Method, apparatus, storage medium and computer program product for similar image retrieval

Abstract

The application provides a similar image retrieval method and device, a storage medium and a computer program product, wherein the method comprises the following steps: based on multiple types of features of multiple images, the similarity between the multiple images is determined through a hierarchical matching operation; and based on the similarity, similar images in the multiple images are retrieved. The application is based on multiple types of features of images to be retrieved, and a hierarchical image similarity calculation strategy is adopted to realize fast image similarity comparison from simple to complex, so that the calculation amount can be effectively reduced, and the operation efficiency can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a similar image retrieval method, apparatus, storage medium, and computer program product. Background Technology

[0002] Image similarity comparison is a classic problem in the field of computer image processing. Currently, deep learning algorithms can be used to solve the problem of image content recognition in image similarity comparison. For example, images can be analyzed from the perspectives of pixel values, color distribution, texture features, etc., to compare similar images.

[0003] However, these methods are not very accurate. To ensure the accuracy of the comparison, more complex similarity matching algorithms are needed, but when the application scenario involves a large-scale image database, the computational load will increase significantly, resulting in low efficiency. Summary of the Invention

[0004] This invention provides a similar image retrieval method, device, storage medium, and computer program product to address the shortcomings of existing technologies that cannot simultaneously ensure accuracy and computational efficiency, thereby achieving the goal of effectively improving computational efficiency while guaranteeing computational accuracy.

[0005] This invention provides a similar image retrieval method, comprising:

[0006] Based on the multi-type features of multiple images, the similarity between the multiple images is determined through a progressive hierarchical matching operation;

[0007] Based on the similarity, similar images are retrieved from the multiple images.

[0008] According to the similar image retrieval method provided by the present invention, before the multi-type features based on multiple images, it further includes:

[0009] Extract the multi-type features from the multiple images. The multi-type features include at least one of the following: scale-invariant feature transformation features, thumbnail pixel features, horizontal and vertical pixel mean features, edge features, and histogram features.

[0010] According to a similar image retrieval method provided by the present invention, the extraction of the multi-type features of the multiple images includes:

[0011] The following processing procedure is used to extract the thumbnail pixel features of the multiple images, the following processing procedure includes:

[0012] The images are reduced to a preset size to obtain thumbnails of the images, and the average value of the R channel, G channel and B channel values ​​of at least one pixel in the thumbnail is calculated.

[0013] The average values ​​corresponding to each pixel in the thumbnail are sorted to obtain a vector of length corresponding to the number of pixels in the thumbnail, and the vector is used as the pixel feature of the thumbnail.

[0014] According to a similar image retrieval method provided by the present invention, when the multi-type features include thumbnail pixel features, edge features, horizontal and / or vertical pixel mean, histogram features, and scale-invariant feature transformation features, the method determines the similarity between the multiple images based on the multi-type features of multiple images through a progressively hierarchical matching operation, including:

[0015] Calculate the first Euclidean distance between the thumbnail pixel features of the multiple images, and if the first Euclidean distance is greater than a first preset threshold, determine whether the first Euclidean distance is less than a second preset threshold;

[0016] If the first Euclidean distance is less than the second preset threshold, then determine whether the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold.

[0017] If the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold, then a horizontal / vertical pixel mean judgment is performed. The horizontal / vertical pixel mean judgment is used to determine whether the second Euclidean distance between the horizontal pixel mean values ​​of the multiple images and / or the third Euclidean distance between the vertical pixel mean values ​​are less than the fourth preset threshold.

[0018] If the result of the horizontal / vertical pixel mean determination is yes, then determine whether the average Wasserstein distance between the histogram features of the multiple images is less than a fifth preset threshold.

[0019] If the average Wasserstein distance is less than the fifth preset threshold, then it is further determined whether the similarity between the scale-invariant feature transformation features of the multiple images is less than the sixth preset threshold. If so, the similarity between the scale-invariant feature transformation features of the multiple images is obtained as the similarity between the multiple images.

[0020] According to a similar image retrieval method provided by the present invention, before further determining whether the similarity between the scale-invariant feature transformation features of the multiple images is less than a sixth preset threshold, the method further includes:

[0021] The plurality of images are input in a first order to obtain a first scale-invariant feature transformation similarity, and the plurality of images are input in a second order to obtain a second scale-invariant feature transformation similarity.

[0022] The average value between the first scale-invariant feature transformation similarity and the second scale-invariant feature transformation similarity is calculated as the similarity between the scale-invariant feature transformation features.

[0023] A similar image retrieval method provided by the present invention further includes:

[0024] If the first Euclidean distance is not greater than the first preset threshold, then the multiple images are determined to be similar images;

[0025] If the first Euclidean distance is not less than the second preset threshold, or the proportion of edge pixels in the edge features of each of the multiple images is not all greater than the third preset threshold, or the result of the horizontal / vertical pixel mean judgment is negative, or the average Wasserstein distance is not less than the fifth preset threshold, or the similarity between the scale-invariant feature transformation features is not less than the sixth preset threshold, then the multiple images are determined to be dissimilar images.

[0026] According to a similar image retrieval method provided by the present invention, after extracting the multi-type features of the multiple images, the method further includes:

[0027] After associating the multiple images with their various types of features, they are stored in an image database.

[0028] The present invention also provides a similar image retrieval device, comprising:

[0029] The matching calculation module is used to determine the similarity between multiple images based on multiple types of features through a progressive hierarchical matching operation.

[0030] The similarity determination module is used to retrieve similar images among the multiple images based on the similarity score.

[0031] A similar image retrieval device according to the present invention further includes:

[0032] The feature extraction module is used to extract the multi-type features of the multiple images, the multi-type features including: scale-invariant feature transformation features and at least one of thumbnail pixel features, horizontal and vertical pixel mean features, edge features and histogram features.

[0033] According to a similar image retrieval device provided by the present invention, the feature extraction module is used for:

[0034] The following processing procedure is used to extract the thumbnail pixel features of the multiple images, the following processing procedure includes:

[0035] The images are reduced to a preset size to obtain thumbnails of the images, and the average value of the R channel, G channel and B channel values ​​of at least one pixel in the thumbnail is calculated.

[0036] The average values ​​corresponding to each pixel in the thumbnail are sorted to obtain a vector of length corresponding to the number of pixels in the thumbnail, and the vector is used as the pixel feature of the thumbnail.

[0037] According to a similar image retrieval device provided by the present invention, when the multiple types of features include thumbnail pixel features, edge features, horizontal and / or vertical pixel mean, histogram features, and scale-invariant feature transformation features, the matching calculation module is used to:

[0038] Calculate the first Euclidean distance between the thumbnail pixel features of the multiple images, and if the first Euclidean distance is greater than a first preset threshold, determine whether the first Euclidean distance is less than a second preset threshold;

[0039] If the first Euclidean distance is less than the second preset threshold, then determine whether the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold.

[0040] If the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold, then a horizontal / vertical pixel mean judgment is performed. The horizontal / vertical pixel mean judgment is used to determine whether the second Euclidean distance between the horizontal pixel mean values ​​of the multiple images and / or the third Euclidean distance between the vertical pixel mean values ​​are less than the fourth preset threshold.

[0041] If the result of the horizontal / vertical pixel mean determination is yes, then determine whether the average Wasserstein distance between the histogram features of the multiple images is less than a fifth preset threshold.

[0042] If the average Wasserstein distance is less than the fifth preset threshold, then it is further determined whether the similarity between the scale-invariant feature transformation features of the multiple images is less than the sixth preset threshold. If so, the similarity between the scale-invariant feature transformation features of the multiple images is obtained as the similarity between the multiple images.

[0043] According to a similar image retrieval device provided by the present invention, the matching calculation module is further configured to:

[0044] The plurality of images are input in a first order to obtain a first scale-invariant feature transformation similarity, and the plurality of images are input in a second order to obtain a second scale-invariant feature transformation similarity.

[0045] The average value between the first scale-invariant feature transformation similarity and the second scale-invariant feature transformation similarity is calculated as the similarity between the scale-invariant feature transformation features.

[0046] According to a similar image retrieval device provided by the present invention, the similarity judgment module is further configured to:

[0047] If the first Euclidean distance is not greater than the first preset threshold, then the multiple images are determined to be similar images;

[0048] If the first Euclidean distance is not less than the second preset threshold, or the proportion of edge pixels in the edge features of each of the multiple images is not all greater than the third preset threshold, or the result of the horizontal / vertical pixel mean judgment is negative, or the average Wasserstein distance is not less than the fifth preset threshold, or the similarity between the scale-invariant feature transformation features is not less than the sixth preset threshold, then the multiple images are determined to be dissimilar images.

[0049] According to a similar image retrieval device provided by the present invention, the feature extraction module is further configured to:

[0050] After associating the multiple images with their various types of features, they are stored in an image database.

[0051] The present invention also provides an electronic device, including a memory, a processor, and a program or instructions stored in the memory and executable on the processor, wherein when the processor executes the program or instructions, it implements the steps of the similar image retrieval method as described above.

[0052] The present invention also provides a non-transitory computer-readable storage medium having a program or instructions stored thereon, wherein when the program or instructions are executed by a computer, the steps of the similar image retrieval method as described above are implemented.

[0053] The present invention also provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the similar image retrieval method as described above.

[0054] The similar image retrieval method, device, storage medium, and computer program product provided by this invention, based on the multi-type features of the image to be retrieved, adopts a hierarchical image similarity calculation strategy, progressing from simple to complex, to achieve rapid image similarity comparison, which can effectively reduce the amount of calculation and effectively improve the computational efficiency. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments of this invention or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is one of the flowcharts illustrating the similar image retrieval method provided by the present invention;

[0057] Figure 2 This is a schematic diagram of image pixels in the similar image retrieval method provided by the present invention;

[0058] Figure 3 The second schematic diagram of the similar image retrieval method provided by the present invention;

[0059] Figure 4 This is a comparative schematic diagram of thumbnail flipping and rotation transformation in the similar image retrieval method provided by the present invention;

[0060] Figure 5 This is a schematic diagram illustrating the process of determining the similarity between multiple images in the similar image retrieval method provided by the present invention;

[0061] Figure 6 A schematic diagram of the structure of the similar image retrieval device provided by the present invention;

[0062] Figure 7 A schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0064] This invention addresses the problem of existing technologies failing to balance accuracy and computational efficiency. Based on multiple features of the image to be retrieved, it employs a hierarchical image similarity calculation strategy, progressing from simple to complex, to achieve rapid image similarity comparison. This effectively reduces computational load and significantly improves computational efficiency. The invention will be further described and illustrated below with reference to the accompanying drawings and several specific embodiments.

[0065] Figure 1 This is one of the flowcharts illustrating the similar image retrieval method provided by the present invention, such as... Figure 1 As shown, the method includes:

[0066] S101, based on the multi-type features of multiple images, the similarity between the multiple images is determined through a progressive hierarchical matching operation.

[0067] This invention can be understood as follows: multiple images to be compared or matched can be obtained in advance, and features can be extracted from each of the multiple images. During feature extraction, multiple types of features are extracted from each image from simple to complex. These can be collectively referred to as multi-type features. Thus, each image corresponds to a set of multi-type features.

[0068] Optionally, before the multi-type features based on multiple images, the method further includes: extracting the multi-type features of the multiple images, wherein the multi-type features include at least one of: scale-invariant feature transformation features and thumbnail pixel features, horizontal and vertical pixel mean features, edge features and histogram features.

[0069] It can be understood that the purpose of this invention is to achieve rapid comparison and matching of similar images while ensuring accuracy. Therefore, the embodiments of this invention can extract the scale-invariant feature transform (SIFT) features of each of the above multiple images to perform accurate matching of similar images based on these features.

[0070] SIFT features are used to detect and describe local features in images. They are primarily obtained by finding extreme points in spatial scale and extracting invariants such as the position, scale, and rotation of these extreme points as feature descriptors. SIFT features are derived from local texture features within objects and are independent of the object's scale and angle, thus effectively avoiding the influence of lighting, noise, and changes in viewpoint. Due to the high saliency of these SIFT features, they can effectively identify targets with high accuracy.

[0071] Furthermore, SIFT features can also be used to handle situations where objects are partially occluded. When more than three SIFT features are extracted from an object, its location can be calculated. The computation speed is fast enough, and the amount of information retained by the features is relatively high compared to other algorithms, resulting in a higher signal-to-noise ratio, making it suitable for the large databases that this invention addresses. The extraction of SIFT features from images in this invention can be achieved using existing algorithms.

[0072] While ensuring accuracy by utilizing SIFT features, in order to reduce computational complexity and improve computational speed, relatively simple features can also be extracted from multiple images, such as one or any combination of the features listed above.

[0073] Among them, thumbnail pixel features are the pixel features extracted from the thumbnails of each image. For two similar images, comparing the differences between the thumbnails after shrinking them will not lead to misjudgment, and will greatly reduce the time required.

[0074] The horizontal and vertical pixel mean features refer to the horizontal pixel mean of each row of an image, obtained by averaging all pixels in each row, and the vertical pixel mean of each column, obtained by averaging all pixels in each column. Alternatively, only the horizontal or vertical pixel mean can be obtained as needed, without necessarily obtaining both simultaneously as the aforementioned horizontal and vertical pixel mean features.

[0075] Taking a 2x3 pixel image as an example, such as Figure 2 As shown, this is a schematic diagram of image pixels in the similar image retrieval method provided by the present invention. The image contains two rows, each containing three pixels, and the pixel values ​​are as follows: Figure 2 As shown. When extracting the horizontal pixel mean feature of this image, for the first row, its pixel mean can optionally be calculated using the following formula:

[0076]

[0077] Similarly, for the second row, its average pixel value is as follows:

[0078]

[0079] but Figure 2 The horizontal pixel mean feature of the image shown is a 2*3 matrix, as follows:

[0080]

[0081] Figure 2 The image shown contains three columns, each with two elements, and the pixel values ​​are as follows: Figure 2 As shown. When extracting the vertical pixel mean feature of this image, for the first column, its pixel mean can optionally be calculated using the following formula:

[0082]

[0083] Similarly, the average pixel values ​​for the second and third columns are as follows:

[0084] Second column: Third column:

[0085] but Figure 2 The horizontal pixel mean feature of the image shown is a 3x3 matrix, as follows:

[0086]

[0087] Edge features are image features extracted from edge pixels. An edge is the boundary line between different regions in an image; it's a point of discontinuity in the distribution of image characteristics (such as pixel grayscale, texture, etc.); it's a collection of pixels whose characteristics exhibit abrupt or roof-like changes. Image edge detection aims to detect edges or discontinuous regions in an image, primarily used to enhance contour edges, details, and grayscale transitions, forming complete object boundaries to separate objects from the image or detect regions representing the same object's surface.

[0088] The extraction of image edge features can be achieved using existing edge detection algorithms such as Canny, differential methods, difference edge detection, Roberts, Sobel, Prewitt, or Laplacian. These edge detection algorithms can detect the edge information of various objects in an image. Generally, images containing more information should also contain more edge information, while images containing less information, such as images of unfinished houses, contain almost no edge information due to their shallow texture. By setting a certain threshold, images with more effective information can be effectively filtered out based on their edge information.

[0089] Image histogram features are the distributions of the R, G, and B channel values. To speed up comparison, the image resolution can be reduced to a set value such as 12x12 before extracting the histogram features. Determining image similarity through histogram features involves comparing the differences between the three pairs of distributions of R, G, and B channel values ​​in the image's histogram features to determine the similarity between images.

[0090] like Figure 3 The diagram shown is a second flowchart of the similar image retrieval method provided by the present invention. Based on the extraction of multiple types of features of each image to be compared according to the above processing, the features are compared in order from simple to complex.

[0091] Specifically, the present invention can select one or more relatively simple features with low computational complexity from the multiple types of features, and first use these features to perform rough similarity matching and filtering on the images. That is, these simple features can be used to calculate the rough similarity of multiple images, and images with large differences (or low similarity) can be filtered out based on the rough similarity.

[0092] For images that are judged to be similar using simple features, one or more features that are slightly more computationally intensive and relatively more complex than the features selected in the previous round can be selected from multiple types of features to perform further similarity calculations, matching, and filtering on the images.

[0093] Thus, based on the complexity of each feature among the multiple types of features, starting with the simplest feature with the least computational load, the images are subjected to progressive similarity calculations, matching, and filtering until the most complex and accurate feature is used for similarity calculation, ultimately determining the similarity between the features of each type among multiple images.

[0094] It is understandable that, after the above-mentioned hierarchical matching operation, images with low similarity can be distinguished by simpler image features. Therefore, only simpler or coarser similarity calculations were performed on these images using simpler features.

[0095] For images with high or greater similarity, using simple image features for similarity matching may result in misjudgments due to relatively low accuracy. Therefore, after calculating similarity using simpler features and filtering based on the results, more complex and accurate features are used for further similarity calculation and matching.

[0096] Therefore, based on the above hierarchical matching calculation, images with low similarity undergo a simple similarity calculation using relatively simple features, and the resulting similarity is based on these simple features. For images with high similarity, a similarity calculation is first performed using relatively simple features to filter them out, and then a further similarity calculation is performed using relatively complex and more accurate features, resulting in a similarity based on these more complex and more accurate features.

[0097] S102, Based on the similarity, retrieve similar images from the multiple images.

[0098] This can be understood as follows: based on the similarity between multiple images calculated according to the above steps, the similarity can be used to determine which of the multiple images are similar by combining the similarity with a preset threshold or other methods.

[0099] Understandably, two images are generally considered similar if their similarity reaches a certain threshold. Therefore, a reasonable preset threshold can be set, and the calculated similarity of the images can be compared with this preset threshold. If the similarity of the images is greater than the preset threshold, then the images can be determined to be similar.

[0100] The similar image retrieval method provided by this invention is based on multiple types of features of the image to be retrieved. By adopting a hierarchical image similarity calculation strategy, it achieves rapid image similarity comparison from simple to complex, which can effectively reduce the amount of calculation and effectively improve the computational efficiency.

[0101] Optionally, in the similar image retrieval method provided by the above embodiments, the extraction of the multi-type features of the multiple images includes: extracting the thumbnail pixel features of the multiple images using the following processing flow, wherein the following processing flow includes:

[0102] The images are reduced to a preset size to obtain thumbnails of the images, and the average value of the R channel, G channel and B channel values ​​of at least one pixel in the thumbnail is calculated.

[0103] The average values ​​corresponding to each pixel in the thumbnail are sorted to obtain a vector of length corresponding to the number of pixels in the thumbnail, and the vector is used as the pixel feature of the thumbnail.

[0104] As can be understood, according to the above embodiments, the present invention can extract thumbnail pixel features of images to calculate image similarity. When extracting thumbnail pixel features of images, for two similar images, reducing their size and then comparing the differences between the thumbnails will not lead to misjudgment and will greatly reduce the required time. For images that are not originally similar, although reducing the size will lead to a decrease in accuracy, they can be eliminated through subsequent multi-level similarity calculations.

[0105] Therefore, the present invention can first reduce the size of the image according to a preset size (i.e., the preset size) to obtain a thumbnail of the image. For example, the preset size of the thumbnail can be set to 6*6, and the size of the original image can be reduced to 6*6 according to the preset size. The 6*6 thumbnail includes 36 pixels.

[0106] Next, obtain the R-channel, G-channel, and B-channel values ​​of each pixel in the thumbnail, and calculate the average value of each pixel's R-channel, G-channel, and B-channel values. For example, for the 6*6 thumbnail mentioned above, for each of the 36 pixels, sum the R-channel, G-channel, and B-channel values ​​and divide by 3 to obtain the average value of the pixel, thus obtaining the average value of all 36 pixels.

[0107] Based on the calculated average value of each pixel in the thumbnail, taking into account factors such as... Figure 4 The illustration shows a comparison of thumbnail flipping and rotation transformations. When the image is horizontally flipped or rotated... Figure 4 The image shown on the left is the original image, the middle image is a horizontally flipped version of the original image, and the right image is a 90-degree clockwise rotation of the original image. Directly comparing the differences between individual pixels yields a large result, significantly impacting the recall rate. Therefore, this invention incorporates sorting when acquiring the thumbnail pixel features of an image. Specifically, the average pixel values ​​corresponding to each pixel in the thumbnail are sorted by size, resulting in a vector with element values ​​sorted from smallest to largest or largest to smallest. This vector is then used as the thumbnail pixel feature of the image.

[0108] For example, for the 6x6 thumbnail example above, by averaging and sorting the RGB values ​​of each pixel, a vector of length 36, arranged in ascending order of values, can be obtained as the thumbnail pixel feature of the image.

[0109] This invention reduces the time required for similarity calculation by reducing the image size and averaging and sorting the pixels of the resulting thumbnails. It can also handle partially mirror-symmetric or rotated similar images with higher accuracy.

[0110] Optionally, in the similar image retrieval method provided by the above embodiments, when the multi-type features include thumbnail pixel features, edge features, horizontal and / or vertical pixel mean, histogram features, and scale-invariant feature transformation features, the similarity between the multiple images is determined through a progressively hierarchical matching operation based on the multi-type features of multiple images. Figure 5 The diagram shown illustrates the process of determining the similarity between multiple images in the similar image retrieval method provided by the present invention, mainly including the following processing steps:

[0111] Step 501: Calculate the first Euclidean distance between the thumbnail pixel features of the multiple images, and if the first Euclidean distance is greater than a first preset threshold, determine whether the first Euclidean distance is less than a second preset threshold.

[0112] This can be understood as, for example Figure 5 As shown, before performing image similarity calculation, the multi-type features 1 of the image to be compared and the multi-type features 2 of the image to be compared can be obtained according to the above embodiments.

[0113] Then, thumbnail pixel features from multi-type feature 1 and multi-type feature 2 are selected respectively, and the Euclidean distance between the thumbnail pixel features of these two multi-type features is calculated. For ease of distinction, it can be called the first Euclidean distance.

[0114] Wherein, Euclidean distance is the straight-line distance between two points in space. In this invention, it represents the straight-line distance between the thumbnail pixel features of multi-type feature 1 and the thumbnail pixel features of multi-type feature 2, and is expressed by the following formula:

[0115]

[0116] In the formula, d(x,y) represents the Euclidean distance between thumbnail pixel feature 1 and thumbnail pixel feature 2, and x1, x2, ..., xy represent the distances between them. n The elements representing pixel feature 1 of the thumbnail, y1, y2, ..., y3, represent the elements of the thumbnail pixel feature 1. n This represents the elements of thumbnail pixel feature 2.

[0117] Next, the present invention compares a first Euclidean distance with a first preset threshold to determine whether the first Euclidean distance is greater than the first preset threshold. The first preset threshold is a sufficiently small value; when the first Euclidean distance between the thumbnail pixel features of two images is greater than this sufficiently small value, it indicates that the two images are not close enough and cannot be identified as similar images.

[0118] To determine whether two images are similar, it is necessary to further judge whether their first Euclidean distance is less than a second preset threshold. The second preset threshold is a benchmark value that is relatively large compared to the first preset threshold. If the first Euclidean distance between the two images is not less than the second preset threshold, it means that the two images are far enough apart and can be considered dissimilar images.

[0119] Step 502: If the first Euclidean distance is less than the second preset threshold, then determine whether the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold.

[0120] This can be understood as follows: based on the above steps, if the first Euclidean distance between two images is less than the second preset threshold, it means that the distance between the two images is not far enough to confirm that the two images are dissimilar. Therefore, the edge features of the images can be used to further determine whether the two images are similar.

[0121] Specifically, edge feature 1 from multi-type feature 1 and edge feature 2 from multi-type feature 2 are selected respectively, and it is determined whether the proportion of edge pixels in edge feature 1 and the proportion of edge pixels in edge feature 2 are both greater than the third preset threshold.

[0122] It should be understood that many image libraries contain images that are "meaningless" or contain little information, which can lead to redundant computational load in similarity comparisons. For example, it's normal for a large number of unfinished house images to appear "similar" to each other. Since these images contain very little information, comparing them all would increase unnecessary computation; in practical business scenarios, there's no need to compare these similar images. From a business perspective, users are more interested in other renovated secondhand houses. This invention can employ edge detection, such as using edge detection methods like Canny, to detect the edge information of each object. The amount of useful information contained in an image can be characterized by calculating the proportion of edge pixels in the edge features.

[0123] Step 503: If the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold, then perform a horizontal / vertical pixel mean judgment. The horizontal / vertical pixel mean judgment is used to determine whether the second Euclidean distance between the horizontal pixel mean and / or the third Euclidean distance between the vertical pixel mean of the multiple images is less than the fourth preset threshold.

[0124] This can be understood as follows: based on the above steps, if the proportion of edge pixels in edge feature 1 and the proportion of edge pixels in edge feature 2 are both greater than the third preset threshold, it means that both images contain enough useful information and can be further similarity matched.

[0125] Specifically, during further similarity matching calculations, the first horizontal pixel mean and / or the first vertical pixel mean of image 1 are selected from multi-type feature 1, and the second horizontal pixel mean and / or the second vertical pixel mean of image 2 are selected from multi-type feature 2. Then, the Euclidean distance between the first and second horizontal pixel mean (referred to as the second Euclidean distance) and / or the Euclidean distance between the first and second vertical pixel mean (referred to as the third Euclidean distance) are calculated. Finally, it is determined whether the calculated second and / or third Euclidean distances are both less than a fourth preset threshold.

[0126] It should be understood that, in addition to the similarity between individual pixels, similar images should also have relatively similar average pixels in either the horizontal or vertical direction, or at least one of them should be relatively similar. By comparing the average pixels in the horizontal and vertical directions of actual images, it has been found that for two images with a certain degree of similarity, although the difference in average pixels in the vertical direction may sometimes be slightly larger, the two horizontal vectors are very similar.

[0127] Step 504: If the result of the horizontal / vertical pixel mean determination is yes, then determine whether the average Wasserstein distance between the histogram features of the multiple images is less than a fifth preset threshold.

[0128] This can be understood as follows: based on the above steps, if the result of the horizontal / vertical pixel mean judgment is yes, it indicates that the two images have a certain degree of similarity in at least one direction. To determine whether they are truly similar images, histogram feature 1 of image 1 can be selected from multi-type feature 1, and histogram feature 2 of image 2 can be selected from multi-type feature 2. Then, the Wasserstein distance between histogram feature 1 and histogram feature 2 is calculated, and it is determined whether this Wasserstein distance is less than a fifth preset threshold.

[0129] The Wasserstein distance between the two distributions is calculated using the following formula:

[0130]

[0131] In the formula, μ and ν are two probability distributions that need to be measured, γ is a joint probability distribution with marginal distributions μ and ν, and d(xy) p Let p represent the p-order norm of xy.

[0132] Understandably, since the image has been compressed before calculating the histogram feature differences, the histogram distribution for any color has only 12*12 = 144 data points. Therefore, there is a considerable probability that the histogram distributions do not overlap, hence the Wasserstein metric is used as the standard. The intuitive understanding of the Wasserstein distance can be described as the difficulty of transforming histogram distribution A into histogram distribution B. When calculating non-overlapping distributions, the output is smooth. For example, if histogram distribution A has a probability of 1 only at x = a, and histogram distribution B has a probability of 1 only at x = b, then...

[0133]

[0134] Wasserstein(A||B)=|ab|.

[0135] In the formula, KL(A||B) represents the KL divergence between distributions A and B, JS(A||B) represents the JS divergence between distributions A and B, and Wasserstein(A||B) represents the Wasserstein distance between the two distributions.

[0136] Step 505: If the average Wasserstein distance is less than the fifth preset threshold, then it is further determined whether the similarity between the scale-invariant feature transformation features of the multiple images is less than the sixth preset threshold. If so, the similarity between the scale-invariant feature transformation features of the multiple images is obtained as the similarity between the multiple images.

[0137] This can be understood as follows: based on the above steps, if the Wasserstein distance between histogram feature 1 of image 1 and histogram feature 2 of image 2 is less than the fifth preset threshold, it means that the color distribution distance between the two images is not far enough to confirm that the two images are dissimilar. Therefore, the scale-invariant feature transformation feature of the images can be used to further determine whether the two images are similar.

[0138] Specifically, scale-invariant feature transform feature 1 from multi-type feature 1 and scale-invariant feature transform feature 2 from multi-type feature 2 are selected respectively, and it is determined whether the similarity between scale-invariant feature transform feature 1 and scale-invariant feature transform feature 2 is less than a sixth preset threshold. If so, it means that the two images are sufficiently similar in texture, and the similarity between the two scale-invariant feature transform features can be obtained as the similarity between the corresponding image 1 and image 2.

[0139] In this invention, the KNN (k=2) algorithm is used to pair descriptors between two graphs. If the distance ratio between a pair of descriptors is less than 0.9, the two descriptors are considered to be matched. Then, the ratio of matched descriptors to the total number of descriptors is calculated, which is the similarity ratio between the two graphs, or simply the similarity.

[0140] It should be understood that if two images are still sufficiently similar after being detected by the features mentioned above, then Scale-invariant feature transform (SIFT) features will be used to determine the similarity of the images at the image texture level. SIFT features are derived from local texture features in objects and are sensitive to scaled similar textures. After magnification, the scale and angle of the texture do not change, allowing for accurate comparison of images before and after scaling.

[0141] This invention employs a similarity comparison strategy that progresses from simple to complex. For most images with large similarity differences, simple matching operations can be used to filter them out, reducing unnecessary calculations and significantly lowering the average computation time.

[0142] Optionally, the similar image retrieval method provided in the above embodiments further includes, before determining whether the similarity between the scale-invariant feature transformation features of the plurality of images is less than a sixth preset threshold: inputting each of the plurality of images in a first order to obtain a first scale-invariant feature transformation similarity, and inputting each of the plurality of images in a second order to obtain a second scale-invariant feature transformation similarity; calculating the average value between the first scale-invariant feature transformation similarity and the second scale-invariant feature transformation similarity as the similarity between the scale-invariant feature transformation features.

[0143] This can be understood as follows: because the SIFT similarity algorithm does not conform to the commutative law, changing the image input order will yield different results. Therefore, when comparing the SIFT similarity of two images, this invention swaps the input order of the two images and compares them twice, obtaining two scale-invariant feature transform similarities respectively, and outputs the average of the two as the final similarity.

[0144] This invention performs two similarity comparisons by swapping the order of the images and takes the average of the two comparisons as the final result, which can effectively avoid misjudgments caused by the input order and achieve higher accuracy.

[0145] Furthermore, based on the similar image retrieval method provided in the above embodiments, the method further includes: if the first Euclidean distance is not greater than the first preset threshold, then the plurality of images are determined to be similar images; if the first Euclidean distance is not less than the second preset threshold, or the proportion of edge pixels in the edge features of each of the plurality of images is not all greater than the third preset threshold, or the result of the horizontal / vertical pixel mean judgment is negative, or the average Wasserstein distance is not less than the fifth preset threshold, or the similarity between the scale-invariant feature transformation features is not less than the sixth preset threshold, then the plurality of images are determined to be dissimilar images.

[0146] This can be understood as, for example Figure 5 As shown, when performing hierarchical similarity matching operations on multiple images using multi-type features, if the first Euclidean distance between the thumbnail pixel features of two images is not greater than a sufficiently small first preset threshold, it indicates that the two images are close enough and can be identified as similar images.

[0147] If the first Euclidean distance between two images is not less than a relatively large second preset threshold, it means that the distance between the two images is far enough, that is, the similarity is small, and they can be considered as dissimilar images.

[0148] If the ratio of edge pixels in edge feature 1 and the ratio of edge pixels in edge feature 2 are at least not greater than the third preset threshold, it means that at least one of the two images does not contain enough useful information. Performing similarity matching calculations on it would bring unnecessary computation and slow down the computation speed. Therefore, it can be identified as a dissimilar image and filtered out.

[0149] If the result of the horizontal / vertical pixel mean judgment is negative, it means that the two images do not reach sufficient similarity in either direction, so they can be identified as dissimilar images and filtered out.

[0150] If the Wasserstein distance between histogram feature 1 of image 1 and histogram feature 2 of image 2 is not less than the fifth preset threshold, it means that the color distribution of the two images is far enough apart, and it can be confirmed that the two images are dissimilar images and should be filtered out.

[0151] If the similarity between scale-invariant feature transform feature 1 and scale-invariant feature transform feature 2 is not less than the sixth preset threshold, it indicates that the two images are not sufficiently similar in texture, and they are confirmed as dissimilar images and filtered out.

[0152] Furthermore, based on the similar image retrieval methods provided in the above embodiments, after extracting the multi-type features of the multiple images, the method further includes: associating the multiple images and the multi-type features of the multiple images, and then storing them in an image database.

[0153] This invention can be understood as follows: based on the extraction of multiple types of features from an image, the invention can associate and store the image with the multiple types of features extracted from the image, that is, while saving the image, its features can be calculated and saved.

[0154] This invention solves the performance problem of image similarity comparison in large-scale image libraries by associating and storing images with their multiple types of features, and simplifies the subsequent calculation process.

[0155] Based on the same inventive concept, the present invention also provides a similar image retrieval device according to the above embodiments, which is used to implement similar image retrieval in the above embodiments. Therefore, the descriptions and definitions in the similar image retrieval methods of the above embodiments can be used to understand the various execution modules in the present invention, and specific details can be found in the above method embodiments, which will not be repeated here.

[0156] According to an embodiment of the present invention, the structure of the similar image retrieval device is as follows: Figure 6 The diagram shown is a structural schematic of the similar image retrieval device provided by the present invention. This device can be used to implement similar image retrieval in the above-described method embodiments. The device includes: a matching calculation module 601 and a similarity judgment module 602. Wherein:

[0157] The matching calculation module 601 is used to determine the similarity between multiple images based on multiple types of features and through progressive hierarchical matching operations; the similarity judgment module 602 is used to retrieve similar images among the multiple images based on the similarity.

[0158] Specifically, this invention can acquire multiple images to be compared or matched in advance, and extract features from each of these images. During feature extraction, multiple types of features are extracted from each image, starting from simple to complex. These can be collectively referred to as multi-type features, so each image corresponds to a set of multi-type features. Based on the extracted multi-type features of each image to be compared, the matching calculation module 601 compares each feature sequentially from simple to complex.

[0159] Specifically, the matching calculation module 601 can select one or more relatively simple features with low computational complexity from the multiple types of features, and first use these features to perform rough similarity matching and filtering on the images. In other words, the matching calculation module 601 can first use these simple features to calculate the rough similarity of multiple images, and can filter out some images with large differences (or low similarity) based on the rough similarity.

[0160] For images that are judged to be similar using simple features, the matching calculation module 601 can select one or more features from multiple types of features that are slightly more computationally intensive and relatively more complex than the features selected in the previous round, and perform further similarity calculations, matching, and filtering on the images.

[0161] Thus, based on the complexity of each feature among the multiple types of features, starting with the simplest feature with the least computational load, the images are subjected to progressive similarity calculations, matching, and filtering until the most complex and accurate feature is used for similarity calculation, ultimately determining the similarity between the features of each type among multiple images.

[0162] It is understandable that, after the above-mentioned hierarchical matching operation, images with low similarity can be distinguished by simpler image features. Therefore, only simpler or coarser similarity calculations were performed on these images using simpler features.

[0163] For images with high or greater similarity, using simple image features for similarity matching may result in misjudgments due to relatively low accuracy. Therefore, the matching calculation module 601 performs similarity calculations on these images using simpler features and filters them based on the calculation results. Then, it uses features with higher complexity and accuracy for further similarity calculations and matching.

[0164] Therefore, based on the above hierarchical matching calculation, images with low similarity undergo a simple similarity calculation using relatively simple features, and the resulting similarity is based on these simple features. For images with high similarity, a similarity calculation is first performed using relatively simple features to filter them out, and then a further similarity calculation is performed using relatively complex and more accurate features, resulting in a similarity based on these more complex and more accurate features.

[0165] Subsequently, the similarity judgment module 602 can determine which of the above multiple images are similar images based on the above similarity and by combining preset thresholds and other methods.

[0166] The similar image retrieval device provided by this invention, based on the multi-type features of the image to be retrieved, adopts a hierarchical image similarity calculation strategy, progressing from simple to complex, to achieve rapid image similarity comparison, which can effectively reduce the amount of calculation and effectively improve the computational efficiency.

[0167] Optionally, the similar image retrieval device further includes:

[0168] The feature extraction module is used to extract the multi-type features of the multiple images, the multi-type features including: scale-invariant feature transformation features and at least one of thumbnail pixel features, horizontal and vertical pixel mean features, edge features and histogram features.

[0169] Optionally, the feature extraction module is used for:

[0170] The following processing procedure is used to extract the thumbnail pixel features of the multiple images, the following processing procedure includes:

[0171] The images are reduced to a preset size to obtain thumbnails of the images, and the average value of the R channel, G channel and B channel values ​​of at least one pixel in the thumbnail is calculated.

[0172] The average values ​​corresponding to each pixel in the thumbnail are sorted to obtain a vector of length corresponding to the number of pixels in the thumbnail, and the vector is used as the pixel feature of the thumbnail.

[0173] Optionally, when the multiple types of features include thumbnail pixel features, edge features, horizontal and / or vertical pixel mean, histogram features, and scale-invariant feature transformation features, the matching calculation module is used to:

[0174] Calculate the first Euclidean distance between the thumbnail pixel features of the multiple images, and if the first Euclidean distance is greater than a first preset threshold, determine whether the first Euclidean distance is less than a second preset threshold;

[0175] If the first Euclidean distance is less than the second preset threshold, then determine whether the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold.

[0176] If the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold, then a horizontal / vertical pixel mean judgment is performed. The horizontal / vertical pixel mean judgment is used to determine whether the second Euclidean distance between the horizontal pixel mean values ​​of the multiple images and / or the third Euclidean distance between the vertical pixel mean values ​​are less than the fourth preset threshold.

[0177] If the result of the horizontal / vertical pixel mean determination is yes, then determine whether the average Wasserstein distance between the histogram features of the multiple images is less than a fifth preset threshold.

[0178] If the average Wasserstein distance is less than the fifth preset threshold, then it is further determined whether the similarity between the scale-invariant feature transformation features of the multiple images is less than the sixth preset threshold. If so, the similarity between the scale-invariant feature transformation features of the multiple images is obtained as the similarity between the multiple images.

[0179] Optionally, the matching calculation module is further configured to:

[0180] The plurality of images are input in a first order to obtain a first scale-invariant feature transformation similarity, and the plurality of images are input in a second order to obtain a second scale-invariant feature transformation similarity.

[0181] The average value between the first scale-invariant feature transformation similarity and the second scale-invariant feature transformation similarity is calculated as the similarity between the scale-invariant feature transformation features.

[0182] Optionally, the similarity determination module is further configured to:

[0183] If the first Euclidean distance is not greater than the first preset threshold, then the multiple images are determined to be similar images;

[0184] If the first Euclidean distance is not less than the second preset threshold, or the proportion of edge pixels in the edge features of each of the multiple images is not all greater than the third preset threshold, or the result of the horizontal / vertical pixel mean judgment is negative, or the average Wasserstein distance is not less than the fifth preset threshold, or the similarity between the scale-invariant feature transformation features is not less than the sixth preset threshold, then the multiple images are determined to be dissimilar images.

[0185] Optionally, the feature extraction module is further configured to:

[0186] After associating the multiple images with their various types of features, they are stored in an image database.

[0187] It is understood that the relevant program modules in the apparatus of the above embodiments can be implemented by a hardware processor in this invention. Furthermore, the similar image retrieval apparatus of this invention, utilizing the above program modules, can implement the similar image retrieval process of the above method embodiments. When used to implement similar image retrieval in the above method embodiments, the beneficial effects produced by the apparatus of this invention are the same as those in the corresponding above method embodiments, and can be referred to the above method embodiments, which will not be repeated here.

[0188] As another aspect of the present invention, this embodiment provides an electronic device according to the above embodiments. The electronic device includes a memory, a processor, and a program or instructions stored in the memory and executable on the processor. When the processor executes the program or instructions, it implements the steps of the similar image retrieval method as described in the above embodiments.

[0189] Furthermore, the electronic device of the present invention may also include a communication interface and a bus. (See reference) Figure 7 The above is a schematic diagram of the physical structure of the electronic device provided by the present invention, including: at least one memory 701, at least one processor 702, a communication interface 703 and a bus 704.

[0190] The memory 701, processor 702, and communication interface 703 communicate with each other via bus 704. The communication interface 703 is used for information transmission between the electronic device and the image library device. The memory 701 stores a program or instruction that can be run on the processor 702. When the processor 702 executes the program or instruction, it implements the steps of the similar image retrieval method as described in the above embodiments.

[0191] This electronic device can be understood to include at least a memory 701, a processor 702, a communication interface 703, and a bus 704. The memory 701, processor 702, and communication interface 703 are interconnected via the bus 704, enabling communication between them. For example, the processor 702 can read program instructions for a similar image retrieval method from the memory 701. Furthermore, the communication interface 703 can also establish a communication connection between the electronic device and an image library device, facilitating information transmission between them, such as reading multiple images through the communication interface 703.

[0192] When the electronic device is running, the processor 702 calls the program instructions in the memory 701 to execute the methods provided in the above-described method embodiments, such as: determining the similarity between the multiple images based on the multi-type features of multiple images through a progressive hierarchical matching operation; and retrieving similar images from the multiple images based on the similarity.

[0193] When the program instructions in the aforementioned memory 701 can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Alternatively, all or part of the steps of the above method embodiments can be implemented by hardware related to the program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0194] The present invention also provides a non-transitory computer-readable storage medium according to the above embodiments, wherein a program or instructions are stored thereon, which, when executed by a computer, implement the steps of the similar image retrieval method as described in the above embodiments, for example including: determining the similarity between the multiple images based on multiple types of features of multiple images through a progressive hierarchical matching operation; and retrieving similar images among the multiple images based on the similarity.

[0195] As another aspect of the present invention, this embodiment also provides a computer program product according to the above embodiments. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the similar image retrieval method provided in the above method embodiments. The method includes, for example, determining the similarity between the multiple images based on multiple types of features of multiple images through a progressively hierarchical matching operation; and retrieving similar images from the multiple images based on the similarity.

[0196] The electronic device, non-transitory computer-readable storage medium, and computer program product provided by the present invention, by executing the steps of the similar image retrieval method described in the above embodiments, based on the multi-type features of the image to be retrieved, and by adopting a hierarchical image similarity calculation strategy, from simple to complex, achieves rapid image similarity comparison, which can effectively reduce the amount of calculation and effectively improve the computational efficiency.

[0197] It is understood that the embodiments of the devices, electronic devices, and storage media described above are merely illustrative. The units described as separate components may or may not be physically separate; they may be located in one place or distributed across different network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0198] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a USB flash drive, mobile hard drive, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the above method embodiments or some parts of the method embodiments.

[0199] Furthermore, those skilled in the art should understand that in the application documents of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0200] Numerous specific details are set forth in this specification. However, it should be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification. Similarly, it should be understood that, in order to simplify the disclosure of this invention and aid in the understanding of one or more aspects of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A similar image retrieval method, characterized in that, include: Based on the multi-type features of multiple images, the similarity between the multiple images is determined through a progressive hierarchical matching operation; Based on the similarity, similar images are retrieved from the multiple images; Before the multi-image-based multi-type features, the following is also included: Extract the multi-type features from the multiple images. The multi-type features include: scale-invariant feature transformation features and relatively simple features. The relatively simple features include thumbnail pixel features, horizontal and vertical pixel mean features, edge features, and histogram features. When the multi-type features include thumbnail pixel features, edge features, horizontal and / or vertical pixel mean, histogram features, and scale-invariant feature transformation features, the multi-type features based on multiple images determine the similarity between the multiple images through a progressively hierarchical matching operation, including: Calculate the first Euclidean distance between the thumbnail pixel features of the multiple images, and if the first Euclidean distance is greater than a first preset threshold, determine whether the first Euclidean distance is less than a second preset threshold; If the first Euclidean distance is less than the second preset threshold, then determine whether the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold. If the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold, then a horizontal / vertical pixel mean judgment is performed. The horizontal / vertical pixel mean judgment is used to determine whether the second Euclidean distance between the horizontal pixel mean values ​​of the multiple images and / or the third Euclidean distance between the vertical pixel mean values ​​are less than the fourth preset threshold. If the result of the horizontal / vertical pixel mean determination is yes, then determine whether the average Wasserstein distance between the histogram features of the multiple images is less than a fifth preset threshold. If the average Wasserstein distance is less than the fifth preset threshold, then it is further determined whether the similarity between the scale-invariant feature transformation features of the multiple images is less than the sixth preset threshold. If so, the similarity between the scale-invariant feature transformation features of the multiple images is obtained as the similarity between the multiple images.

2. The similar image retrieval method according to claim 1, characterized in that, The extraction of the multiple types of features from the multiple images includes: The following processing procedure is used to extract the thumbnail pixel features of the multiple images, the following processing procedure includes: The images are reduced to a preset size to obtain thumbnails of the images, and the average value of the R channel, G channel and B channel values ​​of at least one pixel in the thumbnail is calculated. The average values ​​corresponding to each pixel in the thumbnail are sorted to obtain a vector of length corresponding to the number of pixels in the thumbnail, and the vector is used as the pixel feature of the thumbnail.

3. The similar image retrieval method according to claim 1, characterized in that, Before further determining whether the similarity between the scale-invariant feature transformation features of the multiple images is less than a sixth preset threshold, the method further includes: The plurality of images are input in a first order to obtain a first scale-invariant feature transformation similarity, and the plurality of images are input in a second order to obtain a second scale-invariant feature transformation similarity. The average value between the first scale-invariant feature transformation similarity and the second scale-invariant feature transformation similarity is calculated as the similarity between the scale-invariant feature transformation features.

4. The similar image retrieval method according to claim 1, characterized in that, Also includes: If the first Euclidean distance is not greater than the first preset threshold, then the multiple images are determined to be similar images; If the first Euclidean distance is not less than the second preset threshold, or the proportion of edge pixels in the edge features of each of the multiple images is not all greater than the third preset threshold, or the result of the horizontal / vertical pixel mean judgment is negative, or the average Wasserstein distance is not less than the fifth preset threshold, or the similarity between the scale-invariant feature transformation features is not less than the sixth preset threshold, then the multiple images are determined to be dissimilar images.

5. The similar image retrieval method according to claim 1, characterized in that, After extracting the multi-type features from the multiple images, the method further includes: After associating the multiple images with their various types of features, they are stored in an image database.

6. A similar image retrieval device, characterized in that, include: The matching calculation module is used to determine the similarity between multiple images based on multiple types of features through a progressive hierarchical matching operation. A similarity determination module is used to retrieve similar images among the multiple images based on the similarity score. The feature extraction module is used to extract the multi-type features of the multiple images. The multi-type features include: scale-invariant feature transformation features and relatively simple features. The relatively simple features include thumbnail pixel features, horizontal and vertical pixel mean features, edge features and histogram features. When the multiple feature types include thumbnail pixel features, edge features, horizontal and / or vertical pixel mean, histogram features, and scale-invariant feature transformation features, the matching calculation module is used to: Calculate the first Euclidean distance between the thumbnail pixel features of the multiple images, and if the first Euclidean distance is greater than a first preset threshold, determine whether the first Euclidean distance is less than a second preset threshold; If the first Euclidean distance is less than the second preset threshold, then determine whether the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold. If the proportion of edge pixels in the edge features of each of the multiple images is greater than the third preset threshold, then a horizontal / vertical pixel mean judgment is performed. The horizontal / vertical pixel mean judgment is used to determine whether the second Euclidean distance between the horizontal pixel mean values ​​of the multiple images and / or the third Euclidean distance between the vertical pixel mean values ​​are less than the fourth preset threshold. If the result of the horizontal / vertical pixel mean determination is yes, then determine whether the average Wasserstein distance between the histogram features of the multiple images is less than a fifth preset threshold. If the average Wasserstein distance is less than the fifth preset threshold, then it is further determined whether the similarity between the scale-invariant feature transformation features of the multiple images is less than the sixth preset threshold. If so, the similarity between the scale-invariant feature transformation features of the multiple images is obtained as the similarity between the multiple images.

7. A non-transitory computer-readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by a computer, they implement the steps of the similar image retrieval method as described in any one of claims 1 to 5.

8. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the similar image retrieval method as described in any one of claims 1 to 5.

Images