Image retrieval method, device and storage medium

Through multiple multi-dimensional comparisons and calculations of feature vectors and pixel clustering vectors, the problem of low image retrieval accuracy in the prior art is solved, high-precision and high-efficiency image retrieval is achieved, and the search of application scenarios such as e-commerce is met. Accuracy requirements.

CN113886628BActive Publication Date: 2024-11-15SHANGHAI SHIZHUANG INFORMATION TECHNOLOGY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111176709.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-09
Publication Date
2024-11-15
Estimated Expiration
2041-10-09

AI Technical Summary

Technical Problem

The existing image retrieval methods have low accuracy and cannot meet the problems of high retrieval accuracy in some application scenarios.

Method used

By receiving the images to be retrieved, performing initial screening to narrow the range of the image library, calculating image feature vectors and pixel clustering vectors, and comparing multiple multi-dimensionalities to determine the final similar image, improving the retrieval accuracy.

Benefits of technology

It realizes high-precision image retrieval, reduces the error recognition rate and miss recognition rate, improves user experience and retrieval efficiency, and meets real-time requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113886628B_ABST
    Figure CN113886628B_ABST
Patent Text Reader

Abstract

The present invention discloses an image retrieval method, device and storage medium, the method comprising: receiving a retrieval request; preliminarily screening images in an image library according to the image to be retrieved to obtain a first image set similar to the image to be retrieved; calculating a first image feature vector of the image to be retrieved, and determining a second image set according to the first image feature vector and a second image feature vector of an image in the first image set; calculating a first image pixel clustering vector of the image to be retrieved and a second image pixel clustering vector of an image in the second image set, and obtaining a final similar image of the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector. Through the embodiment of the present invention, when performing image retrieval, images in the retrieval image library whose similarity with the image to be retrieved meets the required standard and related information of the image can be returned.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image retrieval method, device and storage medium. Background Art

[0002] Image retrieval refers to inputting a picture to be searched and then retrieving the same or similar pictures and information related to the picture in the picture library. Image retrieval has a wide range of practical applications. For unfamiliar animals and plants, you can use image retrieval software, website applets, etc. to take a photo and upload the picture, and then identify the category of the animal or plant. In the e-commerce platform APP, you can use image retrieval to take a photo and upload the product picture to find the target product.

[0003] At present, in some application scenarios, the retrieval accuracy is very high. For example, e-commerce platforms search for the same product display pictures to prevent plagiarism and infringement of promotional images, and the platform repeatedly lists the same products. When the goods on the e-commerce platform are damaged, the defective pictures are used for after-sales claims. The customer service system automatically searches for defective images to determine whether the claimant has used the pictures that were previously claimed to make a second claim to defraud compensation. These image retrieval application scenarios require that the retrieval result pictures are exactly the same or highly similar to the pictures to be retrieved, and the retrieval accuracy is high. The retrieval results are not allowed to have missed returns or wrong returns.

[0004] However, the commonly used technology in the current image retrieval process is: using the convolutional neural network method to extract image semantic features, then vectorizing the features to obtain feature vectors, and comparing the similarity between images by calculating the distance between vectors. This image retrieval method has low accuracy and only returns the first few similar images that meet the set criteria. It requires further screening by users or secondary judgment by back-end manual reviewers, resulting in poor user experience, low review efficiency, and failure to meet high-precision retrieval requirements. Summary of the invention

[0005] The main purpose of the present invention is to provide an image retrieval method, device and storage medium, aiming to solve the technical problem that the image retrieval method in the prior art has low accuracy and cannot meet the requirements of some application scenarios with high retrieval accuracy requirements.

[0006] To achieve the above object, the present invention provides an image retrieval method, which comprises the following steps:

[0007] receiving a search request, wherein the search request carries an image to be searched;

[0008] Preliminarily screening images in an image library according to the image to be retrieved to obtain a first image set similar to the image to be retrieved;

[0009] Calculating a first image feature vector of the image to be retrieved, and determining a second image set according to the first image feature vector and second image feature vectors of images in the first image set;

[0010] The first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set are calculated, and a final similar image of the image to be retrieved is obtained according to the first image pixel clustering vector and the second image pixel clustering vector.

[0011] Optionally, the step of preliminarily screening images in an image library according to the image to be retrieved to obtain a first image set similar to the image to be retrieved includes:

[0012] Calculate the hash code of the image to be retrieved according to a preset algorithm;

[0013] Obtaining a central cluster hash code of an image cluster in the image library;

[0014] Determining whether the image cluster is a similar cluster to the image to be retrieved according to the hash code of the image to be retrieved and the central cluster hash code of the image cluster;

[0015] If it is determined to be a similar cluster, the first image set is determined according to the hash codes of all images in the image cluster and the hash code of the image to be retrieved.

[0016] Optionally, the step of determining whether the image cluster is a similar cluster of the image to be retrieved based on the hash code of the image to be retrieved and the central cluster hash code of the image cluster includes:

[0017] Calculating a first Hamming distance between a central hash code of the image cluster and a hash code of the image to be retrieved;

[0018] If the first Hamming distance is less than a first preset threshold, the image cluster is determined to be a similar cluster of the image to be retrieved.

[0019] Optionally, the step of determining the first image set according to the hash codes of all images in the image cluster and the hash codes of the image to be retrieved includes:

[0020] Traversing all images in the image cluster;

[0021] Calculating a second Hamming distance between a hash code of a first target image in the image cluster and a hash code of the image to be retrieved;

[0022] If the second Hamming distance is less than a second preset threshold, the target image is determined to be a similar image to the image to be retrieved, until all images in the image cluster are traversed to obtain the first image set.

[0023] Optionally, the step of determining the second image set according to the first image feature vector and a second image feature vector of an image in the first image set includes:

[0024] Obtaining second image feature vectors of all images in the first image set;

[0025] respectively calculating a first cosine distance between a first image feature vector of the image to be retrieved and a second image feature vector of a second target image in the first image set;

[0026] If the first cosine distance is less than a third preset threshold, the second target image is determined to be a similar image to the image to be retrieved, until all images in the image cluster are traversed to obtain the second image set.

[0027] Optionally, the step of calculating the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set includes:

[0028] Performing image pixel value clustering processing on the RGB three channels of the image to be retrieved and the RGB three channels of the images in the second image set respectively, to obtain a first pixel value clustering cluster of the image to be retrieved and a second pixel value clustering cluster of the images in the second image set;

[0029] The central value of the first pixel value clustering cluster and the central value of the second pixel value clustering cluster are calculated to obtain the first image pixel clustering vector and the second image pixel clustering vector.

[0030] Optionally, the second image set includes a third target image, and the step of obtaining a similar image to the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector comprises:

[0031] respectively calculating a second cosine distance between the first image pixel clustering vector and the second image feature vector of the third target image;

[0032] If the second cosine distance is less than a fourth preset threshold, the second target image is determined to be a similar image to the image to be retrieved, until all images in the second image set are traversed to obtain a final similar image to the image to be retrieved.

[0033] Optionally, the method further comprises the following steps:

[0034] The index number of the final similar image is output, so that the image corresponding to the index number and related information thereof can be obtained from the image library according to the index number and displayed to the user.

[0035] In addition, to achieve the above object, the present invention also provides an image retrieval device, the device comprising:

[0036] A receiving unit, configured to receive a search request, wherein the search request carries an image to be searched;

[0037] A preliminary screening unit, configured to perform preliminary screening on images in an image library according to the image to be retrieved received by the receiving unit, to obtain a first image set similar to the image to be retrieved;

[0038] A first processing unit, configured to calculate a first image feature vector of the image to be retrieved, and determine a second image set according to the first image feature vector and a second image feature vector of an image in the first image set;

[0039] The second processing unit is used to calculate the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set, and obtain the final similar image of the image to be retrieved based on the first image pixel clustering vector and the second image pixel clustering vector.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the image retrieval method described above are implemented.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image retrieval method described above are implemented.

[0042] The technical solution of the present invention, when there is an image retrieval request, narrows the retrieval range of images in the image library by performing multiple multi-dimensional comparisons between the image to be retrieved and the images in the image library, so that images in the retrieval image library whose similarity with the image to be retrieved meets the required standards and related information of the image can be returned, the retrieval accuracy is high, all images that meet the conditions can be retrieved, and the retrieval result has low false recognition rate and missed recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of a flow chart of a first embodiment of the image retrieval method provided by the present invention;

[0044] Figure 2 for Figure 1A flow chart of an embodiment of step 102;

[0045] Figure 3 for Figure 1 A flow chart of an embodiment of step 103;

[0046] Figure 4 for Figure 1 A flow chart of an embodiment of step 104;

[0047] Figure 5 A schematic diagram of a second embodiment of the image retrieval method provided by the present invention

[0048] Figure 6 A structural block diagram of the image retrieval device provided by the present invention;

[0049] Figure 7 The present invention provides a schematic structural diagram of an electronic device.

[0050] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0052] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0054] In one embodiment, Figure 1 As shown, the present invention provides an image retrieval method, the method comprising:

[0055] Step 101: Receive a search request, wherein the search request carries an image to be searched.

[0056] In an embodiment of the present application, a user or operator uploads an image to be retrieved on an APP or website, and the retrieval system obtains the image to be retrieved and related information and stores them on a server, and then the retrieval system sends a retrieval request to the server. After receiving the retrieval request from the retrieval system, the server runs an image retrieval program to obtain similar images of the image to be retrieved. It can be understood that the retrieval request should carry the image to be retrieved so that the server can run the image retrieval program based on the image to be retrieved to obtain similar images.

[0057] Step 102: Preliminarily screen the images in the image library according to the image to be retrieved to obtain a first image set similar to the image to be retrieved.

[0058] When this step is specifically performed, the server uses the perceptual hash algorithm to roughly screen all images stored in the image library according to the image to be retrieved, so as to narrow the scope of the retrieval image and obtain the first image set. For example, the initial screening method can be: the server can first calculate the hash code of the image to be retrieved, and then obtain the central cluster hash code of each image cluster in the image library, and determine the similar cluster of the image to be retrieved according to the hash code of the image to be retrieved and the central cluster hash code of the image cluster, and then find images similar to the image to be retrieved from the similar clusters, thereby narrowing the retrieval scope and reducing the computational workload of the retrieval work, which can improve the retrieval efficiency to a certain extent.

[0059] Step 103: Calculate the first image feature vector of the image to be retrieved, and determine the second image set according to the first image feature vector and the second image feature vector of the image in the first image set.

[0060] In this step, when calculating the image feature vector of the image to be retrieved, for example, the server may use a trained convolutional neural network to extract the image feature vector, wherein the convolutional neural network may include VggNet, ResNet, DenseNet, EfficieNet, etc., and the feature vector size is 1*512. The second image feature vectors of all images in the first image set are extracted by the server in advance using the above-mentioned trained convolutional neural network and stored in the database where the image library is located.

[0061] In this step, when determining the second image set, for example, the cosine distance between the feature vector of the image to be retrieved and the feature vectors of all images in the first image set can be calculated one by one, and when the cosine distance is less than a set value, the image is judged to be a similar image of the image to be retrieved, and finally the similar images obtained by judgment are combined to form the second image set.

[0062] Step 104: Calculate the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set, and obtain a final similar image of the image to be retrieved based on the first image pixel clustering vector and the second image pixel clustering vector.

[0063] In this step, K-Means is used to cluster the image pixel values ​​of the RGB three channels of the image to be retrieved, and a set number of pixel value clusters are obtained. The center value of each pixel value cluster is calculated to obtain the pixel cluster vector of the image to be retrieved. Each channel corresponds to a cluster vector, and the vector length is 10. Using the same method, the image pixel cluster vectors of all images in the second image set can be calculated. Then, when determining similar images, the cosine distance between the cluster vector of the image to be retrieved and the cluster vectors of all images in the second image set can be calculated one by one, and when the cosine distance is less than the set value, it is determined to be the similar image to be retrieved in the end.

[0064] In an embodiment of the present invention, the image to be retrieved is first screened, a first image set similar to the image to be retrieved is obtained, and a second image set is determined based on the image feature vector of the image to be retrieved and the image feature vector of the image in the first image set, and then the image pixel clustering vector of the image to be retrieved and the image pixel clustering vector of the image in the second image set are compared with each other, thereby narrowing the search scope step by step by making multiple multi-dimensional comparisons between the image to be retrieved and the images in the image library, and returning the relevant information of images and similar images in the image library whose similarity with the image to be retrieved meets the required standards, and all images that meet the conditions can be retrieved, and the false recognition rate and missed recognition rate of the search results are low, so that the obtained search results are highly accurate. At the same time, the search process takes less time, and a single search takes less than 200 milliseconds, which meets the real-time search performance requirements. At the same time, there is no need for users to further screen or make secondary judgments on the search results, which improves the user experience.

[0065] Please refer to Figure 2 , Figure 2 Provided Figure 1 The specific implementation of step 102 in the embodiment, that is, when the images in the image library are preliminarily screened according to the image to be retrieved to obtain a first image set similar to the image to be retrieved, the specific process can be as follows Figure 2 As shown, the following steps are included:

[0066] Step 1021: Calculate the hash code of the image to be retrieved according to a preset algorithm.

[0067] In this step, the server converts the image to be retrieved into a grayscale image, resulting in a channel number of 1. The grayscale image is then scaled to 64x64, and then the scaled grayscale image is discrete cosine transformed to obtain a frequency domain data matrix. The data matrix is ​​converted into a one-dimensional vector from left to right and from top to bottom, and then the average value of all values ​​in the vector is calculated. Each value in the vector is then compared. If it is greater than or equal to the average value, it is 1, and if it is less than the average value, it is 0, and the hash code of the image to be detected is obtained.

[0068] Step 1022: Obtain the central cluster hash code of the image cluster in the image library.

[0069] Before this step is executed, the hash codes of all images in the image library have been calculated and stored in the database of the image library according to the method described in step 1021. K-means is used for clustering to obtain several clusters, each of which has a central cluster hash code. The purpose of this step is to obtain the central cluster hash code of each image cluster from the database of the image library.

[0070] Step 1023: Determine whether the image cluster is a similar cluster to the image to be retrieved based on the hash code of the image to be retrieved and the central cluster hash code of the image cluster.

[0071] Step 1024: If it is determined to be a similar cluster, the first image set is determined according to the hash codes of all images in the image cluster and the hash code of the image to be retrieved.

[0072] When step 1023-step 1024 is specifically executed, it can be: calculate the Hamming distance between the hash code of the image to be retrieved and the central cluster code of each image clustering cluster, if the Hamming distance is less than a preset threshold, they are similar, and the corresponding image clustering cluster is a similar cluster. Then calculate the Hamming distance between the hash code of the image to be retrieved and the image hash codes corresponding to all images in the similar cluster, if the Hamming distance is less than a preset threshold, they are similar images.

[0073] Specifically, when determining the first image set according to the hash codes of all images in the image cluster and the hash code of the image to be retrieved, the operation method may be:

[0074] In the image clustering cluster determined as a similar cluster, all images in the image clustering cluster are traversed in sequence, and the Hamming distance between the hash code of each target image in the image clustering cluster and the hash code of the image to be retrieved is calculated one by one; if the Hamming distance is less than a second preset threshold, the target image is determined to be a similar image of the image to be retrieved, until all images in the image clustering cluster are traversed. Finally, all the retrieved similar images are combined into a first image set.

[0075] In this embodiment, when there is an image retrieval request, the retrieval scope of the images in the image library is gradually narrowed by comparing the image to be retrieved with the images in the image library in multiple dimensions, and then the relevant information of the images in the image library that meet the required standards and similar images with the similarity to the image to be retrieved can be returned. The retrieval accuracy is high, all images that meet the conditions can be retrieved, and the retrieval result has low false recognition rate and missed recognition rate. In addition, by first calculating the Hamming distance between the central hash code in the image cluster and the hash code of the image to be retrieved, similar clusters are found and image similarity retrieval is performed only in similar clusters. In this way, the retrieval scope of the image library can be narrowed, the amount of calculation can be reduced, the retrieval time can be shortened, the retrieval efficiency can be improved, and the real-time retrieval requirements can be met.

[0076] Please refer to Figure 3 , Figure 3 Provided Figure 1 In the specific implementation of step 103 in the embodiment, that is, when determining the second image set according to the first image feature vector and the second image feature vector of the image in the first image set, the specific process can be as follows: Figure 3 As shown, the following steps are included:

[0077] Step 1031: Obtain second image feature vectors of all images in the first image set.

[0078] Among them, all images in the image library have used the trained convolutional neural network described above to extract image feature vectors and store them in the database where the image library is located, which of course also includes the image feature vectors of all images in the first image set. Therefore, when this step is specifically performed, it is only necessary to read the image feature vectors of all images in the first image set from the database of the image library.

[0079] Step 1032: Calculate first cosine distances between a first image feature vector of the image to be retrieved and a second image feature vector of a second target image in the first image set.

[0080] Step 1033: If the first cosine distance is less than a third preset threshold, determine that the second target image is a similar image to the image to be retrieved, until all images in the image cluster are traversed to obtain the second image set.

[0081] In the specific operation of step 1032-step 1033, all images in the first image set are traversed in sequence, and the cosine distance between the feature vector of each target image in the first image set and the feature vector of the image to be retrieved is calculated one by one; if the cosine distance is less than the third preset threshold, the target image is determined to be a similar image of the image to be retrieved, until all images in the first image set are traversed. Finally, all the retrieved similar images are combined into the second image set.

[0082] In this embodiment, when there is an image retrieval request, the retrieval scope of images in the image library is gradually narrowed by multiple multi-dimensional comparisons, so that images in the image library whose similarity to the image to be retrieved meets the required standards and related information of similar images can be returned, the retrieval accuracy is high, all images that meet the conditions can be retrieved, and the retrieval result has low false recognition rate and missed recognition rate. In addition, by calculating the cosine distance between the feature vector of the image to be detected and all images in the first image set, the retrieval scope of the image library can be further narrowed, the amount of calculation can be reduced, the retrieval time can be shortened, the retrieval efficiency can be improved, and the real-time retrieval requirements can be met.

[0083] Please refer to Figure 4 , Figure 4 Provided Figure 1 The specific implementation of step 104 in the embodiment, that is, when calculating the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set, and obtaining similar images of the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector, the specific process can be as follows: Figure 4 As shown, the following steps are included:

[0084] Specifically, when calculating the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set, steps 1041 and 1042 are included. When obtaining similar images of the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector, steps 1043 and 1044 are included.

[0085] Step 1041 , performing image pixel value clustering processing on the RGB three channels of the image to be retrieved and the RGB three channels of the images in the second image set, respectively, to obtain a first pixel value clustering cluster of the image to be retrieved and a second pixel value clustering cluster of the images in the second image set.

[0086] Step 1042: Calculate the center value of the first pixel value cluster and the center value of the second pixel value cluster to obtain the first image pixel cluster vector and the second image pixel cluster vector.

[0087] In step 1041-step 1042, when specifically executed, K-Means can be used to cluster the image pixel values ​​of the RGB three channels of the image to be retrieved, obtain a set number of pixel value clusters, and calculate the center value of each pixel value cluster to obtain the pixel cluster vector of the image to be retrieved, each channel corresponds to a cluster vector, and the vector length is 10. Using the same method, the image pixel cluster vectors of all images in the second image set can be calculated.

[0088] Step 1043 , respectively calculating the second cosine distance between the first image pixel clustering vector and the second image feature vector of the third target image.

[0089] Step 1044: if the second cosine distance is less than a fourth preset threshold, determine that the third target image is a similar image to the image to be retrieved, until all images in the second image set are traversed to obtain similar images to the image to be retrieved.

[0090] In step 1041-step 1042, since the second image set includes multiple images, when determining similar images, the cosine distance between the cluster vector of the image to be retrieved and the cluster vectors of all images in the second image set (i.e., multiple third target objects) can be calculated one by one. If the cosine distance is less than the fourth preset threshold, the target image is determined to be a similar image of the image to be retrieved, until all images in the second image set are traversed. Finally, all retrieved similar images are output.

[0091] In this embodiment, when there is an image retrieval request, the retrieval scope of images in the image library is narrowed by multiple multi-dimensional comparisons, so that the images in the image library whose similarity with the image to be retrieved meets the required standards and the related information of similar images can be returned, the retrieval accuracy is high, all images that meet the conditions can be retrieved, and the retrieval results have low false recognition rate and missed recognition rate. In addition, by calculating the cosine distance between the cluster vector of the image to be detected and all images in the second image set, the retrieval scope of the image library can be further narrowed, the amount of calculation can be reduced, the retrieval time can be shortened, the retrieval efficiency can be improved, and the real-time retrieval requirements can be met.

[0092] Please refer to Figure 5 , Figure 5 Provided Figure 1 The flowchart after step 104 in the embodiment, that is, after executing step 104, may also include:

[0093] Step 105: output the index number of the final similar image, so that the image corresponding to the index number and related information thereof can be obtained from the image library according to the index number and displayed to the user.

[0094] The server can output the search results to the search system. In specific operations, the server returns the index number of the final similar image retrieved in the image library to the search system. The search system uses the search number to access the image library and obtain the image and image-related information corresponding to the index number, and displays the image and image-related information on the display interface of the APP or website where the search initiator is located.

[0095] In this embodiment, when there is an image retrieval request, the retrieval scope of the images in the image library is narrowed by multiple multi-dimensional comparisons, so that the images in the image library whose similarity to the image to be retrieved meets the required standards and the related information of similar images can be returned. The retrieval accuracy is high, all images that meet the conditions can be retrieved, and the retrieval results have low false recognition rate and missed recognition rate. In addition, the retrieved similar pictures are returned to the retrieval system in the form of index numbers. In this way, the retrieval system can access the image library according to the index number and obtain the image and image-related information corresponding to the index number, and display the image and image-related information on the display interface of the APP or website where the retrieval initiator is located.

[0096] In addition, the embodiment of the present invention also provides an image retrieval device, referring to Figure 6 , the image retrieval device comprises:

[0097] The receiving unit 10 is used to receive a search request, wherein the search request carries an image to be searched;

[0098] A preliminary screening unit 20, configured to perform preliminary screening on images in an image library according to the image to be retrieved received by the receiving unit 10, to obtain a first image set similar to the image to be retrieved;

[0099] A first processing unit 30 is used to calculate a first image feature vector of the image to be retrieved, and determine a second image set according to the first image feature vector and a second image feature vector of an image in the first image set;

[0100] The second processing unit 40 is used to calculate the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set, and obtain the final similar image of the image to be retrieved based on the first image pixel clustering vector and the second image pixel clustering vector.

[0101] Through the above scheme, this embodiment first performs a preliminary screening of the image to be retrieved, obtains a first image set similar to the image to be retrieved, and determines the second image set based on the image feature vector of the image to be retrieved and the image feature vector of the image in the first image set, and then based on the image pixel clustering vector of the image to be retrieved and the image pixel clustering vector of the image in the second image set, the search scope is gradually narrowed by performing multiple multi-dimensional comparisons between the image to be retrieved and the images in the image library, and can return the relevant information of images and similar images in the search image library whose similarity with the image to be retrieved meets the required standards, and can retrieve all images that meet the conditions, and the search result has low false recognition rate and missed recognition rate, so that the obtained search result has high accuracy. At the same time, the search process takes less time, and a single search takes less than 200 milliseconds, which meets the real-time search performance requirements. At the same time, there is no need for users to further screen or make secondary judgments on the search results, which improves the user experience.

[0102] It should be noted that each unit in the above-mentioned device can be used to implement each step in the above-mentioned method and achieve corresponding technical effects, which will not be described in detail in this embodiment.

[0103] Reference Figure 7 , Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0104] like Figure 7 As shown, the electronic device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as WI-FI, 4G, 5G interface). The memory 1005 may be a high-speed RAM memory, or it may be a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0105] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0106] like Figure 7As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an image retrieval program.

[0107] exist Figure 7 In the electronic device shown, the network interface 1004 is mainly used for data communication with an external network; the user interface 1003 is mainly used for receiving user input instructions; the electronic device calls the image retrieval program stored in the memory 1005 through the processor 1001, and performs the following operations:

[0108] receiving a search request, wherein the search request carries an image to be searched;

[0109] Preliminarily screening images in an image library according to the image to be retrieved to obtain a first image set similar to the image to be retrieved;

[0110] Calculating a first image feature vector of the image to be retrieved, and determining a second image set according to the first image feature vector and second image feature vectors of images in the first image set;

[0111] A first image pixel clustering vector of the image to be retrieved and a second image pixel clustering vector of the image in the second image set are calculated, and a final similar image of the image to be retrieved is obtained according to the first image pixel clustering vector and the second image pixel clustering vector.

[0112] Optionally, the step of preliminarily screening images in an image library according to the image to be retrieved to obtain a first image set similar to the image to be retrieved includes:

[0113] Calculate the hash code of the image to be retrieved according to a preset algorithm;

[0114] Obtaining a central cluster hash code of an image cluster in the image library;

[0115] Determining whether the image cluster is a similar cluster to the image to be retrieved according to the hash code of the image to be retrieved and the central cluster hash code of the image cluster;

[0116] If it is determined to be a similar cluster, the first image set is determined according to the hash codes of all images in the image cluster and the hash code of the image to be retrieved.

[0117] Optionally, the step of determining whether the image cluster is a similar cluster of the image to be retrieved based on the hash code of the image to be retrieved and the central cluster hash code of the image cluster includes:

[0118] Calculating a first Hamming distance between a central hash code of the image cluster and a hash code of the image to be retrieved;

[0119] If the first Hamming distance is less than a first preset threshold, the image cluster is determined to be a similar cluster of the image to be retrieved.

[0120] Optionally, the step of determining the first image set according to the hash codes of all images in the image cluster and the hash codes of the image to be retrieved includes:

[0121] Traversing all images in the image cluster;

[0122] Calculating a second Hamming distance between a hash code of a first target image in the image cluster and a hash code of the image to be retrieved;

[0123] If the second Hamming distance is less than a second preset threshold, the first target image is determined to be a similar image to the image to be retrieved, until all images in the image cluster are traversed to obtain the first image set.

[0124] Optionally, the step of determining the second image set according to the first image feature vector and a second image feature vector of an image in the first image set includes:

[0125] Obtaining second image feature vectors of all images in the first image set;

[0126] respectively calculating a first cosine distance between a first image feature vector of the image to be retrieved and a second image feature vector of a second target image in the first image set;

[0127] If the first cosine distance is less than a third preset threshold, the second target image is determined to be a similar image to the image to be retrieved, until all images in the image cluster are traversed to obtain the second image set.

[0128] Optionally, the step of calculating the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set includes:

[0129] Performing image pixel value clustering processing on the RGB three channels of the image to be retrieved and the RGB three channels of the images in the second image set respectively, to obtain a first pixel value clustering cluster of the image to be retrieved and a second pixel value clustering cluster of the images in the second image set;

[0130] The central value of the first pixel value clustering cluster and the central value of the second pixel value clustering cluster are calculated to obtain the first image pixel clustering vector and the second image pixel clustering vector.

[0131] Optionally, the second image set includes a third target image, and the step of obtaining a final similar image of the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector comprises:

[0132] respectively calculating a second cosine distance between the first image pixel clustering vector and the second image feature vector of the third target image;

[0133] If the second cosine distance is less than a fourth preset threshold, the third target image is determined to be a similar image to the image to be retrieved, until all images in the second image set are traversed to obtain a final similar image to the image to be retrieved.

[0134] Optionally, the method further comprises the following steps:

[0135] The index number of the final similar image is output, so that the image corresponding to the index number and related information thereof can be obtained from the image library according to the index number and displayed to the user.

[0136] Through the above scheme, when there is an image retrieval request, this embodiment narrows the retrieval scope of images in the image library through multiple multi-dimensional comparisons, so that images in the image library that meet the required standards and related information of similar images can be returned. The retrieval accuracy is high, all images that meet the conditions can be retrieved, and the retrieval results have low false recognition rate and missed recognition rate. In addition, the retrieved similar images are returned to the retrieval system in the form of index numbers. In this way, the retrieval system can access the image library according to the index number and obtain the image and image-related information corresponding to the index number, and display the image and image-related information on the display interface of the APP or website where the retrieval initiator is located.

[0137] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which an image retrieval program is stored. When the image retrieval program is executed by a processor, the following operations are implemented:

[0138] receiving a search request, wherein the search request carries an image to be searched;

[0139] Preliminarily screening images in an image library according to the image to be retrieved to obtain a first image set similar to the image to be retrieved;

[0140] Calculating a first image feature vector of the image to be retrieved, and determining a second image set according to the first image feature vector and second image feature vectors of images in the first image set;

[0141] A first image pixel clustering vector of the image to be retrieved and a second image pixel clustering vector of the image in the second image set are calculated, and a final similar image of the image to be retrieved is obtained according to the first image pixel clustering vector and the second image pixel clustering vector.

[0142] Optionally, the step of preliminarily screening images in an image library according to the image to be retrieved to obtain a first image set similar to the image to be retrieved includes:

[0143] Calculate the hash code of the image to be retrieved according to a preset algorithm;

[0144] Obtaining a central cluster hash code of an image cluster in the image library;

[0145] Determining whether the image cluster is a similar cluster to the image to be retrieved according to the hash code of the image to be retrieved and the central cluster hash code of the image cluster;

[0146] If it is determined to be a similar cluster, the first image set is determined according to the hash codes of all images in the image cluster and the hash code of the image to be retrieved.

[0147] Optionally, the step of determining whether the image cluster is a similar cluster of the image to be retrieved based on the hash code of the image to be retrieved and the central cluster hash code of the image cluster includes:

[0148] Calculate the hash code between the center of the image cluster and the hash code of the image to be retrieved

[0149] The first Hamming distance of

[0150] If the first Hamming distance is less than a first preset threshold, the image cluster is determined to be a similar cluster of the image to be retrieved.

[0151] Optionally, the step of determining the first image set according to the hash codes of all images in the image cluster and the hash codes of the image to be retrieved includes:

[0152] Traversing all images in the image cluster;

[0153] Calculating a second Hamming distance between a hash code of a first target image in the image cluster and a hash code of the image to be retrieved;

[0154] If the second Hamming distance is less than a second preset threshold, the first target image is determined to be a similar image to the image to be retrieved, until all images in the image cluster are traversed to obtain the first image set.

[0155] Optionally, the step of determining the second image set according to the first image feature vector and a second image feature vector of an image in the first image set includes:

[0156] Obtaining second image feature vectors of all images in the first image set;

[0157] respectively calculating a first cosine distance between a first image feature vector of the image to be retrieved and a second image feature vector of a second target image in the first image set;

[0158] If the first cosine distance is less than a third preset threshold, the second target image is determined to be a similar image to the image to be retrieved, until all images in the image cluster are traversed to obtain the second image set.

[0159] Optionally, the step of calculating the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set includes:

[0160] Performing image pixel value clustering processing on the RGB three channels of the image to be retrieved and the RGB three channels of the images in the second image set respectively, to obtain a first pixel value clustering cluster of the image to be retrieved and a second pixel value clustering cluster of the images in the second image set;

[0161] The central value of the first pixel value clustering cluster and the central value of the second pixel value clustering cluster are calculated to obtain the first image pixel clustering vector and the second image pixel clustering vector.

[0162] Optionally, the second image set includes a third target image, and the step of obtaining a final similar image of the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector comprises:

[0163] respectively calculating a second cosine distance between the first image pixel clustering vector and the second image feature vector of the third target image;

[0164] If the second cosine distance is less than a fourth preset threshold, the third target image is determined to be a similar image to the image to be retrieved, until all images in the second image set are traversed to obtain a final similar image to the image to be retrieved.

[0165] Optionally, the method further comprises the following steps:

[0166] Output the index number of the final similar image, so that the image corresponding to the index number and its related information can be obtained from the image library according to the index number and displayed to the user

[0167] Through the above scheme, when there is an image retrieval request, this embodiment narrows the retrieval scope of images in the image library through multiple multi-dimensional comparisons, so that images in the image library that meet the required standards and related information of similar images can be returned. The retrieval accuracy is high, all images that meet the conditions can be retrieved, and the retrieval results have low false recognition rate and missed recognition rate. In addition, the retrieved similar images are returned to the retrieval system in the form of index numbers. In this way, the retrieval system can access the image library according to the index number and obtain the image and image-related information corresponding to the index number, and display the image and image-related information on the display interface of the APP or website where the retrieval initiator is located.

[0168] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0169] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, controller, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0171] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image retrieval method, characterized in that: The method comprises the following steps: The retrieval server receives a retrieval request sent by the retrieval system, wherein the retrieval request is received by the retrieval system from a user and carries an image to be retrieved; The retrieval server calculates the hash code of the image to be retrieved according to a preset algorithm, obtains the hash code of the center cluster of the image cluster in the image library, determines whether the image cluster is a similar cluster of the image to be retrieved according to the hash code of the image to be retrieved and the hash code of the center cluster of the image cluster, and if it is determined to be a similar cluster, determines the first image set according to the hash codes of all images in the image cluster and the hash code of the image to be retrieved; wherein the hash code of the center cluster of the image cluster in the image library has been pre-calculated and stored in the database of the image library; The retrieval server calculates a first image feature vector of the image to be retrieved, and determines a second image set according to the first image feature vector and second image feature vectors of images in the first image set; The retrieval server calculates a first image pixel clustering vector of the image to be retrieved and a second image pixel clustering vector of the image in the second image set, and obtains a final similar image of the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector; The step of determining the second image set according to the first image feature vector and the second image feature vector of the image in the first image set comprises: The retrieval server obtains the second image feature vectors of all images in the first image set; the second image feature vectors of all images in the first image set have been pre-calculated and stored in a database of the image library; The retrieval server calculates the first cosine distance between the first image feature vector of the image to be retrieved and the second image feature vector of the second target image in the first image set one by one; If the first cosine distance is less than a third preset threshold, the retrieval server determines that the second target image is an intermediate similar image of the image to be retrieved, until all images in the first image set are traversed, all intermediate similar images are combined, and the second image set is obtained; The step of calculating the first image pixel clustering vector of the image to be retrieved and the second image pixel clustering vector of the image in the second image set includes: The retrieval server performs image pixel value clustering processing on the RGB three channels of the image to be retrieved and the RGB three channels of the images in the second image set, respectively, to obtain a first pixel value clustering cluster of the image to be retrieved and a second pixel value clustering cluster of the images in the second image set; The retrieval server calculates the central value of the first pixel value cluster and the central value of the second pixel value cluster to obtain the first image pixel cluster vector and the second image pixel cluster vector.

2. The method according to claim 1, characterized in that The step of determining whether the image cluster is a similar cluster of the image to be retrieved based on the hash code of the image to be retrieved and the central cluster hash code of the image cluster comprises: The retrieval server calculates the first Hamming distance between the central hash code of the image cluster and the hash code of the image to be retrieved one by one; If the first Hamming distance is less than a first preset threshold, the search server determines that the image cluster is a similar cluster of the image to be searched.

3. The method according to claim 2, characterized in that The step of determining the first image set according to the hash codes of all images in the image cluster and the hash codes of the image to be retrieved comprises: The retrieval server obtains the hash codes of all images in the image cluster; the hash codes of all images in the image cluster have been pre-calculated and stored in the database of the image library; The retrieval server calculates the second Hamming distance between the hash code of each target image in the image cluster and the hash code of the image to be retrieved one by one; If the second Hamming distance is less than a second preset threshold, the retrieval server determines that the first target image is an initial similar image of the image to be retrieved until all images in the image cluster are traversed; and combines all the initial similar images to obtain the first image set.

4. The method according to claim 1, characterized in that: The second image set includes a third target image, and the step of obtaining a final similar image of the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector includes: The retrieval server calculates the second cosine distance between the first image pixel clustering vector and the second image feature vector of the third target image respectively; If the second cosine distance is less than a fourth preset threshold, the retrieval server determines that the third target image is a similar image to the image to be retrieved, until all images in the second image set are traversed to obtain a final similar image to the image to be retrieved.

5. The method according to claim 1, characterized in that The method further comprises the following steps: The retrieval server outputs the index number of the final similar image, so that the retrieval system can obtain the image corresponding to the index number and related information from the image library according to the index number and display them to the user.

6. An image retrieval device, characterized in that: The device comprises: A receiving unit, configured to receive a search request sent by a search system, wherein the search request is received by the search system from a user and carries an image to be searched; A primary screening unit, used to calculate the hash code of the image to be retrieved according to a preset algorithm, obtain the central cluster hash code of the image clustering cluster in the image library, determine whether the image clustering cluster is a similar cluster of the image to be retrieved according to the hash code of the image to be retrieved and the central cluster hash code of the image clustering cluster, and if it is determined to be a similar cluster, determine the first image set according to the hash codes of all images in the image clustering cluster and the hash code of the image to be retrieved; wherein the central cluster hash code of the image clustering cluster in the image library has been pre-calculated and stored in the database of the image library; A first processing unit, configured to calculate a first image feature vector of the image to be retrieved, and determine a second image set according to the first image feature vector and a second image feature vector of an image in the first image set; a second processing unit, configured to calculate a first image pixel clustering vector of the image to be retrieved and a second image pixel clustering vector of the image in the second image set, and obtain a final similar image of the image to be retrieved according to the first image pixel clustering vector and the second image pixel clustering vector; Wherein, the first processing unit is further used for: The retrieval server obtains the second image feature vectors of all images in the first image set; the second image feature vectors of all images in the first image set have been pre-calculated and stored in a database of the image library; The retrieval server calculates the first cosine distance between the first image feature vector of the image to be retrieved and the second image feature vector of the second target image in the first image set one by one; If the first cosine distance is less than a third preset threshold, the retrieval server determines that the second target image is an intermediate similar image of the image to be retrieved, until all images in the first image set are traversed, all intermediate similar images are combined, and the second image set is obtained; Wherein, the second processing unit is further used for: The retrieval server performs image pixel value clustering processing on the RGB three channels of the image to be retrieved and the RGB three channels of the images in the second image set, respectively, to obtain a first pixel value clustering cluster of the image to be retrieved and a second pixel value clustering cluster of the images in the second image set; The retrieval server calculates the central value of the first pixel value cluster and the central value of the second pixel value cluster to obtain the first image pixel cluster vector and the second image pixel cluster vector.

7. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and an image retrieval program stored in the memory and executable on the processor, wherein the image retrieval program is configured to implement the steps of the image retrieval method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image retrieval method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Ciphertext image retrieval method and system under a cloud environment

    CN108959478A

  • Image retrieval method and device, electronic equipment and storage medium

    CN111651625A

  • Image index generation based on similarities of image features

    US20130195361A1

  • Automatic Canonical Digital Image Selection Method and Apparatus

    US20180005088A1

  • Image search method, apparatus, device, and computer-readable storage medium

    WO2020182019A1