An image retrieval method and device, electronic equipment and storage medium
By segmenting the original image into segmented images of overlapping regions and calculating hash feature values, the problem of computational complexity and low accuracy in existing image retrieval methods is solved, achieving more efficient and accurate similar image retrieval.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING QIYI CENTURY SCI & TECH CO LTD
- Filing Date
- 2023-12-27
- Publication Date
- 2026-07-21
AI Technical Summary
Existing image retrieval methods are computationally complex and have low accuracy, especially when searching by image, where the computational load is large and the accuracy is insufficient.
The original image is segmented into multiple segments with overlapping regions. The hash feature value of each image is calculated, and these feature values are grouped and converted into different number systems. A pre-set image library is used to search for similar images, and similar images are determined by calculating a similarity threshold.
It improves the accuracy and computational efficiency of image retrieval, especially when dealing with images with varying brightness or after rotation and cropping, maintaining high retrieval accuracy and stability.
Smart Images
Figure CN117786152B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information technology, and in particular to an image retrieval method, apparatus, electronic device, and storage medium. Background Technology
[0002] Currently, image retrieval is increasingly widely used in work, study, and entertainment. Among these applications, image-based search is becoming increasingly popular. For example, on video websites and e-commerce sites, users can upload an image to search for similar images, such as identical clothing or similar plants.
[0003] However, current image search methods often calculate the similarity between user-uploaded images and images in the database. This method is not only computationally complex and computationally intensive, but also often has low accuracy. Summary of the Invention
[0004] The purpose of this invention is to provide an image retrieval method, apparatus, electronic device, and storage medium to achieve high accuracy in image retrieval. The specific technical solution is as follows:
[0005] In a first aspect of this invention, an image retrieval method is provided, comprising:
[0006] Obtain the original image;
[0007] The original image is segmented to obtain multiple segmented images, wherein there is an overlapping region between two adjacent segmented images corresponding to the original image.
[0008] Calculate the hash feature values of the multiple segmented images to obtain multiple first hash feature values, wherein each segmented image corresponds to one first hash feature value;
[0009] The plurality of first hash feature values are grouped to obtain a plurality of second hash feature values;
[0010] Based on the multiple sets of second hash feature values and the feature values of each image in the preset image library, similar images are retrieved to determine similar images in the preset image library, wherein the similarity between the feature values of the similar images and the second hash feature values is greater than a preset similarity threshold.
[0011] In one possible implementation, calculating the hash values of the multiple segmented images to obtain multiple first hash feature values includes:
[0012] For each segmented image, calculate the average pixel value of that segmented image to obtain the first average pixel value;
[0013] Calculate the average of the first pixel mean values corresponding to the multiple segmented images to obtain the second pixel mean value;
[0014] The value corresponding to each segmented image is calculated based on the second pixel mean and the first pixel mean corresponding to each segmented image;
[0015] The first hash feature value is obtained by calculating the corresponding hash feature value based on the assigned values of the multiple segmented images.
[0016] In one possible implementation, grouping the plurality of first hash feature values to obtain a plurality of second hash feature values includes:
[0017] The first hash feature value is grouped to obtain a first preset number of first feature values;
[0018] For each first feature value, the remaining first feature values are merged and grouped to obtain a second preset number of second feature values; the first feature value is then merged with each of the second feature values to obtain the multiple groups of second hash feature values.
[0019] In one possible implementation, segmenting the original image to obtain multiple segmented images includes:
[0020] The original image is segmented according to a preset segmentation direction to obtain multiple segmented images, wherein there is overlap between any two adjacent segmented images.
[0021] In one possible implementation, the step of retrieving similar images based on the plurality of sets of second hash feature values and the feature values of each image in the preset image library, and determining the similar images in the preset image library, includes:
[0022] The multiple sets of second hash feature values are converted into bases to obtain the target feature values corresponding to the first base, wherein the multiple sets of second hash feature values are binary numbers and the number of bits in the first base is higher than that in the binary number.
[0023] Select images from the preset image library whose corresponding first base feature values are the same as the target feature values to obtain multiple first images. The feature values of each image in the preset image library include a feature value corresponding to the first base and a feature value corresponding to the second base.
[0024] For each first image, calculate the similarity between the binary feature value and the second hash feature value corresponding to that image;
[0025] The similarity score is used to determine whether the first image is a similar image.
[0026] In one possible implementation, determining whether the first image is a similar image based on the calculated similarity includes:
[0027] For each first image, the calculated similarity of the first image is compared with a preset similarity threshold. If the similarity is greater than the threshold, then the first image is considered a similar image.
[0028] In one possible implementation, segmenting the original image to obtain multiple segmented images includes:
[0029] The original image is converted to grayscale to obtain a grayscale image;
[0030] The grayscale image is segmented to obtain multiple segmented images.
[0031] In one possible implementation, the step of converting the original image to grayscale to obtain a grayscale image includes:
[0032] The original image is scaled to obtain a scaled image;
[0033] The scaled image is converted to grayscale to obtain the grayscale image.
[0034] In a second aspect of the present invention, an image retrieval device is also provided, the device comprising:
[0035] The image acquisition module is used to acquire the original image;
[0036] The image segmentation module is used to segment the original image to obtain multiple segmented images, wherein there is an overlapping region between two adjacent segmented images corresponding to the original image.
[0037] The first feature value calculation module is used to calculate the hash feature values of the multiple segmented images to obtain multiple first hash feature values, wherein each segmented image corresponds to one first hash feature value;
[0038] The second feature value calculation module is used to group the plurality of first hash feature values to obtain a plurality of second hash feature values;
[0039] The similarity judgment module is used to retrieve similar images based on the multiple sets of second hash feature values and the feature values of each image in the preset image library, and to determine the similar images in the preset image library, wherein the similarity between the feature values of the similar images and the second hash feature values is greater than a preset similarity threshold.
[0040] In one possible implementation, the first feature value calculation module is specifically used to calculate the pixel mean of each segmented image to obtain a first pixel mean; calculate the average of the first pixel mean values corresponding to the multiple segmented images to obtain a second pixel mean; calculate the assignment corresponding to each segmented image based on the second pixel mean and the first pixel mean value corresponding to each segmented image; and calculate the corresponding hash feature value based on the assignment values corresponding to the multiple segmented images to obtain the first hash feature value.
[0041] In one possible implementation, the second feature value calculation module is specifically used to group the first hash feature value to obtain a first preset number of first feature values; for each first feature value, merge the remaining first feature values and group them to obtain a second preset number of second feature values; and merge the first feature value with each second feature value to obtain the plurality of groups of second hash feature values.
[0042] In one possible implementation, the image segmentation module is specifically used to segment the original image according to a preset segmentation direction to obtain the multiple segmented images, wherein there is overlap between any two adjacent segmented images.
[0043] In one possible implementation, the similarity judgment module is specifically used to convert the plurality of sets of second hash feature values into a base to obtain a target feature value corresponding to a first base, wherein the plurality of sets of second hash feature values are binary numbers and the number of bits in the first base is higher than that in the binary number; to select images from the preset image library whose corresponding first base feature values are the same as the target feature values, thereby obtaining a plurality of first images, wherein the feature values of each image in the preset image library include a feature value corresponding to the first base and a feature value corresponding to the binary number; for each first image, to calculate the similarity between the binary number feature value and the second hash feature value of the image; and to determine whether the first image is a similar image based on the calculated similarity.
[0044] In one possible implementation, the similarity judgment module is specifically used to compare the calculated similarity of each first image with a preset similarity threshold. If the similarity is greater than the threshold, then the first image is considered to be a similar image.
[0045] In one possible implementation, the image segmentation module is specifically used to perform grayscale processing on the original image to obtain a grayscale image; and to segment the grayscale image to obtain multiple segmented images.
[0046] In one possible implementation, the image segmentation module is specifically used to scale the original image to obtain a scaled image; and to convert the scaled image to grayscale to obtain a grayscale image.
[0047] In another aspect of the present invention, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.
[0048] Memory, used to store computer programs;
[0049] The processor, when executing a program stored in memory, implements any of the image retrieval methods described above.
[0050] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the image retrieval methods described above.
[0051] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image retrieval methods described above.
[0052] This invention provides an image retrieval method, apparatus, electronic device, and storage medium. The method involves: acquiring an original image; segmenting the original image to obtain multiple segmented images, wherein two adjacent segmented images corresponding to the original image have overlapping regions; calculating hash feature values for the multiple segmented images to obtain multiple first hash feature values, wherein each segmented image corresponds to one first hash feature value; grouping the multiple first hash feature values to obtain multiple groups of second hash feature values; and performing similar image retrieval based on the multiple groups of second hash feature values and feature values of images in a preset image library to determine similar images in the preset image library, wherein the similarity between the feature values of the similar images and the second hash feature values is greater than a preset similarity threshold. This method can segment the original image into multiple images with overlapping regions, and then calculate the hash feature value for each image, thereby improving the continuity of the sampled images and the calculated hash feature values, and improving the accuracy of similar image retrieval when performing similar image retrieval based on the calculated hash feature values. Attached Figure Description
[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0054] Figure 1This is a schematic flowchart of an image retrieval method provided in an embodiment of this application;
[0055] Figure 2 A schematic diagram illustrating the process of acquiring multiple segmented images provided in an embodiment of this application;
[0056] Figure 3 This is a schematic diagram illustrating the process of segmenting the original image according to an embodiment of this application;
[0057] Figure 4 A schematic diagram illustrating the determination of segmented image positions as provided in an embodiment of this application;
[0058] Figure 5 This is a schematic diagram of the process for obtaining the first hash feature value provided in an embodiment of this application;
[0059] Figure 6 This is a schematic diagram of the process for obtaining the second hash feature value provided in an embodiment of this application;
[0060] Figure 7 This is a schematic diagram illustrating the calculation of the second hash feature value provided in an embodiment of this application;
[0061] Figure 8 A schematic diagram illustrating the process of retrieving similar images provided in an embodiment of this application;
[0062] Figure 9 Another schematic flowchart of the image retrieval method provided in the embodiments of this application;
[0063] Figure 10 This is a schematic diagram of the structure of an image retrieval device provided in an embodiment of this application;
[0064] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0065] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.
[0066] To address the problem of low accuracy in image retrieval in existing technologies, this application provides an image retrieval method, apparatus, electronic device, and storage medium.
[0067] In a first aspect of this invention, an image retrieval method is provided, see [link to relevant documentation]. Figure 1 , Figure 1 A flowchart illustrating an image retrieval method provided in this application embodiment includes:
[0068] Step S11: Obtain the original image;
[0069] Step S12: Segment the original image to obtain multiple segmented images, wherein there is an overlapping region between two adjacent segmented images corresponding to the original image in the multiple segmented images;
[0070] Step S13: Calculate the hash feature values of the multiple segmented images to obtain multiple first hash feature values, wherein each segmented image corresponds to a first hash feature value;
[0071] Step S14: Group the plurality of first hash feature values to obtain a plurality of second hash feature values;
[0072] Step S15: Based on the multiple sets of second hash feature values and the feature values of each image in the preset image library, perform a search for similar images to determine the similar images in the preset image library, wherein the similarity between the feature values of the similar images and the second hash feature values is greater than a preset similarity threshold.
[0073] As can be seen, the method of this application embodiment can segment the original image into multiple images with overlapping regions, and then calculate the hash feature value of each image, thereby improving the continuity of the sampled images and the continuity of the calculated hash feature values, and improving the accuracy of similar image retrieval when retrieving similar images based on the calculated hash feature values.
[0074] Corresponding to step S11 above, the original image can be in various forms, such as images of people, animals, plants, or clothing. In one example, the method of this embodiment can be applied to the field of image search. The original image can be an image input by the user for similar image retrieval; for example, if the user is searching for the same style of clothing, the original image can be an image for searching the same style of clothing.
[0075] Corresponding to step S12 above, there is an overlapping area between two adjacent segmented images in the original image among the multiple segmented images. Specifically, when segmenting the original image, the overlap between adjacent blocks can be used to take into account the image features of the block boundaries, increase the continuity of sampling, make the feature value granularity of the image finer, and be compatible with similar image retrieval scenarios with large changes in brightness or color, or after rotation and cropping. This overcomes the problem in the existing feature extraction process that when there may be discontinuous features in the segmented image, the extracted features become abnormal after rotation and other operations, affecting the accuracy of retrieval.
[0076] Corresponding to step S13 above, each segmented image corresponds to a first hash feature value. When calculating the hash feature values of the multiple segmented images, a pre-built hash algorithm can be used to obtain the first hash feature value. Specifically, this hash algorithm can convert data of arbitrary length into a fixed-length value. For example, by performing a series of calculations and transformations on the input data, a unique binary string is generated, which is the first hash feature value. This first hash feature value can be used to represent the features of the multiple segmented images. For example, in practical use, calculating the hash feature values of the multiple segmented images can be achieved by first calculating the feature value of each segmented image, and then combining and transforming the feature values of each segmented image to obtain the first hash feature value.
[0077] Corresponding to step S14 above, the first hash feature value is grouped. This grouping can be done using various methods. Specifically, it can first be grouped using a first grouping method to obtain multiple groups, then grouped using a second grouping method to obtain multiple groups again, and finally combined by combining the groups obtained from different grouping methods to obtain the final group. For example, the different grouping methods can correspond to different group lengths. For instance, for a certain hash feature value, it can first be grouped using a first length to obtain four groups: a, b, c, and d. Then, each group can be further subdivided. For example, when grouping 'a', the remaining parts (b, c, and d) can be grouped using a second length to obtain 1, 2, 3, and 4. After combination, a1, a2, a3, and a4 can be obtained. Then, further subdividing b, c, and d can yield 16 groups.
[0078] Corresponding to step S15 above, the similarity between the feature value of the similar image and the second hash feature value is greater than a preset similarity threshold. In this embodiment, the hash feature value corresponding to each image in the preset image library can be pre-calculated, and the length of the hash feature value of the image can be the same as the length of the second hash feature value. When searching for similar images based on the multiple sets of second hash feature values and the feature values of each image in the preset image library, the similarity between the two can be calculated, and then judged by comparing the calculated similarity with a preset threshold. Images with a similarity greater than the preset threshold are determined as similar images.
[0079] In one possible implementation, see Figure 2 , Figure 2 This is a schematic diagram of the process for obtaining multiple segmented images provided in an embodiment of this application. Step S12 segments the original image to obtain multiple segmented images, including:
[0080] Step S121: Perform grayscale processing on the original image to obtain a grayscale image;
[0081] Step S122: Segment the grayscale image to obtain multiple segmented images.
[0082] In one possible implementation, the step of converting the original image to grayscale to obtain a grayscale image includes: scaling the original image to obtain a scaled image; and performing grayscale processing on the scaled image to obtain the grayscale image.
[0083] In this embodiment, after obtaining the original image, it can be converted to grayscale, and then segmented to obtain multiple segmented images. This not only facilitates subsequent processing but also reduces computational load. In practical use, scaling can also be performed during grayscale conversion to a preset size, facilitating subsequent processing. Grayscale conversion of the scaled image also facilitates pixel feature extraction. When scaling the original image, it can be scaled to a preset size. Specific specifications can be set according to actual conditions, such as width: 16 pixels, height: 64 pixels, or width: 16 pixels, height: 32 pixels, or width: 32 pixels, height: 64 pixels, etc. Similarly, segmentation of the grayscale image can be set according to actual conditions, such as width: 4 pixels, height: 8 pixels, or width: 8 pixels, height: 8 pixels, or width: 4 pixels, height: 16 pixels, or width: 8 pixels, height: 16 pixels, etc. See one example. Figure 3 , Figure 3 This is a schematic diagram of the process for segmenting the original image provided in an embodiment of this application. The original image is scaled to obtain an image with a width of 16 pixels and a height of 64 pixels. Then, the image is converted to grayscale to obtain a grayscale image. After grayscale conversion, the 16×64 image is cut into 64 small blocks. The block size of these 64 small blocks is 4 pixels wide and 8 pixels high. Then, the average hash value of each block is calculated to obtain a 64-bit binary feature value. The 64-bit hash value is then cut and stored in 16 tables.
[0084] In one possible implementation, segmenting the original image to obtain multiple segmented images includes: segmenting the original image according to a preset segmentation direction to obtain the multiple segmented images, wherein there is overlap between any two adjacent segmented images.
[0085] The preset segmentation direction can be any pre-defined direction, such as from left to right, from top to bottom, or along a diagonal. In this application, it is only necessary to segment adjacent regions step by step according to this direction. Before segmentation, the overlap between two adjacent images can be set, and then segmentation is performed according to this overlap. Specifically, the overlap can be along the horizontal or vertical axis. The overlap can be set according to the actual situation; for example, the overlap between adjacent blocks can be 50%, 45%, 40%, 55%, 60%, etc. For example, when segmenting images, adjacent blocks maintain a 40% overlap along the horizontal axis. See the example below. Figure 4 , Figure 4 This is a schematic diagram of determining the position of the segmented image provided in the embodiment of this application. The image is cut into 4×8 (4 pixels wide and 8 pixels high). In the figure, different line types represent different blocks. In the horizontal axis direction, adjacent blocks maintain a 50% overlap. For example, the first block and the second block in the figure have a 50% overlap. In the order from left to right and from top to bottom, with the X and Y axes as references in the figure, a total of 64 blocks are cut out.
[0086] In one possible implementation, see Figure 5 , Figure 5 This is a schematic diagram of the process for obtaining the first hash feature value provided in an embodiment of this application. Step S13 calculates the hash values of the multiple segmented images to obtain multiple first hash feature values, including:
[0087] Step S131: For each segmented image, calculate the average pixel value of the segmented image to obtain the first average pixel value.
[0088] Step S132: Calculate the average of the first pixel mean values corresponding to the multiple segmented images to obtain the second pixel mean value;
[0089] Step S133: Calculate the assigned value for each segmented image based on the second pixel mean and the first pixel mean corresponding to each segmented image;
[0090] Step S134: Calculate the corresponding hash feature value based on the assigned values of the multiple segmented images to obtain the first hash feature value.
[0091] Specifically, for each segmented image, the pixel mean of that segmented image can be calculated by calculating the pixel values in each segmented image and then averaging them. In this application, the average of the first pixel mean values corresponding to the multiple segmented images is calculated to compare with the first pixel mean values corresponding to the multiple segmented images, thereby calculating the corresponding first hash feature value based on the comparison result. For example, if the first pixel mean value corresponding to a segmented image is greater than the second pixel mean value, the comparison result is 1; if it is less, it is 0. Thus, a two-dimensional feature value, namely the first hash feature value, is created based on the comparison result of each segmented image. For example, if there are 4 segmented images with corresponding pixel mean values of 12, 24, 18, and 22, the second pixel mean value obtained by calculating the average is 19. Then, the first pixel mean value and the second pixel mean value corresponding to the segmented image are compared; if it is greater, it is 1; if it is less, it is 0, resulting in the first hash feature value: 0, 1, 0, 1.
[0092] For example, see Figure 3 Calculate the pixel mean of each of the 64 image blocks: {M1, M2, M3, ..., M64}. Then calculate the average of these 64 values: Md = median(Mi), where i = 1, 2, ..., 64. Finally, convert the mean sequence into a binary hash value using the following method:
[0093] 0,Mi <Md
[0094] h(i) =
[0095] 1,Mi>=Md
[0096] After conversion, a 64-bit image hash feature value is obtained.
[0097] In one possible implementation, see Figure 6 , Figure 6 This is a schematic diagram of the process for obtaining the second hash feature value provided in an embodiment of this application. Step S14 groups the plurality of first hash feature values to obtain a plurality of groups of second hash feature values, including:
[0098] Step S141: Group the first hash feature values to obtain a first preset number of first feature values;
[0099] Step S142: For each first feature value, merge the remaining first feature values and group them to obtain a second preset number of second feature values; merge each first feature value with each second feature value to obtain the multiple groups of second hash feature values.
[0100] When grouping the first hash feature value to obtain a first preset number of first feature values, and then merging and grouping the remaining first feature values for each first feature value, the hash feature values can be first grouped according to a first ratio, then one portion is taken, and the remaining portions are grouped according to a second ratio, and then a portion of those portions is taken. In actual use, the first ratio can be set according to the actual situation. For example, the first hash feature value can be divided into 4 equal parts, 2 equal parts, 3 equal parts, 5 equal parts, 6 equal parts, etc. Then, a portion is selected as the first feature value, and after merging the remaining portions, they are grouped according to the second ratio. Specifically, the second ratio can be the same as or different from the first ratio. For example, the remaining portions can be divided into 4 equal parts, 2 equal parts, 3 equal parts, 5 equal parts, 6 equal parts, etc. For example, see Figure 7 , Figure 7 The diagram illustrates the calculation of the second hash feature value provided in this application embodiment. First, the 64-bit binary data is divided into four equal parts. One part is taken, each containing 16 bits, and one of these parts is taken as the first part (there are four ways to take one part). The remaining 48-bit binary data is divided into four equal parts. One part is taken, i.e., the remaining 3 / 4 is further divided into four equal parts, each containing 12 bits, and one of these parts is taken as the second part (there are four ways to take one part). The 16 bits of the first part are concatenated with the 12 bits of the second part, and then concatenated in sequence to obtain a 28-bit binary data. There are a total of 16 combinations, resulting in 16 second hash feature values.
[0101] In one possible implementation, see Figure 8 , Figure 8 This is a schematic diagram of the process for retrieving similar images provided in an embodiment of this application. Step S15 retrieves similar images based on the multiple sets of second hash feature values and the feature values of each image in the preset image library, and determines the similar images in the preset image library, including:
[0102] Step S151: Convert the multiple sets of second hash feature values into bases to obtain the target feature values corresponding to the first base, wherein the multiple sets of second hash feature values are binary numbers and the number of bits in the first base is higher than that in the binary number.
[0103] Step S152: Select images from the preset image library whose corresponding first base feature values are the same as the target feature values to obtain multiple first images. The feature values of each image in the preset image library include the corresponding first base feature values and the corresponding binary feature values.
[0104] Step S153: For each first image, calculate the similarity between the binary feature value and the second hash feature value corresponding to that image;
[0105] Step S154: Determine whether the first image is a similar image based on the calculated similarity.
[0106] In one possible implementation, determining whether the first image is a similar image based on the calculated similarity includes: for each first image, comparing the calculated similarity corresponding to the first image with a preset similarity threshold; if the similarity is greater than the threshold, then determining whether the first image is a similar image.
[0107] Wherein, the multiple sets of second hash feature values are binary numbers, and the number of bits in the first base is higher than that in the second base. For example, the first base is decimal, and the second base is binary. By selecting images from the preset image library whose corresponding first base feature values are the same as the target feature values, multiple first images are obtained. Then, for each first image, the similarity between the corresponding binary feature value and the second hash feature value can be calculated, which can reduce the computational load of the similarity calculation process and thus improve computational efficiency. In this application, similarity can be calculated using various methods. In actual use, the similarity is used to determine whether the first image is a similar image. If so, it is output, and a link to the corresponding output image can also be output. For example, when searching for the same clothing, a purchase link corresponding to the image of the same clothing can also be output.
[0108] For example, after obtaining the second hash feature value, this 28-bit binary number is converted into a decimal number. There are 16 possible combinations, resulting in 16 decimal numbers, which are stored in 16 tables (table_1 to table_16). These tables also store the original 64-bit feature value of the image. When performing similar image retrieval, the user-input image is processed, and the 16 feature numbers are extracted using the previous steps. Then, images with the same value are searched for in the corresponding tables (table_1 to table_16). The retrieved images are then further analyzed using their 64-bit feature values to calculate the Hamming distance between them. Images with a Hamming distance between 0 and 3 are considered similar.
[0109] To illustrate the methods of the embodiments of this application, the following description is provided in conjunction with specific examples. (See attached examples.) Figure 9 , Figure 9 Another schematic flowchart of the image retrieval method provided in the embodiments of this application includes:
[0110] 1. After a user uploads an image, the original image is scaled to 16×64 (16 pixels wide and 64 pixels high), and then the image is converted to grayscale to obtain a grayscale image; after grayscale conversion, the 16×64 image is cut into 64 small blocks;
[0111] 2. The image is cut into 4×8 blocks (block size: 4 pixels wide, 8 pixels high). Adjacent blocks maintain a 50% overlap along the horizontal axis. A total of 64 blocks are created, arranged from left to right and top to bottom. The average hash value of each block is then calculated and concatenated in order to obtain a 64-bit binary sequence. The method for calculating the average hash value of image blocks is as follows:
[0112] Calculate the mean hash value for each block to obtain a 64-bit binary feature value. Calculate the pixel mean for each of the 64 image blocks: {M1, M2, M3, ..., M64}. Then calculate the average of these 64 values: Md = median(Mi), where i = 1, 2, ..., 64. Finally, convert the mean sequence into a binary hash value using the following method:
[0113] 0,Mi <Md
[0114] h(i) =
[0115] 1,Mi>=Md
[0116] After conversion, a 64-bit image hash feature value is obtained;
[0117] 3. After splitting the 64-bit hash value, store it in 16 tables. First, divide the 64 bits into 4 groups of 16 bits each, and take one group as the first part (there are 4 ways to do this). Divide the remaining 48 bits into 4 groups of 12 bits each, and take one group as the second part (there are 4 ways to do this). Concatenate the first 16 bits and the second 12 bits in order to get a 28-bit binary number. Then convert this 28-bit binary number into a decimal number. There are a total of 16 combinations, resulting in 16 decimal numbers, which are stored in 16 tables (table_1 to table_16). The 16 values are then searched for in the corresponding 16 tables to find the same image data. The tables also store the original 64-bit feature value of the image.
[0118] 4. Similar Image Retrieval: Following steps (1), (2), and (3) above, the user-input image is scaled to a width of 16 pixels and a height of 64 pixels. Then, the image is converted to grayscale. After grayscale conversion, the 16×64 image is divided into 64 smaller blocks, each with a width of 4 pixels and a height of 8 pixels. The mean hash value of each block is calculated to obtain a 64-bit binary feature value. This 64-bit hash value is then divided and stored in 16 tables. Each of the 16 values is used to search for identical image data in its corresponding table. Specifically, after extracting the 16 feature numbers, the images with the same values are searched in the corresponding 16 tables (table_1~table_16), resulting in 16 search results. The retrieved image data is then further processed using the 64-bit feature values of the images to calculate the Hamming distance. Specifically, for each of the 16 search results, the 64-bit hash value of each image is compared with the 64-bit hash value of the user-input image to calculate the Hamming distance. Images with a Hamming distance between 0 and 3 are considered similar, and thus images with a Hamming distance between 0 and 3 are returned to the user.
[0119] Common mean hash algorithms suffer from decreased retrieval accuracy when encountering images with significant variations in brightness or color, or images that have been rotated or cropped. The improved image hash algorithm in this application maintains high accuracy even in these situations, significantly enhancing algorithm stability. While this method can handle similar image retrieval in most scenarios, the data width of the image hash feature value remains 64 bits. To further improve accuracy, the width of the image hash feature value can be increased, for example, to 128 bits. This method utilizes overlapping blocks and appropriate block sizes to significantly improve the accuracy and stability of short video cover image similarity retrieval, enhancing accuracy and user experience in finding similar short videos using cover images.
[0120] In a second aspect of the invention, an image retrieval device is also provided, see [link to relevant documentation]. Figure 10 , Figure 10 This is a schematic diagram of an image retrieval device provided in an embodiment of this application. The device includes:
[0121] Image acquisition module 1001 is used to acquire raw images;
[0122] The image segmentation module 1002 is used to segment the original image to obtain multiple segmented images, wherein there is an overlapping region between two adjacent segmented images corresponding to the original image in the multiple segmented images;
[0123] The first feature value calculation module 1003 is used to calculate the hash feature values of the multiple segmented images to obtain multiple first hash feature values, wherein each segmented image corresponds to a first hash feature value;
[0124] The second feature value calculation module 1004 is used to group the plurality of first hash feature values to obtain a plurality of second hash feature values;
[0125] The similarity judgment module 1005 is used to retrieve similar images based on the multiple sets of second hash feature values and the feature values of each image in the preset image library, and to determine the similar images in the preset image library, wherein the similarity between the feature values of the similar images and the second hash feature values is greater than a preset similarity threshold.
[0126] In one possible implementation, the first feature value calculation module is specifically used to calculate the pixel mean of each segmented image to obtain a first pixel mean; calculate the average of the first pixel mean values corresponding to the multiple segmented images to obtain a second pixel mean; calculate the assignment corresponding to each segmented image based on the second pixel mean and the first pixel mean value corresponding to each segmented image; and calculate the corresponding hash feature value based on the assignment values corresponding to the multiple segmented images to obtain the first hash feature value.
[0127] In one possible implementation, the second feature value calculation module is specifically used to group the first hash feature value to obtain a first preset number of first feature values; for each first feature value, merge the remaining first feature values and group them to obtain a second preset number of second feature values; and merge the first feature value with each second feature value to obtain the plurality of groups of second hash feature values.
[0128] In one possible implementation, the image segmentation module is specifically used to segment the original image according to a preset segmentation direction to obtain the multiple segmented images, wherein there is overlap between any two adjacent segmented images.
[0129] In one possible implementation, the similarity judgment module is specifically used to convert the plurality of sets of second hash feature values into a base to obtain a target feature value corresponding to a first base, wherein the plurality of sets of second hash feature values are binary numbers and the number of bits in the first base is higher than that in the binary number; to select images from the preset image library whose corresponding first base feature values are the same as the target feature values, thereby obtaining a plurality of first images, wherein the feature values of each image in the preset image library include a feature value corresponding to the first base and a feature value corresponding to the binary number; for each first image, to calculate the similarity between the binary number feature value and the second hash feature value of the image; and to determine whether the first image is a similar image based on the calculated similarity.
[0130] In one possible implementation, the similarity judgment module is specifically used to compare the calculated similarity of each first image with a preset similarity threshold. If the similarity is greater than the threshold, then the first image is considered to be a similar image.
[0131] In one possible implementation, the image segmentation module is specifically used to perform grayscale processing on the original image to obtain a grayscale image; and to segment the grayscale image to obtain multiple segmented images.
[0132] In one possible implementation, the image segmentation module is specifically used to scale the original image to obtain a scaled image; and to convert the scaled image to grayscale to obtain a grayscale image.
[0133] As can be seen, the apparatus of this application embodiment can segment the original image into multiple images with overlapping regions, and then calculate the hash feature value of each image, thereby improving the continuity of the sampled images and the continuity of the calculated hash feature values, and improving the accuracy of similar image retrieval when retrieving similar images based on the calculated hash feature values.
[0134] This invention also provides an electronic device, such as... Figure 11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104, wherein the processor 1101, the communication interface 1102, and the memory 1103 communicate with each other through the communication bus 1104.
[0135] Memory 1103 is used to store computer programs;
[0136] When processor 1101 executes the program stored in memory 1103, it performs the following steps:
[0137] Obtain the original image;
[0138] The original image is segmented to obtain multiple segmented images, wherein there is an overlapping region between two adjacent segmented images corresponding to the original image.
[0139] Calculate the hash feature values of the multiple segmented images to obtain multiple first hash feature values, wherein each segmented image corresponds to one first hash feature value;
[0140] The plurality of first hash feature values are grouped to obtain a plurality of second hash feature values;
[0141] Based on the multiple sets of second hash feature values and the feature values of each image in the preset image library, similar images are retrieved to determine similar images in the preset image library, wherein the similarity between the feature values of the similar images and the second hash feature values is greater than a preset similarity threshold.
[0142] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.
[0143] The communication interface is used for communication between the aforementioned terminal and other devices.
[0144] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0145] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0146] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the image retrieval methods described in the above embodiments.
[0147] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image retrieval methods described in the above embodiments.
[0148] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).
[0149] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0150] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, storage media, and computer program products are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0151] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. An image retrieval method, characterized in that, The method includes: Obtain the original image; The original image is segmented to obtain multiple segmented images, wherein there is an overlapping region between two adjacent segmented images corresponding to the original image. Calculate the hash feature values of the multiple segmented images to obtain multiple first hash feature values, wherein each segmented image corresponds to one first hash feature value; The plurality of first hash feature values are grouped to obtain a plurality of second hash feature values; The multiple sets of second hash feature values are converted into bases to obtain the target feature values corresponding to the first base, wherein the multiple sets of second hash feature values are binary numbers and the number of bits in the first base is higher than that in the binary number. Select images from a preset image library whose corresponding first base feature values are the same as the target feature values to obtain multiple first images. The feature values of each image in the preset image library include a feature value corresponding to the first base and a feature value corresponding to the second base. For each first image, calculate the similarity between the feature value of the corresponding binary number of the image and the plurality of first hash feature values; The similarity is used to determine whether the first image is a similar image, wherein the similarity between the feature value of the similar image and the plurality of first hash feature values is greater than a preset similarity threshold.
2. The method according to claim 1, characterized in that, The calculation of the hash values of the multiple segmented images yields multiple first hash feature values, including: For each segmented image, calculate the average pixel value of that segmented image to obtain the first average pixel value; Calculate the average of the first pixel mean values corresponding to the multiple segmented images to obtain the second pixel mean value; The value corresponding to each segmented image is calculated based on the second pixel mean and the first pixel mean corresponding to each segmented image; The first hash feature value is obtained by calculating the corresponding hash feature value based on the assigned values of the multiple segmented images.
3. The method according to claim 1, characterized in that, The process of grouping the plurality of first hash feature values to obtain a plurality of second hash feature values includes: The plurality of first hash feature values are grouped to obtain a first preset number of first feature values; For each first feature value, the remaining first feature values are merged and grouped to obtain a second preset number of second feature values; the first feature value is then merged with each of the second feature values to obtain the multiple groups of second hash feature values.
4. The method according to claim 1, characterized in that, The process of segmenting the original image to obtain multiple segmented images includes: The original image is segmented according to a preset segmentation direction to obtain multiple segmented images, wherein there is overlap between any two adjacent segmented images.
5. The method according to claim 1, characterized in that, The step of determining whether the first image is a similar image based on the calculated similarity includes: For each first image, the calculated similarity of the first image is compared with a preset similarity threshold. If the similarity is greater than the threshold, then the first image is considered a similar image.
6. The method according to claim 1, characterized in that, The process of segmenting the original image to obtain multiple segmented images includes: The original image is converted to grayscale to obtain a grayscale image; The grayscale image is segmented to obtain multiple segmented images.
7. The method according to claim 6, characterized in that, The step of converting the original image to grayscale to obtain a grayscale image includes: The original image is scaled to obtain a scaled image; The scaled image is converted to grayscale to obtain the grayscale image.
8. An image retrieval device, characterized in that, The device includes: The image acquisition module is used to acquire the original image; The image segmentation module is used to segment the original image to obtain multiple segmented images, wherein there is an overlapping region between two adjacent segmented images corresponding to the original image. The first feature value calculation module is used to calculate the hash feature values of the multiple segmented images to obtain multiple first hash feature values, wherein each segmented image corresponds to one first hash feature value; The second feature value calculation module is used to group the plurality of first hash feature values to obtain a plurality of second hash feature values; The similarity judgment module is used to perform base conversion on the multiple sets of second hash feature values to obtain the target feature value corresponding to the first base, wherein the multiple sets of second hash feature values are binary numbers and the number of bits in the first base is higher than that in the binary number. The similarity judgment module is further configured to select images from a preset image library whose corresponding first base feature values are the same as the target feature values, thereby obtaining multiple first images. The feature values of each image in the preset image library include a feature value corresponding to the first base and a feature value corresponding to the second base. The similarity judgment module is further configured to calculate the similarity between the binary feature value corresponding to each first image and the plurality of first hash feature values for each first image; and to determine whether the first image is a similar image based on the calculated similarity, wherein the similarity between the feature value of the similar image and the plurality of first hash feature values is greater than a preset similarity threshold.
9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.