Image similarity determination method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image similarity determination method and device, electronic equipment and a storage medium. The method comprises the steps that at least one feature vector corresponding to a first image and at least one feature vector corresponding to a second image are determined respectively; the feature vector comprises any one of a histogram feature vector, a texture feature vector, a local feature vector and a global feature vector; fusing the histogram eigenvector, the texture eigenvector and the local eigenvector corresponding to the first image and the histogram eigenvector, the texture eigenvector and the local eigenvector corresponding to the second image to obtain a first fused eigenvector corresponding to the first image and a second fused eigenvector corresponding to the second image; and determining the target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, and the global feature vector corresponding to the second image, and the second fusion feature vector, thereby realizing accurate determination of the target similarity, and improving the accuracy of the image similarity.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular, to a method, apparatus, electronic device, and storage medium for determining image similarity. Background Art

[0002] With the rapid development of computer vision, image processing technology has also been continuously advancing and is widely applied in fields such as image detection, image filtering, and object detection. As a basic research in the field of image processing, image similarity measurement calculates the similarity of images by studying features such as the spatial position, color distribution, and texture distribution of images, so as to analyze the similarity degree between two or more images.

[0003] Although current image similarity algorithms are constantly being updated, how to effectively improve the accuracy of image similarity remains an urgent problem to be solved in the field of image processing. Summary of the Invention

[0004] In view of the above problems existing in the prior art, embodiments of the present application provide a method, apparatus, electronic device, and storage medium for determining image similarity.

[0005] In a first aspect, embodiments of the present application provide a method for determining image similarity, including:

[0006] Respectively determining at least one feature vector corresponding to a first image and a second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector;

[0007] Fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image respectively, to obtain a first fused feature vector corresponding to the first image and a second fused feature vector corresponding to the second image;

[0008] Based on the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector, determining the target similarity between the first image and the second image.

[0009] Optionally, according to the method for determining image similarity of an embodiment of the present application, fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image to obtain a first fused feature vector corresponding to the first image, including:

[0010] Transpose the histogram feature vector and the texture feature vector corresponding to the first image to obtain the transposed histogram feature vector and the transposed texture feature vector;

[0011] Connect the transposed histogram feature vector and the transposed texture feature vector to obtain a connected feature vector;

[0012] Based on the connected feature vector and the local feature vector, determine the first fusion feature vector corresponding to the first image.

[0013] Optionally, according to the image similarity determination method of an embodiment of the present application, determining the target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector includes:

[0014] Based on the first fusion feature vector and the second fusion feature vector, determine the first similarity between the first image and the second image;

[0015] Based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image, determine the second similarity between the first image and the second image;

[0016] Based on the first similarity and the second similarity, determine the target similarity between the first image and the second image.

[0017] Optionally, according to the image similarity determination method of an embodiment of the present application, determining the first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector includes:

[0018] Based on the first fusion feature vector and the second fusion feature vector, use the nearest neighbor algorithm to determine the first similarity between the first image and the second image.

[0019] Optionally, according to the image similarity determination method of an embodiment of the present application, determining the second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image includes:

[0020] Perform an inner product calculation on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image to obtain an inner product result;

[0021] Square the inner product result to determine the second similarity between the first image and the second image.

[0022] Optionally, according to the image similarity determination method of an embodiment of the present application, determining the target similarity between the first image and the second image based on the first similarity and the second similarity includes:

[0023] Perform a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image; the weight corresponding to the first similarity is greater than the weight corresponding to the second similarity.

[0024] Optionally, according to the image similarity determination method of an embodiment of the present application, determining the histogram feature vector corresponding to the first image includes:

[0025] Perform a block processing on the first image to obtain a plurality of image blocks;

[0026] Respectively determine the histogram feature vectors of each of the image blocks;

[0027] Perform a weighted summation on the histogram feature vectors of each of the image blocks to obtain the histogram feature vector corresponding to the first image.

[0028] Optionally, according to the image similarity determination method of an embodiment of the present application, the step of respectively determining the histogram feature vectors of each of the image blocks includes:

[0029] For each image block, determine the pixel region corresponding to the pixel value of each pixel point in the image block; the number of pixel regions is determined based on the value range of the pixel value;

[0030] Count the number of pixel points in each of the pixel regions;

[0031] Based on the number of pixel points in each of the pixel regions, determine the histogram feature vector of the image block.

[0032] Optionally, according to the image similarity determination method of an embodiment of the present application, determining the texture feature vector corresponding to the first image includes

[0033] Use the LBP algorithm to determine the texture feature image corresponding to the first image;

[0034] Perform a block processing on the texture feature image to obtain a plurality of texture image blocks;

[0035] Respectively determine the histogram feature vectors of each of the texture image blocks;

[0036] Weightedly sum the histogram feature vectors of each of the texture image blocks to obtain the texture feature vector corresponding to the first image.

[0037] In a second aspect, an embodiment of the present application further provides an electronic device, including a memory, a transceiver, and a processor:

[0038] The memory is used to store a computer program; the transceiver is used to transmit and receive data under the control of the processor; the processor is used to read the computer program in the memory and perform the following operations:

[0039] Respectively determine at least one feature vector corresponding to each of the first image and the second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector;

[0040] Fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image, respectively, to obtain a first fused feature vector corresponding to the first image and a second fused feature vector corresponding to the second image;

[0041] Based on the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector, determine the target similarity between the first image and the second image.

[0042] Optionally, for the electronic device according to an embodiment of the present application, fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image to obtain the first fused feature vector corresponding to the first image includes:

[0043] Transpose the histogram feature vector and the texture feature vector corresponding to the first image to obtain a transposed histogram feature vector and a transposed texture feature vector;

[0044] Connect the transposed histogram feature vector and the transposed texture feature vector to obtain a connected feature vector;

[0045] Based on the connected feature vector and the local feature vector, determine the first fused feature vector corresponding to the first image.

[0046] Optionally, for an electronic device according to an embodiment of the present application, determining the target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector includes:

[0047] Determining a first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector;

[0048] Determining a second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image;

[0049] Determining the target similarity between the first image and the second image based on the first similarity and the second similarity.

[0050] Optionally, for an electronic device according to an embodiment of the present application, determining the first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector includes:

[0051] Determining the first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector by using the nearest neighbor algorithm.

[0052] Optionally, for an electronic device according to an embodiment of the present application, determining the second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image includes:

[0053] Performing an inner product calculation on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image to obtain an inner product result;

[0054] Determining the square of the inner product result as the second similarity between the first image and the second image.

[0055] Optionally, for an electronic device according to an embodiment of the present application, determining the target similarity between the first image and the second image based on the first similarity and the second similarity includes:

[0056] Performing a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image; the weight corresponding to the first similarity is greater than the weight corresponding to the second similarity.

[0057] Optionally, an electronic device according to an embodiment of the present application determines a histogram feature vector corresponding to a first image, including:

[0058] Performing a blocking process on the first image to obtain a plurality of image blocks;

[0059] Respectively determining the histogram feature vectors of the image blocks;

[0060] Performing weighted summation on the histogram feature vectors of the image blocks to obtain the histogram feature vector corresponding to the first image.

[0061] Optionally, for the electronic device according to an embodiment of the present application, the step of respectively determining the histogram feature vectors of the image blocks includes:

[0062] For each image block, determining a pixel region corresponding to the pixel value of each pixel point in the image block; the number of pixel regions is determined based on the value range of the pixel value;

[0063] Counting the number of pixel points in each pixel region;

[0064] Based on the number of pixel points in each pixel region, determining the histogram feature vector of the image block.

[0065] Optionally, for the electronic device according to an embodiment of the present application, determining a texture feature vector corresponding to a first image includes

[0066] Using the LBP algorithm to determine a texture feature image corresponding to the first image;

[0067] Performing a blocking process on the texture feature image to obtain a plurality of texture image blocks;

[0068] Respectively determining the histogram feature vectors of the texture image blocks;

[0069] Performing weighted summation on the histogram feature vectors of the texture image blocks to obtain the texture feature vector corresponding to the first image.

[0070] In a third aspect, an embodiment of the present application further provides an image similarity determination device, including:

[0071] A first determination module, configured to respectively determine at least one feature vector corresponding to a first image and a second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector;

[0072] A feature fusion module, configured to fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image respectively, to obtain a first fusion feature vector corresponding to the first image and a second fusion feature vector corresponding to the second image;

[0073] A second determination module, configured to determine a target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector.

[0074] Optionally, the feature fusion module is specifically configured to:

[0075] Transpose the histogram feature vector and the texture feature vector corresponding to the first image to obtain a transposed histogram feature vector and a transposed texture feature vector;

[0076] Concatenate the transposed histogram feature vector and the transposed texture feature vector to obtain a concatenated feature vector;

[0077] Determine the first fusion feature vector corresponding to the first image based on the concatenated feature vector and the local feature vector.

[0078] Optionally, the second determination module is specifically configured to:

[0079] Determine a first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector;

[0080] Determine a second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image;

[0081] Determine the target similarity between the first image and the second image based on the first similarity and the second similarity.

[0082] Optionally, the second determination module is further configured to:

[0083] Determine the first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector by using the nearest neighbor algorithm.

[0084] Optionally, the second determination module is further configured to:

[0085] Perform an inner product calculation on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image to obtain an inner product result;

[0086] Square the inner product result to determine the second similarity between the first image and the second image.

[0087] Optionally, the second determination module is further configured to:

[0088] Perform a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image; the weight corresponding to the first similarity is greater than the weight corresponding to the second similarity.

[0089] Optionally, the first determination module is specifically configured to:

[0090] Perform a block processing on the first image to obtain a plurality of image blocks;

[0091] Respectively determine the histogram feature vectors of each of the image blocks;

[0092] Perform a weighted summation on the histogram feature vectors of each of the image blocks to obtain the histogram feature vector corresponding to the first image.

[0093] Optionally, the first determination module is further configured to:

[0094] For each image block, determine the pixel region corresponding to the pixel value of each pixel point in the image block; the number of pixel regions is determined based on the value range of the pixel value;

[0095] Count the number of pixel points in each of the pixel regions;

[0096] Based on the number of pixel points in each of the pixel regions, determine the histogram feature vector of the image block.

[0097] Optionally, the first determination module is further configured to:

[0098] Use the LBP algorithm to determine the texture feature image corresponding to the first image;

[0099] Perform a block processing on the texture feature image to obtain a plurality of texture image blocks;

[0100] Respectively determine the histogram feature vectors of each of the texture image blocks;

[0101] Perform a weighted summation on the histogram feature vectors of each of the texture image blocks to obtain the texture feature vector corresponding to the first image.

[0102] Fourthly, an embodiment of the present application further provides a processor-readable storage medium storing a computer program for causing the processor to execute the steps of the image similarity determination method described in the first aspect above.

[0103] Fifthly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program for causing a computer to execute the steps of the image similarity determination method provided in the first aspect above.

[0104] The image similarity determination method, device, electronic device, and storage medium provided by the embodiments of the present application determine at least one feature vector corresponding to each of the first image and the second image respectively; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector; the histogram feature vector, texture feature vector, and local feature vector corresponding to the first image, and the histogram feature vector, texture feature vector, and local feature vector corresponding to the second image are respectively fused to obtain a first fused feature vector corresponding to the first image and a second fused feature vector corresponding to the second image; then, according to the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector, the target similarity between the first image and the second image is determined. Through the fusion of the histogram feature vector, texture feature vector, and local feature vector, the first fused feature vector and the second fused feature vector with rich image feature information can be determined, and thus the target similarity between the first image and the second image can be accurately determined, improving the accuracy of image similarity. Description of the Drawings

[0105] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following briefly introduces the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0106] Figure 1 is one of the flowcharts of the image similarity determination method provided by the embodiments of the present application;

[0107] Figure 2 is a schematic diagram of multiple image blocks corresponding to the first image provided by the embodiments of the present application;

[0108] Figure 3 is a schematic diagram of multiple texture image blocks corresponding to the first image provided by the embodiments of the present application;

[0109] Figure 4It is the second schematic flowchart of the image similarity determination method provided by an embodiment of the present application;

[0110] Figure 5 It is the schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0111] Figure 6 It is the schematic structural diagram of an image similarity determination device provided by an embodiment of the present application. Detailed implementation manners

[0112] In the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0113] In the embodiments of the present application, the term "plurality" means two or more, and other quantifiers are similar thereto.

[0114] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0115] To facilitate a clearer understanding of the embodiments of the present application, some related technical knowledge is introduced as follows.

[0116] Currently, there are various methods for determining image similarity, but the accuracy and efficiency are poor. For example, the classic Scale-Invariant Feature Transform (SIFT) based on content detects and describes the local features of an image. It finds extreme points in different spatial scales and then extracts position, scale, and rotation invariants. Although the SIFT algorithm has been widely applied to image similarity matching, the features extracted by the SIFT algorithm are local features of the image, without global information and semantic information. In addition, the SIFT algorithm has no way to distinguish the textures of different images, and it is difficult to extract feature points for dark images or flat areas. In practical applications, the accuracy of the SIFT image similarity algorithm is limited.

[0117] The color histogram is a commonly used image similarity algorithm that measures the similarity in color distribution between two images. The histogram algorithm presents the comparison of image similarity by counting the number of pixels of different colors in the image and presenting it in the form of a histogram. The advantages of the histogram algorithm are that it is simple and easy to understand, has a fast calculation speed, and has a certain robustness to image transformations such as rotation and scaling. However, the color histogram ignores the position information of the image and may misjudge some images with similar color distributions but different textures, considering these dissimilar images to have a high similarity.

[0118] In view of the above problems existing in the prior art, the embodiments of the present application provide an image similarity determination method, apparatus, electronic device, and storage medium, which can improve the accuracy of image similarity.

[0119] Figure 1 is one of the schematic flowcharts of the image similarity determination method provided by the embodiments of the present application, as Figure 1 shown, the method includes step 101-step 103; wherein,

[0120] Step 101, respectively determine at least one feature vector corresponding to the first image and the second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector.

[0121] It should be noted that the image similarity determination method provided by the embodiments of the present application can be applied to the scenario of image similarity matching. The execution subject of this method can be an image similarity determination apparatus, such as an electronic device, or a control module in the image similarity determination apparatus for executing the image similarity determination method.

[0122] Specifically, the first image and the second image are images to be matched. The texture feature vector represents the local binary pattern (LBP) feature of the image texture, the local feature vector represents the SIFT feature of the image, and the global feature vector represents the global Gist feature of the image.

[0123] Before respectively determining at least one feature vector corresponding to the first image and the second image, it is necessary to scale the first image and the second image respectively so that the sizes of the first image and the second image are the same. For example, the sizes of the scaled first image and the second image are both 256*256. Then convert the scaled first image and the second image from RGB images to grayscale images, that is, the three RGB components become one grayscale component, aiming to reduce the amount of calculation while maintaining the distribution characteristics of the overall and local chromaticity and brightness levels of the image.

[0124] For an RGB image, R, G, and B represent the red, green, and blue color channels respectively, and each color is divided into 256 levels of brightness. The conversion formula between RGB and grayscale (Gray) is:

[0125] Gray = R·0.229 + G·0.587 + B·0.114 (1)

[0126] Based on the first image and the second image after grayscale conversion, the histogram feature vectors, texture feature vectors, and local feature vectors of the first image and the second image can be determined respectively. For the first image and the second image without grayscale conversion, the global feature vector of the first image and the global feature vector of the second image can be determined respectively.

[0127] Step 102: Fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image respectively, to obtain the first fusion feature vector corresponding to the first image and the second fusion feature vector corresponding to the second image.

[0128] Specifically, by using a multi-feature fusion algorithm to fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, the first fusion feature vector corresponding to the first image can be obtained; by using a multi-feature fusion algorithm to fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image, the second fusion feature vector corresponding to the second image can be obtained. Among them, the first fusion feature vector and the second fusion feature vector contain rich image feature information, which can improve the accuracy of image similarity.

[0129] Step 103: Based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector, determine the target similarity between the first image and the second image.

[0130] Specifically, according to the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector, the target similarity between the first image and the second image can be determined, that is, by combining the global feature vector and the fusion feature vector, the target similarity can be accurately determined, thereby improving the accuracy of image similarity.

[0131] The image similarity determination method provided by the embodiments of the present application determines at least one feature vector corresponding to the first image and the second image respectively; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector; the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image are respectively fused to obtain a first fused feature vector corresponding to the first image and a second fused feature vector corresponding to the second image; then, according to the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector, the target similarity between the first image and the second image is determined. Through the fusion of the histogram feature vector, the texture feature vector, and the local feature vector, the first fused feature vector and the second fused feature vector with rich image feature information can be determined, and thus the target similarity between the first image and the second image can be accurately determined, improving the accuracy of image similarity.

[0132] Optionally, determining the histogram feature vector corresponding to the first image includes:

[0133] Performing a block processing on the first image to obtain a plurality of image blocks; respectively determining the histogram feature vectors of the image blocks; and performing a weighted summation on the histogram feature vectors of the image blocks to obtain the histogram feature vector corresponding to the first image.

[0134] Specifically, for the first image after grayscale conversion, the first image is subjected to block processing to obtain a plurality of image blocks. For example, Figure 2 is a schematic diagram of a plurality of image blocks corresponding to the first image provided by the embodiments of the present application. As Figure 2 shown, the 256*256 first image is segmented into a plurality of 4*4 image blocks (block), forming two circles. Among them, the inner circle has 4 image blocks (6, 7, 8, and 9 respectively represent an image block), and the outer circle has 12 image blocks (1, 2, 3, 4, 5, 8, 9, 12, 13, 14, 15, and 16 respectively represent an image block), and the size of each image block is 64*64.

[0135] Respectively determining the histogram feature vectors of each image block, and performing a weighted summation on the histogram feature vectors of the image blocks can obtain the histogram feature vector corresponding to the first image. For example, multiplying the histogram feature vectors of the image blocks in the Figure 2 outer circle by the weight a = 0.5, and multiplying the histogram feature vectors of the image blocks in the inner circle by the weight a = 1, which is expressed by the formula as:

[0136]

[0137] where, Hj denotes the histogram feature vector of the j-th image block, and H denotes the histogram feature vector after weighted summation of each image block.

[0138] Then, normalize H, that is, divide the histogram feature vector of each image block by H, and the histogram feature vector v1 corresponding to the first image can be obtained. The dimension of v1 is 128.

[0139] It should be noted that the determination method of the histogram feature vector corresponding to the second image is the same as that of the histogram feature vector corresponding to the first image. To avoid repetition, this application will not repeat it here.

[0140] Optionally, the step of separately determining the histogram feature vector of each image block includes:

[0141] For each image block, determine the pixel region corresponding to the pixel value of each pixel point in the image block; the number of pixel regions is determined based on the value range of the pixel value; count the number of pixel points in each pixel region; and determine the histogram feature vector of the image block based on the number of pixel points in each pixel region.

[0142] Specifically, the value range of the pixel value of each pixel point in the image block is [0, 255]. The value range is divided into 128 pixel regions, and each pixel region contains 2 pixel values. For example, pixel values 0 and 1 correspond to pixel region 1, pixel values 2 and 3 correspond to pixel region 2,..., pixel values 254 and 255 correspond to pixel region 128. Then, for each image block, determine the pixel region corresponding to the pixel value of each pixel point in the image block; count the number of pixel points h(x k ) = m k , k ∈ [0, 255], m k represents the number of pixels with pixel value m in the k-th pixel region. Then, the histogram feature vector of the j-th image block is expressed as:

[0143] H j = [h(x0), h(x1), …, h(x n )] = [h(x0), h(x1), …, h(x 127 )] (3)

[0144] where j ∈ [1, 16], n represents the number of pixel regions, and H j is a 128-dimensional feature vector.

[0145] In the embodiments of the present application, by performing block processing on the first image, a plurality of image blocks are obtained; for each image block, a pixel region corresponding to the pixel value of each pixel point in the image block is determined; the number of pixel regions is determined based on the value range of the pixel values; the number of pixel points in each pixel region is counted; based on the number of pixel points in each pixel region, a histogram feature vector of the image block is determined; the histogram feature vectors of the respective image blocks are weighted and added together to obtain a histogram feature vector corresponding to the first image. By determining the histogram feature vector of each image block and then determining the histogram feature vector corresponding to the first image, more detailed feature information of the first image can be extracted, thereby improving the accuracy of image similarity.

[0146] Optionally, determining the texture feature vector corresponding to the first image includes

[0147] Using the LBP algorithm to determine the texture feature image corresponding to the first image; performing block processing on the texture feature image to obtain a plurality of texture image blocks; respectively determining the histogram feature vectors of the respective texture image blocks; and weighting and adding the histogram feature vectors of the respective texture image blocks to obtain the texture feature vector corresponding to the first image.

[0148] Specifically, the LBP algorithm is a non-parametric algorithm for describing the gray relationship between the feature pixel points of an image and each pixel point, and is also an efficient texture description algorithm.

[0149] For the first image after gray conversion, the LBP algorithm is used to determine the texture feature image corresponding to the first image. Among them, within a region of size 3x3, with the pixel value of the neighborhood center pixel as the threshold, each pixel point in the neighborhood is successively compared with the pixel value of the neighborhood center pixel and encoded, that is, the encoding for greater than or equal to the center pixel value is 1, and the encoding for less than the center pixel value is 0. Then, this group of binary characters is connected in a certain direction and converted into a decimal value, so that the LBP value within this region can be obtained.

[0150]

[0151] Among them, (x c , y c ) represents the center pixel point, i c represents the pixel value of the center pixel point, i p represents the pixel value of the p-th pixel point in the neighborhood, and P represents the total number of pixel points in the neighborhood.

[0152] According to the above LBP algorithm, the texture features of the first image are calculated to obtain the texture feature image corresponding to the first image.

[0153] Perform block processing on the texture feature image to obtain a plurality of texture image blocks; for example,Figure 3 It is a schematic diagram of multiple texture image blocks corresponding to the first image provided by an embodiment of the present application. As Figure 3 shown, the texture feature image is segmented into 8*8, a total of 64 texture image blocks (blocks), forming four circles. The first outer circle (the outermost circle) has 28 texture image blocks (1 to 8, 57 to 64, 9, 17, 25, 33, 41, 49, 16, 24, 32, 40, 48, 56, each representing a texture image block), the second outer circle has 20 texture image blocks (10 to 15, 50 to 55, 18, 26, 34, 42, 50, 23, 31, 39, 47), the third outer circle has 12 texture image blocks (19 to 22, 43 to 46, 27, 35, 30, 38), and the innermost circle has 4 texture image blocks (28, 29, 36, and 37). The size of each image block is 32*32.

[0154] For each texture image block, the value range of the pixel value of each pixel point in the texture image block is [0, 255]. The value range is divided into 128 pixel regions, and each pixel region contains 2 pixel values. For example, pixel values 0 and 1 correspond to pixel region 1, pixel values 2 and 3 correspond to pixel region 2,..., pixel values 254 and 255 correspond to pixel region 128. Then, for each texture image block, determine the pixel region corresponding to the pixel value of each pixel point in the texture image block; count the number of pixel points h(y k ) = m k , k ∈ [0, 255], m k represents the number of pixels with pixel value m in the kth pixel region. Then, the histogram feature vector of the jth texture image block is expressed as:

[0155] LBPH j = [h(y0), h(y1),..., h(y n )] = [h(y0), h(y1),..., h(y 127 )] (5)

[0156] where, j ∈ [1, 64], n represents the number of pixel regions, and LBPH j is a 128-dimensional feature vector.

[0157] Perform weighted addition on the histogram feature vectors of each texture image block. For example, multiply the histogram feature vector of the texture image blocks in the first outer circle (the outermost circle) in Figure 3 by the weight a = 0.25, multiply the histogram feature vector of the texture image blocks in the second outer circle by the weight a = 0.5, multiply the histogram feature vector of the texture image blocks in the third outer circle by the weight a = 0.75, and multiply the histogram feature vector of the texture image blocks in the innermost circle by the weight a = 1. It is expressed by the formula as:

[0158]

[0159] Among them, LBPH j represents the histogram feature vector of the j-th texture image block, and H represents the histogram feature vector after weighted summation of each texture image block.

[0160] Then, normalize LBPH, that is, divide the histogram feature vector of each texture image block by LBPH, and the texture feature vector v2 corresponding to the first image can be obtained. The dimension of v2 is 128 dimensions.

[0161] It should be noted that the determination method of the texture feature vector corresponding to the second image is the same as that of the texture feature vector corresponding to the first image. To avoid repetition, this application will not repeat it here.

[0162] In the embodiment of this application, the LBP algorithm is used to determine the texture feature image corresponding to the first image; the texture feature image is divided into blocks to obtain multiple texture image blocks; the histogram feature vectors of each texture image block are determined respectively; the histogram feature vectors of each texture image block are weighted and summed to obtain the texture feature vector corresponding to the first image. By using the LBP algorithm to extract the texture features of the first image, and then by determining the histogram feature vectors of each texture image block, the texture feature vector corresponding to the first image can be determined, which can extract more texture feature information of the first image, thereby improving the accuracy of image similarity.

[0163] Optionally, the SIFI algorithm is used to determine the local feature vector corresponding to the first image.

[0164] Specifically, for the first image after grayscale conversion, the Gaussian convolution is used to define different scale spaces of the image to construct a Gaussian pyramid; based on the Gaussian difference scale space obtained by subtracting the adjacent upper and lower layers of each group in the Gaussian pyramid, a Gaussian difference pyramid is constructed; then, compare the adjacent upper and lower layers of the same group in the Gaussian difference pyramid to determine the local extreme points, and determine these extreme points as key points. Finally, with the key points as the center, the first image is divided into multiple sub-regions. For example, 16*16 sub-regions, and the size of each sub-region is 16*16. Then, according to the preset window in the scale space of the key points, calculate the scale information in multiple directions. For example, the preset window is 4*4 and the number of directions is 8, and the global feature vector corresponding to each key point can be generated. For example, a global feature vector of 4×4×8 = 128 dimensions, then for c key points, a c*128-dimensional global feature vector v3 can be generated.

[0165] It should be noted that the determination method of the local feature vector corresponding to the second image is the same as that of the local feature vector corresponding to the first image. To avoid repetition, this application will not repeat it here.

[0166] Optionally, fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image to obtain a first fusion feature vector corresponding to the first image includes:

[0167] Transposing the histogram feature vector and the texture feature vector corresponding to the first image to obtain a transposed histogram feature vector and a transposed texture feature vector; concatenating the transposed histogram feature vector and the transposed texture feature vector to obtain a concatenated feature vector; determining the first fusion feature vector corresponding to the first image based on the concatenated feature vector and the local feature vector.

[0168] Specifically, the specific implementation manner of the multi-feature fusion algorithm is as follows: Transposing the histogram feature vector v1 (e.g., 1*128 dimension) and the texture feature vector v2 (e.g., 1*128 dimension) corresponding to the first image, the transposed histogram feature vector (e.g., 128*1) and the transposed texture feature vector (e.g., 128*1) can be obtained.

[0169] Concatenating the transposed histogram feature vector and the transposed texture feature vector by column vectors to obtain a concatenated feature vector v, that is The concatenated feature vector v is a 128*2 feature vector.

[0170] Performing a matrix dot product on the concatenated feature vector v and the local feature vector v3 to obtain a first fusion feature vector Des corresponding to the first image. It is represented by the formula:

[0171] Des = v3v (7)

[0172] where v is a 128*2 dimensional feature vector, v3 is a c*128 dimensional feature vector, c is the number of key points (feature points) extracted by the SIFI algorithm, and Des is a c*2 dimensional feature vector.

[0173] It should be noted that the determination method of the second fusion feature vector corresponding to the second image is the same as that of the first fusion feature vector corresponding to the first image. To avoid repetition, this application will not repeat it here.

[0174] In an embodiment of the present application, by transposing the histogram feature vector and the texture feature vector corresponding to the first image, the transposed histogram feature vector and the transposed texture feature vector are obtained; the transposed histogram feature vector and the transposed texture feature vector are concatenated to obtain a concatenated feature vector; based on the concatenated feature vector and the local feature vector, the first fusion feature vector corresponding to the first image is determined, realizing the fusion of the histogram feature vector, the texture feature vector and the local feature vector, and further being able to accurately determine the target similarity between the first image and the second image, improving the accuracy of the image similarity.

[0175] Optionally, the GIST algorithm is used to determine the global feature vector corresponding to the first image.

[0176] Specifically, for the first image that has not undergone grayscale conversion, the first image is filtered by a Gabor filter, the filtered image is divided into a t×t grid, and the global feature information of each grid image is extracted by using the discrete Fourier transform and the window Fourier transform.

[0177] The steps are as follows: The first image is normalized to obtain a normalized image. An image pyramid is constructed for the normalized image on each of the R, G, and B color channels, and each layer is an image with a different resolution; on a certain layer image of the image pyramid corresponding to each color channel, the image is filtered by using Gabor filters with different directions and corresponding to the resolution of the image. The Gabor filter is expressed by the formula:

[0178]

[0179] where l is the scale of the filter, K is a positive constant, σ is the standard deviation of the Gaussian function, exp[2πj(u0x rθ +v0y rθ )] is a two-dimensional modulation function, u0 and v0 represent the coordinates of the two-dimensional modulation function in the frequency domain, and j represents an imaginary number.

[0180]

[0181] where i = 1, 2…, θ i , θ l , θ i , θ is the total number of directions at the scale, and x and y respectively represent the pixel coordinate positions.

[0182] The filtered image is expressed as:

[0183]

[0184] where I represents the first image.

[0185] Divide the filtered image into a grid of t×t, and use the discrete Fourier transform and the window Fourier transform to extract the global feature information of each grid image. For example, divide the filtered image into a 4×4 grid, and count the histogram of the filtered image corresponding to each block of the grid, that is, represent the histogram of each channel in each direction and each scale with a vector, and use this histogram as the global feature information corresponding to the grid.

[0186] Determine the global feature information corresponding to all grids as the global feature vector v4 corresponding to the first image.

[0187] It should be noted that the method for determining the global feature vector corresponding to the second image is the same as that for determining the global feature vector corresponding to the first image. To avoid repetition, this application will not repeat it.

[0188] Optionally, determining the target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector includes:

[0189] Determine a first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector; determine a second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image; determine the target similarity between the first image and the second image based on the first similarity and the second similarity.

[0190] Specifically, the first similarity represents the similarity between the fusion feature vectors, and the second similarity represents the similarity between the global feature vectors. According to the first fusion feature vector and the second fusion feature vector, the first similarity between the first image and the second image can be determined; then, according to the global feature vector corresponding to the first image and the global feature vector corresponding to the second image, the second similarity between the first image and the second image can be determined; then, according to the first similarity and the second similarity, the target similarity between the first image and the second image can be determined.

[0191] In the embodiments of the present application, based on the first fusion feature vector and the second fusion feature vector, the first similarity between the first image and the second image is determined; then, based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image, the second similarity between the first image and the second image is determined; and then, based on the first similarity and the second similarity, the target similarity between the first image and the second image is determined. Since the first fusion feature vector and the second fusion feature vector express rich feature information of the image, the first similarity can be accurately determined; and then, according to the first similarity and the second similarity, the target similarity can be accurately determined, improving the accuracy and efficiency of image similarity.

[0192] Optionally, determining the first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector includes:

[0193] Based on the first fusion feature vector and the second fusion feature vector, the first similarity between the first image and the second image is determined by using the nearest neighbor algorithm.

[0194] Specifically, according to the first fusion feature vector and the second fusion feature vector, feature point matching is performed by using the nearest neighbor algorithm (for example, the Fast Library for Approximate Nearest Neighbors (FLANN)). That is, the parameters are first initialized, and then the constant parameters for establishing the indexing method of a random k-d tree are established. The K-Nearest Neighbor (KNN) algorithm is used for matching retrieval, the number of successfully matched feature points is calculated, and then the qualified paired feature points are screened. The ratio of the screened qualified paired feature points to the number of successfully matched feature points is determined as the first similarity S1 between the first image and the second image.

[0195] Optionally, determining the second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image includes:

[0196] The inner product of the global feature vector corresponding to the first image and the global feature vector corresponding to the second image is calculated to obtain an inner product result; the square of the inner product result is determined as the second similarity between the first image and the second image.

[0197] Specifically, the global feature vector corresponding to the first image and the global feature vector corresponding to the second image are subjected to inner product calculation to obtain an inner product result Then, the square of the inner product result Determine the second similarity S2 between the first image and the second image.

[0198] Optionally, determining the target similarity between the first image and the second image based on the first similarity and the second similarity includes:

[0199] Performing a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image; the weight corresponding to the first similarity is greater than the weight corresponding to the second similarity.

[0200] Specifically, the weight corresponding to the first similarity S1 is greater than the weight corresponding to the second similarity S2. For example, the weight b1 corresponding to the first similarity S1 is 0.7, and the weight b2 corresponding to the second similarity S2 is 0.3. By performing a weighted calculation on the first similarity and the second similarity, the target similarity S between the first image and the second image can be determined. The calculation formula for the target similarity S is expressed as:

[0201] S = b1 × S1 + b2 × S2 (11)

[0202] Figure 4 is the second schematic flow chart of the image similarity determination method provided by the embodiments of the present application. As Figure 4 shown, first, respectively determine the histogram feature vector, texture feature vector, local feature vector, and global feature vector corresponding to the first image and the second image; for the first image or the second image, respectively use a multi-feature fusion algorithm, that is, transpose the histogram feature vector and the texture feature vector to obtain the transposed histogram feature vector and the transposed texture feature vector; connect the transposed histogram feature vector and the transposed texture feature vector to obtain a connected feature vector; perform a matrix dot product on the connected feature vector and the local feature vector to obtain the first fusion feature vector corresponding to the first image and the second fusion feature vector corresponding to the second image. Based on the first fusion feature vector and the second fusion feature vector, use the Fast Library for Approximate Nearest Neighbors (FLANN) algorithm to determine the first similarity S1 between the first image and the second image. Perform an inner product calculation on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image to obtain an inner product result; square the inner product result to determine the second similarity S2 between the first image and the second image. Perform a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image.

[0203] In the embodiments of the present application, since the local features extracted by SIFT lack global information, color features, and texture features, by combining the advantages of histogram feature extraction, LBP algorithm, and SIFT algorithm, the histogram feature vector, texture feature vector, and local feature vector are fused to generate new feature information, which can reduce the deficiencies of the extracted image features; and then by combining the fused features with the global feature vector, the image similarity can be accurately determined, thereby improving the accuracy of the image similarity.

[0204] Figure 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application, as Figure 5 shown, the electronic device includes: a memory 520, a transceiver 500, and a processor 510, where:

[0205] The memory 520 is used to store a computer program; the transceiver 500 is used to transmit and receive data under the control of the processor; the processor 510 is used to read the computer program in the memory and perform the following operations:

[0206] respectively determine at least one feature vector corresponding to each of the first image and the second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector;

[0207] fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image, respectively, to obtain a first fused feature vector corresponding to the first image and a second fused feature vector corresponding to the second image;

[0208] determine a target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector.

[0209] Specifically, the transceiver 500 is used to receive and transmit data under the control of the processor 510.

[0210] wherein, in Figure 5Among them, the bus architecture may include any number of interconnected buses and bridges, and various circuits represented by one or more processors represented by processor 510 and memory 520 represented by the memory are specifically linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, and thus will not be further described herein. The bus interface provides an interface. The transceiver 500 may be a plurality of components, that is, including a transmitter and a receiver, and provides a unit for communicating with various other devices on a transmission medium, and these transmission media include transmission media such as wireless channels, wired channels, and optical fibers. The processor 510 is responsible for managing the bus architecture and general processing, and the memory 520 may store data used by the processor 510 when performing operations.

[0211] The processor 510 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD), and the processor may also adopt a multi-core architecture.

[0212] Optionally, fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image to obtain a first fusion feature vector corresponding to the first image includes:

[0213] Transposing the histogram feature vector and the texture feature vector corresponding to the first image to obtain a transposed histogram feature vector and a transposed texture feature vector;

[0214] Connecting the transposed histogram feature vector and the transposed texture feature vector to obtain a connected feature vector;

[0215] Based on the connected feature vector and the local feature vector, determining the first fusion feature vector corresponding to the first image.

[0216] Optionally, the determining the target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector includes:

[0217] Determine a first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector;

[0218] Determine a second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image;

[0219] Determine the target similarity between the first image and the second image based on the first similarity and the second similarity.

[0220] Optionally, the determining a first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector includes:

[0221] Determine the first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector by using the nearest neighbor algorithm.

[0222] Optionally, the determining a second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image includes:

[0223] Perform an inner product calculation on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image to obtain an inner product result;

[0224] Determine the square of the inner product result as the second similarity between the first image and the second image.

[0225] Optionally, the determining the target similarity between the first image and the second image based on the first similarity and the second similarity includes:

[0226] Perform a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image; the weight corresponding to the first similarity is greater than the weight corresponding to the second similarity.

[0227] Optionally, determining a histogram feature vector corresponding to a first image includes:

[0228] Perform a block processing on the first image to obtain a plurality of image blocks;

[0229] Determine the histogram feature vector of each of the image blocks respectively;

[0230] Perform a weighted addition on the histogram feature vectors of each of the image blocks to obtain the histogram feature vector corresponding to the first image.

[0231] Optionally, the step of respectively determining the histogram feature vectors of the image blocks includes:

[0232] For each image block, determining a pixel region corresponding to the pixel value of each pixel point in the image block; the number of the pixel regions is determined based on the value range of the pixel value;

[0233] Counting the number of pixel points in each pixel region;

[0234] Based on the number of pixel points in each pixel region, determining the histogram feature vector of the image block.

[0235] Optionally, determining the texture feature vector corresponding to the first image includes

[0236] Using the LBP algorithm to determine the texture feature image corresponding to the first image;

[0237] Performing a blocking process on the texture feature image to obtain a plurality of texture image blocks;

[0238] Respectively determining the histogram feature vectors of the texture image blocks;

[0239] Performing weighted summation on the histogram feature vectors of the texture image blocks to obtain the texture feature vector corresponding to the first image.

[0240] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the same parts and beneficial effects as those in the method embodiment will not be specifically described in this embodiment.

[0241] Figure 6 is a structural schematic diagram of an image similarity determination device provided in an embodiment of the present application. As Figure 6 shown, the image similarity determination device 600 includes: a first determination module 601, a feature fusion module 602, and a second determination module 603; where:

[0242] The first determination module 601 is configured to respectively determine at least one feature vector corresponding to each of the first image and the second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector;

[0243] A feature fusion module 602, configured to fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image respectively, to obtain a first fusion feature vector corresponding to the first image and a second fusion feature vector corresponding to the second image;

[0244] A second determination module 603, configured to determine a target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector.

[0245] The image similarity determination device provided in the embodiment of the present application determines at least one feature vector corresponding to each of the first image and the second image respectively; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector; fuses the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image respectively, to obtain a first fusion feature vector corresponding to the first image and a second fusion feature vector corresponding to the second image; and then determines a target similarity between the first image and the second image according to the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector. Through the fusion of the histogram feature vector, the texture feature vector, and the local feature vector, the first fusion feature vector and the second fusion feature vector with rich image feature information can be determined, and further the target similarity between the first image and the second image can be accurately determined, improving the accuracy of image similarity.

[0246] Optionally, the feature fusion module 602 is specifically configured to:

[0247] Transpose the histogram feature vector and the texture feature vector corresponding to the first image to obtain a transposed histogram feature vector and a transposed texture feature vector;

[0248] Connect the transposed histogram feature vector and the transposed texture feature vector to obtain a connected feature vector;

[0249] Determine the first fusion feature vector corresponding to the first image based on the connected feature vector and the local feature vector.

[0250] Optionally, the second determination module 603 is specifically configured to:

[0251] Determine a first similarity between the first image and the second image based on the first fused feature vector and the second fused feature vector;

[0252] Determine a second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image;

[0253] Determine the target similarity between the first image and the second image based on the first similarity and the second similarity.

[0254] Optionally, the second determination module 603 is further configured to:

[0255] Determine the first similarity between the first image and the second image based on the first fused feature vector and the second fused feature vector by using the nearest neighbor algorithm.

[0256] Optionally, the second determination module 603 is further configured to:

[0257] Perform an inner product calculation on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image to obtain an inner product result;

[0258] Determine the square of the inner product result as the second similarity between the first image and the second image.

[0259] Optionally, the second determination module 603 is further configured to:

[0260] Perform a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image; the weight corresponding to the first similarity is greater than the weight corresponding to the second similarity.

[0261] Optionally, the first determination module 601 is specifically configured to:

[0262] Perform a block processing on the first image to obtain a plurality of image blocks;

[0263] Respectively determine the histogram feature vectors of the respective image blocks;

[0264] Perform a weighted summation on the histogram feature vectors of the respective image blocks to obtain the histogram feature vector corresponding to the first image.

[0265] Optionally, the first determination module 601 is further configured to:

[0266] For each image block, determine a pixel region corresponding to the pixel value of each pixel point in the image block; the number of the pixel regions is determined based on the value range of the pixel value;

[0267] Count the number of pixel points in each of the pixel regions;

[0268] Based on the number of pixel points in each of the pixel regions, determine a histogram feature vector of the image block.

[0269] Optionally, the first determination module 601 is further configured to:

[0270] Use the LBP algorithm to determine a texture feature image corresponding to the first image;

[0271] Perform a blocking process on the texture feature image to obtain a plurality of texture image blocks;

[0272] Respectively determine the histogram feature vectors of the texture image blocks;

[0273] Perform weighted summation on the histogram feature vectors of the texture image blocks to obtain a texture feature vector corresponding to the first image.

[0274] The methods and apparatuses provided in the embodiments of the present application are based on the same application concept. Since the principles for the methods and apparatuses to solve problems are similar, the implementation of the apparatus and the method can be referred to each other, and the repeated parts will not be described again.

[0275] It should be noted that the division of units in the embodiments of the present application is illustrative, merely a logical function division. In actual implementation, there may be other division methods. In addition, in each embodiment of the present application, the functional units may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above integrated units may be implemented in the form of hardware or in the form of software functional units.

[0276] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0277] It should be noted here that the above-mentioned device provided in the embodiments of this application can implement all the method steps implemented in the above method embodiments and can achieve the same technical effects. Therefore, the parts and beneficial effects that are the same as those in the method embodiments will not be specifically described in this embodiment.

[0278] On the other hand, the embodiments of this application also provide a processor-readable storage medium. The processor-readable storage medium stores a computer program, and the computer program is used to cause the processor to execute the image similarity determination method provided in the above various embodiments. For example, it includes: respectively determining at least one feature vector corresponding to the first image and the second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector; fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image respectively to obtain a first fused feature vector corresponding to the first image and a second fused feature vector corresponding to the second image; based on the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector, determining the target similarity between the first image and the second image.

[0279] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memories (such as floppy disks, hard disks, magnetic tapes, magneto-optical discs (MO), etc.), optical memories (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memories (such as ROM, EPROM, EEPROM, non-volatile memories (NANDFLASH), solid-state drives (SSD)).

[0280] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0281] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0282] These processor-executable instructions can also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the processor-readable memory generate a manufactured article including instruction means that implement the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0283] These processor-executable instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0284] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. An image similarity determination method, characterized in that, Including: Respectively determine at least one feature vector corresponding to each of the first image and the second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector; Fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image respectively, to obtain a first fused feature vector corresponding to the first image and a second fused feature vector corresponding to the second image; Based on the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector, determine the target similarity between the first image and the second image.

2. The method for determining image similarity according to claim 1, wherein Fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image to obtain a first fused feature vector corresponding to the first image, includes: Transpose the histogram feature vector and the texture feature vector corresponding to the first image to obtain a transposed histogram feature vector and a transposed texture feature vector; Connect the transposed histogram feature vector and the transposed texture feature vector to obtain a connected feature vector; Based on the connected feature vector and the local feature vector, determine the first fused feature vector corresponding to the first image.

3. The method for determining image similarity according to claim 1, wherein The determining the target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector, includes: Based on the first fused feature vector and the second fused feature vector, determine a first similarity between the first image and the second image; Based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image, determine a second similarity between the first image and the second image; Based on the first similarity and the second similarity, determine the target similarity between the first image and the second image.

4. The method for determining image similarity according to claim 3, wherein The determining the first similarity between the first image and the second image based on the first fused feature vector and the second fused feature vector, includes: Based on the first fused feature vector and the second fused feature vector, use the nearest neighbor algorithm to determine the first similarity between the first image and the second image.

5. The method for determining image similarity according to claim 3, wherein The determining the second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image, includes: Perform an inner product calculation on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image to obtain an inner product result; Square the inner product result and determine it as the second similarity between the first image and the second image.

6. The method for determining image similarity according to claim 3, wherein Determining the target similarity between the first image and the second image based on the first similarity and the second similarity includes: Performing a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image; the weight corresponding to the first similarity is greater than the weight corresponding to the second similarity.

7. The method for determining image similarity according to any one of claims 1 to 6, characterized in that, Determining the histogram feature vector corresponding to the first image includes: Performing a block processing on the first image to obtain a plurality of image blocks; Respectively determining the histogram feature vectors of the respective image blocks; Performing a weighted summation on the histogram feature vectors of the respective image blocks to obtain the histogram feature vector corresponding to the first image.

8. The method for determining image similarity according to claim 7, wherein The respectively determining the histogram feature vectors of the respective image blocks includes: For each image block, determining the pixel region corresponding to the pixel value of each pixel point in the image block; the number of pixel regions is determined based on the value range of the pixel value; Counting the number of pixel points in each pixel region; Based on the number of pixel points in the respective pixel regions, determining the histogram feature vector of the image block.

9. The method for determining image similarity according to any one of claims 1 to 6, characterized in that Determining the texture feature vector corresponding to the first image includes Using the LBP algorithm to determine the texture feature image corresponding to the first image; Performing a block processing on the texture feature image to obtain a plurality of texture image blocks; Respectively determining the histogram feature vectors of the respective texture image blocks; Performing a weighted summation on the histogram feature vectors of the respective texture image blocks to obtain the texture feature vector corresponding to the first image.

10. An electronic device, characterized in that, Including a memory, a transceiver, and a processor: The memory is used for storing a computer program; the transceiver is used for transceiving data under the control of the processor; the processor is used for reading the computer program in the memory and performing the following operations: Respectively determining at least one feature vector corresponding to each of the first image and the second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector; Respectively fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image to obtain a first fused feature vector corresponding to the first image and a second fused feature vector corresponding to the second image; Based on the global feature vector corresponding to the first image, the first fused feature vector, the global feature vector corresponding to the second image, and the second fused feature vector, determining the target similarity between the first image and the second image.

11. The electronic device according to claim 10, wherein Fusing the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image to obtain a first fused feature vector corresponding to the first image, includes: Transposing the histogram feature vector and the texture feature vector corresponding to the first image to obtain a transposed histogram feature vector and a transposed texture feature vector; Connect the transposed histogram feature vector and the transposed texture feature vector to obtain a connected feature vector; Based on the connected feature vector and the local feature vector, determine the first fusion feature vector corresponding to the first image.

12. The electronic device according to claim 10, wherein The determining the target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector includes: Based on the first fusion feature vector and the second fusion feature vector, determine the first similarity between the first image and the second image; Based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image, determine the second similarity between the first image and the second image; Based on the first similarity and the second similarity, determine the target similarity between the first image and the second image.

13. The electronic device according to claim 12, characterized in that, The determining the first similarity between the first image and the second image based on the first fusion feature vector and the second fusion feature vector includes: Based on the first fusion feature vector and the second fusion feature vector, use the nearest neighbor algorithm to determine the first similarity between the first image and the second image.

14. The electronic device according to claim 12, wherein The determining the second similarity between the first image and the second image based on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image includes: Perform an inner product calculation on the global feature vector corresponding to the first image and the global feature vector corresponding to the second image to obtain an inner product result; Determine the square of the inner product result as the second similarity between the first image and the second image.

15. The electronic device according to claim 12, wherein The determining the target similarity between the first image and the second image based on the first similarity and the second similarity includes: Perform a weighted calculation on the first similarity and the second similarity to determine the target similarity between the first image and the second image; the weight corresponding to the first similarity is greater than the weight corresponding to the second similarity.

16. The electronic device according to any one of claims 10 to 15, characterized in that, Determining the histogram feature vector corresponding to the first image includes: Perform a block processing on the first image to obtain a plurality of image blocks; Respectively determine the histogram feature vectors of the image blocks; Perform a weighted summation on the histogram feature vectors of the image blocks to obtain the histogram feature vector corresponding to the first image.

17. The electronic device according to claim 16, wherein The respectively determining the histogram feature vectors of the image blocks includes: For each image block, determine the pixel region corresponding to the pixel value of each pixel point in the image block; the number of pixel regions is determined based on the value range of the pixel value; Count the number of pixel points in each pixel region; Based on the number of pixel points in each pixel region, determine the histogram feature vector of the image block.

18. The electronic device according to any one of claims 10 to 15, characterized in that, Determining the texture feature vector corresponding to the first image includes Use the LBP algorithm to determine the texture feature image corresponding to the first image; The texture feature image is segmented to obtain a plurality of texture image blocks; The histogram feature vectors of the respective texture image blocks are determined separately; The histogram feature vectors of the respective texture image blocks are weighted and added together to obtain the texture feature vector corresponding to the first image.

19. An image similarity determination device, characterized in that, It includes: A first determination module, configured to separately determine at least one feature vector corresponding to each of a first image and a second image; the feature vector includes any one of a histogram feature vector, a texture feature vector, a local feature vector, and a global feature vector; A feature fusion module, configured to fuse the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the first image, and the histogram feature vector, the texture feature vector, and the local feature vector corresponding to the second image respectively, to obtain a first fusion feature vector corresponding to the first image and a second fusion feature vector corresponding to the second image; A second determination module, configured to determine the target similarity between the first image and the second image based on the global feature vector corresponding to the first image, the first fusion feature vector, the global feature vector corresponding to the second image, and the second fusion feature vector.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is used to cause a computer to execute the method according to any one of claims 1 to 9.

Images