Image search method and device

Abstract

The present invention discloses an image search method and device, and relates to the field of computer technology. A specific implementation of the method includes: according to the binary bits representing the color depth, the monochrome matrices of the color channels of the target image are layered to obtain the corresponding matrix slices of the monochrome matrix; according to the matrix slices of the source image in the image database and the matrix slices of the target image, the similarity between the source image and the target image is calculated; according to the similarity, the image identical or similar to the target image is determined in the source image. The method obtains matrix slices by layering the monochrome matrices of multiple color channels of the target image according to the bits, and calculates the similarity between the matrix slices of the target image and the matrix slices of the source image, so as to determine the identical or similar image of the target image from the source image according to the similarity.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to an image search method and device. Background Art

[0002] The popularity of the Internet and the development of search engine technology have brought great convenience to people's lives. People can find what they need on the Internet quickly and accurately. Image search is an emerging search method that uses images as query requirements and aims to search for the same or similar images. Image search can identify the searched images from the massive images on the Internet, improving user experience.

[0003] The image search methods in the prior art mainly include color distribution method, content feature method and perceptual hash algorithm. Among them, the basic principle of color distribution method is: each image can generate a color distribution histogram, if the histograms of two images are very close, then the two images are considered to be similar images; the basic principle of content feature method is: convert the image into a black and white image, if the black and white contours of the two images are similar, then the two images are considered to be similar images; the basic principle of perceptual hash algorithm is: generate a fingerprint string for each image, if the fingerprints of two images are very close, then the two images are considered to be similar images. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an image search method and device, which obtains matrix slices by bit-by-bit layering of the monochrome matrices of multiple color channels of a target image, calculates the similarity between the matrix slices of the target image and the matrix slices of a source image, and determines the same or similar images of the target image from the source image according to the similarity. The embodiment of the present invention proposes a new implementation method to search for the same or similar images of the target image.

[0005] To achieve the above objective, according to one aspect of an embodiment of the present invention, an image search method is provided.

[0006] An image search method according to an embodiment of the present invention includes: hierarchically processing the monochrome matrices of the color channels of a target image according to binary bits representing color depth to obtain matrix slices corresponding to the monochrome matrix; calculating the similarity between the source image and the target image according to the matrix slices of the source image and the matrix slices of the target image in an image database; and determining, in the source image, an image that is identical or similar to the target image according to the similarity.

[0007] Optionally, the calculating the similarity between the source image and the target image includes: merging matrix slices of the same bit layer of the color channel of the target image into a slice vector group of the bit layer; cross-calculating the color channel similarity of the slice vector group of the target image and the slice vector group of the source image at the same bit layer; wherein the slice vector group of the source image is obtained by merging matrix slices of the same bit layer of the source image; and weighting and adding the color channel similarities of at least one bit layer to obtain the similarity between the source image and the target image.

[0008] Optionally, the method further comprises: determining, according to the image search level, the number of bit levels of the matrix slices required for calculating the similarity.

[0009] Optionally, the weighted addition of the color channel similarities of at least one bit layer to obtain the similarity between the source image and the target image includes: initialization: setting the current bit layer to the highest bit of the binary bit representing the color depth, and the similar image set of the current bit layer to the source image in the image database; calculating the bit layer similarity: calculating the bit layer similarity of the current bit layer according to the color channel similarity of the slice vector group of the target image in the current bit layer and the slice vector group of the source image in the similar image set of the current bit layer; determining the similar image set: sorting the bit layer similarities of the current bit layer, and determining the similar image set of the next bit layer according to the sorting result and the similar image set of the current bit layer; decrementing the current bit layer, repeating the calculation of the bit layer similarity and the determination of the similar image set until the current bit layer is smaller than the image search level, and weighted addition of the calculated bit layer similarities to obtain the similarity between the source image and the target image.

[0010] Optionally, the image search level is 8 or 7.

[0011] Optionally, the color channel similarity is one of the following: an angle value between a slice vector group of the target image and a slice vector group of the source image, a Manhattan distance, a Euclidean distance, or a Pearson correlation coefficient.

[0012] Optionally, the target image and the source image have the same pixel size.

[0013] To achieve the above objective, according to another aspect of an embodiment of the present invention, an image search device is provided.

[0014] An image search device according to an embodiment of the present invention includes: a layering module, used to layer the monochrome matrices of the color channels of a target image according to binary bits representing color depth to obtain matrix slices corresponding to the monochrome matrix; a calculation module, used to calculate the similarity between the source image and the target image based on the matrix slices of the source image in an image database and the matrix slices of the target image; and a determination module, used to determine an image in the source image that is identical or similar to the target image based on the similarity.

[0015] Optionally, the calculation module is also used to: merge matrix slices of the same bit layer of the color channel of the target image into a slice vector group of the bit layer; cross-calculate the color channel similarity of the slice vector group of the target image and the slice vector group of the source image at the same bit layer; wherein the slice vector group of the source image is obtained by merging matrix slices of the same bit layer of the source image; and weightedly add the color channel similarities of at least one bit layer to obtain the similarity between the source image and the target image.

[0016] Optionally, the device further comprises: a bit layer number determination module, configured to determine, according to an image search level, the number of bit layers of the matrix slices required for calculating the similarity.

[0017] Optionally, the calculation module is also used to: initialize: set the current bit layer to the highest bit of the binary bit representing the color depth, and the similar image set of the current bit layer to the source image in the image database; calculate the bit layer similarity: calculate the bit layer similarity of the current bit layer according to the color channel similarity of the slice vector group of the target image in the current bit layer and the slice vector group of the source image in the similar image set of the current bit layer; determine the similar image set: sort the bit layer similarities of the current bit layer, and determine the similar image set of the next bit layer according to the sorting result and the similar image set of the current bit layer; and decrement the current bit layer, repeat the calculation of the bit layer similarity and the determination of the similar image set until the current bit layer is smaller than the image search level, and add the calculated bit layer similarities after weighting to obtain the similarity between the source image and the target image.

[0018] Optionally, the image search level is 8 or 7.

[0019] Optionally, the color channel similarity is one of the following: an angle value between a slice vector group of the target image and a slice vector group of the source image, a Manhattan distance, a Euclidean distance, or a Pearson correlation coefficient.

[0020] Optionally, the target image and the source image have the same pixel size.

[0021] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0022] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement an image search method according to an embodiment of the present invention.

[0023] To achieve the above objective, according to another aspect of the embodiments of the present invention, a computer readable medium is provided.

[0024] A computer-readable medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, an image search method according to an embodiment of the present invention is implemented.

[0025] One embodiment of the above invention has the following advantages or beneficial effects: by dividing the monochrome matrices of multiple color channels of the target image into layers according to the bit positions to obtain matrix slices, the similarity between the matrix slices of the target image and the matrix slices of the source image is calculated, so as to determine the same or similar image of the target image from the source image according to the similarity; the number of bit positions of the slice matrix used to calculate the similarity is determined according to the image search level, and then only the similarity of the matrix slices corresponding to the number of bit positions is calculated. This method reduces the amount of calculation and improves the speed of feedback to the user while ensuring the search accuracy.

[0026] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0028] Figure 1 is a schematic diagram of main steps of an image search method according to an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of the main flow of the image search method according to an embodiment of the present invention;

[0030] Figure 3 Schematic diagram of the index structure of an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the principle of calculating the angle value of a slice vector group according to an embodiment of the present invention;

[0032] Figure 5 A schematic diagram of color channels of each bit layer of a color source image according to an embodiment of the present invention;

[0033] Figure 6 A schematic diagram of a main process of calculating the similarity between a source image and a target image according to the bit layer weight according to an embodiment of the present invention;

[0034] Figure 7 is a schematic diagram of main modules of an image search device according to an embodiment of the present invention;

[0035] Figure 8 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0036] Fig. 9 It is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0038] Figure 1 FIG. 1 is a schematic diagram of the main steps of the image search method according to an embodiment of the present invention. Figure 1 As shown, the image search method of the embodiment of the present invention mainly includes the following steps:

[0039] Step S101: According to the binary bits representing the color depth, the monochrome matrices of multiple color channels of the target image are layered and processed respectively to obtain the matrix slices corresponding to the monochrome matrix. The target image may be a search item input by the user, which may be a color image or a black and white image. In one embodiment, the image is represented by RGB three-color channels representing the three colors of red, green, and blue; in another embodiment, the image is represented by CMYK four-color channels representing cyan, magenta, yellow, and black. In this application, the RGB three-color channels are used to describe the image, but the application is not limited thereto. For example, the CYMK four-color channels may be used to describe the image.

[0040] In the embodiment, the monochrome matrix of the RGB three-color channel of the target image is extracted respectively, and the three monochrome matrices are layered respectively according to the binary bit representing the color depth, and the submatrix corresponding to each monochrome matrix is ​​obtained. In this article, the submatrix is ​​also referred to as a matrix slice. For example, the color depth of the target image can be 4 bits, 8 bits, 16 bits, 24 bits, 32 bits, etc., and the number of the matrix slices corresponding to each monochrome matrix is ​​related to the binary bit representing the color depth of the target image. For example, when the target image is a 32-bit RGB image, that is, when each color channel of a pixel is represented by an 8-bit binary number (8-bit red channel, 8-bit green channel, 8-bit blue channel, 8-bit grayscale channel), the number of the submatrix (matrix slices) corresponding to the monochrome matrix of each color channel is 8.

[0041] Step S102: Calculate the similarity between the source image and the target image according to the matrix slices of the source image and the matrix slices of the target image in the image database. Obtain the source image from the image database, and extract the monochrome matrices of the RGB channels of the source image respectively, and perform layered processing on the monochrome matrix of the R channel, the monochrome matrix of the G channel and the monochrome matrix of the B channel of the source image according to the binary bits representing the color depth, and obtain the matrix slices corresponding to each monochrome matrix of the source image. The similarity between the source image and the target image can be calculated based on the matrix slices corresponding to the monochrome matrices of the source image and the target image respectively using algorithms such as the Pearson correlation coefficient, the Euclidean distance, the Manhattan distance or the cosine similarity.

[0042] Step S103: According to the similarity, determine the image that is the same or similar to the target image in the source image, and return the same or similar image as the search result. In one embodiment, the calculated similarity between the source image and the target image is sorted in descending order, and the greater the similarity, the more similar the source image is to the target image. When determining the image that is the same or similar to the target image, the source image whose similarity exceeds a first threshold can be regarded as the image that is the same or similar to the target image; or the first K source images in the sorting result can be regarded as the image that is the same or similar to the target image, where K is an integer. The determined same or similar image is obtained from the image database, and returned to the user as the search result.

[0043] Figure 2 FIG. 1 is a schematic diagram of the main flow of the image search method according to an embodiment of the present invention. Figure 2 As shown, the image search method of the embodiment of the present invention mainly includes the following steps:

[0044] Step S201: Load the original image into the image database, and compress the original image into a source image of preset pixels. The image database stores multiple original images, and the pixels of the original images may be of various types, such as 200×100, 344×234, etc. For the convenience of processing, the pixels of the original image can be compressed into preset pixels, such as 100×100, to obtain the corresponding source image. In the embodiment, the imresize function can be used to implement image compression. The imresize function is used to scale the image. The calling format can be: B=imresize(A, [numrows numcols]), where A is the original image, B is the compressed image, numrows is the compressed height, and numcols is the compressed width.

[0045] Step S202: Extract the monochrome matrices of the RGB channels of the source image respectively. Each RGB image can be represented as three two-dimensional matrices. In one embodiment, because the image search mainly focuses on searching for the similarity of the image content, the color image can be converted into a grayscale image, and the monochrome matrix can be operated. In this step, the monochrome matrix of the R channel, the monochrome matrix of the G channel, and the monochrome matrix of the B channel of the source image are respectively extracted. In the embodiment, the source image is a 32-bit RGB image, and each color channel thereof is represented by a decimal monochrome matrix corresponding to an 8-bit binary number.

[0046] Here is an example of a partial monochrome matrix for the R channel of a 32-bit RGB image:

[0047]

[0048] Each element in the matrix is ​​the grayscale value of the corresponding pixel; the matrix represents an image size of 4×4 pixels, but it should be understood that the image size is not limited thereto.

[0049] In a preferred embodiment, in this step, the source image is first read, and then the source image is subjected to histogram equalization processing, and the monochrome matrix of the RGB channels of the source image after the equalization processing is extracted. Among them, histogram equalization is a basic method of image enhancement, which can improve the contrast of the image, that is, stretching a narrow image grayscale range to a larger range (within the entire grayscale range) according to certain rules. The purpose of histogram equalization is to obtain an image with uniform distribution in the entire grayscale range.

[0050] Step S203: The monochrome matrix of the source image is layered according to the binary bits representing the color depth to obtain the matrix slices corresponding to the monochrome matrix. The monochrome matrices of the R channel, G channel and B channel of each source image are layered according to the binary bits to obtain the matrix slices corresponding to each monochrome matrix. Still taking the above 32-bit RGB source image as an example, the binary representation of the partial monochrome matrix of its R channel is:

[0051]

[0052] The same binary bits of each pixel are taken to form a submatrix, that is, the matrix slices of the 8 bit layers of the monochrome matrix of the R channel are obtained through layered processing, which are represented by R1-R8 respectively:

[0053]

[0054]

[0055] Similarly, the monochrome matrix of the G channel may be split into corresponding matrix slices, which are represented by G1-G8 in the embodiment; and the monochrome matrix of the B channel may be split into corresponding matrix slices, which are represented by B1-B8 in the embodiment.

[0056] Step S204: merge the matrix slices of the same bit layer of the color channel of the source image into the slice vector group of the bit layer. In the matrix slices of each bit layer of each source image, the matrix slices of the RGB channels of the same bit layer are merged respectively to obtain the slice vector group of each source image and each bit layer. The slice vector group corresponding to the matrix slices of each bit layer of a certain source image is (R i , G i , B i ), where i=1, 2,…, 8.

[0057] The following is an example of a slice vector group for bit level 1:

[0058]

[0059] Step S205: Create an index structure to store the slice vector group. In the embodiment, an index structure is created for each slice vector group of each bit layer, that is, 8 index structures are created. Each index structure includes the slice vector group corresponding to the current bit layer of multiple source images, and the index of the slice vector group corresponding to the next bit layer of the current bit layer. The slice vector groups of different bit layers of the same source image may be stored in different rows of the index structure. In the embodiment, the connection between the slice vector groups of each layer of the same source image is maintained by the layer index.

[0060] Figure 3 Schematic diagram of the index structure of an embodiment of the present invention. Figure 3The index structure of the 8th layer slice vector group and the 7th layer slice vector group of 5 source images are shown in FIG. The first column of each index structure is the sequence number, and the sequence numbers 1-5 correspond to a source image in the image database; the second column is the slice vector group of the current layer; and the third column is the next layer index of the current layer.

[0061] Figure 3 In the figure, layer index P5 represents the 8th layer slice vector group corresponding to sequence number 1, and the 7th layer slice vector group corresponding to sequence number 5 is derived from the same source image; layer index P3 represents the 8th layer slice vector group corresponding to sequence number 2, and the 7th layer slice vector group corresponding to sequence number 3 is derived from the same source image; layer index P4 represents the 8th layer slice vector group corresponding to sequence number 3, and the 7th layer slice vector group corresponding to sequence number 4 is derived from the same source image; layer index P2 represents the 8th layer slice vector group corresponding to sequence number 4, and the 7th layer slice vector group corresponding to sequence number 2 is derived from the same source image; layer index P1 represents the 8th layer slice vector group corresponding to sequence number 5, and the 7th layer slice vector group corresponding to sequence number 1 is derived from the same source image.

[0062] Steps S201 to S205 are a process of preprocessing the images in the image database, and subsequent steps S206 to S210 are a process of searching the image database for images that are identical or similar to the target image.

[0063] Step S206: receiving the image to be searched, and compressing the image to be searched into the target image of the preset pixels. This step is used to compress the pixels of the image to be searched into the preset pixels, such as 100×100, and its implementation process is the same as the implementation process of compressing the original image into the source image in step S201.

[0064] Step S207: Extract the monochrome matrices of the RGB channels of the target image respectively. This step is used to extract the monochrome matrix of the R channel, the monochrome matrix of the G channel, and the monochrome matrix of the B channel of the target image respectively. The specific implementation process is the same as the implementation process of extracting the monochrome matrix corresponding to the RGB channels of the source image in step S202. In a preferred embodiment, the target image is first subjected to histogram equalization processing, and then the monochrome matrix of the RGB channels of the target image after the histogram equalization processing is extracted.

[0065] Step S208: The monochrome matrix of the target image is layered according to the binary bits representing the color depth to obtain matrix slices corresponding to the monochrome matrix. The monochrome matrices of the R channel, G channel and B channel of the target image are layered according to the binary bits to obtain matrix slices corresponding to each monochrome matrix. The specific implementation process is the same as the implementation process of layering the monochrome matrix of the source image according to the bit in step S203.

[0066] Step S209: Calculate the similarity between the source image and the target image based on the matrix slice of the source image and the matrix slice of the target image. Taking cosine similarity as an example, the specific implementation process of this step can be:

[0067] (1) The matrix slices of the same bit layer of the color channel of the target image are merged into the slice vector group of the bit layer, and the matrix slices of the same bit layer are respectively merged into the corresponding slice vector group. In the matrix slices of each layer of the target image, the matrix slices of the RGB channels of the same bit layer are respectively merged to obtain the target image and the slice vector group of each bit layer. Among them, the slice vector group corresponding to the matrix slices of each layer of the target image is (R' i , G' i , B' i ), where i=1, 2,…, 8.

[0068] (2) Cross-calculate the angle values ​​between the slice vector group of the target image and the slice vector group of the source image at the same bit layer. Taking the slice vector group of the 8th bit layer as an example, the angle values ​​between the slice vector group of the target image and the slice vector group of the source image are: V(R'8, R8), V(R'8, G8), V(R'8, B8), V(G'8, R8), V(G'8, G8), V(G'8, B8), V(B'8, R8), V(B'8, G8) and V(B'8, B8). Each of the above angle values ​​is a multi-dimensional space angle. In the embodiment, the subspace function can be used to calculate the multi-dimensional space angle.

[0069] In the embodiment, when calculating the angle value between the slice vector group of the target image at the 7-1 bit layer and the slice vector group of the source image, it is necessary to find the next bit layer index corresponding to the slice vector group of the current bit layer of the source image according to the index structure created in step S205, obtain the slice vector group of the next bit layer according to the next bit layer index, and then calculate the angle value between it and the slice vector group of the same bit layer of the target image. The following is an example of calculating the angle value between the slice vector group of the 7th bit layer of the target image and the slice vector group of the 7th bit layer of the source image: find the 7th bit layer index corresponding to the slice vector group of the 8th bit layer of the source image according to the index structure created in step S205, obtain the slice vector group of the 7th bit layer according to the 7th bit layer index, and then calculate the angle value between it and the slice vector group of the 7th bit layer of the target image.

[0070] Figure 4 FIG. 1 is a schematic diagram showing the principle of calculating the angle value of a slice vector group according to an embodiment of the present invention. Figure 4 As shown, the angle values ​​between the matrix slices R'8, G'8, B'8 of the target image and the matrix slices R8, G8, B8 of the source image are cross-calculated in pairs.

[0071] (3) The angle values ​​are weighted and added to obtain the similarity between the source image and the target image at the corresponding bit layer. The similarity between the source image and the target image can be the weighted sum of the similarities of the slice vector groups of each bit layer. Assume that the weights set for each angle value of the slice vector group of the 8th bit layer are x k , where k = 1, 2, ..., 9, then the calculation formula for the similarity Corr8 of the 8th layer slice vector group is:

[0072] Corr8=x1×V(R'8,R8)+x2×V(R'8,G8)+x3×V(R'8,B8)+x4×V(G'8,R8)+x5× V(G'8,G8)+x6×V(G'8,B8)+x7×V(B'8,R8)+x8×V(B'8,G8)+x9×V(B'8,B8)

[0073] Formula 2

[0074] The similarity of the slice vector group of each bit layer is calculated in the above manner, and the similarity corresponding to each bit layer is weighted and summed according to the preset bit layer weight, and finally the similarity between the source image and the target image is obtained.

[0075] Step S210: According to the similarity, an image identical or similar to the target image is determined in the source image, and the identical or similar image is returned as a search result. The similarity between the source image and the target image is sorted in descending order, and the greater the similarity, the more similar the source image and the target image are. In an embodiment, all source images with a similarity exceeding a first threshold are regarded as images identical or similar to the target image and returned to the user.

[0076] Figure 5 FIG. 1 is a schematic diagram of the color channels of each bit layer of a color source image according to an embodiment of the present invention. Figure 5 It can be seen that the matrix slices of the low-level layers (1st to 4th layers, i.e. R1-R4, G1-G4, B1-B4) are mainly noise information, and the matrix slices of the high-level layers (5th to 8th layers, i.e. R5-R8, G5-G8, B5-B8) contain more texture information. In addition, according to the above example, the matrix slices of the 1st layer corresponding to the three RGB channels of the source image are:

[0077]

[0078] By comparing the elements at corresponding positions in the above three matrix slices, it can be seen that the information contained in the matrix slices of the corresponding layers of the three channels is highly correlated. Since the information contained in the slice matrices of the corresponding layers of the three RGB channels is highly correlated, and the matrix slices of the low-level layers mainly involve noise information, which contributes less to image similarity; the matrix slices of the high-level layers contain more texture information and contribute more to image similarity - therefore, when searching for images that are identical or similar to the target image, the similarity of the matrix slices of the high-level layers can be calculated first, and then, for example, according to the requirements of the search accuracy, the source images with high similarity can be found layer by layer as the same or similar images of the target image.

[0079] Further, in a preferred embodiment, the specific implementation of step S209 may also be:

[0080] (1) According to the image search level, determine the number of bit layers of the matrix slices to be merged. The image search level indicates the accuracy requirement of the image search. If the image search level is 8, only the matrix slices of the 8th bit layer are needed to calculate the similarity, and the corresponding number of bit layers is 8; if the image search level is 7, the matrix slices of the 8th bit layer and the 7th bit layer are needed to calculate the similarity, and the corresponding number of bit layers is 8 and 7; and so on.

[0081] (2) Merge the matrix slices corresponding to the number of bit layers into corresponding slice vector groups. If the image search level is 8, merge the matrix slices of the RGB channels of the 8th bit layer to obtain the slice vector group (R'8, G'8, B'8) of the 8th bit layer of the target image. If the image search level is 7, merge the matrix slices of the RGB channels of the 8th bit layer to obtain the slice vector group (R'8, G'8, B'8) of the 8th bit layer of the target image; and merge the matrix slices of the RGB channels of the 7th bit layer to obtain the slice vector group (R'7, G'7, B'7) of the 7th bit layer of the target image.

[0082] (3) Cross-calculate the angle values ​​between the slice vector group of the target image and the slice vector group of the source image at the same bit layer. Still taking the slice vector group of the 8th bit layer as an example, the angle values ​​between the slice vector group of the target image and the slice vector group of the source image calculated according to Formula 1 are: V(R'8, R8), V(R'8, G8), V(R'8, B8), V(G'8, R8), V(G'8, G8), V(G'8, B8), V(B'8, R8), V(B'8, G8) and V(B'8, B8).

[0083] (4) According to formula 2, the angle values ​​are weighted and added to obtain the similarity between the source image and the target image. If the image search level is 8, the angle values ​​of the slice vector group corresponding to the 8th bit layer are weighted and added, and the addition result is recorded as Corr8, which is used as the similarity between the source image and the target image. If the image search level is less than 8, the bit-level similarity between the source image and the target image corresponding to the bit layer of the image search level is calculated, and the bit-level similarities are weighted and added to obtain the similarity between the source image and the target image. The specific implementation process of this step is described as follows.

[0084] Figure 6 FIG. 1 is a schematic diagram of the main process of calculating the similarity between a source image and a target image according to the bit-level weights of an embodiment of the present invention. Figure 6 As shown, the similarity between the source image and the target image is calculated according to the bit layer weight, which mainly includes the following steps:

[0085] Step S601: Calculate the bit-level similarity between the target image of the current bit-level and the set of similar images according to Formula 2. The initial value of the current bit-level is the highest bit of the binary bit representing the color depth; the initial value of the set of similar images of the current bit-level is the source image in the image database. In the embodiment, the initial value of the current bit-level is 8, that is, the bit-level similarity Corr8 between the target image of the 8th bit-level and the set of similar images is calculated according to Formula 2.

[0086] Step S602: Sort the bit-level similarities of the current bit-level, and determine the similar image set of the next bit-level according to the sorting result and the similar image set of the current bit-level. In the embodiment, the plurality of source images are sorted according to the similarity Corr8 of the 8th bit-level, that is, the source images with high similarity to the target image are preferentially selected. In addition, in this step, the source images whose similarity Corr8 is greater than the second threshold value can be used as the similar image set of the 7th bit-level; or the source images corresponding to the first M similarities in the sorting result can be used as the similar image set of the 7th bit-level, where M is an integer, or some images can be selected from the source images whose similarity Corr8 is greater than the second threshold value as the similar image set of the 7th bit-level.

[0087] Step S603: Decrement the current bit layer, and determine whether the number of bit layers after decrement is greater than or equal to the image search level. If so, execute step S601; if not, execute step S604. Assuming that the image search level is 7, after decrementing the current bit layer, the number of bit layers obtained is 7, and the two are equal, execute step S601; assuming that the image search level is 8, after decrementing the current bit layer, the number of bit layers obtained is 7, 7<8, execute step S604.

[0088] Step S604: The calculated bit-level similarities are weighted and added to obtain the similarity between the source image and the target image. Assuming that the image search level is 7, similarity Corr8 for the 8th bit-level and similarity Corr7 for the 7th bit-level are weighted and added according to Formula 3, and the source images in the similar image set corresponding to the 7th bit-level are re-sorted according to the weighted total similarity, and the source images whose total similarity Corr is greater than the similarity threshold are selected and returned as search results.

[0089] Corr=α×Corr8+β×Corr7 Formula 3

[0090] In one embodiment, the similarities of each layer (the 8th and 7th layers) contribute equally to the total similarity Corr, and α=1, β=1; in another embodiment, the similarity Corr8 of the 8th layer contributes more to the total similarity Corr than the similarity Corr7 of the 7th layer, and α>β is set.

[0091] It can be seen from the image search method of the embodiment of the present invention that the monochrome matrices of multiple color channels of the target image are layered bit by bit to obtain matrix slices, and the similarity between the matrix slices of the target image and the matrix slices of the source image is calculated, so as to determine the same or similar image of the target image from the source image based on the similarity; the number of bit layers of the slice matrix used to calculate the similarity is determined according to the image search level, and then only the similarity of the matrix slices corresponding to the number of bit layers is calculated. This method reduces the amount of calculation and improves the speed of feedback to the user while ensuring the search accuracy.

[0092] Figure 7 Schematic diagram of the main modules of the image search device according to an embodiment of the present invention. Figure 7 As shown, the image search device 700 of the embodiment of the present invention mainly includes:

[0093] The layering module 701 is used to layer the monochrome matrices of multiple color channels of the target image according to the binary bits representing the color depth, and obtain the matrix slices corresponding to the monochrome matrices. In the embodiment, the target image can be a search item input by the user, which can be a color image or a black and white image. In one embodiment, the image is represented by the RGB three-color channels representing the three colors of red, green and blue; in another embodiment, the image is represented by the CMYK four-color channels representing cyan, magenta, yellow and black. In the embodiment, the monochrome matrices of the RGB three-color channels of the target image are extracted respectively, and the three monochrome matrices are layered according to the binary bits representing the color depth to obtain the sub-matrices corresponding to each monochrome matrix. In this article, the sub-matrices are also referred to as matrix slices.

[0094] The calculation module 702 is used to calculate the similarity between the source image and the target image according to the matrix slices of the source image and the matrix slices of the target image in the image database. The source image is obtained from the image database, and the monochrome matrices of the RGB channels of the source image are extracted respectively. The monochrome matrix of the R channel, the monochrome matrix of the G channel and the monochrome matrix of the B channel of the source image are layered according to the binary bits representing the color depth to obtain the matrix slices corresponding to each monochrome matrix of the source image. The algorithms such as Pearson correlation coefficient, Euclidean distance, Manhattan distance or cosine similarity can be used to calculate the similarity between the source image and the target image according to the matrix slices corresponding to the monochrome matrices of the source image and the target image.

[0095] The determination module 703 is used to determine, in the source image, an image that is identical or similar to the target image based on the similarity, so as to return the identical or similar image as a search result. In one embodiment, the calculated similarities between the source image and the target image are sorted in descending order, and the greater the similarity, the more similar the source image and the target image are. When determining an image that is identical or similar to the target image, the source image whose similarity exceeds a first threshold value may be regarded as an image that is identical or similar to the target image; or the first K source images in the sorting result may be regarded as images that are identical or similar to the target image, where K is an integer. The determined identical or similar images are obtained from the image database and returned to the user as a search result.

[0096] In addition, the image search device 700 of the embodiment of the present invention may further include: a bit layer number determination module ( Figure 7 (not shown), this module is used to determine the number of bit layers of the matrix slices required to calculate the similarity according to the image search level.

[0097] From the above description, it can be seen that the monochrome matrices of multiple color channels of the target image are layered bit by bit to obtain matrix slices, and the similarity between the matrix slices of the target image and the matrix slices of the source image is calculated, so as to determine the same or similar image of the target image from the source image based on the similarity; the number of bit layers of the slice matrix used to calculate the similarity is determined according to the image search level, and then only the similarity of the matrix slices corresponding to the number of bit layers is calculated. This method reduces the amount of calculation and improves the speed of feedback to the user while ensuring the search accuracy.

[0098] Figure 8 An exemplary system architecture 800 to which the image search method or image search device according to the embodiment of the present invention can be applied is shown.

[0099] like Figure 8As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0100] Users can use terminal devices 801, 802, and 803 to interact with server 805 through network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0101] The terminal devices 801 , 802 , and 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0102] The server 805 may be a server that provides various services, such as a background management server that provides support for image search requests sent by users using the terminal devices 801, 802, and 803. The background management server may analyze and process the received image search requests, and feed back the processing results (e.g., images that are the same as or similar to the target image) to the terminal device.

[0103] It should be noted that the image search method provided in the embodiment of the present application is generally executed by the server 805 , and accordingly, the image search device is generally disposed in the server 805 .

[0104] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0105] According to an embodiment of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0106] The electronic device of the present invention comprises: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement an image search method of an embodiment of the present invention.

[0107] The computer-readable medium of the present invention stores a computer program, and when the program is executed by a processor, an image search method according to an embodiment of the present invention is implemented.

[0108] Reference below Fig. 9, which shows a schematic diagram of the structure of a computer system 900 suitable for implementing an electronic device of an embodiment of the present invention. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0109] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the computer system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0110] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0111] In particular, according to the embodiments disclosed in the present invention, the process described in the main step diagram above can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the main step diagram. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0112] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0113] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0114] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, they may be described as: a processor including a layering module, a calculation module, and a determination module. The names of these modules do not constitute a limitation on the modules themselves in some cases. For example, the layering module may also be described as "a module for layering the monochrome matrices corresponding to the RGB channels of the target image bit by bit to obtain matrix slices corresponding to the monochrome matrix".

[0115] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: bit-by-bit layering of the monochrome matrix corresponding to the RGB channels of the target image to obtain matrix slices corresponding to the monochrome matrix; calculating the similarity between the source image and the target image based on the matrix slices of at least one source image and the matrix slices of the target image; determining the same or similar image of the target image from the source image based on the similarity, and returning the same or similar image of the target image.

[0116] From the above description, it can be seen that the monochrome matrices of multiple color channels of the target image are layered bit by bit to obtain matrix slices, and the similarity between the matrix slices of the target image and the matrix slices of the source image is calculated, so as to determine the same or similar image of the target image from the source image based on the similarity; the number of bit layers of the slice matrix used to calculate the similarity is determined according to the image search level, and then only the similarity of the matrix slices corresponding to the number of bit layers is calculated. This method reduces the amount of calculation and improves the speed of feedback to the user while ensuring the search accuracy.

[0117] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0118] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image search method, characterized in that: include: According to the binary bits representing the color depth, the monochrome matrices of the color channels of the target image are layered respectively to obtain matrix slices corresponding to the monochrome matrices; wherein, after the target image is subjected to histogram equalization processing, the monochrome matrix of each color channel of the target image after the histogram equalization processing is extracted; the layered processing process for a single color channel is as follows: the monochrome matrix of a single channel is represented by binary, and then the same binary bits of each pixel are taken to form a matrix slice, and a plurality of bit-layer matrix slices are obtained; Calculating the similarity between the source image and the target image according to the matrix slices of the source image and the matrix slices of the target image in the image database, including: merging the matrix slices of the same bit layer of the color channel of the target image into the slice vector group of the bit layer; cross-calculating the color channel similarity of the slice vector group of the target image and the slice vector group of the source image at the same bit layer; wherein the slice vector group of the source image is obtained by merging the matrix slices of the same bit layer of the source image; and weighting and adding the color channel similarity of at least one bit layer to obtain the similarity between the source image and the target image; According to the similarity, an image identical to or similar to the target image is determined in the source image.

2. The method according to claim 1, characterized in that The method further comprises: determining the number of bit levels of the matrix slices required for calculating the similarity according to the image search level.

3. The method according to claim 2, characterized in that The weighted addition of the color channel similarities of at least one bit layer to obtain the similarity between the source image and the target image includes: Initialization: setting the current bit layer to the highest bit of the binary bit representing the color depth, and the similar image set of the current bit layer to the source image in the image database; Calculating bit-level similarity: calculating the bit-level similarity of the current bit-level according to the color channel similarity of the slice vector group of the target image in the current bit-level and the slice vector group of the source image in the similar image set of the current bit-level; Determining a similar image set: sorting the bit-level similarities of the current bit-level, and determining a similar image set of the next bit-level according to the sorting result and the similar image set of the current bit-level; The current bit layer is decreased, and the calculation of the bit layer similarity and the determination of the similar image set are repeated until the current bit layer is smaller than the image search level, and the calculated bit layer similarities are weighted and added to obtain the similarity between the source image and the target image.

4. The method according to claim 3, characterized in that The image search level is 8 or 7.

5. The method according to any one of claims 1 to 4, characterized in that: The color channel similarity is one of the following: an angle value between the slice vector group of the target image and the slice vector group of the source image, a Manhattan distance, a Euclidean distance, or a Pearson correlation coefficient.

6. The method according to any one of claims 1 to 4, characterized in that: The target image and the source image have the same pixel size.

7. An image search device, characterized in that: include: A layering module is used to perform layering processing on the monochrome matrices of the color channels of the target image according to the binary bits representing the color depth, and obtain the matrix slices corresponding to the monochrome matrix; wherein, after the target image is subjected to histogram equalization processing, the monochrome matrix of each color channel of the target image after the histogram equalization processing is extracted; the layering processing process for a single color channel is: the monochrome matrix of a single channel is represented by binary, and then the same binary bits of each pixel are taken to form a matrix slice, and a plurality of bit-layer matrix slices are obtained; A calculation module, for calculating the similarity between the source image and the target image according to the matrix slices of the source image and the matrix slices of the target image in the image database, comprising: merging the matrix slices of the same bit layer of the color channel of the target image into the slice vector group of the bit layer; cross-calculating the color channel similarity of the slice vector group of the target image and the slice vector group of the source image at the same bit layer; wherein the slice vector group of the source image is obtained by merging the matrix slices of the same bit layer of the source image; and weighting and adding the color channel similarity of at least one bit layer to obtain the similarity between the source image and the target image; A determination module is used to determine an image that is identical or similar to the target image in the source image according to the similarity.

8. The device according to claim 7, characterized in that The device further comprises: a bit layer number determination module, which is used to determine the bit layer number of the matrix slices required for calculating the similarity according to the image search level.

9. The device according to claim 8, characterized in that The computing module is further used for: Initialization: setting the current bit layer to the highest bit of the binary bit representing the color depth, and the similar image set of the current bit layer to the source image in the image database; Calculating bit-level similarity: calculating the bit-level similarity of the current bit-level according to the color channel similarity of the slice vector group of the target image in the current bit-level and the slice vector group of the source image in the similar image set of the current bit-level; Determining a similar image set: sorting the bit-level similarities of the current bit-level, and determining a similar image set of the next bit-level according to the sorting result and the similar image set of the current bit-level; as well as The current bit layer is decreased, and the calculation of the bit layer similarity and the determination of the similar image set are repeated until the current bit layer is smaller than the image search level, and the calculated bit layer similarities are weighted and added to obtain the similarity between the source image and the target image.

10. The device according to claim 9, characterized in that The image search level is 8 or 7.

11. The device according to any one of claims 7 to 10, characterized in that The color channel similarity is one of the following: an angle value between the slice vector group of the target image and the slice vector group of the source image, a Manhattan distance, a Euclidean distance, or a Pearson correlation coefficient.

12. The device according to any one of claims 7 to 10, characterized in that The target image and the source image have the same pixel size.

13. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

14. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images