Image matching methods, apparatus, electronic devices and readable media

Abstract

This disclosure provides an image matching method, apparatus, electronic device, and readable medium, relating to the field of image retrieval technology. The image matching method includes: selecting a first group of similar images from a source image set based on the Hamming distance between each image in the source image set and a target image; assigning values ​​to the images in the first group of similar images according to the magnitude of the Hamming distance between the target image and each image in the first group of similar images, obtaining a row matrix of the first group of similar images; assigning values ​​to the images in the first group of similar images according to the degree of overlap between the target image and each image in the first group of similar images, obtaining a column matrix of the first group of similar images; multiplying the row matrix and column matrix of the first group of similar images to obtain a target matrix; and selecting a second group of similar images from the first group of similar images based on the target matrix. This disclosure improves the accuracy of image matching by incorporating image content comparison on top of image fingerprint comparison.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image retrieval, and particularly relates to an image matching method and device, an electronic device and a readable medium. BACKGROUND

[0002] Image retrieval is to extract features from a target picture, quickly quantize and mark the image by using a perceptual hashing algorithm (pHash), and then calculate the feature distance by comparing the hash values of the target picture and a source picture group to reflect the similarity of the images, so as to quickly find a picture similar to the target picture and achieve image retrieval.

[0003] The existing image retrieval often has retrieval results in which the picture elements are roughly similar but the core highlighted content is inconsistent. In addition, since the perceptual hashing algorithm does not have mirror invariance, the similarity between the original picture and the picture obtained by only performing horizontal mirror processing or adding a rotation angle is very low according to the existing method.

[0004] Therefore, how to improve the accuracy of image matching has become a technical problem to be solved.

[0005] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The present disclosure provides an image matching method, device, electronic device and readable medium, which at least partially solves the problem of low image matching accuracy in the related art.

[0007] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0008] According to one aspect of the present disclosure, an image matching method is provided, comprising: screening a first group of similar images from a source picture set according to the Hamming distance between each image in the source picture set and a target image; assigning values to the images in the first group of similar images according to the size of the Hamming distance between the target image and each image in the first group of similar images, to obtain a row matrix of the first group of similar images; assigning values to the images in the first group of similar images according to the level of element overlap between the target image and each image in the first group of similar images, to obtain a column matrix of the first group of similar images; multiplying the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and screening a second group of similar images from the first group of similar images according to the target matrix.

[0009] In one embodiment of the present disclosure, the filtering a first group of similar images from the source image set according to the Hamming distance between each image in the source image set and the target image comprises: calculating the hash value of each image in the source image set and the target image by using a perceptual hashing algorithm; calculating the Hamming distance between each image in the source image set and the target image according to the hash value of each image in the source image set and the target image; filtering a first group of similar images from the source image set according to the Hamming distance between each image in the source image set and the target image.

[0010] In one embodiment of the present disclosure, the calculating the hash value of the target image by using a perceptual hashing algorithm comprises: performing grayscale processing on the scaled target image to obtain a grayscale processed target image; performing discrete cosine transform processing on the grayscale processed target image to obtain frequency domain data of the target image; calculating the average value of pixels of the target image, comparing the grayscale of each pixel of the target image with the average value of pixels, and recording 1 when the pixel is greater than or equal to the average value of pixels, otherwise, recording 0; obtaining the hash value of the target image according to the comparison result.

[0011] In one embodiment of the present disclosure, the calculating the Hamming distance between each image in the source image set and the target image according to the hash value of each image in the source image set and the target image comprises: performing XOR operation on the hash value of each image in the source image set and the target image bit by bit; obtaining the Hamming distance between each image in the source image set and the target image according to the XOR operation result.

[0012] In one embodiment of the present disclosure, the assigning values to the images in the first group of similar images according to the high and low of the element overlap degree between the target image and each image in the first group of similar images to obtain the column matrix of the first group of similar images comprises: extracting element data pairs of the target image; performing element comparison on the target image and each image in the first group of similar images to obtain the element overlap degree between the target image and each image in the first group of similar images; assigning values to the images in the first group of similar images according to the high and low of the element overlap degree between the target image and the first group of similar images to obtain the column matrix of the first group of similar images.

[0013] In one embodiment of the present disclosure, the extracting the element data pair of the target image comprises: extracting image features of the target image by using a convolutional neural network (CNN); inputting the image features of the target image into a long short-term memory network (LSMT) to output a natural language sentence for describing the target image; extracting a key-value data pair from the natural language sentence of the target image by using an attention model, wherein the key in the key-value data pair is an image element label, and the value in the key-value data pair is an image after attention focusing processing on the corresponding key; and removing stop words from the key-value data pair to obtain the element data pair of the target image.

[0014] In one embodiment of the present disclosure, the filtering a second group of similar images from the first group of similar images according to the target matrix comprises: arranging elements with non-zero values in the target matrix in ascending order, wherein the smaller the element value is, the higher the matching degree of the image corresponding to the element with the target image is, and filtering a preset number of images from the first group of similar images according to the size of the element value to obtain the second group of similar images.

[0015] According to another aspect of the present disclosure, an image matching device is provided, comprising: a first filtering module configured to filter a first group of similar images from a source image set according to a Hamming distance between each image in the source image set and a target image; a first assigning module configured to assign values to images in the first group of similar images according to the size of the Hamming distance between the target image and each image in the first group of similar images, to obtain a row matrix of the first group of similar images; a second assigning module configured to assign values to images in the first group of similar images according to the degree of element overlap between the target image and each image in the first group of similar images, to obtain a column matrix of the first group of similar images; and a second filtering module configured to multiply the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and filter a second group of similar images from the first group of similar images according to the target matrix.

[0016] According to still another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the image matching method described above by executing the executable instructions.

[0017] According to yet another aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the image matching method described above.

[0018] The embodiment of the present disclosure provides an image matching method, device, electronic equipment and readable medium, wherein the image matching method comprises the following steps: screening a first group of similar images from a source image set according to a Hamming distance between each image in the source image set and a target image; assigning values to the images in the first group of similar images according to the size of the Hamming distance between the target image and each image in the first group of similar images, so as to obtain a row matrix of the first group of similar images; assigning values to the images in the first group of similar images according to the high and low of the element overlap degree between the target image and each image in the first group of similar images, so as to obtain a column matrix of the first group of similar images; multiplying the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and screening a second group of similar images from the first group of similar images according to the target matrix. The present application improves the accuracy of image matching by comparing the image content on the basis of image fingerprint comparison.

[0019] Further, by removing stop words, the information redundancy of the image description is reduced, and the comparison efficiency is improved.

[0020] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0021] The drawings herein are incorporated into the specification and form a part of the specification, show embodiments consistent with the present disclosure, and together with the specification, serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0022] Figure 1 A flow chart of an image matching method in an embodiment of the present disclosure is shown;

[0023] Figure 2 A flow chart of another image matching method in an embodiment of the present disclosure is shown;

[0024] Figure 3 A flow chart of another image matching method in an embodiment of the present disclosure is shown;

[0025] Figure 4 A flow chart of another image matching method in an embodiment of the present disclosure is shown;

[0026] Figure 5 A flow chart of another image matching method in an embodiment of the present disclosure is shown;

[0027] Figure 6 A flow chart of another image matching method in an embodiment of the present disclosure is shown;

[0028] Figure 7A schematic diagram showing an image matching method in an embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram showing an image matching device in an embodiment of the present disclosure; and

[0030] Figure 9 A structural block diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects to those skilled in the art. Features, structures, or characteristics described in connection with one implementation can be combined in any suitable manner with features, structures or characteristics of other implementations.

[0032] Furthermore, the drawings are not necessarily drawn to scale. Like reference numerals in different drawings denote like or similar parts, and the description thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities, which do not necessarily have to correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0033] The existing image retrieval often has a retrieval result that the picture elements are roughly similar but the core highlighted content is inconsistent. In addition, since the pHash does not have mirror invariance, the similarity between the picture obtained by only horizontally mirroring the original picture or adding a rotation angle and the original picture is very low by the existing method.

[0034] Based on this, the present disclosure provides an image matching method, which, on the basis of the existing image fingerprint comparison, integrates the comparison of image content by an attention model to improve the accuracy of image recognition and matching.

[0035] The present example implementation will be described in detail below with reference to the accompanying drawings and examples.

[0036] First, an image matching method is provided in an embodiment of the present disclosure, which can be executed by any electronic device with computing processing capability.

[0037] Figure 1 A flowchart showing an image matching method in an embodiment of the present disclosure is shown in FIG. 1, which includes the following steps: Figure 1

[0038] ​Step S102, filtering a first group of similar images from the source image set according to the Hamming distance between each image in the source image set and the target image;

[0039] Step S104, assigning values to the images in the first group of similar images according to the size of the Hamming distance between the target image and each image in the first group of similar images, to obtain a row matrix of the first group of similar images;

[0040] Step S106, assigning values to the images in the first group of similar images according to the level of element overlap between the target image and each image in the first group of similar images, to obtain a column matrix of the first group of similar images;

[0041] Step S108, multiplying the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and filtering a second group of similar images from the first group of similar images according to the target matrix.

[0042] The image matching method provided by the embodiments of the present disclosure includes: filtering a first group of similar images from a source image set according to the Hamming distance between each image in the source image set and a target image; assigning values to the images in the first group of similar images according to the size of the Hamming distance between the target image and each image in the first group of similar images, to obtain a row matrix of the first group of similar images; assigning values to the images in the first group of similar images according to the level of element overlap between the target image and each image in the first group of similar images, to obtain a column matrix of the first group of similar images; multiplying the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and filtering a second group of similar images from the first group of similar images according to the target matrix. The present application improves the accuracy of image matching by comparing the image content on the basis of image fingerprint comparison.

[0043] In the following, the image matching method in the embodiments of the present disclosure will be further described.

[0044] In step S102, a first group of similar images is filtered from the source image set according to the Hamming distance between each image in the source image set and the target image;

[0045] In this step, the source image set is a gallery that stores a large number of images, and similar images to the target image can be searched in the source image set; the Hamming distance represents the number of different characters of two equal-length strings at corresponding positions, and the smaller the Hamming distance, the higher the similarity between the two images, and vice versa, the larger the Hamming distance, the lower the similarity between the two images; here, a Hamming distance threshold can be set in advance, when the Hamming distance between a certain image in the source image set and the target image is less than the Hamming distance threshold, the image is filtered from the source image set, and all images in the source image set that meet the condition are filtered out in turn, to obtain the first group of similar images.

[0046] In step S104, the images in the first set of similar images are valued according to the size of the Hamming distance between the target image and each image in the first set of similar images, to obtain a row matrix of the first set of similar images;

[0047] In this step, the images in the first set of similar images are valued according to the Hamming distance between the target image and each image in the first set of similar images from small to large, to obtain a row matrix X, such as row matrix X = [1, 2, 3, …, N], where N is the number of images contained in the first set of similar images.

[0048] In step S106, the images in the first set of similar images are valued according to the high and low of the element overlap degree between the target image and each image in the first set of similar images, to obtain a column matrix of the first set of similar images;

[0049] In this step, the images in the first set of similar images are valued according to the high and low of the element overlap degree between the target image and each image in the first set of similar images, and the valuation result is 1, 2, 3, …, L, where N-L images in the first set of similar images are considered to be discarded, and the weight is 0; the valuation result is arranged according to the position order of the row matrix, to obtain a column matrix Y of the first set of similar images, such as column matrix Y = [3, 7, 0, …, L, 5], that is, the element x i in the i th position of the row matrix represents the same image as the element y i in the i th position of the column matrix.

[0050] In step S108, the row matrix and the column matrix of the first set of similar images are multiplied to obtain a target matrix, and a second set of similar images is screened from the first set of similar images according to the target matrix.

[0051] In this step, the row matrix X and the column matrix Y of the first set of similar images are multiplied to obtain a target matrix P, that is, P = X*Y, and the target matrix P is the weight ordering result obtained by joint image fingerprint matching and image content matching. The non-zero elements are arranged in ascending order of value to obtain a final matching sequence, and the smaller the element value in the matching sequence, the higher the matching degree of the image corresponding to the value with the target image.

[0052] In an embodiment of the present disclosure, referring to the flowchart of another image matching method shown in FIG. 2, the steps of screening the first set of similar images from the source image set can specifically include: Figure 2 Figure 2 In step S202, the hash values of each image in the source image set and the target image are calculated by using a perceptual hash algorithm.

[0053] In step S202, the hash values of each image in the source image set and the target image are calculated by using a perceptual hash algorithm.

[0054] ​This step, which uses the perceptual hash algorithm to calculate the hash value, can be divided into five steps: First, reduce the image size to remove high-frequency and detail elements, shrinking the image to an 8x8 pixel size (64 pixels in total). This eliminates differences caused by different sizes and aspect ratios. Second, simplify the colors by converting the 8x8 image into a grayscale image. Third, calculate the average pixel value by averaging the grayscale of all 64 pixels. Fourth, compare the grayscale values ​​of each pixel with the average value; if a pixel's grayscale value is greater than or equal to the average value, it is recorded as 1; otherwise, it is recorded as 0. Fifth, calculate the hash value by combining the comparison results from step four into a 64-bit integer, which is the hash value of the image.

[0055] Step S204: Calculate the Hamming distance between each image in the source image set and the target image based on the hash values ​​of each image in the source image set and the target image.

[0056] In this step, the hash values ​​of each image in the source image set and the target image are compared, which means counting the number of different bits in the two 64-bit sets. The number of different bits is the Hamming distance between the images in the source image set and the target image.

[0057] Step S206: Based on the Hamming distance between each image in the source image set and the target image, select the first group of similar images from the source image set.

[0058] In one embodiment of this disclosure, see Figure 3 The flowchart of another image matching method shown can be followed... Figure 3 The steps shown for calculating the hash value of the target image may specifically include:

[0059] Step S302: Perform grayscale processing on the scaled target image to obtain the grayscale processed target image;

[0060] In this step, the target image is scaled, retaining only the highlights and shadows, and uniformly containing 64 pixels, which means reducing the image to an 8×8 size. The scaled image is then subjected to grayscale processing to uniformly set 64 gray levels.

[0061] Step S304: Perform discrete cosine transform on the grayscale processed target image to obtain the frequency domain data of the target image;

[0062] In this step, a Discrete Cosine Transform (DCT) is performed on the grayscale-processed target image to obtain the frequency domain data of the target image. High-frequency information is discarded, and only the matrix information in the upper left corner is retained. The DCT transformation matrix D = TMT T Where T is the DCT matrix, M is the image data, and D is the DCT compression result.

[0063] Step S306, the average value of the pixels of the target image is calculated, and the gray value of each pixel of the target image is compared with the average value of the pixels. When the pixel is greater than or equal to the average value of the pixels, 1 is recorded, otherwise, 0 is recorded.

[0064] Step S308, the hash value of the target image is obtained according to the comparison result.

[0065] In this step, the comparison results of each pixel of the target image and the average value of the pixels are uniformly sequenced to obtain a 64-bit binary number, which is the hash value of the target image, that is, the fingerprint of the target image. Here, the 64-bit binary number can be converted into a 16-bit hexadecimal number for storage.

[0066] In an embodiment of the present disclosure, referring to another image matching method shown in Figure 4 , the Hamming distance between each image in the source image set and the target image can be calculated according to the steps shown in Figure 4 , which can specifically include:

[0067] Step S402, the hash values of each image in the source image set and the target image are subjected to an exclusive OR operation bit by bit;

[0068] Step S404, the Hamming distance between each image in the source image set and the target image is obtained according to the exclusive OR operation result.

[0069] The more the number of "0" in the operation result represents, the more the same bits of the hash values of the images in the source image set and the target image, and the higher the similarity. Conversely, the more the number of "1" in the operation result represents, the more the different bits of the hash values of the images in the source image set and the target image, and the lower the similarity. Generally, the images with a Hamming distance less than 5 are considered to meet the similarity condition, and the images with a Hamming distance greater than or equal to 10 are considered to be irrelevant and are discarded directly.

[0070] In an embodiment of the present disclosure, referring to another image matching method shown in Figure 5 , a column matrix of the first group of similar images can be obtained according to the steps shown in Figure 5 , which can specifically include:

[0071] Step S502, an element data pair of the target image is extracted;

[0072] In this step, the convolutional neural network is used to extract the features of the target image, the extracted features of the target image are input into the long short-term memory network to output a natural sentence for describing the target image, and the attention model is used to extract a key-value data pair <key, value> of the target image according to the natural sentence. The element data pair is obtained by removing the stop words of the key-value data pair.

[0073] Step S504, element comparison is performed on the target image and each image in the first set of similar images to obtain element overlap degrees between the target image and each image in the first set of similar images.

[0074] In this step, the key value of the element data pair of each image in the first set of similar images is compared with the target image. The higher the key value overlap degree is, the higher the content matching degree of the image is. After the key value comparison is completed, in order to improve the accuracy of image matching, value comparison can be further performed, that is, the value corresponding to the overlap degree key value is focused on and compared. The focus region in the value is used as a new image for fingerprint matching. The higher the similarity of the value corresponding to the overlap degree key value is, the more consistent the elements of the image are, and the higher the matching degree is.

[0075] Step S506, the images in the first set of similar images are valued according to the element overlap degrees of the target image and the first set of similar images, and a column matrix of the first set of similar images is obtained.

[0076] In an embodiment of the present disclosure, referring to another image matching method shown in Figure 6 , the element data pair of the target image can be extracted according to the steps shown in Figure 6 , and specifically can include the following steps.

[0077] Step S602, image features of the target image are extracted by using a convolutional neural network (CNN).

[0078] In this step, the CNN (Convolutional Neural Networks) is used to extract features of the target image, and the image features of the target image are obtained.

[0079] Step S604, the image features of the target image are input into a long short-term memory network (LSMT) to output a natural sentence used to describe the target image.

[0080] In this step, the image features of the target image are input into the LSMT (Long Short-Term Memory) to output a natural sentence as a description of the target image.

[0081] Step S606, key-value data pairs are extracted from the natural sentence of the target image by using an attention model, wherein the key in the key-value data pair is an image element label, and the value in the key-value data pair is an image after attention focusing processing on the corresponding key.

[0082] In this step, the attention model is used to extract the key-value data pair <key, value> from the natural language sentence describing the target image. The key key in the key-value data pair is the element label of the target image, and the value value in the key-value data pair is the image focused on the corresponding key key. For example, if the natural language sentence describing the target image is "a cat on the chair", when the output key key is "cat", the corresponding value value will focus on the cat in the picture, and when the output key key is "chair", the corresponding value value will focus on the chair in the picture.

[0083] In step S608, stop words are removed from the key-value data pair to obtain the element data pair of the target image.

[0084] In this step, stop words are removed from the key-value data pair <key, value>, i.e. the key-value data pair <key, value> corresponding to "a" and "the" in the above example is removed, and only the key-value data pair <key, value> corresponding to "cat", "on", and "chair" is retained to obtain the element data pair of the target image. By removing stop words, the information redundancy of image description is reduced to reduce the calculation amount of image matching and improve the comparison efficiency of image matching.

[0085] In one embodiment of the present disclosure, the second group of similar images is selected from the first group of similar images according to the target matrix. This step includes: arranging the elements with non-zero values in the target matrix in ascending order, and the smaller the element value, the higher the matching degree of the image corresponding to the element with the target image. A predetermined number of images are selected from the first group of similar images according to the size of the element value as the second group of similar images.

[0086] In one embodiment of the present disclosure, referring to the schematic diagram of an image matching method shown in Figure 7 As shown in Figure 7 The image matching method of the present disclosure is divided into image fingerprint matching and image content matching;

[0087] The image fingerprint matching is mainly completed by the pHash image perceptual hash algorithm to perform compressed grayscale processing on the picture, and the hash value of the picture is quantitatively obtained as the distance dependence for judging the similarity of the picture. The Hamming distance between the pictures in the source image set and the target picture is calculated, and the pictures are sorted according to the distance from near to far. The shorter the distance, the higher the similarity of the two pictures. After the image fingerprint matching, a column matrix is obtained.

[0088] The content matching of the image mainly obtains the main elements of the picture elements and the corresponding focus area through the use of the attention model. The traditional picture description uses an Encoder-Decoder model, and the semantic code c is always the same without distinction, while the attention model weights the information of the Encoder for each Decoder output, that is, a small number of important elements are selectively filtered from multiple elements and focused on the information of these important elements. The focusing process represents weight calculation, and the weight represents the importance of information. After the content matching of the image, a row matrix is obtained.

[0089] Finally, the fingerprint matching and the content matching of the image are combined, that is, the row matrix and the column matrix are multiplied to obtain a final matching sequence. The element value in the matching sequence comprehensively reflects the matching degree of each picture in the source image set and the target picture. The smaller the element value is, the higher the matching degree is, and vice versa.

[0090] Based on the same inventive concept, the embodiment of the present disclosure also provides an image matching device, as follows. Since the principle of solving the problem of the device embodiment is similar to that of the above-mentioned method embodiment, the implementation of the device embodiment can be referred to the implementation of the above-mentioned method embodiment, and the repeated parts will not be described here.

[0091] Figure 8 A schematic diagram of an image matching device in an embodiment of the present disclosure is shown, as shown in the figure, the device comprises: Figure 8

[0092] The first screening module 810 is configured to screen a first group of similar images from the source image set according to the Hamming distance between each image in the source image set and the target image.

[0093] The first assignment module 820 is configured to assign values to the images in the first group of similar images according to the size of the Hamming distance between the target image and each image in the first group of similar images, to obtain a row matrix of the first group of similar images.

[0094] The second assignment module 830 is configured to assign values to the images in the first group of similar images according to the height of the element overlap degree between the target image and each image in the first group of similar images, to obtain a column matrix of the first group of similar images.

[0095] The second screening module 840 is configured to multiply the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and to screen a second group of similar images from the first group of similar images according to the target matrix.

[0096] ​In an embodiment of the present disclosure, the first screening module 810 can be further configured to calculate hash values of each image in the source image set and the target image by using a perceptual hashing algorithm; calculate Hamming distances between each image in the source image set and the target image according to the hash values of each image in the source image set and the target image; and screen a first group of similar images from the source image set according to the Hamming distances between each image in the source image set and the target image.

[0097] In an embodiment of the present disclosure, the first screening module 810 can be further configured to perform grayscale processing on the scaled target image to obtain a grayscale-processed target image; perform discrete cosine transform processing on the grayscale-processed target image to obtain frequency domain data of the target image; calculate a pixel average value of the target image, compare a grayscale of each pixel of the target image with the pixel average value, and record 1 when the pixel is greater than or equal to the pixel average value, or record 0 otherwise; and obtain a hash value of the target image according to the comparison result.

[0098] In an embodiment of the present disclosure, the first screening module 810 can be further configured to perform XOR operation on the hash values of each image in the source image set and the target image bit by bit; and obtain Hamming distances between each image in the source image set and the target image according to the XOR operation result.

[0099] In an embodiment of the present disclosure, the second assignment module 830 can be further configured to extract element data pairs of the target image; perform element comparison on the target image and each image in the first group of similar images to obtain element overlap degrees between the target image and each image in the first group of similar images; assign values to the images in the first group of similar images according to the element overlap degrees between the target image and the first group of similar images, and obtain a column matrix of the first group of similar images.

[0100] In an embodiment of the present disclosure, the second assignment module 830 can be further configured to extract image features of the target image by using a convolutional neural network (CNN); input the image features of the target image into a long short-term memory (LSMT) network to output a natural sentence for describing the target image; extract key-value data pairs from the natural sentence of the target image by using an attention model, wherein a key in the key-value data pair is an image element label, and a value in the key-value data pair is an image after attention focusing processing on the corresponding key; and remove stop words from the key-value data pairs to obtain element data pairs of the target image.

[0101] In an embodiment of the present disclosure, the second screening module 840 can be further configured to arrange elements in the target matrix in ascending order, wherein a smaller element value indicates a higher matching degree between the image corresponding to the element and the target image, and screen a preset number of images from the first group of similar images as a second group of similar images according to the element values.

[0102] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied in a form of entirely hardware, entirely software (including firmware, microcode, etc.), or a combination of hardware and software, which can be generically referred to as "circuitry", "module" or "system".

[0103] The electronic device 900 according to this embodiment of the present disclosure will be described below with reference to Figure 9 Figure 9 The electronic device 900 shown is merely an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.

[0104] As Figure 9 shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 can include, but are not limited to, the at least one processing unit 910 described above, the at least one storage unit 920 described above, and a bus 930 connecting different system components, including the storage unit 920 and the processing unit 910.

[0105] The storage unit stores program code that can be executed by the processing unit 910, so that the processing unit 910 performs the steps described in the "Exemplary Method" section above according to various exemplary embodiments of the present disclosure. For example, the processing unit 910 can perform the following steps of the above method embodiments: selecting a first group of similar images from the source image set according to the Hamming distance between each image in the source image set and the target image; assigning values to the images in the first group of similar images according to the size of the Hamming distance between the target image and each image in the first group of similar images, to obtain a row matrix of the first group of similar images; assigning values to the images in the first group of similar images according to the level of element overlap between the target image and each image in the first group of similar images, to obtain a column matrix of the first group of similar images; multiplying the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and selecting a second group of similar images from the first group of similar images according to the target matrix.

[0106] The storage unit 920 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 9201 and / or a cache memory 9202, and can further include a read-only memory (ROM) 9203.

[0107] ​The storage unit 920 can also include the programs / utilities 9204 having a set (at least one) of program modules 9205, such as an operating system, one or more application programs, other program modules, and program data, and each or a combination thereof can include an implementation of a network environment.

[0108] The bus 930 can be representative of one or more of several types of bus structures, including a storage unit bus or bus controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus architectures.

[0109] The electronic device 900 can also communicate with one or more external devices 940 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices such as a storage device or an external effects device; and / or one or more devices that enable a user to interact with the electronic device 900; and / or one or more devices that enable the electronic device 900 to communicate with one or more other computing devices. Such communication can be facilitated by an Input / Output (I / O) interface 950. Still yet, the electronic device 900 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet, via a network adapter 960. As depicted, the network adapter 960 communicates with the other components of the electronic device 900 via the bus 930. It should be appreciated that although the network adapter 960 is depicted as a single component, the network adapter 960 can comprise two or more components that operate together to facilitate the communication of the electronic device 900 with one or more networks. Although not shown, it should be understood that the electronic device 900 can include other hardware and / or software components, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0110] From the above description of the embodiments, those skilled in the art will readily perceive that the example embodiments described herein can be implemented by software and / or by software in combination with the necessary hardware. Accordingly, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0111] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which can be a readable signal medium or a readable storage medium. A program product is stored thereon, which can implement the method of the present disclosure. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing the terminal device to perform the steps described in the above "example method" section according to various example embodiments of the present disclosure when the program product is run on the terminal device.

[0112] More specific examples of the computer readable storage medium in the present disclosure can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the foregoing.

[0113] In the present disclosure, the computer readable storage medium can include a data signal carried in the baseband or as a part of a carrier wave propagating through the program code, in which the readable program code is borne. Such a propagating data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal, or any appropriate combination of the foregoing. The readable signal medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit programs for use by or in connection with an instruction execution system, apparatus or device.

[0114] Optionally, the program code contained in the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any appropriate combination of the foregoing.

[0115] In specific implementation, the program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, etc., and a conventional procedural programming language such as "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected through the Internet by using an Internet service provider).

[0116] It should be noted that, although several modules or units of the devices for action execution are mentioned in the above detailed description, such division is not mandatory. Indeed, according to embodiments of the disclosure, features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, features and functionalities of one module or unit described above can be further divided into embodied by multiple modules or units.

[0117] Moreover, although the various steps of the methods of the disclosure are described in a particular order in the figures, this is not required or implied as to the order of execution of the steps, nor is it required that all of the steps be executed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, one step can be broken into multiple steps, etc.

[0118] From the above description of the embodiments, those skilled in the art will readily perceive that the example embodiments described herein can be implemented by software and / or by software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or network, and includes several instructions to make a computing device (which can be a personal computer, server, mobile terminal, or network device, etc.) execute the method according to the embodiments of the disclosure.

[0119] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features of the disclosure as disclosed herein. It is intended that the disclosure be considered as including any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such steps, compositions, components, and / or elements of the disclosure as come within the true scope of the disclosure. The specification and examples given are considered exemplary only, with the true scope and spirit of the disclosure being indicated by the appended claims.

Claims

1. An image matching method characterized by, The method comprises the following steps: filtering a first group of similar images from the source image set according to the Hamming distance between each image in the source image set and the target image; assigning values to the images in the first group of similar images according to the Hamming distance between the target image and each image in the first group of similar images, and obtaining a row matrix of the first group of similar images; assigning values to the images in the first group of similar images according to the degree of element overlap between the target image and each image in the first group of similar images, and obtaining a column matrix of the first group of similar images; multiplying the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and filtering a second group of similar images from the first group of similar images according to the target matrix; wherein the element xi in the i-th position of the row matrix and the element yi in the i-th position of the column matrix represent the same image; and each element in the target matrix is the multiplication result of the elements in the corresponding positions of the row matrix and the column matrix.

2. The image matching method of claim 1, wherein, The filtering of the first group of similar images from the source image set according to the Hamming distance between each image in the source image set and the target image comprises the following steps: calculating the hash value of each image in the source image set and the target image by using a perceptual hash algorithm; calculating the Hamming distance between each image in the source image set and the target image according to the hash value of each image in the source image set and the target image; filtering a first group of similar images from the source image set according to the Hamming distance between each image in the source image set and the target image.

3. The image matching method of claim 2, wherein, The calculation of the hash value of the target image by using the perceptual hash algorithm comprises the following steps: performing grayscale processing on the scaled target image to obtain a grayscale-processed target image; performing discrete cosine transform processing on the grayscale-processed target image to obtain frequency domain data of the target image; calculating the average value of the pixels of the target image, comparing the grayscale of each pixel of the target image with the average value of the pixels, and recording 1 when the pixel is greater than or equal to the average value of the pixels, or recording 0 otherwise; obtaining the hash value of the target image according to the comparison result.

4. The image matching method according to claim 2, characterized in that, The calculation of the Hamming distance between each image in the source image set and the target image according to the hash value of each image in the source image set and the target image comprises the following steps: performing XOR operation on the hash values of each image in the source image set and the target image bit by bit; obtaining the Hamming distance between each image in the source image set and the target image according to the XOR operation result.

5. The image matching method of claim 1, wherein, The assigning of values to the images in the first group of similar images according to the degree of element overlap between the target image and each image in the first group of similar images to obtain the column matrix of the first group of similar images comprises the following steps: extracting the element data pair of the target image; performing element comparison between the target image and each image in the first group of similar images to obtain the degree of element overlap between the target image and each image in the first group of similar images; assigning values to the images in the first group of similar images according to the degree of element overlap between the target image and each image in the first group of similar images to obtain the column matrix of the first group of similar images.

6. The image matching method of claim 5, wherein, The extracting the element data pair of the target image comprises: extracting image features of the target image by using a convolutional neural network (CNN); inputting the image features of the target image into a long short-term memory (LSMT) network to output a natural language sentence for describing the target image; extracting a key-value data pair from the natural language sentence of the target image by using an attention model, wherein the key in the key-value data pair is an image element label, and the value in the key-value data pair is an image after attention focusing processing on the corresponding key; removing stop words from the key-value data pair to obtain the element data pair of the target image.

7. The image matching method of claim 1, wherein, The filtering a second group of similar images from the first group of similar images according to the target matrix comprises: arranging elements with non-zero values in the target matrix in ascending order, and the smaller the element value is, the higher the matching degree of the element corresponding image with the target image is, and filtering a preset number of images from the first group of similar images as the second group of similar images according to the size of the element value.

8. An image matching apparatus characterized by comprising: comprises: a first filtering module configured to filter a first group of similar images from a source image set according to a Hamming distance between each image in the source image set and a target image; a first assignment module configured to assign values to images in the first group of similar images according to the size of the Hamming distance between the target image and each image in the first group of similar images, to obtain a row matrix of the first group of similar images; a second assignment module configured to assign values to images in the first group of similar images according to the degree of element overlap between the target image and each image in the first group of similar images, to obtain a column matrix of the first group of similar images; a second filtering module configured to multiply the row matrix and the column matrix of the first group of similar images to obtain a target matrix, and filter a second group of similar images from the first group of similar images according to the target matrix; wherein the element xi in the i-th position of the row matrix and the element yi in the i-th position of the column matrix represent the same image; and each element in the target matrix is a multiplication result of the elements in the corresponding positions of the row matrix and the column matrix.

9. An electronic device, comprising: comprises: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the image matching method of any one of claims 1-7 by executing the executable instructions.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the image matching method of any one of claims 1-7. The computer program is executed by the processor to implement the image matching method of any one of claims 1-7.

Images