Image deduplication method and device, electronic equipment and storage medium

Abstract

The invention provides an image deduplication method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: determining an image hash value of an image in a to-be-processed image set; determining an image to be subjected to similarity comparison according to the image hash value; for any two images to be subjected to similarity comparison, performing similarity comparison at least based on image semantic features, and determining whether the any two images are similar or not; and removing similar images in the to-be-processed image set to realize image de-duplication of the to-be-processed image set, thereby comprehensively considering image set similarity comparison times and image similarity comparison precision, adopting a two-stage strategy, using relatively fast comparison in the first stage, and performing fine similarity comparison in the second stage, so that the image de-duplication of the to-be-processed image set is realized. And a relatively good de-weighting effect is ensured while the de-weighting speed of the image set is considered.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to an image deduplication method, device, electronic device and storage medium. Background Art

[0002] Image deduplication is an important part of the dataset construction process. Its purpose is to identify and delete duplicate or similar images in an image collection. Image deduplication has a wide range of application scenarios, such as image deduplication methods can be applied to large-scale image dataset cleaning, similar image location in cloud disks, key information storage of surveillance videos, etc.

[0003] In current image deduplication methods, a common approach is to compare the similarity of every two images in the image set one by one, and filter out duplicate images by setting a similarity threshold. However, in the context of deduplication of large-scale image sets, performing similarity comparisons on every two images will result in a huge computational cost. In order to address these challenges, a fast method for deduplication of massive images has been proposed to quickly remove duplicate images. However, this method only uses the simplest features of the image, ignoring the most significant semantic features between images, and cannot achieve further comparison when the low-level features of the images are similar. Therefore, how to balance the deduplication speed and deduplication accuracy of the image set is a technical problem that needs to be solved urgently in the present invention. Summary of the invention

[0004] Embodiments of the present invention provide an image deduplication method, an apparatus, an electronic device, and a storage medium to overcome the above problems or at least partially solve the above problems.

[0005] A first aspect of an embodiment of the present invention provides an image deduplication method, the method comprising:

[0006] Determining image hash values ​​for images in the set of images to be processed;

[0007] Determine the image to be compared for similarity according to the image hash value;

[0008] For any two images to be compared for similarity, performing a similarity comparison based at least on image semantic features to determine whether the two images are similar;

[0009] Similar images in the image set to be processed are removed to achieve image deduplication in the image set to be processed.

[0010] Optionally, for any two images to be compared for similarity, performing a similarity comparison based at least on image semantic features to determine whether the two images are similar includes:

[0011] Performing category detection on the arbitrary two images respectively to obtain a plurality of categories corresponding to the arbitrary two images respectively;

[0012] For the multiple categories respectively corresponding to the arbitrary two images, semantic segmentation is performed under each category to obtain image semantic features respectively corresponding to the arbitrary two images;

[0013] Determine the appearance features corresponding to the two images respectively;

[0014] Based on the image semantic features respectively corresponding to the arbitrary two images and the appearance features respectively corresponding to the arbitrary two images, an image similarity comparison is performed to determine whether the arbitrary two images are similar.

[0015] Optionally, the image semantic features include: pixel quantity features of an object set of each category; performing image similarity comparison based on the image semantic features corresponding to the arbitrary two images and the appearance features corresponding to the arbitrary two images to determine whether the arbitrary two images are similar includes:

[0016] At least based on the pixel quantity feature of the object set of each category respectively corresponding to the two images, determining whether the two images meet a first similarity condition, wherein the first similarity condition indicates whether the semantic features of the two images are similar;

[0017] Based on the appearance features respectively corresponding to the arbitrary two images, determining whether the arbitrary two images satisfy a second similarity condition, where the second similarity condition indicates whether the appearance features of the arbitrary two images are similar;

[0018] In the case where the arbitrary two images satisfy the first similarity condition and the second similarity condition, it is determined that the arbitrary two images are similar, and a similar image among the arbitrary two images is obtained.

[0019] Optionally, determining whether the arbitrary two images satisfy a first similarity condition based at least on the pixel quantity feature of the object set of each category respectively corresponding to the arbitrary two images includes:

[0020] Based on the pixel quantity characteristics of the object set of each category corresponding to the arbitrary two images, respectively, determining the sum of the pixel differences of the objects of each category corresponding to the arbitrary two images, and the sum of the pixels of all categories;

[0021] Determine a first proportion based on the sum of pixel differences of the objects in each category and the sum of pixels of all categories;

[0022] Determine a second proportion based on the sum of pixel differences of the objects of each category and the number of pixels of the reference image in the arbitrary two images;

[0023] Based on the first proportion and the second proportion, it is determined whether the any two images meet the first similarity condition.

[0024] Optionally, determining whether the any two images satisfy the first similarity condition based on the first proportion and the second proportion includes:

[0025] Comparing the first proportion with a first threshold, and comparing the second proportion with a second threshold;

[0026] When the first proportion is less than the first threshold, and the second proportion is less than the second threshold, determining that the any two images satisfy the first similarity condition;

[0027] When the first proportion is greater than the first threshold, or the second proportion is greater than the second threshold, it is determined that the any two images do not satisfy the first similarity condition.

[0028] Optionally, the determining the sum of pixel differences of objects of each category corresponding to the arbitrary two images and the sum of pixels of all categories based on the pixel quantity features of the object set of each category respectively corresponding to the arbitrary two images comprises:

[0029] For a same category among the multiple categories respectively corresponding to the arbitrary two images, determining the object set pixel quantity features corresponding to the same category of the arbitrary two images based on the object set pixel quantity features of each category respectively corresponding to the arbitrary two images;

[0030] Determine the difference in the number of pixels of objects in the same category based on the pixel number characteristics of the object sets in the same category corresponding to the two arbitrary images respectively;

[0031] Based on the difference in the number of pixels of objects in the same category, determine the sum of the difference in the number of pixels of objects in each category corresponding to the arbitrary two images;

[0032] The sum of the pixel quantity features of the object set of each category corresponding to the arbitrary two images is determined as the sum of all category pixels corresponding to the arbitrary two images.

[0033] Optionally, the method further comprises:

[0034] Determine a first image as a reference image from the arbitrary two images;

[0035] Taking the first image as a reference, performing a uniform scale transformation on the second image of the arbitrary two images except the first image to obtain a third image;

[0036] The performing category detection on the arbitrary two images respectively to obtain a plurality of categories corresponding to the arbitrary two images respectively includes:

[0037] Performing multi-label category detection on the first image and the third image respectively to obtain multiple categories corresponding to the first image and the third image respectively;

[0038] The performing semantic segmentation under each category for the multiple categories respectively corresponding to the arbitrary two images to obtain the image semantic features respectively corresponding to the arbitrary two images comprises:

[0039] For a plurality of categories respectively corresponding to the first image and the third image, semantic segmentation is performed under each category to obtain image semantic features respectively corresponding to the first image and the third image;

[0040] The determining the appearance features respectively corresponding to the arbitrary two images comprises:

[0041] Calculating appearance features corresponding to the first image and the second image respectively;

[0042] The performing image similarity comparison based on the image semantic features respectively corresponding to the arbitrary two images and the appearance features respectively corresponding to the arbitrary two images to determine whether the arbitrary two images are similar includes:

[0043] Based on the image semantic features corresponding to the first image and the third image respectively, and the appearance features corresponding to the first image and the second image respectively, an image similarity comparison is performed to determine whether the arbitrary two images are similar.

[0044] Optionally, determining the image to be compared for similarity according to the image hash value includes:

[0045] According to the image hash value, the mapping pair of each image is sequentially added to the initialized mapping dictionary to construct a hash image mapping dictionary; the index value of the initialized mapping dictionary is the image hash value of the image, and the value of the initialized mapping dictionary is the image identifier of the image; the mapping pair includes: the image hash value of the image and the image identifier of the image;

[0046] Determining one or more images corresponding to each image hash value based on the hash image mapping dictionary;

[0047] Determine multiple images corresponding to the same image hash value as a sub-image cluster to obtain multiple sub-image clusters;

[0048] The images in the same sub-image cluster are determined as the images to be compared for similarity.

[0049] A second aspect of an embodiment of the present invention provides an image deduplication device, the device comprising:

[0050] A hash value determination module, used to determine the image hash value of the image in the image set to be processed;

[0051] An image determination module, used to determine an image to be compared for similarity based on the image hash value;

[0052] An image comparison module, for performing a similarity comparison on any two images to be compared for similarity, at least based on image semantic features, to determine whether the two images are similar;

[0053] The image deduplication module is used to remove similar images in the image set to be processed, so as to achieve image deduplication in the image set to be processed.

[0054] Optionally, the image comparison module includes:

[0055] A classification module, used to perform category detection on the arbitrary two images respectively, to obtain a plurality of categories corresponding to the arbitrary two images respectively;

[0056] A segmentation module, for performing semantic segmentation in each category for a plurality of categories respectively corresponding to the arbitrary two images, to obtain image semantic features respectively corresponding to the arbitrary two images;

[0057] An appearance feature determination module, used to determine the appearance features corresponding to the two arbitrary images respectively;

[0058] The similarity comparison module is used to perform image similarity comparison based on the image semantic features corresponding to the arbitrary two images and the appearance features corresponding to the arbitrary two images, so as to determine whether the arbitrary two images are similar.

[0059] Optionally, the image semantic features include: pixel quantity features of object sets of each category; the similarity comparison module includes:

[0060] A first similarity comparison module is used to determine whether the arbitrary two images meet a first similarity condition based at least on the pixel quantity characteristics of the object set of each category respectively corresponding to the arbitrary two images, wherein the first similarity condition indicates whether the semantic features of the arbitrary two images are similar;

[0061] A second similarity comparison module, used to determine whether the arbitrary two images meet a second similarity condition based on the appearance features respectively corresponding to the arbitrary two images, wherein the second similarity condition indicates whether the appearance features of the arbitrary two images are similar;

[0062] The similar image determination module is used to determine that the arbitrary two images are similar when the arbitrary two images satisfy the first similarity condition and the second similarity condition, and obtain a similar image among the arbitrary two images.

[0063] Optionally, the first similarity comparison module includes:

[0064] A first determination module is used to determine the sum of pixel differences of objects of each category corresponding to the arbitrary two images and the sum of pixels of all categories based on the pixel quantity characteristics of the object set of each category respectively corresponding to the arbitrary two images;

[0065] A second determination module, configured to determine a first proportion based on the sum of pixel differences of the objects in each category and the sum of pixels of all categories;

[0066] A third determination module, configured to determine a second proportion based on the sum of pixel differences of the objects of each category and the number of pixels of the reference image in the any two images;

[0067] The fourth determination module is used to determine whether the any two images meet the first similarity condition based on the first proportion and the second proportion.

[0068] Optionally, the fourth determining module includes:

[0069] A comparison module, configured to compare the first proportion with a first threshold, and to compare the second proportion with a second threshold;

[0070] a fifth determining module, configured to determine that the any two images satisfy the first similarity condition when the first proportion is less than the first threshold and the second proportion is less than the second threshold;

[0071] The sixth determination module is used to determine that the any two images do not satisfy the first similarity condition when the first proportion is greater than the first threshold or the second proportion is greater than the second threshold.

[0072] Optionally, the first determining module includes:

[0073] A pixel quantity determination module, for determining, for a same category among the multiple categories respectively corresponding to the any two images, based on the pixel quantity characteristics of the object set of each category respectively corresponding to the any two images, the pixel quantity characteristics of the object set corresponding to the same category respectively corresponding to the any two images;

[0074] A pixel quantity difference determination module, used to determine the pixel quantity difference of objects in the same category based on the pixel quantity characteristics of the object sets in the same category corresponding to the two arbitrary images respectively;

[0075] A pixel quantity difference sum determination module, used for determining the pixel difference sum of each category of objects corresponding to the arbitrary two images based on the pixel quantity difference of each object under the same category;

[0076] The pixel sum determination module is used to determine the sum of the pixel quantity features of the object set of each category corresponding to the arbitrary two images as the sum of all category pixels corresponding to the arbitrary two images.

[0077] Optionally, the device further comprises:

[0078] A reference image determination module, used to determine a first image as a reference image from the arbitrary two images;

[0079] An image conversion module, used to perform a uniform scale transformation on a second image of the arbitrary two images except the first image, based on the first image, to obtain a third image;

[0080] The classification module comprises:

[0081] A classification submodule, used to perform multi-label category detection on the first image and the third image respectively, to obtain a plurality of categories corresponding to the first image and the third image respectively;

[0082] The segmentation module comprises:

[0083] a segmentation submodule, configured to perform semantic segmentation in each of the multiple categories corresponding to the first image and the third image, respectively, to obtain image semantic features corresponding to the first image and the third image, respectively;

[0084] The appearance feature determination module includes:

[0085] an appearance feature determination submodule, used to calculate appearance features corresponding to the first image and the second image respectively;

[0086] The similarity comparison module comprises:

[0087] The similarity comparison submodule is used to perform image similarity comparison based on the image semantic features corresponding to the first image and the third image respectively, and the appearance features corresponding to the first image and the second image respectively, to determine whether any two images are similar.

[0088] Optionally, the image determination module includes:

[0089] A mapping dictionary construction module is used to add the mapping pair of each image to the initialized mapping dictionary in turn according to the image hash value to construct a hash image mapping dictionary; the index value of the initialized mapping dictionary is the image hash value of the image, and the value of the initialized mapping dictionary is the image identifier of the image; the mapping pair includes: the image hash value of the image and the image identifier of the image;

[0090] An image corresponding module, used to determine one or more images corresponding to each image hash value based on the hash image mapping dictionary;

[0091] A sub-image cluster determination module, used to determine multiple images corresponding to the same image hash value as a sub-image cluster, to obtain multiple sub-image clusters;

[0092] The image determination submodule is used to determine images in the same sub-image cluster as the images to be compared for similarity.

[0093] A third aspect of an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, an image deduplication method according to the first aspect of the embodiment of the present invention is implemented.

[0094] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the image deduplication method of the first aspect of the embodiment of the present invention is implemented.

[0095] In the image deduplication method provided by the embodiment of the present invention, first, the image hash value of the image in the image set to be processed is determined; secondly, the image to be compared for similarity is determined based on the image hash value; then, for any two images to be compared for similarity, a similarity comparison is performed at least based on the image semantic features to determine whether any two images are similar; finally, similar images in the image set to be processed are removed to achieve image deduplication in the image set to be processed.

[0096] In this method, a two-stage strategy is adopted for the image set to be processed that needs to be deduplicated. To address the problem of slow deduplication speed, in the first stage, images are compared more quickly based on image hash values ​​to screen out images to be compared for similarity, thereby reducing the range of image similarity comparison and saving computing resources. To address the problem of inaccurate deduplication accuracy, a fine similarity comparison is performed in the second stage: for any two images to be compared for similarity, the semantic features between the images are taken into account, and at least a similarity comparison is performed based on the semantic features of the images, so as to achieve further similarity comparison when the low-level features of the images are similar, thereby improving the accuracy of similarity comparison between images. By determining whether any two images are similar, the determined similar images are removed, thereby achieving image deduplication for the image set to be processed while taking into account the deduplication speed of the image set and ensuring a good deduplication effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0098] Figure 1 is a flow chart of an image deduplication method shown in one embodiment of the present invention;

[0099] Figure 2 is an example diagram of inter-image scale unification proposed by an embodiment of the present invention;

[0100] Figure 3 is a flow chart of a two-stage image deduplication method based on hash image mapping according to an embodiment of the present invention;

[0101] Figure 4 is a structural block diagram of an image deduplication device provided by an embodiment of the present invention;

[0102] Figure 5 It is a schematic diagram of an electronic device shown in one embodiment of the present invention. DETAILED DESCRIPTION

[0103] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0104] Hereinafter, specific examples of the present solution will be described in more detail with reference to the accompanying drawings.

[0105] refer to Figure 1 , Figure 1 FIG. 1 is a flow chart of an image deduplication method according to an embodiment of the present invention. Figure 1 As shown, the image deduplication method of this embodiment may include the following steps:

[0106] Step S11: Determine the image hash value of the image in the image set to be processed.

[0107] In this embodiment, the image set to be processed is an image set to be deduplicated, and the image set to be processed includes multiple images. The image hash value of each image in the image set to be processed can be determined, wherein the image hash value of this embodiment is a hash value obtained by converting the image through a hash algorithm, and is used to represent the characteristics of the image.

[0108] Step S12: Determine the image to be compared for similarity based on the image hash value.

[0109] In this embodiment, after obtaining the image hash value of each image in the image set to be processed, the images in the image set to be processed can be compared more quickly according to the image hash value, and the images to be compared for similarity can be determined from the image set to be processed, and most of the dissimilar images can be quickly screened out, thereby reducing the image similarity comparison range, and no longer needing to compare the similarity of each two images in the image set to be processed one by one, thereby saving computing resources. The images to be compared for similarity in this embodiment are images that may be similar and to be further compared for image similarity based on the image hash value.

[0110] Step S13: for any two images to be compared for similarity, a similarity comparison is performed based at least on image semantic features to determine whether the two images are similar.

[0111] In this embodiment, the images to be compared for similarity include multiple images. For any two images to be compared for similarity, a similarity comparison can be performed between the any two images based on at least the image semantic features respectively corresponding to the any two images to determine whether the any two images are similar. In this way, when it is determined that any two images are similar, a similar image among the two images is obtained.

[0112] Step S14: removing similar images from the image set to be processed, thereby achieving image deduplication in the image set to be processed.

[0113] In this embodiment, after obtaining similar images of any two images through the above steps, all similar images in the image set to be processed can be obtained, and similar images in the image set to be processed can be removed to achieve image deduplication in the image set to be processed.

[0114] In this embodiment, for the image set to be processed that needs to be deduplicated, a two-stage strategy is adopted. To address the problem of slow deduplication speed, in the first stage, a relatively fast rough comparison of images is performed based on image hash values, most of the dissimilar images are quickly screened out, and images to be compared for similarity are screened out, thereby reducing the image similarity comparison range and saving computing resources; to address the problem of inaccurate deduplication accuracy; in the second stage, a more refined similarity comparison is performed to confirm the final image deduplication result: for any two images to be compared for similarity, the semantic features between the images are considered, and at least the similarity comparison is performed based on the semantic features of the images, so as to achieve further similarity comparison when the low-level features of the images are similar, thereby improving the similarity comparison accuracy between the images; by determining whether any two images are similar, and removing the similar images in the determined any two images, image deduplication is achieved for the image set to be processed while taking into account the deduplication speed of the image set and ensuring a good deduplication effect. The method of this embodiment can quickly reduce the amount of comparison calculation in the first stage, thereby improving the overall deduplication speed, and perform a more accurate comparison of the remaining images in the second stage to ensure accuracy.

[0115] In addition, in a preferred embodiment, in the above step S11, for each image in the image set, the mean hash algorithm can be used to convert the image into an image hash value of a fixed length, so that the image that needs to be compared for similarity in the second stage can be quickly located by the hash value. In this embodiment, the mean hash algorithm is selected. Average Hash (AverageHash, AHash): is an image hash algorithm for converting an image into a binary hash code of a fixed length. By scaling, graying, and calculating the average gray value of the image, the image information is compressed into a hash code of a fixed length for image similarity comparison and retrieval. It has a faster feature extraction speed than other methods. And in the image Resize step, scaling the image size to 5x5 can speed up the feature extraction speed without affecting the image positioning effect.

[0116] In an optional example, determining the image hash value of the image using the mean hash algorithm may specifically include the following steps:

[0117] S1-1, let the original image be I, and reduce the image size to w×h, where w and h are length and width respectively, w and h are both set to 5, I resizeResize represents the scaled image, and the calculation method is shown in the following formula (1):

[0118] I resize d=Resize(I,w,h) (1)

[0119] S1-2, the scaled image I resize , converted to a 25-level (5x5) grayscale image, denoted as I gray , the calculation method is shown in the following formula (2);

[0120] I gray (i,j)=Grayscale(I resized (i,j)) (2)

[0121] Among them, I gray (i, j) is the grayscale of the pixel in the image; i and j represent the horizontal and vertical coordinates of the pixel in the image respectively; Grayscale is grayscale processing.

[0122] S1-3, calculate the average grayscale average of all pixels, traverse each pixel point in turn, and calculate its average value. The calculation method is shown in the following formula (3);

[0123]

[0124] S1-4. Compare the grayscale of each pixel with the average grayscale average. If it is greater than or equal to the average grayscale, it is recorded as 1; if it is less than the average, it is recorded as 0. Finally, a 25-bit integer is formed, which is the image hash value of the image (i.e., a hash value of fixed length).

[0125] In combination with the above embodiments, in one implementation, the present invention also provides an image deduplication method. Specifically, in the method, the above step S13 may include steps S21 to S24:

[0126] Step S21: performing category detection on the arbitrary two images respectively to obtain a plurality of categories corresponding to the arbitrary two images respectively.

[0127] In this embodiment, the category detection can be performed on any two images respectively to obtain multiple categories corresponding to any two images. In an optional example, the labels in the image can be identified by a multi-label classification algorithm, and each category in the image can be detected to obtain multiple categories of image content. Among them, multi-label classification (MLC, Multi-Label Classification): In multi-label classification, each sample has an associated set of labels, rather than belonging to only a single category. This task is common in natural language processing, computer vision, bioinformatics and other fields. For example, in image classification, an image can contain multiple objects such as cats, trees and cars at the same time.

[0128] Step S22: for the multiple categories respectively corresponding to the arbitrary two images, semantic segmentation is performed under each category to obtain image semantic features respectively corresponding to the arbitrary two images.

[0129] In this embodiment, after obtaining multiple categories corresponding to any two images, semantic segmentation can be performed under each category for the multiple categories corresponding to the any two images to obtain image semantic features corresponding to the any two images. Semantic segmentation (SemSeg) is an image segmentation task in the field of computer vision, and its goal is to divide each pixel in the image into a corresponding semantic category. It is different from ordinary image segmentation. Semantic segmentation not only focuses on the boundaries of objects, but also pays attention to the semantic information of each pixel in the image.

[0130] Step S23: Determine the appearance features corresponding to the arbitrary two images respectively.

[0131] In this embodiment, in addition to determining the image semantic features corresponding to any two images, the appearance features corresponding to the any two images are also determined. The appearance features of this embodiment are color histogram appearance features, which can be obtained by calculating the color histogram features of any two images respectively. Among them, the color histogram is an important representation method of image appearance features, which is suitable for describing and analyzing the color distribution in an image. By calculating the color histogram features of an image, the frequency and distribution of various colors in the image can be obtained, thereby reflecting the overall color characteristics of the image.

[0132] Step S24: performing image similarity comparison based on the image semantic features corresponding to the arbitrary two images and the appearance features corresponding to the arbitrary two images to determine whether the arbitrary two images are similar.

[0133] In this embodiment, based on the image semantic features respectively corresponding to the arbitrary two images and the appearance features respectively corresponding to the arbitrary two images, an image similarity comparison may be performed on the arbitrary two images to determine whether the arbitrary two images are similar.

[0134] In this embodiment, for any two images, multi-category detection is first performed, and then semantic segmentation is performed under each category to obtain image semantic features, so as to combine the image semantic features with the image appearance features to complete the image similarity calculation, and perform more refined similarity calculation, which can improve the accuracy of image similarity comparison and thus achieve refined image deduplication.

[0135] In combination with the above embodiments, in another implementation, the present invention also provides an image deduplication method. Specifically, in the method, the above step S24 may include steps S31 to S33:

[0136] Step S31: determining whether the arbitrary two images satisfy a first similarity condition based at least on the pixel quantity characteristics of the object set of each category respectively corresponding to the arbitrary two images, wherein the first similarity condition represents whether the semantic features of the arbitrary two images are similar.

[0137] In this embodiment, the image semantic features include: the pixel quantity feature of each category of the object set. In this embodiment, each object in the image is segmented and the pixel quantity feature of each category of the object set is calculated. The pixel quantity feature of the object set represents the pixel quantity of the semantic object under the category.

[0138] In an optional implementation, a multi-label classification algorithm can be used to detect each category in the image, and then each category is used as a prompt word to perform semantic segmentation on the image to obtain a mask matrix for each category corresponding to the image, and then the length and width of the mask matrix of each category are traversed in turn to calculate the pixel quantity feature of the object set of each category. For example, the mask matrix ClsMaskArray of each category can be calculated by formula (4):

[0139] ClsMaskArray=SemSeg(I,MLC(I)) (4)

[0140] Where I represents image, MLC represents multi-label classification algorithm, MLC(I) represents category classification result, and SemSeg represents semantic segmentation. In this embodiment, semantic segmentation (SemSeg) is to segment the image to obtain the number of image pixels occupied by the multi-label classification categories. Through semantic segmentation, the analysis of image content can be further refined to obtain the specific location and range of each object, as well as the category to which they belong. This step is crucial for the subsequent object set pixel quantity feature calculation, which can more accurately describe and analyze the image content.

[0141] After obtaining the mask matrix ClsMaskArray of each category, the length w and width h of ClsMaskArray can be traversed in sequence through formula (5) to calculate the number of pixels N(clsid) of the object set of each category:

[0142]

[0143] Among them, clsid represents the category, i and j represent the horizontal and vertical coordinates of the pixels in the image respectively.

[0144] In this embodiment, after obtaining the object set pixel quantity features of each category corresponding to any two images, it is possible to determine whether any two images meet the first similarity condition based on the object set pixel quantity features of each category corresponding to any two images. The first similarity condition of this embodiment represents whether the semantic features of any two images are similar.

[0145] Step S32: Based on the appearance features respectively corresponding to the arbitrary two images, determine whether the arbitrary two images meet a second similarity condition, where the second similarity condition represents whether the appearance features of the arbitrary two images are similar.

[0146] In this embodiment, it is also necessary to determine whether the two images meet a second similarity condition based on the appearance features corresponding to the two images. The second similarity condition in this embodiment represents whether the appearance features of the two images are similar. Step S33: When it is determined that the two images meet the first similarity condition and the second similarity condition, the two images are determined to be similar, and a similar image among the two images is obtained.

[0147] In this embodiment, when any two images satisfy the first similarity condition and the second similarity condition, that is, when the semantic features and the appearance features of any two images respectively satisfy the corresponding first similarity condition and the second similarity condition, it can be determined that the any two images are similar, and one image can be selected from the any two images as the similar image of the any two images for removal.

[0148] In one implementation, a threshold corresponding to the appearance feature may be set, and based on the appearance features corresponding to any two images and the threshold corresponding to the appearance feature, it is determined whether any two images meet the second similarity condition. In one example, a value greater than the threshold corresponding to the appearance feature satisfies the second similarity condition, and a value less than the threshold corresponding to the appearance feature does not satisfy the second similarity condition.

[0149] In another optional implementation, when it is determined that any two images satisfy the first similarity condition, it is determined whether any two images satisfy the second similarity condition based on the appearance features corresponding to the two images. When it is determined that the two images satisfy the first similarity condition, the second similarity condition is further compared based on the appearance features of the images, so that it can be determined whether any two images satisfy the first similarity condition and the second similarity condition at the same time while saving computing resources.

[0150] In combination with the above embodiments, in one implementation, the present invention also provides an image deduplication method. Specifically, in the method, the above step S31 may include steps S41 to S44:

[0151] Step S41: based on the pixel quantity characteristics of the object set of each category corresponding to the arbitrary two images, determine the sum of the pixel differences of the objects of each category corresponding to the arbitrary two images, and the sum of the pixels of all categories.

[0152] In this embodiment, based on the pixel quantity characteristics of each category of objects corresponding to any two images, the sum of pixel differences of each category of objects corresponding to the any two images and the sum of pixels of all categories corresponding to the any two images can be determined.

[0153] Step S42: determining a first proportion based on the sum of pixel differences of the objects in each category and the sum of pixels of all categories.

[0154] In this embodiment, after obtaining the sum of the pixel differences of each category of objects corresponding to any two images and the sum of all category pixels corresponding to the any two images, a first ratio can be determined based on the sum of the pixel differences of each category of objects corresponding to the any two images and the sum of all category pixels corresponding to the any two images to characterize the relationship between the difference in the number of category pixels and the sum of the category pixels.

[0155] Step S43: determining a second proportion based on the sum of pixel differences of the objects of each category and the number of pixels of the reference image in the arbitrary two images.

[0156] In this embodiment, one image can be selected from any two images as a reference image, and the number of pixels of the reference image can be determined. The number of pixels of the reference image is the length multiplied by the width of the reference image, that is, w×h. Then, based on the sum of the pixel differences of each category object corresponding to the any two images and the number of pixels of the reference image in the any two images, the second proportion is determined to characterize the relationship between the difference in the number of category pixels and the number of pixels of the reference image.

[0157] Step S44: Based on the first proportion and the second proportion, determine whether the any two images meet the first similarity condition.

[0158] In this embodiment, after the first proportion and the second proportion are obtained, it can be determined whether the arbitrary two images meet the first similarity condition based on the first proportion and the second proportion.

[0159] In combination with the above embodiments, in one implementation, the present invention also provides an image deduplication method. Specifically, in the method, the above step S44 may include steps S51 to S53:

[0160] Step S51: comparing the first proportion with a first threshold, and comparing the second proportion with a second threshold.

[0161] In this embodiment, the first proportion can be compared with a first threshold, and the second proportion can be compared with a second threshold. In this embodiment, a first threshold is set in advance for the first proportion, and the first threshold represents the maximum value of the first proportion corresponding to the similarity of the two images, which can be freely specified according to experience or project requirements, for example, the first threshold can be 1 / 25, etc. In addition, a second threshold is set in advance for the second proportion, and the second threshold represents the maximum value of the second proportion corresponding to the similarity of the two images, which can be freely specified according to experience or project requirements, for example, the first threshold can be 1 / 1000, etc.

[0162] Step S52: when the first proportion is smaller than the first threshold value, and the second proportion is smaller than the second threshold value, determining that the any two images satisfy the first similarity condition.

[0163] In this embodiment, when it is determined that the first proportion is less than the first threshold and the second proportion is less than the second threshold, it can be determined that the arbitrary two images meet the first similarity condition, indicating that the semantic features of the arbitrary two images are similar.

[0164] Step S53: when the first proportion is greater than the first threshold, or the second proportion is greater than the second threshold, determining that the any two images do not satisfy the first similarity condition.

[0165] In this embodiment, when it is determined that the first proportion is greater than the first threshold value, or the second proportion is greater than the second threshold value, it can be determined that any two images do not meet the first similarity condition, indicating that the semantic features of the any two images are not similar. For example, when the first proportion is greater than 1 / 25, or the second proportion is greater than 1 / 1000, the any two images do not meet the first similarity condition, the semantic features of the any two images are not similar, that is, the any two images are dissimilar images.

[0166] In combination with the above embodiments, in one implementation, the present invention also provides an image deduplication method. Specifically, in the method, the above step S41 may include steps S61 to S64:

[0167] Step S61: for the same category among the multiple categories respectively corresponding to the arbitrary two images, determine the object set pixel quantity features corresponding to the same category of the arbitrary two images based on the object set pixel quantity features of each category respectively corresponding to the arbitrary two images.

[0168] In this embodiment, for the same category among the multiple categories respectively corresponding to any two images, the object set pixel quantity features of each category respectively corresponding to any two images obtained based on semantic segmentation can be used to determine the object set pixel quantity features of the same category respectively corresponding to the any two images. For example, if the same category is category A, it is necessary to determine the object set pixel quantity features of category A respectively corresponding to the any two images (i.e., the object set pixel quantity features under category A).

[0169] Step S62: determining the difference in the number of pixels of objects in the same category based on the pixel number characteristics of the object sets of the same category corresponding to the two arbitrary images respectively.

[0170] In this embodiment, the difference in the number of object pixels of the same category can be determined based on the obtained object set pixel number features of the same category corresponding to the two images. For example, based on the object set pixel number feature of one of the two images under category A and the object set pixel number feature of the other of the two images under category A, the difference in the number of object pixels of the two images under category A is determined.

[0171] Step S63: Based on the difference in the number of object pixels of each category, determine the sum of the difference in the number of object pixels of each category corresponding to the arbitrary two images.

[0172] In this embodiment, after obtaining the difference in the number of object pixels of each same category corresponding to any two images, the sum of the difference in the number of object pixels of each category corresponding to any two images can be determined based on the difference in the number of object pixels of each same category.

[0173] Step S64: Determine the sum of the pixel quantity features of the object set of each category corresponding to the arbitrary two images as the sum of all category pixels corresponding to the arbitrary two images.

[0174] In this embodiment, the sum of the pixel quantity features of the object set of each category corresponding to any two images can be determined as the sum of all category pixels corresponding to any two images.

[0175] In an optional implementation, the first ratio DiffRatio1 can be determined by the following formula (6), and the second ratio DiffRatio2 can be determined by the following formula (7), that is, DiffRatio1 is the ratio of the sum of the pixel differences of each category to the sum of all category pixels, and DiffRatio2 is the ratio of the sum of the pixel differences of each category to the number of pixels in the reference image.

[0176]

[0177] Among them, firstCls and endCls are the upper and lower limits of the category obtained by multi-label classification, and clsid represents the category. is the number of object set pixels of the clsid category in the first image of any two images; N is the number of pixels of the object set of the clsid category in the second image of any two images; base is the number of pixels of the reference image in S3-1. That is, the sum of pixel differences of objects in each category. It is the sum of all category pixels.

[0178] In combination with the above embodiments, in one implementation, an embodiment of the present invention further provides an image deduplication method. Specifically, in addition to the above steps, the method may also include step S71 and step S72, and the above step S21 may specifically include step S73, the above step S22 may specifically include step S74, the above step S23 may specifically include step S75, and the above step S24 may specifically include step S76:

[0179] Step S71: Determine a first image as a reference image from the arbitrary two images.

[0180] In this embodiment, any two images can be scaled in the second stage. Scaling (Scale Standardization) refers to adjusting the size of the images to make them have the same size or scaling ratio. In image processing, the purpose of scale standardization is to enable images of different sizes to be effectively compared or analyzed, especially in tasks such as image similarity calculation.

[0181] In this embodiment, a first image as a reference image can be determined from any two images. In an optional example, the reference image can be the image with the shortest side of any two images. The shortest side is the shortest length or width. For example, the image with the shortest width of the two images is the reference image, that is, the first image; for another example, when the widths of the two images are equal, the image with the shortest length is used as the reference image, that is, the first image.

[0182] Step S72: Taking the first image as a reference, performing a uniform scale transformation on the second image of the arbitrary two images except the first image to obtain a third image.

[0183] In this embodiment, the image other than the first image in any two images is the second image, and the first image is used as a reference to perform a uniform scale transformation on the second image to obtain a third image. Specifically, based on the image ratio of the second image (i.e., on the premise that the image ratio of the second image remains consistent), the corresponding side of the second image is scaled to be consistent with the shortest side of the first image to obtain the scaled third image. For example, if the shortest side of the first image is the length of the first image, then based on the image ratio of the second image, the length of the second image is scaled to be consistent with the length of the first image to obtain the third image. In this embodiment, the first image as the reference image remains unchanged.

[0184] like Figure 2 As shown, Figure 2 is an example diagram of inter-image scale unification proposed by an embodiment of the present invention. Figure 2 In the example, the first image is I a , whose length and width are w a 、h a ; The second image is recorded as I b , whose length and width are w b 、h b . Take the first image I a As a reference, in the second image I b Under the premise of keeping the image ratio consistent, the second image I b h b Zoom to the same image as the reference image (first image I a )'s short side (h a ) are consistent, and the third image I is obtainedresized-b .

[0185] Step S73: performing multi-label category detection on the first image and the third image respectively to obtain multiple categories corresponding to the first image and the third image respectively.

[0186] In this embodiment, after the scale is unified, multi-label category detection is performed on the first image and the third image to obtain multiple categories corresponding to the first image and the third image, respectively. The specific method is the same or similar to the aforementioned step 21.

[0187] Step S74: for the multiple categories corresponding to the first image and the third image respectively, semantic segmentation is performed under each category to obtain image semantic features corresponding to the first image and the third image respectively.

[0188] In this embodiment, semantic segmentation is performed on multiple categories corresponding to the first image and the third image respectively, and image semantic features corresponding to the first image and the third image respectively are obtained. The specific method is the same or similar to the aforementioned step 22.

[0189] Step S75: Calculate the appearance features corresponding to the first image and the second image respectively.

[0190] In this embodiment, the appearance features of any two original images, ie, the first image and the second image, are calculated to obtain the appearance features corresponding to the first image and the second image respectively.

[0191] Step S76: Based on the image semantic features corresponding to the first image and the third image respectively, and the appearance features corresponding to the first image and the second image respectively, an image similarity comparison is performed to determine whether any two images are similar.

[0192] In this embodiment, when obtaining the image semantic features corresponding to the first image and the third image respectively, and the appearance features corresponding to the first image and the second image respectively, an image similarity comparison is performed based on the image semantic features corresponding to the first image and the third image respectively, and the appearance features corresponding to the first image and the second image respectively, to determine whether the two arbitrary images are similar, and when the two arbitrary images are similar, a similar image in the two arbitrary images is obtained. The specific method may be the same or similar to the aforementioned step S24 or the preferred implementation method of step S24.

[0193] In this embodiment, the size of the image is adjusted by resizing, and any two images to be compared are rescaled to eliminate the possible impact of size differences on subsequent feature calculations and similarity comparisons. Based on the method of image resizing, multi-label classification, semantic segmentation, pixel number comparison, and color histogram appearance feature comparison in this embodiment, the content and features of the image can be considered more comprehensively, thereby improving the accuracy and robustness of image deduplication.

[0194] In combination with the above embodiments, in one implementation, the embodiment of the present invention further provides an image deduplication method. Specifically, in the method, the above step S12 may specifically include steps S81 to S84, the above step S23 may specifically include step S75, and the above step S24 may specifically include step S76:

[0195] Step S81: according to the image hash value, add the mapping pair of each image to the initialized mapping dictionary in turn to construct a hash image mapping dictionary.

[0196] In this embodiment, a mapping dictionary is first initialized, the index value of the initialized mapping dictionary is the image hash value of the image, and the value of the initialized mapping dictionary is the image identifier of the image. Then, according to the image hash value, the mapping pair of each image in the image set to be processed is added to the initialized mapping dictionary in turn to construct a hash image mapping dictionary. Among them, the mapping pair of the image includes: the image hash value of the image and the image identifier of the image.

[0197] Among them, if the image hash value exists in the index of the initialized mapping dictionary, its corresponding image identifier is inserted into the image identifier list corresponding to the image hash value in the initialized mapping dictionary; if the image hash value does not exist in the index of the initialized mapping dictionary, a mapping pair between the image hash value and the image identifier is established and added to the initialized mapping dictionary, thereby constructing a hash image mapping dictionary. For example, the sample of the hash image mapping dictionary finally constructed is as follows:

[0198]

[0199] Step S82: Based on the hash image mapping dictionary, determine one or more images corresponding to each image hash value.

[0200] In this embodiment, one or more image identifiers corresponding to each image hash value in the image set to be processed may be determined based on the constructed hash image mapping dictionary, thereby obtaining one or more images corresponding to each image hash value.

[0201] Step S83: multiple images corresponding to the same image hash value are determined as a sub-image cluster to obtain multiple sub-image clusters.

[0202] In this embodiment, multiple images corresponding to the same image hash value are determined as a sub-image cluster to obtain multiple sub-image clusters in the image set to be processed. In other words, the images in the same sub-image cluster have the same image hash value.

[0203] Step S84: determining images in the same sub-image cluster as the images to be compared for similarity.

[0204] In this embodiment, after obtaining multiple sub-image clusters in the image set to be processed, images in the same sub-image cluster are determined as images to be compared for similarity. That is, in this embodiment, the second-stage similarity comparison is performed between two images with the same hash value, and the second-stage similarity comparison is not performed between images with different hash values, thereby reducing the traversal and comparison of the entire image set, saving computing resources and time. It should be noted that when the image hash value corresponds to only one image, it can be directly determined that there is no similar image to the image corresponding to the image hash value.

[0205] In this embodiment, a mapping dictionary of hash values ​​and images is constructed based on the image hash values, and the image set to be processed is divided into one or more sub-image clusters. The hash values ​​are used as features to classify similar images into the same cluster, and the second stage of fine similarity calculation is performed for the images in each sub-image cluster. The method based on the construction of the mapping dictionary of hash values ​​and sub-image clusters can effectively index and search images quickly, thereby improving the efficiency of image deduplication. Through the mapping of hash values, images with the same hash value can be quickly located, reducing the traversal and comparison of the entire image set, saving computing resources and time.

[0206] In one embodiment, if Figure 3 As shown, Figure 3 FIG. 1 is a flowchart of a two-stage image deduplication method based on hash image mapping according to an embodiment of the present invention. Figure 3 In the method, the mean hash of each image in the image set (i.e., the image set to be processed) is first calculated in turn to obtain the image hash value corresponding to each image; then, an image hash mapping table (i.e., hash image mapping dictionary) is constructed based on the image hash values, each hash value corresponds to one or more images, and the image set is divided into several image clusters (i.e., sub-image clusters); finally, the images in the image cluster are compared for similarity between each other (e.g., image similarity comparison is performed based on a combination of traditional features and deep semantic features) to obtain similar images in the sub-image cluster, and similar images are filtered to achieve image deduplication for the image set to be processed while taking into account the deduplication speed of the image set and ensuring good deduplication effect.

[0207] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0208] Based on the same inventive concept, an embodiment of the present invention provides an image deduplication device 400. Figure 4 , Figure 4 FIG. 1 is a structural block diagram of an image deduplication device provided by an embodiment of the present invention. Figure 4 As shown, the device 400 includes:

[0209] A hash value determination module 401 is used to determine the image hash value of an image in the image set to be processed;

[0210] An image determination module 402 is used to determine an image to be compared for similarity based on the image hash value;

[0211] An image comparison module 403 is used to compare the similarity of any two images to be compared based on at least the semantic features of the images to determine whether the two images are similar.

[0212] The image deduplication module 404 is used to remove similar images in the image set to be processed, so as to achieve image deduplication in the image set to be processed.

[0213] Optionally, the image comparison module 403 includes:

[0214] A classification module, used to perform category detection on the arbitrary two images respectively, to obtain a plurality of categories corresponding to the arbitrary two images respectively;

[0215] A segmentation module, for performing semantic segmentation in each category for a plurality of categories respectively corresponding to the arbitrary two images, to obtain image semantic features respectively corresponding to the arbitrary two images;

[0216] An appearance feature determination module, used to determine the appearance features corresponding to the two arbitrary images respectively;

[0217] The similarity comparison module is used to perform image similarity comparison based on the image semantic features corresponding to the arbitrary two images and the appearance features corresponding to the arbitrary two images, so as to determine whether the arbitrary two images are similar.

[0218] Optionally, the image semantic features include: pixel quantity features of object sets of each category; the similarity comparison module includes:

[0219] A first similarity comparison module is used to determine whether the arbitrary two images meet a first similarity condition based at least on the pixel quantity characteristics of the object set of each category respectively corresponding to the arbitrary two images, wherein the first similarity condition indicates whether the semantic features of the arbitrary two images are similar;

[0220] A second similarity comparison module, used to determine whether the arbitrary two images meet a second similarity condition based on the appearance features respectively corresponding to the arbitrary two images, wherein the second similarity condition indicates whether the appearance features of the arbitrary two images are similar;

[0221] The similar image determination module is used to determine that the arbitrary two images are similar when the arbitrary two images satisfy the first similarity condition and the second similarity condition, and obtain a similar image among the arbitrary two images.

[0222] Optionally, the first similarity comparison module includes:

[0223] A first determination module is used to determine the sum of pixel differences of objects of each category corresponding to the arbitrary two images and the sum of pixels of all categories based on the pixel quantity characteristics of the object set of each category respectively corresponding to the arbitrary two images;

[0224] A second determination module, configured to determine a first proportion based on the sum of pixel differences of the objects in each category and the sum of pixels of all categories;

[0225] A third determination module, configured to determine a second proportion based on the sum of pixel differences of the objects of each category and the number of pixels of the reference image in the any two images;

[0226] The fourth determination module is used to determine whether the any two images meet the first similarity condition based on the first proportion and the second proportion.

[0227] Optionally, the fourth determining module includes:

[0228] A comparison module, configured to compare the first proportion with a first threshold, and to compare the second proportion with a second threshold;

[0229] a fifth determining module, configured to determine that the any two images satisfy the first similarity condition when the first proportion is less than the first threshold and the second proportion is less than the second threshold;

[0230] The sixth determination module is used to determine that the any two images do not satisfy the first similarity condition when the first proportion is greater than the first threshold or the second proportion is greater than the second threshold.

[0231] Optionally, the first determining module includes:

[0232] A pixel quantity determination module, for determining, for a same category among the multiple categories respectively corresponding to the any two images, based on the pixel quantity characteristics of the object set of each category respectively corresponding to the any two images, the pixel quantity characteristics of the object set corresponding to the same category respectively corresponding to the any two images;

[0233] A pixel quantity difference determination module, used to determine the pixel quantity difference of objects in the same category based on the pixel quantity characteristics of the object sets in the same category corresponding to the two arbitrary images respectively;

[0234] A pixel quantity difference sum determination module, used for determining the pixel difference sum of each category of objects corresponding to the arbitrary two images based on the pixel quantity difference of each object under the same category;

[0235] The pixel sum determination module is used to determine the sum of the pixel quantity features of the object set of each category corresponding to the arbitrary two images as the sum of all category pixels corresponding to the arbitrary two images.

[0236] Optionally, the device 400 further includes:

[0237] A reference image determination module, used to determine a first image as a reference image from the arbitrary two images;

[0238] An image conversion module, used to perform a uniform scale transformation on a second image of the arbitrary two images except the first image, based on the first image, to obtain a third image;

[0239] The classification module comprises:

[0240] A classification submodule, used to perform multi-label category detection on the first image and the third image respectively, to obtain a plurality of categories corresponding to the first image and the third image respectively;

[0241] The segmentation module comprises:

[0242] a segmentation submodule, configured to perform semantic segmentation in each of the multiple categories corresponding to the first image and the third image, respectively, to obtain image semantic features corresponding to the first image and the third image, respectively;

[0243] The appearance feature determination module includes:

[0244] an appearance feature determination submodule, used to calculate appearance features corresponding to the first image and the second image respectively;

[0245] The similarity comparison module comprises:

[0246] The similarity comparison submodule is used to perform image similarity comparison based on the image semantic features corresponding to the first image and the third image respectively, and the appearance features corresponding to the first image and the second image respectively, to determine whether any two images are similar.

[0247] Optionally, the image determination module 402 includes:

[0248] A mapping dictionary construction module is used to add the mapping pair of each image to the initialized mapping dictionary in turn according to the image hash value to construct a hash image mapping dictionary; the index value of the initialized mapping dictionary is the image hash value of the image, and the value of the initialized mapping dictionary is the image identifier of the image; the mapping pair includes: the image hash value of the image and the image identifier of the image;

[0249] An image corresponding module, used to determine one or more images corresponding to each image hash value based on the hash image mapping dictionary;

[0250] A sub-image cluster determination module, used to determine multiple images corresponding to the same image hash value as a sub-image cluster, to obtain multiple sub-image clusters;

[0251] The image determination submodule is used to determine images in the same sub-image cluster as the images to be compared for similarity.

[0252] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the image deduplication method described in any of the above embodiments of the present invention are implemented.

[0253] Based on the same inventive concept, another embodiment of the present invention provides an electronic device 500, such as Figure 5 shown. Figure 5 1 is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device includes a memory 502, a processor 501, and a computer program stored in the memory and executable on the processor, and when executed by the processor, the steps in the image deduplication method according to any of the above embodiments of the present invention are implemented.

[0254] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0255] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0256] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, embodiments of the present invention may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware. Furthermore, embodiments of the present invention may take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0257] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0258] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0259] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0260] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0261] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0262] The above is a detailed introduction to an image deduplication method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An image deduplication method, characterized in that: The method comprises: Determining image hash values ​​for images in the set of images to be processed; Determine the image to be compared for similarity according to the image hash value; For any two images to be compared for similarity, performing a similarity comparison based at least on image semantic features to determine whether the two images are similar; Similar images in the image set to be processed are removed to achieve image deduplication in the image set to be processed.

2. The image deduplication method according to claim 1, characterized in that: The similarity comparison is performed on any two images to be compared based on at least image semantic features to determine whether the two images are similar, including: Performing category detection on the arbitrary two images respectively to obtain a plurality of categories corresponding to the arbitrary two images respectively; For the multiple categories respectively corresponding to the arbitrary two images, semantic segmentation is performed under each category to obtain image semantic features respectively corresponding to the arbitrary two images; Determine the appearance features corresponding to the two images respectively; Based on the image semantic features respectively corresponding to the arbitrary two images and the appearance features respectively corresponding to the arbitrary two images, an image similarity comparison is performed to determine whether the arbitrary two images are similar.

3. The image deduplication method according to claim 2, characterized in that: The image semantic features include: pixel quantity features of an object set of each category; performing image similarity comparison based on the image semantic features corresponding to the arbitrary two images and the appearance features corresponding to the arbitrary two images to determine whether the arbitrary two images are similar includes: At least based on the pixel quantity feature of the object set of each category respectively corresponding to the two images, determining whether the two images meet a first similarity condition, wherein the first similarity condition indicates whether the semantic features of the two images are similar; Based on the appearance features respectively corresponding to the arbitrary two images, determining whether the arbitrary two images satisfy a second similarity condition, where the second similarity condition indicates whether the appearance features of the arbitrary two images are similar; In the case where the arbitrary two images satisfy the first similarity condition and the second similarity condition, it is determined that the arbitrary two images are similar, and a similar image among the arbitrary two images is obtained.

4. The image deduplication method according to claim 3, characterized in that: The determining whether the arbitrary two images satisfy the first similarity condition based on at least the pixel quantity feature of the object set of each category respectively corresponding to the arbitrary two images comprises: Based on the pixel quantity characteristics of the object set of each category corresponding to the arbitrary two images, respectively, determining the sum of the pixel differences of the objects of each category corresponding to the arbitrary two images, and the sum of the pixels of all categories; Determine a first proportion based on the sum of pixel differences of the objects in each category and the sum of pixels of all categories; Determine a second proportion based on the sum of pixel differences of the objects of each category and the number of pixels of the reference image in the arbitrary two images; Based on the first proportion and the second proportion, it is determined whether the any two images meet the first similarity condition.

5. The image deduplication method according to claim 4, characterized in that: The determining, based on the first proportion and the second proportion, whether the any two images satisfy the first similarity condition includes: Comparing the first proportion with a first threshold, and comparing the second proportion with a second threshold; When the first proportion is less than the first threshold, and the second proportion is less than the second threshold, determining that the any two images satisfy the first similarity condition; When the first proportion is greater than the first threshold, or the second proportion is greater than the second threshold, it is determined that the any two images do not satisfy the first similarity condition.

6. The image deduplication method according to claim 4, characterized in that: The method of determining the sum of pixel differences of objects of each category corresponding to the two images based on the pixel quantity characteristics of the object set of each category respectively corresponding to the two images, and the sum of pixels of all categories, comprises: For a same category among the multiple categories respectively corresponding to the arbitrary two images, determining the object set pixel quantity features corresponding to the same category of the arbitrary two images based on the object set pixel quantity features of each category respectively corresponding to the arbitrary two images; Determine the difference in the number of pixels of objects in the same category based on the pixel number characteristics of the object sets in the same category corresponding to the two arbitrary images respectively; Based on the difference in the number of pixels of objects in the same category, determine the sum of the difference in the number of pixels of objects in each category corresponding to the arbitrary two images; The sum of the pixel quantity features of the object set of each category corresponding to the arbitrary two images is determined as the sum of all category pixels corresponding to the arbitrary two images.

7. The image deduplication method according to any one of claims 2 to 6, characterized in that: The method further comprises: Determine a first image as a reference image from the arbitrary two images; Taking the first image as a reference, performing a uniform scale transformation on the second image of the arbitrary two images except the first image to obtain a third image; The performing category detection on the arbitrary two images respectively to obtain a plurality of categories corresponding to the arbitrary two images respectively includes: Performing multi-label category detection on the first image and the third image respectively to obtain multiple categories corresponding to the first image and the third image respectively; The performing semantic segmentation under each category for the multiple categories respectively corresponding to the arbitrary two images to obtain the image semantic features respectively corresponding to the arbitrary two images comprises: For a plurality of categories respectively corresponding to the first image and the third image, semantic segmentation is performed under each category to obtain image semantic features respectively corresponding to the first image and the third image; The determining the appearance features respectively corresponding to the arbitrary two images comprises: Calculating appearance features corresponding to the first image and the second image respectively; The performing image similarity comparison based on the image semantic features respectively corresponding to the arbitrary two images and the appearance features respectively corresponding to the arbitrary two images to determine whether the arbitrary two images are similar includes: Based on the image semantic features corresponding to the first image and the third image respectively, and the appearance features corresponding to the first image and the second image respectively, an image similarity comparison is performed to determine whether the arbitrary two images are similar.

8. The image deduplication method according to any one of claims 1 to 6, characterized in that: The step of determining the image to be compared for similarity according to the image hash value includes: According to the image hash value, the mapping pair of each image is sequentially added to the initialized mapping dictionary to construct a hash image mapping dictionary; the index value of the initialized mapping dictionary is the image hash value of the image, and the value of the initialized mapping dictionary is the image identifier of the image; the mapping pair includes: the image hash value of the image and the image identifier of the image; Determining one or more images corresponding to each image hash value based on the hash image mapping dictionary; Determine multiple images corresponding to the same image hash value as a sub-image cluster to obtain multiple sub-image clusters; The images in the same sub-image cluster are determined as the images to be compared for similarity.

9. An image deduplication device, characterized in that: The device comprises: A hash value determination module, used to determine the image hash value of the image in the image set to be processed; An image determination module, used to determine an image to be compared for similarity based on the image hash value; An image comparison module, for performing a similarity comparison on any two images to be compared for similarity, at least based on image semantic features, to determine whether the two images are similar; The image deduplication module is used to remove similar images in the image set to be processed, so as to achieve image deduplication in the image set to be processed.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the image deduplication method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image deduplication method according to any one of claims 1 to 8 is implemented.

Images