Image Processing Method and Device Based on Deep Learning

Abstract

This invention discloses a deep learning-based image processing method and apparatus. The method includes: inputting each image to be processed from a set of images to be processed into an image processing model to obtain a hash value corresponding to each image to be processed; and, based on the hash value corresponding to each image to be compared, calculating a similarity representation value between each image to be compared and every remaining image to be compared among all images to be compared, and determining that the image to be compared corresponding to the target similarity representation value satisfies a preset similarity condition when the target similarity representation value is greater than a preset similarity threshold. Therefore, implementing this invention can combine the hash value of the image to determine the similarity representation value of the image, improving the accuracy of the similarity representation value between images in representing image differences, and improving the accuracy and reliability of determining whether images are similar, thereby improving the accuracy and reliability of image deduplication.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method and apparatus based on deep learning. Background Technology

[0002] With the rapid development of the internet, people are using image and video data more and more frequently, especially in the e-commerce sector, where e-commerce material libraries often contain many similar or duplicate image or video materials. However, as the amount of similar or duplicate image or video data in storage space increases, it not only occupies storage space but also increases the difficulty for people to find the target image or video data. Therefore, deduplication of similar or duplicate image or video data has become very important.

[0003] Currently, several commonly used deduplication techniques, such as MD5, hash deduplication, and feature extraction deduplication, all suffer from low accuracy in determining whether images are duplicates, resulting in poor deduplication performance. Therefore, improving the accuracy of determining the similarity between multiple images is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] This invention provides an image processing method and apparatus based on deep learning, which can help improve the accuracy and reliability of determining the similarity between multiple images.

[0005] To address the aforementioned technical problems, the first aspect of this invention discloses a deep learning-based image processing method, the method comprising:

[0006] Each image to be processed in the set of images to be processed is input into the image processing model to obtain the hash value corresponding to each image to be processed. The set of images to be processed includes at least two images to be processed.

[0007] At least two images are selected from the set of images to be processed as images to be compared.

[0008] Based on the hash value corresponding to each of the images to be compared, for each image to be compared, calculate the similarity representation value between the image to be compared and each of the remaining images to be compared among all the images to be compared, and obtain the set of similarity representation values ​​of the image to be compared.

[0009] For each image to be compared, if there is a target similarity value in the set of similarity representation values ​​of the image to be compared that is greater than a preset similarity threshold, it is determined that the image to be compared corresponding to the target similarity representation value meets the preset similarity condition.

[0010] As an optional implementation, in the first aspect of the present invention, the step of inputting each image to be processed in the set of images to be processed into an image processing model to obtain a hash value corresponding to each image to be processed includes:

[0011] Each image to be processed in the set of images to be processed is input into the image processing model to obtain a first output and a second output corresponding to each image to be processed;

[0012] Calculate the hash value corresponding to each image to be processed based on the first and second outputs corresponding to each image to be processed.

[0013] Furthermore, the method further includes:

[0014] The hash value corresponding to each image to be processed is binarized by a preset data conversion threshold to obtain the binarized hash value corresponding to each image to be processed, which is used as the hash value corresponding to each image to be processed.

[0015] As an optional implementation, in the first aspect of the present invention, the first output corresponding to each of the images to be processed includes a first feature vector corresponding to the image to be processed, and the second output corresponding to each of the images to be processed includes a second feature vector corresponding to the image to be processed.

[0016] The image processing model is a deep learning-type image processing model, and the image processing model includes a feature mapping layer, a bidirectional LSTM layer, and a fully connected layer;

[0017] The feature mapping layer is used to map the feature vector corresponding to each image to be processed to a preset first dimension. The bidirectional LSTM layer is used to capture the context feature information corresponding to each image to be processed. The number of fully connected layers is at least two. At least one of the fully connected layers is used to reduce the dimensionality of the context feature information to a preset second dimension to obtain and output the first output. At least one of the fully connected layers is used to output the second output.

[0018] As an optional implementation, in the first aspect of the present invention, calculating the hash value corresponding to each of the images to be processed based on the first output and the second output corresponding to each image to be processed includes:

[0019] Based on the first and second outputs corresponding to each image to be processed, and combined with the predetermined hash value calculation formula, calculate the hash value corresponding to each image to be processed.

[0020] The formula for calculating the hash value of the i-th image in each image to be processed is as follows:

[0021]

[0022] Among them, H i This represents the hash value corresponding to the i-th image to be processed, k represents the gradient amplification factor, and F1 represents the hash value. i F2 represents the first output corresponding to the i-th image to be processed. i This represents the second output corresponding to the i-th image to be processed.

[0023] As an optional implementation, in the first aspect of the present invention, the method further includes:

[0024] A predetermined set of training images is input into the image processing model to obtain the hash value of each training image in the set. The set of training images includes at least one pair of samples and a sample label value corresponding to each pair. All the sample pairs included in the set of training images include positive sample pairs and / or negative sample pairs. The sample label value of the positive sample pair is a positive sample label value, and the sample label value of the negative sample pair is a negative sample label value. Each sample pair includes at least two training images.

[0025] For each of the sample pairs, a similarity representation value for the sample pair is calculated based on the hash value of each of the training images in the sample pair;

[0026] For each sample pair, the loss corresponding to the sample pair is calculated based on the similarity representation value and the sample label value of the sample pair;

[0027] The parameters of the image processing model are updated based on the loss corresponding to each sample pair.

[0028] As an optional implementation, in the first aspect of the present invention, the step of calculating a similarity representation value between each image to be compared and every remaining image to be compared (excluding the image to be compared) based on the hash value corresponding to each image to be compared, to obtain a set of similarity representation values ​​for the images to be compared, includes:

[0029] Among all the images to be compared, a first image to be compared and a second image to be compared are determined, wherein the number of the second image to be compared is at least one;

[0030] For each of the second images to be compared, a similarity representation value between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, thus obtaining the similarity representation value between the first image to be compared and the second image to be compared.

[0031] Based on the similarity representation values ​​between the first image to be compared and each of the second images to be compared, a set of similarity representation values ​​for the first image to be compared is generated.

[0032] And, after determining that the image to be compared corresponding to the target similarity representation value meets the preset similarity condition, the method further includes:

[0033] Based on the predetermined image deduplication conditions, the target duplicate image is determined and removed from all the images to be compared that satisfy the similarity conditions;

[0034] The image deduplication conditions include at least one of the following: image storage time, image size, and image storage space usage.

[0035] As an optional implementation, in a first aspect of the present invention, the step of calculating a similarity representation value between the first image to be compared and the second image to be compared, based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, to obtain the similarity representation value between the first image to be compared and the second image to be compared, includes:

[0036] For each second image to be compared, the cosine similarity between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared.

[0037] or,

[0038] For each of the second images to be compared, the Hamming distance between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared.

[0039] The similarity representation value includes the cosine similarity or the Hamming distance.

[0040] A second aspect of the present invention discloses an image processing apparatus based on deep learning, the apparatus comprising:

[0041] The first processing module is used to input each image to be processed in the image processing model to obtain a hash value corresponding to each image to be processed. The image processing set includes at least two images to be processed.

[0042] The determining module is used to determine at least two images from the set of images to be processed as images to be compared;

[0043] The calculation module is used to calculate, based on the hash value corresponding to each of the images to be compared, the similarity representation value between each image to be compared and each of the remaining images to be compared (excluding the image to be compared) among all the images to be compared, to obtain a set of similarity representation values ​​for the images to be compared.

[0044] The determining module is further configured to, for each image to be compared, determine that the image to be compared corresponding to the target similarity value satisfies the preset similarity condition when there is a target similarity value in the set of similarity representation values ​​of the image to be compared that is greater than the preset similarity threshold.

[0045] As an optional implementation, in a second aspect of the invention,

[0046] The first processing module inputs each image to be processed from the set of images to be processed into the image processing model to obtain the hash value corresponding to each image to be processed. Specifically, this includes:

[0047] Each image to be processed in the set of images to be processed is input into the image processing model to obtain a first output and a second output corresponding to each image to be processed;

[0048] Calculate the hash value corresponding to each image to be processed based on the first and second outputs corresponding to each image to be processed.

[0049] The device also includes:

[0050] The second processing module is used to perform binarization processing on the hash value corresponding to each image to be processed by a preset data conversion threshold, so as to obtain the binarized hash value corresponding to each image to be processed, which is used as the hash value corresponding to each image to be processed.

[0051] As an optional implementation, in a second aspect of the present invention, the first output corresponding to each of the images to be processed includes a first feature vector corresponding to the image to be processed, and the second output corresponding to each of the images to be processed includes a second feature vector corresponding to the image to be processed.

[0052] The image processing model is a deep learning-type image processing model, and the image processing model includes a feature mapping layer, a bidirectional LSTM layer, and a fully connected layer;

[0053] The feature mapping layer is used to map the feature vector corresponding to each image to be processed to a preset first dimension. The bidirectional LSTM layer is used to capture the context feature information corresponding to each image to be processed. The number of fully connected layers is at least two. At least one of the fully connected layers is used to reduce the dimensionality of the context feature information to a preset second dimension to obtain and output the first output. At least one of the fully connected layers is used to output the second output.

[0054] As an optional implementation, in a second aspect of the present invention, the method by which the first processing module calculates the hash value corresponding to each of the images to be processed based on the first output and the second output corresponding to each image to be processed specifically includes:

[0055] Based on the first and second outputs corresponding to each image to be processed, and combined with the predetermined hash value calculation formula, calculate the hash value corresponding to each image to be processed.

[0056] The formula for calculating the hash value of the i-th image in each image to be processed is as follows:

[0057]

[0058] Among them, H i This represents the hash value corresponding to the i-th image to be processed, k represents the gradient amplification factor, and F1 represents the hash value. i F2 represents the first output corresponding to the i-th image to be processed. i This represents the second output corresponding to the i-th image to be processed.

[0059] As an optional implementation, in a second aspect of the present invention, the first processing module is further configured to input a predetermined set of training images into the image processing model to obtain a hash value for each training image in the set of training images, wherein the set of training images includes at least one pair of samples and a sample label value corresponding to each pair of samples, all the sample pairs included in the set of training images include positive sample pairs and / or negative sample pairs, wherein the sample label value of the positive sample pair is a positive sample label value, the sample label value of the negative sample pair is a negative sample label value, and each pair of samples includes at least two training images;

[0060] The calculation module is further configured to, for each sample pair, calculate a similarity representation value for the sample pair based on the hash value of each training image in the sample pair; and for each sample pair, calculate the loss corresponding to the sample pair based on the similarity representation value and the sample label value of the sample pair.

[0061] The device further includes:

[0062] An update module is used to update the parameters of the image processing model based on the corresponding loss for each sample pair.

[0063] As an optional implementation, in the second aspect of the present invention, the calculation module calculates, based on the hash value corresponding to each of the images to be compared, a similarity representation value between each image to be compared and each of the remaining images to be compared (excluding the image to be compared) to obtain the set of similarity representation values ​​for the images to be compared. Specifically, this includes:

[0064] Among all the images to be compared, a first image to be compared and a second image to be compared are determined, wherein the number of the second image to be compared is at least one;

[0065] For each of the second images to be compared, a similarity representation value between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, thus obtaining the similarity representation value between the first image to be compared and the second image to be compared.

[0066] Based on the similarity representation values ​​between the first image to be compared and each of the second images to be compared, a set of similarity representation values ​​for the first image to be compared is generated.

[0067] The device also includes:

[0068] The removal module is used to determine and remove target duplicate images from all the images to be compared that satisfy the similarity conditions after the determining module determines that the image to be compared corresponding to the target similarity representation value meets the preset similarity conditions, according to the preset image deduplication conditions;

[0069] The image deduplication conditions include at least one of the following: image storage time, image size, and image storage space usage.

[0070] As an optional implementation, in a second aspect of the present invention, the calculation module calculates a similarity representation value between the first image to be compared and the second image to be compared for each second image to be compared, based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared. The specific method for obtaining the similarity representation value between the first image to be compared and the second image to be compared includes:

[0071] For each second image to be compared, the cosine similarity between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared.

[0072] or,

[0073] For each of the second images to be compared, the Hamming distance between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared.

[0074] The similarity representation value includes the cosine similarity or the Hamming distance.

[0075] A third aspect of the present invention discloses another image processing apparatus based on deep learning, the apparatus comprising:

[0076] Memory containing executable program code;

[0077] A processor coupled to the memory;

[0078] The processor calls the executable program code stored in the memory to execute the deep learning-based image processing method disclosed in the first aspect of the present invention.

[0079] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute the deep learning-based image processing method disclosed in the first aspect of the present invention.

[0080] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0081] In this embodiment of the invention, each image to be processed in the image set is input into an image processing model to obtain a hash value corresponding to each image. At least two images are selected from the image set as comparison images. Based on the hash value of each comparison image, for each comparison image, a similarity representation value is calculated between the comparison image and each of the remaining comparison images (excluding the comparison image itself) to obtain a set of similarity representation values ​​for that comparison image. For each comparison image, if there is a target similarity representation value in the set of similarity representation values ​​that is greater than a preset similarity threshold, the comparison image corresponding to the target similarity representation value is determined to meet a preset similarity condition. Therefore, implementing this invention can combine the hash value of the image to determine the image similarity representation value, improving the accuracy of the determined similarity representation values ​​for image differences, enhancing the accuracy and reliability of determining whether images are similar, and thus improving the accuracy and reliability of image deduplication. Attached Figure Description

[0082] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0083] Figure 1 This is a flowchart illustrating an image processing method based on deep learning disclosed in an embodiment of the present invention;

[0084] Figure 2 This is a flowchart illustrating another deep learning-based image processing method disclosed in an embodiment of the present invention.

[0085] Figure 3 This is a schematic diagram of the structure of an image processing device based on deep learning disclosed in an embodiment of the present invention;

[0086] Figure 4 This is a schematic diagram of another deep learning-based image processing device disclosed in an embodiment of the present invention;

[0087] Figure 5 This is a schematic diagram of the structure of another deep learning-based image processing device disclosed in an embodiment of the present invention. Detailed Implementation

[0088] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0089] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0090] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0091] This invention discloses a deep learning-based image processing method and apparatus, which can determine the similarity representation value of images by combining the hash value of the images, thereby improving the accuracy of the similarity representation value in representing image differences, improving the accuracy and reliability of determining whether images are similar, and thus improving the accuracy and reliability of image deduplication. Detailed descriptions follow.

[0092] Example 1

[0093] Please see Figure 1 , Figure 1 This is a flowchart illustrating an image processing method based on deep learning disclosed in an embodiment of the present invention. Wherein, Figure 1 The described deep learning-based image processing method can be applied to a deep learning-based image processing device. This device can include any one of an intelligent server or an intelligent platform for controlling the deep learning-based image processing device. The intelligent server can be a local server or a cloud server; this embodiment of the invention does not limit the scope. Figure 1 As shown, this deep learning-based image processing method can include the following operations:

[0094] 101. Input each image to be processed in the set of images to be processed into the image processing model to obtain the hash value corresponding to each image to be processed.

[0095] In this embodiment of the invention, optionally, the set of images to be processed includes at least two images to be processed. These images may be pre-processed images, and the pre-processing methods include, but are not limited to, uniform size, image enhancement, and image normalization. Optionally, the types of images to be processed in the set may be the same or different. For example, the images to be processed in the set may all be product images from the e-commerce field, or they may include various types of images such as portraits, landscapes, and product images. This invention does not impose any limitations.

[0096] In this embodiment of the invention, optionally, the image processing model can be a pre-trained deep learning type image processing model. Specifically, the image processing model can be designed and trained using convolutional neural networks or autoencoders, etc., and this invention does not limit it.

[0097] 102. Identify at least two images from the set of images to be processed as images to be compared.

[0098] In this embodiment of the invention, optionally, the images to be compared can be determined according to the type of the images, that is, images of the same type are determined as a group of images to be compared, or the images to be compared can be determined according to the image information during image preprocessing. This invention does not limit the scope of the invention.

[0099] 103. Based on the hash value corresponding to each image to be compared, for each image to be compared, calculate the similarity representation value between the image to be compared and each of the remaining images to be compared, except for the image to be compared, to obtain the set of similarity representation values ​​of the image to be compared.

[0100] In this embodiment of the invention, optionally, a similarity representation value can be calculated between a single image to be compared and multiple images to be compared excluding the image to be compared, or a similarity representation value can be calculated between a single image to be compared and a single image to be compared excluding the image to be compared. The similarity representation value can be used to represent the similarity between two images or to represent the similarity between multiple images. For example, three images to be compared and their hash values ​​are determined, which can be Figure A: [5c5c49cb49cb5c5c], Figure B: [53cea728d4e9ad92] and Figure C: [cb5c495ccb49cb49]. The similarity representation value between Figure A and Figure B can be calculated, or the similarity representation value between Figure A and Figure B and Figure C can be calculated. This invention does not limit the scope of the invention.

[0101] 104. For each image to be compared, if there is a target similarity value in the set of similarity representation values ​​of the image to be compared that is greater than the preset similarity threshold, it is determined that the image to be compared corresponding to the target similarity representation value meets the preset similarity condition.

[0102] In this embodiment of the invention, optionally, the preset similarity threshold can be determined based on the image deduplication requirements, or it can be determined by the image processing model based on historical image processing records, or it can be manually set; this invention does not impose any limitations. Optionally, the image to be compared corresponding to the target similarity representation value includes at least two images to be compared. That is, when there is a target similarity representation value in the set of similarity representation values ​​of a certain image to be compared that is greater than the preset similarity threshold, it is determined that the other images to be compared corresponding to the target similarity representation value, excluding the image to be compared, satisfy a preset similarity condition. The preset similarity condition can be used to indicate that the image to be compared is similar to the other images to be compared corresponding to the target similarity representation value, excluding the image to be compared, or it can be used to indicate that the image to be compared is not similar to the other images to be compared corresponding to the target similarity representation value, excluding the image to be compared; this invention does not impose any limitations.

[0103] It is evident that implementation Figure 1The described deep learning-based image processing method can input each image in the image set into the image processing model to obtain the hash value corresponding to each image. At least two images are selected from the image set as comparison images. Based on the hash value of each comparison image, for each comparison image, a similarity representation value is calculated between the comparison image and each of the remaining comparison images (excluding the comparison image itself), resulting in a set of similarity representation values ​​for that comparison image. This method can combine the image hash value to determine the image similarity representation value, improving the accuracy of the determined similarity representation values ​​in representing image differences. For each comparison image, when there is a target similarity representation value in the similarity representation value set that is greater than a preset similarity threshold, the comparison image corresponding to the target similarity representation value is determined to meet the preset similarity condition. This improves the accuracy and reliability of determining whether images are similar, thereby improving the accuracy and reliability of image deduplication.

[0104] In an optional embodiment, the deep learning-based image processing method may further include the following operations:

[0105] A predetermined set of training images is input into an image processing model to obtain the hash value of each training image in the training image set. The training image set includes at least one pair of samples and the sample label value corresponding to each pair of samples. All sample pairs included in the training image set include positive sample pairs and / or negative sample pairs. The sample label value of the positive sample pair is the positive sample label value, and the sample label value of the negative sample pair is the negative sample label value. Each sample pair includes at least two training images.

[0106] For each sample pair, the similarity representation value of the sample pair is calculated based on the hash value of each training image in the sample pair;

[0107] For each sample pair, calculate the loss corresponding to the sample pair based on the similarity representation value and the sample label value of the sample pair;

[0108] The parameters of the image processing model are updated based on the corresponding loss for each sample pair.

[0109] In this optional embodiment, the training image set may include at least one pair of samples and a sample label value corresponding to each pair. Each sample pair may contain at least two training images. The sample pairs in the training image set can be positive or negative pairs, and may include both simultaneously. The sample label value of a positive sample pair is a positive sample label value, and the sample label value of a negative sample pair is a negative sample label value. Optionally, when the training image in the positive sample pair satisfies a preset similarity condition, the training image in the negative sample pair is an image that does not satisfy the preset similarity condition; or, when the training image in the positive sample pair does not satisfy the preset similarity condition, the training image in the negative sample pair satisfies the preset similarity condition. Optionally, the positive sample label value may be defined as 1, and the negative sample label value may be defined as 0. Alternatively, when the training image in the sample pair satisfies the preset similarity condition, the sample label value of the sample pair may be defined as 1; when the training image in the sample pair does not satisfy the preset similarity condition, the sample label value of the sample pair may be defined as 0. This embodiment does not impose any limitations.

[0110] In this optional embodiment, the formula for calculating the loss for a sample pair can be:

[0111]

[0112] Where N can be used to represent the number of sample pairs in the training image set, y i It can be used to represent the sample label value of the i-th sample pair in the training image set, P(y i The similarity representation value of the i-th sample pair in the training image set can be used, but this embodiment does not limit it.

[0113] In this optional embodiment, the parameters of the image processing model can be updated using the backpropagation algorithm. Specifically, the loss represents the difference between the model's predicted value and the actual value. The gradient information of the model can be calculated by taking the partial derivative of the model's parameter weights or biases with respect to the calculated sample pairs and the chain rule. Then, each parameter of the model is updated based on the gradient information. This embodiment does not impose any limitations on this.

[0114] As can be seen, implementing this optional embodiment allows inputting a predetermined set of training images into an image processing model to obtain the hash value of each training image in the set. For each sample pair, a similarity representation value is calculated based on the hash value of each training image in the sample pair. For each sample pair, the loss corresponding to the sample pair is calculated based on the similarity representation value and the sample label value of the sample pair. The parameters of the image processing model are updated based on the loss corresponding to each sample pair. This allows for continuous training and optimization of the model, improving the accuracy of the hash values ​​calculated by the model and enhancing the representativeness of the hash values ​​for image differences. Consequently, the accuracy and reliability of image deduplication are improved.

[0115] In another optional embodiment, based on the hash value corresponding to each image to be compared, for each image to be compared, calculating the similarity representation value between the image to be compared and each of the remaining images to be compared (excluding the image to be compared) to obtain the set of similarity representation values ​​for the images to be compared may include the following operations:

[0116] Among all the images to be compared, a first image to be compared and a second image to be compared are determined, wherein the number of second images to be compared is at least one;

[0117] For each second image to be compared, the similarity representation value between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, thus obtaining the similarity representation value between the first image to be compared and the second image to be compared.

[0118] Based on the similarity representation values ​​between the first image to be compared and each second image to be compared, a set of similarity representation values ​​for the first image to be compared is generated.

[0119] Furthermore, after determining that the images to be compared corresponding to the target similarity representation value meet the preset similarity conditions, this deep learning-based image processing method may further include the following operations:

[0120] Based on the predetermined image deduplication conditions, the target duplicate image is identified and removed from all images to be compared that meet the similarity conditions;

[0121] The image deduplication conditions include at least one of the following: image storage time, image size, and image storage space usage.

[0122] In this optional embodiment, at least one second image to be compared is selected to calculate the corresponding similarity representation value with the first image to be compared. That is, the similarity representation value between a single image to be compared and multiple images to be compared other than the image to be compared can be calculated, or the similarity representation value between a single image to be compared and a single image to be compared other than the image to be compared can be calculated. The similarity representation value can be used to represent the similarity between two images or to represent the similarity between multiple images. This invention does not limit this.

[0123] In this optional embodiment, the pre-determined image deduplication conditions may optionally include at least one of image storage duration conditions, image size conditions, and image storage space occupancy conditions. These conditions can be determined based on manually set image deduplication requirements or based on historical image processing records. For example, when the image deduplication conditions include image storage duration conditions, one or more images with the longest storage duration can be selected from all images to be compared that meet the similarity conditions as the target duplicate image; or one or more images with the shortest storage duration can be selected from all images to be compared that meet the similarity conditions as the target duplicate image. When the image deduplication conditions include image size conditions, one or more images with the largest image size can be selected from all images to be compared that meet the similarity conditions as the target duplicate image. The image deduplication method can be either to select one or several images with the smallest image size from all images to be compared that meet the similarity criteria, or to select one or several images with an image size that meets a preset image size range from all images to be compared that meet the similarity criteria. When the image deduplication criteria include the image storage space occupancy criteria, the image deduplication method can be either to select one or several images with the largest image storage space occupancy from all images to be compared that meet the similarity criteria, or to select one or several images with the smallest image storage space occupancy from all images to be compared that meet the similarity criteria, or to select one or several images with an image storage space occupancy range that meets a preset image storage space occupancy criteria. This embodiment does not impose any limitations on this method.

[0124] As can be seen, implementing this optional embodiment can determine the first image to be compared and the second image to be compared among all images to be compared, wherein the number of second images to be compared is at least one. For each second image to be compared, a similarity representation value between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, thus obtaining the similarity representation value between the first image to be compared and the second image to be compared. Based on the similarity representation value between the first image to be compared and each second image to be compared, a set of similarity representation values ​​for the first image to be compared is generated. This can determine the similarity representation value between images based on the image's hash value, and can determine the similarity representation value between one-to-one corresponding images or one-to-many images, improving the efficiency and accuracy of determining the similarity representation value between images. Based on the predetermined image deduplication conditions, the target duplicate image is determined and removed from all images to be compared that meet the similarity conditions, improving the accuracy and reliability of removing images that meet the image deduplication conditions from all images to be compared that meet the similarity conditions. At the same time, the image deduplication conditions can be adaptively changed according to the deduplication requirements, improving the flexibility of the image deduplication operation.

[0125] In another optional embodiment, for each second image to be compared, calculating the similarity representation value between the first image to be compared and the second image to be compared based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and obtaining the similarity representation value between the first image to be compared and the second image to be compared may include the following operations:

[0126] For each second image to be compared, the cosine similarity between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared.

[0127] or,

[0128] For each second image to be compared, the Hamming distance between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared.

[0129] Similarity metrics include cosine similarity or Hamming distance.

[0130] In this optional embodiment, the formula for calculating the cosine similarity between the first image to be compared and the second image to be compared is optionally:

[0131]

[0132] Wherein, H1 represents the hash value of the first image to be compared, H2 represents the hash value of the second image to be compared, and the cosine similarity cosθ obtained by the formula takes the value range of [0,1]. When the cosine similarity cosθ approaches 1, it means that the first image to be compared and the second image to be compared are similar. When the cosine similarity cosθ approaches 0, it means that the first image to be compared and the second image to be compared are not similar. Optionally, the preset similarity threshold can be 0.8, that is, when the cosine similarity of the first image to be compared and the second image to be compared is greater than 0.8, it is determined that the first image to be compared and the second image to be compared are similar. This embodiment does not limit this.

[0133] In this optional embodiment, the Hamming distance can be used to represent the number of different characters in two strings. That is, it can be achieved by comparing the number of different characters in the hash values ​​of the first image to be compared and the hash values ​​of the second image to be compared. For example, the hash value of the first image to be compared can be converted into a string "01011100" and the hash value of the second image to be compared can be converted into a string "01001001". In this case, the number of different characters is 3, that is, the Hamming distance between the first image to be compared and the second image to be compared is 3. Optionally, the preset similarity threshold can be 7. That is, when the Hamming distance between the first image to be compared and the second image to be compared is less than 7, it is determined that the first image to be compared and the second image to be compared are similar. Furthermore, the size of the preset similarity threshold (i.e., the size of the Hamming distance) needs to be less than the number of bits in the hash value of the first image to be compared or the second image to be compared. This embodiment does not limit this.

[0134] As can be seen, implementing this optional embodiment can calculate the cosine similarity between the first image to be compared and the second image to be compared based on the hash values ​​corresponding to the first image to be compared and the second image to be compared, and use this as a similarity representation value between the first image to be compared and the second image to be compared; or it can calculate the Hamming distance between the first image to be compared and the second image to be compared based on the hash values ​​corresponding to the first image to be compared and the second image to be compared, and use this as a similarity representation value between the first image to be compared and the second image to be compared. This can represent the similarity representation value between images in a variety of different forms, thereby improving the accuracy and reliability of the determined similarity representation value.

[0135] Example 2

[0136] Please see Figure 2 , Figure 2 This is a flowchart illustrating an image processing method based on deep learning disclosed in an embodiment of the present invention. Wherein, Figure 2The described deep learning-based image processing method can be applied to a deep learning-based image processing device. This device can include any one of an intelligent server or an intelligent platform for controlling the deep learning-based image processing device. The intelligent server can be a local server or a cloud server; this embodiment of the invention does not limit the scope. Figure 2 As shown, this deep learning-based image processing method can include the following operations:

[0137] 201. Input each image to be processed in the set of images to be processed into the image processing model to obtain the first output and the second output corresponding to each image to be processed.

[0138] In this embodiment of the invention, optionally, the first output corresponding to the image to be processed may include a first feature vector corresponding to the image to be processed, and the second output corresponding to the image to be processed may include a second feature vector corresponding to the image to be processed. Optionally, the second output of the image to be processed may be used as a hash threshold of the first output of the image to be processed, which is not limited in this invention.

[0139] 202. Calculate the hash value corresponding to each image to be processed based on the first and second outputs corresponding to each image to be processed.

[0140] In this embodiment of the invention, optionally, the hash value corresponding to each image to be processed can be understood as an approximate hash matrix obtained by performing a binary approximation on the first and second outputs of the image to be processed. The whole process is differentiable, so that the gradient will not be nondifferentiable during model calculation and training, thereby improving the performance and stability of the model.

[0141] 203. Binarize the hash value corresponding to each image to be processed by using a preset data conversion threshold to obtain the binarized hash value corresponding to each image to be processed, which is used as the hash value corresponding to each image to be processed.

[0142] In this embodiment of the invention, optionally, the data conversion threshold can convert the hash value corresponding to the image to be processed from floating point to integer, which can significantly reduce the computational overhead and improve the testing speed with minimal loss of precision.

[0143] 204. Identify at least two images from the set of images to be processed as images to be compared.

[0144] 205. Based on the hash value corresponding to each image to be compared, for each image to be compared, calculate the similarity representation value between the image to be compared and each of the remaining images to be compared, except for the image to be compared, to obtain the set of similarity representation values ​​of the image to be compared.

[0145] 206. For each image to be compared, if there is a target similarity value in the set of similarity representation values ​​of the image to be compared that is greater than the preset similarity threshold, it is determined that the image to be compared corresponding to the target similarity representation value satisfies the preset similarity condition.

[0146] In this embodiment of the invention, for other descriptions of steps 204-206, please refer to the detailed description of steps 102-104 in Embodiment 1. These descriptions will not be repeated in this embodiment of the invention.

[0147] It is evident that implementation Figure 2 The described deep learning-based image processing method can input each image in a set of images to be processed into an image processing model, obtaining a first output and a second output corresponding to each image. Based on the first and second outputs, a hash value is calculated for each image. This method improves the accuracy and reliability of the determined hash values ​​by using the first and second outputs, and also makes the hash values ​​more realistically reflect the image details. At least two images are selected from the set of images to be processed as comparison images. Based on the hash value of each comparison image, for each... For each image to be compared, the similarity representation value between the image to be compared and each of the remaining images to be compared (excluding the image to be compared) is calculated, resulting in a set of similarity representation values ​​for the image to be compared. This can be combined with the image's hash value to determine the image's similarity representation value, improving the accuracy of the determined similarity representation values ​​in reflecting image differences. For each image to be compared, if there is a target similarity representation value in the set of similarity representation values ​​that is greater than a preset similarity threshold, the image to be compared corresponding to the target similarity representation value is determined to meet the preset similarity condition. This can improve the accuracy and reliability of determining whether images are similar, thereby improving the accuracy and reliability of image deduplication.

[0148] In an optional embodiment, the first output corresponding to each image to be processed includes a first feature vector corresponding to the image to be processed, and the second output corresponding to each image to be processed includes a second feature vector corresponding to the image to be processed.

[0149] The image processing model is a deep learning-based image processing model, and it includes a feature mapping layer, a bidirectional LSTM layer, and a fully connected layer.

[0150] The feature mapping layer is used to map the feature vector corresponding to each image to be processed to a preset first dimension. The bidirectional LSTM layer is used to capture the context feature information corresponding to each image to be processed. The number of fully connected layers is at least two. At least one fully connected layer is used to reduce the dimensionality of the context feature information to a preset second dimension to obtain and output the first output. At least one fully connected layer is used to output the second output.

[0151] In this optional embodiment, the first output corresponding to the image to be processed may include a first feature vector corresponding to the image to be processed, and the second output corresponding to the image to be processed may include a second feature vector corresponding to the image to be processed. Optionally, the second output of the image to be processed may be used as a hash threshold of the first output of the image to be processed, which is not limited in this invention.

[0152] In this optional embodiment, taking a convolutional neural network (CNN) model as an example, a Res-Net (Residual Neural Network) or VGG (Visual Geometry Group) model can be used as the base model. Then, the last classification layer is removed, and a feature mapping layer is added to map the dimension of the feature vector corresponding to the image to be processed to a preset first dimension, such as 256 dimensions. Optionally, at least one bidirectional LSTM (Long Short-Term Memory) layer, preferably two bidirectional LSTM layers, is included to capture global contextual information and output the features of the preset first dimension. Optionally, at least two fully connected layers are included, one of which is used to fuse the global contextual information captured by the bidirectional LSTM layer and reduce its dimension to a preset second dimension, such as 128 dimensions, to obtain and output the first output of the preset second dimension, and the other layer is used to output the second output of the preset second dimension. This embodiment is not limited.

[0153] As can be seen, implementing this optional embodiment can, by setting the various layers of the image processing model accordingly and using a bidirectional LSTM layer to capture global contextual information, enable the two feature vectors of the final fused and output image to more accurately reflect the image details and represent the global information of the image. Using two branches to output the first output and the second output respectively, each feature value of the first output has a separate threshold, which is beneficial to improving the hash value representation ability of the image to be processed in subsequent calculations, and also makes the gradient differentiable in the entire training process, thereby improving the stability and reliability of the model.

[0154] In another optional embodiment, calculating the hash value corresponding to each image to be processed based on the first output and the second output corresponding to each image to be processed may include the following operations:

[0155] Based on the first and second outputs corresponding to each image to be processed, and combined with the predetermined hash value calculation formula, calculate the hash value corresponding to each image to be processed.

[0156] The formula for calculating the hash value of the i-th image in each image to be processed is as follows:

[0157]

[0158] Among them, H i This represents the hash value corresponding to the i-th image to be processed, k represents the gradient amplification factor, and F1 represents the hash value. i F2 represents the first output corresponding to the i-th image to be processed. i This represents the second output corresponding to the i-th image to be processed.

[0159] In this optional embodiment, the gradient information amplification coefficient k can be adjusted according to the training data or manually. Preferably, the value of k can be set to 60, but this embodiment does not limit it.

[0160] As can be seen, implementing this optional embodiment can calculate the hash value corresponding to each image to be processed based on the first and second outputs corresponding to each image to be processed, combined with the predetermined hash value calculation formula, and obtain an approximate hash matrix. The whole process is differentiable, so that the gradient will not be non-differentiable during model calculation and training, thereby improving the performance and stability of the model.

[0161] Example 3

[0162] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an image processing device based on deep learning disclosed in an embodiment of the present invention. Figure 3 The described deep learning-based image processing apparatus may include any one of an intelligent server or an intelligent platform for controlling the deep learning-based image processing apparatus, wherein the intelligent server includes a local server or a cloud server, and the embodiments of the present invention are not limited thereto. Figure 3 As shown, the deep learning-based image processing device may include:

[0163] The first processing module 301 is used to input each image to be processed in the image processing model to obtain the hash value corresponding to each image to be processed. The image processing set includes at least two images to be processed.

[0164] The determining module 302 is used to determine at least two images from the set of images to be processed as images to be compared;

[0165] The calculation module 303 is used to calculate the similarity representation value between each image to be compared and each of the remaining images to be compared, excluding the image to be compared, based on the hash value corresponding to each image to be compared, so as to obtain the set of similarity representation values ​​of the image to be compared.

[0166] The determining module 302 is further configured to, for each image to be compared, determine that the image to be compared corresponding to the target similarity value satisfies the preset similarity condition when there is a target similarity value in the set of similarity representation values ​​of the image to be compared that is greater than the preset similarity threshold.

[0167] It is evident that implementation Figure 3 The described deep learning-based image processing apparatus can input each image in a set of images to be processed into an image processing model to obtain a hash value corresponding to each image. It determines at least two images in the set as comparison images. Based on the hash value of each comparison image, it calculates a similarity representation value between the comparison image and each of the remaining comparison images (excluding the comparison image itself) to obtain a set of similarity representation values ​​for the comparison image. This apparatus can combine the hash values ​​of the images to determine their similarity representation values, improving the accuracy of the determined similarity representation values ​​in representing image differences. For each comparison image, when there is a target similarity representation value in the set of similarity representation values ​​that is greater than a preset similarity threshold, it determines that the comparison image corresponding to the target similarity representation value meets the preset similarity condition. This improves the accuracy and reliability of determining whether images are similar, thereby improving the accuracy and reliability of image deduplication.

[0168] In an optional embodiment, such as Figure 4 As shown, the first processing module 301 inputs each image to be processed from the set of images to be processed into the image processing model, and obtains the hash value corresponding to each image to be processed in the following specific ways:

[0169] Each image in the set of images to be processed is input into the image processing model to obtain the first output and the second output corresponding to each image to be processed.

[0170] Calculate the hash value corresponding to each image to be processed based on the first and second outputs corresponding to each image to be processed.

[0171] Furthermore, the deep learning-based image processing apparatus may also include:

[0172] The second processing module 304 is used to perform binarization processing on the hash value corresponding to each image to be processed by a preset data conversion threshold, so as to obtain the binarized hash value corresponding to each image to be processed, which is used as the hash value corresponding to each image to be processed.

[0173] It is evident that implementation Figure 4 The described deep learning-based image processing apparatus can input each image in a set of images to be processed into an image processing model, obtaining a first output and a second output corresponding to each image. Based on the first and second outputs, it calculates a hash value for each image. This ability to calculate the hash value of an image using its first and second outputs improves the accuracy and reliability of the determined hash value, while also ensuring that the hash value more accurately reflects the image details. The apparatus identifies at least two images in the set as comparison images, and calculates the hash value for each image based on its hash value. For each image to be compared, the similarity representation value between the image to be compared and each of the remaining images to be compared (excluding the image to be compared) is calculated, resulting in a set of similarity representation values ​​for the image to be compared. This can be combined with the image's hash value to determine the image's similarity representation value, improving the accuracy of the determined similarity representation values ​​in reflecting image differences. For each image to be compared, if there is a target similarity representation value in the set of similarity representation values ​​that is greater than a preset similarity threshold, the image to be compared corresponding to the target similarity representation value is determined to meet the preset similarity condition. This can improve the accuracy and reliability of determining whether images are similar, thereby improving the accuracy and reliability of image deduplication.

[0174] In another alternative embodiment, such as Figure 4 As shown, the first output corresponding to each image to be processed includes the first feature vector corresponding to the image to be processed, and the second output corresponding to each image to be processed includes the second feature vector corresponding to the image to be processed.

[0175] The image processing model is a deep learning-based image processing model, and it includes a feature mapping layer, a bidirectional LSTM layer, and a fully connected layer.

[0176] The feature mapping layer is used to map the feature vector corresponding to each image to be processed to a preset first dimension. The bidirectional LSTM layer is used to capture the context feature information corresponding to each image to be processed. The number of fully connected layers is at least two. At least one fully connected layer is used to reduce the dimensionality of the context feature information to a preset second dimension to obtain and output the first output. At least one fully connected layer is used to output the second output.

[0177] It is evident that implementation Figure 4 The described deep learning-based image processing device can capture global contextual information by setting the various layers of the image processing model accordingly. This allows the two feature vectors of the final fused and output image to more accurately reflect the image details and represent the global information of the image. By using two branches to output the first and second outputs respectively, each feature value of the first output has a separate threshold, which is beneficial to improving the hash value representation ability of the image to be processed in subsequent calculations and also makes the gradient differentiable throughout the training process, thereby improving the stability and reliability of the model.

[0178] In yet another alternative embodiment, such as Figure 4 As shown, the specific method by which the first processing module 301 calculates the hash value corresponding to each image to be processed based on the first output and the second output corresponding to each image to be processed includes:

[0179] Based on the first and second outputs corresponding to each image to be processed, and combined with the predetermined hash value calculation formula, calculate the hash value corresponding to each image to be processed.

[0180] The formula for calculating the hash value of the i-th image in each image to be processed is as follows:

[0181]

[0182] Among them, H i This represents the hash value corresponding to the i-th image to be processed, k represents the gradient amplification factor, and F1 represents the hash value. i F2 represents the first output corresponding to the i-th image to be processed. i This represents the second output corresponding to the i-th image to be processed.

[0183] It is evident that implementation Figure 4 The described deep learning-based image processing device can calculate the hash value corresponding to each image to be processed based on the first and second outputs corresponding to each image to be processed, combined with a predetermined hash value calculation formula, and obtain an approximate hash matrix. The whole process is differentiable, so that the gradient will not be non-differentiable during model calculation and training, thereby improving the performance and stability of the model.

[0184] In yet another alternative embodiment, such as Figure 4As shown, the first processing module 301 is further configured to input a predetermined set of training images into an image processing model to obtain the hash value of each training image in the set of training images. The set of training images includes at least one pair of samples and the sample label value corresponding to each pair of samples. All sample pairs included in the set of training images include positive sample pairs and / or negative sample pairs. The sample label value of the positive sample pair is the positive sample label value, and the sample label value of the negative sample pair is the negative sample label value. Each sample pair includes at least two training images.

[0185] The calculation module 303 is further configured to, for each sample pair, calculate the similarity representation value of the sample pair based on the hash value of each training image in the sample pair; and for each sample pair, calculate the loss corresponding to the sample pair based on the similarity representation value of the sample pair and the sample label value of the sample pair.

[0186] The deep learning-based image processing device may also include:

[0187] The update module 305 is used to update the parameters of the image processing model based on the loss corresponding to each sample pair.

[0188] It is evident that implementation Figure 4 The described deep learning-based image processing device can input a pre-determined set of training images into an image processing model to obtain the hash value of each training image in the set. For each sample pair, a similarity representation value is calculated based on the hash value of each training image in the sample pair. For each sample pair, a loss is calculated based on the similarity representation value and the sample label value of the sample pair. The parameters of the image processing model are updated based on the loss of each sample pair. This allows for continuous training and optimization of the model, improving the accuracy of the hash values ​​calculated by the model and enhancing the representativeness of the hash values ​​for image differences. Consequently, the accuracy and reliability of image deduplication are improved.

[0189] In yet another alternative embodiment, such as Figure 4 As shown, the calculation module 303 calculates the similarity representation value between each image to be compared and every remaining image to be compared among all images to be compared, based on the hash value corresponding to each image to be compared, and obtains the set of similarity representation values ​​for the image to be compared in the following specific ways:

[0190] Among all the images to be compared, a first image to be compared and a second image to be compared are determined, wherein the number of second images to be compared is at least one;

[0191] For each second image to be compared, the similarity representation value between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, thus obtaining the similarity representation value between the first image to be compared and the second image to be compared.

[0192] Based on the similarity representation values ​​between the first image to be compared and each second image to be compared, a set of similarity representation values ​​for the first image to be compared is generated.

[0193] Furthermore, the deep learning-based image processing apparatus may also include:

[0194] The removal module 306 is used to determine and remove the target duplicate image from all the images to be compared that meet the similarity conditions after the determination module 302 determines that the image to be compared corresponding to the target similarity representation value meets the preset similarity conditions.

[0195] The image deduplication conditions include at least one of the following: image storage time, image size, and image storage space usage.

[0196] It is evident that implementation Figure 4 The described deep learning-based image processing apparatus can determine a first image to be compared and a second image to be compared from all images to be compared, wherein the number of second images to be compared is at least one. For each second image to be compared, a similarity representation value between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, thereby obtaining the similarity representation value between the first image to be compared and the second image to be compared. Based on the similarity representation value between the first image to be compared and each second image to be compared, a set of similarity representation values ​​for the first image to be compared is generated. It can determine the similarity representation value between images based on the hash value of the images, and can determine the similarity representation value between one-to-one corresponding images or one-to-many images, thereby improving the efficiency and accuracy of determining the similarity representation value between images. Based on the predetermined image deduplication conditions, it identifies and removes target duplicate images from all images to be compared that meet the similarity conditions, thereby improving the accuracy and reliability of removing images that meet the image deduplication conditions from all images to be compared that meet the similarity conditions. At the same time, the image deduplication conditions can be adaptively changed according to the deduplication requirements, thereby improving the flexibility of the image deduplication operation.

[0197] In yet another alternative embodiment, such as Figure 4As shown, for each second image to be compared, the calculation module 303 calculates the similarity representation value between the first image to be compared and the second image to be compared based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared. The specific method for obtaining the similarity representation value between the first image to be compared and the second image to be compared includes:

[0198] For each second image to be compared, the cosine similarity between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared.

[0199] or,

[0200] For each second image to be compared, the Hamming distance between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared.

[0201] Similarity metrics include cosine similarity or Hamming distance.

[0202] It is evident that implementation Figure 4 The described deep learning-based image processing apparatus can calculate the cosine similarity between the first and second images to be compared based on the hash values ​​corresponding to the first and second images to be compared, and use this as a similarity representation value between the two images. Alternatively, it can calculate the Hamming distance between the first and second images to be compared based on the hash values ​​corresponding to the first and second images to be compared, and use this as a similarity representation value between the two images. This allows the similarity representation value between images to be expressed in various different forms, thereby improving the accuracy and reliability of the determined similarity representation value.

[0203] Example 4

[0204] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of another image processing device based on deep learning disclosed in an embodiment of the present invention. For example... Figure 5 As shown, the deep learning-based image processing device may include:

[0205] Memory 401 storing executable program code;

[0206] Processor 402 coupled to memory 401;

[0207] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the deep learning-based image processing method described in Embodiment 1 or Embodiment 2 of the present invention.

[0208] Example 5

[0209] This invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute the steps in the deep learning-based image processing method described in Embodiment 1 or Embodiment 2 of this invention.

[0210] Example 6

[0211] This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the deep learning-based image processing method described in Embodiment 1 or Embodiment 2.

[0212] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0213] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0214] Finally, it should be noted that the image processing method and apparatus based on deep learning disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based image processing method, characterized in that, The method includes: Each image to be processed in the set of images to be processed is input into the image processing model to obtain the hash value corresponding to each image to be processed. The set of images to be processed includes at least two images to be processed. At least two images are selected from the set of images to be processed as images to be compared. Based on the hash value corresponding to each of the images to be compared, for each image to be compared, calculate the similarity representation value between the image to be compared and each of the remaining images to be compared among all the images to be compared, and obtain the set of similarity representation values ​​of the image to be compared. For each image to be compared, if there is a target similarity value in the set of similarity representation values ​​of the image to be compared that is greater than a preset similarity threshold, it is determined that the image to be compared corresponding to the target similarity representation value satisfies the preset similarity condition. And, the step of inputting each image to be processed in the set of images to be processed into the image processing model to obtain the hash value corresponding to each image to be processed includes: Each image to be processed in the set of images to be processed is input into the image processing model to obtain a first output and a second output corresponding to each image to be processed; Calculate the hash value corresponding to each image to be processed based on the first and second outputs corresponding to each image to be processed. Furthermore, the first output corresponding to each of the images to be processed includes a first feature vector corresponding to the image to be processed, and the second output corresponding to each of the images to be processed includes a second feature vector corresponding to the image to be processed; The image processing model is a deep learning-type image processing model, and the image processing model includes a feature mapping layer, a bidirectional LSTM layer, and a fully connected layer; The feature mapping layer is used to map the feature vector corresponding to each image to be processed to a preset first dimension. The bidirectional LSTM layer is used to capture the context feature information corresponding to each image to be processed. The number of fully connected layers is at least two. At least one of the fully connected layers is used to reduce the dimensionality of the context feature information to a preset second dimension to obtain and output the first output. At least one of the fully connected layers is used to output the second output. And, the step of calculating the hash value corresponding to each of the images to be processed based on the first output and the second output corresponding to each image to be processed includes: Based on the first and second outputs corresponding to each image to be processed, and combined with the predetermined hash value calculation formula, calculate the hash value corresponding to each image to be processed. The formula for calculating the hash value of the i-th image in each image to be processed is as follows: ； in, This represents the hash value corresponding to the i-th image to be processed, and k represents the gradient amplification factor. This represents the first output corresponding to the i-th image to be processed. This represents the second output corresponding to the i-th image to be processed.

2. The image processing method based on deep learning according to claim 1, characterized in that, The method further includes: The hash value corresponding to each image to be processed is binarized by a preset data conversion threshold to obtain the binarized hash value corresponding to each image to be processed, which is used as the hash value corresponding to each image to be processed.

3. The deep learning-based image processing method according to claim 1 or 2, characterized in that, The method further includes: A predetermined set of training images is input into the image processing model to obtain the hash value of each training image in the set. The set of training images includes at least one pair of samples and a sample label value corresponding to each pair. All the sample pairs included in the set of training images include positive sample pairs and / or negative sample pairs. The sample label value of the positive sample pair is a positive sample label value, and the sample label value of the negative sample pair is a negative sample label value. Each sample pair includes at least two training images. For each of the sample pairs, a similarity representation value for the sample pair is calculated based on the hash value of each of the training images in the sample pair; For each sample pair, the loss corresponding to the sample pair is calculated based on the similarity representation value and the sample label value of the sample pair; The parameters of the image processing model are updated based on the loss corresponding to each sample pair.

4. The deep learning-based image processing method according to claim 1 or 2, characterized in that, Based on the hash value corresponding to each of the images to be compared, for each image to be compared, a similarity representation value is calculated between the image to be compared and each of the remaining images to be compared (excluding the image to be compared) to obtain a set of similarity representation values ​​for the images to be compared, including: Among all the images to be compared, a first image to be compared and a second image to be compared are determined, wherein the number of the second image to be compared is at least one; For each of the second images to be compared, a similarity representation value between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, thus obtaining the similarity representation value between the first image to be compared and the second image to be compared. Based on the similarity representation values ​​between the first image to be compared and each of the second images to be compared, a set of similarity representation values ​​for the first image to be compared is generated. And, after determining that the image to be compared corresponding to the target similarity representation value meets the preset similarity condition, the method further includes: Based on the predetermined image deduplication conditions, the target duplicate image is determined and removed from all the images to be compared that satisfy the similarity conditions; The image deduplication conditions include at least one of the following: image storage time, image size, and image storage space usage.

5. The image processing method based on deep learning according to claim 4, characterized in that, For each second image to be compared, based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, a similarity representation value between the first image to be compared and the second image to be compared is calculated to obtain the similarity representation value between the first image to be compared and the second image to be compared, including: For each second image to be compared, the cosine similarity between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared. or, For each of the second images to be compared, the Hamming distance between the first image to be compared and the second image to be compared is calculated based on the hash value corresponding to the first image to be compared and the hash value corresponding to the second image to be compared, and is used as the similarity representation value between the first image to be compared and the second image to be compared. The similarity representation value includes the cosine similarity or the Hamming distance.

6. An image processing device based on deep learning, characterized in that, The device includes: The first processing module is used to input each image to be processed in the image processing model to obtain a hash value corresponding to each image to be processed. The image processing set includes at least two images to be processed. The determining module is used to determine at least two images from the set of images to be processed as images to be compared; The calculation module is used to calculate, based on the hash value corresponding to each of the images to be compared, the similarity representation value between each image to be compared and each of the remaining images to be compared (excluding the image to be compared) among all the images to be compared, to obtain a set of similarity representation values ​​for the images to be compared. The determining module is further configured to, for each image to be compared, determine that the image to be compared corresponding to the target similarity value satisfies the preset similarity condition when there is a target similarity value in the set of similarity representation values ​​of the image to be compared that is greater than the preset similarity threshold. Furthermore, the method by which the first processing module inputs each image to be processed from the set of images to be processed into the image processing model to obtain the hash value corresponding to each image to be processed specifically includes: Each image to be processed in the set of images to be processed is input into the image processing model to obtain a first output and a second output corresponding to each image to be processed; Calculate the hash value corresponding to each image to be processed based on the first and second outputs corresponding to each image to be processed. Furthermore, the first output corresponding to each of the images to be processed includes a first feature vector corresponding to the image to be processed, and the second output corresponding to each of the images to be processed includes a second feature vector corresponding to the image to be processed; The image processing model is a deep learning-type image processing model, and the image processing model includes a feature mapping layer, a bidirectional LSTM layer, and a fully connected layer; The feature mapping layer is used to map the feature vector corresponding to each image to be processed to a preset first dimension. The bidirectional LSTM layer is used to capture the context feature information corresponding to each image to be processed. The number of fully connected layers is at least two. At least one of the fully connected layers is used to reduce the dimensionality of the context feature information to a preset second dimension to obtain and output the first output. At least one of the fully connected layers is used to output the second output. Furthermore, the method by which the first processing module calculates the hash value corresponding to each image to be processed based on the first output and the second output corresponding to each image to be processed specifically includes: Based on the first and second outputs corresponding to each image to be processed, and combined with the predetermined hash value calculation formula, calculate the hash value corresponding to each image to be processed. The formula for calculating the hash value of the i-th image in each image to be processed is as follows: ； in, This represents the hash value corresponding to the i-th image to be processed, and k represents the gradient amplification factor. This represents the first output corresponding to the i-th image to be processed. This represents the second output corresponding to the i-th image to be processed.

7. An image processing device based on deep learning, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the deep learning-based image processing method as described in any one of claims 1-5.

8. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the deep learning-based image processing method as described in any one of claims 1-5.

Images