A method and system for image duplicate checking based on deep learning

Abstract

The present invention discloses a method and system for image duplicate checking based on deep learning, including: obtaining a to-be-checked image, extracting the overall image feature vector of the to-be-checked image, retrieving several images with the closest distance in the query image library and judging one by one; if the retrieved feature distance is less than a set threshold one, it is directly determined to be the same; if it is greater than threshold two, it is directly judged to be different; if the feature distance is greater than threshold one and less than threshold two, the local feature vector of the to-be-checked image is extracted and the feature distance value from the retrieved local image is calculated, and the overall image feature distance is recalculated based on the calculated local distance value; if the finally calculated distance value is less than threshold one, it is judged to be the same, otherwise it is judged to be different. Advantages: The present invention can avoid the reduction of similarity caused by watermarks and image editing, and at the same time does not reduce the query speed.

Description

Technical Field

[0001] The present invention relates to a method and system for picture duplicate checking based on deep learning, belonging to the technical field of computer vision. Background Art

[0002] SFA (Sales Force Automation) is an important business component of the CRM customer relationship management system. Through a series of functions such as visit target setting, route planning, setting execution specifications, task execution, and execution result analysis, SFA standardizes and guides the on-site behavior of business personnel, helping business personnel correctly and efficiently complete the specified visit steps.

[0003] With the development of artificial intelligence, the photos submitted during visits can already be used to automatically identify products in photos such as shelves and refrigerators through object detection technology, helping enterprises conduct further display detection, stock placement rate statistics, etc. The visit requires business personnel to submit real photos, but in the actual visit process, business personnel will reuse previously taken photos to avoid making real visits to the business site. When the photo is submitted, the business system will automatically add watermark information such as location, time, and customer information. At the same time, business personnel will edit the existing photo to add noise to avoid being discovered as reused. In this way, the current method of querying and comparing by extracting the feature values of the entire picture will cause missed detection of the retrieved pictures. The present invention proposes a method and device for picture duplicate checking based on deep learning, which can effectively reduce the influence of watermark information or noise on the retrieval result, thereby improving the success rate of picture duplicate checking and ensuring the authenticity of business data. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a method and system for picture duplicate checking based on deep learning.

[0005] To solve the above technical problem, the present invention provides a method for picture duplicate checking based on deep learning, including:

[0006] Obtain the picture to be queried, extract the whole-picture feature vector of the picture to be queried, compare it with the whole-picture feature vectors in the pre-constructed query picture library, and retrieve several detected pictures with the lowest distance of the whole-picture feature vector from the query picture library;

[0007] For the detected pictures, judge according to the ascending order of the distance of the whole-picture feature vector. If the distance of the whole-picture feature vector is lower than threshold one, it is judged as the same picture; if it is greater than threshold two, it is judged as a non-same picture, where threshold two is greater than threshold one;

[0008] If the eigenvector distance of the whole image is between the first threshold and the second threshold, the image to be queried is divided into several local images, the eigenvectors of each local image are extracted, and the eigenvector distances of each local image are calculated one by one with the eigenvectors of the local images of the detected image, and the eigenvector distance of the whole image is calculated by combining according to the eigenvector distances of each local image. If it is less than the first threshold, it is judged as the same image, otherwise it is judged as a non - same image.

[0009] Further, the determination of the eigenvector of the whole image in the query image library includes:

[0010] A query image library is established to store the query images judged as non - same images after each query into the query image library. The eigenvector of the whole image of each image and the eigenvector of the local image of each image in the query image library are extracted by using a deep neural network and stored in the eigenvector library.

[0011] Further, the eigenvector distance between two images is calculated by using the cosine distance or the Euclidean distance calculation formula.

[0012] Further, the number of several images retrieved from the query image library with the lowest eigenvector distance from the eigenvector of the image to be queried is not less than 3.

[0013] Further, the number of local images into which the image to be queried is divided is not less than 4.

[0014] Further, the formula for calculating the eigenvector distance D of the whole image by combining according to the eigenvector distances of each local image is:

[0015] Δd i = D0 - d i

[0016]

[0017]

[0018]

[0019]

[0020] Where, Δd i represents the difference between the eigenvector distance d i of the local image and the eigenvector distance D0 of the whole image. D0 represents the eigenvector distance between the image to be queried and the whole image of the detected image, d i represents the eigenvector distance between the image to be queried and the corresponding local image of the detected image, n represents the number of partitions of the local images between the image to be queried and the detected image, and μ represents the difference Δd between the eigenvector distances of n local images and the eigenvector distance of the image to be queriedi The average, where σ represents the difference Δd between the eigenvector distances of n local images and the eigenvector distance of the image to be queried i The standard deviation, and f(Δd i ) represents the difference Δd between the eigenvector distances of the local image and the eigenvector distance of the image to be queried i The weight

[0021] A deep learning-based image duplicate checking system, comprising:

[0022] An extraction module, configured to obtain an image to be queried, extract the eigenvector of the entire image of the image to be queried, compare it with the eigenvectors of the entire images in a pre-constructed query image library, and retrieve several detected images with the lowest eigenvector distance from the entire image of the image to be queried from the query image library;

[0023] A first judgment module, configured to judge the detected images in ascending order of the eigenvector distance of the entire image. If the eigenvector distance of the entire image is lower than a first threshold, it is judged as the same image; if it is greater than a second threshold, it is judged as a non-same image, where the second threshold is greater than the first threshold;

[0024] A second judgment module, configured to, if the eigenvector distance of the entire image is between the first threshold and the second threshold, divide the image to be queried into several local images, extract the eigenvector of each local image, calculate the eigenvector distance of each local image one by one with the eigenvector of the local image of the detected image, and calculate the eigenvector distance of the entire image based on the eigenvector distances of each local image. If it is less than the first threshold, it is judged as the same image; otherwise, it is judged as a non-same image

[0025] A computer-readable storage medium storing one or more programs, where the one or more programs include instructions that, when executed by a computing device, cause the computing device to execute any of the methods in the above method

[0026] A computing device, comprising

[0027] One or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods in the above method

[0028] The beneficial effects achieved by the present invention: The present invention provides a method and device for image duplicate checking based on deep learning, which can effectively reduce the influence of watermark information or noise on the retrieval result, thereby improving the success rate of image duplicate checking and ensuring the authenticity of business data Description of the Drawings

[0029] Figure 1It is the process schematic diagram of the present invention. Specific Embodiments

[0030] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.

[0031] As Figure 1 shown, a method for picture duplicate checking based on deep learning includes:

[0032] Obtain the picture to be queried, extract the whole-picture feature vector of the picture to be queried, compare it with the whole-picture feature vectors in the pre-constructed query picture library, and retrieve several pictures with the lowest distance of the whole-picture feature vector from the query picture library;

[0033] For the retrieved pictures, judge them in ascending order of the distance of the whole-picture feature vector. If the distance of the whole-picture feature vector is lower than threshold one, it is judged as the same picture; if it is greater than threshold two, it is judged as a non-same picture, where threshold two is greater than threshold one; by making the overall feature vector judgment, it is possible to quickly judge obvious same pictures or non-same pictures.

[0034] If the distance of the whole-picture feature vector is between threshold one and threshold two, divide the picture to be queried into several local pictures, extract the local-picture feature vectors of each local picture, and calculate them one by one with the local-picture feature vectors in the pre-constructed query picture library to obtain the minimum feature vector distance of each local picture. Combine and calculate the minimum feature vector distances of each local picture as the distance of the whole-picture feature vector. If it is less than threshold one, it is judged as the same picture; if it is not less than threshold one, it is judged as a non-same picture. By making the local feature vector judgment, it is possible to further determine pictures that cannot be clearly determined as the same or non-same, and improve the recognition accuracy.

[0035] The determination of the whole-picture feature vectors in the query picture library includes:

[0036] Establish a query picture library for storing the query pictures judged as non-same pictures after each query into the query picture library. Use deep neural networks such as VGG16 and ResNet to extract the whole-picture feature vectors of each picture in the query picture library and the local-picture feature vectors of each local picture of each picture, and store them in the feature vector library.

[0037] The distance between the feature vectors of two pictures is obtained by using distance calculation formulas such as cosine distance.

[0038] The number of several pictures retrieved from the query picture library with the lowest distance of the whole-picture feature vector from the picture to be queried is not less than 3 pictures, preferably 5 pictures.

[0039] The number of local images obtained by dividing the image to be queried is not less than 4, preferably 4.

[0040] The formula for calculating the overall image feature vector distance by combining the minimum feature vector distances of each local image is:

[0041] Δd i = D0 - d i

[0042]

[0043]

[0044]

[0045]

[0046] D0: Feature vector distance between the image to be retrieved and the retrieved image

[0047] d i : Feature vector distance between the local image corresponding to the image to be retrieved and the retrieved image

[0048] Δd i : Represents the difference between the feature vector distance d of the local image i and the feature vector distance D0 of the overall image

[0049] n: Number of partitions of the local images of the image to be retrieved and the retrieved image

[0050] μ: Average of the difference between the feature vector distance of the local image and the feature vector distance of the original image to be retrieved

[0051] σ: Standard deviation of the difference between the feature vector distance of the local image and the feature vector distance of the original image to be retrieved

[0052] f(Δd i ): Weight of the difference between the feature vector distance of the local image and the feature vector distance of the original image to be retrieved

[0053] D: Feature vector distance between the image to be queried and the retrieved image obtained by combined calculation.

[0054] Correspondingly, the present invention further provides a system for image duplicate checking based on deep learning, including:

[0055] An extraction module, configured to obtain the image to be queried, extract the overall image feature vector of the image to be queried, compare it with the overall image feature vectors in the pre-constructed query image library, and retrieve several retrieved images with the lowest feature vector distance from the overall image of the image to be queried from the query image library;

[0056] The first judgment module is used to judge the detected pictures in ascending order of the feature vector distance of the whole picture. If the feature vector distance of the whole picture is lower than threshold one, it is judged as the same picture; if it is greater than threshold two, it is judged as a non - same picture, where threshold two is greater than threshold one.

[0057] The second judgment module is used to, if the feature vector distance of the whole picture is between threshold one and threshold two, divide the picture to be queried into several local pictures, extract the feature vectors of each local picture, calculate the feature vector distance between each local picture and the local picture feature vectors of the detected picture one by one, and calculate the feature vector distance of the whole picture based on the combined calculation of the feature vector distances of each local picture. If it is less than threshold one, it is judged as the same picture; otherwise, it is judged as a non - same picture.

[0058] Correspondingly, the present invention also provides a computer - readable storage medium storing one or more programs, where the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device is caused to execute any one of the methods in the above - mentioned method.

[0059] Correspondingly, the present invention also provides a computing device, which is characterized by including

[0060] One or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods in the above - mentioned method.

[0061] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer - usable storage media (including but not limited to disk memories, CD - ROMs, optical memories, etc.) containing computer - usable program codes.

[0062] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general - purpose computers, special - purpose computers, embedded processors, or other programmable data - processing devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data - processing devices generate for implementing in the process Figure 1 a process or multiple processes and / or blocks Figure 1a device for the functions specified in one or more boxes.

[0063] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device that implements the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 a box or more boxes.

[0064] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide for implementing the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 a box or more boxes.

[0065] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for image duplicate checking based on deep learning, characterized in that, Including: Obtain the picture to be queried, extract the overall picture feature vector of the picture to be queried, compare it with the overall picture feature vectors in the pre-constructed query picture library, and retrieve several detected pictures with the lowest feature vector distance from the overall picture of the picture to be queried from the query picture library; For the detected pictures, judge them in ascending order of the feature vector distance of the overall picture. If the feature vector distance of the overall picture is lower than threshold one, it is judged as the same picture. If it is greater than threshold two, it is judged as a non-same picture, where threshold two is greater than threshold one; If the feature vector distance of the overall picture is between threshold one and threshold two, divide the picture to be queried into several local pictures, extract the feature vectors of each local picture, and calculate the feature vector distance of each local picture with the feature vectors of the local pictures of the detected picture one by one, and calculate the feature vector distance of the overall picture by combining according to the feature vector distances of each local picture. If it is less than threshold one, it is judged as the same picture, otherwise it is judged as a non-same picture; The formula for calculating the feature vector distance of the overall picture by combining according to the feature vector distances of each local picture is: ； ； ； ； ； Among them, D represents the feature vector distance of the whole image, represents the feature vector distance of the local image and the difference between the feature vector distance of the whole image . represents the feature vector distance between the image to be queried and the whole image of the detected image, represents the feature vector distance between the image to be queried and the corresponding local image of the detected image, and n represents the number of partitions of the local images of the image to be queried and the detected image, represents the average of the differences between the feature vector distances of n local images and the feature vector distance of the image to be queried . represents the standard deviation of the differences between the feature vector distances of n local images and the feature vector distance of the image to be queried . represents the difference between the feature vector distance of the local image and the feature vector distance of the image to be queried .

2. The method for image duplicate checking based on deep learning according to claim 1, characterized in that, The determination of the overall picture feature vectors in the query picture library includes: Establish a query picture library for storing the query pictures judged as non-same pictures after each query into the query picture library, and use a deep neural network to extract the overall picture feature vectors of each picture in the query picture library and the local picture feature vectors of the local pictures of each picture, and store them in the feature vector library.

3. The method for image duplicate checking based on deep learning according to claim 1, characterized in that, The feature vector distance between two pictures is calculated using the cosine distance or Euclidean distance calculation formula.

4. The method for image duplicate checking based on deep learning according to claim 1, characterized in that, The number of several pictures retrieved from the query picture library with the lowest feature vector distance from the overall picture of the picture to be queried is not less than 3 pictures.

5. The method for image duplicate checking based on deep learning according to claim 1, characterized in that, The number of local pictures into which the picture to be queried is divided is not less than 4 pictures.

6. A system for image duplicate checking based on deep learning, characterized in that, Including: An extraction module for obtaining the picture to be queried, extracting the overall picture feature vector of the picture to be queried, comparing it with the overall picture feature vectors in the pre-constructed query picture library, and retrieving several detected pictures with the lowest feature vector distance from the overall picture of the picture to be queried from the query picture library; A first judgment module for judging the detected pictures in ascending order of the feature vector distance of the overall picture. If the feature vector distance of the overall picture is lower than threshold one, it is judged as the same picture. If it is greater than threshold two, it is judged as a non-same picture, where threshold two is greater than threshold one; A second judgment module for, if the feature vector distance of the overall picture is between threshold one and threshold two, dividing the picture to be queried into several local pictures, extracting the feature vectors of each local picture, and calculating the feature vector distance of each local picture with the feature vectors of the local pictures of the detected picture one by one, and calculating the feature vector distance of the overall picture by combining according to the feature vector distances of each local picture. If it is less than threshold one, it is judged as the same picture, otherwise it is judged as a non-same picture; The formula for calculating the feature vector distance of the overall picture by combining according to the feature vector distances of each local picture is: ； ； ； ； ； Among them, D represents the feature vector distance of the entire image, represents the feature vector distance of the local image and the feature vector distance of the entire image difference, represents the feature vector distance between the image to be queried and the entire image of the detected image, represents the feature vector distance between the image to be queried and the corresponding local image of the detected image, and n represents the number of partitions of the local images of the image to be queried and the detected image, represents the average of the differences between the feature vector distances of n local images and the feature vector distance of the image to be queried average, represents the difference between the feature vector distances of n local images and the feature vector distance of the image to be queried standard deviation, represents the difference between the feature vector distance of the local image and the feature vector distance of the image to be queried weight.

7. A computer-readable storage medium storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to execute any of the methods according to claims 1 to 5.

8. A computing device, characterized in that, Including, One or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods according to claims 1 to 5.

Images