Image de-weighting method and system based on multilayer feature fusion and improved similarity clustering

Abstract

The invention discloses an image deduplication method and system based on multilayer feature fusion and improved similarity clustering, and relates to the technical field of computer image processing, and the method comprises the steps: carrying out the first feature extraction of an input image, and generating an index; obtaining image similarity according to the feature vectors; reading the index, and performing first analysis on the image; according to the multi-layer feature fusion and improved similarity clustering-based image deduplication method provided by the invention, main features and detail features of the image are respectively extracted by utilizing three-layer wavelet packet decomposition and two-layer wavelet packet decomposition, and the features are combined to form a three-dimensional vector, so that the feature expression capability and the anti-interference capability are improved, and the image deduplication efficiency is improved. According to the method, the calculation efficiency is improved by preferentially matching main features, the comprehensiveness and reliability of repeated image removal are improved by abstracting a similarity result as the distance between samples and carrying out unsupervised clustering analysis, and the method has better effects in the aspects of interference resistance, calculation efficiency and reliability.

Description

Technical Field

[0001] The invention relates to the technical field of computer image processing, and in particular to an image deduplication method based on multi-layer feature fusion and improved similarity clustering. Background Art

[0002] With the development of science and technology, and the continuous updating and iteration of image data acquisition methods, the amount of image data is gradually increasing. It is inevitable that one or more duplicate images will be mixed in the image set. Using manual methods to classify or remove the duplicate images is not only time-consuming and labor-intensive, but also difficult to ensure that there are no omissions. Therefore, it is necessary to develop an efficient, fast and accurate similar image redundancy elimination algorithm.

[0003] Duplicate images usually refer to images that are exactly the same in content, color, size, orientation, etc. These images are exactly the same in characteristics, so it is relatively easy to deduplicate such images, and this can be achieved using general methods. In actual situations, in most cases, duplicate images are not exactly the same, and may also be images with relatively similar content, especially images taken continuously with a camera, which may not appear to be much different in content to the human eye, but may have undergone significant changes to a computer. These may also be images that have been processed by a computer, such as flipping, translation, brightness changes, etc., which may also be significant changes to a computer, but are still duplicate images in terms of image content.

[0004] Currently, the deduplication algorithms for duplicate images can be roughly divided into two categories: one is the deduplication algorithm for exactly the same images, and the other is the algorithm for deduplication of similar images through feature extraction.

[0005] For completely identical images, the commonly used deduplication algorithms include histogram comparison, MD5 value comparison, cosine similarity comparison, and hash value comparison. The deduplication algorithm based on histogram comparison represents the pixel intensity distribution of the image in the form of a histogram, and compares the difference in the histogram to determine whether the image is repeated. Therefore, this algorithm is greatly affected by the image pixels and cannot obtain good results when the image changes in brightness. The deduplication algorithm based on MD5 value requires that the data of the repeated images is highly consistent. If there is any difference, the calculated MD5 value will also be different. Therefore, this method can only deduplicate the completely identical images, and the actual application effect is poor. The image deduplication algorithm based on cosine similarity needs to convert the image information into a set of vectors first, and then calculate the cosine value of the angle in the inner product space based on the vectors. The calculation is complex and the amount of calculation is large. The deduplication algorithm based on hash value needs to convert the image into a grayscale image first, and compress the image resolution to ensure the calculation accuracy. Although this method has a fast processing speed, a lot of image information and details are lost during the processing process. Therefore, the similarity calculated based on the hash value is only a rough similarity, and the actual image deduplication effect is poor.

[0006] The algorithms for deduplication of similar images based on feature extraction generally include deduplication algorithms based on ORB feature extraction, deduplication algorithms based on SIFT (Scale Invariant Feature Transform) feature extraction, and deduplication algorithms based on machine learning. Although the algorithm based on ORB to extract local binary feature points has a fast feature extraction speed, it is greatly affected by the size difference and deformation between images. Although the deduplication algorithm based on SIFT feature extraction has a good effect, the calculation speed is limited by the complexity of feature extraction and feature matching calculation. At the same time, the feature extraction effect of images after flipping and other operations is poor. The deduplication algorithm based on machine learning requires a large amount of data and time for training, and the deduplication effect depends on the training results, and the actual application effect is poor.

[0007] Most of the existing image deduplication algorithms currently perform deduplication when there is only one duplicate image. They cannot handle the situation where there are multiple duplicate images well. Even if they do, they only compare the subsequent duplicate images with the first one, ignoring the correlation between the multiple duplicate images, so that the query of similar images is missed or inaccurate. Summary of the invention

[0008] In view of the above-mentioned problems, the present invention is proposed.

[0009] Therefore, the technical problem solved by the present invention is: the existing image deduplication methods have insufficient processing of multiple duplicate images, limited feature extraction efficiency and accuracy, low efficiency of similarity calculation and clustering analysis, and how to achieve optimization of efficient image feature extraction and similarity calculation.

[0010] To solve the above technical problems, the present invention provides the following technical solutions: an image deduplication method based on multi-layer feature fusion and improved similarity clustering, including performing a first feature extraction on the input image to generate an index; obtaining image similarity based on the feature vector; reading in the index and performing a first analysis on the image.

[0011] As a preferred solution of the image deduplication method based on multi-layer feature fusion and improved similarity clustering described in the present invention, wherein: the first feature extraction of the input image includes extracting features of the input image to obtain a feature vector.

[0012] As a preferred solution of the image deduplication method based on multi-layer feature fusion and improved similarity clustering described in the present invention, wherein: generating the index includes generating an index of the file name plus the feature.

[0013] As a preferred solution of the image deduplication method based on multi-layer feature fusion and improved similarity clustering described in the present invention, wherein: the step of based on the feature vector includes reading in the feature vector.

[0014] As a preferred solution of the image deduplication method based on multi-layer feature fusion and improved similarity clustering described in the present invention, wherein: the obtaining of image similarity includes integrating the similarities of various components.

[0015] As a preferred solution of the image deduplication method based on multi-layer feature fusion and improved similarity clustering described in the present invention, wherein: the read-in index includes the read-in file name plus feature index.

[0016] As a preferred solution of the method for image deduplication based on multi-layer feature fusion and improved similarity clustering described in the present invention, wherein: the first analysis of the image includes obtaining an image clustering result.

[0017] Another object of the present invention is to provide an image deduplication system based on multi-layer feature fusion and improved similarity clustering, which can calculate the similarity between images according to the feature vector through a similarity calculation module using a hierarchical comparison method, thereby solving the current problems of high similarity calculation overhead, low efficiency, and susceptibility to noise or subtle changes.

[0018] As a preferred solution of the image deduplication system based on multi-layer feature fusion and improved similarity clustering described in the present invention, it includes: a feature extraction module, a similarity calculation module, and a cluster analysis module; the feature extraction module is used to perform a first feature extraction on the input image and generate an index; the similarity calculation module is used to obtain image similarity based on the feature vector; the cluster analysis module is used to read in the index and perform a first analysis on the image.

[0019] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that the processor executes the computer program to implement the steps of an image deduplication method based on multi-layer feature fusion and improved similarity clustering.

[0020] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of an image deduplication method based on multi-layer feature fusion and improved similarity clustering are implemented.

[0021] Beneficial effects of the present invention: The image deduplication method based on multi-layer feature fusion and improved similarity clustering provided by the present invention utilizes three-layer wavelet packet decomposition and two-layer wavelet packet decomposition to respectively extract the main features and detail features of the image, and combines the features to form a three-dimensional vector, thereby improving the feature expression ability and anti-interference ability. By preferentially matching the main features, the calculation efficiency is improved, and the risk of misjudgment due to noise or minor changes is reduced. By abstracting the similarity results as the distance between samples and performing unsupervised clustering analysis, the comprehensiveness and reliability of duplicate image removal are improved. The present invention achieves better results in terms of anti-interference, calculation efficiency and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0023] Figure 1 The overall flow chart of the image deduplication method based on multi-layer feature fusion and improved similarity clustering provided for the first and second embodiments of the present invention.

[0024] Figure 2 A schematic diagram of image feature extraction based on multi-layer feature fusion and improved similarity clustering image deduplication method provided for the first and second embodiments of the present invention.

[0025] Figure 3 A schematic diagram of similarity calculation based on a multi-layer feature fusion and improved similarity clustering image deduplication method provided in a second embodiment of the present invention.

[0026] Figure 4 A schematic diagram of DBScan clustering analysis based on improved similarity, which is an image deduplication method based on multi-layer feature fusion and improved similarity clustering, provided in the second embodiment of the present invention.

[0027] Figure 5This is an overall module diagram of an image deduplication system based on multi-layer feature fusion and improved similarity clustering provided in the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0029] Example 1

[0030] Reference Figure 1-Figure 2 , which is an embodiment of the present invention, provides an image deduplication method based on multi-layer feature fusion and improved similarity clustering, including:

[0031] S1: Perform the first feature extraction on the input image and generate an index.

[0032] Furthermore, performing a first feature extraction on the input image includes performing feature extraction on the input image to obtain a feature vector.

[0033] In the embodiment of the present application, the first feature extraction is image feature extraction, and the input is a similar redundant image data set including at least two categories and at least one or no redundant images;

[0034] Determine whether the input single image is in HIS space. If not, convert it from RGB space to HIS space, decompose the converted image according to HIS space to obtain three components: H, I, and S, and convert each decomposed component from a two-dimensional matrix to a one-dimensional matrix;

[0035] The three components are decomposed by three-layer wavelet packets respectively, and eight nodes are obtained for each component, namely eight frequency bands. The energy proportions of the eight frequency bands are arranged from high to low according to the frequency size, as a group of one-dimensional feature vectors A1, recorded as the main features;

[0036] The two with the largest energy values ​​are further subjected to two-layer wavelet packet decomposition to obtain two groups of four nodes and four frequency bands. The energy proportions of the two groups of four frequency bands are sorted according to the frequency size, and the two groups of results are combined as a group of one-dimensional feature vectors A2, which are recorded as detail features;

[0037] The two one-dimensional vectors are combined to obtain a set of two-dimensional vectors, and the feature vectors of the three components of HIS are merged to obtain the three-dimensional feature vector of the entire image.

[0038] In an optional embodiment, the first feature extraction can also be implemented in other ways, such as feature extraction based on deep learning. Before feature extraction, the input image data set is standardized, including resizing, pixel value normalization and data enhancement. All images are uniformly adjusted to a fixed size (such as 224×224 pixels) to ensure the consistency of network input, and the pixel values ​​are normalized to the [0,1] interval to reduce the complexity of numerical calculations and improve the stability of the model.

[0039] The pre-trained deep convolutional neural network model is used as the basic framework for feature extraction. By removing the fully connected layers in the pre-trained model and retaining only the convolutional layers and pooling layers, the network is adjusted to a feature extraction module. The first few layers of the convolutional network are used to extract low-level features of the image, such as edges, textures, and color distribution. These features are mainly used to represent the basic structural information of the image. High-level semantic features, such as the shape of objects, local structures, and overall layout in the image, are extracted through deep convolutional layers. These features have higher expressive power and can effectively distinguish the categories and similarities of images.

[0040] The low-level features and high-level semantic features are integrated through feature fusion. The features extracted by different convolutional layers are spliced ​​according to the channel dimension to form a multi-dimensional feature matrix. Different weights are given to features at different levels to highlight the features that are more discriminative for image deduplication. The fused high-dimensional features are reduced in dimension using principal component analysis or a dimensionality reduction layer (such as a global average pooling layer) to obtain a feature vector of fixed length.

[0041] Furthermore, generating the index includes generating an index of the file name plus the feature.

[0042] It should be noted that the file name and the extracted feature vector are added to the index dictionary as a set of key-value pairs. If the number of images is large, the index dictionary can be written into a json file for subsequent use.

[0043] Example 2

[0044] Reference Figure 1-Figure 4 , which is an embodiment of the present invention, provides an image deduplication method based on multi-layer feature fusion and improved similarity clustering, including:

[0045] S2: Obtain image similarity based on feature vectors.

[0046] Furthermore, the feature vector includes reading in the feature vector, and reading in the feature vectors of two images to be compared.

[0047] Furthermore, obtaining the image similarity includes integrating the similarities of each component.

[0048] It should be noted that, according to the eigenvector, the eigenvector with the largest energy value in the three components A1 of HIS is compared to see whether it belongs to the same frequency band. If True is output, the next one is compared in the order of energy proportion. If False is output, the comparison is directly jumped out, the similarity is given, and the cycle is repeated until all the values ​​in the A1 eigenvector are matched. The eigenvector with the largest energy value in the three components A2 of HIS is compared to see whether it belongs to the same frequency band. The two groups of A2 are performed in turn. If True is output, the next one is compared in the order of energy proportion. If False is output, the comparison is jumped out, and the similarity is given. The cycle is repeated until all the values ​​in the eigenvector are compared. The similarity of the three components of HIS is comprehensively considered to give the final result of similarity.

[0049] It should also be noted that the main features are compared first, and the detailed features are compared later. The ones with higher energy proportion values ​​are compared first, and the ones with lower energy proportion values ​​are compared later. After the priority features are matched, the others are compared. If the main features do not match, there is no need to compare the detailed features. The lower the similarity, the faster the calculation speed, and the higher the similarity, the longer the calculation time.

[0050] S3: Read the index and perform a first analysis on the image.

[0051] Furthermore, reading the index includes reading the index of the file name plus the feature, and selecting the object P in the index in sequence.

[0052] Furthermore, performing a first analysis on the image includes obtaining an image clustering result.

[0053] In the embodiment of the present application, the first analysis is a DBScan cluster analysis based on similarity improvement to determine whether P belongs to a certain cluster. If it does, the object is reselected;

[0054] Determine whether P is a core point. If it is a core point, mark P as processed. If it is not a core point, determine whether P is a noise point. If it is a noise point, separate P into a cluster and mark it as processed. If it is not a noise point, reselect the object.

[0055] Traverse all unlabeled data in the neighborhood of the core point P, group all unlabeled points and the core point P into one cluster, and use the distance parameter used to calculate the neighborhood range as the abstracted similarity, i.e., d = 1-[S(P1, P2)], mark all unlabeled points as processed, and determine whether they are core points in turn. If they are core points, repeat the traversal. If all core points and all unlabeled points in their neighborhood are processed, reselect the object;

[0056] Repeat the loop until all items in the index are marked as processed, and output all clusters obtained by clustering, each cluster being a class of similar images.

[0057] In an optional embodiment, the first analysis may also be implemented in other ways, such as cluster analysis based on hierarchical clustering. Before the cluster analysis, it is necessary to calculate the similarity distance matrix between all images based on the feature vectors generated by the feature extraction module. The distance calculation may be performed using the Euclidean distance, the cosine distance, or a similarity formula generated based on the similarity calculation module. The feature vector of each image is read from the index, and the distance between the feature vectors is calculated for each pair of images to form a symmetrical distance matrix.

[0058] Treat each image as an independent cluster, that is, there are n clusters in the initial state, and each cluster contains only one image;

[0059] According to the value of the distance matrix, select the two clusters with the smallest distance to merge, and update the distance matrix. The distance can be updated by the single chain method. The distance between the new cluster and other clusters is the distance between the closest points in the two clusters.

[0060] The full chain method uses the distance between the new cluster and other clusters as the distance between the farthest points in the two clusters;

[0061] In the average chain method, the distance between the new cluster and other clusters is the average of all the distances between the two clusters.

[0062] In the centroid method, the distance between the new cluster and other clusters is the distance between the cluster centers;

[0063] The clustering process continues until any of the following conditions is met: the minimum distance between all clusters is greater than the preset threshold, the total number of clusters reaches the specified target number of clusters, and no further merging is possible;

[0064] The final clustering result includes all clusters and the images they contain. Each cluster represents a group of similar images, while images that are not clustered are regarded as noise points.

[0065] Example 3

[0066] An embodiment of the present invention provides an image deduplication method based on multi-layer feature fusion and improved similarity clustering. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0067] A dataset containing multiple categories of similar redundant pictures was selected, containing a total of 5,000 pictures. The dataset comes from various types of images taken in different scenes, including pictures after various pre-processing such as flipping, brightness adjustment, and scaling, as well as several continuously shot pictures with high similarity.

[0068] Each image is input into feature extraction, and the main features are extracted by three-layer wavelet packet decomposition, and the detail features are extracted by two-layer wavelet packet decomposition. The main features and detail features are fused to form a three-dimensional feature vector. During the feature extraction process, the image name is associated with the feature vector using an index file to facilitate subsequent processing. For any two images, the matching rate of the main features is first compared. If the main feature matching rate is lower than the threshold, it is directly judged as dissimilar. If the main feature matching rate meets the requirements, the detail features are further compared. The improved DBScan algorithm is used, and the similarity results of the images are used as the distance parameter. Similar images are classified into the same group through density clustering. The algorithm does not need to pre-set the number of image categories, and can dynamically identify similar categories in the image set. The clustering results are compared with the manually labeled similar groups, and the clustering accuracy and error rate are calculated. At the same time, the time for processing each group of images is recorded to evaluate the efficiency.

[0069] As shown in Table 1, the matching rates of main features are all above 88%, and the matching rates of detail features are all above 85%, indicating that feature extraction has a high expressive ability in extracting global and local features of images, and can effectively distinguish similar images from non-similar images. The processing time is generally between 0.45 and 0.60 seconds, which is significantly improved compared to the processing time of more than 1 second of traditional algorithms. The hierarchical strategy of priority matching of main features reduces the frequency of complex calculations, and the clustering error rate is generally lower than 1.2%. The error rate of most samples is controlled in the range of 0.5%-0.7%, reflecting that the improved DBScan algorithm can accurately process large-scale image data sets and reduce the omission or classification errors of similar images.

[0070] Table 1 Experimental data table

[0071]

[0072] Example 4

[0073] Reference Figure 5 , as an embodiment of the present invention, provides an image deduplication system based on multi-layer feature fusion and improved similarity clustering, including: a feature extraction module, a similarity calculation module, and a cluster analysis module.

[0074] The feature extraction module is used to extract the first feature of the input image and generate an index; the similarity calculation module is used to obtain the image similarity according to the feature vector; and the cluster analysis module is used to read the index and perform the first analysis on the image.

[0075] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0076] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0077] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0078] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0079] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An image deduplication method based on multi-layer feature fusion and improved similarity clustering, characterized in that: include: Performing a first feature extraction on the input image and generating an index; According to the feature vector, the image similarity is obtained; Read in the index and perform a first analysis of the image.

2. The image deduplication method based on multi-layer feature fusion and improved similarity clustering as claimed in claim 1, characterized in that: The first feature extraction of the input image includes extracting features of the input image to obtain a feature vector.

3. The image deduplication method based on multi-layer feature fusion and improved similarity clustering as claimed in claim 2, characterized in that: The generating the index includes generating an index of the file name plus features.

4. The image deduplication method based on multi-layer feature fusion and improved similarity clustering as claimed in claim 3, characterized in that: The step of following the feature vector includes reading in the feature vector.

5. The image deduplication method based on multi-layer feature fusion and improved similarity clustering as claimed in claim 4, characterized in that: The obtaining of image similarity includes integrating similarities of various components.

6. The image deduplication method based on multi-layer feature fusion and improved similarity clustering as claimed in claim 5, characterized in that: The read-in index includes an index of a read-in file name plus features.

7. The image deduplication method based on multi-layer feature fusion and improved similarity clustering as claimed in claim 6, characterized in that: The first analysis of the image includes obtaining an image clustering result.

8. A system using the image deduplication method based on multi-layer feature fusion and improved similarity clustering as claimed in any one of claims 1 to 7, characterized in that: Including feature extraction module, similarity calculation module and cluster analysis module; The feature extraction module is used to extract a first feature from the input image and generate an index; The similarity calculation module is used to obtain image similarity based on the feature vector; The cluster analysis module is used to read the index and perform a first analysis on the image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the image deduplication method based on multi-layer feature fusion and improved similarity clustering as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image deduplication method based on multi-layer feature fusion and improved similarity clustering as described in any one of claims 1 to 7 are implemented.

Images