Similar picture detection method and system based on multi-feature fusion and dynamic adjustment strategy

The method and system enhance similar image detection by integrating multi-feature fusion and dynamic threshold adjustment, addressing inefficiencies in existing technologies to provide accurate and efficient image similarity judgments in diverse scenarios.

CN119068218AActive Publication Date: 2024-12-03NANJING DAOCHENG NETWORK TECH CO LTD

Patent Information

Application Number
CN202411177512.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-12-03
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

Existing similar image detection technologies struggle with poor adaptability to complex scenes, difficulty in balancing efficiency and accuracy, and insufficient dynamism and intelligence, leading to inaccurate similarity judgments and high computational complexity, especially in mobile devices.

Method used

A method and system utilizing multi-feature fusion and dynamic adjustment strategies, combining perceptual hash and histogram algorithms with convolutional neural networks, to capture global and local features, dynamically adjust similarity thresholds based on user needs, and incorporate shooting time and location for enhanced accuracy and efficiency.

Benefits of technology

Improves the accuracy of similar image detection in complex environments by efficiently balancing speed and precision, reducing resource consumption, and maintaining reliable results under varying conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068218A_ABST
    Figure CN119068218A_ABST
Patent Text Reader

Abstract

The invention discloses a mobile phone similar picture detection method and system based on multi-feature fusion and a dynamic adjustment strategy, and relates to the technical field of similar picture detection. The method comprises the following steps: sorting pictures in a photo album according to a time sequence, and carrying out preliminary similarity judgment on the pictures to evaluate global similarity; if the judgment result is dissimilar, further judging whether the shooting time and the shooting position of the picture are close or not, if so, dynamically adjusting a similarity threshold value, and performing depth similarity judgment by using the adjusted similarity threshold value; if not, the output result is that the pictures are not similar. According to the method, depth similarity judgment is carried out by adopting the convolutional neural network and the twin network, fusion of global and local features is realized, the accuracy of similar picture recognition in a complex environment is greatly improved, the processing capacity of complex pictures is also improved while efficient detection is ensured, and the method is suitable for popularization and application. The method is especially suitable for high-precision similarity detection tasks of large-scale picture libraries.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of similar image detection, and in particular to a similar image detection method and system based on multi-feature fusion and dynamic adjustment strategy. Background Art

[0002] With the popularity of smartphones and the continuous improvement of photography technology, the amount of image data generated by users has increased dramatically, resulting in increasingly tight storage space in mobile phones. How to efficiently manage and organize these massive images, especially how to accurately detect and delete duplicate or similar images, has become an urgent problem that needs to be solved.

[0003] At present, there are many similar image detection technologies on the market, most of which are based on traditional image processing methods, such as histogram similarity algorithm and perceptual hashing algorithm. The histogram algorithm is a classic image similarity detection method. It calculates the color histogram of the image and compares the histogram distribution of different images, and uses the Bhattacharyya distance to preliminarily judge the similarity of two images. This method is simple to implement and has high computational efficiency, but it is mainly based on the color distribution of the image for comparison, and cannot effectively capture the local features and structural information of the image. Therefore, it is easy to make misjudgments when dealing with illumination changes, color shifts or small image transformations. In addition, the histogram algorithm has difficulty in dealing with fine-grained image comparisons in complex scenes, resulting in low accuracy of the detection results. Perceptual hashing algorithm is a technology for generating image fingerprints. It generates a hash value that reflects the essential characteristics of the image content by performing operations such as image preprocessing and feature extraction, and uses the Hamming distance to measure the similarity between images. The perceptual hashing algorithm can better handle some minor image transformations, such as size changes and format compression. However, the perceptual hashing algorithm may not be accurate enough when processing color information, and has limited ability to recognize image subjects in complex backgrounds. In addition, there is room for improvement in the computational efficiency of the perceptual hashing algorithm, which makes it difficult to meet the rapid retrieval requirements of large-scale image libraries.

[0004] Therefore, although the traditional methods can meet the basic requirements of users for similar image detection to a certain extent, with the increase in the number and diversity of images, the existing technology lacks sufficient sensitivity to the local features and global structure of the images, especially when dealing with complex backgrounds, lighting changes and partial deformation of the image, which easily leads to inaccurate similarity judgment and poor adaptability to complex scenes; although the histogram algorithm has high computational efficiency, it is insufficient in accuracy, and although the perceptual hash algorithm has improved accuracy, it has high computational complexity and is difficult to be efficiently applied in large-scale image libraries, and it is difficult to achieve a balance between efficiency and accuracy; most of the current similar image detection algorithms are based on fixed thresholds or rules for judgment, lacking the ability to intelligently adapt to user personalized needs and specific usage scenarios, and are difficult to cope with the diversity of user operation behaviors and the complexity of image content. For mobile phone users, multiple images generated in a short period of time are not processed in a targeted manner, resulting in a low success rate and little benefit to users. Summary of the invention

[0005] In view of the problems existing in the existing similar image detection, such as poor adaptability to complex scenes, difficulty in balancing efficiency and accuracy, and insufficient dynamism and intelligence, the present invention proposes a similar image detection method and system based on multi-feature fusion and dynamic adjustment strategy, which can simultaneously capture the global and local features of the image, and can maintain high-precision feature extraction and fusion strategies in complex scenes. It has the ability to dynamically adjust the similarity threshold under the personalized needs of users and diversified usage scenarios, while taking into account the accuracy and speed of retrieval, and can still maintain efficient operating performance in environments with limited computing resources such as mobile devices.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0007] A similar image detection method based on multi-feature fusion and dynamic adjustment strategy, the method comprising:

[0008] Sort the pictures in the album in chronological order and perform similarity detection on any two pictures in the album;

[0009] Preliminary similarity judgment: The perceptual hash value and histogram hash value of the two images are calculated by the perceptual hash algorithm and the histogram algorithm respectively, and then the Hamming distance between the perceptual hash values ​​and the Hamming distance between the histogram hash values ​​are calculated respectively. The two Hamming distances are compared with the preset similarity threshold to evaluate the global similarity;

[0010] If the judgment result is not similar, then further determine whether the shooting time and shooting location of the pictures are similar. If so, dynamically adjust the similarity threshold and use the adjusted similarity threshold to perform deep similarity judgment; if not, the output result is that the pictures are not similar;

[0011] Deep similarity judgment: The local features of the image are extracted through a convolutional neural network, and weighted fused with the global features extracted by the histogram algorithm. The fused features are input into the twin network to calculate the deep similarity score, which is compared with the adjusted similarity threshold. If the deep similarity score is greater than the adjusted similarity threshold, the output result is that the images are similar, otherwise the output result is that the images are not similar.

[0012] A further improvement of the present invention is that the perceptual hash value of the image is calculated by a perceptual hash algorithm, and the method includes:

[0013] Preprocess the image, scale it to a standard size of 32×32, and convert it into a grayscale image;

[0014] Reduce the image bit depth from 256 grayscale levels to 64 levels;

[0015] The image is divided into blocks, each block is subjected to discrete cosine transform, a frequency coefficient matrix is ​​generated, and the DCT coefficients of the low-frequency part are selected as the extracted image frequency features;

[0016] Calculating the average value of the DCT coefficients of the low-frequency part to generate an average threshold;

[0017] Each DCT coefficient is compared with the average value threshold, and the hash bit corresponding to the DCT coefficient above the average value threshold is set to 1, otherwise it is set to 0, and a fixed-length binary hash string is generated as the perceptual hash value of the image.

[0018] A further improvement of the present invention is that a histogram hash value of a picture is calculated by a histogram algorithm, and the method includes:

[0019] Convert the image's color space from RGB mode to three independent channels for processing;

[0020] Construct histogram: Count the pixel values ​​of the three RGB channels independently, divide each channel into 32 intervals according to the method of dividing [0,256) into [0,8), count the frequency of occurrence of pixel values ​​in each interval, and form a global histogram;

[0021] Normalization: Convert each interval frequency value of the histogram into a normalized value so that the sum of all frequency values ​​is equal to 1;

[0022] Simplified processing: Divide the normalized value of the histogram into 16 intervals, generate a binary hash bit for each interval, and finally form a fixed-length hash string as the histogram hash value.

[0023] A further improvement of the present invention is that the comprehensive evaluation of the global similarity by two Hamming distances specifically includes:

[0024] The two Hamming distances are combined into a comprehensive similarity score using weighted summation or simple averaging, where

[0025] The weighted summation formula is: comprehensive similarity score = α × Hamming distance of perceptual hash value + β × Hamming distance of histogram hash value, where α and β are weight coefficients;

[0026] The simple average formula is: comprehensive similarity score = (Hamming distance of perceptual hash value + Hamming distance of histogram hash value) / 2;

[0027] The comprehensive similarity score is compared with a preset similarity threshold. If the comprehensive similarity score is greater than the similarity threshold, the two images are determined to be similar in global features, and the system directly outputs the result that the images are similar; if not, the two images are determined to be significantly different in global features, and the next step of analysis is performed based on time and location factors.

[0028] A further improvement of the present invention is that the method for determining whether the shooting time and shooting location of the pictures are close, and if so, dynamically adjusting the similarity threshold comprises:

[0029] Extracting shooting timestamp and shooting location information from metadata of the image, wherein the shooting location information is geographic coordinates recorded by GPS data;

[0030] Shooting time judgment: set a time threshold, calculate the shooting time difference between two pictures, if the shooting time difference between the two pictures is less than or equal to the time threshold, it is considered that the shooting time of the two pictures is close, and the similarity threshold is increased by a%;

[0031] Shooting location judgment: set a distance threshold, calculate the distance between the geographical coordinates of the two pictures, if the shooting locations of the two pictures are less than or equal to the distance threshold, then the shooting locations of the two pictures are considered to be close, and the similarity threshold is increased by b%;

[0032] If the shooting time and shooting location of the two pictures are close, the similarity threshold is increased by c%, and c>b>a;

[0033] Re-evaluate the similarity using the adjusted similarity threshold.

[0034] A further improvement of the present invention is that the method for extracting local features from an image by using a convolutional neural network comprises:

[0035] Preprocess the image, resize it to a standard size of 224×224 pixels, and normalize the pixel values ​​to the range of [0,1];

[0036] Use a multi-scale convolutional neural network to perform multi-scale convolution operations on the image. At each scale, convolution operations and pooling operations are performed to obtain feature maps of different scales.

[0037] An attention module is introduced at each position of the feature map, the attention weight is calculated, and it is multiplied pixel by pixel with the original feature map to generate a weighted feature map;

[0038] The feature map weighted by the attention mechanism is flattened, the two-dimensional feature map is converted into a one-dimensional feature vector, and the local feature vector F is generated through the fully connected layer. CNN-增强 .

[0039] A further improvement of the present invention is that the method for extracting global features using a histogram algorithm comprises:

[0040] The frequency of each interval in the normalized global histogram is used as the global feature representation F 全局 , the global eigenvector F is 全局 Sparse coding is performed, the formula is:

[0041]

[0042] Among them, D is the dictionary matrix, Θ is the sparse representation matrix, T 0 is the sparsity threshold, which is used to limit the kth sparse vector θ k The number of non-zero elements in ;

[0043] Perform content analysis on the image to identify the target object and background area, and introduce an adaptive weight learning mechanism to train weight parameters through a gradient descent optimization strategy to dynamically adjust the weights of different dimensions in the global feature representation. The formula is:

[0044]

[0045] Where W is the weight matrix; L(·) is the cross entropy loss function or the mean square error loss function; is the global feature vector of the jth image; y (j) is the label of the jth image, y=1 means similar, y=0 means dissimilar; J is the number of images;

[0046] The final generated weight matrix W is applied to the sparse representation matrix Θ after sparse coding to obtain the optimized global eigenvector F Hist-全局 .

[0047] A further improvement of the present invention is that the method for weighted fusion of local features and global features includes:

[0048] The local feature vector F extracted by the convolutional neural network CNN-增强and the sparsely coded global feature vector F Hist-全局 Perform nonlinear weighted combination, the formula is as follows:

[0049] F 融合 =ω·tanh(F CNN-增强 )+λ·tanh(F Hist-全局 );

[0050] Among them, F 融合 is the fused feature representation; ω and λ are the adaptive weight parameters obtained through training, and tanh(·) is the hyperbolic tangent function, which is used to map the local feature representation vector to the interval [-1,1];

[0051] Introducing the residual connection mechanism, the local feature vector F CNN-增强 and the global eigenvector F Hist-全局 And the fused feature representation F 融合 Perform weighted combination to obtain the final fusion feature vector F 融合-增强 , the formula is:

[0052] F 融合-增强 =F 融合 +γ·(F CNN-增强 +F Hist-全局 );

[0053] Among them, γ is the residual weight coefficient.

[0054] A further improvement of the present invention is that the method of inputting the fused features into the twin network to calculate the deep similarity score includes:

[0055] The final fusion feature vector F 融合-增强 Input into a Siamese network with a multi-layer embedding structure, where each layer of the embedding structure consists of multiple fully connected layers and activation functions;

[0056] After each layer of the twin network is embedded, a recursive neural network is introduced for time series modeling to capture the temporal relationship between features;

[0057] When training the twin network, the contrastive learning loss function is introduced to further optimize the distance metric between features, so that the embedding vector distance of similar images is minimized and the distance of non-similar images is maximized. The contrastive learning loss function formula is:

[0058]

[0059] Where d is the distance between two embedding vectors, m is the distance threshold of the embedding vectors; y is the label, y=1 means similarity, y=0 means dissimilarity;

[0060] Calculate two embedding vectors E 1 、E2 The improved Euclidean distance between them is as follows:

[0061]

[0062] Among them, E 1 、E 2 are the two embedding vectors output from the twin network; E 1i 、E 2i Denote the embedding vector E 1 、E 2 The i-th eigenvalue in i is the weight coefficient corresponding to the i-th eigenvalue; n represents the dimension of the embedding vector;

[0063] D 改进 (E 1 ,E 2 ) is the embedding vector E 1 、E 2 The modified Euclidean distance between ;

[0064] The modified Euclidean distance is used as a depth similarity score and compared with the adjusted similarity threshold.

[0065] A similar image detection system based on multi-feature fusion and dynamic adjustment strategy, the system comprising:

[0066] A processor for performing computing operations;

[0067] A memory for storing image data and program codes;

[0068] An image processing module, used to execute the similarity detection method based on multi-feature fusion and dynamic adjustment strategy as described above, and perform similarity detection on the images;

[0069] The output module is used to output the results of similar image detection and determine whether the images are similar;

[0070] The image processing module specifically includes:

[0071] The picture sorting module is used to sort the pictures in the album in chronological order;

[0072] Preliminary similarity judgment module: used to calculate the perceptual hash value and histogram hash value of two pictures respectively through the perceptual hash algorithm and the histogram algorithm, and then calculate the Hamming distance between the perceptual hash values ​​and the Hamming distance between the histogram hash values ​​respectively, and compare the two Hamming distances with the preset similarity threshold to evaluate the global similarity;

[0073] The shooting information analysis module is used to further determine whether the shooting time and shooting location of the pictures are close, and dynamically adjust the similarity threshold;

[0074] Local feature extraction module, used to extract local features of images through convolutional neural networks;

[0075] The global feature optimization module is used to perform sparse coding processing and adaptive weight adjustment on the global features of the image to generate an optimized global feature vector;

[0076] The deep similarity judgment module is used to extract the local features of the image using the convolutional neural network, perform weighted fusion with the global features extracted by the histogram algorithm, input the fused features into the twin network to calculate the deep similarity score, and compare it with the adjusted similarity threshold to determine whether the images are similar.

[0077] The beneficial effects of the present invention are: through time sorting and preliminary similarity judgment, dissimilar pictures can be quickly excluded to reduce computing resource consumption; when dissimilar is initially judged, the similarity threshold is dynamically adjusted in combination with shooting time and location information to enhance the flexibility of detection; convolutional neural networks and twin networks are used for deep similarity judgment to achieve the fusion of global and local features, greatly improving the accuracy of similar picture recognition in complex environments. For individual users, this method can efficiently retrieve and remove similar pictures in mobile devices, greatly reducing disk storage usage. It can also maintain efficient detection performance in the detection of large-scale image libraries, and can also provide more stable similar photo inspection results when processing photos with lighting changes and partial differences. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0079] in:

[0080] Figure 1 is a flow chart of the method of the present invention;

[0081] Figure 2 4 is a system structure diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0083] Take a common scenario in life as an example: we pass by a beautiful building, and we take out the camera to take a picture of the building. However, in addition to the building, we will also take pictures of people around the building in the background. They will change the structural information and color information of the entire picture, but the main building has not changed. According to the results of the existing similar image detection algorithm, two pictures taken continuously may be judged as dissimilar, but according to the user's usage habits, these two pictures can be counted as similar pictures because our shooting subject has not changed. In response to this scenario, the present invention introduces time and place factors, dynamically adjusts the similarity threshold, and performs segmentation judgment on the subject, providing users with a reliable and fast similar image detection method that is suitable for image data sets of various sizes and types.

[0084] like Figure 1 As shown in FIG. 1 , an embodiment of the present invention is provided, which provides a similar image detection method with multi-feature fusion and dynamic adjustment strategy, comprising the following steps:

[0085] S1: Sort the pictures in the album in chronological order and perform similarity detection on any two pictures in the album;

[0086] The system reads all the images in the dataset, which may come from mobile phone albums, cloud storage or other image databases, extracts their shooting time metadata, and sorts the images according to the timestamp to form a time-linear image list. The sorted image list will be used as input for the subsequent similarity detection step to ensure that the images are compared in chronological order during the similarity detection.

[0087] Step S1 is to organize the pictures preliminarily, filter out the pictures that are unlikely to be related, thereby optimizing the use of computing resources and reducing the workload of similarity detection, so as to ensure that the subsequent similarity detection process can be carried out on the basis of time series. This helps to improve the detection efficiency, especially when processing a large number of pictures, the time sequence can effectively reduce the number of comparisons of irrelevant pictures.

[0088] S2: Preliminary similarity judgment: The perceptual hash value and histogram hash value of the two images are calculated by the perceptual hash algorithm and the histogram algorithm respectively, and then the Hamming distance between the perceptual hash values ​​and the Hamming distance between the histogram hash values ​​are calculated respectively, and the two Hamming distances are compared with the preset similarity threshold to evaluate the global similarity;

[0089] In one embodiment, step S2 comprises:

[0090] S21: Calculating the perceptual hash value of the image by using a perceptual hash algorithm, including the following steps:

[0091] S211: preprocessing the image, scaling the image to a standard size of 32×32 pixels, and converting the image to a grayscale image to eliminate the influence of the image size and color dimension;

[0092] S212: Reduce the image bit depth from 256 gray levels to 64 gray levels;

[0093] S213: Divide the image into blocks, perform discrete cosine transform on each block, generate a frequency coefficient matrix, and select the DCT coefficient of the low-frequency part as the extracted image frequency feature;

[0094] After applying DCT to an image block (for example, an 8×8 block), a matrix of the same size is obtained. The coefficients in the upper left corner of the matrix represent low-frequency components, which represent the main brightness and structural information of the image, and the coefficients in the lower right corner represent high-frequency components, which represent the details and texture information of the image. The most common method is to select the upper left corner area of ​​the DCT matrix, which contains most of the low-frequency coefficients. Specifically, you can choose a 2×2, 3×3, or 4×4 upper left submatrix, depending on how much low-frequency information you want to retain;

[0095] S214: Calculate the average value of the DCT coefficients of the low-frequency part to generate an average threshold;

[0096] S215: Compare each DCT coefficient with the average value threshold, set the hash bit corresponding to the DCT coefficient higher than the average value threshold to 1, otherwise set to 0, and generate a fixed-length binary hash string as the perceptual hash value of the image.

[0097] S22: Calculate the histogram hash value of the image through a histogram algorithm, including:

[0098] S221: performing color space conversion on the image, converting the image from RGB mode to independent three-channel processing to reduce the correlation between color channels;

[0099] S222: Constructing a histogram: Counting the pixel values ​​of the three channels of RGB independently, dividing each channel into 32 intervals according to the method of dividing [0,256) into [0,8), counting the frequency of occurrence of pixel values ​​in each interval, and forming a global histogram;

[0100] S223: Normalization processing: converting each interval frequency value of the histogram into a normalized value so that the sum of all frequency values ​​is equal to 1, thereby eliminating the influence of the image size;

[0101] S224: Simplified processing: divide the normalized value of the histogram into 16 intervals, generate a binary hash bit for each interval, and finally form a hash string of fixed length as the histogram hash value;

[0102] By utilizing the global statistical characteristics of the histogram, the color distribution of the image can be stably reflected, and the generated hash value is convenient for fast calculation and comparison.

[0103] S23: Calculate the Hamming distance between the perceptual hash values ​​and the Hamming distance between the histogram hash values ​​respectively. The Hamming distance is a measure of the difference between two strings of equal length, which is defined by counting the number of different characters in the two strings. In image processing, the Hamming distance is used to measure the difference between two images, so as to match and recognize images.

[0104] S24: Combine the two Hamming distances to evaluate the global similarity, including:

[0105] S241: Combine the two Hamming distances into a comprehensive similarity score using weighted summation or simple average, where

[0106] The weighted summation formula is: comprehensive similarity score = α × Hamming distance of perceptual hash value + β × Hamming distance of histogram hash value, where α and β are weight coefficients;

[0107] The simple average formula is: comprehensive similarity score = (Hamming distance of perceptual hash value + Hamming distance of histogram hash value) / 2;

[0108] S242: Compare the comprehensive similarity score with the preset similarity threshold. If the comprehensive similarity score is greater than the similarity threshold, the two images are determined to be similar in global features, and the system directly outputs the result that the images are similar. If not, the two images are determined to be significantly different in global features, and the next step of analysis is performed based on time and location factors.

[0109] Assume that the hash values ​​of images A and B are calculated as follows:

[0110] Image A:

[0111] Perceptual hash value: 1010101111001101;

[0112] Histogram hash value: 1110001111001111;

[0113] Image B:

[0114] Perceived hash value: 1010101111011101;

[0115] Histogram hash value: 1110000111001111;

[0116] Hamming distance of perceptual hashing: the distance between 1010101111001101 and 1010101111011101 is 1;

[0117] Hamming distance of histogram hash: the distance between 1110001111001111 and 1110000111001111 is 2;

[0118] The composite similarity score assumes a simple average:

[0119] Comprehensive similarity score = (1 + 2) ÷ 2 = 1.5;

[0120] The system compares 1.5 with the preset threshold (assuming the threshold is 2). Since 1.5 is less than 2, the system will determine that images A and B are not similar in global features.

[0121] S3: If the judgment result is not similar, further judge whether the shooting time and shooting location of the pictures are close. If so, dynamically adjust the similarity threshold and use the adjusted similarity threshold to perform depth similarity judgment; if not, output the result that the pictures are not similar;

[0122] By analyzing the shooting time and location, it is determined whether the two pictures may have been taken in similar scenes. If so, the similarity threshold is increased so that pictures that are slightly different can also be judged as similar.

[0123] Furthermore, step S3 specifically includes:

[0124] S31: extracting shooting timestamp and shooting location information from metadata of the image, where the shooting location information is geographic coordinates recorded by GPS data;

[0125] Metadata is usually included in image files. The system extracts shooting time and GPS coordinate information by reading these metadata fields. This can be achieved through standard image processing libraries or tools. The extraction of timestamps and GPS data is to provide a basic comparison standard. This information is an objective record and is not affected by human factors. Therefore, it can improve the accuracy and credibility of judgments as a basis for judgment.

[0126] S32: Shooting time judgment: set a time threshold, calculate the shooting time difference between the two pictures, if the shooting time difference between the two pictures is less than or equal to the time threshold, it is considered that the shooting time of the two pictures is close, and the similarity threshold is increased by a%;

[0127] The purpose of setting the time threshold is to infer that the pictures may be taken in the same or similar environment through temporal proximity. This setting can reduce the misjudgment caused by slightly different times, especially for a series of pictures taken quickly, which can better capture the potential similarities between pictures.

[0128] Shooting location judgment: set a distance threshold and calculate the distance between the geographic coordinates of the two pictures. If the shooting locations of the two pictures are less than or equal to the distance threshold, it is considered that the shooting locations of the two pictures are close, and the similarity threshold is increased by b%;

[0129] The system can use the Haversine formula to calculate the distance on the earth's surface between two points to evaluate the location difference. For example, if the shooting location of picture A is (30.2672, -97.7431) and the shooting location of picture B is (30.2674, -97.7433), the geographical distance between the two is calculated and a location threshold (such as 500 meters) is set. If the shooting locations of the two pictures are within this distance, the shooting locations of the two pictures are considered to be close.

[0130] The purpose of setting the distance threshold is to further confirm whether the pictures are taken in similar scenes through the proximity of geographical locations. The proximity of locations usually means the similarity of scenes or environments, thus reasonably improving the similarity judgment criteria and reducing incorrect classification.

[0131] If the shooting time and shooting location of the two pictures are close, the similarity threshold is increased by c%, and c>b>a;

[0132] This setting is to emphasize the impact of double proximity. Being close in time and location means that the pictures may have been taken in very similar scenes, so significantly increasing the similarity threshold can more accurately capture the weak similarities between pictures. In implementation, the system first determines the proximity of the shooting time and shooting location separately. If both conditions are met, the system will adjust the similarity threshold by c% instead of simply adding a% and b%; if only one of the conditions is met, a% or b% will be increased accordingly.

[0133] S33: re-evaluate the similarity using the adjusted similarity threshold;

[0134] The purpose of re-evaluating the similarity is to incorporate time and location factors into the final similarity determination. By dynamically adjusting the similarity threshold, the accuracy of the evaluation is improved, especially in complex scenarios or conditions, this method can effectively avoid misjudgment.

[0135] S4: Depth similarity judgment: The local features of the image are extracted through a convolutional neural network, and weighted fused with the global features extracted by the histogram algorithm. The fused features are input into the twin network to calculate the depth similarity score, which is compared with the adjusted similarity threshold. If the depth similarity score is greater than the adjusted similarity threshold, the output result is that the images are similar, otherwise the output result is that the images are not similar.

[0136] In a specific embodiment, step S4 includes:

[0137] S41: Extracting local features from images using convolutional neural networks:

[0138] Preprocess the image, resize it to a standard size of 224×224 pixels, and normalize the pixel values ​​to the range of [0,1];

[0139] Use a multi-scale convolutional neural network to perform multi-scale convolution operations on the image. At each scale, convolution operations and pooling operations are performed to obtain feature maps of different scales.

[0140] An attention module is introduced at each position of the feature map, the attention weight is calculated, and it is multiplied pixel by pixel with the original feature map to generate a weighted feature map;

[0141] The feature map weighted by the attention mechanism is flattened, the two-dimensional feature map is converted into a one-dimensional feature vector, and the local feature vector F is generated through the fully connected layer. CNN-增强 .

[0142] S42: Histogram algorithm to extract global features:

[0143] The frequency of each interval in the normalized global histogram is used as the global feature representation F 全局 , the global eigenvector F is 全局 Sparse coding is performed, the formula is:

[0144]

[0145] Among them, D is the dictionary matrix, Θ is the sparse representation matrix, T 0 is the sparsity threshold, which is used to limit the kth sparse vector θ k The number of non-zero elements in ;

[0146] Perform content analysis on the image to identify the target object and background area, and introduce an adaptive weight learning mechanism to train weight parameters through a gradient descent optimization strategy to dynamically adjust the weights of different dimensions in the global feature representation. The formula is:

[0147]

[0148] Where W is the weight matrix; L(·) is the cross entropy loss function or the mean square error loss function; is the global feature vector of the jth image; y (j) is the label of the jth image, y=1 means similar, y=0 means dissimilar; J is the number of images;

[0149] The final generated weight matrix W is applied to the sparse representation matrix Θ after sparse coding to obtain the optimized global eigenvector F Hist-全局 .

[0150] S43: Weighted fusion of local features and global features:

[0151] The local feature vector F extracted by the convolutional neural network CNN-增强 and the sparsely coded global feature vector F Hist-全局 Perform nonlinear weighted combination, the formula is as follows:

[0152] F 融合 =ω·tanh(F CNN-增强 )+λ·tanh(F Hist-全局 );

[0153] Among them, F 融合 is the fused feature representation; ω and λ are the adaptive weight parameters obtained through training, and tanh(·) is the hyperbolic tangent function, which is used to map the local feature representation vector to the interval [-1,1];

[0154] Introducing the residual connection mechanism, the local feature vector F CNN-增强 and the global eigenvector F Hist-全局 And the fused feature representation F 融合 Perform weighted combination to obtain the final fusion feature vector F 融合-增强 , the formula is:

[0155] F 融合-增强 =F 融合 +γ·(F CNN-增强 +F Hist-全局 );

[0156] Among them, γ is the residual weight coefficient.

[0157] S44: Input the fused features into the twin network to calculate the deep similarity score:

[0158] The final fusion feature vector F 融合-增强 Input into a Siamese network with a multi-layer embedding structure, where each layer of the embedding structure consists of multiple fully connected layers and activation functions;

[0159] After each layer of the twin network is embedded, a recursive neural network is introduced for time series modeling to capture the temporal relationship between features;

[0160] When training the twin network, the contrastive learning loss function is introduced to further optimize the distance metric between features, so that the embedding vector distance of similar images is minimized and the distance of non-similar images is maximized. The contrastive learning loss function formula is:

[0161]

[0162] Where d is the distance between two embedding vectors, m is the distance threshold of the embedding vectors; y is the label, y=1 means similarity, y=0 means dissimilarity;

[0163] Calculate two embedding vectors E 1 、E 2 The improved Euclidean distance between them is as follows:

[0164]

[0165] Among them, E 1 、E 2 are the two embedding vectors output from the twin network; E 1i 、E 2i Denote the embedding vector E 1 、E 2 The i-th eigenvalue in i is the weight coefficient corresponding to the i-th eigenvalue; n represents the dimension of the embedding vector;

[0166] D 改进 (E 1 ,E 2 ) is the embedding vector E 1 、E 2 The modified Euclidean distance between ;

[0167] The modified Euclidean distance is used as the depth similarity score and compared with the adjusted similarity threshold.

[0168] like Figure 2 As shown, another embodiment of the present invention provides a similar image detection system based on multi-feature fusion and dynamic adjustment strategy, including:

[0169] A processor for performing computing operations;

[0170] A memory for storing image data and program codes;

[0171] An image processing module, used to execute the similarity detection method based on multi-feature fusion and dynamic adjustment strategy as described above, and perform similarity detection on the images;

[0172] The output module is used to output the results of similar image detection and determine whether the images are similar;

[0173] The image processing module specifically includes:

[0174] The picture sorting module is used to sort the pictures in the album in chronological order;

[0175] Preliminary similarity judgment module: used to calculate the perceptual hash value and histogram hash value of two pictures respectively through the perceptual hash algorithm and the histogram algorithm, and then calculate the Hamming distance between the perceptual hash values ​​and the Hamming distance between the histogram hash values ​​respectively, and compare the two Hamming distances with the preset similarity threshold to evaluate the global similarity;

[0176] The shooting information analysis module is used to further determine whether the shooting time and shooting location of the pictures are close, and dynamically adjust the similarity threshold;

[0177] Local feature extraction module, used to extract local features of images through convolutional neural networks;

[0178] The global feature optimization module is used to perform sparse coding processing and adaptive weight adjustment on the global features of the image to generate an optimized global feature vector;

[0179] The deep similarity judgment module is used to extract the local features of the image using the convolutional neural network, perform weighted fusion with the global features extracted by the histogram algorithm, input the fused features into the twin network to calculate the deep similarity score, and compare it with the adjusted similarity threshold to determine whether the images are similar.

[0180] In summary, the present invention can more efficiently locate and compare pictures taken in a similar time period by sorting the pictures in the album in chronological order, thereby improving the accuracy of similarity detection, and also helps to optimize the allocation of computing resources and avoid unnecessary global similarity calculations; by using the perceptual hash algorithm and the histogram algorithm to respectively calculate the perceptual hash value and the histogram hash value of the picture, and performing a preliminary similarity evaluation based on the Hamming distance, it is possible to quickly identify globally similar or dissimilar pictures, greatly reducing the number of pictures entering the deep similarity judgment, thereby saving computing resources and time; when the preliminary judgment result is dissimilar, the similarity judgment is assisted by comparing the shooting time and location of the pictures. If the shooting time and location of the two pictures are close, it is possible that the shooting content is similar but not recognized by the preliminary algorithm. At this time, the similarity threshold is dynamically adjusted, which increases the flexibility and accuracy of the system in specific situations and further improves the reliability of detection. In deep similarity judgment, local features of the image are extracted through convolutional neural networks and weightedly fused with global features extracted by the histogram algorithm. The fused features are input into the twin network for similarity scoring. By combining the advantages of local and global features, similar images in complex situations can be identified more accurately, and a high similarity judgment accuracy can be maintained even when there are partial differences or lighting changes in the images.

[0181] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, it can be implemented in whole or in part in the form of a computer program product, which includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0182] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A similar image detection method based on multi-feature fusion and dynamic adjustment strategy, characterized in that: The method comprises: sorting the pictures in the album in chronological order, and performing similarity detection on any two pictures in the album; Preliminary similarity judgment: The perceptual hash value and histogram hash value of the two images are calculated by the perceptual hash algorithm and the histogram algorithm respectively, and then the Hamming distance between the perceptual hash values ​​and the Hamming distance between the histogram hash values ​​are calculated respectively. The two Hamming distances are compared with the preset similarity threshold to evaluate the global similarity; If the judgment result is not similar, then further determine whether the shooting time and shooting location of the pictures are similar. If so, dynamically adjust the similarity threshold and use the adjusted similarity threshold to perform deep similarity judgment; if not, the output result is that the pictures are not similar; Deep similarity judgment: The local features of the image are extracted through a convolutional neural network, and weighted fused with the global features extracted by the histogram algorithm. The fused features are input into the twin network to calculate the deep similarity score, which is compared with the adjusted similarity threshold. If the deep similarity score is greater than the adjusted similarity threshold, the output result is that the images are similar, otherwise the output result is that the images are not similar.

2. According to claim 1, a similar image detection method based on multi-feature fusion and dynamic adjustment strategy is characterized in that: The perceptual hash value of the image is calculated using the perceptual hash algorithm. The method includes: Preprocess the image, scale it to a standard size of 32×32, and convert it into a grayscale image; Reduce the image bit depth from 256 gray levels to 64 levels; The image is divided into blocks, each block is subjected to discrete cosine transform, a frequency coefficient matrix is ​​generated, and the DCT coefficients of the low-frequency part are selected as the extracted image frequency features; Calculating the average value of the DCT coefficients of the low-frequency part to generate an average threshold; Each DCT coefficient is compared with the average value threshold, and the hash bit corresponding to the DCT coefficient above the average value threshold is set to 1, otherwise it is set to 0, and a fixed-length binary hash string is generated as the perceptual hash value of the image.

3. According to claim 2, a similar image detection method based on multi-feature fusion and dynamic adjustment strategy is characterized in that: The histogram hash value of the image is calculated using the histogram algorithm. The method includes: Convert the image's color space from RGB mode to three independent channels for processing; Construct histogram: Count the pixel values ​​of the three RGB channels independently, divide each channel into 32 intervals according to the method of dividing [0,256) into [0,8), count the frequency of occurrence of pixel values ​​in each interval, and form a global histogram; Normalization processing: convert each interval frequency value of the histogram into a normalized value so that the sum of all frequency values ​​is equal to 1; simplification processing: divide the normalized value of the histogram into 16 intervals, generate a binary hash bit for each interval, and finally form a fixed-length hash string as the histogram hash value.

4. According to claim 3, a similar image detection method based on multi-feature fusion and dynamic adjustment strategy is characterized in that: The comprehensive evaluation of the global similarity by combining the two Hamming distances specifically includes: The two Hamming distances are combined into a comprehensive similarity score using weighted summation or simple averaging, where The weighted summation formula is: comprehensive similarity score = α × Hamming distance of perceptual hash value + β × Hamming distance of histogram hash value, where α and β are weight coefficients; The simple average formula is: comprehensive similarity score = (Hamming distance of perceptual hash value + Hamming distance of histogram hash value) / 2; compare the comprehensive similarity score with the preset similarity threshold. If the comprehensive similarity score is greater than the similarity threshold, the two pictures are judged to be similar in global features, and the system directly outputs the result that the pictures are similar; if not, the two pictures are judged to be quite different in global features, and the next step of analysis is performed based on time and location factors.

5. According to claim 1, a similar image detection method based on multi-feature fusion and dynamic adjustment strategy is characterized in that: The method of determining whether the shooting time and shooting location of the pictures are close, and if so, dynamically adjusting the similarity threshold comprises: Extracting shooting timestamp and shooting location information from metadata of the image, wherein the shooting location information is geographic coordinates recorded by GPS data; Shooting time judgment: set a time threshold, calculate the shooting time difference between two pictures, if the shooting time difference between the two pictures is less than or equal to the time threshold, it is considered that the shooting time of the two pictures is close, and the similarity threshold is increased by a%; Shooting location judgment: set a distance threshold, calculate the distance between the geographical coordinates of the two pictures, if the shooting locations of the two pictures are less than or equal to the distance threshold, then the shooting locations of the two pictures are considered to be close, and the similarity threshold is increased by b%; If the shooting time and shooting location of the two pictures are close, the similarity threshold is increased by c%, and c>b>a; Re-evaluate the similarity using the adjusted similarity threshold.

6. The similar image detection method based on multi-feature fusion and dynamic adjustment strategy according to claim 1 is characterized in that: Methods for extracting local features from images using convolutional neural networks include: Preprocess the image, resize it to a standard size of 224×224 pixels, and normalize the pixel values ​​to the range of [0,1]; Use a multi-scale convolutional neural network to perform multi-scale convolution operations on the image. At each scale, convolution operations and pooling operations are performed to obtain feature maps of different scales. An attention module is introduced at each position of the feature map, the attention weight is calculated, and it is multiplied pixel by pixel with the original feature map to generate a weighted feature map; The feature map weighted by the attention mechanism is flattened, the two-dimensional feature map is converted into a one-dimensional feature vector, and the local feature vector F is generated through the fully connected layer. CNN-增强 .

7. The similar image detection method based on multi-feature fusion and dynamic adjustment strategy according to claim 3 is characterized in that: The methods of extracting global features by histogram algorithm include: The frequency of each interval in the normalized global histogram is used as the global feature representation F 全局 , the global eigenvector F is 全局 Sparse coding is performed, the formula is: Where D is the dictionary matrix, Θ is the sparse representation matrix, and T0 is the sparsity threshold used to limit the kth sparse vector θ k The number of non-zero elements in ; Perform content analysis on the image to identify the target object and background area, and introduce an adaptive weight learning mechanism to train weight parameters through a gradient descent optimization strategy to dynamically adjust the weights of different dimensions in the global feature representation. The formula is: Where W is the weight matrix; L(·) is the cross entropy loss function or the mean square error loss function; is the global feature vector of the jth image; y (j) is the label of the jth image, y=1 indicates similarity, y=0 indicates dissimilarity; J represents the number of images; the final generated weight matrix W is applied to the sparse representation matrix Θ after sparse coding to obtain the optimized global feature vector F Hist-全局 .

8. The similar image detection method based on multi-feature fusion and dynamic adjustment strategy according to claim 1 is characterized in that: Methods for weighted fusion of local features and global features include: The local feature vector F extracted by the convolutional neural network CNN-增强 and the sparsely coded global feature vector F Hist-全局 Perform nonlinear weighted combination, the formula is as follows: F 融合 =ω·tanh(F CNN-增强 )+λ·tanh(F Hist-全局 ); Among them, F 融合 is the fused feature representation; ω and λ are the adaptive weight parameters obtained through training, and tanh(·) is the hyperbolic tangent function, which is used to map the local feature representation vector to the interval [-1,1]; Introducing the residual connection mechanism, the local feature vector F CNN-增强 and the global eigenvector F Hist-全局 And the fused feature representation F 融合 Perform weighted combination to obtain the final fusion feature vector F 融合-增强 , the formula is: F 融合-增强 =F 融合 +γ·(F CNN-增强 +F Hist-全局 ); Among them, γ is the residual weight coefficient.

9. The similar image detection method based on multi-feature fusion and dynamic adjustment strategy according to claim 8 is characterized in that: Methods for inputting the fused features into the Siamese network to calculate the deep similarity score include: The final fusion feature vector F 融合-增强 Input into a Siamese network with a multi-layer embedding structure, where each layer of the embedding structure consists of multiple fully connected layers and activation functions; After each layer of the twin network is embedded, a recursive neural network is introduced for time series modeling to capture the temporal relationship between features. When training the twin network, a contrastive learning loss function is introduced to further optimize the distance metric between features, so that the embedding vector distance of similar images is minimized and the distance of non-similar images is maximized. The formula of the contrastive learning loss function is: Where d is the distance between two embedding vectors, m is the distance threshold of the embedding vectors; y is the label, y=1 means similarity, y=0 means dissimilarity; Calculate the modified Euclidean distance between two embedding vectors E1 and E2. The formula is as follows: Among them, E1 and E2 are two embedding vectors output from the twin network; E 1i 、E 2i Respectively represent the i-th eigenvalue in the embedding vectors E1 and E2; w i is the weight coefficient corresponding to the i-th eigenvalue; n represents the dimension of the embedding vector; D 改进 (E1, E2) is the modified Euclidean distance between the embedding vectors E1 and E2; The modified Euclidean distance is used as a depth similarity score and compared with the adjusted similarity threshold.

10. A similar image detection system based on multi-feature fusion and dynamic adjustment strategy, characterized in that: The system includes: a processor for performing computing operations; A memory for storing image data and program codes; An image processing module, used to execute the similarity detection method of any one of claims 1 to 9 to perform similarity detection on the images; The output module is used to output the results of similar image detection and determine whether the images are similar; The image processing module specifically includes: The picture sorting module is used to sort the pictures in the album in chronological order; Preliminary similarity judgment module: used to calculate the perceptual hash value and histogram hash value of two pictures respectively through the perceptual hash algorithm and the histogram algorithm, and then calculate the Hamming distance between the perceptual hash values ​​and the Hamming distance between the histogram hash values ​​respectively, and compare the two Hamming distances with the preset similarity threshold to evaluate the global similarity; The shooting information analysis module is used to further determine whether the shooting time and shooting location of the pictures are close, and dynamically adjust the similarity threshold; Local feature extraction module, used to extract local features of images through convolutional neural networks; The global feature optimization module is used to perform sparse coding processing and adaptive weight adjustment on the global features of the image to generate an optimized global feature vector; The deep similarity judgment module is used to extract the local features of the image using the convolutional neural network, perform weighted fusion with the global features extracted by the histogram algorithm, input the fused features into the twin network to calculate the deep similarity score, and compare it with the adjusted similarity threshold to determine whether the images are similar.

Citation Information

Patent Citations

  • Image retrieval method, system and device and medium

    CN112632314A

  • Method and device for determining threshold value for face image comparison and electronic equipment

    CN116737974A

  • Natural fermented starter and method of manufacturing fermented bread using it

    KR102740779B1

  • Method and electronic device for retrieving an image and computer readable storage medium

    US20200242422A1

  • Facial feature detection

    US8515139B1

Cited By

  • Work meeting picture uploading control method and system based on image similarity detection

    CN120910298A

  • PPT static page detection method and system based on efficient compression and similarity judgment

    CN121033460A