Image similarity detection method and device

Abstract

The invention discloses an image similarity detection method and device, and belongs to the technical field of image processing. The method comprises the following steps: firstly, quickly de-duplicating an original picture set by utilizing perceptual hash and a color histogram algorithm to obtain a candidate picture set; then, an improved OfficientViT network is adopted to extract image features, the network captures global context information and local details respectively through parallel self-attention branches and multi-scale local feature branches, and the global context information and the local details are spliced in a channel dimension; further, multi-level features in the network are integrated through a cross-layer feature fusion module, and the cross-layer feature fusion module performs weighted fusion on the features by using adaptive convolution and a channel attention mechanism; and finally, calculating cosine similarity based on the fused feature vectors, and judging whether the images are similar or not according to a preset threshold value. According to the invention, the accuracy and efficiency of image similarity detection are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image similarity detection method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, many classic algorithms have emerged in image similarity technology. These algorithms can be categorized as follows: Algorithms for calculating image similarity based on traditional techniques commonly include pixel-by-pixel comparison algorithms, histogram comparison methods, SSIM algorithms, and hash algorithms. While these methods show certain advantages in image similarity calculation and are computationally efficient, they rely on low-level image features and are therefore susceptible to interference from external factors such as changes in image illumination, rotation, scaling, and positional shifts, resulting in low robustness in similarity calculations.

[0003] Image similarity acquisition using deep learning technology has become increasingly prevalent in recent years, with the advancement of deep learning. By leveraging techniques such as convolutional neural networks (CNNs), attention neural networks (NNNs), and Transformer networks, significant differences in features between two images can be extracted, directly outputting their similarity and greatly improving the accuracy and stability of image similarity assessment. From the initial CNNs to the Transformer based on global feature modeling, and now to the latest efficient hybrid architectures, CNNs, with their local receptive fields, extract local features of an image through convolutional kernels, excelling in handling detailed information and local changes, but exhibiting weaker perception of global information. Transformer networks introduce a self-attention mechanism, capturing global feature associations in an image and significantly improving the ability to model overall structural changes; however, their high computational complexity limits their speed in high-resolution scenes. Most methods employ a fusion strategy of first extracting local features using convolutional blocks and then extracting global features using Transformer blocks. However, such sequential or parallel combinations are not deep enough to fully utilize the advantages of CNNs and visual Transformers. EfficientViT combines the lightweight design of EfficientNet with the global feature extraction capabilities of ViT, achieving a balance between speed and performance through a multi-scale attention mechanism (MSA), making it suitable for large-scale image similarity detection scenarios. However, EfficientViT's multi-scale attention mechanism may have certain limitations when processing multi-resolution and multi-level features simultaneously. For example, it may struggle to effectively fuse features at different scales, leading to some features not being fully utilized and thus reducing the model's similarity detection performance. Therefore, there is still room for improvement in EfficientViT's handling of angle and local structural changes. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] To address this, this invention discloses an image similarity detection method. This method utilizes perceptual hashing and color histogram algorithms to quickly deduplicate the original image set, generating a candidate image set. Feature extraction is performed based on an improved EfficientViT network, which extracts global and local features through parallel self-attention branches and multi-scale local feature branches, and then fuses them. A cross-layer feature fusion module integrates features from different levels, employing adaptive convolution and channel attention mechanisms to weight the feature vectors. Finally, cosine similarity is calculated based on the weighted fused feature vectors, and image similarity is determined according to a preset threshold. This achieves accurate and efficient detection of image similarity.

[0006] Another objective of this invention is to provide an image similarity detection device.

[0007] The third objective of this invention is to provide a computer device.

[0008] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0009] To achieve the above objectives, the present invention provides an image similarity detection method, comprising: S1, using the perceptual hash algorithm and color histogram algorithm to quickly remove duplicates from the original image set and generate a candidate image set; S2, feature extraction is performed on the candidate image set based on the improved EfficientViT network structure; wherein, the EfficientViT network structure includes a parallel self-attention branch and a multi-scale local feature extraction branch; the self-attention branch uses a standard multi-head self-attention mechanism based on Softmax to model global features; the multi-scale local feature extraction branch uses parallel multi-scale deep convolution to extract local features, and concatenates the global features and local features in the channel dimension; S3 uses a cross-layer feature fusion module to fuse features from different layers in the improved EfficientViT network structure. The cross-layer feature fusion module includes a global average pooling operation, an adaptive convolutional layer, and a channel attention weight generation unit. The adaptive convolutional layer dynamically adjusts the kernel size according to the channel dimension. The channel attention weight generation unit generates attention weights using the Sigmoid function and then weights and fuses the feature vectors. S4 calculates the cosine similarity based on the weighted fused feature vectors, and determines whether the images are similar according to a preset threshold.

[0010] An image similarity detection method according to an embodiment of the present invention may also have the following additional technical features: In one embodiment of the present invention, the step of using a perceptual hash algorithm and a color histogram algorithm to quickly deduplicate and filter the original image set to generate a candidate image set includes: S11 treats the image's PHash value as a fixed-length binary vector, calculates the color histogram and normalizes it to a floating-point vector, then directly concatenates the binary vector and the floating-point vector to form a comprehensive feature vector, using the following formula: ;in, It is a binary vector of fixed length. It is a normalized floating-point vector; S12, using the cosine similarity formula Calculate the similarity of the combined feature vectors; if the cosine similarity is greater than 0.99, it is considered a duplicate image. , These represent the combined feature vectors of the two images.

[0011] In one embodiment of the present invention, feature extraction of a candidate image set is performed based on an improved EfficientViT network structure, including: S21, the multi-scale local feature extraction branch is composed of parallel depthwise convolutions with kernel sizes of 1×1, 3×3, and 5×5, and the corresponding output is represented as follows. ;in, , , This represents convolutions with depths of 1×1, 3×3, and 5×5. , , These represent query, key, and value tensors, respectively. S22, the global features output by the self-attention branch. With multi-scale local features The channel concatenation operation `Concat` combines the channels, and the output feature representation is as follows: .

[0012] In one embodiment of the present invention, a cross-layer feature fusion module is used to fuse features from different layers in the improved EfficientViT network structure, including: S31, uses global average pooling to aggregate shallow and deep features of the input; S32 uses an adaptive convolutional layer to determine the kernel size, which is dynamically calculated using a function. ; in, Let t be the nearest odd neighbor. and Setting them to 2 and 1 respectively corresponds to the slope and intercept in a linear relationship. This represents the number of channels for the input feature; S33 generates channel attention weights using the Sigmoid function and performs a dot product operation with the original features to obtain the final fused features.

[0013] In one embodiment of the present invention, it further includes: S5. The HNSW algorithm is used to construct an index and perform approximate nearest neighbor retrieval on the high-dimensional feature vectors generated by the improved EfficientViT: S51, store the high-dimensional feature vector in the HNSW index structure and construct the initial index; S52: For the query image, the set of candidate high-dimensional feature vectors is quickly retrieved using the HNSW index, and then cosine similarity is used for accurate calculation.

[0014] To achieve the above objectives, another aspect of the present invention provides an image similarity detection device, comprising: The fast deduplication and filtering module is used to quickly deduplicatize the original image set using the perceptual hash algorithm and the color histogram algorithm, and generate a candidate image set. The feature extraction module is used to extract features from the candidate image set based on the improved EfficientViT network structure. The EfficientViT network structure includes a parallel self-attention branch and a multi-scale local feature extraction branch. The self-attention branch uses a standard multi-head self-attention mechanism based on Softmax to model global features. The multi-scale local feature extraction branch uses parallel multi-scale deep convolution to extract local features and concatenates the global features and local features in the channel dimension. The cross-layer feature fusion module is used to fuse features from different layers in the improved EfficientViT network structure. The cross-layer feature fusion module includes a global average pooling operation, an adaptive convolutional layer, and a channel attention weight generation unit. The adaptive convolutional layer dynamically adjusts the kernel size according to the channel dimension. The channel attention weight generation unit generates attention weights through the Sigmoid function and then weights and fuses the feature vectors. The similarity calculation and determination module is used to calculate cosine similarity based on the weighted fused feature vectors and determine whether the images are similar according to a preset threshold.

[0015] In one embodiment of the present invention, it further includes: The index optimization module is used to construct an index and perform approximate nearest neighbor retrieval on the high-dimensional feature vectors generated by the improved EfficientViT algorithm using the HNSW algorithm. The high-dimensional feature vectors are stored in the HNSW index structure and an initial index is constructed. For the query image, the set of candidate high-dimensional feature vectors is quickly retrieved using the HNSW index, and then cosine similarity is used for accurate calculation.

[0016] This invention discloses an image similarity detection method and apparatus. By integrating rapid deduplication screening, improved neural network feature extraction, and cross-layer feature fusion mechanisms, it effectively overcomes the technical bottlenecks of existing technologies, such as the difficulty in simultaneously considering global features and local details, and the challenge in balancing computational efficiency and retrieval accuracy. This method achieves end-to-end optimization from initial deduplication, multi-scale feature extraction, adaptive feature fusion to efficient similarity calculation, significantly improving the accuracy and retrieval efficiency of image similarity detection. Furthermore, by introducing an approximate nearest neighbor retrieval mechanism, it enhances the practicality and scalability of the method on large-scale image datasets, providing reliable technical support for image retrieval and recognition applications.

[0017] To achieve the above objectives, a third aspect of this application provides a computer device, including a processor and a memory; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, for implementing an image similarity detection method as described in the first aspect embodiment.

[0018] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an image similarity detection method as described in the first aspect.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of an image similarity detection method according to an embodiment of the present invention; Figure 2 This is a flowchart of an image similarity detection hardware according to an embodiment of the present invention; Figure 3 This is a network structure diagram of an image similarity detection method according to an embodiment of the present invention; Figure 4 This is a detailed structural diagram of the multi-scale local-global attention mechanism of an image similarity detection method according to an embodiment of the present invention; Figure 5This is a detailed structural diagram of a cross-layer feature fusion module in an image similarity detection method according to an embodiment of the present invention; Figure 6 This is an overall flowchart of an image plagiarism detection system according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an image similarity detection device according to an embodiment of the present invention; Figure 8 It is a computer device according to an embodiment of the present invention. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] The following description, with reference to the accompanying drawings, describes an image similarity detection method, apparatus, device, and storage medium according to embodiments of the present invention.

[0024] The core idea of ​​this invention is to construct a hierarchical detection architecture that integrates rapid screening, multi-scale local-global feature extraction, and adaptive feature fusion, combining the fast comparison capability of perceptual hashing with the deep representation capability of an improved visual Transformer. First, efficient image deduplication is achieved using perceptual hashing and color histograms to generate a candidate set. Then, through parallel self-attention mechanisms and multi-scale deep convolutional branches, global semantic information and local detail features of the image are captured respectively, and effectively concatenated at the channel dimension. Further, a cross-layer feature fusion module is introduced, dynamically integrating multi-layer feature representations in the network using adaptive convolutional kernels and channel attention mechanisms. Finally, similarity measurement and judgment are performed based on the weighted fused feature vectors. This technical approach elevates traditional single-feature-dependent similarity detection to an end-to-end solution that balances computational efficiency and discrimination accuracy, effectively solving the challenge of balancing accuracy and real-time performance in large-scale image similarity detection, and providing reliable technical support for applications such as image retrieval and copyright recognition.

[0025] Example 1 To achieve the above invention, embodiments of the present invention provide an image similarity detection method, such as... Figure 1 As shown, it includes: S1 uses a perceptual hash algorithm and a color histogram algorithm to quickly remove duplicates from the original image set and generate a candidate image set.

[0026] Specifically, this step aims to eliminate obviously duplicated or highly similar images by using feature extraction methods with low computational complexity, thereby significantly reducing the computational overhead of subsequent deep feature extraction and similarity calculation.

[0027] Specifically, the PHash algorithm generates a fixed-length binary hash vector based on the global visual features of an image. Its core process includes: first, converting the image to grayscale; then performing a Discrete Cosine Transform (DCT) to retain low-frequency coefficients that reflect the overall structural features of the image; and finally, generating a hash value through threshold quantization. The PHash vector is typically 64 bits long, suitable for fast hash comparison. Simultaneously, the color histogram algorithm divides pixel values ​​into several intervals (usually 32 or 64 intervals) by statistically analyzing the frequency information of color distribution in the image, and calculates the number of pixels in each interval, forming a normalized floating-point vector. This vector reflects the color distribution characteristics of the image, enhancing its sensitivity to color changes.

[0028] Furthermore, the PHash vector and the color histogram vector are concatenated along the feature dimension to form a comprehensive feature vector. This vector integrates the image's structural and color information, enabling a more comprehensive representation of the image's visual features. Subsequently, the cosine similarity formula is used... Perform similarity calculations on image pairs. Set a similarity threshold of 0.99; if the calculated result is greater than this threshold, the image is identified as a duplicate and is removed.

[0029] Specifically, this step is widely applicable to the preprocessing stage of large-scale image databases, especially in insurance claims scenarios where users may upload a large number of duplicate or similar supporting images, such as medical reports and on-site photos. Using this method, the system can complete preliminary screening within milliseconds, significantly improving processing efficiency.

[0030] Specifically, by combining PHash and color histogram algorithms, the efficiency of traditional methods is retained while improving robustness to color changes. Its innovation lies in the optimization of the comprehensive feature vector construction method and the similarity determination strategy, providing high-quality input data for subsequent deep feature extraction based on the improved EfficientViT, thus achieving a balance between computational cost and detection accuracy in the overall process.

[0031] Furthermore, S1 includes: S11 treats the image's PHash value as a fixed-length binary vector, calculates the color histogram and normalizes it to a floating-point vector, then directly concatenates the binary vector and the floating-point vector to form a comprehensive feature vector, using the following formula: ;in, It is a binary vector of fixed length. It is a normalized floating-point vector.

[0032] Specifically, this step effectively compensates for the shortcomings of the traditional PHash algorithm in being insensitive to color information by combining the global structural features and color distribution features of the image, thereby achieving efficient and accurate deduplication and similarity judgment in large-scale image processing.

[0033] Furthermore, the PHash algorithm extracts the low-frequency energy distribution of the image by converting it to grayscale and performing a Discrete Cosine Transform (DCT), ultimately generating a fixed-length binary hash vector, typically 64 or 128 bits. This vector reflects the overall visual structure of the image and is suitable for fast hash comparison. Meanwhile, the color histogram divides pixel values ​​into several intervals (e.g., 32 or 64 intervals) by statistically analyzing the pixel distribution of each color channel (e.g., RGB or HSV) in the image, and calculates the number of pixels in each interval. To eliminate the influence of image size and brightness differences, the histogram needs to be normalized, converting it into a floating-point vector, so that the vector's... The norm is 1. Finally, the PHash binary vector and the normalized color histogram vector are concatenated along the feature dimension to form a comprehensive feature vector. This is used for subsequent similarity calculations.

[0034] Furthermore, the length of the PHash vector is typically 64 bits or 128 bits, depending on the image scaling and the range of DCT coefficients selected. The number of intervals in the color histogram. The normalization value is typically set to 32 or 64 to balance computational complexity and feature representation capability. Normalization, i.e. This ensures that the vectors are distributed across the unit sphere. The concatenated composite feature vector... The dimension is or Specifically, it depends on the configuration of PHash and histogram.

[0035] Specifically, this step is widely used in image deduplication modules of insurance claims systems. When users upload a large number of images, the system first performs rapid extraction and stitching of PHash and color histograms on all images to form preliminary feature representations. Then, it uses the cosine similarity formula... The method calculates the similarity between image pairs and sets a threshold of 0.99. Images with similarity exceeding this threshold are considered duplicates. This method demonstrates good robustness in handling common image transformations such as rotation, scaling, and partial occlusion, while significantly reducing the computational burden of subsequent deep feature extraction stages.

[0036] Specifically, this step constructs a comprehensive feature vector that takes into account both structural and color information by fusing PHash and color histograms, thus improving the accuracy and recall in the initial screening stage. In large-scale image databases, this method can complete the initial screening within milliseconds, providing a high-quality candidate set for subsequent deep feature extraction based on the improved EfficientViT, thereby optimizing the overall efficiency and performance of image similarity detection.

[0037] S12, using the cosine similarity formula Calculate the similarity of the combined feature vectors; if the cosine similarity is greater than 0.99, it is considered a duplicate image. , These represent the combined feature vectors of the two images.

[0038] Specifically, this step is based on the concatenated composite feature vector. ,in, A perceptual hash vector, typically a 64-bit or 128-bit binary vector, is used to characterize the global structural features of an image. The normalized color histogram vector, typically a 256-dimensional floating-point vector, is used to describe the color distribution characteristics of an image. By concatenating these two types of features in vector space, a unified feature representation is formed. This integrates the structural and color information of the image in terms of feature dimensions, thereby improving the robustness and accuracy of the initial screening.

[0039] Furthermore, the cosine similarity formula is defined as: ; in, This represents the dot product operation of two eigenvectors. and Let represent the Euclidean norms of the two vectors. This formula measures the directional similarity by calculating the cosine of the angle between the two vectors, with a value range of [-1, 1]. The closer the value is to 1, the more similar the two images are.

[0040] Specifically, the threshold for determining cosine similarity is set at 0.99. When the calculated... When the system determines that the image pair is a duplicate image, the threshold is set based on a balance between experimental verification and the needs of real-world scenarios. This ensures the ability to recognize highly duplicate images while avoiding misjudgments caused by noise or slight variations.

[0041] Specifically, after rapid filtering using perceptual hashing and color histograms, the system uses cosine similarity to make a final judgment on candidate image pairs, ensuring that only highly similar image pairs are retained for subsequent deep feature extraction and precise localization stages. This method has significant application value in insurance claims scenarios, effectively identifying duplicate or forged images uploaded by users, thereby improving review efficiency and risk control capabilities. Furthermore, when this step is combined with the HNSW algorithm, it can further optimize retrieval performance in large-scale image databases, achieving collaborative work between rapid near-nearest neighbor filtering and precise similarity calculation.

[0042] S2, feature extraction is performed on the candidate image set based on the improved EfficientViT network structure; wherein, the EfficientViT network structure includes a parallel self-attention branch and a multi-scale local feature extraction branch; the self-attention branch adopts the standard multi-head self-attention mechanism based on Softmax to model global features; the multi-scale local feature extraction branch adopts parallel multi-scale deep convolution to extract local features, and concatenates the global features and local features in the channel dimension.

[0043] Specifically, the improved structure (IMEfficientViT) of this step integrates a self-attention mechanism and a multi-scale local feature extraction module, thereby significantly enhancing the ability to model global dependencies and multi-scale local features of the image while maintaining the model's lightweight nature.

[0044] Furthermore, IMEfficientViT introduces parallel self-attention branches and multi-scale local feature extraction branches on top of the original EfficientViT. Specifically, the input image first extracts preliminary features through convolutional blocks and then undergoes layer normalization to improve the stability and learnability of the features. Subsequently, learnable feature vectors are generated through group convolution mapping. The feature vector A is divided into Q, K, and V according to the image dimensions. This feature vector is then divided into two parts: one half... Used for the self-attention branch, the other half This branch is used for multi-scale local feature extraction. The self-attention branch employs a standard multi-head attention mechanism, calculating global features through the Softmax function. Its output can be expressed as: ; in, , , These are query, key, and value vectors, respectively. This represents the channel dimension of the key vector. This branch is primarily responsible for modeling long-range semantic dependencies in the image, enhancing the model's robustness to changes in the overall structure.

[0045] Furthermore, the multi-scale local feature extraction module consists of parallel depthwise separable convolutions, with kernel sizes of [sizes to be filled in]. , and These correspond to local feature extraction at different scales. The output of this module is: ; in, , , This is the feature vector input to the local feature extraction module. This branch, through multi-scale convolution operations, can effectively capture local details such as edges and textures in the image, thereby improving the model's ability to perceive changes in local structure.

[0046] Furthermore, the global features output by the self-attention branch With multi-scale local features The features are concatenated along the channel dimension to form the final multi-scale local-global attention mechanism output features. Its expression is: .

[0047] Specifically, this stitching operation increases the feature dimension, enabling the model to utilize both global semantic information and multi-scale local features simultaneously, thereby improving the accuracy and robustness of image similarity judgment.

[0048] Specifically, in the self-attention mechanism The value of is usually consistent with the number of channels in the feature map. For example, in the intermediate layers of EfficientViT, the number of channels can be set to 256 or 512. In the multi-scale convolution module, the number of output channels in each branch is kept consistent to ensure the feasibility of the concatenation operation. In addition, the number of groups in the grouped convolution can be adjusted according to the actual hardware resources, usually set to 8 or 16, to achieve a balance between computational efficiency and feature representation capability.

[0049] Specifically, this step is mainly applied to the deep feature extraction stage of the candidate image set. In insurance claims scenarios, user-uploaded images may undergo complex transformations such as rotation, scaling, and partial occlusion, which traditional feature extraction methods struggle to handle effectively. IMEfficientViT, through its parallel architecture design, can improve its ability to model multi-scale features while maintaining high inference speed, thereby significantly enhancing the model's recognition performance under complex transformations.

[0050] Specifically, by introducing a multi-scale local-global attention mechanism, the shortcomings of the original EfficientViT in local feature extraction are effectively addressed, improving the model's adaptability to changes in image structure. Simultaneously, the channel-dimensional concatenation operation enhances feature expressiveness, providing higher-quality feature vectors for subsequent cosine similarity calculations, thereby significantly improving the accuracy and robustness of image similarity detection.

[0051] Furthermore, S2 includes: S21, the multi-scale local feature extraction branch is composed of parallel depthwise convolutions with kernel sizes of 1×1, 3×3, and 5×5, and the corresponding output is represented as follows. ;in, , , This represents convolutions with depths of 1×1, 3×3, and 5×5. , , These represent query, key, and value tensors, respectively. Specifically, this module, as part of a multi-scale local-global attention mechanism, processes feature vectors in parallel with the self-attention mechanism branch, extracting global dependency information and multi-scale local texture information of the image, respectively.

[0052] Specifically, this branch first receives the feature vectors that have undergone layer normalization. The feature vectors are then mapped to learnable feature representations through grouped convolutions. The image is divided into multiple sub-feature maps according to its spatial dimensions. Half of the features are fed into the self-attention mechanism branch, while the other half are fed into the multi-scale local feature extraction module. Specifically, this module consists of three parallel deep convolutional layers with kernel sizes of [sizes to be filled in]. , and These layers are used to extract local features at different scales from images, including point features, edges, and region structures. Each convolutional layer employs a depthwise separable convolution structure to reduce computational complexity while maintaining feature representation capabilities. In further processing, the output features of each branch are summed element-wise to form a multi-scale local feature vector. Its expression is: ; in, , , These represent the query, key, and value parts of the input feature vector, respectively, with DWConv representing the depthwise convolution operation. The output channel number of this module is the same as the input, ensuring that it can be concatenated with the output of the self-attention mechanism branch in the channel dimension.

[0053] Furthermore, the stride of the convolutional kernel is typically set to 1 to maintain spatial resolution; the number of channels is consistent with the input features to avoid information loss. Additionally, this module is usually deployed in the intermediate layers of EfficientViT to capture the shallow texture information extracted by the initial convolution and fuse it with deep semantic features.

[0054] Specifically, this module is widely used in systems requiring high-precision image similarity assessment, such as image retrieval, content moderation, and insurance claims. Especially when images undergo transformations such as local occlusion, rotation, and scaling, multi-scale local feature extraction can effectively capture the structural details of the image, thereby improving the model's robustness to complex transformations.

[0055] Specifically, this module achieves multi-granularity modeling of local image features by paralleling depthwise convolutions of different scales, thus overcoming the shortcomings of traditional attention mechanisms in local perception. Compared with structures that rely solely on global attention, this design significantly enhances the model's adaptability to changes in local image structure while improving feature representation capabilities, providing a richer feature foundation for subsequent feature fusion and similarity calculation.

[0056] S22, the global features output by the self-attention branch. With multi-scale local features The channel concatenation operation `Concat` combines the channels, and the output feature representation is as follows: .

[0057] Specifically, the core of this step lies in proposing a mechanism for dynamically determining the convolutional kernel size. This is achieved through a mechanism that uses the number of input feature channels... Function with independent variable Calculate. The mathematical expression of this function is: The calculation result is then rounded to an odd value. This design makes the kernel size no longer a pre-fixed hyperparameter, but a dynamic variable that can adaptively adjust according to changes in the channel complexity of the input feature map, thus achieving a greater degree of flexibility in the model structure.

[0058] Furthermore, this design directly addresses the challenge posed by the significant differences in the number of feature channels across different layers in deep neural networks. In the feature fusion module, the number of channels for shallow and deep features typically differs, making it difficult for fixed-size convolutional kernels to achieve optimal feature extraction results across all layers. This solution adaptively adjusts the convolutional kernel size to ensure that, regardless of whether processing low-level high-resolution, low-channel features or high-level low-resolution, multi-channel semantic features, the convolutional operation can effectively extract and fuse feature information with a matching receptive field, enhancing the model's ability to capture features at different scales.

[0059] Specifically, it fundamentally solves the limitations of fixed-size convolutional kernels in traditional convolutional operations for cross-layer feature fusion scenarios. By associating the kernel size with the channel dimension of the input features, this technique achieves more refined and adaptive feature processing capabilities. This not only significantly improves the expressive power of the feature fusion module, enabling the fused feature vector to contain richer and higher-quality information, but also, as an innovation in the entire network structure, lays a crucial technical foundation for improving the accuracy of final image similarity detection, demonstrating the significant progress of the invention.

[0060] S3 uses a cross-layer feature fusion module to fuse features from different layers in the improved EfficientViT network structure. The cross-layer feature fusion module includes a global average pooling operation, an adaptive convolutional layer, and a channel attention weight generation unit. The adaptive convolutional layer dynamically adjusts the kernel size according to the channel dimension. The channel attention weight generation unit generates attention weights using the Sigmoid function and then weights and fuses the feature vectors.

[0061] Specifically, this step consists of three parts: global average pooling operation, adaptive convolutional layer, and channel attention weight generation unit. Its technical implementation principle is based on the collaborative optimization and dynamic weighting mechanism of multi-scale features.

[0062] Specifically, in the input stage of feature fusion, the cross-layer fusion module receives feature maps from different layers of the network, including shallow features (such as low-level texture and edge information) and deep features (such as semantic information and structural features). To achieve global information integration of cross-layer features, the module employs Global Average Pooling (GAP) to compress feature maps from different layers into channel vectors in spatial dimension, thereby extracting globally representative feature information. The output dimension of the GAP operation is... That is, the number of channels in the feature map.

[0063] Furthermore, the adaptive convolutional layer adjusts according to the channel dimension. The convolution kernel size is dynamically adjusted to adapt to the spatial distribution characteristics of features at different levels. The kernel size is determined by a function. Adaptive calculation, its formula is: ; in, Let t be the nearest odd neighbor. and Setting them to 2 and 1 respectively corresponds to the slope and intercept in a linear relationship. The value represents the number of channels in the input features, ensuring that the convolutional kernel size has good computational efficiency on actual hardware. This adaptive mechanism effectively avoids the feature alignment bias that may be introduced in cross-layer fusion with a fixed convolutional kernel.

[0064] Furthermore, the channel attention weight generation unit weights the fused features using a sigmoid function. Specifically, this unit first uses a 1D convolutional layer to reduce the channel dimension, generating channel attention weight vectors. Its length is Then, through the Sigmoid function... right Perform nonlinear activation to obtain normalized attention weights. Ultimately, Perform a channel-wise dot product operation with the original feature map to generate the fused feature output. The calculation method is as follows: .

[0065] Specifically, in practical applications, this module can be deployed between multiple intermediate layers of the improved EfficientViT network to fuse multi-scale features extracted from CNN and Transformer structures. For example, after a multi-scale local-global attention mechanism, the output features of different Transformer blocks can be fused across layers to enhance the model's ability to perceive changes in image structure.

[0066] Specifically, this cross-layer feature fusion module significantly improves the hierarchy and relevance of feature representation by dynamically adjusting the convolution kernel size and introducing a channel attention mechanism. Experiments show that this module can effectively suppress noise interference and enhance the model's ability to focus on key features when handling complex transformations such as image rotation, scaling, and local occlusion, thereby improving the accuracy and generalization ability of image similarity judgment. This step is one of the key technical links in achieving high-precision, low-latency image similarity detection in this invention and has significant engineering practical value.

[0067] Furthermore, S3 includes: S31 uses global average pooling to aggregate shallow and deep features of the input.

[0068] Specifically, this step introduces a global average pooling operation to uniformly aggregate the input shallow and deep features. Specifically, global average pooling calculates the mean along the spatial dimension of each feature map, compressing it into a single value, thereby mapping features at different levels to feature vectors of fixed dimensions. This method does not rely on additional parameters, effectively preserves the overall response intensity of each feature channel, achieves information fusion of multi-level features, and significantly reduces feature dimensionality, which is beneficial for the efficient computation of subsequent processing modules.

[0069] Specifically, this feature aggregation method is applicable to various computer vision tasks, such as image classification, object detection, and semantic segmentation. By fusing shallow detail information with deep semantic information, the generated feature representation possesses both local fine structure and global context awareness, which helps improve the model's recognition accuracy and robustness for multi-scale targets, especially demonstrating stronger generalization ability in complex scenes.

[0070] Furthermore, global average pooling, as a parameter-free operation, does not introduce any trainable weights; it only reduces the dimensionality and summarizes information from the input feature map. Its pooling window automatically adapts to the spatial size of the input feature map, achieving averaging of the response across the entire image, thus possessing high adaptability and being suitable for input images of different resolutions.

[0071] Specifically, by introducing global average pooling into the feature extraction network, a compact representation of multi-level features is effectively achieved, enhancing the model's ability to extract key features and its robustness against interference. This design not only reduces computational complexity and avoids the risk of overfitting, but also improves the stability and interpretability of the entire network structure, providing important technical support for the construction of high-performance vision systems.

[0072] S32 uses an adaptive convolutional layer to determine the kernel size, which is determined by a function. Dynamic calculation, in which The channel dimension, and Set them to 2 and 1 respectively; where, This represents the number of channels in the input feature. and These correspond to the slope and intercept in a linear relationship, respectively.

[0073] Specifically, the kernel size is determined by the function Dynamic calculation, in which This indicates the channel dimension of the current feature map. and These represent the slope and intercept in the linear relationship, respectively, and are empirically set to 2 and 1. The function adaptively selects a suitable convolution kernel size based on the number of channels in the feature map, ensuring it is odd to satisfy the requirements of symmetry and center alignment in the convolution operation.

[0074] Specifically, in the cross-layer fusion module, shallow and deep features are first compressed in spatial dimension using Global Average Pooling (GAP) to obtain feature vectors with two channel dimensions. These two feature vectors are then input into an adaptive convolutional layer, the kernel size of which is determined by the aforementioned function. It is calculated dynamically. For example, when the number of channels... hour, After taking the nearest odd neighbor, the result is 3, therefore the convolution kernel size is 3. This adaptive mechanism can automatically adjust the convolution kernel size according to the number of channels in the feature map, thereby maintaining good receptive field matching and information transfer efficiency in feature fusion at different levels.

[0075] Furthermore, the features output by the adaptive convolutional layer are adjusted for channel dimensions through 1D convolution and then input into a sigmoid activation function to generate channel attention weights. These weights reflect the correlation strength between shallow and deep features, and are subsequently subjected to channel-wise dot product operations with the original features to obtain the final fused features. This fusion method has stronger expressive power than the traditional shortcut connection, effectively suppressing irrelevant features and enhancing the expression of key features, thereby improving the robustness and accuracy of image similarity detection.

[0076] Specifically, through adaptive convolutional kernel design, the system can maintain the flexibility and effectiveness of feature fusion across different channel dimensions. Especially when processing multi-scale, multi-resolution images, it can adaptively adjust the local receptive field, enhancing the model's adaptability to changes in image structure. In insurance claims scenarios, this technical solution can effectively identify similar or forged images uploaded by users, significantly improving the system's detection accuracy and operational efficiency.

[0077] S33 generates channel attention weights using the Sigmoid function and performs a dot product operation with the original features to obtain the final fused features.

[0078] Specifically, this step is a key component of the cross-layer feature fusion module, designed to enhance the correlation between shallow texture features and deep structural features, thereby improving the robustness and accuracy of image similarity detection.

[0079] Specifically, this step first performs global average pooling (GAP) on both shallow and deep features to obtain statistical information about the feature maps in the spatial dimension. Then, these statistical features are fed into an adaptive convolutional layer, the kernel size of which is determined by a function. The adaptive determination is based on the following formula: ; in, Let t be the nearest odd neighbor. and Setting them to 2 and 1 respectively corresponds to the slope and intercept in a linear relationship. This represents the number of channels in the input features. This function ensures that the kernel size has a non-linear relationship with the number of channels, thus maintaining the flexibility and effectiveness of feature fusion under different channel configurations.

[0080] Furthermore, the feature vector output by the adaptive convolutional layer is further adjusted in channel dimension using 1D convolution to match the number of channels in the original feature map. Then, the adjusted feature vector is input into the Sigmoid function to generate channel attention weights. The Sigmoid function outputs in the range [0, 1], assigning a weight value between 0 and 1 to each channel, representing the importance of that channel in the fusion process. The generation process of channel attention weights can be represented as follows: ; in, This represents the shallow and deep features of the input. For channel attention weights, This refers to the Sigmoid function.

[0081] Specifically, this step is mainly used in the image feature fusion stage, especially in insurance claims scenarios where user-uploaded images may have variations such as partial occlusion, rotation, and scaling. By introducing a channel attention mechanism, the system can adaptively enhance its response to key features and suppress the influence of irrelevant or interfering features, thereby improving the model's robustness to complex transformations.

[0082] Specifically, this step achieves non-uniform fusion of shallow and deep features by assigning dynamic weights to different channels, avoiding the simple superposition of feature information in traditional quick connection methods, thereby improving the richness and discriminative power of feature representation. The final fused features exhibit higher semantic consistency in subsequent cosine similarity calculations, which helps to more accurately identify similar images.

[0083] S4 calculates the cosine similarity based on the weighted fused feature vectors, and determines whether the images are similar according to a preset threshold.

[0084] Specifically, at the technical implementation level, this step first normalizes the high-dimensional feature vectors extracted by the IMEfficientViT network to eliminate the influence of vector length on similarity calculation. Specifically, let the feature vectors of the two images be... and Its dimensions are Then the formula for calculating cosine similarity is: ; in, Represents the dot product of two vectors. and Let and represent the Euclidean norms of the two vectors, respectively. This formula assesses the similarity of image content by measuring the cosine of the angle between the two vectors in a high-dimensional space, reflecting the consistency of their directions.

[0085] Specifically, this step sets the threshold for cosine similarity to 0.99. When the calculated... If two images are similar, they are considered similar; otherwise, they are considered dissimilar. This threshold is set based on the sensitivity requirements for image forgery in insurance claims scenarios, and is combined with experimental verification and actual test data to ensure a balance between high precision and high recall. Furthermore, the dimension of the feature vector... The value is typically set to 128 or 256, depending on the output configuration of the IMEfficientViT network, to balance feature representation capability and computational efficiency.

[0086] Specifically, this step is widely used in insurance industry image deduplication systems. For example, when users upload images such as medical reports and accident scene photos, the system uses this step to quickly identify whether there are duplicate or highly similar images, thus assisting reviewers in determining whether fraudulent activity exists. In large-scale image databases, this step can be combined with the HNSW algorithm, first performing approximate nearest neighbor retrieval, and then performing precise cosine similarity calculation on candidate image pairs, significantly improving retrieval efficiency and accuracy.

[0087] Specifically, this step overcomes the sensitivity of traditional methods to changes in illumination, rotation, and scaling by introducing high-dimensional semantic features and normalization processing, thus improving the robustness and generalization ability of image similarity judgment. Simultaneously, by combining a multi-scale local-global attention mechanism and a cross-layer fusion module, the feature vectors can more comprehensively express the structural and semantic information of the image, thereby maintaining high recognition accuracy even in complex transformation scenarios. As the final decision-making step in the entire image similarity detection process, this step plays a crucial role in the overall performance of the system, demonstrating the dual advantages of this invention in terms of accuracy and efficiency.

[0088] S5 uses the HNSW algorithm to construct an index and perform approximate nearest neighbor retrieval on the high-dimensional feature vectors generated by the improved EfficientViT.

[0089] Specifically, HNSW is an efficient approximate nearest neighbor search algorithm based on a hierarchical graph structure. Its core idea is to construct a multi-layer graph structure so that the candidate vector most similar to the query vector can be quickly located in a high-dimensional space, thereby significantly improving the efficiency and scalability of large-scale image retrieval.

[0090] Specifically, firstly, the candidate image set is subjected to deep feature extraction using the IMEfficientViT network to generate fixed-length high-dimensional feature vectors (e.g., 512-dimensional or 768-dimensional). These vectors can accurately represent the semantic information and global dependencies of the images. Then, all feature vectors are input into the HNSW indexing module, which, based on set parameters (e.g., maximum number of layers),... The hierarchical graph structure is constructed using the number of entry points (ef_{construction}) and the expansion factor (ef_{search}) during retrieval. The number of connections for each node in each layer of the graph is controlled. ef_{construction} determines the search scope during index construction, while ef_{search} affects the balance between recall quality and speed during retrieval.

[0091] Furthermore, the HNSW retrieval process is divided into two stages: the first stage is coarse-grained retrieval, which quickly locates candidate regions through the upper-level graph structure; the second stage is fine-grained retrieval, which further searches for the most similar approximate neighbors in the lower-level graph. In this invention, the candidate feature vectors returned by HNSW will be input into the cosine similarity calculation module, with the formula as follows: .

[0092] Furthermore, the system filters out image pairs with high similarity based on preset thresholds (such as 0.95 or 0.98) for subsequent similarity determination and display.

[0093] Specifically, through HNSW's efficient indexing and near nearest neighbor retrieval, the system can quickly locate potentially similar images in image databases of millions or even hundreds of millions of images, significantly reducing computational complexity. Simultaneously, by combining the high-dimensional semantic features extracted by IMEfficientViT, HNSW can improve the accuracy and robustness of similarity calculation while maintaining retrieval efficiency, making it particularly suitable for insurance claims scenarios where images undergo complex transformations such as rotation, scaling, and partial occlusion.

[0094] Furthermore, S5 includes: S51, store the high-dimensional feature vector in the HNSW index structure and construct the initial index.

[0095] Specifically, HNSW is an Approximate Nearest Neighbor (ANN) search algorithm based on a hierarchical graph structure. Its core idea is to build a multi-layer graph structure so that the candidate image most similar to the query vector can be found quickly in a high-dimensional feature space with low computational cost.

[0096] Specifically, this step first extracts deep features from the candidate image set using the IMEfficientViT network, generating fixed-length high-dimensional feature vectors (e.g., 512-dimensional or 768-dimensional). These feature vectors are then imported into the HNSW index structure as a vector database. The HNSW index construction process includes two main stages: hierarchical graph construction and optimization of the connection strategy. In hierarchical construction, the algorithm first inserts all feature vectors into the highest layer (entry layer), and then gradually expands to lower layers according to the connection strategy, forming a multi-layered graph network. Nodes in each layer connect to their nearest neighbors, forming navigation paths, thus enabling rapid navigation to the target region during the search.

[0097] Furthermore, the construction of HNSW involves several key parameters, including the number of levels in the entry layer (M), the maximum number of connections (efConstruction), and the distance metric. In this invention, cosine similarity is used as the distance metric, and its calculation formula is as follows: ; in, and These represent the feature vectors of the two images. When building the index, the entry layer is typically set to 16 levels and the maximum number of connections to 64, to strike a balance between retrieval efficiency and accuracy. Furthermore, HNSW supports dynamic insertion and updates, making it suitable for online image retrieval scenarios.

[0098] Specifically, this step is widely used in large-scale image deduplication and similarity detection in insurance claims systems. In actual deployment, the system can import the feature vectors of millions of images into the HNSW index at once, and after construction, it can support fast retrieval. For example, in the catalog batch detection mode, the system can index the entire image library, and subsequent queries only need to input the feature vector of the target image to return the set of candidate images with the highest similarity in milliseconds.

[0099] Specifically, this step significantly improves the efficiency and scalability of image retrieval through the HNSW index structure. Compared to traditional linear search or KD-Tree structures, HNSW offers superior search performance in high-dimensional spaces, making it particularly suitable for the high-dimensional semantic features extracted using the improved EfficientViT method in this invention. By constructing a high-quality initial index, the system can achieve rapid response to large-scale image databases while maintaining the accuracy of similarity calculations, thereby effectively supporting the real-time risk control needs of the insurance industry in claims review.

[0100] S52: For the query image, the set of candidate high-dimensional feature vectors is quickly retrieved using the HNSW index, and then cosine similarity is used for accurate calculation.

[0101] Specifically, this step achieves efficient approximate nearest neighbor search in a large-scale image database, significantly reducing the computational overhead of full comparison of feature vectors while ensuring the accuracy of the final similarity judgment.

[0102] Specifically, HNSW is an efficient approximate nearest neighbor search algorithm based on a hierarchical graph structure. Its core idea is to construct a multi-layered graph structure, where each layer has fewer node connections, allowing for rapid navigation to the target region during the search. In practice, the system first extracts high-dimensional feature vectors from all training or processed images using the IMEfficientViT network, typically 512-dimensional or 768-dimensional floating-point vectors. These feature vectors are then used to construct the HNSW index structure, where the construction of each graph layer is based on the number of connections. The system employs an entry point selection strategy. During the retrieval phase, the system inputs the feature vector of the query image into the HNSW index, and through a multi-level graph skip search, quickly locates the image with the closest matching query features. There are candidate feature vectors, among which The value is typically set between 100 and 500, depending on the size of the database and the required precision.

[0103] Furthermore, HNSW performance is affected by several key parameters, including the maximum number of connections. Maximum number of floors, number of entry points and the number of nearest neighbors during the search In this invention, It is usually set to 16 or 32. Set to 100 to strike a balance between retrieval speed and recall. Furthermore, the HNSW index is built using either Euclidean distance or cosine similarity, with cosine similarity being preferred to ensure consistency with subsequent precise calculations.

[0104] Specifically, this step is widely used in large-scale image deduplication and similarity detection in insurance claims systems. For example, after a user uploads a claim image, the system can quickly retrieve similar images from the database to identify duplicate submissions, forged images, or tampered images. HNSW's efficient retrieval capabilities enable the system to achieve millisecond-level response times in a database of tens of thousands to millions of images, significantly improving the automation level and processing efficiency of image review.

[0105] Specifically, this step uses an approximate nearest neighbor search of the HNSW index to transform what would originally require... The linear search complexity is reduced to This significantly reduces the consumption of computing resources. Simultaneously, the precise calculation of cosine similarity ensures the accuracy of the final similarity judgment. This combined strategy combines efficiency and robustness in large-scale image retrieval scenarios, providing a crucial guarantee for achieving both real-time performance and accuracy optimization in image similarity detection systems.

[0106] This invention discloses an image similarity detection method that effectively overcomes the technical bottlenecks of existing technologies, such as the difficulty in balancing detection efficiency and accuracy, and the challenge in coordinating global features and local details, by integrating rapid deduplication screening, improved neural network feature extraction, and cross-layer feature fusion mechanisms. This method achieves end-to-end optimization from initial deduplication, multi-scale feature extraction, adaptive feature fusion to efficient similarity calculation, significantly improving the accuracy and retrieval efficiency of similarity detection on large-scale image sets. Furthermore, by introducing an approximate nearest neighbor retrieval mechanism, it enhances the scalability and real-time performance of the method in practical applications, providing reliable technical support for image retrieval and recognition applications.

[0107] Example 2 To achieve the above invention, embodiments of the present invention also provide a specific process for an image similarity detection method, including: Specifically, firstly, a candidate image set is generated. Perceptual hashing and color histogram algorithms are used to quickly deduplicate a large number of images, retaining only those with high similarity as input for the next stage, thus reducing overall computational cost. Subsequently, in the feature vector generation stage, IMEfficientViT (Improved-EfficientViT) is used to extract deep features from the candidate image set, generating a high-dimensional feature vector for each image to fully express its semantic features and global dependencies. Finally, cosine similarity is used to calculate the similarity value of the extracted feature vectors, and a set threshold is used to determine whether the images are similar.

[0108] Furthermore, IMEfficientViT is specifically based on the EfficientViT network structure. On the basis of designing the CNN and visual Transformer serial structure, it constructs a multi-scale local-global attention mechanism. This mechanism integrates a multi-scale local feature extraction module on the basis of the multi-head attention mechanism, thereby realizing the deep integration of CNN and visual Transformer to fully mine multi-scale information, shallow texture information and deep structural information pairs.

[0109] Furthermore, the core idea of ​​perceptual hashing (PHash) is to generate hash values ​​based on the global visual features of an image, and to determine the degree of similarity between two images by comparing the similarity of their hash values. However, its main limitation is that PHash processes the brightness (grayscale) of an image and is not sensitive to color features, easily overlooking color information. This invention combines PHash values ​​with color histogram features to improve the accuracy of the initial screening.

[0110] Furthermore, the histogram method statistically analyzes the distribution of image colors, dividing the pixel value range into several intervals. The number of pixels in each interval is the statistical measure for that interval. The Bhattacharyya distance is used to calculate the correlation of the pixel distribution histogram, and the formula is as follows: ; in, , This represents the number of intervals the histogram divides the data into. The smaller the value, the higher the correlation; the minimum value is 0, and the maximum value is 1.

[0111] Specifically, the PHash value is treated as a fixed-length binary vector, and the color histogram is calculated and normalized to a floating-point vector. The two are then directly concatenated into a comprehensive feature vector, as shown in the formula: .

[0112] Furthermore, the cosine similarity is then used to calculate the similarity between the two, using the following formula: .

[0113] Specifically, a threshold of 0.99 is set; images with a cosine similarity greater than this threshold are identified as duplicates. After removing duplicate images, the candidate image set is further fed into the improved EfficientVit network to extract deeper features.

[0114] Specifically, the EfficientViT module consists of a lightweight multi-scale global self-attention (MSA) module and an MBConv module. The lightweight MSA module is responsible for capturing global information, while MBConv is responsible for capturing local information.

[0115] Specifically, MSA abandons the complex softmax attention mechanism and adopts ReLU linear attention to operate on the global receptive field. Compared with softmax attention, ReLU linear attention cannot produce a sharp attention distribution due to the lack of a non-linear similarity function, and its local information extraction capability is weaker than that of softmax attention. Therefore, to improve the local feature extraction capability of EfficientViT, this invention constructs a scale-local-global attention mechanism.

[0116] Furthermore, the specific structure of the multi-scale local-global attention mechanism is as follows: First, the features extracted from the convolutional blocks are normalized by layers, and then a learnable feature vector is generated through a grouped convolutional mapping. Secondly, the feature vectors are divided according to the image dimensions. Divided into , and and half of them , and The other half is fed into the self-attention mechanism branch. , and The data is fed into the multi-scale local feature extraction module branch. The self-attention mechanism is responsible for mining global features of the image, while the multi-scale local feature extraction module is responsible for capturing multi-scale local information. The output of the self-attention mechanism branch can be represented as... ; in, Representing the eigenvector The multi-scale feature extraction module is designed with a channel dimension of 1×1, 3×3, and 5×5 depthwise convolutions in parallel, which can extract local feature information at multiple scales. The output of this module... Represented as: .

[0117] Finally, capture the global features from both branches. and multi-scale local features Channel concatenation is performed to obtain the output features of a multi-scale local-global attention mechanism. : ; Concat represents a concatenation operation at the channel level.

[0118] Specifically, different targets in an image possess rich texture information and complex overall structures, and the effective extraction and fusion of features at different levels is beneficial to improving detection performance. Therefore, this invention stacks multiple multi-scale visual Transformers to mine shallow texture information and deep structural information of images. Secondly, to effectively combine shallow information and deep features and enhance their correlation, this invention uses a cross-layer fusion module to fuse feature information from different levels in the network. First, global average pooling is used to aggregate the shallow and deep features of the input. Second, adaptive convolutional layer depth is used to fuse shallow and deep features. Specifically, to determine the optimal value of the convolutional kernel, a function is used... The size of the convolution kernel is selected adaptively, and the formula is: ; in, Represents the nearest odd neighbor. Let r represent the channel dimension, and b correspond to the slope and intercept in a linear relationship, respectively, empirically set to 2 and 1. Then, a 1D convolutional layer is used to adjust the channel dimension, and the channel attention weights are obtained through the sigmoid function. Finally, the channel attention weights are multiplied by the original features to obtain the final fused features. Compared to directly using shortcut connections for feature fusion, the cross-layer feature fusion method achieves effective fusion of shallow and deep features by assigning feature attention weights.

[0119] Finally, cosine similarity is used to calculate feature similarity, and the similarity values ​​are sorted in descending order.

[0120] This invention discloses a specific process for an image similarity detection method. By constructing a hierarchical detection architecture that integrates rapid deduplication screening, multi-scale feature extraction, and cross-layer feature fusion, it effectively overcomes the technical bottlenecks of existing technologies, such as the difficulty in balancing detection efficiency and accuracy, and the difficulty in coordinating global features and local details. It achieves end-to-end optimization from initial screening and deep feature extraction to adaptive feature fusion, significantly improving the accuracy and retrieval efficiency of image similarity detection. Furthermore, through an improved neural network structure and feature fusion mechanism, it enhances the method's adaptability to complex image transformations, providing reliable technical support for large-scale image retrieval and recognition applications.

[0121] Example 3 To achieve the above-mentioned invention, embodiments of the present invention also provide a scenario application of the image similarity detection method, including: Completely duplicate images are quickly filtered out using perceptual hashing and color histogram algorithms. Among the images filtered in the first step, a more in-depth similarity evaluation is performed using an optimized EfficientViT algorithm. A multi-scale local-global attention mechanism is constructed, which adds multi-scale convolutional modules in parallel to its lightweight multi-scale MSA to enhance local feature extraction capabilities and further improve the overall accuracy and robustness of the network, while maintaining efficient computational performance to meet the dual requirements of accuracy and real-time performance in practical applications.

[0122] The embodiments of this invention evaluated the proposed method on the collected dataset, showing good performance compared to most traditional image similarity comparison algorithms. Compared to other methods, it achieves higher accuracy and faster processing time on large-scale image databases. Specifically, it includes: Scenario Application 1 (e.g.) Figure 2 (as shown) Specifically, the image to be detected is input into a fast deduplication module, which consists of a perceptual hashing algorithm and a color histogram algorithm. The core idea of ​​perceptual hashing (PHash) is to generate hash values ​​based on the global visual features of an image, and to determine the degree of similarity by comparing the similarity of the hash values ​​of two images. However, its main limitation is that PHash processes the brightness (grayscale) of an image and is not sensitive to color features, easily overlooking color information. This invention combines PHash values ​​and color histogram features to calculate a comprehensive similarity, improving the accuracy of the initial screening.

[0123] Furthermore, the histogram method statistically analyzes the distribution of image colors, dividing the pixel value range into several intervals. The number of pixels in each interval is the statistical measure for that interval. The Bhattacharyya distance is used to calculate the correlation of the pixel distribution histogram, and the formula is as follows: ; in, , This represents the number of intervals the histogram divides the data into. The smaller the value, the higher the correlation; the minimum value is 0, and the maximum value is 1.

[0124] Specifically, the PHash value is treated as a fixed-length binary vector, and the color histogram is calculated and normalized to a floating-point vector. The two are then directly concatenated into a comprehensive feature vector, as shown in the formula: .

[0125] Furthermore, the cosine similarity is then used to calculate the similarity between the two, using the following formula: .

[0126] Specifically, a threshold of 0.99 is set; images with a cosine similarity greater than this threshold are identified as duplicates. After removing duplicate images, the candidate image set is further fed into the improved EfficientVit network to extract deeper features.

[0127] Furthermore, IMEfficientViT is specifically based on the EfficientViT network structure (such as...). Figure 3 As shown, based on the design of the CNN and visual Transformer serial structure, a multi-scale local-global attention mechanism is constructed. This mechanism integrates a multi-scale local feature extraction module on the basis of the multi-head attention mechanism, thereby realizing the deep integration of CNN and visual Transformer.

[0128] Specifically, the EfficientViT module consists of a lightweight multi-scale global self-attention (MSA) module and an MBConv module. The lightweight MSA module is responsible for capturing global information, while MBConv is responsible for capturing local information.

[0129] Specifically, MSA abandons the complex softmax attention mechanism and adopts ReLU linear attention to operate on the global receptive field. A comparison of Softmax and ReLU linear attention reveals that, unlike softmax attention, ReLU linear attention cannot produce a sharp attention distribution due to the lack of a non-linear similarity function. Its local information extraction capability is weaker than that of softmax attention. Therefore, to improve the local feature extraction capability of EfficientViT, this invention constructs a scale-based local-global attention mechanism.

[0130] Furthermore, embodiments of the present invention also propose a multi-scale local-global attention mechanism (such as...). Figure 4As shown in the diagram, firstly, the features extracted from the convolutional blocks are normalized by the layers, and then a learnable feature vector is generated through a grouped convolutional mapping. Secondly, the feature vectors are divided according to the image dimensions. Divided into , and and half of them , and The other half is fed into the self-attention mechanism branch. , and The data is fed into the multi-scale local feature extraction module branch. The self-attention mechanism is responsible for mining global features of the image, while the multi-scale local feature extraction module is responsible for capturing multi-scale local information. The output of the self-attention mechanism branch can be expressed as: ; in, Representing the eigenvector The multi-scale feature extraction module is designed with a channel dimension of 1×1, 3×3, and 5×5 depthwise convolutions in parallel, which can extract local feature information at multiple scales. The output of this module... Represented as: .

[0131] Finally, capture the global features from both branches. and multi-scale local features Channel concatenation is performed to obtain the output features of a multi-scale local-global attention mechanism. : ; Concat represents a concatenation operation at the channel level.

[0132] Specifically, different targets in an image possess rich texture information and complex overall structures, and the effective extraction and fusion of features at different levels is beneficial to improving detection performance. Therefore, this invention stacks multiple multi-scale visual Transformers to mine shallow texture information and deep structural information of images. Secondly, in order to effectively combine shallow information and deep features and enhance the correlation between them, this invention uses a cross-layer fusion module to fuse feature information from different levels in the network (such as... Figure 5 (As shown). First, global average pooling is used to aggregate the shallow and deep features of the input. Second, an adaptive convolutional layer depth is used to fuse the shallow and deep features. Specifically, to determine the optimal value of the convolutional kernel, a function is used... The size of the convolution kernel is selected adaptively, and the formula is: ; in, Represents the nearest odd neighbor. Let r represent the channel dimension, and b correspond to the slope and intercept in a linear relationship, respectively, empirically set to 2 and 1. Then, a 1D convolutional layer is used to adjust the channel dimension, and the channel attention weights are obtained through the sigmoid function. Finally, the channel attention weights are multiplied by the original features to obtain the final fused features. Compared to directly using shortcut connections for feature fusion, the cross-layer feature fusion method achieves effective fusion of shallow and deep features by assigning feature attention weights.

[0133] Finally, cosine similarity is used to calculate feature similarity, and the similarity values ​​are sorted in descending order.

[0134] Scenario Application 2: The application scenario in this embodiment is basically the same as that in Scenario Application Two, with the following differences: The process of training the network involves acquiring training image pairs and their corresponding label values, and training the image similarity recognition model based on these sample image pairs. Specifically, the training image pairs consist of two different sample images. When preparing the training data, similar pairs (positive samples) are image pairs with similar content or attributes, such as different angles or sizes of the same object; while dissimilar pairs (negative samples) are image pairs with completely different content, such as image pairs of different objects or different scenes. Image pairs from the same category are selected as positive samples, and image pairs from different categories are selected as negative samples. Simultaneously, to improve sample diversity and avoid overfitting, data augmentation techniques such as rotation, scaling, translation, and cropping are used to expand the dataset.

[0135] In this application scenario, a contrastive loss function is used for supervised learning of feature similarity, and the formula is as follows: ; in, It is the similarity between two images (calculated using cosine similarity). These are tags (1 indicates similarity, 0 indicates dissimilarity).

[0136] Specifically, to compare the similarity between the feature vectors of two images, a set of sample pairs is defined. Each sample pair consists of two images and their similarity labels, which are binary (similar / dissimilar). During training, the two images of each sample pair are first input into the IMEfficientViT network to extract their corresponding feature vectors. Then, the similarity is calculated using a contrastive loss function, and the corresponding loss value is generated. Finally, the network parameters are updated based on the backpropagation algorithm, allowing the model to be gradually optimized.

[0137] Specifically, by employing a contrastive loss function to supervise the learning of image feature similarity, the model can more accurately fit feature similarity in practical applications, providing effective support for image similarity recognition. After the model is trained, it is constructed as an ONNX model file for subsequent use.

[0138] Scenario Application 3: This scenario application is basically the same as Scenario Application 1, with the following differences: Image detection systems can perform image deduplication screening and similarity calculation in the following ways.

[0139] Batch Directory Detection: The system allows users to input an image search directory and click the "Start Detection" button. The system will then perform duplicate detection and similarity calculation on the images in the directory. After algorithm processing, the system will prioritize displaying duplicate images in a list, and then sort the image pairs in descending order based on the calculated similarity values, displaying only image pairs with a similarity exceeding a set threshold.

[0140] Single Image Detection: Users can upload a single image for comparison with images in the detection directory. After clicking the "Start Detection" button, the system will display a list of image pairs with a similarity exceeding a set threshold, showing the corresponding similarity value to the right of each image. The results are sorted in descending order of similarity value for easy viewing of the most similar images.

[0141] Scenario Application 4: This scenario application is basically the same as Scenario Application 1, with the following differences: The image detection system used in this scenario (such as...) Figure 6 The image shown employs the HNSW (Hierarchical Navigable Small World) algorithm for large-scale image retrieval. HNSW achieves efficient approximate nearest neighbor search through a hierarchical graph structure. The specific implementation process is as follows: In an embodiment of the present invention, feature vector generation and index construction are as follows: First, deep features are extracted from the input image using the IMEfficientViT network to generate high-dimensional feature vectors. Then, all processed image feature vectors are stored in the HNSW index structure, and an initial index is constructed. The HNSW index construction process fully utilizes its hierarchical graph structure, achieving efficient data storage and organization.

[0142] In an embodiment of the present invention, image retrieval: for a query image (or an image to be matched), the system quickly retrieves the approximate nearest neighbor feature vector that is closest to the features of the query image through the HNSW index, and generates a corresponding candidate image set.

[0143] In an embodiment of the present invention, the similarity is calculated precisely: the set of candidate feature vectors returned by HNSW is further calculated precisely using cosine similarity to filter out image pairs with similarity exceeding a set threshold.

[0144] In an embodiment of the present invention, the result output is as follows: the system outputs the final filtered similar image pairs for users to view or process later.

[0145] Specifically, by combining the efficient retrieval capabilities of the HNSW algorithm with the accurate calculation of cosine similarity, this embodiment can significantly improve the speed and performance of image retrieval while maintaining accuracy. This method is suitable for rapid deduplication and similarity detection scenarios in large-scale image databases.

[0146] The technical solution provided by the application scenarios of this invention can ensure the timeliness and accuracy of image similarity judgment even when images are rotated, scaled, partially occluded, shifted, or have different colors.

[0147] This invention discloses an image similarity detection method applicable to various scenarios. By constructing a hierarchical detection architecture that integrates rapid deduplication screening, multi-scale feature extraction, and cross-layer 65 feature fusion, it effectively overcomes the technical bottlenecks of existing technologies, such as the difficulty in balancing detection efficiency and accuracy, and the challenge of coordinating global features with local details. It achieves end-to-end optimization from initial screening and deep feature extraction to adaptive feature fusion, significantly improving the accuracy and retrieval efficiency of image similarity detection. Furthermore, through an improved neural network structure and feature fusion mechanism, it enhances the method's adaptability to complex image transformations, providing reliable technical support for large-scale image retrieval and recognition applications.

[0148] Example 4 To achieve the above invention, such as Figure 7 As shown, this embodiment also provides an image similarity detection device 10, which includes: The fast deduplication filtering module 100 is used to quickly deduplicate the original image set using a perceptual hash algorithm and a color histogram algorithm to generate a candidate image set.

[0149] The feature extraction module 200 is used to extract features from a candidate image set based on an improved EfficientViT network structure. The EfficientViT network structure includes a parallel self-attention branch and a multi-scale local feature extraction branch. The self-attention branch uses a standard multi-head self-attention mechanism based on Softmax to model global features. The multi-scale local feature extraction branch uses parallel multi-scale deep convolution to extract local features and concatenates the global features with the local features in the channel dimension.

[0150] The cross-layer feature fusion module 300 is used to fuse features from different layers in the improved EfficientViT network structure. The cross-layer feature fusion module includes a global average pooling operation, an adaptive convolutional layer, and a channel attention weight generation unit. The adaptive convolutional layer dynamically adjusts the kernel size according to the channel dimension. The channel attention weight generation unit generates attention weights through the Sigmoid function and weights the fused feature vectors.

[0151] The similarity calculation and judgment module 400 is used to calculate the cosine similarity based on the weighted fused feature vectors and determine whether the images are similar according to a preset threshold.

[0152] In one embodiment of the present invention, it further includes: an index optimization module, used to construct an index and perform approximate nearest neighbor retrieval on the high-dimensional feature vectors generated by the improved EfficientViT using the HNSW algorithm: storing the high-dimensional feature vectors in the HNSW index structure and constructing an initial index; quickly retrieving a set of candidate high-dimensional feature vectors for the query image using the HNSW index, and then using cosine similarity for accurate calculation.

[0153] This invention discloses an image similarity detection device that effectively overcomes the technical bottlenecks of existing technologies, such as the difficulty in balancing detection efficiency and accuracy, and the challenge of coordinating global features and local details, by constructing a modular architecture for rapid deduplication screening, deep feature extraction, and cross-layer feature fusion. The device achieves end-to-end optimization from initial screening, multi-scale feature extraction, adaptive feature fusion to efficient similarity calculation, significantly improving the accuracy of image similarity detection and system processing efficiency. Furthermore, by introducing an approximate nearest neighbor retrieval mechanism, it enhances the device's practicality and scalability in large-scale image datasets, providing a reliable system-level solution for image retrieval and recognition applications.

[0154] To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 8 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads the executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the image similarity detection method described above.

[0155] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an image similarity detection method as described in the foregoing embodiments.

[0156] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0157] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. An image similarity detection method, characterized in that, include: S1, using the perceptual hash algorithm and color histogram algorithm to quickly remove duplicates from the original image set and generate a candidate image set; S2, feature extraction is performed on the candidate image set based on the improved EfficientViT network structure; wherein, the EfficientViT network structure includes a parallel self-attention branch and a multi-scale local feature extraction branch; the self-attention branch uses a standard multi-head self-attention mechanism based on Softmax to model global features; the multi-scale local feature extraction branch uses parallel multi-scale deep convolution to extract local features, and concatenates the global features and local features in the channel dimension; S3 uses a cross-layer feature fusion module to fuse features from different layers in the improved EfficientViT network structure. The cross-layer feature fusion module includes a global average pooling operation, an adaptive convolutional layer, and a channel attention weight generation unit. The adaptive convolutional layer dynamically adjusts the kernel size according to the channel dimension. The channel attention weight generation unit generates attention weights using the Sigmoid function and then weights and fuses the feature vectors. S4 calculates the cosine similarity based on the weighted fused feature vectors, and determines whether the images are similar according to a preset threshold.

2. The method as described in claim 1, characterized in that, The process of using a perceptual hashing algorithm and a color histogram algorithm to quickly deduplicate and filter the original image set to generate a candidate image set includes: S11 treats the image's PHash value as a fixed-length binary vector, calculates the color histogram and normalizes it to a floating-point vector, then directly concatenates the binary vector and the floating-point vector to form a comprehensive feature vector, using the following formula: ;in, It is a binary vector of fixed length. It is a normalized floating-point vector; S12, using the cosine similarity formula Calculate the similarity of the combined feature vectors; if the cosine similarity is greater than 0.99, it is considered a duplicate image; where, , These represent the combined feature vectors of the two images.

3. The method as described in claim 1, characterized in that, Feature extraction of candidate image sets is performed based on the improved EfficientViT network structure, including: S21, the multi-scale local feature extraction branch is composed of parallel depthwise convolutions with kernel sizes of 1×1, 3×3, and 5×5, and the corresponding output is represented as follows. ;in, , , This represents convolutions with depths of 1×1, 3×3, and 5×5. , , These represent query, key, and value tensors, respectively. S22, the global features output by the self-attention branch. With multi-scale local features The channel concatenation operation `Concat` combines the channels, and the output feature representation is as follows: .

4. The method as described in claim 1, characterized in that, The improved EfficientViT network structure fuses features from different levels through a cross-layer feature fusion module, including: S31, uses global average pooling to aggregate shallow and deep features of the input; S32 uses an adaptive convolutional layer to determine the kernel size, which is dynamically calculated using a function. ； in, Let t be the nearest odd neighbor. and Setting them to 2 and 1 respectively corresponds to the slope and intercept in a linear relationship. This represents the number of channels for the input feature; S33 generates channel attention weights using the Sigmoid function and performs a dot product operation with the original features to obtain the final fused features.

5. The method as described in claim 1, characterized in that, Also includes: S5. The HNSW algorithm is used to construct an index and perform approximate nearest neighbor retrieval on the high-dimensional feature vectors generated by the improved EfficientViT: S51, store the high-dimensional feature vector in the HNSW index structure and construct the initial index; S52: For the query image, the set of candidate high-dimensional feature vectors is quickly retrieved using the HNSW index, and then cosine similarity is used for accurate calculation.

6. An image similarity detection device, characterized in that, include: The fast deduplication and filtering module is used to quickly deduplicatize the original image set using the perceptual hash algorithm and the color histogram algorithm, and generate a candidate image set. The feature extraction module is used to extract features from the candidate image set based on the improved EfficientViT network structure. The EfficientViT network structure includes a parallel self-attention branch and a multi-scale local feature extraction branch. The self-attention branch uses a standard multi-head self-attention mechanism based on Softmax to model global features. The multi-scale local feature extraction branch uses parallel multi-scale deep convolution to extract local features and concatenates the global features and local features in the channel dimension. The cross-layer feature fusion module is used to fuse features from different layers in the improved EfficientViT network structure. The cross-layer feature fusion module includes a global average pooling operation, an adaptive convolutional layer, and a channel attention weight generation unit. The adaptive convolutional layer dynamically adjusts the kernel size according to the channel dimension. The channel attention weight generation unit generates attention weights through the Sigmoid function and then weights and fuses the feature vectors. The similarity calculation and determination module is used to calculate cosine similarity based on the weighted fused feature vectors and determine whether the images are similar according to a preset threshold.

7. The apparatus as claimed in claim 6, characterized in that, Also includes: The index optimization module is used to construct an index and perform approximate nearest neighbor retrieval on the high-dimensional feature vectors generated by the improved EfficientViT algorithm using the HNSW algorithm. The high-dimensional feature vectors are stored in the HNSW index structure and an initial index is constructed. For the query image, the set of candidate high-dimensional feature vectors is quickly retrieved using the HNSW index, and then cosine similarity is used for accurate calculation.

8. An electronic device, comprising: processor; The memory stores executable instructions; when the processor executes the instructions, it implements an image similarity detection method as described in any one of claims 1-5.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements an image similarity detection method as claimed in any one of claims 1-5.

Images