A method and system for image similarity recognition based on multi-feature fusion

Abstract

A method and system for image similarity recognition based on multi-feature fusion, relating to the field of image processing technology, is disclosed. The method includes: calculating the hash features of an input image and generating hash similarity and hash confidence values; when the hash confidence value is less than a hash confidence reference value, performing local feature extraction on the input image to generate local similarity and local confidence values; when the local confidence value is less than the local confidence reference value, performing deep learning feature extraction on the input image to obtain deep features and deep confidence values; determining the weight coefficients of the hash features, local features, and deep features based on the hash confidence value, local confidence value, and deep confidence value, respectively; performing feature fusion to generate a fused feature vector; and calculating the final similarity between the input image and the image to be compared based on the fused feature vector. Implementing this application can improve processing efficiency while ensuring the accuracy of image similarity recognition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image similarity recognition method and system based on multi-feature fusion. Background Technology

[0002] With the rapid development of computer vision technology, image similarity recognition has been widely used in image retrieval, visual positioning, and product recognition. Accurate image similarity recognition can help systems quickly find images similar to the target image from massive amounts of image data, improving image processing efficiency and providing users with a better search experience.

[0003] Image similarity recognition methods in related technologies are mainly divided into two categories: one category uses a single feature for image matching, such as using only a certain type of algorithm for similarity calculation; to ensure the accuracy of image similarity recognition, the other category uses multiple features for simple fusion, that is, extracting multi-dimensional features of the image for feature fusion calculation, and then making similarity judgment.

[0004] However, in practical applications, different types of images exhibit different features. Some images can be accurately judged using only simple features, while others require more complex feature analysis to obtain a reliable judgment; related technologies suffer from wasted computational resources and low processing efficiency. For example, for images with significant feature differences, even simple feature recognition can yield accurate results, but related technologies are less efficient when processing such images based on multi-dimensional features. Summary of the Invention

[0005] This application provides an image similarity recognition method and system based on multi-feature fusion, which improves processing efficiency while ensuring the accuracy of image similarity recognition.

[0006] Firstly, this application provides an image similarity recognition method based on multi-feature fusion, applied to an image processing system. The method includes: calculating the perceptual hash value of an input image to obtain hash features, and calculating the difference between the hash features and corresponding features of the image to be compared, generating hash similarity and hash confidence values; when the hash confidence value is less than a hash confidence reference value, performing local feature extraction on the input image to obtain local features; performing feature point matching on the image to be compared based on the local features to generate local similarity and local confidence values; when the local confidence value is less than a local confidence reference value, performing deep learning feature extraction on the input image to obtain deep features and deep confidence values; determining the weight coefficients of the hash features, local features, and deep features based on the hash confidence value, local confidence value, and deep confidence value, respectively; fusing the hash features, local features, and deep features according to the weight coefficients to generate a fused feature vector; and calculating the final similarity between the input image and the image to be compared based on the fused feature vector.

[0007] In the above embodiments, the image processing system first calculates the perceptual hash value of the image to obtain preliminary features. Local feature extraction is only performed when the hash confidence value is low, and deep learning feature extraction is only executed when the local confidence value is insufficient. This progressive feature extraction strategy allows the system to dynamically adjust the computation depth according to the complexity of the image features. Combined with a feature fusion mechanism using weighted coefficients, accurate similarity judgment results are ensured in different scenarios, while avoiding resource waste caused by using complex features on simple images, thus improving the overall computational efficiency and recognition accuracy of the system.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the step of determining the weight coefficients of hash features, local features, and deep features based on hash confidence values, local confidence values, and deep confidence values ​​specifically includes: calculating the information entropy of hash confidence values, local confidence values, and deep confidence values ​​respectively to obtain an entropy value sequence; normalizing the entropy value sequence to generate initial weights; calculating the feature importance coefficients of hash confidence values, local confidence values, and deep confidence values ​​based on the initial weights; correcting and adjusting the initial weights according to the feature importance coefficients to obtain corrected weights; and performing nonlinear mapping processing on the corrected weights to obtain weight coefficients.

[0009] In the above embodiments, the image processing system realizes the quantitative evaluation of feature importance by calculating the information entropy of each confidence value; it generates initial weights by normalization and corrects the weights based on the feature importance coefficients, and finally obtains the final weight coefficients through nonlinear mapping. This can more accurately reflect the contribution of different features in similarity judgment, improve the accuracy of feature fusion, and make the final similarity calculation results more accurate and reliable.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the step of fusing hash features, local features, and deep features according to weight coefficients to generate a fused feature vector specifically includes: projecting hash features, local features, and deep features onto a preset dimensional space to generate projected features; weighting the projected features according to weight coefficients to generate initial fused features; performing feature selection based on the variance distribution of the initial fused features to determine the main feature components, and performing orthogonal transformation on the main feature components to obtain key features; and processing the key features based on a nonlinear activation function to obtain a fused feature vector.

[0011] In the above embodiments, the image processing system projects features of different dimensions onto a unified dimensional space and performs weighted combination, and selects features based on variance distribution to ensure that the most representative feature components are retained. Through orthogonal transformation and nonlinear activation function processing, the expressive power of features is further enhanced, the discriminative performance of the fused feature vector is improved, and the final similarity calculation result is more accurate.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, the input image is multiple; before the steps of calculating the perceptual hash value of the input image, obtaining hash features, and calculating the difference value between the hash features and the corresponding features of the image to be compared, and generating hash similarity and hash confidence value, the method further includes: grouping the multiple input images according to image size to generate an image group sequence; calculating the batch size based on the number of images in each image group sequence to generate batch partitioning information; establishing a cache index for the images in each image group sequence according to the batch partitioning information to obtain an image cache table; determining the feature extraction order based on the image cache table and creating a parallel computing task queue.

[0013] In the above embodiments, the image processing system performs size-based grouping management on multiple input images, optimizes data access efficiency through batch partitioning and caching indexing mechanisms, and achieves efficient processing of multiple images by establishing a parallel computing task queue, thereby improving the throughput of batch image processing and ensuring the rational utilization of memory resources, effectively avoiding resource bottleneck problems.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, after determining the feature extraction order based on the image cache table and creating a parallel computing task queue, the method further includes: determining the maximum memory usage of each image group sequence based on batch partitioning information, and adjusting the batch size based on the available system memory; determining the priority identifier of each image group sequence based on user settings; prioritizing the parallel computing task queue according to the priority identifier, and generating a task execution sequence.

[0015] In the above embodiments, the image processing system dynamically adjusts the batch size based on the available system memory and optimizes the sorting of the task queue by combining the priority identifier set by the user, thereby realizing intelligent scheduling of computing resources. This not only ensures the stable operation of the system but also improves the response speed of task processing, enhancing the practicality and reliability of the system.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, there are multiple images to be compared; the step of calculating the final similarity between the input image and the images to be compared based on the fused feature vector specifically includes: calculating the feature distribution density of the multiple images to be compared to generate feature cluster centers; constructing a feature index tree based on the feature cluster centers to obtain a retrieval structure; performing hierarchical screening of the images to be compared according to the retrieval structure to obtain a candidate image set; and performing parallel similarity calculation on the candidate image set based on the fused feature vector to obtain a similarity result containing multiple final similarities.

[0017] In the above embodiments, the image processing system constructs a feature index tree based on feature distribution density, realizes a multi-level filtering mechanism, reduces the number of images that need to be compared in detail, accelerates the similarity calculation process through parallel computing technology, improves the efficiency of large-scale image library retrieval, and ensures the accuracy of retrieval results.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, the step of performing hierarchical filtering of the images to be compared according to the retrieval structure to obtain a candidate image set specifically includes: acquiring spatial distribution data of feature cluster centers and generating a feature index tree; the feature index tree includes feature distance parameters and node capacity parameters; calculating the spatial density value of the feature index tree to obtain a node distribution sequence, such that the node distribution sequence and the feature distance parameters form a mapping relationship; statistically analyzing node access data according to the mapping relationship to obtain node preloaded data, and storing the node preloaded data in the retrieval cache; reading the node preloaded data from the retrieval cache and generating a fast retrieval table based on the access frequency of the node preloaded data; matching the feature data of the images to be compared with the fast retrieval table to obtain a preliminary image sequence; calculating the feature distance value of the preliminary image sequence, and filtering based on the feature distance value to generate a candidate image set.

[0019] In the above embodiments, the image processing system constructs a feature index tree by combining spatial distribution data and optimizes retrieval efficiency through node preloading and a fast retrieval table mechanism. The quality of the candidate image set is ensured by filtering based on feature distance values, guaranteeing both the accuracy of the retrieval results and improving retrieval speed, thus achieving efficient and accurate retrieval of a large-scale image database.

[0020] In a second aspect, embodiments of this application provide an image processing system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, which includes computer instructions, and the one or more processors call the computer instructions to cause the image processing system to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on an image processing system, cause the image processing system to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an image processing system, cause the image processing system to perform the method described in the first aspect and any possible implementation thereof.

[0023] It is understood that the image processing system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0025] 1. By employing a progressive feature extraction and adaptive fusion approach, the system first rapidly acquires preliminary features through perceptual hash value calculation. Only when the reliability is insufficient is it progressively deepened with local and deep feature extraction, and intelligent fusion is performed using weighted coefficients. Therefore, the system can automatically adjust the computational depth based on the complexity of image features, avoiding unnecessary computational overhead. This effectively solves the problem of resource waste caused by applying the same processing depth to all images in existing technologies, thereby improving computational efficiency and recognition accuracy. Especially for simple images with significant feature differences, accurate results can be obtained quickly, while for complex images, multi-level feature extraction ensures recognition accuracy.

[0026] 2. By employing image size-based grouping management and a parallel computing task scheduling mechanism, optimizing data access efficiency through batch partitioning and cache indexing, and creating a parallel computing task queue to achieve concurrent processing of multiple images, the system can efficiently process a large number of input images, improving data processing throughput. This effectively solves the problems of low efficiency and unreasonable resource utilization in batch image processing in existing technologies, thereby improving system processing performance. It not only ensures the rational use of memory resources and avoids resource bottlenecks, but also improves the overall operating efficiency and stability of the system through an intelligent task scheduling mechanism.

[0027] 3. By employing a retrieval mechanism based on feature clustering and multi-level filtering, and constructing an index tree structure by calculating feature distribution density, combined with hierarchical filtering and parallel computing techniques to accelerate similarity calculation, the system can quickly locate potentially similar images in a massive image database. This effectively solves the problems of low efficiency and insufficient accuracy in large-scale image database retrieval in existing technologies, thus achieving a dual improvement in retrieval performance and accuracy. It reduces the number of images requiring detailed comparison and accelerates the similarity calculation process through parallel computing, ensuring improved retrieval efficiency while maintaining accuracy. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating an image similarity recognition method based on multi-feature fusion in an embodiment of this application;

[0029] Figure 2 This is another flowchart illustrating the image similarity recognition method based on multi-feature fusion in the embodiments of this application;

[0030] Figure 3 This is a schematic diagram of the physical device structure of an image processing system in an embodiment of this application. Detailed Implementation

[0031] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0032] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0033] To facilitate understanding, the application scenarios of the embodiments of this application are described below.

[0034] In product image retrieval on e-commerce platforms, the system needs to process millions of product images daily to determine if similar or duplicate products exist. Traditional image similarity recognition systems use the same processing depth and feature extraction methods for all images, leading to significant waste of computational resources. For example, for some simple product images (such as standard product photos with a solid color background), accurate judgment can be made through basic feature comparison, but the system still performs a complete deep feature extraction process. However, for some complex product images (such as real-life photos taken from multiple angles with complex backgrounds), more in-depth feature analysis may be needed to accurately determine similarity; using only simple features can easily lead to misjudgments. This lack of targeted processing reduces system efficiency and fails to guarantee recognition accuracy.

[0035] In related technologies, image similarity recognition can be achieved by using single feature extraction or simple feature overlay. However, this method applies the same processing depth to all images, leading to wasted computational resources and unstable recognition accuracy. The following describes a scenario using an image similarity recognition method based on multi-feature fusion from related technologies.

[0036] Current image similarity recognition systems typically employ two approaches: The first is a single-feature approach, such as the perceptual hashing algorithm used by some e-commerce platforms for image deduplication. While fast, this approach is prone to misclassification for images with significant changes in angle or lighting. The second is a multi-feature fusion approach, such as an image retrieval system that simultaneously extracts SIFT and deep learning features and simply overlays them. While this improves accuracy, it significantly increases computational overhead. For example, even for two identical product images, the system must complete all feature extraction processes from low to high levels, resulting in a substantial waste of computational resources. Furthermore, due to the lack of an effective feature importance evaluation mechanism, simple feature overlay can lead to noisy features interfering with the final judgment.

[0037] The image similarity recognition method based on multi-feature fusion in this application embodiment achieves efficient utilization of computing resources by dynamically adjusting the feature extraction depth and intelligent weight allocation, which not only improves processing efficiency but also ensures the accuracy of recognition results. The following describes a scenario where the image similarity recognition method based on multi-feature fusion in this application is used.

[0038] By adopting the progressive feature fusion scheme of this application, the system can adaptively adjust the processing depth according to the complexity of the image. For example, for product main images with a high degree of standardization, the system first calculates the perceptual hash value. If the confidence of the hash similarity is high (e.g., exceeding 0.9), the judgment result is directly obtained without further complex feature extraction. For product images with more complex compositions, when the confidence of the hash features is insufficient, the system will further extract local features such as SIFT; only if the confidence of the local features is still low will the computationally intensive deep learning feature extraction be initiated. Through this progressive processing strategy, the system ensures both the accuracy of the recognition results and significantly improves processing efficiency. Actual tests show that, while maintaining the same accuracy, the processing speed is improved by approximately 40%.

[0039] As can be seen, the image similarity recognition method based on multi-feature fusion in this application embodiment can not only accurately identify image similarity, but also effectively solve the problem of wasted computing resources, thereby achieving a dual improvement in recognition efficiency and accuracy.

[0040] To facilitate understanding, the method provided in this implementation will be described in detail below, using the above scenario as an example. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating an image similarity recognition method based on multi-feature fusion in an embodiment of this application.

[0041] S101. Calculate the perceptual hash value of the input image to obtain the hash feature, and calculate the difference between the hash feature and the corresponding feature of the image to be compared, and generate hash similarity and hash confidence value.

[0042] The input image represents the image data to be processed for similarity recognition; the perceptual hash value is a binary string used to characterize the visual features of the image, calculated by the perceptual hash algorithm; the hash feature represents the feature vector converted from the perceptual hash value; the difference value is the Hamming distance or Euclidean distance between the hash features of two images; the hash similarity is used to represent the degree of similarity between two images calculated based on the hash feature, and the value ranges from 0 to 1; the hash confidence value represents the credibility of the hash similarity calculation result.

[0043] Upon receiving an image for similarity recognition, the image processing system first performs basic feature extraction. Specifically, the system first scales the input image and the image to be compared to the same size, then calculates their perceptual hash values. For each image, the system first converts it to grayscale, then uses DCT transform to extract frequency domain features, and finally generates a binary perceptual hash value based on the distribution of the frequency domain coefficients. The system converts the obtained hash value into feature vector form and calculates the difference between the two feature vectors. Based on the difference value, it calculates hash similarity and evaluates the reliability of the calculation result by combining the distribution characteristics of the difference value, generating a hash confidence value.

[0044] In some embodiments, the calculation of perceptual hash values ​​and similarity assessment can be implemented in several ways: Optionally, the system can employ a mean hash algorithm to generate binary features by calculating the average gray value of image patches and comparing it with the overall mean, then using normalized Hamming distance to calculate similarity, and assessing confidence based on the variance of the distance distribution; alternatively, the system can also employ a perceptual hash algorithm to first perform DCT transformation on the image to obtain low-frequency coefficients, then quantize and compare the low-frequency coefficients to generate binary features, and finally use weighted Euclidean distance to calculate similarity, and assess confidence based on the frequency domain energy distribution. It is understood that other image hashing algorithms and similarity calculation methods can also be used to implement this step, and no limitation is made here.

[0045] It should be noted that in the perceptual hash value calculation process, the input image I(x, y) is first scaled to 8×8 pixels and then converted into a grayscale image G(x, y). The average pixel value μ = 1 / 64∑G(x, y) of G(x, y) is used as the binarization threshold to generate a 64-bit binary hash value H. For any pixel (i, j), H(i, j) = 1 when G(i, j) > μ, otherwise H(i, j) = 0. The hash features are represented as a vector h = [h1, h2, ..., h64]. The Hamming distance d = ∑|h1(i) - h2(i)| between the hash feature vectors h1 and h2 of two images is calculated as the difference value. The hash similarity is obtained by normalization calculation s = 1 - d / 64, with a value range of [0, 1]. The hash confidence value c is calculated by analyzing the distribution characteristics of the difference values; the more concentrated the distribution of d and the smaller the standard deviation, the higher the confidence value. For example, for identical images, d=0, s=1, and c is close to 1; for completely different images, d is close to 32, s is close to 0.5, and c is close to 0.

[0046] S102. When the hash confidence value is less than the hash confidence reference value, perform local feature extraction on the input image to obtain local features.

[0047] Among them, the hash confidence reference value represents the threshold standard for judging the reliability of hash features; local feature extraction refers to the process of extracting discriminative feature points and descriptors from local regions of an image; local features represent a set of vectors describing the features of local regions of an image.

[0048] When an image processing system detects that the reliability of a hash-based judgment result is insufficient, it needs to perform more in-depth feature analysis. Specifically, the image processing system first compares the currently obtained hash confidence value with a preset hash confidence reference value. When the hash confidence value is low, it indicates that relying solely on hash features cannot yield a reliable similarity judgment result. In this case, the system performs local feature extraction on the input image. The system first detects feature points in the image, then extracts feature descriptors for local regions around each feature point, ultimately obtaining a set of feature vectors that characterize the local structural features of the image.

[0049] In some embodiments, local feature extraction can be achieved in several ways: Optionally, the system can employ the SIFT algorithm to construct scale-space detection keypoints, calculate the principal orientation of the keypoints, generate a 128-dimensional feature descriptor in the neighborhood of the feature points, and finally normalize the descriptor to improve its robustness; alternatively, the system can also employ the SURF algorithm to detect feature points using Haar wavelet responses, generate a 64-dimensional feature descriptor by calculating Haar wavelet responses around the feature points, and finally apply PCA for dimensionality reduction to optimize the feature representation. It is understood that other local feature extraction algorithms can also be used to implement this step, and no limitation is made here.

[0050] S103. Based on local features, perform feature point matching on the images to be compared to generate local similarity and local confidence values.

[0051] Feature point matching refers to the process of finding corresponding feature points in two images; local similarity represents the degree of similarity of local regions of an image calculated based on the feature point matching results; local confidence value is used to represent the reliability of the local similarity calculation results; feature point correspondence refers to the spatial mapping relationship between matched feature points in two images.

[0052] After obtaining local features, image processing systems need to evaluate the local structural similarity between images through feature matching. Specifically, the image processing system first calculates the similarity of local feature descriptors extracted from the input image and the image to be compared, establishing a preliminary correspondence between feature points. Then, the system uses algorithms such as RANSAC to remove erroneous matching points and optimize the correspondence between feature points. Based on the optimized matching results, the system calculates statistical features such as the number, distribution, and similarity of matching point pairs, and comprehensively evaluates the local similarity. Simultaneously, the system analyzes the spatial distribution characteristics and matching quality of the matching points, generating a local confidence value that reflects the reliability of the matching.

[0053] In some embodiments, feature point matching and similarity evaluation can be implemented in multiple ways: Optionally, the system can employ a nearest neighbor ratio matching strategy, establishing initial matching by calculating the Euclidean distance between feature descriptors, then using the RANSAC algorithm to estimate the geometric transformation model to remove mismatches, and finally calculating similarity and confidence based on the number and distribution of valid matching point pairs; Optionally, the system can also employ a graph matching algorithm, constructing feature points into a graph structure, establishing the correspondence between feature points through graph similarity metrics, and evaluating the matching quality by combining the topological features of the graph. It is understood that other feature matching and similarity calculation methods can also be used to implement this step, and no limitation is made here.

[0054] It should be noted that local feature extraction is based on scale-space theory, constructing a Gaussian difference pyramid DoG(x, y, σ) = G(x, y, kσ) ⊗ I(x, y) - G(x, y, σ) ⊗ I(x, y), where G(x, y, σ) is the Gaussian kernel function and k is the scale coefficient. In the DoG space, candidate feature points are extracted by finding three-dimensional local extrema. For each candidate point (x, y, σ), the Hessian matrix H = [DxxDxy; DxyDyy] is calculated, where Dxx, Dxy, and Dyy are second-order partial derivatives. Edge response points are removed based on the eigenvalue ratio of H, retaining stable feature points. For the retained feature points, the principal direction θ = arctan(L(x+1, y) - L(x-1, y)) / (L(x, y+1) - L(x, y-1)), where L is the Gaussian blurred image. Finally, a 128-dimensional descriptor vector d = [d1, d2, ..., d128] is generated in the neighborhood of the feature points. The reliability of the features is evaluated by analyzing the distribution characteristics and matching quality of the descriptors. Preferably, the feature distance d(x, y) = G(x, y) ⊗ I(x, y) can also be calculated, where G(x, y) is a preset Gaussian kernel function. For each feature point p, a CostMatrix is ​​constructed based on the feature distance to calculate the minimum matching cost. Invalid matches are filtered by setting thresholds ConstantThreshold and DistanceThreshold. For matching point pairs that meet the conditions, the geometric transformation model H(x, y) = {H|d(x, y) ≥ r} is further calculated, where r is the edge threshold, usually taken as 0.05. The local similarity s and local confidence value c are calculated based on the matching results of the geometric consistency constraint.

[0055] S104. When the local confidence value is less than the local confidence reference value, perform deep learning feature extraction on the input image to obtain deep features and deep confidence values.

[0056] Among them, the local confidence reference value represents the threshold standard for judging the reliability of local feature matching; deep learning feature extraction refers to the extraction of high-level semantic features of an image using a pre-trained deep neural network model; deep features represent the feature vector output by the neural network; and the depth confidence value is used to represent the reliability of the deep feature extraction results.

[0057] When an image processing system detects insufficient reliability in local feature matching results, it needs to perform higher-level feature analysis. Specifically, the image processing system first compares the local confidence value with a preset confidence reference value. When the confidence value is low, the system invokes a pre-trained deep neural network model to process the input image. The model extracts feature representations of the image from low to high levels layer by layer through multi-layer convolution operations and nonlinear transformations. The system obtains the outputs of the intermediate or final layers of the network as deep features and calculates the depth confidence value based on the activation intensity and distribution characteristics of the features.

[0058] During training, deep neural network models receive a large number of image pairs with similarity labels as input. Each image pair contains raw pixel data and its similarity / dissimilarity label information. The training process uses a cross-entropy loss function to optimize network parameters by minimizing the difference between the predicted similarity and the true label. A feature discriminative loss term is also introduced to ensure that the extracted features can effectively distinguish images of different categories. The model uses stochastic gradient descent for parameter optimization and employs a mini-batch training approach, dynamically adjusting the learning rate in each batch to improve convergence efficiency. In terms of model architecture, the deep neural network model adopts a multi-layer convolutional neural network structure, including basic components such as convolutional layers, pooling layers, and fully connected layers. Lower-level convolutional kernels mainly extract local features such as edges and textures, middle layers capture local structural and shape information, and higher layers learn semantic-level abstract features. A skip connection mechanism is used to fuse features from different levels, enhancing the expressive power of the features. During the usage phase, a single image to be processed is input into the deep neural network model, which can choose to extract feature maps from intermediate or final layers as deep features. By analyzing the activation intensity and distribution characteristics of the feature maps, a reliability metric for the features is calculated, generating confidence values. For example, the stability and discriminative power of features can be assessed based on the mean and variance statistics of feature maps.

[0059] In some embodiments, deep feature extraction and confidence evaluation can be achieved in various ways: Optionally, the system can employ a pre-trained CNN model to extract feature maps from different convolutional layers, obtain multi-scale feature descriptions through spatial pyramid pooling of the feature maps, and finally evaluate the feature confidence based on the statistical properties of feature activation; alternatively, the system can also employ an autoencoder model to extract latent feature representations of the image through the encoder, then reconstruct the image using the decoder, and evaluate the reliability of the features based on the reconstruction error. It is understood that other deep learning models and feature extraction methods can also be used to implement this step, and no limitation is made here.

[0060] During the training phase, the autoencoder model processes unlabeled image data and inputs multiple types of feature representations (including hash features, local features, and deep features). The training objective is to minimize the reconstruction error, minimizing the difference between the features reconstructed by the decoder and the original input features. Mean squared error is used as the reconstruction loss function, while a regularization term is introduced to control the feature distribution in the latent space. The model also includes a learning mechanism for feature fusion weights, automatically adjusting the importance weights of different features through backpropagation. In terms of model structure, the encoder and decoder employ a symmetrical multi-layer neural network architecture. The encoder maps features of different dimensions to a unified-dimensional latent space through nonlinear transformations, while the decoder reconstructs the fused features back to the original feature space. A feature fusion layer is designed in the latent space, dynamically adjusting feature importance through an attention mechanism to achieve adaptive feature fusion. In practical applications, the model receives multiple feature vectors of different types as input, maps them to the latent space through the encoder, performs feature fusion and dimensionality reduction operations in the latent space, and finally outputs a unified feature representation. For example, 64-dimensional hash features, 128-dimensional local features, and 256-dimensional deep features can be uniformly mapped to a 96-dimensional latent space for fusion.

[0061] It should be noted that deep feature extraction utilizes a pre-trained ResNet-50 model. The input image is first uniformly scaled to 224×224 pixels and normalized: I'=(I-μ) / σ, where μ and σ are the mean and standard deviation of the ImageNet dataset. The feature extraction process includes multiple residual blocks, each residual block F(x)=[Conv-BN-ReLU-Conv-BN], with an output y=F(x)+x. The feature map f∈R^{2048} before the last pooling layer of the network is extracted, and the feature vector v is obtained through global average pooling. The deep confidence value is calculated based on the feature activation intensity. First, the activation distribution of each dimension of the feature vector A(i)=ReLU(v(i)) is analyzed, and the activation entropy H=-∑p(i)log p(i) is calculated, where p(i)=A(i) / ∑A(i). The confidence value c is negatively correlated with the entropy value: c=exp(-αH), where α is the scaling factor. The more significant and concentrated the feature activation, the higher the confidence value. Preferably, deep features can also be extracted using a pre-trained CNN model. After scaling and preprocessing, the input image is used to extract features through a convolutional layer Fdeep. The depth confidence value is calculated based on the statistical properties of the feature vector: Confidence = P(Fdeep), where P() is a preset probability distribution function. When the confidence value is less than a set threshold, the next level of feature extraction is triggered.

[0062] S105. Determine the weight coefficients of hash features, local features, and deep features based on hash confidence values, local confidence values, and deep confidence values, respectively.

[0063] Among them, the weight coefficient represents the importance of different features in the fusion process; feature importance refers to the contribution of each feature to the final similarity judgment; the weight allocation strategy is used to guide how to determine reasonable weight coefficients based on confidence values.

[0064] After obtaining features at three different levels and their confidence values, the image processing system needs to determine the weight allocation for feature fusion. Specifically, the system first analyzes the distribution characteristics of each confidence value, calculates their information entropy, and evaluates the information content of the features. Then, the system normalizes the entropy values ​​to obtain initial weights and adjusts them based on the prior importance of the features. The system further analyzes the complementarity between features, optimizes the weight allocation scheme, and finally obtains the final weight coefficients through nonlinear mapping.

[0065] In some embodiments, the weight coefficients can be determined in several ways: Optionally, the system can employ an adaptive weight allocation method based on information entropy, which assesses the uncertainty of features by calculating the entropy value of the confidence value, then weights the features based on their prior importance, and finally normalizes the weight coefficients using the Softmax function; alternatively, the system can also employ a weight learning method based on collaborative optimization, constructing an objective function for feature fusion and learning the optimal weight parameters through an iterative optimization algorithm. It is understood that other weight determination methods can also be used to achieve this step, and no limitation is made here.

[0066] It should be noted that the weight coefficients are calculated using an adaptive weighting method based on confidence. The confidence values ​​ch (hash), cl (local), and cd (depth) of the three features are first normalized to obtain the probability distribution p = [ph, pl, pd]. The confidence entropy H = -∑pilog(pi) is calculated to assess uncertainty. A nonlinear mapping function g(H) = 1 / (1 + exp(β(H - H0))) is designed based on the entropy value to adjust the weight influence, where β and H0 are adjustment parameters. The initial weights wi = pig(H) of each feature are normalized using Softmax to obtain the final weight coefficient w'i = exp(wi) / ∑exp(wi). This method can dynamically adjust the weights according to the reliability of the features; when the confidence of a certain type of feature is significantly higher than that of other features, its weight will be adaptively enhanced.

[0067] S106. Perform feature fusion on hash features, local features and deep features according to weight coefficients to generate a fused feature vector.

[0068] Among them, feature fusion represents the process of combining different types of features into a unified representation; fused feature vector refers to the comprehensive feature representation obtained through feature fusion; projected feature representation is the representation of the original features after mapping to a unified feature space; feature selection refers to the process of selecting the most representative feature components from the fused features; orthogonal transformation is used to optimize the expressive power of features and reduce feature redundancy.

[0069] After determining the weighting coefficients, the image processing system needs to effectively fuse multiple features into a unified feature representation. Specifically, the system first projects hash features, local features, and depth features onto a feature space of the same dimension through feature transformation. Then, the system weights and combines the projected features according to the determined weighting coefficients to generate initial fused features. The system further analyzes the variance distribution of the fused features, selects the most discriminative feature components, and optimizes the feature representation through orthogonal transformation. Finally, the system uses a nonlinear activation function to process the features, enhancing their expressive power, and obtains the final fused feature vector.

[0070] In some embodiments, feature fusion can be achieved in several ways: Optionally, the system can employ an attention-based feature fusion method, which adaptively adjusts the importance of different features by calculating attention weights for feature channels and spatial dimensions, and then obtains fused features through residual connections and nonlinear transformations; alternatively, the system can also employ an autoencoder-based feature fusion method, which takes multiple features as input, learns a unified feature representation through an encoder, and uses reconstruction loss to guide the feature fusion process. It is understood that other feature fusion methods can also be used to achieve this step, and no limitation is made here.

[0071] It should be noted that feature fusion first unifies the dimensionality of feature vectors from different dimensions. For hash features h∈R^64, local features l∈R^128, and depth features d∈R^2048, they are mapped to the same dimensional space v'=Pv using a linear projection matrix P. The projection matrix P is optimized by minimizing the reconstruction error, with the objective function J=||vP^TPv||²+λ||P||², where λ is the regularization coefficient. After feature concatenation, PCA dimensionality reduction is performed, retaining principal components with a cumulative variance contribution rate of 95%. Let the feature matrix X=[v'h; v'l; v'd], and its covariance matrix C=X^TX, with eigenvalue decomposition C=UΣU^T. The first k feature vectors are selected to construct the projection matrix W. The fused feature f=WX is then L2 normalized to obtain the final feature vector. Preferably, the perceptual hash value, local features and deep learning features can be concatenated: Fconcat=[Hash_Fuzz, FSIFT, Fdeep]. Then, the concatenated feature vector is reduced in dimensionality by PCA: F'concat=PCA(Fconcat). 95% of the variance contribution is retained to obtain the dimensionality-reduced fused feature vector: Ffused=NonLinear(F'concat).

[0072] S107. Based on the fused feature vector, calculate the final similarity between the input image and the image to be compared.

[0073] Among them, the final similarity represents the image similarity evaluation result obtained after comprehensively considering multiple features; the similarity calculation strategy refers to the method of calculating image similarity based on fused feature vectors; the distance metric is used to represent the method of calculating feature differences in the feature space; and similarity normalization refers to the process of mapping the calculated similarity to a standard interval.

[0074] After obtaining the fused feature vectors, the image processing system needs to calculate the final image similarity evaluation result. Specifically, the system first selects an appropriate distance metric to calculate the distance between the fused feature vectors of the input image and the image to be compared. The system considers the statistical distribution characteristics of the features and weights the distance calculation results. Then, the system calculates a similarity score based on the distance values ​​and calibrates it using a pre-established similarity mapping model. Finally, the system normalizes the similarity scores to ensure that the results fall within a predetermined range, thus obtaining the final similarity evaluation result.

[0075] In some embodiments, the final similarity can be calculated in several ways: Optionally, the system can employ a kernel-based similarity calculation method, which maps features to a high-dimensional space, calculates the inner product of features in the kernel space as a similarity metric, and uses kernel parameters to adjust the distribution characteristics of the similarity; alternatively, the system can also employ a metric learning-based similarity calculation method, which learns the Mahalanobis distance matrix to assign different importance to different feature dimensions, thereby obtaining a similarity metric that better reflects perceptual characteristics. It is understood that other similarity calculation methods can also be used to implement this step, and no limitation is made here.

[0076] It should be noted that the final similarity calculation is based on a weighted cosine distance metric. Given two fused feature vectors f1 and f2, their standardized inner product sim(f1, f2) = (f1·f2) / (||f1||·||f2||) is first calculated. Considering the different importance of each dimension of the features, a learned dimension weight vector α is introduced, and the corrected similarity calculation is sim'(f1, f2) = ∑αi(f1i·f2i) / (||f1||·||f2||). The weight α is optimized by maximizing the interval between positive and negative sample pairs, and the loss function is L = max(0, m + sim'(f-, f) - sim'(f+, f)), where m is the interval threshold, and f+ / f- represent positive / negative sample features, respectively. The final similarity is mapped to the interval [0, 1] s = 1 / (1 + exp(-γ·sim')), where γ is the temperature parameter.

[0077] In the above embodiment, intelligent feature fusion is achieved through progressive feature extraction and adaptive weight allocation. In practical applications, this method can automatically select an appropriate processing depth based on the complexity of the image, avoiding unnecessary computational overhead. The following section supplements the scenario described in this embodiment.

[0078] By introducing grouped batch processing and parallel computing mechanisms, the system's processing power is further enhanced. In a product deduplication scenario on a large e-commerce platform, the system first groups the product images to be processed according to their size, such as grouping all 1920×1080 resolution images into one group. Then, it calculates the optimal batch size for each group and creates a parallel computing task queue. For example, for simple image groups, the system can process more images simultaneously; for complex image groups requiring deep feature extraction, the batch size is reduced accordingly to avoid memory overflow. Simultaneously, by establishing a multi-level caching structure and a preloading mechanism, the system can preload the next batch of image data, reducing IO wait time. Test results show that this optimization strategy increases the system's peak processing power several times over while maintaining a low memory footprint.

[0079] In light of the above scenarios, the method provided in this implementation will now be described in more detail. Please refer to [link / reference]. Figure 2 This is another flowchart illustrating the image similarity recognition method based on multi-feature fusion in this application embodiment.

[0080] S201. Group the multiple input images according to their image size to generate an image group sequence.

[0081] Here, the image group sequence represents a collection of images categorized according to their size; image size refers to the pixel dimensions of an image; grouping refers to classifying images according to a preset size range. For example, images with a resolution of 1920×1080 can be grouped into one group, and images with a resolution of 640×480 can be grouped into another.

[0082] After receiving multiple input images, image processing systems need to preprocess and organize them to improve the efficiency of subsequent processing. Specifically, the image processing system first obtains the size information of each input image, then sets up several groups according to a preset size range, grouping images of similar sizes into the same group, ultimately forming multiple image group sequences containing images of similar sizes. This grouping method allows subsequent feature extraction and processing to be performed in batches, improving computational efficiency.

[0083] In some embodiments, grouping can be achieved through an adaptive size range partitioning method: first, statistical analysis is performed on the size distribution of all input images to calculate the mean and standard deviation of the sizes; then, a dynamic threshold is set based on the Gaussian distribution principle, and images falling within adjacent threshold ranges are grouped into the same group. This method can automatically adjust the grouping boundaries according to the actual image size distribution characteristics, making the grouping more reasonable. Optionally, a fixed size range grouping method can also be used: several standard size levels are pre-defined, such as small size (below 640×480), medium size (640×480 to 1920×1080), large size (above 1920×1080), etc., and the images are directly mapped to the corresponding size level groups. It is understood that other grouping strategies can also be used to achieve image size classification, and the specific grouping method can be flexibly adjusted according to the actual application scenario.

[0084] S202. Calculate the batch size based on the number of images in each image group sequence and generate batch partitioning information.

[0085] The batch size represents the number of images processed in parallel each time; the batch partitioning information includes the image index of each batch, the corresponding memory usage estimate, and the processing priority; the number of images refers to the number of images contained in each image group sequence.

[0086] Image processing systems need to determine an appropriate batch size for each image sequence to achieve efficient utilization of computing resources. Specifically, the system first counts the number of images in each image sequence, then calculates the maximum number of images that can be processed simultaneously based on system hardware resource limitations (such as memory capacity and GPU memory), and uses this number as the initial batch size. Simultaneously, considering the differences in memory usage for images of different sizes, the batch size is dynamically adjusted, ultimately generating batch partitioning information containing detailed information for each batch.

[0087] In some embodiments, the batch size can be determined adaptively using resources: first, a mapping model between image size and memory usage is established; based on this model, the expected memory requirement for each image group sequence is calculated; then, the batch size is dynamically adjusted based on the system's currently available memory, ensuring maximum system resource utilization while avoiding memory overflow. Alternatively, a fixed-ratio approach can be used: the total system memory is divided into different processing stages according to a preset ratio, and then the corresponding batch size is calculated based on the memory quota for each stage. It is understood that other batch processing strategies can also be used to achieve reasonable allocation of computing resources, and the specific allocation method can be optimized and adjusted according to the actual system performance.

[0088] S203. Based on the batch division information, establish a cache index for the images in each image group sequence to obtain the image cache table.

[0089] The cache index represents the mapping relationship of the storage location of images in memory; the image cache table contains information such as image identifier, memory address, and loading status; creating a cache index refers to allocating and recording memory space for images in a batch.

[0090] Image processing systems require efficient management of image data in memory to avoid redundant loading and frequent disk I / O operations. Specifically, based on batch segmentation information, the image processing system pre-allocates memory space for each batch of images, establishes a mapping relationship between image identifiers and memory addresses, and forms a complete image cache table. The cache table not only records the storage location of image data but also contains information such as the image processing status and access frequency to optimize subsequent data access efficiency.

[0091] In some embodiments, efficient image data management can be achieved through a multi-level caching architecture: a hierarchical storage structure including memory cache and disk cache is constructed, with frequently accessed image data stored in the memory cache and less frequently accessed data stored in the disk cache, and dynamic data scheduling achieved through a cache replacement algorithm. Optionally, a pre-fetch mechanism can also be used: the next batch of image data is loaded into the cache in advance according to the image processing order, reducing the waiting time during processing. It is understood that other caching strategies can also be used to optimize the efficiency of image data access, and the specific implementation can be adjusted according to the actual application requirements.

[0092] S204. Determine the feature extraction order based on the image cache table and create a parallel computing task queue.

[0093] Among them, the feature extraction order indicates the order in which features are calculated on the image; the parallel computing task queue contains information such as task ID, dependency relationship, and priority; the task queue is used to manage and schedule computing tasks executed in parallel.

[0094] Image processing systems need to rationally arrange the execution order of feature extraction to fully utilize parallel computing resources. Specifically, based on the image information recorded in the image cache table, the image processing system analyzes the dependencies and resource requirements between tasks and constructs a directed acyclic graph (DAG) of computational tasks. Then, according to factors such as task priority and expected execution time, the tasks are rationally allocated to different computing units, forming a task queue that can be executed in parallel.

[0095] In some embodiments, parallel computing efficiency can be optimized through dynamic task scheduling strategies: a task execution cost model is established, considering factors such as computational load and data transmission overhead, to dynamically adjust the execution order and resource allocation of tasks, thereby achieving load balancing. Optionally, a static task allocation scheme can also be used: tasks are pre-divided into several relatively independent sub-task sets, with each computing unit responsible for processing a fixed subset of tasks. It is understood that other task scheduling strategies can also be used to achieve efficient utilization of computing resources, and the specific scheduling method can be optimized according to the actual hardware configuration.

[0096] In some embodiments, the image processing system dynamically optimizes and schedules resources for parallel computing tasks. Specifically, the image processing system determines the maximum memory usage of each image group sequence based on batch partitioning information and adjusts the batch size based on available system memory. Based on user settings, it determines the priority identifier of each image group sequence and sorts the parallel computing task queue according to the priority identifier to generate a task execution sequence.

[0097] Among them, maximum memory usage represents the peak memory size required to process the image sequence; batch size refers to the number of images that can be processed simultaneously in each batch; priority identifier is used to indicate the priority order of task processing; task execution sequence represents the computation task queue sorted by priority.

[0098] Before executing parallel computing tasks, image processing systems need to allocate computing resources appropriately and determine the task execution order. Specifically, the image processing system first estimates the memory requirements for processing each image group sequence based on the image size and the complexity of the processing algorithm. The estimated results are compared with the system's currently available memory, and the batch size is dynamically adjusted to avoid memory overflow. Then, the system reads the user-configured processing priority information and assigns a priority identifier to each image group sequence. Finally, the computing tasks in the task queue are sorted based on the priority identifiers to generate the final execution sequence.

[0099] In some embodiments, efficient task processing can be achieved through dynamic scheduling based on resource prediction: first, a task resource consumption model is established to predict the CPU utilization, memory usage, and execution time of each task; then, task priorities are dynamically adjusted based on system resource status, and the task scheduling strategy is optimized in real time through a feedback control mechanism. Optionally, an optimization scheduling method based on task dependencies can also be used: a task dependency graph is constructed, the parallelism possibilities between tasks are analyzed, and the optimal task execution order is determined through a critical path algorithm. It is understood that other scheduling optimization strategies can also be used to achieve efficient utilization of computing resources, and these are not limited here.

[0100] S205. Calculate the perceptual hash value of the input image to obtain the hash feature, and calculate the difference between the hash feature and the corresponding feature of the image to be compared, and generate hash similarity and hash confidence value.

[0101] Referring to step S101, the image processing system generates hash similarity and hash confidence values.

[0102] S206. When the hash confidence value is less than the hash confidence reference value, perform local feature extraction on the input image to obtain local features.

[0103] Referring to step S102, the image processing system extracts local features.

[0104] S207. Based on local features, perform feature point matching on the images to be compared to generate local similarity and local confidence values.

[0105] Referring to step S103, the image processing system determines local similarity and local confidence values.

[0106] S208. When the local confidence value is less than the local confidence reference value, perform deep learning feature extraction on the input image to obtain deep features and deep confidence values.

[0107] Referring to step S104, the image processing system extracts depth features and determines depth confidence values.

[0108] S209. Determine the weight coefficients of hash features, local features, and deep features based on hash confidence values, local confidence values, and deep confidence values, respectively.

[0109] Referring to step S105, the image processing system determines the weight coefficients for each feature.

[0110] In some embodiments, the image processing system determines the feature weight allocation scheme through information entropy analysis. Specifically, the image processing system calculates the information entropy of the hash confidence value, local confidence value, and depth confidence value respectively to obtain an entropy value sequence; normalizes the entropy value sequence to generate initial weights; calculates the feature importance coefficients of the hash confidence value, local confidence value, and depth confidence value based on the initial weights; adjusts the initial weights according to the feature importance coefficients to obtain corrected weights; and performs nonlinear mapping processing on the corrected weights to obtain weight coefficients.

[0111] Among them, information entropy represents the uncertainty measure of the distribution of feature confidence values; entropy sequence refers to the numerical sequence composed of the information entropy of multiple feature confidence values; initial weight is used to represent the initial importance of features; feature importance coefficient is a numerical index that reflects the degree of influence of features on the recognition result; adjusted weight represents the feature weight value after adjustment; nonlinear mapping refers to the mathematical transformation that maps the weight value to a suitable interval.

[0112] After acquiring the confidence values ​​of multiple features, the image processing system needs to scientifically determine the weights of each feature in the fusion process. Specifically, the image processing system first calculates the probability distribution of each confidence value sequence, and then calculates the information entropy of each sequence based on the distribution to quantitatively assess the uncertainty of the confidence values ​​of different features. Next, it performs maximum-minimum normalization on all entropy values ​​to obtain initial weight coefficients. Then, it analyzes the contribution of each feature to the recognition accuracy, calculates the feature importance coefficient, and multiplies this coefficient by the initial weights to obtain the corrected weights. Finally, it maps the corrected weights to the [0, 1] interval using nonlinear functions such as the sigmoid function to obtain the final weight coefficients.

[0113] The image processing system employs an information gain-based approach to calculate feature importance coefficients. First, a feature evaluation dataset is constructed, containing image pairs with known similarity labels. For each feature class, its conditional entropy across different similarity intervals is calculated; the difference between this entropy and the entropy of the original similarity distribution is the information gain for that feature. Simultaneously, considering the time complexity of the features, a computational cost factor is introduced to weight and adjust the information gain. The system normalizes the adjusted information gain using a Softmax function to obtain the importance coefficients for each feature. In dynamic scenes, the system continuously updates the feature evaluation dataset and periodically recalculates the importance coefficients to adapt to changes in data distribution. For example, for high-quality standard product images, hash features often provide higher information gain, resulting in a correspondingly higher importance coefficient; while for complex real-world images, deep features may contribute more information gain, leading to a higher importance coefficient.

[0114] In some embodiments, feature fusion weights can be determined using an adaptive weight allocation method based on confidence distribution: First, a confidence distribution based on a Gaussian mixture model is constructed, and the entropy and mutual information of each distribution are calculated. Then, a feature correlation matrix is ​​constructed based on the entropy and mutual information, and the importance coefficients of the features are extracted using principal component analysis. Finally, the final weights are generated using an exponential weighted average method. Optionally, a weight optimization method based on fuzzy theory can also be used: By establishing a fuzzy membership function to characterize the uncertainty of the confidence value, the feature weights are calculated using a fuzzy comprehensive evaluation method, and then the weights are globally optimized using a genetic algorithm. It is understood that other weight determination strategies can also be used to achieve a quantitative assessment of feature importance, which is not limited here.

[0115] S210. Perform feature fusion on hash features, local features and deep features according to weight coefficients to generate a fused feature vector.

[0116] Referring to step S106, the image processing system will perform feature fusion.

[0117] In some embodiments, the image processing system optimizes the feature fusion effect through feature space projection and nonlinear transformation. Specifically, the image processing system projects hash features, local features, and depth features onto a preset dimensional space to generate projected features; it then weights and combines the projected features according to weight coefficients to generate initial fused features; based on the variance distribution of the initial fused features, it selects features to determine the main feature components and performs orthogonal transformation on the main feature components to obtain key features; finally, it processes the key features based on a nonlinear activation function to obtain a fused feature vector.

[0118] Among them, the preset dimension space represents a unified feature representation space; the projected feature refers to the feature vector after dimensional transformation; the weighted combination represents the feature fusion operation process based on weights; the initial fusion feature is used to represent the feature representation after preliminary fusion; the variance distribution refers to the degree of numerical dispersion of each dimension of the feature; the main feature components represent the feature dimensions that have a greater impact on the recognition result; the orthogonal transformation refers to the linear transformation that maintains the independence between features; the key features are used to represent the key features after dimensionality reduction and transformation; and the nonlinear activation function refers to the mathematical function used for nonlinear transformation of features.

[0119] After acquiring multiple different types of features, image processing systems need to effectively fuse them into a unified feature representation. Specifically, the image processing system first maps features of different dimensions to a feature space of the same dimension using methods such as principal component analysis or linear discriminant analysis to ensure feature comparability. Then, it performs a weighted summation of each feature based on pre-calculated weight coefficients to obtain the initial fused features. Next, it calculates the variance of each dimension of the fused features, selects the dimension with the larger variance as the principal feature component, and orthogonals these components using methods such as SVD decomposition to obtain mutually independent key features. Finally, it uses nonlinear functions such as ReLU or tanh to activate the key features, enhancing their expressive power.

[0120] The image processing system employs an attention mechanism to achieve weighted combination of features. First, a linear transformation maps different types of features to a feature space of the same dimension, generating query vectors, key vectors, and value vectors. The dot product of the query vector and key vector is calculated and normalized using a scaling factor (usually the square root of the feature dimension) to obtain the attention weight matrix. Multiplying the attention weights by the value vector yields a fused representation that considers the correlations between features. The system also introduces a multi-head attention mechanism, capturing feature correlations across different subspaces through multiple independent linear transformations. Finally, the outputs of the multi-head attention are concatenated and processed by a feedforward network to generate the initial fused features. For example, when processing product images containing multiple viewpoints, the attention mechanism can automatically identify and strengthen the most representative viewpoint features while weakening occluded or blurred viewpoint features, thus obtaining a more reliable fused representation.

[0121] In some embodiments, a unified representation of multiple features can be achieved through feature fusion based on deep autoencoders: first, a multi-layer neural network structure is constructed, and different features are input into their respective encoders for dimensionality reduction and feature extraction; then, feature fusion is performed in the hidden layer space; finally, the features are reconstructed through a decoder, and the reconstruction error is minimized to achieve adaptive feature fusion. Alternatively, a feature fusion method based on tensor decomposition can be used: multiple features are constructed into a high-order tensor, and the core features of the tensor are extracted using methods such as Tucker decomposition or CP decomposition; then, the fused features are obtained by tensor reconstruction. It is understood that other feature fusion strategies can also be used to effectively integrate heterogeneous features, and this is not limited here.

[0122] S211. Calculate the feature distribution density of multiple images to be compared and generate feature cluster centers.

[0123] Among them, the feature distribution density represents the distribution of the images to be compared in the feature space; the feature cluster center refers to the center point that can represent the features of a class of similar images; the calculation of the distribution density involves the distance measurement and probability statistics between feature vectors.

[0124] Image processing systems need to analyze the feature distribution patterns of the set of images to be compared in order to establish an index structure for subsequent fast retrieval. Specifically, the image processing system first calculates the distance matrix between the feature vectors of each image to be compared, and then calculates the local density distribution in the feature space based on the kernel density estimation method. According to the density distribution, a clustering algorithm is used to automatically determine the appropriate number of clusters, calculate the center point position of each category, and form a set of feature cluster centers.

[0125] The image processing system employs a density peak clustering algorithm to generate feature cluster centers. First, it calculates the Euclidean distance matrix between all point pairs in the feature space. Based on this matrix, it calculates the local density ρi and the minimum distance δi to high-density points for each point. The local density is calculated using a Gaussian kernel function, with the kernel width adaptively set to 2% of the average of all distances. For each point, a discriminant factor γi = ρi × δi is calculated, and points with larger discriminant factors are selected as candidate cluster centers. The system determines the final number of cluster centers by calculating the mutual information between candidate points, stopping selection when the information gain from adding new cluster centers is less than a threshold. For example, when processing an image library containing multiple product categories, the algorithm can automatically identify high-density feature regions as cluster centers; these centers often correspond to representative product category prototypes.

[0126] In some embodiments, feature distribution analysis can be achieved using an adaptive density clustering method: first, a density field of the feature space is constructed, density peaks and valleys are identified, then feature points are automatically clustered based on the density connectivity principle, and finally, the centroid of each category is calculated as the cluster center. Optionally, a hierarchical clustering method can also be used: through a bottom-up merging strategy, similar feature points are gradually combined into larger categories until the desired number of clusters is reached. It is understood that other clustering methods can also be used to partition the feature space, and the specific clustering strategy can be selected based on the characteristics of the feature distribution.

[0127] S212. Construct a feature index tree based on feature cluster centers to obtain the retrieval structure.

[0128] Among them, the feature index tree is a hierarchical feature organization structure; the retrieval structure contains information such as parent-child relationships between nodes and distance thresholds; the construction process involves spatial partitioning and the generation of tree structures.

[0129] Image processing systems require efficient feature retrieval structures to support rapid similar image searches. Specifically, the image processing system uses feature clustering centers as a foundation and recursively constructs a multi-level tree-like index structure. Each node stores local region information in the feature space, including the region's boundaries and the number of samples it contains. By appropriately setting node splitting conditions and merging rules, the balance of the tree structure and retrieval efficiency are ensured.

[0130] In some embodiments, feature indexing can be implemented using an adaptive KD-tree: the segmentation dimension and segmentation points are dynamically selected based on the principal direction of the feature distribution, constructing a non-uniform spatial partition that adapts to the feature distribution and improving pruning efficiency during retrieval. Optionally, a spherical tree structure can also be used: nested hyperspherical structures are constructed through recursive partitioning with the feature cluster centers as the spheres, enabling fast retrieval based on distance metrics. It is understood that other indexing structures can also be used to achieve efficient organization of the feature space; the specific implementation can be optimized according to retrieval requirements.

[0131] S213. Based on the retrieval structure, perform hierarchical filtering on the images to be compared to obtain a candidate image set.

[0132] Among them, hierarchical filtering refers to the process of gradually narrowing the search scope through multiple levels; the candidate image set refers to the set of potentially similar images after preliminary screening; and the retrieval structure is used to guide the search path in the filtering process.

[0133] Image processing systems need to efficiently filter out potentially similar images from a large number of images to be compared. Specifically, the image processing system starts from the root node of the feature index tree and selectively visits child nodes based on the distance relationship between the current query feature and the node region. During the traversal, a priority queue is maintained to record the most similar candidate image found so far. By setting reasonable pruning conditions, traversing branches that are unlikely to contain similar images is avoided, thereby improving the filtering efficiency.

[0134] In some embodiments, efficient filtering can be achieved through an approximate nearest neighbor search algorithm: setting a distance threshold and traversal depth limit, and improving retrieval speed by appropriately relaxing accuracy requirements while ensuring recall. Optionally, a multi-path parallel search strategy can also be adopted: simultaneously exploring multiple potential search paths, utilizing parallel computing resources to accelerate the discovery process of candidate images. It is understood that other filtering strategies can also be used to achieve rapid localization of candidate images, and the specific filtering method can be adjusted according to the actual application scenario.

[0135] In some embodiments, the image processing system achieves efficient hierarchical image retrieval based on a feature index structure. Specifically, the system acquires spatial distribution data of feature cluster centers and generates a feature index tree. This feature index tree includes a feature distance parameter and a node capacity parameter. The system calculates the spatial density value of the feature index tree to obtain a node distribution sequence, creating a mapping relationship between the node distribution sequence and the feature distance parameter. Based on this mapping relationship, the system statistically analyzes node access data to obtain pre-loaded node data, which is then stored in the retrieval cache. The system reads the pre-loaded node data from the retrieval cache and generates a fast retrieval table based on the access frequency of the pre-loaded node data. The system matches the feature data of the images to be compared with the fast retrieval table to obtain a preliminary image sequence. Finally, the system calculates the feature distance value of the preliminary image sequence and filters it based on the feature distance value to generate a candidate image set.

[0136] Among them, spatial distribution data represents the distribution of features in high-dimensional space; feature distance parameter is a metric for measuring feature similarity; node capacity parameter is used to represent the number of samples that an index tree node can hold; spatial density value represents the degree of clustering of samples in feature space; node distribution sequence refers to the spatial distribution information of each node in the index tree; node preload data represents pre-cached node information; access frequency is used to represent the frequency statistics of node access; fast retrieval table refers to the data structure used to accelerate retrieval; and feature distance value represents the similarity measurement result between features.

[0137] When constructing a feature index structure, image processing systems need to comprehensively consider both retrieval efficiency and index quality. Specifically, the system first analyzes the distribution patterns of feature cluster centers in the feature space, constructs a hierarchical index tree structure based on these distribution features, and sets appropriate distance thresholds and capacity limits for each node. Then, it calculates the sample density of each node region in the index tree, generating a node sequence reflecting the spatial distribution. Based on this information, it statistically analyzes node access patterns and pre-loads frequently accessed node data into the retrieval cache. Next, it constructs a fast retrieval table based on node access frequency to accelerate the matching process of similar features. Finally, it calculates the feature distance between the image to be compared and the retrieval results, and filters out the final candidate image set by setting a threshold.

[0138] The image processing system uses a hierarchical prediction model to generate node preload data. First, a time-series model of node access history is constructed, employing an LSTM network to predict the access probability of each node within a future time window. The model input includes the node's historical access frequency, time factor, and contextual features, with the output being an access probability distribution. Based on the predicted access probabilities and node data size, the system optimizes the preload strategy using a knapsack algorithm to maximize the preload hit rate within a limited cache space. Simultaneously, an LRU-based eviction queue is maintained, prioritizing the eviction of node data with lower predicted access probabilities when cache space is insufficient. For example, in e-commerce scenarios, popular product categories often have corresponding index tree nodes with high predicted access probabilities, and the system prioritizes loading the data of these nodes into the cache.

[0139] The image processing system employs an adaptive threshold strategy for feature distance filtering. First, kernel density estimation is performed on the distance values ​​in the initial image sequence to obtain the probability density function of the distance distribution. By analyzing the local extrema of the probability density function, the main patterns of the distance distribution are determined. The system uses an expectation-maximization-based method to cluster the distance values ​​into multiple intervals and calculates the statistical characteristics of each interval. The filtering threshold is determined by optimizing the goal of minimizing the variance within the interval and maximizing the distance between intervals. A dynamic adjustment mechanism is also introduced, adjusting the threshold parameters based on the accuracy feedback of historical filtering results. For example, when processing highly similar product images, the distance values ​​often exhibit obvious clustering characteristics, allowing the system to accurately locate the optimal filtering threshold; while for scenarios with more continuous similarity distributions, a relatively lenient threshold strategy is adopted.

[0140] In some embodiments, efficient feature retrieval can be achieved through index optimization based on locality-sensitive hashing (LSH): first, multiple LSH functions are designed to map high-dimensional features to low-dimensional hash codes; then, a hash bucket index structure is constructed; multi-table joint retrieval improves recall; and a probabilistic elimination strategy reduces computational overhead. Alternatively, a nearest-neighbor graph-based index construction method can be used: a nearest-neighbor graph structure is constructed through iterative optimization, utilizing the connectivity features of the graph to guide the retrieval process, and combining pruning strategies to reduce invalid search paths. It is understood that other index optimization strategies can also be used to achieve fast retrieval of large-scale features; these are not limited here.

[0141] S214. Based on the fused feature vector, perform parallel similarity calculation on the candidate image set to obtain a similarity result containing multiple final similarities.

[0142] Parallel similarity calculation refers to the simultaneous processing of similarity evaluations for multiple candidate images; the final similarity includes distance metrics and confidence information; the similarity results are used to quantify the degree of similarity between images.

[0143] Image processing systems need to accurately calculate the similarity between input images and candidate image sets. Specifically, the image processing system divides the candidate image set into multiple batches and uses parallel computing resources to process multiple similarity calculation tasks simultaneously. For each candidate image, the distance between its fused feature vector and the feature vector of the input image is calculated, and a comprehensive similarity score is generated by combining the importance weights of the features. Finally, the similarity results are normalized to obtain a standardized similarity index.

[0144] In some embodiments, similarity can be calculated using a weighted distance metric: weight coefficients are set according to the discriminative power of different feature dimensions to construct an adaptive distance metric function, thereby improving the accuracy of similarity calculation. Optionally, a multi-scale matching strategy can also be employed: local similarity is calculated at different levels of feature abstraction, and then a global similarity score is obtained through hierarchical integration. It is understood that other similarity calculation methods can also be used to achieve accurate evaluation of image similarity; the specific calculation method can be designed according to application requirements.

[0145] In this embodiment, by employing a progressive feature extraction and adaptive fusion technique, the system can automatically adjust the computational depth based on the complexity of image features. It only progressively delves into more complex features when simple features fail to yield reliable results, effectively avoiding wasted computational resources. Simultaneously, through feature importance evaluation based on information entropy and a dynamic weight allocation mechanism, the system ensures the reasonable fusion of different types of features, effectively solving the problem of insufficient feature fusion accuracy in traditional methods. Furthermore, by introducing group batch processing and parallel computing mechanisms, the system's computational efficiency is further optimized, thereby achieving the goal of significantly improving processing speed while maintaining recognition accuracy.

[0146] The image processing system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference needed]. Figure 3 This is a schematic diagram of the physical device structure of an image processing system in an embodiment of this application.

[0147] It should be noted that, Figure 3 The structure of the image processing system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0148] like Figure 3 As shown, the image processing system includes a CPU 301, which can perform various appropriate actions and processes according to a program stored in ROM 302 or a program loaded from storage section 308 into RAM 303, such as executing the methods described in the above embodiments. RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via bus 304. I / O interface 305 is also connected to bus 304.

[0149] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including hard disks, etc.; and communication section 309 including network interface cards such as LAN (Local Area Network) cards, modems, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0150] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by CPU 301, it performs the various functions defined in the present invention.

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0152] Specifically, the image processing system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the image similarity recognition method based on multi-feature fusion provided in the above embodiment.

[0153] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the image processing system described in the above embodiments; or it may exist independently and not assembled into the image processing system. The storage medium carries one or more computer programs that, when executed by a processor of the image processing system, cause the image processing system to implement the image similarity recognition method based on multi-feature fusion provided in the above embodiments.

[0154] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0155] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

Claims

1. An image similarity recognition method based on multi-feature fusion, characterized in that, Applied to an image processing system, the method includes: Calculate the perceptual hash value of the input image to obtain hash features, and calculate the difference between the hash features and the corresponding features of the image to be compared to generate hash similarity and hash confidence value; When the hash confidence value is less than the hash confidence reference value, local feature extraction is performed on the input image to obtain local features; Based on the local features, feature point matching is performed on the images to be compared to generate local similarity and local confidence values. When the local confidence value is less than the local confidence reference value, deep learning feature extraction is performed on the input image to obtain deep features and deep confidence values; Based on the hash confidence value, local confidence value, and deep confidence value, the weight coefficients of the hash feature, the local feature, and the deep feature are determined respectively. Specifically, this includes: calculating the information entropy of the hash confidence value, local confidence value, and deep confidence value respectively to obtain an entropy value sequence; normalizing the entropy value sequence to generate initial weights; calculating the feature importance coefficients of the hash confidence value, local confidence value, and deep confidence value based on the initial weights; adjusting the initial weights according to the feature importance coefficients to obtain adjusted weights; and performing nonlinear mapping processing on the adjusted weights to obtain weight coefficients. The hash feature, the local feature, and the deep feature are fused according to the weighting coefficients to generate a fused feature vector; Based on the fused feature vector, the final similarity between the input image and the image to be compared is calculated.

2. The method according to claim 1, characterized in that, The step of fusing the hash feature, the local feature, and the deep feature according to the weight coefficient to generate a fused feature vector specifically includes: The hash features, local features, and depth features are projected onto a preset dimension space to generate projected features; The projected features are weighted and combined according to weight coefficients to generate initial fused features; Feature selection is performed based on the variance distribution of the initial fusion features to determine the main feature components, and the main feature components are orthogonally transformed to obtain key features. The key features are processed using a nonlinear activation function to obtain a fused feature vector.

3. The method according to claim 1, characterized in that, The input images are multiple; Before the steps of calculating the perceptual hash value of the input image, obtaining hash features, calculating the difference between the hash features and the corresponding features of the image to be compared, and generating hash similarity and hash confidence values, the method further includes: Multiple input images are grouped according to their image size to generate an image group sequence; The batch size is calculated based on the number of images in each image group sequence, and batch partitioning information is generated. Based on the batch division information, a cache index is established for the images in each image group sequence to obtain an image cache table; The feature extraction order is determined based on the image cache table, and a parallel computing task queue is created.

4. The method according to claim 3, characterized in that, After the step of determining the feature extraction order based on the image cache table and creating a parallel computing task queue, the method further includes: The maximum memory usage of each image group sequence is determined based on the batch division information, and the batch size is adjusted based on the available system memory. Based on user settings, a priority identifier is determined for each of the image group sequences; The parallel computing task queue is sorted by priority according to the priority identifier to generate a task execution sequence.

5. The method according to claim 1, characterized in that, The images to be compared are multiple; The step of calculating the final similarity between the input image and the image to be compared based on the fused feature vector specifically includes: Calculate the feature distribution density of multiple images to be compared, and generate feature cluster centers; Based on the feature cluster centers, a feature index tree is constructed to obtain the retrieval structure; Based on the retrieval structure, the images to be compared are hierarchically filtered to obtain a candidate image set; Based on the fused feature vector, parallel similarity calculations are performed on the candidate image set to obtain similarity results containing multiple final similarities.

6. The method according to claim 5, characterized in that, The step of performing hierarchical filtering of the images to be compared according to the retrieval structure to obtain a candidate image set specifically includes: Obtain the spatial distribution data of the feature cluster centers and generate a feature index tree; the feature index tree includes feature distance parameters and node capacity parameters. Calculate the spatial density value of the feature index tree to obtain the node distribution sequence, such that the node distribution sequence forms a mapping relationship with the feature distance parameter; Based on the mapping relationship, the node access data is statistically analyzed to obtain the node preload data, and the node preload data is stored in the retrieval cache. Read the node preloaded data from the retrieval cache and generate a fast retrieval table based on the access frequency of the node preloaded data; The feature data of the images to be compared are matched with the fast retrieval table to obtain a preliminary image sequence; Calculate the feature distance value of the initially selected image sequence, and filter based on the feature distance value to generate a candidate image set.

7. An image processing system, characterized in that, The image processing system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the image processing system to perform the method as described in any one of claims 1-6.

8. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the image processing system, the image processing system performs the method as described in any one of claims 1-6.

9. A computer program product, characterized in that, When the computer program product is run on the image processing system, the image processing system performs the method as described in any one of claims 1-6.

Images