Image retrieval method based on average point embedding vector learning and related device

Abstract

The invention belongs to the technical field of image retrieval, and discloses an image retrieval method based on average point embedding vector learning and a related device, an average point embedding vector is obtained by calculating the average value of all sample feature embedding vectors of each category, and a similarity item of a baseline loss function is reconstructed according to the average point embedding vector, so that the image retrieval efficiency is improved. And forming a loss function of the positive and negative sample pairs, further constructing an overall loss function for training a deep neural network model, and finally outputting an image retrieval model. By adopting the method, the sampling complexity and computing resource requirements are greatly reduced, the training efficiency and expandability of the model on a large-scale data set are improved, meanwhile, the robustness to noise and outliers is enhanced, the training process is more stable, and the generalization performance of the model is improved.

Description

Technical Field

[0001] This invention belongs to the field of image retrieval technology, and particularly relates to an image retrieval method and related apparatus based on average point embedding vector learning. Background Technology

[0002] In the field of image retrieval technology, with the rapid development of deep learning, deep metric learning has become a key technological support for improving the performance of core tasks such as image semantic recognition and target matching. Its core task is to train deep neural networks to map input data such as images into a high-dimensional embedding space, so that semantically similar samples are close to each other in this space, while semantically dissimilar samples are far apart. This provides effective feature representations for complex needs such as fine-grained recognition and cross-scene matching in image retrieval. Currently, deep metric learning has been widely applied in many important fields such as autonomous driving, security monitoring, and medical image analysis. Optimizing the performance of related technologies is of great significance for promoting the intelligent upgrading of these industries. To achieve the above core objectives, researchers have designed various loss functions, such as contrastive loss, triplet loss, and N-pair loss. These sample combination-based methods constitute the mainstream framework of existing deep metric learning technologies.

[0003] However, existing deep metric learning methods based on sample sampling strategies have significant problems and drawbacks, making it difficult to meet the demands of image retrieval technology towards large-scale, high-precision, and high-stability applications. On the one hand, the sampling complexity is extremely high; as the amount of training data increases, the number of sample pairs or triples that may be formed explodes exponentially, for example, in a mini-batch containing m samples, it can result in O(m... 2 ) sample pairs or O(m 3 Selecting effective sample combinations for model training from a massive number of triples is not only difficult but also computationally expensive, severely limiting the efficiency of models on large-scale image datasets. Furthermore, existing methods are extremely sensitive to outliers and noise. Most methods employ hard example mining strategies to accelerate convergence, focusing on the most difficult-to-distinguish samples. However, mislabeled data, extremely poor-quality images, and other noise or outliers easily become these "hard examples," causing the model to overemphasize such samples, leading to training instability and significantly impairing the model's generalization performance. Although some studies have proposed smoother weighting strategies such as logarithmic summation and exponential weighting to replace hard hard example mining, these still do not fundamentally solve the problem of low sampling efficiency caused by the explosion of sample combinations.

[0004] Therefore, it is evident that existing image retrieval methods based on sample sampling in the current field of image retrieval technology suffer from high sampling complexity and sensitivity to outliers and noise. Summary of the Invention

[0005] This invention provides an image retrieval method and related apparatus based on average point embedding vector learning. This method can effectively solve the problems of high sampling complexity and sensitivity to outliers and noise in existing image retrieval methods based on sample sampling.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: An image retrieval method based on average point embedding vector learning includes: Acquire raw image data; The original image data is used as the training dataset to train a pre-built deep neural network model, outputting an image retrieval model. The training process of the deep neural network model is as follows: Extract feature embedding vectors from the training dataset; Calculate the average of the feature embedding vectors of all samples for each category to obtain the average point embedding vector for each category; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector to obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding vector loss function is constructed based on the loss function for positive sample pairs and the loss function for negative sample pairs. The deep neural network model is trained by combining the average point embedding vector and the average point embedding vector loss function to obtain the image retrieval model; An image retrieval model is used to retrieve the image data to be retrieved, and the image retrieval results are output.

[0007] Furthermore, before calculating the average of the feature embedding vectors for all samples corresponding to each category, the method further includes: Construct a logarithmic summation exponential function and a baseline loss function; wherein, the logarithmic summation exponential function is used to smoothly weight all positive and negative sample pairs of the baseline loss function, and the logarithmic summation exponential function... The specific expression is as follows:

[0008] In the formula, x represents a set of similarity scores; t represents the temperature coefficient, used to adjust the smoothness of the weighting; exp represents the natural exponential function; and i represents the index number. The baseline loss function It is constructed based on the sum of the weighted similarities of all negative sample pairs and the sum of the weighted similarities of all positive sample pairs, and is specifically expressed as follows:

[0009] In the formula, and Let a and m represent the similarity sets between the anchor sample and all negative samples and all positive samples, respectively; a represents the number of samples; m represents the total number of samples. express The corresponding temperature coefficient, express The corresponding temperature coefficient; This indicates an indicator function.

[0010] Further, the calculation of the average value of the feature embedding vectors for all samples corresponding to each category includes: For each class C in the training dataset, calculate the average of the feature embedding vectors of all samples belonging to the current class to obtain the average point embedding vector of the corresponding class. The specific formula is as follows:

[0011] In the formula, This represents the set of samples belonging to category C; i is the sample index. Let represent the feature embedding vector of the i-th sample.

[0012] Furthermore, the step of reconstructing the similarity term in the predefined baseline loss function using the average point embedding vector includes: Obtain the calculated average point embedding vector; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector. The specific expression is as follows: ; In the formula, Let the transpose of the feature embedding vector of the i-th sample be denoted as . This represents the average point embedding vector of the j-th sample; The loss functions for positive sample pairs and negative sample pairs are constructed based on the similarity term. The loss function of the positive sample pairs The specific expression is as follows:

[0013] In the formula, a represents the anchor point; m represents the number of anchor points; and p represents the positive sample. The set representing positive samples; This indicates the number of positive samples corresponding to anchor point a; This represents the average point embedding vector of anchor point a; This represents the average point embedding vector of the positive sample p; C represents the average embedding vector of positive samples corresponding to the category to which anchor point a belongs; C represents the total number of categories; The loss function of the negative sample pairs The specific expression is as follows:

[0014] In the formula, n represents the negative sample; It is a set of negative samples; It is the number of negative samples corresponding to anchor point a; This represents the average point embedding vector of the positive sample n; Indicates a category index; Indicates the category to which anchor point 'a' belongs; Indicates the first The average embedding vector of each negative class; This represents the average embedding vector of all negative classes corresponding to anchor point a.

[0015] Furthermore, the construction of the average point embedding vector loss function based on the loss function for positive sample pairs and the loss function for negative sample pairs includes: Obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding loss function is constructed based on the loss functions for positive and negative sample pairs. The specific expression is as follows: .

[0016] Furthermore, before training the pre-built deep neural network model using the original image data as the training dataset, the process further includes: A deep neural network model is constructed, which serves as a feature extractor to extract features from each input sample to obtain a multidimensional feature embedding vector.

[0017] Furthermore, after extracting the feature embedding vectors from the training dataset, the process further includes: The extracted feature embedding vectors are normalized so that they are distributed on the unit hypersphere.

[0018] An image retrieval system based on average point embedding vector learning includes: The data acquisition module is used to acquire raw image data; The model training module is used to train a pre-built deep neural network model using the original image data as the training dataset, and outputs an image retrieval model. The training process of the deep neural network model is as follows: Extract feature embedding vectors from the training dataset; Calculate the average of the feature embedding vectors of all samples for each category to obtain the average point embedding vector for each category; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector to obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding vector loss function is constructed based on the loss function for positive sample pairs and the loss function for negative sample pairs. The deep neural network model is trained by combining the average point embedding vector and the average point embedding vector loss function to obtain the image retrieval model; The retrieval module is used to retrieve the image data to be retrieved using an image retrieval model and output the image retrieval results.

[0019] An image retrieval device based on average point embedding vector learning includes: Memory, used to store computer programs; A processor is used to implement the steps of the image retrieval method based on average point embedding vector learning described above when executing the computer program.

[0020] A computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the steps of the image retrieval method based on average point embedding vector learning described above.

[0021] Compared with the prior art, the present invention has the following beneficial effects: This invention provides an image retrieval method based on average point embedding vector learning. It calculates the average of the feature embedding vectors of all samples in each category to obtain the average point embedding vector, and uses this to reconstruct the similarity term of the baseline loss function, forming a loss function for positive and negative sample pairs. This overall loss function is then used to train a deep neural network model, ultimately outputting an image retrieval model. This method uses the average point of each category as a surrogate point, replacing the direct sampling of traditional sample pairs or triples, avoiding the combinatorial explosion problem. Because the calculation only involves category-level averaging rather than individual sample combinations, it significantly simplifies similarity calculation. Simultaneously, the averaging operation smooths out the influence of noise and outliers, weakening the interference of mislabeled or low-quality samples, thereby alleviating the sensitivity of hard example mining to noise. This method significantly reduces sampling complexity and computational resource requirements, improves the training efficiency and scalability of the model on large-scale datasets, enhances robustness to noise and outliers, makes the training process more stable, and improves the model's generalization performance. Attached Figure Description

[0022] Figure 1 A table illustrating the recall rate @1 on the Cars-196 and SOP datasets provided in this embodiment of the invention; Figure 2A table illustrating the search results on the Market1501, DukeMTMC, and CUHK03-detected datasets provided in this embodiment of the invention; Figure 3 A flowchart of an image retrieval method based on average point embedding vector learning provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an image retrieval system based on average point embedding vector learning, provided in an embodiment of the present invention. Detailed Implementation

[0023] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0024] The technical terms used in this invention are explained as follows: APE Learning: short for Average Point Embedding Learning.

[0025] APE loss: short for Average Point Embedding loss, which is the average point embedding learning loss.

[0026] Contrastive Loss:

[0027] Triplet Loss: The loss of a triplet.

[0028] N-pair Loss: N pairs of losses.

[0029] LSE stands for Log-Sum-Exp, which is the logarithmic summation exponential function.

[0030] Hard Mining: Digging out difficult cases.

[0031] SOTA stands for State-Of-The-Art, referring to the current best level. In this invention, the SOTA method represents the current optimal method.

[0032] Baseline: refers to the baseline method.

[0033] Proxy-NCA: short for No Fuss Distance Metric Learning using Proxies, is a metric learning method based on proxies.

[0034] Cars-196 Dataset: The full name is Stanford Cars Dataset, which is a classic fine-grained image classification and retrieval dataset containing 16,185 images of 196 car categories for vehicle model recognition (brand, model, year).

[0035] SOP dataset: short for Stanford Online Products, is a dataset commonly used for metric learning and image retrieval. It contains approximately 120,000 product images and is used to learn image similarity.

[0036] Recall@1: Refers to recall rate@1, which is a core metric used to evaluate model performance in the fields of information retrieval and machine learning.

[0037] Market-1501, DukeMTMC-reID, and CUHK03-Detected are three widely used public datasets in the field of person re-identification.

[0038] mAP stands for Mean Average Precision.

[0039] MS stands for Multi-Similarity.

[0040] Contrastive loss is a fundamental loss function in metric learning. Its core idea is to bring similar samples (positive sample pairs) closer together and push dissimilar samples (negative sample pairs) further apart.

[0041] Lifted Struct: This refers to improving structural loss.

[0042] HDC stands for Cascaded Semi-Hard Approach for Hard Example Mining.

[0043] Clustering: also known as clustering, is an unsupervised learning method.

[0044] Smart Mining: refers to intelligent mining.

[0045] Margin: In classification tasks, it refers to the distance between the decision boundary and samples of each class. A larger margin usually means that the model has better generalization ability.

[0046] HTL stands for Hierarchical Transfer Learning.

[0047] A-BIER stands for Adaptive Deep Metric Learning with Band-based ImportanceEstimation.

[0048] ABE stands for Attribute-Based Embedding.

[0049] Pose-guided ReID: A pose-guided re-identification method.

[0050] GLAD stands for Global-Local Attention Detection.

[0051] PIE stands for Pose Invariant Embedding.

[0052] PSE stands for Pose-Sensitive Embedding.

[0053] Mask-guided: Mask-guided.

[0054] SP ReID: full name Semantic Parsing guided Re-Identification.

[0055] Mask ReID: A mask-based method for pedestrian re-identification.

[0056] Stripe-based ReID: A re-identification method based on stripe patterns.

[0057] Aligned ReID: An aligned method for pedestrian re-identification.

[0058] PCB: Full name Part-based Convolutional Baseline.

[0059] Pyramid: Methods related to pyramid structures.

[0060] BFE stands for Batch Feature Erasing.

[0061] Attention-based ReID: A re-identification method based on an attention mechanism.

[0062] Mancs stands for A Multi-task Attentional Network with Curriculum Sampling for Re-identification.

[0063] DuATM stands for Dual-Attention Matching network.

[0064] HA - CNN: short for Harmonious Attention Convolutional Neural Network.

[0065] DAAF - BoT: A deep learning method that combines a deep attention-based feature learning mechanism with the classic Bagof Tricks (BoT) model, aiming to make the network pay more attention to the discriminative regions of pedestrians. ("Bag of Tricks" is a specific term, generally referring to a set of techniques, strategies, or improvements that can effectively enhance model performance. These techniques may involve data augmentation, loss function design, network structure optimization, and other aspects. In technical literature, for accuracy and conciseness, the original English term "Bag of Tricks" or the abbreviation "BoT" is often used directly.)

[0066] GAN-based ReID: A re-identification method based on generative adversarial networks.

[0067] Camstyle: A data augmentation method that reduces style differences between different cameras and increases the diversity of training data through camera style transfer, thereby improving the model's generalization ability.

[0068] PN-GAN: Part-based Neural Network with GAN; where GAN is an abbreviation for Generative Adversarial Network.

[0069] DGNet: short for Disentangled Generative Network.

[0070] Part – feature: partial features.

[0071] BPBReID: A pedestrian re-identification method that, in occluded scenarios, learns a more robust pedestrian representation to occlusion by extracting body features.

[0072] Text-based ReID: A text-based re-identification method.

[0073] PLIP stands for Parallel-data-free Image-to-image Translation with Limited textual supervision.

[0074] IDE: Full name ID - discriminative Embedding, which is identity discrimination embedding.

[0075] SVDNet: short for Singular Vector Decomposition Network.

[0076] TriNet: short for Triplet-based Network, which is a network based on triples.

[0077] AWTL: An improved loss function, namely Adaptive Weighted Triplet Loss, which makes model training more efficient by dynamically adjusting the weights of hard samples.

[0078] Trans ReID: short for Transformer-based Re-Identification, is a re-identification method based on Transformer.

[0079] Flip-ReID: A flip-based re-identification method.

[0080] ProNet++: Rethinks pedestrian re-identification from a new perspective of prototype projection, emphasizing the learning of better features by projecting sample vectors onto representative prototypes in the feature space.

[0081] LUP: A large-scale unlabeled pedestrian re-identification dataset for unsupervised pre-training, designed to improve model performance by leveraging large amounts of unlabeled data and methods.

[0082] ResNet-50: full name Residual Network with 50 Layers.

[0083] APE + ResNet-50: Average point embedding learning with a 50-layer residual network.

[0084] APE + ResNet-50+re-ranking: Average point embedding learning with a 50-layer residual network combined with re-ranking.

[0085] As mentioned in the background section, current image retrieval methods suffer from the following drawbacks: First, high sampling complexity: As the amount of training data increases, the number of possible sample pairs or triples explodes exponentially, on a quadratic or cubic scale. For example, in a minibatch containing m samples, it can result in O(m... 2 ) sample pairs or O(m 3 There are 30 triplet pairs. Selecting "effective" (i.e., helpful for model training) sample combinations from a massive number of combinations is itself a very difficult and time-consuming problem. Second, it is sensitive to outliers and noise: To accelerate convergence, many methods adopt a "hard example mining" strategy, which focuses on the most difficult-to-distinguish samples (such as the farthest positive sample pair and the closest negative sample pair). However, this strategy is highly susceptible to noise or outliers in the data. A mislabeled image or a very poor-quality image can become the "hardest" sample. If the model focuses too much on these outliers, it will lead to instability in the training process and even impair the model's generalization performance.

[0086] To address the aforementioned issues, this embodiment provides an image retrieval method based on average point embedding vector learning. This method replaces a massive number of individual samples with "average features of categories," thereby simplifying the complex "point-to-point" comparison problem into an efficient "point-to-class" comparison problem. This effectively solves the problem that most existing methods over-design mining, weighting, and sampling strategies, resulting in slow convergence speed and poor learning performance. This method introduces a novel average point embedding (APE) loss function, which is learned through average point embedding vectors rather than original data samples, thereby significantly reducing the sampling complexity from exponential to linear. Extensive experimental validation on image retrieval and pedestrian re-retrieval tasks confirms the superiority of the proposed method, achieving state-of-the-art performance while significantly improving convergence speed.

[0087] like Figure 3 As shown, this embodiment provides an image retrieval method based on average point embedding vector learning, including: Acquire raw image data; The original image data is used as the training dataset to train a pre-built deep neural network model, outputting an image retrieval model. The training process of the deep neural network model is as follows: Extract feature embedding vectors from the training dataset; Calculate the average of the feature embedding vectors of all samples for each category to obtain the average point embedding vector for each category; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector to obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding vector loss function is constructed based on the loss function for positive sample pairs and the loss function for negative sample pairs. The deep neural network model is trained by combining the average point embedding vector and the average point embedding vector loss function to obtain the image retrieval model; An image retrieval model is used to retrieve the image data to be retrieved, and the image retrieval results are output.

[0088] The retrieval method provided in this embodiment will be further explained below with reference to the accompanying drawings: This embodiment provides an image retrieval method based on average point embedding vector learning, the specific steps of which are as follows: Step 1: Data Acquisition and Feature Extraction S11. Data Collection: Obtain the training dataset with category labels. ,in It is the input sample. It is its category label.

[0089] S12. Feature Extraction: Construct a deep neural network (backbone network, such as ResNet) as a feature extractor for each input sample. Extract its d-dimensional feature embedding vector To ensure the effectiveness of the measurement, this embodiment performs a normalization operation on all extracted feature embedding vectors, preferably using L2 normalization (Euclidean norm normalization), so that the feature embedding vectors are distributed on a unit hypersphere, thereby optimizing the measurement properties of the features, stabilizing the training process, and enhancing discriminability.

[0090] Step 2: Construct the baseline weighted loss function: This embodiment first constructs a smoothly weighted baseline loss function for subsequent improvements. The smoothly weighted baseline loss function uses the Log-Sum-Exp (LSE, logarithmic summation exponential function) function to smoothly weight all positive and negative sample pairs, thereby avoiding the drawbacks of hard example mining.

[0091] S21. Define the LSE function (construct a logarithmic summation exponential function):

[0092] In the formula, x represents a set of similarity scores; t represents the temperature coefficient, used to adjust the smoothness of the weighting; exp represents the natural exponential function; and i represents the index number. The larger t is, the closer it is to the maximum value (hard mining); the smaller t is, the closer it is to the average value.

[0093] S22. Constructing the baseline loss: For an anchor sample, the baseline loss function of the anchor sample is composed of the sum of the weighted similarities of all negative sample pairs and the sum of the weighted similarities of all positive sample pairs.

[0094]

[0095] In the formula, and Let a and m represent the similarity sets between the anchor sample and all negative samples and all positive samples, respectively; a represents the number of samples; m represents the total number of samples. express The corresponding temperature coefficient, express The corresponding temperature coefficient; This indicates the indicator function. It is defined when the hardest positive sample is "harder" than the hardest negative sample (i.e., ...). If the sample size is 1, then the loss is calculated to ignore simple samples, thereby accelerating convergence. Indicates the minimum value; This represents the maximum value.

[0096] Step 3: Introduce average point embedding for optimization: This step is the core of this method. It uses the "average point embedding" of each class to represent all samples of that class, thus completely eliminating the reliance on individual sample pairs.

[0097] S31. Calculate the average point embedding vector: For each class C in the training dataset, calculate the average of the feature embedding vectors of all samples belonging to that class to obtain the average point embedding vector for that class. .

[0098]

[0099] in, It is a sample set belonging to category C.

[0100] S32. Reconstructing the Loss Function: Redefining the similarity term in the baseline loss using the average point embedding vector. Furthermore, the loss functions for positive sample pairs and negative sample pairs are reconstructed based on the similarity term; where: Loss function for positive sample pairs The specific expression is as follows:

[0101] In the formula, a represents the anchor point; m represents the number of anchor points; and p represents the positive sample. The set representing positive samples; This indicates the number of positive samples corresponding to anchor point a; This represents the average point embedding vector of anchor point a; This represents the average point embedding vector of the positive sample p; C represents the average embedding vector of positive samples corresponding to the category to which anchor point a belongs; C represents the total number of categories; Loss function for negative sample pairs The specific expression is as follows:

[0102] In the formula, n represents the negative sample; It is a set of negative samples; It is the number of negative samples corresponding to anchor point a; This represents the average point embedding vector of the positive sample n; Indicates a category index; Indicates the category to which anchor point 'a' belongs; Indicates the first The average embedding vector of each negative class; This represents the average embedding vector of all negative classes corresponding to anchor point a.

[0103] S33. Construct the final APE loss function (Average Point Embedding Vector Loss Function): Substitute the similarity based on average point embedding above into the baseline loss function framework to obtain the final average point embedding loss function, which is to combine the loss function of positive sample pairs and the loss function of negative sample pairs to obtain the average point embedding loss function. The specific expression is as follows: .

[0104] Step 4: Train the pre-built deep neural network model using the average point embedding vector loss function and the training dataset to optimize and obtain the image retrieval model.

[0105] Step 5: Input the image data to be retrieved into the image retrieval model and output the image retrieval results.

[0106] Therefore, in this embodiment, the average point embedding loss function indicates that for a given anchor point embedding... If we can calculate similar positive average point embeddings Other Embedded with dissimilar negative average points Then the sampling complexity can be reduced from exponential to linear. .

[0107] Therefore, this embodiment provides an image retrieval method based on average point embedding vector learning, which has the following advantages compared to existing image retrieval methods: First, this method greatly reduces the sampling complexity: This invention reduces the O(m) sampling complexity of traditional methods. 2 ) or O(m 3 The complexity of sample pair comparison is reduced to O(m×C) (where m is the batch size and C is the number of classes), where C is typically much smaller than m. This eliminates the need for complex online sampling during training, resulting in faster convergence and higher computational efficiency.

[0108] Second, this method enhances robustness to noise and outliers: by using the "average" features of the classes, the influence of individual outliers or noisy samples on class centers is greatly reduced. The model is no longer misled by a few extremely difficult samples, the training process is more stable, and the learned features have stronger generalization ability.

[0109] Third, it makes full use of global information: The APE loss function implicitly uses the class structure information within the entire batch to guide the learning of each sample, providing a more global optimization perspective compared to methods that only focus on local sample pairs or triples.

[0110] Fourth, superior performance: Experiments have shown that on several mainstream pedestrian re-retrieval and image retrieval benchmark datasets, the model trained using the APE loss function of this invention significantly outperforms the model using traditional sampling and hard example mining methods.

[0111] For example, specific experimental verification was carried out on the image retrieval method based on average point embedding vector learning provided in this embodiment, such as... Figure 1 As shown, Figure 1 The baseline method and Average Point Embedding Loss (APE loss) proposed in this embodiment are highlighted in bold for clear comparison with other existing methods. Experiments show that the baseline loss function outperforms most state-of-the-art (SOTA) methods. Furthermore, the APE loss function proposed in this embodiment improves the Recall@1 metric by 16.4% on the CARS-196 dataset and by 5.2% on the SOP dataset compared to the typical agent-based metric learning method Proxy-NCA. While the latest multi-similarity loss function MS loss employs a sample pair weighting strategy, its Recall@1 metric is still 5.5% and 0.7% lower than our proposed method on the CARS-196 and SOP datasets, respectively.

[0112] In summary, the APE learning loss function proposed in this embodiment achieves state-of-the-art performance on both fine-grained datasets (such as CARS-196) and large-scale datasets with a large number of categories (such as SOP).

[0113] like Figure 2 As shown, Figure 2 middle, The number of features in the inference phase is represented by the value '[']. It can be seen that the baseline loss function, using only a single global feature, achieved mAP scores of 81.0%, 69.3%, and 65.5% on these three datasets, respectively. The APE loss further improved the baseline performance, achieving additional mAP improvements of +5.0%, +7.9%, and +5.8% on the corresponding datasets. Through re-ranking techniques, the APE loss achieved an mAP score of 94.3% on Market-1501, 88.9% on DukeMTMC, and 82.7% on CUHK03-Detected, demonstrating significant effectiveness.

[0114] like Figure 4 As shown, this embodiment also provides an image retrieval system based on average point embedding vector learning, including: The data acquisition module is used to acquire raw image data; The model training module is used to train a pre-built deep neural network model using the original image data as the training dataset, and outputs an image retrieval model. The training process of the deep neural network model is as follows: Extract feature embedding vectors from the training dataset; Calculate the average of the feature embedding vectors of all samples for each category to obtain the average point embedding vector for each category; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector to obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding vector loss function is constructed based on the loss function for positive sample pairs and the loss function for negative sample pairs. The deep neural network model is trained by combining the average point embedding vector and the average point embedding vector loss function to obtain the image retrieval model; The retrieval module is used to retrieve the image data to be retrieved using an image retrieval model and output the image retrieval results.

[0115] The present invention also provides an image retrieval device based on average point embedding vector learning, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the image retrieval method based on average point embedding vector learning.

[0116] The present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the image retrieval method based on average point embedding vector learning.

[0117] When the processor executes the computer program, it implements the above-mentioned steps of image retrieval based on average point embedding vector learning, for example: acquiring original image data; using the original image data as a training dataset to train a pre-constructed deep neural network model and outputting an image retrieval model; wherein, the training process of the deep neural network model is as follows: extracting feature embedding vectors from the training dataset; calculating the average value of the feature embedding vectors of all samples corresponding to each category to obtain the average point embedding vector of each category; reconstructing the similarity term in the predefined baseline loss function using the average point embedding vector to obtain the loss function for positive sample pairs and the loss function for negative sample pairs; constructing the average point embedding vector loss function based on the loss function for positive sample pairs and the loss function for negative sample pairs; training the deep neural network model by combining the average point embedding vector and the average point embedding vector loss function to obtain the image retrieval model; using the image retrieval model to retrieve the image data to be retrieved and outputting the image retrieval result.

[0118] Exemplarily, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing preset functions, wherein the instruction segments describe the execution process of the computer program in the image retrieval device based on average point embedding vector learning. For example, the computer program can be divided into a data acquisition module, a model training module, and a retrieval module, with the following specific functions: The data acquisition module is used to acquire raw image data; The model training module is used to train a pre-built deep neural network model using the original image data as the training dataset, and outputs an image retrieval model. The training process of the deep neural network model is as follows: Extract feature embedding vectors from the training dataset; Calculate the average of the feature embedding vectors of all samples for each category to obtain the average point embedding vector for each category; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector to obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding vector loss function is constructed based on the loss function for positive sample pairs and the loss function for negative sample pairs. The deep neural network model is trained by combining the average point embedding vector and the average point embedding vector loss function to obtain the image retrieval model; The retrieval module is used to retrieve the image data to be retrieved using an image retrieval model and output the image retrieval results.

[0119] The image retrieval device based on average point embedding vector learning can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The image retrieval device based on average point embedding vector learning may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above are examples of image retrieval devices based on average point embedding vector learning and do not constitute a limitation on such devices. The device may include more components than described above, or combine certain components, or use different components. For example, the image retrieval device based on average point embedding vector learning may also include input / output devices, network access devices, buses, etc.

[0120] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or any conventional processor. The processor is the control center of the image retrieval based on average point embedding vector learning, connecting various parts of the image retrieval device using various interfaces and lines.

[0121] The memory can be used to store the computer program and / or modules. The processor implements various functions of the image retrieval device based on average point embedding vector learning by running or executing the computer program and / or modules stored in the memory and calling the data stored in the memory.

[0122] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, image playback, etc.). The data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory may include high-speed random access memory and non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0123] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the image retrieval method based on average point embedding vector learning.

[0124] If the modules / units integrated in the image retrieval system based on average point embedding vector learning are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0125] Based on this understanding, the present invention can implement all or part of the processes in the image retrieval method based on average point embedding vector learning, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the image retrieval method based on average point embedding vector learning. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or a preset intermediate form, etc.

[0126] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0127] It should be noted that the content contained in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0128] The above embodiments are merely one of the implementation methods for achieving the technical solution of the present invention. The scope of protection claimed by the present invention is not limited to this embodiment, but also includes any variations, substitutions and other implementation methods that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. An image retrieval method based on average point embedding vector learning, characterized in that, include: Acquire raw image data; The original image data is used as the training dataset to train a pre-built deep neural network model, outputting an image retrieval model. The training process of the deep neural network model is as follows: Extract feature embedding vectors from the training dataset; Calculate the average of the feature embedding vectors for all samples in each category to obtain the average point embedding vector for each category; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector to obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding vector loss function is constructed based on the loss function for positive sample pairs and the loss function for negative sample pairs. The deep neural network model is trained by combining the average point embedding vector and the average point embedding vector loss function to obtain the image retrieval model; An image retrieval model is used to retrieve the image data to be retrieved, and the image retrieval results are output.

2. The image retrieval method based on average point embedding vector learning according to claim 1, characterized in that, Before calculating the average of the feature embedding vectors for all samples corresponding to each category, the method further includes: Construct a logarithmic summation exponential function and a baseline loss function; wherein, the logarithmic summation exponential function is used to smoothly weight all positive and negative sample pairs of the baseline loss function, and the logarithmic summation exponential function... The specific expression is as follows: In the formula, x represents a set of similarity scores; t represents the temperature coefficient, used to adjust the smoothness of the weighting; exp represents the natural exponential function; and i represents the index number. The baseline loss function It is constructed based on the sum of the weighted similarities of all negative sample pairs and the sum of the weighted similarities of all positive sample pairs, and is specifically expressed as follows: In the formula, and Let a and m represent the similarity sets between the anchor sample and all negative samples and all positive samples, respectively; a represents the number of samples; m represents the total number of samples. express The corresponding temperature coefficient, express The corresponding temperature coefficient; Indicates an indicator function.

3. The image retrieval method based on average point embedding vector learning according to claim 1, characterized in that, The calculation of the average value of the feature embedding vectors for all samples corresponding to each category includes: For each class C in the training dataset, calculate the average of the feature embedding vectors of all samples belonging to the current class to obtain the average point embedding vector of the corresponding class. The specific formula is as follows: In the formula, This represents the set of samples belonging to category C; i is the sample index. Let represent the feature embedding vector of the i-th sample.

4. The image retrieval method based on average point embedding vector learning according to claim 3, characterized in that, The process of reconstructing the similarity term in the predefined baseline loss function using the average point embedding vector includes: Obtain the calculated average point embedding vector; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector. The specific expression is as follows: ； In the formula, Let the transpose of the feature embedding vector of the i-th sample be denoted as . This represents the average point embedding vector of the j-th sample; The loss functions for positive sample pairs and negative sample pairs are constructed based on the similarity term. The loss function of the positive sample pairs The specific expression is as follows: In the formula, a represents the anchor point; m represents the number of anchor points; and p represents the positive sample. The set representing positive samples; This indicates the number of positive samples corresponding to anchor point a; This represents the average point embedding vector of anchor point a; This represents the average point embedding vector of a positive sample p; C represents the average embedding vector of positive samples corresponding to the category to which anchor point a belongs; C represents the total number of categories; The loss function of the negative sample pairs The specific expression is as follows: In the formula, n represents the negative sample; It is a set of negative samples; It is the number of negative samples corresponding to anchor point a; This represents the average point embedding vector of the positive sample n; Indicates a category index; Indicates the category to which anchor point 'a' belongs; Indicates the first The average embedding vector of each negative class; This represents the average embedding vector of all negative classes corresponding to anchor point a.

5. The image retrieval method based on average point embedding vector learning according to claim 4, characterized in that, The construction of the average point embedding vector loss function based on the loss function for positive sample pairs and the loss function for negative sample pairs includes: Obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding loss function is constructed based on the loss functions for positive and negative sample pairs. The specific expression is as follows: 。 6. The image retrieval method based on average point embedding vector learning according to claim 1, characterized in that, Before training the pre-built deep neural network model using the original image data as the training dataset, the process also includes: A deep neural network model is constructed, which serves as a feature extractor to extract features from each input sample to obtain a multidimensional feature embedding vector.

7. The image retrieval method based on average point embedding vector learning according to claim 1, characterized in that, After extracting the feature embedding vectors from the training dataset, the process also includes: The extracted feature embedding vectors are normalized so that they are distributed on the unit hypersphere.

8. An image retrieval system based on average point embedding vector learning, characterized in that, include: The data acquisition module is used to acquire raw image data; The model training module is used to train a pre-built deep neural network model using the original image data as the training dataset, and outputs an image retrieval model. The training process of the deep neural network model is as follows: Extract feature embedding vectors from the training dataset; Calculate the average of the feature embedding vectors for all samples in each category to obtain the average point embedding vector for each category; The similarity term in the predefined baseline loss function is reconstructed using the average point embedding vector to obtain the loss function for positive sample pairs and the loss function for negative sample pairs; The average point embedding vector loss function is constructed based on the loss function for positive sample pairs and the loss function for negative sample pairs. The deep neural network model is trained by combining the average point embedding vector and the average point embedding vector loss function to obtain the image retrieval model; The retrieval module is used to retrieve the image data to be retrieved using an image retrieval model and output the image retrieval results.

9. An image retrieval device based on average point embedding vector learning, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the image retrieval method based on average point embedding vector learning as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it is used to implement the steps of the image retrieval method based on average point embedding vector learning as described in any one of claims 1-7.

Images