Data enhancement method, framework and application in image retrieval
By introducing data augmentation methods and a semantic retention estimation module into deep hash image retrieval, the problems of semantic misalignment and spurious feature interference are solved, the hash space structure is optimized, and the retrieval accuracy and cross-modal feature alignment accuracy are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Huairou Laboratory Xinjiang Research Institute
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-12
AI Technical Summary
Existing deep hashing image retrieval methods suffer from semantic misalignment, metric target mismatch, interference from augmentation pseudo-features, and difficulty in quantifying semantic changes during the data augmentation process, leading to a decrease in retrieval accuracy.
By employing a data augmentation method, a semantic retention estimation module is introduced by discarding some information in the image. The metric loss is then corrected using a semantically weighted softened Bernoulli likelihood loss function, thereby achieving dynamic adjustment of the semantic retention of augmented samples.
It improves hash retrieval accuracy, reduces false feature interference, optimizes hash space structure, and enhances the model's resistance to occlusion and distortion, making it suitable for multi-task scenarios and cross-modal retrieval.
Smart Images
Figure CN122023213A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and information retrieval, specifically relating to a data augmentation method, framework, and its application in image retrieval, and in particular to a data augmentation method, framework, and its application in image retrieval based on semantic preservation estimation. Background Technology
[0002] In the field of deep learning image retrieval, deep hashing has become the mainstream technique. This method maps high-dimensional image features to compact binary hash codes using an end-to-end neural network model, thereby achieving efficient similar image retrieval. To improve the robustness and generalization ability of the model, researchers commonly employ data augmentation techniques, such as rotation, scaling, and color perturbation. These augmentation methods expand the distribution of training data and mitigate overfitting by generating variant samples that are semantically consistent with the original samples.
[0003] Currently, mainstream deep hashing image retrieval methods both domestically and internationally primarily rely on three types of data augmentation strategies: Single-sample augmentation, including random cropping, horizontal flipping, brightness adjustment, rotation, and scaling, which enhances data diversity by performing geometric or color transformations on individual samples; Multi-sample augmentation, typically represented by methods such as Mixup and Manifold Mixup, which fuses two images and their labels through linear interpolation, thereby introducing continuous intermediate domains in the feature space; and Deep model-based augmentation (Adversarial / Deep Augmentation), such as adversarial training and generative adversarial augmentation, which uses neural networks to generate perturbed image samples to improve model robustness.
[0004] Although the aforementioned enhancement methods have achieved good results in tasks such as classification, they still have significant shortcomings in image retrieval, especially in deep hashing retrieval tasks, specifically: 1. Semantic misalignment issue Existing augmentation methods generally assume that the semantics of the augmented image are completely consistent with the original image (i.e., the labels remain unchanged). However, in image retrieval scenarios, this assumption often does not hold true. For example, when an image is rotated or partially cropped, its visual features and semantic representation have changed, but the model is still forced to learn "semantically identical," leading to incorrect constraints on the distance distribution between samples in the hash space.
[0005] 2. Measuring target mismatch The loss metrics of deep hashing (such as Pairwise Loss and Triplet Loss) rely on the label similarity matrix to guide feature clustering. When data augmentation introduces semantic drift, the label matrix remains unchanged, causing the model to incorrectly cluster semantically inconsistent samples during optimization, thus compromising the distinguishability of the hash space and reducing retrieval accuracy.
[0006] 3. Enhanced spurious feature interference During large-scale training, models are prone to learning pseudo-features (such as rotated edges, scaling noise, etc.) introduced during the enhancement process as retrieval criteria. This leads to the model tending to return images with similar pseudo-features rather than semantically similar images during actual retrieval, significantly reducing the reliability of retrieval results.
[0007] 4. Difficulty in quantifying semantic changes Traditional augmentation methods cannot assess the impact of each augmentation operation on the semantic retention of samples, and lack a mechanism to determine "how much original semantic information the augmented sample still retains", thus making it impossible to perform dynamic weight adjustment in loss calculation.
[0008] The data augmentation assumption commonly used in existing image retrieval (especially deep hashing) training is "augmentation does not change the label". However, in retrieval tasks, augmentation can cause semantic drift, mismatch, or introduce false features, resulting in metric loss that incorrectly pulls semantically inconsistent samples closer together, thereby reducing retrieval accuracy. Summary of the Invention
[0009] To address the technical problems existing in the prior art, this invention provides a data augmentation method, framework, and its application in image retrieval, aiming to improve the semantic mismatch and pseudo-feature bias problems caused by traditional augmentation in deep hash retrieval. By intentionally and randomly discarding parts of the image content during the augmentation process, and introducing a semantic change estimation module to calculate the semantic preservation degree of each augmented sample, a semantically weighted softened Bernoulli likelihood is used to correct the traditional metric loss, enabling dynamic adjustment of the gradient of augmented samples during the optimization process. Compared with existing technologies, this invention can automatically evaluate the semantic shift caused by the augmentation operation; correct metric mismatch caused by augmentation; reduce pseudo-feature interference and improve hash retrieval accuracy; maintain a simple model structure, be plug-and-play, and have wide applicability.
[0010] This invention provides a data augmentation method, comprising the following steps: Step S01: In the data augmentation stage, augmented samples with quantifiable semantic loss are obtained by discarding some information of the image. Step S02: In the data augmentation stage, semantic retention is estimated to obtain the retention estimation factor e; Step S03: The obtained retention estimation factor e is incorporated into the loss function so that the constraint of the measurement loss on the augmented samples is corrected according to the semantic retention.
[0011] Preferably, in step S01, a model-independent translation discarding method is used to perform random translation on the input image i, translating the pixels as a whole, and discarding the parts that exceed the image boundary.
[0012] Preferably, the sampling point pixel position coordinates (δx, δy) and scaling factor σ are used. t Control the width displacement distance Δx, Δx = σ t (2δx 1) w , where σ t Within the range (0, 1), w This refers to the image width; the height is similar.
[0013] Preferably, in step S01, when using the model-independent translational discarding method, the geometric area ratio is used as the retention estimation factor e, and the retention estimation factor e is calculated in the following way:
[0014] Where Δy is the height displacement distance.
[0015] Preferably, in step S01, a model-aware cutting and discarding method is adopted. First, the saliency distribution Att of the image is calculated according to the existing model. The image is divided into an n×m patch matrix. Based on the saliency distribution Att and the set saliency threshold, a number of image patches are randomly discarded in the high saliency region, where n and m correspond to the width and height dimensions of the feature map of a specific layer of the model, respectively.
[0016] Preferably, in the model-aware cut-and-drop method, the saliency distribution Att is generated using the following method: If the backbone of the model is a convolutional neural network (CNN), the output feature maps of the specified network layers are directly summed to construct an approximate class activation map; then the Softmax function is used to normalize the approximate class activation map to obtain the saliency distribution Att of the model's perception of different image regions. If the model is a ViT network, the patch-level attention is obtained directly by using the dot product similarity between the token and the class token (CLS token). The Softmax function is also used for normalization to obtain the saliency distribution Att.
[0017] Preferably, in step S01, when using the model-aware cutting and discarding method, the attention-weighted ratio is used as the retention estimation factor e, and the retention estimation factor e is calculated as follows: Let the saliency distribution Att be the patch-level attention matrix, and let the set of tokens to be removed be... Att cut The set that is retained is Att remain The retention estimation factor e is calculated using the following formula: .
[0018] Preferably, step S03 specifically includes: make S O The similarity matrix calculated for the original labels. B Given the hash code matrix, we obtain the likelihood term of the retention estimation factor e. :
[0019] in This is the current calculation example. k No. a After data augmentation, the sample was compared with the subsequent execution. m Similarity, This is the current calculation example. k No. a Retention estimation factor after data augmentation in the second execution For example k With sample m The similarity, σ(·) is the Sigmoid activation function; Take the likelihood term of the retention estimation factor e The negative log-likelihood form yields the semantically weighted softened Bernoulli likelihood loss function. , this loss function As the revised metric target:
[0020] During the training of the deep hashing model, a modified metric objective is used to guide the deep learning process.
[0021] The present invention also provides a data augmentation framework for implementing the above-described data augmentation method, including a data augmentation module, a semantic estimation module, and a loss correction module; The data augmentation module obtains augmented samples with quantifiable semantic loss by discarding some information in the image during the data augmentation stage; The semantic estimation module performs semantic retention estimation during the data augmentation stage, obtaining the retention estimation factor e. The loss correction module incorporates the obtained retention estimate factor e into the loss function, so that the constraint of the loss measurement on the augmented samples is corrected according to the semantic retention.
[0022] The present invention also provides an application of the above-described data augmentation method in image retrieval.
[0023] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention proposes a highly efficient data augmentation strategy that employs a discard-style data augmentation transformation. By randomly translating and segmenting the input image, it intentionally "discards" some visual content to generate augmented samples with quantifiable semantic changes. This breaks the traditional assumption that "augmentation does not change semantics," enabling the model to explicitly learn the relationship of "semantic preservation under missing information." This data augmentation strategy effectively reduces semantic noise, improves the model's resistance to occlusion and distortion, and possesses advantages such as semantic controllability, high generation efficiency, and compatibility with any visual model.
[0024] 2. This invention introduces the concept of semantic retention estimation for the first time, constructing a lightweight semantic estimation module to calculate the semantic retention estimate e∈[0,1] of the enhanced sample relative to the original image. This module can be implemented in the following ways: geometric ratio estimation (for translation or cropping enhancement); attention weight ratio (for saliency-based cropping enhancement). It plays a role in quantifying the semantic shift of the enhancement and introducing dynamic confidence adjustment into the training process, possessing the advantages of simple calculation, strong versatility, and easy adaptation to multiple models.
[0025] 3. This invention proposes a softened Bernoulli likelihood loss function based on semantic retention. It integrates the semantic retention estimate *e* into the traditional pairwise or triplet metric loss. By dynamically adjusting the supervision strength, the model applies strong constraints only to samples with high semantic consistency, while weakening constraints on augmented samples with large semantic shifts. This effectively solves the problems of label mismatch and metric misleading caused by data augmentation, optimizing the hash space structure, improving retrieval accuracy and semantic relevance, and possessing advantages such as end-to-end differentiability, stable training, and strong mathematical interpretability.
[0026] 4. In this invention, the proposed data augmentation framework is a fully modular design, which can be embedded into any image retrieval model (such as ResNet, GoogLeNet, Vision Transformer, etc.) without modifying the backbone network structure. This design enables independent and combined training of the data augmentation module, semantic estimation module, and loss correction module. It achieves rapid migration and loosely coupled deployment, and possesses advantages such as high compatibility, low computational burden, and adaptability to multi-task scenarios.
[0027] 5. The framework of this invention is not only applicable to single-modal image retrieval, but can also be extended to cross-modal retrieval (image-text alignment) or multi-source oil and gas data retrieval. By calculating semantic preservation scores and weighted losses for samples from different modalities, enhancement strategies across different modalities can be unified. This achieves the effect of unifying enhancement and measurement mechanisms, improving the accuracy of cross-modal feature alignment, and possessing the advantages of cross-domain scalability and task universality. Attached Figure Description
[0028] Figure 1 This is a flowchart of a data augmentation method according to an embodiment of the present invention.
[0029] Figure 2 This is a comparison chart of precision-recall (PR) curves and P@N metrics for different numbers of returned images according to an embodiment of the present invention. Detailed Implementation
[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0031] like Figure 1 As shown, the present invention provides a data augmentation method, comprising the following steps: Step S01: In the data augmentation stage, augmented samples with quantifiable semantic loss are obtained by discarding some information of the image. Step S02: In the data augmentation stage, semantic retention is estimated to obtain the retention estimation factor e; Step S03: The obtained retention estimation factor e is incorporated into the loss function so that the constraint of the measurement loss on the augmented samples is corrected according to the semantic retention.
[0032] In one embodiment of the present invention, in step S01, a model-independent translation discarding method is used to perform random translation on the input image i, translating the pixels as a whole, and discarding the parts that exceed the image boundary.
[0033] In one embodiment of the present invention, the sampling point pixel position coordinates (δx, δy) and scaling factor σ are used. t Control the width displacement distance Δx, Δx = σ t (2δx 1) w , where σ t Within the range (0, 1), wThis refers to the image width; the height is similar.
[0034] In one embodiment of the present invention, in step S01, when using the model-independent translational discarding method, the geometric area ratio is used as the retention estimation factor e, and the retention estimation factor e is calculated in the following manner:
[0035] Where Δy is the height displacement distance.
[0036] In one embodiment of the present invention, in step S01, a model-aware cutting and discarding method is adopted. First, the saliency distribution Att of the image is calculated according to the existing model, and the image is divided into an n×m patch matrix. Based on the saliency distribution Att and the set saliency threshold, a number of image patches are randomly discarded in the high saliency region, where n and m correspond to the width and height dimensions of the feature map of a specific layer of the model, respectively.
[0037] In one embodiment of the present invention, the saliency distribution Att is generated in model-aware cut-off and discarding using the following method: If the backbone of the model is a convolutional neural network (CNN), the output feature maps of the specified network layers are summed directly to construct an approximate class activation map (approximate-CAM); then the approximate-CAM is normalized using the Softmax function to obtain the saliency distribution Att of the model's perception of different image regions. If the model is a ViT network, the patch-level attention is obtained directly by using the dot product similarity between the token and the class token (CLS token). The Softmax function is also used for normalization to obtain the saliency distribution Att.
[0038] In one embodiment of the present invention, in step S01, when using the model-aware cutting and discarding method, the attention-weighted ratio is used as the retention estimation factor e, and the retention estimation factor e is calculated as follows: Let the saliency distribution Att be the patch-level attention matrix, and let the set of tokens to be cut be... Att cut The set that is retained is Att remain The retention estimation factor e is calculated using the following formula: .
[0039] In one embodiment of the present invention, step S03 specifically includes: make S O The similarity matrix calculated for the original labels. B Given the hash code matrix, we obtain the likelihood term of the retention estimation factor e. :
[0040] in This is the current calculation example. k No. a After data augmentation, the sample was compared with the subsequent execution. m Similarity, This is the current calculation example. k No. a Retention estimation factor after data augmentation in the second execution For example k With sample m The similarity, σ(·) is the Sigmoid activation function; Take the likelihood term of the retention estimation factor e The negative log-likelihood form yields the semantically weighted softened Bernoulli likelihood loss function. , this loss function As the revised metric target:
[0041] During the training of the deep hashing model, a modified metric objective is used to guide the deep learning process.
[0042] The present invention also provides a data augmentation framework for implementing the above-described data augmentation method, including a data augmentation module, a semantic estimation module, and a loss correction module; The data augmentation module obtains augmented samples with quantifiable semantic loss by discarding some information in the image during the data augmentation stage; The semantic estimation module performs semantic retention estimation during the data augmentation stage, obtaining the retention estimation factor e. The loss correction module incorporates the obtained retention estimate factor e into the loss function, so that the constraint of the loss measurement on the augmented samples is corrected according to the semantic retention.
[0043] The present invention also provides an application of the above-described data augmentation method in image retrieval.
[0044] Example 1 This invention provides a data augmentation method, comprising the following steps: Step S01: In the data augmentation stage, augmented samples with quantifiable semantic loss are obtained by discarding some information of the image. Step S02: In the data augmentation stage, semantic retention is estimated to obtain the retention estimation factor e; Step S03: The obtained retention estimation factor e is incorporated into the loss function so that the constraint of the measurement loss on the augmented samples is corrected according to the semantic retention.
[0045] Example 2 This invention provides a data augmentation method, which differs from Embodiment 1 in that: In this embodiment, in step S01, a model-independent translation discarding method is used to perform random translation on the input image i, translating the entire pixel and discarding the part that exceeds the image boundary.
[0046] In this embodiment, the sampling point pixel position coordinates (δx, δy) and scaling factor σ are used. t Control the width displacement distance Δx, Δx = σ t (2δx 1) w , where σ t Within the range (0, 1), w This refers to the image width; the height is similar.
[0047] In this embodiment, in step S01, when using the model-independent translational discarding method, the geometric area ratio is used as the retention estimation factor e, and the retention estimation factor e is calculated in the following way:
[0048] Where Δy is the height displacement distance.
[0049] In this embodiment, in step S01, a model-aware cutting and discarding method is adopted. First, the saliency distribution Att of the image is calculated according to the existing model. The image is divided into an n×m patch matrix. Based on the saliency distribution Att and the set saliency threshold, a number of image patches are randomly discarded in the high saliency region, where n and m correspond to the width and height dimensions of the feature map of a specific layer of the model, respectively.
[0050] In this embodiment, the saliency distribution Att is generated in model-aware cut-and-drop using the following method: If the backbone of the model is a convolutional neural network (CNN), the output feature maps of the specified network layers are summed directly to construct an approximate class activation map (approximate-CAM); then the approximate-CAM is normalized using the Softmax function to obtain the saliency distribution Att of the model's perception of different image regions. If the model is a ViT network, the patch-level attention is obtained directly by using the dot product similarity between the token and the class token (CLS token). Similarly, the Softmax function is used for normalization to obtain the saliency distribution Att.
[0051] In this embodiment, in step S01, when using the model-aware cutting and discarding method, the attention-weighted ratio is used as the retention estimation factor e. The retention estimation factor e is calculated as follows: Let the saliency distribution Att be the patch-level attention matrix, and let the set of tokens to be removed be... Att cut The set that is retained is Att remain The retention estimation factor e is calculated using the following formula: .
[0052] In this embodiment, step S03 specifically includes: make S O The similarity matrix calculated for the original labels. B Given the hash code matrix, we obtain the likelihood term of the retention estimation factor e. :
[0053] in This is the current calculation example. k No. a After data augmentation, the sample was compared with the subsequent execution. m Similarity, This is the current calculation example. k No. a Retention estimation factor after data augmentation in the second execution For example k With sample m The similarity, σ(·) is the Sigmoid activation function; Take the likelihood term of the retention estimation factor e The negative log-likelihood form yields the semantically weighted softened Bernoulli likelihood loss function. , this loss function As the revised metric target:
[0054] During the training of the deep hashing model, a modified metric objective is used to guide the deep learning process.
[0055] Example 3 To verify the performance of the framework in other tasks, this embodiment applies the data augmentation framework (GG framework) of the present invention to real-valued retrieval and classification tasks. The data augmentation method is implemented in the specific manner of Embodiment 2, as shown in the table below. In the real-valued retrieval scenario, the GG framework not only outperforms the baseline methods and other augmentation techniques, but also achieves best or near-best performance in mean recall (mR). This fully demonstrates the high compatibility between hashing and real-valued retrieval tasks—both rely on the nearest neighbor search mechanism within the embedding space. It also performs excellently on the domain-specific datasets RSITM and DeepFashion, a result entirely consistent with the aforementioned conclusions.
[0056] Table 1. Experimental Results
[0057] Where "*" indicates no data augmentation, "†mixup" indicates additional mixed insertion, "†m-mixup" indicates additional popular mixed insertion, "†manifold" indicates additional popular mixed insertion, "†G" indicates additional mixed insertion, and "†G" indicates additional mixed insertion. t "This indicates that G was used in addition to..." t "†cut" indicates that G was used additionally. cut .
[0058] The following are the experimental details: 1. Deep Hash Workflow The effectiveness of the GG framework is evaluated among four visual encoders. Four state-of-the-art (SOTA) models are selected as the visual encoders: 1) CNN-based, including two: Residual Network (ResNet) and Google Inception Network (GoogLeNet); 2) Visual Transformer (ViT) based methods, including two: VIT-based hashing (VTS) and multi-scale visual Transformer (MSViT).
[0059] To improve experimental efficiency, VTS was implemented using the basic Visual Transformer (ViT-B) architecture, and MSViT was implemented using the lightweight Visual Transformer (ViT-S) architecture. Furthermore, CLIP and the ViT-B architecture were used for cross-modal real-valued retrieval.
[0060] 2. Sampling Strategy When performing each intuitive transformation operation: 1) For model-independent moves, abandon G. t From random values that follow a uniform distribution (0,1) Sampling, and through Scale the image to obtain the displacement offset, Δx = σ t (2δx 1) w , where σ t Within the range (0, 1), w σ is the image width, and the height is similar, where σ is the image width. t Control the average degree of movement.
[0061] 2) For model-aware segmentation, abandon G. cut Similarly, a random value The number of tokens discarded has been determined. in Control the average proportion of discards.
[0062] 3. Other experimental setups During training, the batch size was set to 128, the RMSprop optimizer was used, the weight decay was 1e-5, and the learning rate was set to 3e-5 (3e-6 for VTS and 1e-2 for CLIP). All experiments were performed on a single GPU, with the following configuration: Intel Core i7-12700KF CPU, NVIDIA GeForce RTX 3090 graphics card with 24GB GDDR6X VRAM, and 32GB DDR4 RAM (2 x 2).
[0063] Example 4 Unlike Example 3, this example selects six datasets (including four commonly used benchmark datasets, two single-label datasets CIFAR-10 and mageNet-100, two multi-label datasets NUS-WIDE and COCO, and two domain datasets RSITMD and DeepFashion) to verify the effectiveness of the data augmentation framework proposed in this invention. The data augmentation method used is the method described in Example 2.
[0064] To ensure the reliability of the experimental results, six mainstream evaluation metrics (mean Average Precision (mAP), top N precision (P@N), Fowlkes-Mallows Index (FMI), Local Intrinsic Dimensionality (LIID), top K recall (R@K), mR, and accuracy (ACC)) were selected to comprehensively evaluate the framework performance. The experimental results are as follows: The effect of data augmentation frameworks on single-label datasets: The data augmentation method of this invention exhibits superior performance compared to traditional data augmentation methods, employing G... t and G cut The hashing workflow achieved best or second-best results on single-label datasets, as shown in the table below. "×" represents no data augmentation, "†" represents basic data augmentation, "†*" represents additional random rotation, "†mixup" represents additional mixed insertion, "†m-mixup" represents additional popular mixed insertion, and "†G" represents additional random mixed insertion. t "This indicates that G was used in addition to..." t "†cut" indicates that G was used additionally. cut .
[0065] Table 2. Comparison results with traditional data augmentation methods
[0066] The effect of data augmentation frameworks on multi-label datasets: Compared to intuitive conversion G t Method, Perception Transformation G cut Significant performance improvements were achieved on multi-label datasets, as shown in the table below.
[0067] Table 3.G t With G cut Comparison of experimental data
[0068] Ablation experiment: like Figure 2 As shown, this demonstrates the use of (labeled ") from the loss function ") and not using (marked as "N") After estimating the retention factor e, the precision-recall (PR) curves and the top N precision (P@N) metrics are presented on six datasets with different numbers of returned images. For example, "ResNet-50_N" "This indicates that the network backbone is a residual network (ResNet-50) and no retention estimation factor is used; "GoogLeNet-V3_ "The backbone of the network is represented by the Google Inception network (GoogLeNet-V3), using a retention estimation factor." Figure 2 (a) is the application of basic augmentation and G on the Ciafr-10 dataset. t The effect; Figure 2 (b) is the application of basic augmentation and G on the ImageNet-100 dataset. t The effect; Figure 2 (c) is the application of basic augmentation and G on the Ciafr-10 dataset. cut The effect; Figure 2 (d) is to use basic augmentation and G on the ImageNet-100 dataset. cut The effect. After removing the retention estimate factor from the metric loss, G t and G cut Performance on single datasets has declined across the board, and G cut Its performance on multiple datasets is also inferior to the full GG framework. (Dashed lines are below the solid lines) The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A data augmentation method, characterized in that, Includes the following steps: Step S01: In the data augmentation stage, augmented samples with quantifiable semantic loss are obtained by discarding some information of the image. Step S02: In the data augmentation stage, semantic retention is estimated to obtain the retention estimation factor e; Step S03: The obtained retention estimation factor e is incorporated into the loss function so that the constraint of the measurement loss on the augmented samples is corrected according to the semantic retention.
2. The data augmentation method according to claim 1, characterized in that, In step S01, a model-independent translation discarding method is used to perform random translation on the input image i, shifting the pixels as a whole, and discarding the parts that exceed the image boundary.
3. The data augmentation method according to claim 2, characterized in that, Using the pixel position coordinates (δx, δy) of the sampling point and the scaling factor σ t Control the width displacement distance Δx, Δx = σ t (2δx 1) w , where σ t Within the range (0, 1), w This refers to the image width; the height is similar.
4. The data augmentation method according to claim 3, characterized in that, In step S01, when using the model-independent translational discarding method, the geometric area ratio is used as the retention estimation factor e, which is calculated as follows: Where Δy is the height displacement distance.
5. The data augmentation method according to claim 1, characterized in that, In step S01, a model-aware cutting and discarding method is adopted. First, the saliency distribution Att of the image is calculated according to the existing model. The image is divided into an n×m patch matrix. Based on the saliency distribution Att and the set saliency threshold, a number of image patches are randomly discarded in the high saliency region, where n and m correspond to the width and height dimensions of the feature map of a specific layer of the model, respectively.
6. The data augmentation method according to claim 5, characterized in that, In the model-aware cut-and-drop method, the saliency distribution Att is generated using the following method: If the backbone of the model is a convolutional neural network (CNN), the output feature maps of the specified network layers are directly summed to construct an approximate class activation map; then the Softmax function is used to normalize the approximate class activation map to obtain the saliency distribution Att of the model's perception of different image regions. If the model is a ViT network, the patch-level attention is obtained directly by using the dot product similarity between the token and the class token (CLS token). The Softmax function is also used for normalization to obtain the saliency distribution Att.
7. The data augmentation method according to claim 5, characterized in that, In step S01, when using the model-aware cutting and discarding method, the attention-weighted ratio is used as the retention estimation factor e. The retention estimation factor e is calculated as follows: Let the saliency distribution Att be the patch-level attention matrix, and let the set of tokens to be removed be... Att cut The set that is retained is Att remain The retention estimation factor e is calculated using the following formula: 。 8. The data augmentation method according to claim 1, characterized in that, Step S03 specifically includes: make S O The similarity matrix calculated for the original labels. B Given the hash code matrix, we obtain the likelihood term of the retention estimation factor e. : in This is the current calculation example. k No. a After data augmentation, the sample was compared with the subsequent execution. m Similarity, This is the current calculation example. k No. a Retention estimation factor after data augmentation in the second execution For example k With sample m The similarity, σ(·) is the Sigmoid activation function; Take the likelihood term of the retention estimation factor e The negative log-likelihood form yields the semantically weighted softened Bernoulli likelihood loss function. , this loss function As the revised metric target: During the training of the deep hashing model, a modified metric objective is used to guide the deep learning process.
9. A data augmentation framework, characterized in that, The data augmentation method for implementing any one of claims 1-8 includes a data augmentation module, a semantic estimation module, and a loss correction module; The data augmentation module obtains augmented samples with quantifiable semantic loss by discarding some information in the image during the data augmentation stage; The semantic estimation module performs semantic retention estimation during the data augmentation stage, obtaining the retention estimation factor e. The loss correction module incorporates the obtained retention estimate factor e into the loss function, so that the constraint of the loss measurement on the augmented samples is corrected according to the semantic retention.
10. The application of the data augmentation method according to any one of claims 1-8 in image retrieval.