An image retrieval adversarial training method based on target enhancement

Abstract

The application is suitable for the field of adversarial sample technology of deep learning, and provides an image retrieval adversarial training method based on target enhancement, which comprises the following steps: obtaining an initial image data set, wherein the initial image data set comprises a plurality of initial target sample images; constructing a substitute image data set based on the initial image data set, training a substitute model of a victim model by using the substitute image data set, and obtaining a plurality of local substitute models; obtaining an enhanced image data set based on the substitute image data set and the plurality of local substitute models, and generating an adversarial sample based on the enhanced image data set; and performing adversarial training based on the adversarial sample and a training data set of the victim model to improve the robustness of the victim model. The application can solve the problem that the attack effect on image retrieval is not ideal in the prior art, so that the effectiveness of the generated adversarial sample is insufficient in adversarial training, and the robustness of the victim model is still not high.

Description

Technical Field

[0001] This invention relates to the field of adversarial example technology in deep learning, and in particular to an adversarial training method for image retrieval based on target enhancement. Background Technology

[0002] With the rapid development of image acquisition technology and the widespread application of social media platforms in daily life, the number of images has exploded exponentially. Statistics show that over 1.4 trillion images are uploaded annually, Instagram has over 50 billion images, over 300 million images are uploaded to Facebook daily, and over one billion images are uploaded to WeChat Moments daily. With the rapid development of image retrieval technology, image retrieval systems are playing an increasingly important role in people's daily lives. To address the significant challenges in image retrieval, researchers have developed and deployed deep learning retrieval models based on deep neural networks (DNNs). However, the development of deep learning image retrieval technology is a double-edged sword. Image retrieval results may contain sensitive information such as facial features and biometric identifiers. If serious vulnerabilities exist, deploying the model could pose a security risk. Examples include Google's search engine storing users' image data for seven days without authorization to improve Google's products and services, and Clearview AI being able to retrieve all images of a person online using only a single facial image. Attackers can use adversarial examples to upload an unrelated image and retrieve private images containing information about personal interests and work. Attackers can choose to upload copyrighted adversarial examples instead of the original image to avoid detection of copyright infringement. This is because copyright infringement detection often uses image retrieval models to confirm whether a newly uploaded image already exists in the database. In this scenario, adversarial examples are likely to avoid retrieving copyrighted images. In reality, adversarial attacks on image retrieval can have serious consequences, thus necessitating in-depth research into the robustness of retrieval models.

[0003] Current research primarily focuses on attacks from the attacker's perspective, categorizing them into white-box and black-box attacks. The basic idea of ​​a white-box attack is to first extract image features, then minimize the feature distance between the query image and the target image by adding a small perturbation to achieve a targeted attack. However, white-box attacks have significant limitations; most practical image retrieval models are black-box because service providers like Google and Baidu do not share model information (architecture and parameters) with the public. The basic idea of ​​a black-box attack is to continuously query the target model to build a substitute dataset. The attacker trains the substitute model locally and leverages the transferability of adversarial examples to attack the target model. Regardless of whether it's a white-box or black-box attack, the attack performance is highly dependent on the returned target images. That is, the attack can only be used on selected target images, not the majority of images, especially when retrieving target images by querying other non-target categories with a high probability. If the target sample is closer to other non-target category samples, the generated adversarial examples are still ineffective. For attackers, forcing the retrieval model to return as many target images as possible is advantageous, allowing them to steal more private data and obtain higher value. Compared to non-target attacks, targeted attacks are more challenging to design and process. Current attacks on image retrieval are not very effective, limiting their applicability to other victim models. Secondly, prior attacks based on directly perturbing the original query image show limitations such as imperceptible perturbation or insufficient effectiveness; the imperceptibility of adversarial examples is also a key area requiring further research.

[0004] Therefore, there is an urgent need for an image retrieval adversarial training method based on target enhancement to solve the problems in existing methods where the attack effect on image retrieval is not ideal, resulting in insufficient effectiveness of the generated adversarial examples in adversarial training and low robustness of the victim model. Summary of the Invention

[0005] This invention provides an image retrieval adversarial training method based on target enhancement, aiming to address the problems in existing methods where the attack effect on image retrieval is unsatisfactory, resulting in insufficient effectiveness of generated adversarial examples in adversarial training and low robustness of the victim model. By acquiring an initial image dataset, constructing a substitute image dataset from it, and training a substitute model of the victim model using the substitute image dataset to obtain multiple local substitute models, an enhanced image dataset is obtained based on the substitute image dataset and the multiple local substitute models. Adversarial examples are then generated using the enhanced image dataset. This method effectively solves the problems in existing methods where the attack effect on image retrieval is unsatisfactory, resulting in insufficient effectiveness of generated adversarial examples in adversarial training and low robustness of the victim model.

[0006] Initially, embodiments of the present invention provide an image retrieval adversarial training method based on target enhancement, the image retrieval adversarial training method based on target enhancement includes the following steps:

[0007] Obtain an initial image dataset, which includes multiple initial target sample images;

[0008] An alternative image dataset is constructed based on the initial image dataset. The alternative image dataset is then used to train an alternative model of the victim model, resulting in multiple local alternative models.

[0009] Based on the alternative image dataset and the multiple local alternative models, an enhanced image dataset is obtained, and adversarial examples are generated based on the enhanced image dataset;

[0010] The victim model is trained adversarially using the training data of the victim model as positive samples and the adversarial samples as negative samples.

[0011] Optionally, the step of constructing an alternative image dataset based on the initial image dataset, and training an alternative model of the victim model using the alternative image dataset to obtain multiple local alternative models includes:

[0012] The initial target sample image is randomly selected from the initial image dataset and input into the victim model for querying. Based on the query results, an alternative image dataset is obtained. The alternative image dataset includes multiple alternative target sample images, and each initial target sample image corresponds to at least one alternative target sample image.

[0013] The alternative models of the victim model are trained using the alternative image dataset to obtain multiple local alternative models.

[0014] Optionally, the step of training an alternative model of the victim model using the alternative image dataset to obtain multiple local alternative models includes:

[0015] Initial features of the initial target sample image are extracted using the alternative model of the victim model, and alternative features of the alternative target sample image are extracted using the alternative model of the victim model.

[0016] The loss values ​​of the initial feature and the alternative feature are calculated using a preset loss function;

[0017] Based on the loss value, the model parameters of the alternative model are adjusted, and after training, multiple local alternative models are obtained.

[0018] Optionally, the loss function is as follows:

[0019]

[0020] in, It is an alternative model The parameters, It is the initial feature. It is a substitute feature. express and Distance between features yes The projection features, It is a marginal constant to ensure that the loss value remains positive.

[0021] Optionally, CSQ can be used to generate the hash code vector for the alternative model.

[0022] Optionally, the step of obtaining an enhanced image dataset based on the alternative image dataset and multiple local alternative models, and generating adversarial examples based on the enhanced image dataset, includes:

[0023] The average feature center of the alternative image dataset is calculated using the multiple local substitution models.

[0024] For the alternative image dataset, an enhanced image dataset is generated by combining the alternative target sample image with the average feature center and a preset perturbation amount;

[0025] Adversarial examples are generated based on the enhanced image dataset.

[0026] Optionally, the step of calculating the average feature center of the alternative image dataset using the plurality of local substitution models includes:

[0027] Calculate the features of each of the alternative target sample images in the alternative image dataset;

[0028] Based on the features of each of the alternative target sample images in the alternative image dataset, the average feature center of the alternative image dataset is calculated.

[0029] Optionally, the step of calculating the average feature center of the alternative image dataset based on the features of each of the alternative target sample images in the alternative image dataset includes:

[0030] The average feature center of the alternative image dataset is calculated using the following formula:

[0031] in, This represents an alternative image dataset. Indicates an alternative model, Indicates input Alternative models The projected eigenvectors.

[0032] Optionally, the step of generating an enhanced image dataset by combining the alternative target sample image with the average feature center and a preset perturbation amount for the alternative image dataset includes:

[0033] For the alternative image dataset, the enhanced image dataset is generated using the following formula:

[0034]

[0035] in, As a weighting factor, ; This represents the perturbation amount, which, through iterations of the training process, ultimately yields alternative target samples. ; By calculating Exported and The mean square error of the eigenvectors is used to represent and The feature similarity is calculated using the following formula:

[0036]

[0037] Used to evaluate the two search results and The similarity is calculated using the following formula:

[0038]

[0039] in, for The i-th return sample's score. The calculation is as follows:

[0040]

[0041] according to ∈ exist The position j is used to measure The score; if If it appears, then the fraction is Otherwise, the score is 0.

[0042]

[0043] The augmented image dataset is obtained, represented as .

[0044] Optionally, generating adversarial examples based on the enhanced image dataset includes:

[0045] RIE-based adversarial example generation performs adversarial example generation processing on the enhanced image dataset to generate adversarial examples. The RIE-based adversarial example generation satisfies the following:

[0046]

[0047] in and These represent DWT and IDWT, respectively. This represents the four networks that make up INN. For any unrelated image, To perturb the target image in the frequency domain;

[0048] Based on enhanced image dataset Generate adversarial examples using RIE The total loss function can be expressed as follows:

[0049] in, and These represent the feature exchange loss and the reconstruction loss, respectively. For regularization parameters; It is expressed as follows:

[0050]

[0051] This is for the purpose of constraint and The distance between them is used to limit the range of the disturbance and ensure its imperceptibility, as shown below:

[0052]

[0053] in, These represent the low-frequency and high-frequency components of the DWT, respectively. The weighting factor is the corresponding component; adversarial examples are obtained through iterative updates.

[0054] In this embodiment of the invention, an initial image dataset is obtained, comprising multiple initial target sample images; an alternative image dataset is constructed based on the initial image dataset, and the alternative image dataset is used to train an alternative model of the victim model to obtain multiple local alternative models; an enhanced image dataset is obtained based on the alternative image dataset and the multiple local alternative models, and adversarial examples are generated based on the enhanced image dataset; adversarial training is performed on the victim model using the training data of the victim model as positive samples and the adversarial examples as negative samples. This invention can solve the problems in existing methods where the attack effect on image retrieval is not ideal, resulting in insufficient effectiveness of the generated adversarial examples in adversarial training, and the victim model still lacks robustness. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of an image retrieval adversarial training method based on target enhancement provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, Figure 1 This is a flowchart of an image retrieval adversarial training method based on target enhancement provided in an embodiment of the present invention. The image retrieval adversarial training method based on target enhancement includes the following steps:

[0059] 101. Obtain the initial image dataset.

[0060] In this embodiment of the invention, the initial image dataset includes multiple initial target sample images.

[0061] The aforementioned initial target sample images can be understood as a set of image samples containing a specific target object.

[0062] The initial image dataset mentioned above can be a set of image data of various types, such as photographs, medical images, and satellite images.

[0063] 102. Construct an alternative image dataset based on the initial image dataset, and use the alternative image dataset to train the alternative model of the victim model to obtain multiple local alternative models.

[0064] In this embodiment of the invention, the victim model is the target attacked in the adversarial sample attack.

[0065] The victim model described above is a neural network model trained on a large amount of data, which can be used to perform image retrieval tasks on input images.

[0066] The aforementioned alternative model refers to a local model that has a high degree of similarity to the victim model in terms of retrieval capabilities, aiming to reduce computational costs, simulate the victim model, and thus improve the effectiveness of the attack.

[0067] The aforementioned alternative model can be a pre-trained machine learning model. Alternatively, it can be trained using an alternative dataset built upon a query victim model.

[0068] In this embodiment of the invention, an initial target sample image can be randomly selected from the initial image dataset and input into the victim model for querying to obtain a return list, which is then used as a replacement image dataset.

[0069] Furthermore, the substitution model is trained using a substitute image dataset, resulting in multiple local substitution models.

[0070] The training described above can be supervised training, which can be understood as adjusting the model's parameters using a set of samples with known categories to achieve the desired performance. In supervised learning, each training sample has a known output value.

[0071] The aforementioned alternative image dataset includes multiple replacement target sample images. Each initial target sample image corresponds to at least one replacement target sample image.

[0072] 103. Based on the alternative image dataset and multiple local alternative models, an enhanced image dataset is obtained, and adversarial examples are generated based on the enhanced image dataset.

[0073] In this embodiment of the invention, adversarial example generation based on RIE can be employed to improve image quality and transferability while reducing the storage size of adversarial examples. The key idea of ​​RIE is to exchange the frequency domain information of the original sample and the target sample so that the generated adversarial example can retain the feature information of both.

[0074] The aforementioned RIE refers to the Reverse Information Exchange Module, which consists of DWT, Invertible Neural Networks (INNs), and IDWT. The INNs are networks composed of four fully connected layers.

[0075] Furthermore, adversarial examples are generated from the enhanced image dataset using RIE-based adversarial example generation. This adversarial example generation process can be understood as adding subtle perturbations to the original images to test and improve the model's resistance to attacks.

[0076] It should be noted that augmenting image datasets can reduce the high dependence of adversarial examples on the selection of target samples.

[0077] In one embodiment, an initial sample image can be randomly selected from the initial dataset and input into the substitution model for querying, obtaining a return list, which is then used as the substitution image dataset. The substitution model is trained using the substitution image dataset, resulting in multiple local substitution models. A substitution target sample image is randomly selected from the substitution image dataset and input into the local substitution model for querying, obtaining a return list, which is then used as the substitution image dataset. The local substitution model is trained using the substitution image dataset, resulting in the first new local substitution model. This process is repeated N times to obtain the enhanced image dataset.

[0078] 104. Use the training data of the victim model as positive samples and the adversarial samples as negative samples to perform adversarial training on the victim model.

[0079] In this embodiment of the invention, the adversarial examples are generated by adding small perturbations to the training data of the victim model. Adversarial examples enable the model to learn more complex features, improving its adaptability and robustness to different inputs.

[0080] The adversarial training described above can be understood as the training process for defending against adversarial attacks. Adversarial training improves the robustness of the model by adding adversarial examples to the original training data. Specifically, it involves adding generated adversarial examples (negative samples) to the training set as a data augmentation, allowing the model to learn the adversarial examples during training. The purpose of this is to enable the model to recognize these specially processed inputs, thereby improving the model's security and robustness.

[0081] The adversarial training process described above aims to adjust model parameters by minimizing positive sample distance and maximizing negative sample distance, thus completing the adversarial training. Minimizing positive sample distance means minimizing the distance between data of the same class during training, allowing the model to better identify data of the same category. Maximizing negative sample distance means maximizing the distance between data of different classes during training, allowing the model to better resist adversarial attacks and improve its robustness.

[0082] In this embodiment of the invention, an initial image dataset is obtained, comprising multiple initial target sample images; an alternative image dataset is constructed based on the initial image dataset, and the alternative image dataset is used to train an alternative model of the victim model to obtain multiple local alternative models; an enhanced image dataset is obtained based on the alternative image dataset and the multiple local alternative models, and adversarial examples are generated based on the enhanced image dataset; adversarial training is performed on the victim model using the training data of the victim model as positive samples and the adversarial examples as negative samples. This invention can solve the problems in existing methods where the attack effect on image retrieval is not ideal, resulting in insufficient effectiveness of the generated adversarial examples in adversarial training, and the victim model still lacks robustness.

[0083] Optionally, in the steps of constructing an alternative image dataset based on the initial image dataset, training the alternative model of the victim model with the alternative image dataset, and obtaining multiple local alternative models, an initial target sample image can be randomly selected from the initial image dataset and input into the victim model for querying. The alternative image dataset is obtained based on the query results. The alternative image dataset is then used to train the alternative model of the victim model to obtain multiple local alternative models.

[0084] In this embodiment of the invention, the alternative image dataset includes multiple replacement target sample images, and each initial target sample image corresponds to at least one replacement target sample image.

[0085] It should be noted that when the same initial target sample image is input into the victim model for querying, one or more replacement target sample images may be retrieved.

[0086] The initial image dataset mentioned above includes multiple initial target sample images.

[0087] The training described above can be supervised training, which can be understood as adjusting the model's parameters using a set of samples with known categories to achieve the desired performance. In supervised learning, each training sample has a known output value.

[0088] Specifically, an initial target sample image is randomly selected from the initial image dataset and input into the victim model for querying to obtain a return list. Based on the return list, an alternative image dataset is formed, and the alternative model is trained using the initial image dataset and the alternative image dataset to obtain multiple local alternative models.

[0089] In this embodiment of the invention, an initial target sample image is randomly selected from the initial image dataset and input into the victim model for querying, and the query result is output in the form of a return list. This return list contains the mapping relationship between the initial target image and the corresponding output query result, and a new initial image dataset is formed based on the return list. A new target sample image is randomly selected from the new image dataset and input into the victim model for querying, and a return list is obtained. The return list is used as the new query image dataset. The above process is repeated T times to obtain a replacement image dataset.

[0090] Furthermore, an initial target sample image is randomly selected from the initial image dataset. Input the data into the victim model and query it to get the returned list. This will return to the list. and target sample image Form a mapping , >; then use each ∈ As a new query image dataset, a new target sample image is randomly selected from the new query image dataset. Input the data into the victim model and query it to get a new return list. and return a new list. and new target sample images Form a mapping , Repeat the above process T times to obtain the alternative image dataset.

[0091] Optionally, in the step of training the alternative model of the victim model with the alternative image dataset to obtain multiple local alternative models, the initial features of the initial target sample image can be extracted through the alternative model of the victim model, and the alternative features of the alternative target sample image can be extracted through the alternative model of the victim model; the loss value of the initial features and the alternative features can be calculated through a preset loss function; the model parameters of the alternative model can be adjusted based on the loss value, and after training, multiple local alternative models are obtained.

[0092] In this embodiment of the invention, the initial features are obtained by feature extraction of the initial target sample image by an alternative model of the victim model, and the alternative features are obtained by feature extraction of the alternative target sample image by an alternative model of the victim model. Feature extraction can be understood as a process of converting raw data into meaningful features that the model can understand and use. Feature extraction not only maps data to a new feature space but also reduces the dimensionality of the data, thereby reducing computational complexity and avoiding the curse of dimensionality.

[0093] The preset loss function mentioned above can be the cross-entropy loss function, mean absolute error, mean squared error loss function, etc. The loss function is used to evaluate the degree of error between the initial feature and the substitute feature. A smaller loss value indicates that the initial feature and the substitute feature are closer, and vice versa, it indicates a larger error.

[0094] The loss values ​​described above reflect the accuracy of the feature representation between the initial features and the replacement features. Furthermore, optimization algorithms such as gradient descent can be used to minimize the loss value.

[0095] Optional, the loss function is as follows:

[0096]

[0097] in, It is an alternative model The parameters, It is the initial feature. It is a substitute feature. express and The distance between features, where yes The projection features, It is a marginal constant to ensure that the loss value remains positive.

[0098] In the embodiments of the present invention, the above Used to measure the similarity or difference between two features.

[0099] The above express Features obtained after some transformation or mapping Projection refers to a linear combination of features, or a more complex transformation.

[0100] The above It is a marginal constant to ensure that the loss value remains positive. It is used to avoid the loss value becoming negative due to some reason (such as the feature value being too large or too small) when calculating the loss function, so as to avoid affecting the normal training of the model.

[0101] Optionally, CSQ (Central Similarity Quantization) can be used to generate hash code vectors for the local alternative model. CSQ is a hash network layer.

[0102] The hash network layer described above is an image retrieval optimization technique that reduces the dimensionality of feature vectors extracted by the model into fixed-length hash code vectors, thereby reducing storage space and computation time. The hash network layer is used to accelerate computation and improve retrieval accuracy.

[0103] In one embodiment, to avoid overfitting, the alternative model framework can be AlexNet, VGG16, ResNet50, and DenseNet121, etc. Assuming the alternative model differs from the victim model, if the victim model is AlexNet, then the alternative model will be based on VGG16, ResNet50, or DenseNet121. CSQ can generate more accurate hash codes, and all alternative models use CSQ to generate hash codes.

[0104] Optionally, in the step of obtaining an enhanced image dataset based on the alternative image dataset and multiple local substitution models, and generating adversarial examples based on the enhanced image dataset, the average feature center of the alternative image dataset can be calculated using multiple local substitution models; for the alternative image dataset, the enhanced image dataset is generated by combining the average feature center with the substitute target sample image and a preset perturbation amount; and adversarial examples are generated based on the enhanced image dataset.

[0105] In this embodiment of the invention, the average feature center can reflect the feature distribution in the dataset.

[0106] The perturbation amounts mentioned above are used to make minor modifications to the alternative sample images, thereby producing different sample variants.

[0107] The aforementioned adversarial example generation process can be understood as a process of testing and improving the model's ability to resist attacks by adding small perturbations to the original image.

[0108] Optionally, in the step of calculating the average feature center of the alternative image dataset through multiple local substitution models, the features of each replacement target sample image in the alternative image dataset can be calculated; and the average feature center of the alternative image dataset can be calculated based on the features of each replacement target sample image in the alternative image dataset.

[0109] In this embodiment of the invention, for each replacement target sample image, information features contained in each replacement target sample image are extracted. These information features can be used to describe the visual content of the sample image.

[0110] Furthermore, the features of each replacement target sample image in the replacement image dataset are weighted and averaged to obtain the average feature center of the replacement image dataset. The average feature center of the replacement image dataset can be regarded as the "center point" of the entire dataset and has the ability to characterize the entire dataset. The average feature center can reflect the feature distribution in the replacement image dataset.

[0111] Optionally, the step of calculating the average feature center of the alternative image dataset based on the features of each alternative target sample image in the alternative image dataset includes:

[0112] The average feature center of the alternative image dataset is calculated using the following formula:

[0113]

[0114] in, This represents an alternative image dataset. Indicates an alternative model, Indicates input Alternative models The projected eigenvectors.

[0115] In this embodiment of the invention, the above-mentioned feature vector is only The hash code vector after the hash network layer.

[0116] It should be noted that the surrogate image is optimized by calculating the average feature center of the surrogate image dataset, so that the model can more closely approximate the calculated feature center during training. The average feature center reflects the feature distribution in the surrogate image dataset.

[0117] Optionally, in the step of generating an enhanced image dataset by replacing the target sample image with the average feature center and a preset perturbation amount for the alternative image dataset, the enhanced image dataset is generated using the following formula:

[0118]

[0119] in, As a weighting factor, ; This represents the perturbation amount, which, through iterations of the training process, ultimately yields alternative target samples. ; By calculating Exported and The mean square error of the eigenvectors is used to represent and The feature similarity is calculated using the following formula:

[0120]

[0121] Used to evaluate the two search results and The similarity is calculated using the following formula:

[0122]

[0123] in for The i-th return sample's score. It can be calculated as follows:

[0124]

[0125] according to ∈ exist The position j is used to measure The score; if If it appears, then the fraction is Otherwise, the score is 0.

[0126]

[0127] The augmented image dataset is obtained, represented as .

[0128] In this embodiment of the invention, the mean square error is an indicator used to measure the degree of difference between two vectors.

[0129] Optionally, adversarial examples are generated based on the augmented image dataset, including:

[0130] RIE-based adversarial example generation processes enhanced image datasets to generate adversarial examples. The RIE-based adversarial example generation satisfies the following conditions:

[0131]

[0132] in and These represent DWT and IDWT, respectively. This represents the four networks that make up INN. For any unrelated image, To perturb the target image in the frequency domain;

[0133] Based on enhanced image dataset Generate adversarial examples using RIE The total loss function can be expressed as follows:

[0134]

[0135] in, and These represent the feature exchange loss and the reconstruction loss, respectively. For regularization parameters; It can be represented as follows:

[0136]

[0137] This is for the purpose of constraint and The distance between them is used to limit the range of the disturbance and ensure its imperceptibility, as shown below:

[0138]

[0139] in, These represent the low-frequency and high-frequency components of the DWT, respectively. The weighting factor is the corresponding component; adversarial examples are generated through iterative updates.

[0140] In this embodiment of the invention, the aforementioned adversarial sample generation refers to the process of generating a set of adversarial samples with strong robustness by using irrelevant images uploaded by attackers and target images with frequency domain information perturbation, injecting attack images into neural networks and perturbing the target images.

[0141] The aforementioned enhanced image dataset can be used to test the robustness of the model and improve its resistance to attacks.

[0142] The aforementioned RIE structure refers to the Reverse Information Exchange Module, which consists of a DWT, Invertible Neural Networks (INNs), and an IDWT. The INNs are networks composed of four fully connected layers. The DWT is used to decompose the input image, the INNs are used to exchange features, and the IDWT is used to reconstruct the image.

[0143] The aforementioned DWT (Discrete Wavelet Transform) discretizes the image signal using wavelets, decomposing it into different frequency components for analysis at different scales. IDWT (Inverse Discrete Wavelet Transform), on the other hand, reconstructs the original image from the obtained coefficients through the inverse operation.

[0144] Furthermore, the RIE loss function of the image classification model is modified to a loss function suitable for image retrieval models. In image retrieval tasks, the quality of retrieval results is typically evaluated based on the similarity between the input image and the target image in the retrieval list. Therefore, the loss function needs to be adjusted accordingly to better measure similarity.

[0145] The above total loss function is used to handle the sample variability problem within the target dataset.

[0146] The above This represents the feature exchange loss, which can be understood as the loss of features exchanged between two image datasets in order to reduce the differences between the image datasets.

[0147] The above Represents the reconstruction loss, which can be understood as the reconstruction loss in generative adversarial networks. The reconstruction loss measures the difference between the image generated by the generator and the real image. By minimizing the reconstruction loss, the goal is to make the generator generate more realistic images.

[0148] The above For regularization parameters, This parameter controls the weights of feature exchange loss and reconstruction loss in the total loss function. By adjusting this parameter, the trade-off between the two losses can be balanced.

[0149] The low-frequency components represent the main features of the image, while the high-frequency components represent the detailed information of the image.

[0150] The above This refers to the weighting factor for the corresponding component, which is used to weight the low-frequency and high-frequency components. The weighting factor can be adjusted according to the specific application scenario to control the importance of different frequency components.

[0151] The aforementioned iterative update can be understood as continuously optimizing and enhancing the image dataset through iterative updates to reduce the differences between samples and generate adversarial examples.

[0152] It should be noted that discrete wavelet transform is used to handle the sample differences within the image dataset. The importance of different frequency components is preset by weighting factors, and the image dataset is optimized by iterative updates, ultimately generating adversarial examples with smaller sample differences.

[0153] It should be noted that adversarial example generation based on RIE aims to improve image quality and transferability while reducing the storage size of adversarial examples.

[0154] In this embodiment of the invention, by combining the idea of ​​inverse adversarial samples, the target sample is moved away from the original sample, and the generated adversarial sample will also be moved away from the original sample, thereby achieving a better attack effect.

[0155] In this embodiment of the invention, the target sample set is optimized by constructing strong target samples based on inverse adversarial examples, thereby reducing the impact of extreme samples. This invention introduces frequency domain perturbation to reduce perturbation and image storage size, and proposes a targeted attack involving feature exchange. This invention considers the transferability of perturbations and utilizes model aggregation to reduce overfitting of adversarial examples to a single model, effectively addressing the shortcomings of existing methods in image retrieval attack performance, limiting applicability to other victim models, and addressing the limitations of prior attacks based on directly perturbing the original query image, which exhibit perceptible perturbation or insufficient effectiveness, as well as the problem of the imperceptibility of adversarial examples.

[0156] This invention also provides an image retrieval adversarial training device based on target enhancement. This device can implement all the processes of the image retrieval adversarial training method based on target enhancement described in the above embodiments, and can achieve the same beneficial effects. To avoid repetition, further details are omitted here.

[0157] This invention also provides an electronic device, including: a memory, a processor, and a computer program for an image retrieval adversarial training method based on target enhancement, stored in the memory and executable on the processor. The electronic device provided by this invention can implement all the processes of the image retrieval adversarial training method based on target enhancement described in the above-described method embodiments, and can achieve the same beneficial effects. To avoid repetition, further details are omitted here.

[0158] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the image retrieval adversarial training method based on target enhancement provided in this invention, and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0159] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. An image retrieval adversarial training method based on target enhancement, characterized in that, The method includes the following steps: Obtain an initial image dataset, which includes multiple initial target sample images; An alternative image dataset is constructed based on the initial image dataset. The alternative image dataset is then used to train an alternative model of the victim model, resulting in multiple local alternative models. Based on the alternative image dataset and the multiple local alternative models, an enhanced image dataset is obtained, and adversarial examples are generated based on the enhanced image dataset; The victim model is trained adversarially using the training data of the victim model as positive samples and the adversarial samples as negative samples. For the alternative image dataset, the steps of generating the enhanced image dataset by replacing the target sample image with the average feature center and a preset perturbation amount include: For the alternative image dataset, the enhanced image dataset is generated using the following formula: in, As a weighting factor, ; This represents the perturbation amount, which, through iterations of the training process, ultimately yields alternative target samples. ; By calculating Exported and The mean square error of the eigenvectors is used to represent and The feature similarity is calculated using the following formula: Used to evaluate the two search results and The similarity is calculated using the following formula: in, for The i-th sample in the algorithm returns its score. The calculation is as follows: according to ∈ exist The position j is used to measure The score; if If it appears, then the fraction is Otherwise, the score is 0. The augmented image dataset is obtained, represented as .

2. The image retrieval adversarial training method based on target enhancement as described in claim 1, characterized in that, The steps of constructing an alternative image dataset based on the initial image dataset, and training an alternative model of the victim model using the alternative image dataset to obtain multiple local alternative models include: The initial target sample image is randomly selected from the initial image dataset and input into the victim model for querying. Based on the query results, an alternative image dataset is obtained. The alternative image dataset includes multiple alternative target sample images, and each initial target sample image corresponds to at least one alternative target sample image. The alternative models of the victim model are trained using the alternative image dataset to obtain multiple local alternative models.

3. The image retrieval adversarial training method based on target enhancement as described in claim 2, characterized in that, The step of training an alternative model of the victim model using the alternative image dataset to obtain multiple local alternative models includes: Initial features of the initial target sample image are extracted using the alternative model of the victim model, and alternative features of the alternative target sample image are extracted using the alternative model of the victim model. The loss values ​​of the initial feature and the alternative feature are calculated using a preset loss function; Based on the loss value, the model parameters of the alternative model are adjusted, and after training, multiple local alternative models are obtained.

4. The image retrieval adversarial training method based on target enhancement as described in claim 3, characterized in that, The loss function is as follows: in, It is an alternative model The parameters, It is the initial feature. It is a substitute feature. express and Distance between features yes The projection features, It is a marginal constant to ensure that the loss value remains positive.

5. The image retrieval adversarial training method based on target enhancement as described in claim 1, characterized in that, CSQ is used to generate the hash code vector of the local alternative model.

6. The image retrieval adversarial training method based on target enhancement as described in any one of claims 1 to 5, characterized in that, The steps of obtaining an enhanced image dataset based on the alternative image dataset and the multiple local alternative models, and generating adversarial examples based on the enhanced image dataset, include: The average feature center of the alternative image dataset is calculated using the multiple local substitution models. For the alternative image dataset, an enhanced image dataset is generated by combining the alternative target sample image with the average feature center and a preset perturbation amount; Adversarial examples are generated based on the enhanced image dataset.

7. The image retrieval adversarial training method based on target enhancement as described in claim 6, characterized in that, The step of calculating the average feature center of the alternative image dataset using the multiple local substitution models includes: Calculate the features of each of the alternative target sample images in the alternative image dataset; Based on the features of each of the alternative target sample images in the alternative image dataset, the average feature center of the alternative image dataset is calculated.

8. The image retrieval adversarial training method based on target enhancement as described in claim 7, characterized in that, The step of calculating the average feature center of the alternative image dataset based on the features of each of the alternative target sample images in the alternative image dataset includes: The average feature center of the alternative image dataset is calculated using the following formula: in, This represents an alternative image dataset. Indicates an alternative model, Indicates input Alternative models The projected eigenvectors.

9. The image retrieval adversarial training method based on target enhancement as described in claim 6, characterized in that, The generation of adversarial examples based on the enhanced image dataset includes: RIE-based adversarial example generation performs adversarial example generation processing on the enhanced image dataset to generate adversarial examples. The RIE-based adversarial example generation satisfies the following: in, and These represent DWT and IDWT, respectively. This represents the four networks that make up INNs. For any unrelated image, To perturb the target image in the frequency domain; Based on enhanced image dataset Generate adversarial examples using RIE The total loss function can be expressed as follows: in, and These represent the feature exchange loss and the reconstruction loss, respectively. For regularization parameters; It is expressed as follows: This is for the purpose of constraint and The distance between them is used to limit the range of the disturbance and ensure its imperceptibility, as shown below: in, These represent the low-frequency and high-frequency components of the DWT, respectively. The weighting factor is the corresponding component; adversarial examples are generated through iterative updates.

Images