Image retrieval system performance optimization method and device based on confrontation sample attack, equipment and medium

Abstract

The invention discloses an image retrieval system performance optimization method and device based on confrontation sample attack, equipment and a medium, and relates to the field of image processing, and the method comprises the following steps: inputting an original query image into an image retrieval system to obtain a first retrieval result; generating an initial confrontation sample corresponding to the original query image, and taking the initial confrontation sample as a current confrontation sample; determining a current objective function value corresponding to the objective function based on the first retrieval result and the current confrontation sample, and adjusting the current confrontation sample according to the current objective function value and a constraint condition, and taking the adjusted confrontation sample as a new current confrontation sample, skipping to a step of determining a current objective function value corresponding to the objective function based on the first retrieval result and the current confrontation sample until the current objective function value is zero, inputting the current confrontation sample into the image retrieval system to obtain a second retrieval result, and returning the second retrieval result to the image retrieval system. And if the intersection of the two retrieval results is empty, performing performance optimization on the image retrieval system based on the target adversarial sample.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method, apparatus, device and medium for performance optimization of an image retrieval system based on adversarial sample attacks. Background Technology

[0002] With the rapid development of deep learning technology, image retrieval systems, thanks to their efficient feature extraction and matching capabilities, have been widely applied in key fields such as security monitoring, e-commerce, and medical image analysis. These systems typically extract image features based on deep convolutional neural networks and return results related to the query image from a database by calculating feature similarity. However, recent research has shown that deep neural networks have an inherent vulnerability—they are highly susceptible to adversarial example attacks. Adversarial examples are special images generated by adding subtle noise imperceptible to the human eye to the original image. When input into the neural network, these images can cause the model's output to deviate significantly from reality. In image retrieval scenarios, attackers can construct adversarial examples to cause the image retrieval system to return results unrelated to the query image, severely compromising the system's reliability.

[0003] Currently, the industry lacks a framework and methodology that combines adversarial example attacks with system performance optimization. This involves analyzing the generation mechanism, attack paths, and misjudgment patterns of adversarial examples to pinpoint system weaknesses in feature extraction, similarity calculation, and retrieval strategies, thereby guiding targeted system hardening. Furthermore, research on adversarial examples targeting image retrieval systems is still in its early stages. Existing attack methods suffer from significant limitations in terms of offensiveness and stealth, making it difficult to effectively guide improvements in image retrieval system performance. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, device, and medium for optimizing the performance of an image retrieval system based on adversarial sample attacks. The adversarial samples generated in this application can effectively attack the image retrieval system. By effectively analyzing the attack process, the system's defense capability against adversarial sample attacks can be improved, and system performance can be optimized. The specific solution is as follows:

[0005] Firstly, this application provides a method for performance optimization of an image retrieval system based on adversarial sample attacks, including:

[0006] The original query image is input into a pre-built target image retrieval system to obtain a first retrieval result; the first retrieval result includes K first target images sorted from largest to smallest similarity to the original query image;

[0007] Construct an objective function and corresponding constraints; the objective function is used to control the target image retrieval system to return retrieval results unrelated to the original query image during image retrieval; the constraints are used to ensure that the difference between the original query image and the corresponding adversarial sample is visually indistinguishable.

[0008] Generate an initial adversarial sample corresponding to the original query image, and use the initial adversarial sample as the current adversarial sample;

[0009] Based on the first search result and the current adversarial sample, the current objective function value corresponding to the objective function is determined. The current adversarial sample is adjusted according to the current objective function value and the constraint condition. The adjusted adversarial sample is used as the new current adversarial sample and the process jumps to the step of determining the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, until the current objective function value is zero. The current adversarial sample is then used as the final target adversarial sample.

[0010] The target adversarial sample is input into the target image retrieval system to obtain a second retrieval result. If the intersection of the first retrieval result and the second retrieval result is empty, the performance of the target image retrieval system is optimized based on the target adversarial sample.

[0011] Optionally, the performance optimization method for the image retrieval system based on adversarial sample attacks further includes:

[0012] A pre-defined convolutional neural network is used as a feature extractor to construct an image retrieval system, and the image retrieval system is trained on a target training set using a pre-defined loss function.

[0013] The retrieval performance of the trained image retrieval system is tested using a target test set to obtain test results. Based on the test results, the trained image retrieval system is optimized to obtain a target image retrieval system that meets the preset retrieval performance indicators.

[0014] Optionally, the step of inputting the original query image into a pre-built target image retrieval system to obtain a first retrieval result includes:

[0015] The original query image is input into a pre-built target image retrieval system to extract features from the original query image based on the target image retrieval system, thereby obtaining a first feature vector corresponding to the original query image.

[0016] The similarity between the first feature vector and the second feature vector corresponding to each image in the target database of the target image retrieval system is calculated to obtain the similarity between the original query image and each image in the target database.

[0017] The images in the target database are sorted in descending order of similarity, and the top K sorted first target images are used as the first search results.

[0018] Optionally, the construction of the objective function and the corresponding constraints includes:

[0019] Construct an objective function with the maximum similarity between the feature vector of the current adversarial sample and the feature vectors of each first target image in the first retrieval result, and the Kth largest similarity between the feature vector of the current adversarial sample and the feature vector of the second target image as variables;

[0020] Wherein, the second target image is all the images remaining in the target database of the target image retrieval system after removing the first target image;

[0021] The constraint condition is that the infinite norm of the difference between the pixel value of the current adversarial example and the pixel value of the original query image does not exceed a preset maximum infinite norm.

[0022] Optionally, determining the current objective function value corresponding to the objective function based on the first retrieval result and the current adversarial example includes:

[0023] The maximum similarity between the feature vector of the current adversarial sample and the feature vectors of each first target image in the first retrieval result is subtracted from the Kth largest similarity between the feature vector of the current adversarial sample and the feature vector of the second target image, and then the target result is added to the target constant; the target constant is a constant that is greater than zero and meets the preset minimum constant standard.

[0024] If the target result is positive, then the target result is used as the current target function value corresponding to the target function; if the target result is not positive, then zero is used as the current target function value corresponding to the target function.

[0025] Optionally, adjusting the current adversarial example based on the current objective function value and the constraints includes:

[0026] If the current objective function value is not zero, the current adversarial sample is updated using the gradient descent algorithm to obtain the updated adversarial sample;

[0027] Determine whether the updated adversarial sample satisfies the constraint condition. If the updated adversarial sample does not satisfy the constraint condition, then crop the pixel values ​​of the updated adversarial sample so that the infinity norm of the difference between the pixel values ​​of the cropped adversarial sample and the pixel values ​​of the original query image does not exceed the preset maximum infinity norm.

[0028] Optionally, inputting the target adversarial sample into the target image retrieval system to obtain a second retrieval result includes:

[0029] The target adversarial sample is input into the target image retrieval system, and features are extracted from the target adversarial sample based on the target image retrieval system to obtain a third feature vector corresponding to the target adversarial sample.

[0030] The similarity between the third feature vector and the second feature vector corresponding to each image in the target database of the target image retrieval system is calculated to obtain the similarity between the target adversarial sample and each image in the target database.

[0031] The images in the target database are sorted in descending order of similarity, and the top K third target images after sorting are used as the second search results.

[0032] Accordingly, after inputting the target adversarial sample into the target image retrieval system to obtain the second retrieval result, the method further includes:

[0033] Determine whether there is a common image between the third target image and the first target image corresponding to the original query image. If not, determine that the intersection of the first search result and the second search result is empty.

[0034] Secondly, this application provides a performance optimization device for an image retrieval system based on adversarial sample attacks, comprising:

[0035] The result acquisition module is used to input the original query image into a pre-built target image retrieval system to obtain a first retrieval result; the first retrieval result includes K first target images sorted from largest to smallest similarity to the original query image;

[0036] A function construction module is used to construct an objective function and corresponding constraints. The objective function is used to control the target image retrieval system to return retrieval results unrelated to the original query image during image retrieval. The constraints are used to ensure that the difference between the original query image and the corresponding adversarial sample is visually indistinguishable.

[0037] An initial sample determination module is used to generate an initial adversarial sample corresponding to the original query image, and to use the initial adversarial sample as the current adversarial sample.

[0038] The target sample determination module is used to determine the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, adjust the current adversarial sample according to the current objective function value and the constraint conditions, take the adjusted adversarial sample as the new current adversarial sample and jump to the step of determining the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, until the current objective function value is zero, and take the current adversarial sample as the final target adversarial sample;

[0039] The performance optimization module is used to input the target adversarial sample into the target image retrieval system to obtain a second retrieval result. If the intersection of the first retrieval result and the second retrieval result is empty, the performance of the target image retrieval system is optimized based on the target adversarial sample.

[0040] Thirdly, this application provides an electronic device, comprising:

[0041] Memory, used to store computer programs;

[0042] A processor is used to execute the computer program to implement the aforementioned performance optimization method for image retrieval systems based on adversarial sample attacks.

[0043] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned performance optimization method for an image retrieval system based on adversarial sample attacks.

[0044] In this application, the original query image is input into a pre-constructed target image retrieval system to obtain a first retrieval result; the first retrieval result includes K first target images sorted from largest to smallest similarity to the original query image; an objective function and corresponding constraints are constructed; the objective function is used to control the target image retrieval system to return retrieval results unrelated to the original query image during image retrieval; the constraints are used to ensure that the difference between the original query image and the corresponding adversarial sample is visually indistinguishable; an initial adversarial sample corresponding to the original query image is generated, and the initial adversarial sample is used as the current adversarial sample; based on the first retrieval result and the current adversarial sample... The sample determines the current objective function value corresponding to the objective function. Based on the current objective function value and the constraints, the current adversarial sample is adjusted. The adjusted adversarial sample is used as the new current adversarial sample, and the process jumps to the step of determining the current objective function value based on the first search result and the current adversarial sample, until the current objective function value is zero. The current adversarial sample is then used as the final target adversarial sample. The target adversarial sample is input into the target image retrieval system to obtain a second search result. If the intersection of the first search result and the second search result is empty, the performance of the target image retrieval system is optimized based on the target adversarial sample. As can be seen from the above, this application explicitly targets an attack with the goal of "making the intersection of the final target adversarial sample's search result and the original query image's search result empty," and generates the final target adversarial sample based on this attack goal, thus improving the attack success rate of the target adversarial sample. On the other hand, by strictly controlling the pixel differences between the adversarial sample and the original image to be within a range imperceptible to the human eye through constraints, the concealment of the target adversarial sample is ensured. By using the target adversarial sample with good attack and concealment to attack the image retrieval system, the system's ability to resist adversarial sample attacks can be effectively trained, the recall rate of the image retrieval system can be improved, and thus the system performance can be improved. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0046] Figure 1 This is a flowchart of a performance optimization method for an image retrieval system based on adversarial sample attacks disclosed in this application;

[0047] Figure 2This is a schematic diagram illustrating a specific performance optimization method for an image retrieval system based on adversarial sample attacks disclosed in this application.

[0048] Figure 3 This is a schematic diagram of the structure of a performance optimization device for an image retrieval system based on adversarial sample attacks disclosed in this application;

[0049] Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] Currently, the industry lacks ideas and methods for optimizing the performance of image retrieval systems using adversarial example attacks. Furthermore, existing research on adversarial examples targeting image retrieval systems is still in its early stages, and existing attack methods have significant limitations in terms of offensiveness and stealth. Utilizing these limited attack methods is not conducive to guiding improvements in image retrieval system performance. Therefore, this application provides a performance optimization method for image retrieval systems based on adversarial example attacks. The adversarial examples generated in this application can effectively attack the image retrieval system. By effectively analyzing the attack process, the system's defense capabilities against adversarial example attacks can be improved, thereby optimizing system performance.

[0052] See Figure 1 As shown in the figure, this application discloses a performance optimization method for an image retrieval system based on adversarial sample attacks, including:

[0053] Step S11: Input the original query image into the pre-constructed target image retrieval system to obtain the first retrieval result; the first retrieval result includes K first target images sorted from largest to smallest similarity to the original query image.

[0054] In this embodiment, to implement non-targeted adversarial example attacks against the image retrieval system, a high-performance image retrieval system can first be constructed. Specifically, this can include: using a pre-defined convolutional neural network as a feature extractor to build the image retrieval system, and training the system on a target training set using a preset loss function. Then, the retrieval performance of the trained system is tested using a target test set to obtain test results. Based on these results, the system is optimized to obtain a target image retrieval system that meets preset performance metrics.

[0055] For example, an image retrieval system typically consists of a convolutional neural network (CNN) and a database. The CNN is used to extract image features. In this embodiment, a ResNet50 CNN can be selected as the network used by the system. The Margin loss function is used to train the ResNet50 CNN on the CUB_200 dataset. After training, the recall rates Recall@1, Recall@2, Recall@4, Recall@8, and Recall@16 are obtained, as shown in Table 1.

[0056] Table 1

[0057] Model loss function Recall@1 Recall@2 Recall@4 Recall@8 Recall@16 Resnet50 Margin 62.3% 73.4% 82.5% 89.0% 93.5%

[0058] Among them, Recall@1, Recall@2, Recall@4, Recall@8, and Recall@16 represent recall, which is a metric for measuring the performance of an image retrieval system. Their specific meanings are as follows:

[0059] Recall@1: refers to the proportion of images that are relevant to the query image among the first-ranked images returned by the image retrieval system from the database.

[0060] Recall@2: This refers to the proportion of the top two images returned by the system, where at least one is related to the query image.

[0061] Recall@4: This refers to the proportion of the top 4 images returned by the system, where at least one is related to the query image.

[0062] Recall@8: This refers to the proportion of the top 8 images returned by the system, where at least one is related to the query image.

[0063] Recall@16: This refers to the proportion of the top 16 images returned by the system that are related to the query image, at least one of which is a relevant image.

[0064] The recall value ranges from 0 to 1; a higher recall indicates better retrieval performance of the trained system. Based on the obtained recall, it can be seen that the trained target image retrieval system exhibits high retrieval performance on the CUB_200 dataset.

[0065] Furthermore, the original query image can be input into the pre-constructed target image retrieval system mentioned above to extract features from the original query image, obtaining a first feature vector corresponding to the original query image. Then, the first feature vector can be compared with the second feature vectors corresponding to each image in the target database within the target image retrieval system to calculate the similarity between the original query image and each image in the target database. Finally, the images in the target database can be sorted according to their similarity scores from highest to lowest, and the top K sorted first target images are used as the first retrieval results.

[0066] It should be noted that the target image retrieval system is composed of convolutional neural networks. and database Composition, combining the images in the database After being input into the target image retrieval system, the convolutional neural network in the system extracts the features of each image to obtain the corresponding feature vector. Similarly, the original query image After the target image is input into the target image retrieval system, the convolutional neural network in the target image retrieval system extracts the original query image. The features are used to obtain the feature vector. The original query image The system calculates the similarity between the feature vectors of the images and the feature vectors of each image in the database. The images in the database are then sorted in descending order of similarity, and the top K images are returned as the search results. , ;in, The target image retrieval system is based on each image and the original query image. Similarity from database Retrieved from Zhang image.

[0067] Step S12: Construct the objective function and corresponding constraints; the objective function is used to control the target image retrieval system to return retrieval results that are unrelated to the original query image when performing image retrieval; the constraints are used to ensure that the difference between the original query image and the corresponding adversarial sample cannot be visually distinguished.

[0068] In this embodiment, an objective function can be constructed with the maximum similarity between the feature vector of the current adversarial sample and the feature vectors of each first target image in the first search result, and the Kth largest similarity between the feature vector of the current adversarial sample and the feature vector of the second target image as variables; wherein, the second target image is all images remaining in the target database of the target image retrieval system excluding the first target images. Simultaneously, a constraint condition is constructed that the infinity norm of the difference between the pixel values ​​of the current adversarial sample and the pixel values ​​of the original query image does not exceed a preset maximum infinity norm.

[0069] Specifically, in this embodiment, the objective function and corresponding constraints for non-target adversarial example attacks against the target image retrieval system can be constructed in the following specific form:

[0070] ;

[0071] Among them, non-targeted adversarial example attacks against target image retrieval systems refer to attacks on the original query image. Adding subtle noise to obtain adversarial examples Then Input the target image into the image retrieval system and obtain the retrieval results. , ;in, It is a target image retrieval system that compares each image with adversarial examples. Similarity from database The K images retrieved from the query result are not targeted attacks, which refer to the original query image. Search results Adversarial Examples Search results There are no duplicate elements, that is and The intersection of them is the empty set. .

[0072] in, Represents the original query image Search results The maximum similarity between the feature vectors of each image and the feature vectors of the adversarial examples; In addition to the search results Each image in the database, and all other images in the database; express The Kth largest similarity value between the feature vectors of each image and the feature vectors of the adversarial sample; It is a minimal constant greater than 0; constraint conditions Limiting adversarial examples Compared with the original query image The maximum infinite norm is , It is usually set to 8 / 255.

[0073] Step S13: Generate an initial adversarial sample corresponding to the original query image, and use the initial adversarial sample as the current adversarial sample.

[0074] In this embodiment, the original query image is used. Generate the corresponding initial adversarial sample, and use the initial adversarial sample as the current adversarial sample for iteration.

[0075] Step S14: Based on the first search result and the current adversarial sample, determine the current objective function value corresponding to the objective function; adjust the current adversarial sample according to the current objective function value and the constraint conditions; take the adjusted adversarial sample as the new current adversarial sample and jump to the step of determining the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, until the current objective function value is zero, and take the current adversarial sample as the final target adversarial sample.

[0076] In this embodiment, firstly, the maximum similarity between the feature vector of the current adversarial sample and the feature vectors of each first target image in the first search result is determined, minus the Kth largest similarity between the feature vector of the current adversarial sample and the feature vector of the second target image, and then the target result of the target constant is added; wherein, the target constant is a constant greater than zero that satisfies the preset minimum constant standard. If the target result is a positive number, then the target result is used as the current target function value corresponding to the target function, i.e., the current... If the target result is not positive, zero is used as the current objective function value. Further, if the current objective function value is not zero, the gradient descent algorithm is used to update the current adversarial sample, resulting in an updated adversarial sample. It is then determined whether the updated adversarial sample satisfies the constraints in step S13. If the updated adversarial sample does not satisfy the constraints, the pixel values ​​of the updated adversarial sample are cropped so that the infinity norm of the difference between the cropped adversarial sample's pixel values ​​and the original query image does not exceed a preset maximum infinity norm.

[0077] Understandably, for target image retrieval systems, adversarial examples can be generated using gradient descent based on the optimization objective of non-target adversarial example attacks. During the generation of adversarial examples, it is necessary to... The pixel values ​​of the generated adversarial examples are cropped under norm constraints to limit the size of added subtle noise and adjust pixel values ​​that exceed the difference limit to the allowable range, ensuring that the cropped adversarial examples meet the visual concealment requirements and can continue to participate in optimization.

[0078] The above process, through a closed-loop logic of "objective function value + gradient descent + constraint control", ensures that the adversarial example gradually approaches the "attack target" during iteration.

[0079] Step S15: Input the target adversarial sample into the target image retrieval system to obtain a second retrieval result. If the intersection of the first retrieval result and the second retrieval result is empty, then optimize the performance of the target image retrieval system based on the target adversarial sample.

[0080] In this embodiment, a target adversarial sample can be input into a target image retrieval system to extract features from the target adversarial sample, obtaining a third feature vector corresponding to the target adversarial sample. Then, the third feature vector is compared with the second feature vectors corresponding to each image in the target database within the target image retrieval system to calculate the similarity between the target adversarial sample and each image in the target database. The images in the target database are then sorted according to their similarity scores from highest to lowest, and the top K sorted third target images are used as the second retrieval results. Further, it is determined whether there is a common image between the third target image and the first target image corresponding to the original query image. If not, the intersection of the first and second retrieval results is considered empty. If the intersection of the first and second retrieval results is empty, the performance of the target image retrieval system can be optimized based on the target adversarial sample.

[0081] As can be seen from the above, after generating adversarial examples for the target image retrieval system, the original query image can be... Adversarial examples Input the target image into the image retrieval system and obtain the respective retrieval results. and If the two do not have the same image, that is and If the intersection of the two sets is empty, then the adversarial sample has successfully attacked the target image retrieval system, achieving a non-targeted attack. Furthermore, by analyzing the generation mechanism, attack path, and misjudgment pattern of the adversarial sample, the system's defects in feature extraction, similarity calculation, retrieval strategies, etc., can be located, guiding the system to carry out targeted reinforcement to optimize system performance.

[0082] The effectiveness of the method for generating adversarial examples in this embodiment is verified through a specific experimental process.

[0083] 1. Experimental setup

[0084] The image retrieval system consists of a convolutional neural network and a database. The system uses the ResNet50 model for the convolutional neural network. During model training, the training set of the CUB_200 dataset is used as the database. During model testing, the test set of the CUB_200 dataset is used as the database.

[0085] In the adversarial example attack testing phase, 100 images were randomly selected from the test set of the CUB_200 dataset as query images. Minor noise was added to these query images to obtain adversarial examples, which were then used to attack the image retrieval system.

[0086] 2. Experimental Indicators

[0087] Non-targeted adversarial attacks on image retrieval systems need to achieve three objectives:

[0088] Objective 1: The adversarial example and the query image need to be very similar.

[0089] Objective 2: Input adversarial examples and query images into the image retrieval system and obtain their respective retrieval results. and ,like and If the two images are not identical, then the adversarial example successfully attacked the image retrieval system.

[0090] Objective 3: Adversarial examples can significantly reduce the recall rate (Recall@1) of image retrieval systems.

[0091] For objective 1, since norm, The norm, SSIM (Structural Similarity Index), and PSNR (Peak Signal-to-Noise Ratio) are four metrics used to measure image similarity. Therefore, these four metrics were used in the experiment to calculate the similarity between adversarial examples and the query image. For target 2, the Attack Success Rate (ASR) was used to count the proportion of adversarial examples that could successfully attack the image retrieval system. For target 3, Recall@1 was used directly.

[0092] 3. Experimental Results

[0093] The number of images K in the search results of the image retrieval system is set to 8, 16, and 32 respectively. The corresponding indicators for target 1, target 2, and target 3 are shown in Table 2.

[0094] Table 2

[0095] Number of images retrieved norm norm SSIM PSNR Attack success rate Recall@1 K=8 1.399 0.0053 0.996 50.52 100% 35.7% K=16 2.055 0.0068 0.994 48.98 100% 21.7% K=32 2.727 0.0075 0.992 47.70 100% 6.2%

[0096] As shown in Table 2, in the experiment norm 0.0075, far below the threshold of human visual perception. The results show that the adversarial examples are visually indistinguishable from the original images, verifying the stealth capability. The ASR (Average Recall Rate) reached 100% in the experiment, indicating that the method can stably achieve non-target attacks. The Recall@1 value decreased significantly with increasing K value, dropping to 6.2% at K=32, demonstrating that the adversarial examples effectively compromise the system's retrieval accuracy.

[0097] As can be seen from the above, see Figure 2 As shown, this embodiment mainly consists of five parts: 1. Training an image retrieval system with high retrieval performance, including training the ResNet50 model on the CUB_200 training dataset and testing the ResNet50 model on the CUB_200 test dataset; 2. Designing the objective function for non-targeted adversarial attacks, including ensuring the intersection of the retrieval results of the adversarial sample and the query image is empty, and designing the objective function based on the empty intersection of the retrieval results; 3. Generating adversarial samples by minimizing the objective function value under constraints, including constraints such as... The process involves: 1) generating and cropping adversarial examples by minimizing the objective function value using norms; 2) launching adversarial attacks on the image retrieval system to achieve non-targeted attacks, including inputting adversarial examples and query images into the system. If no identical images are found in the search results, a non-targeted attack is achieved; 3) optimizing the performance of the target image retrieval system based on the non-targeted attack. In this way, this embodiment generates aggressive and covert target adversarial examples and uses them to effectively launch non-targeted adversarial attacks on the image retrieval system. By analyzing the generation mechanism, attack path, and misjudgment patterns of the target adversarial examples, the system's deficiencies in feature extraction, similarity calculation, and retrieval strategies are identified, effectively training the system's ability to resist adversarial attacks and improving the performance of the image retrieval system.

[0098] See Figure 3 As shown in the figure, this application also discloses a performance optimization device for an image retrieval system based on adversarial sample attacks, including:

[0099] The result acquisition module 11 is used to input the original query image into a pre-built target image retrieval system to obtain a first retrieval result; the first retrieval result includes K first target images sorted from largest to smallest similarity to the original query image;

[0100] The function construction module 12 is used to construct the objective function and the corresponding constraints; the objective function is used to control the target image retrieval system to return retrieval results that are unrelated to the original query image when performing image retrieval; the constraints are used to ensure that the difference between the original query image and the corresponding adversarial sample cannot be visually distinguished.

[0101] The initial sample determination module 13 is used to generate an initial adversarial sample corresponding to the original query image, and to use the initial adversarial sample as the current adversarial sample.

[0102] The target sample determination module 14 is used to determine the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, adjust the current adversarial sample according to the current objective function value and the constraint conditions, take the adjusted adversarial sample as the new current adversarial sample and jump to the step of determining the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, until the current objective function value is zero, and take the current adversarial sample as the final target adversarial sample;

[0103] The performance optimization module 15 is used to input the target adversarial sample into the target image retrieval system to obtain a second retrieval result. If the intersection of the first retrieval result and the second retrieval result is empty, the performance of the target image retrieval system is optimized based on the target adversarial sample.

[0104] As shown above, this application explicitly targets the attack by ensuring that the retrieval results of the final adversarial sample have no intersection with the retrieval results of the original query image. Based on this target, the final adversarial sample is generated, thus improving the success rate of the adversarial sample attack. Furthermore, by strictly controlling the pixel differences between the adversarial sample and the original image to be imperceptible to the human eye through constraints, the stealth of the adversarial sample is guaranteed. Attacking the image retrieval system with a highly aggressive and stealthy adversarial sample can effectively train the system's ability to resist adversarial attacks, improving the recall rate of the image retrieval system and consequently enhancing its performance.

[0105] In some specific embodiments, the image retrieval system performance optimization device based on adversarial sample attacks further includes:

[0106] The system training unit is used to construct an image retrieval system by using a pre-set convolutional neural network as a feature extractor, and to train the image retrieval system on a target training set using a pre-set loss function.

[0107] The system optimization unit is used to test the retrieval performance of the trained image retrieval system using a target test set, obtain test results, and optimize the tested image retrieval system based on the test results to obtain a target image retrieval system that meets preset retrieval performance indicators.

[0108] In some specific embodiments, the result acquisition module 11 includes:

[0109] The first feature extraction unit is used to input the original query image into a pre-constructed target image retrieval system, so as to extract features from the original query image based on the target image retrieval system and obtain a first feature vector corresponding to the original query image.

[0110] The first similarity calculation unit is used to calculate the similarity between the first feature vector and the second feature vector corresponding to each image in the target database of the target image retrieval system, so as to obtain the similarity between the original query image and each image in the target database.

[0111] The first retrieval result determination unit is used to sort the images in the target database according to the order of similarity from large to small, and take the first K sorted first target images as the first retrieval results.

[0112] In some specific embodiments, the function construction module 12 includes:

[0113] The function construction unit is used to construct an objective function with the maximum similarity between the feature vector of the current adversarial sample and the feature vectors of each first target image in the first retrieval result and the Kth largest similarity between the feature vector of the current adversarial sample and the feature vector of the second target image as variables.

[0114] Wherein, the second target image is all the images remaining in the target database of the target image retrieval system after removing the first target image;

[0115] The constraint construction unit is used to construct a constraint that the infinite norm of the difference between the pixel value of the current adversarial sample and the pixel value of the original query image does not exceed a preset maximum infinite norm.

[0116] In some specific embodiments, the target sample determination module 14 includes:

[0117] The result determination unit is used to determine the maximum similarity between the feature vector of the current adversarial sample and the feature vector of each first target image in the first retrieval result, minus the Kth largest similarity between the feature vector of the current adversarial sample and the feature vector of the second target image, and add the target result of the target constant; the target constant is a constant that is greater than zero and meets the preset minimum constant standard;

[0118] The function value determination unit is used to determine the target value as the current target function value if the target result is positive, and to determine the target function value as zero if the target result is not positive.

[0119] In some specific embodiments, the target sample determination module 14 includes:

[0120] The sample update unit is used to update the current adversarial sample using the gradient descent algorithm if the current objective function value is not zero, so as to obtain the updated adversarial sample.

[0121] The first judgment unit is used to determine whether the updated adversarial sample satisfies the constraint condition. If the updated adversarial sample does not satisfy the constraint condition, the pixel values ​​of the updated adversarial sample are cropped so that the infinity norm of the difference between the pixel values ​​of the cropped adversarial sample and the pixel values ​​of the original query image does not exceed the preset maximum infinity norm.

[0122] In some specific embodiments, the performance optimization module 15 includes:

[0123] The second feature extraction unit is used to input the target adversarial sample into the target image retrieval system, so as to extract features from the target adversarial sample based on the target image retrieval system and obtain a third feature vector corresponding to the target adversarial sample.

[0124] The second similarity calculation unit is used to calculate the similarity between the third feature vector and the second feature vector corresponding to each image in the target database of the target image retrieval system, so as to obtain the similarity between the target adversarial sample and each image in the target database.

[0125] The second retrieval result determination unit is used to sort the images in the target database according to the order of similarity from large to small, and take the first K third target images after sorting as the second retrieval result;

[0126] Accordingly, the performance optimization module 15 also includes:

[0127] The second judgment unit is used to determine whether there is a common image between the third target image and the first target image corresponding to the original query image. If there is no common image, the intersection of the first search result and the second search result is determined to be empty.

[0128] Furthermore, embodiments of this application also disclose an electronic device, Figure 4This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0129] Figure 4 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the performance optimization method for the image retrieval system based on adversarial sample attacks disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0130] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0131] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0132] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the performance optimization method for the image retrieval system based on adversarial sample attacks disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0133] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned performance optimization method for an image retrieval system based on adversarial sample attacks. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0135] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0137] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0138] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for performance optimization of an image retrieval system based on adversarial sample attacks, characterized in that, include: The original query image is input into a pre-built target image retrieval system to obtain the first retrieval result; The first search result includes K first target images sorted from largest to smallest similarity to the original query image; Construct an objective function and corresponding constraints; the objective function is used to control the target image retrieval system to return retrieval results unrelated to the original query image during image retrieval; the constraints are used to ensure that the difference between the original query image and the corresponding adversarial sample is visually indistinguishable. Generate an initial adversarial sample corresponding to the original query image, and use the initial adversarial sample as the current adversarial sample; Based on the first search result and the current adversarial sample, the current objective function value corresponding to the objective function is determined. The current adversarial sample is adjusted according to the current objective function value and the constraint condition. The adjusted adversarial sample is used as the new current adversarial sample and the process jumps to the step of determining the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, until the current objective function value is zero. The current adversarial sample is then used as the final target adversarial sample. The target adversarial sample is input into the target image retrieval system to obtain a second retrieval result. If the intersection of the first retrieval result and the second retrieval result is empty, the performance of the target image retrieval system is optimized based on the target adversarial sample.

2. The performance optimization method for an image retrieval system based on adversarial sample attacks according to claim 1, characterized in that, Also includes: A pre-defined convolutional neural network is used as a feature extractor to construct an image retrieval system, and the image retrieval system is trained on a target training set using a pre-defined loss function. The retrieval performance of the trained image retrieval system is tested using a target test set to obtain test results. Based on the test results, the trained image retrieval system is optimized to obtain a target image retrieval system that meets the preset retrieval performance indicators.

3. The performance optimization method for an image retrieval system based on adversarial sample attacks according to claim 1, characterized in that, The step of inputting the original query image into a pre-built target image retrieval system to obtain the first retrieval result includes: The original query image is input into a pre-built target image retrieval system to extract features from the original query image based on the target image retrieval system, thereby obtaining a first feature vector corresponding to the original query image. The similarity between the first feature vector and the second feature vector corresponding to each image in the target database of the target image retrieval system is calculated to obtain the similarity between the original query image and each image in the target database. The images in the target database are sorted in descending order of similarity, and the top K sorted first target images are used as the first search results.

4. The performance optimization method for an image retrieval system based on adversarial sample attacks according to any one of claims 1 to 3, characterized in that, The construction of the objective function and the corresponding constraints includes: Construct an objective function with the maximum similarity between the feature vector of the current adversarial sample and the feature vectors of each first target image in the first retrieval result, and the Kth largest similarity between the feature vector of the current adversarial sample and the feature vector of the second target image as variables; Wherein, the second target image is all the images remaining in the target database of the target image retrieval system after removing the first target image; The constraint condition is that the infinite norm of the difference between the pixel value of the current adversarial example and the pixel value of the original query image does not exceed a preset maximum infinite norm.

5. The performance optimization method for an image retrieval system based on adversarial sample attacks according to claim 4, characterized in that, The step of determining the current objective function value corresponding to the objective function based on the first retrieval result and the current adversarial sample includes: The maximum similarity between the feature vector of the current adversarial sample and the feature vectors of each first target image in the first retrieval result is subtracted from the Kth largest similarity between the feature vector of the current adversarial sample and the feature vector of the second target image, and then the target result is added to the target constant; the target constant is a constant that is greater than zero and meets the preset minimum constant standard. If the target result is positive, then the target result is used as the current target function value corresponding to the target function; if the target result is not positive, then zero is used as the current target function value corresponding to the target function.

6. The performance optimization method for an image retrieval system based on adversarial sample attacks according to claim 5, characterized in that, The step of adjusting the current adversarial sample based on the current objective function value and the constraints includes: If the current objective function value is not zero, the current adversarial sample is updated using the gradient descent algorithm to obtain the updated adversarial sample; Determine whether the updated adversarial sample satisfies the constraint condition. If the updated adversarial sample does not satisfy the constraint condition, then crop the pixel values ​​of the updated adversarial sample so that the infinity norm of the difference between the pixel values ​​of the cropped adversarial sample and the pixel values ​​of the original query image does not exceed the preset maximum infinity norm.

7. The performance optimization method for an image retrieval system based on adversarial sample attacks according to claim 1, characterized in that, The step of inputting the target adversarial sample into the target image retrieval system to obtain a second retrieval result includes: The target adversarial sample is input into the target image retrieval system, and features are extracted from the target adversarial sample based on the target image retrieval system to obtain a third feature vector corresponding to the target adversarial sample. The similarity between the third feature vector and the second feature vector corresponding to each image in the target database of the target image retrieval system is calculated to obtain the similarity between the target adversarial sample and each image in the target database. The images in the target database are sorted in descending order of similarity, and the top K third target images after sorting are used as the second search results. Accordingly, after inputting the target adversarial sample into the target image retrieval system to obtain the second retrieval result, the method further includes: Determine whether there is a common image between the third target image and the first target image corresponding to the original query image. If not, determine that the intersection of the first search result and the second search result is empty.

8. A performance optimization device for an image retrieval system based on adversarial sample attacks, characterized in that, include: The result acquisition module is used to input the original query image into a pre-built target image retrieval system to obtain the first retrieval result; The first search result includes K first target images sorted from largest to smallest similarity to the original query image; A function construction module is used to construct an objective function and corresponding constraints. The objective function is used to control the target image retrieval system to return retrieval results unrelated to the original query image during image retrieval. The constraints are used to ensure that the difference between the original query image and the corresponding adversarial sample is visually indistinguishable. An initial sample determination module is used to generate an initial adversarial sample corresponding to the original query image, and to use the initial adversarial sample as the current adversarial sample. The target sample determination module is used to determine the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, adjust the current adversarial sample according to the current objective function value and the constraint conditions, take the adjusted adversarial sample as the new current adversarial sample and jump to the step of determining the current objective function value corresponding to the objective function based on the first search result and the current adversarial sample, until the current objective function value is zero, and take the current adversarial sample as the final target adversarial sample; The performance optimization module is used to input the target adversarial sample into the target image retrieval system to obtain a second retrieval result. If the intersection of the first retrieval result and the second retrieval result is empty, the performance of the target image retrieval system is optimized based on the target adversarial sample.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the performance optimization method for an image retrieval system based on adversarial sample attacks as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the image retrieval system performance optimization method based on adversarial sample attacks as described in any one of claims 1 to 7.

Images