Adversarial robustness evaluation method and device for deep hashing model

By calculating the tolerance and semantic expressive power of hash codes to select anchor points, and generating general target adversarial perturbations under the guidance of anchor points, the accuracy problem of adversarial robustness assessment of deep hash models is solved, and a significant improvement in adversarial robustness is achieved.

CN117312627BActive Publication Date: 2026-03-24INSTITUTE OF INFORMATION ENGINEERING CHINESE ACADEMY OF SCIENCES
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-09
Publication Date
2026-03-24

AI Technical Summary

Technical Problem

Existing methods for evaluating the robustness of deep hashing models against adversarial attacks cannot reliably and accurately assess the model's ability to resist attacks. This is mainly because anchor generation methods ignore the correlation between anchors and adversarial examples, which limits the ability to retrieve adversarial examples.

Method used

This paper proposes an adversarial robustness evaluation method for deep hashing models. By calculating the tolerance and semantic expressive power of hash codes, high-quality anchor points are selected. Under the guidance of anchor points, a general target adversarial perturbation is generated iteratively. Gradient descent is used to update the adversarial perturbation, and a loss function is designed to optimize the Hamming distance, thereby improving the accuracy of adversarial robustness evaluation.

Benefits of technology

It significantly improves the accuracy and reliability of adversarial robustness assessment, and the stability and attack capability of the generated adversarial perturbations are significantly enhanced, which can effectively assess the adversarial robustness of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117312627B_ABST
    Figure CN117312627B_ABST
Patent Text Reader

Abstract

The application relates to a kind of facing deep hash model's method and device for evaluating the robustness of confrontation, which comprises the following steps: calculating the tolerance of hash code in the database;Filtering out the hash code with a tolerance greater than a threshold to form a candidate anchor set;Based on the semantic expression ability of the hash code in the candidate anchor set, an anchor point is obtained;Under the guidance of the anchor point, a general target adversarial perturbation is generated iteratively using training data;According to the attack effect of the general target adversarial perturbation on the test data, the robustness of the deep hash model to confrontation is evaluated.The application can reliably and accurately evaluate the robustness of hash model to confrontation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of deep neural network technology, and in particular relates to a method and apparatus for evaluating the adversarial robustness of deep hash models. Background Technology

[0002] With the advent of the era of artificial intelligence and big data, large-scale visual retrieval tasks have received widespread attention. Deep hashing technology, which utilizes deep neural networks to learn compact hash codes for managing and analyzing large-scale visual data, has advantages such as small storage space and fast retrieval speed, making it a key technology for solving large-scale visual retrieval problems. However, with the widespread application of deep hashing models, their robustness and security issues have gradually attracted attention. Recent studies have shown that deep neural networks are vulnerable to attacks from carefully designed adversarial examples. Because deep hashing models use deep neural networks to extract features, they also face security risks from adversarial examples. Specifically, by adding adversarial perturbations invisible to the human eye to the query data, adversarial examples can interfere with the retrieval model and return incorrect or irrelevant images. In order to fully explore the "vulnerabilities" in the robustness of models, discover potential security risks, and thus comprehensively improve the ability of deep hashing models to resist adversarial attacks, many methods for evaluating the adversarial robustness of deep hashing models have been proposed. Among them, implementing adversarial attacks on the model and then testing and analyzing the model's ability to resist adversarial examples is an important method for evaluating model adversarial robustness. However, due to the limitations of the attack capabilities of current adversarial attack methods, the adversarial robustness of deep hashing models cannot be reliably and accurately evaluated. Therefore, how to improve the adversarial attack capability of deep hashing models, and thus enhance the accuracy and reliability of adversarial robustness assessment, is a problem worth exploring.

[0003] This invention aims to reliably and accurately evaluate the robustness of models by exploring a general adversarial perturbation generation method for deep hashing. "General" is defined based on the method of adding adversarial perturbations, indicating that the generated perturbations can be added to any clean data to form adversarial examples with predefined attack effects. "Target" is defined based on the purpose of generating adversarial perturbations, indicating that the generated adversarial examples are designed to retrieve images of a specific category. Currently, deep hashing-based adversarial example generation methods mainly employ a two-stage optimization model: the first stage is anchor generation, and the second stage is adversarial example generation. The first stage generates anchors with semantic representation capabilities of the target category, capable of retrieving a large amount of target category data from the database. The second stage generates adversarial examples under the guidance of the anchors, narrowing the Hamming distance between the adversarial example hash code and the anchor, ensuring that the generated adversarial examples possess the target category information represented by the anchors. However, this two-stage model ignores the deeper correlation between the two stages of optimization. Experiments show that the anchor points generated in the first stage can significantly influence the optimization effect of adversarial examples in the second stage. That is, given a hash code as an anchor point, adversarial examples may not be well predicted as anchor points or pulled to the vicinity of the anchor point. Figure 1 As shown, this limits the ability of adversarial examples to retrieve target data. Therefore, how to generate anchors with strong semantic expressive power while also contributing to the optimization of adversarial examples is an urgent problem to be solved. Summary of the Invention

[0004] To address the problem that current adversarial robustness evaluation methods for deep hashing models cannot reliably and accurately assess model adversarial robustness, this invention discloses an adversarial robustness evaluation method and apparatus for deep hashing models. Specifically, because the Hamming distance between adversarial examples and anchor points cannot be effectively reduced in Hamming space, adversarial attack capabilities are limited, leading to unreliable evaluation results for model robustness. This is defined in this invention as the problem of poor "stability" of adversarial examples (or adversarial perturbations). Simultaneously, a general target adversarial perturbation generation method for deep hashing is proposed, fully exploring the robustness and security risks of the model. Specifically, the tolerance attribute of anchor points is defined, and an important point is proposed: there is a significant correlation between anchor point tolerance and adversarial perturbation stability. This point indicates that the anchor points generated in the first stage can greatly influence the optimization effect of the adversarial perturbation in the second stage; however, previous methods have ignored this correlation. Furthermore, utilizing this point, this invention proposes a new anchor point generation method that takes into account both the semantic expressive power and tolerance attribute of anchor points, thereby improving the stability and attack capability of general target adversarial perturbations generated under anchor point guidance, and thus improving the accuracy of adversarial robustness evaluation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] An adversarial robustness evaluation method for deep hashing models, the method comprising:

[0007] Calculate the tolerance of hash codes in the database;

[0008] Hash codes with a tolerance greater than the threshold are selected to form a candidate anchor set;

[0009] Anchor points are obtained based on the semantic expressive power of the hash codes in the candidate anchor point set;

[0010] Guided by anchor points, general-purpose adversarial perturbations are iteratively generated using training data;

[0011] The adversarial robustness of the deep hashing model is evaluated based on the attack effect of the general target adversarial perturbation on the test data.

[0012] Furthermore, the hash code h i =H(x) i )=sign(F(x i ;θ);where, sth i ∈{-1, 1} K x i Let H represent the i-th input sample, H represent the hash function, F(·θ) represent the hash model with parameter θ, and K represent the hash code h. i The length.

[0013] Further, the step of obtaining anchors based on the semantic expressive power of hash codes in the candidate anchor set includes:

[0014] The semantic expressive power of all hash codes in the candidate anchor set is evaluated using the t-mAP evaluation method;

[0015] The hash code with the strongest semantic expressive power is selected as the anchor point.

[0016] Furthermore, the step of iteratively generating a general target adversarial perturbation using training data under the guidance of the anchor point includes:

[0017] Based on training data and the general objective adversarial perturbation δ obtained in round t. t Generate the input data for the (t+1)th round of the deep hash model;

[0018] The input data of the t-th round is fed into the deep hash model;

[0019] The pixel gradient is calculated using backpropagation based on the loss function, and the general objective adversarial perturbation δ is iteratively updated using gradient descent. tThen, let t = t + 1, and return to the general objective adversarial perturbation δ obtained based on the training data and the t-th round. t The loss function is designed in conjunction with anchor points.

[0020] After reaching the preset number of updates, a general target counter-perturbation is obtained.

[0021] Furthermore, the loss function Where N represents the number of training data in the training dataset, and K represents the hash code length. The anchor point is represented by β, which is used to adjust the gradient magnitude during backpropagation during the update process. F represents the hash model, θ represents the parameters of the hash model, and x represents the setpoint. i Let represent the i-th training data, and δ represent the general objective adversarial perturbation in any round.

[0022] Furthermore, the general objective adversarial perturbation δ is iteratively updated using gradient descent. t ,include:

[0023] Obtaining Adversarial Perturbation Gradients and learning rate α;

[0024] Based on general target counter-disturbance δ t and Obtain the general target counter-perturbation δ t+1 .

[0025] An adversarial robustness evaluation device for deep hashing models, the device comprising:

[0026] An anchor point calculation module is used to calculate the tolerance of hash codes in the database; filter out hash codes with a tolerance greater than a threshold to form a candidate anchor point set; and obtain anchor points based on the semantic expressive power of the hash codes in the candidate anchor point set.

[0027] The perturbation generation module, guided by anchor points, iteratively generates general-purpose adversarial perturbations using training data;

[0028] The model testing module is used to evaluate the adversarial robustness of the deep hash model based on the attack effect of the general target adversarial perturbation on the test data.

[0029] An electronic device, characterized in that the electronic device comprises: a processor and a memory storing computer program instructions; the processor implements any of the methods described above when executing the computer program instructions.

[0030] A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement any of the methods described above.

[0031] Compared with existing methods, this invention proposes a novel anchor point generation method based on candidate anchor point evaluation. This method can generate anchor points with superior properties, namely, good semantic expressiveness and tolerance attributes. Anchor points guided by these superior anchor points exhibit strong adversarial perturbation attack capabilities, thus effectively improving the reliability of adversarial robustness evaluation methods. This anchor point generation method can not only improve the stability of general target adversarial perturbations but also be added to the image-dependent target adversarial example generation framework, demonstrating significant superiority. Therefore, the anchor point generation method proposed in this invention has general applicability to adversarial robustness evaluation tasks for deep hashing-based target attacks. Furthermore, in the two-stage optimization approach, the stability of adversarial perturbations greatly affects the attack capability of adversarial examples; therefore, improving stability is crucial. The stability evaluation index proposed in this invention can effectively assess the degree of stability of adversarial perturbations, thus inspiring more work to focus on improving the stability of adversarial perturbations. Moreover, the viewpoint proposed in this invention—that anchor point tolerance attributes and adversarial perturbation stability are significantly correlated—can inspire subsequent anchor point generation methods to consider the tolerance attributes of anchor points while improving their semantic expressiveness. Finally, extensive experiments show that the general target adversarial perturbation generation method for deep hashing proposed in this invention has good attack performance and can effectively improve the accuracy and reliability of adversarial robustness assessment. Attached Figure Description

[0032] Figure 1 Example diagram of a problem with poor stability.

[0033] Figure 2 A flowchart of an adversarial robustness evaluation method for deep hashing models.

[0034] Figure 3 Tolerance and stability analysis results.

[0035] Figure 4 Visualization results. Detailed Implementation

[0036] To make the above features and advantages of the present invention more apparent and understandable, the technical solution of the present invention will be further described below through specific embodiments.

[0037] The CAE-TUAA adversarial robustness evaluation method for deep hashing models proposed in this invention mainly consists of two stages. The first stage is the anchor generation stage. Firstly, to address the problem of poor stability of adversarial perturbations, this invention formally defines an evaluation method for the "stability" of adversarial perturbations, used to assess the stability of generated adversarial perturbations. Secondly, it proposes an "tolerance" attribute for anchors and formally defines its calculation method. Finally, this invention proposes an important viewpoint: there is a significant correlation between the tolerance of an anchor and the stability of adversarial perturbations generated under its guidance. Based on this viewpoint, this invention proposes an anchor generation method based on Candidate Anchor Code Evaluation (CAE), which considers both the semantic expressive power and tolerance attribute of anchors when generating them. The semantic expressive power of the anchor determines the upper limit of the attack capability, while the tolerance attribute determines the ability of the adversarial perturbation to approach this upper limit. The second stage is the generation of general-purpose adversarial perturbations. This involves designing a loss function, narrowing the anchor point, and adding the Hamming distance between the adversarial examples and the general-purpose adversarial perturbations. Then, an iterative update method based on gradient descent is used to update the adversarial perturbations, constraining them to a small perturbation range to ensure their visual invisibility. The specific update process is as follows: Figure 2 As shown.

[0038] For a well-trained deep hash model F(·θ), this invention will query sample x i The input is fed into the hash model to obtain the prediction result F(x). i Then, the sign(·) function binarizes the model prediction result into a hash code h. i Finally, the hash code is used to retrieve similar images from the database. The specific calculation formula is shown in (1):

[0039] h i =H(x) i )=sign(F(x i ;Θ))

[0040] sth i ∈{-1, 1} K (1)

[0041] Where H(·) represents the hash function, F(·;θ) represents the hash model, θ represents the model parameter, and K represents the hash code length.

[0042] (I) Anchor point generation stage:

[0043] Given an adversarial example test dataset It contains N t There are test data, where x i Let y represent the i-th data.i The corresponding label is indicated. In order to better express the problem that there is a large gap between the anchor point and the hash code of the adversarial sample in the current method, this invention refers to this phenomenon as poor stability of adversarial samples and defines the calculation method of stability as shown in formula (2).

[0044]

[0045] Where d H (·,·) represents the Hamming distance, h a Let IS(δ) represent the anchor point, and δ represent the generated general target adversarial perturbation. This formula means calculating the Hamming distance between the hash codes of all adversarial samples with the general target adversarial perturbation and the anchor point. It describes the degree to which the distance between the adversarial sample and the anchor point is shortened in Hamming space. The smaller IS(δ), the better the stability of δ; the larger IS(δ), the worse the stability of δ.

[0046] This invention, through analysis, finds a strong correlation between the stability of adversarial examples and the size of the anchor feature subspace. The larger the anchor feature subspace, the easier it is to obtain better optimization results for the general target adversarial perturbation generated in the second stage, i.e., it has better stability. However, the size of the anchor feature subspace cannot be accurately calculated. Therefore, in order to better evaluate the size of the anchor feature subspace, this invention defines the "Tolerance" attribute of the anchor and uses the number of hash codes predicted to be this anchor in the database to approximate the size of the anchor feature subspace. As shown in formula (3):

[0047]

[0048] Specifically, Tol(h) represents the tolerance function of hash code h, which calculates the tolerance of hash code h, and I(·) represents the indicator function, indicating whether h is within tolerance. i =h indicates the function value is 1, otherwise it is 0. M represents the amount of data in the database, H u This represents the set of all hash codes in the database that have a tolerance of at least 1.

[0049] To balance the semantic expressiveness and tolerance attributes of anchor points, this invention proposes an anchor point generation method (CAE) based on candidate anchor point evaluation. Specifically, this invention first calculates H... u The tolerance for all hash codes is calculated, and then all codes with a tolerance greater than the threshold T are selected. a The hash codes constitute the candidate anchor set. Then, t-mAP is used to evaluate the semantic expressive power of the anchors, and a candidate anchor set C is selected. a The hash code with the strongest semantic expressive power is used as the final calculated anchor h. a .

[0050] (II) General Target Countermeasure Disturbance Generation Phase:

[0051] The general target adversarial perturbation aims to add to any query data, retrieving images of as many target categories as possible. The hash code represented by the anchor point is representative of the target category, and the adversarial example predicted to have this anchor hash code retrieves the most target category data. Therefore, the optimization objective of the general target adversarial perturbation is to make all adversarial examples with this perturbation close to the anchor point in Hamming space. Since the Hamming distance between hash codes can be calculated from the dot product of the hash codes... Therefore, the loss function designed in this invention is as shown in formula (4):

[0052]

[0053] in The anchor point is generated in the first stage, F(x) i +δ;θ) is the hash model against the adversarial sample x i The prediction result of +δ is used, and β is used to adjust the gradient magnitude during backpropagation in the update process. When generating a general target adversarial perturbation, this invention calculates the pixel gradient based on backpropagation of the loss function and iteratively updates δ using gradient descent. Specifically, the gradient of the loss function with respect to the perturbation δ is first obtained. Then, the gradient is mapped to -1 or +1 using the sign(·) function, and finally, the adversarial perturbation is updated according to the learning rate α. The specific update method is shown in formula (5):

[0054]

[0055] in The loss function is an adversarial gradient, where α represents the learning rate and δ... t+1 It is an updated counter-perturbation.

[0056] In summary, this invention proposes the concept of adversarial example stability, identifies the stability problem prevalent in the two-stage optimization method for generating adversarial examples in deep hashing, and defines a method for evaluating stability.

[0057] This invention proposes an important viewpoint: the tolerance attribute of anchor points is strongly correlated with the stability of adversarial examples, and the correctness of this viewpoint has been proven through extensive experiments. Furthermore, to fully utilize this viewpoint to improve the stability of adversarial examples, this invention proposes a candidate anchor evaluation (CAE) method that takes into account both the tolerance attribute and semantic expressive power of anchor points, thereby comprehensively improving the attack capability against general-purpose adversarial perturbations and thus enhancing the accuracy of adversarial robustness assessment.

[0058] This invention proposes the CAE-TUAA method, which uses gradient descent iterative updates to generate general target adversarial perturbations. Extensive experiments have demonstrated that the CAE-TUAA method significantly improves the accuracy of adversarial robustness assessment due to its superior attack capabilities.

[0059] Experimental verification.

[0060] Datasets: We selected two commonly used datasets for training deep hashing models: FLICKR-25K and NUS-WIDE. FLICKR-25K contains 25,000 images belonging to 38 categories, with each image belonging to multiple categories, making it a classic multi-label dataset. Following the previous partitioning method, 1,700 images were sampled from all images as the query dataset, and the remaining images formed the retrieval database. 5,000 images were randomly sampled from the database for use as the training dataset. The NUS-WIDE dataset contains 269,648 images belonging to 81 categories. However, following the data partitioning method from classic deep hashing articles, 195,834 images from the 21 most commonly used categories were selected for training and testing. Following the classic partitioning method, 100 images from each of the 21 categories were selected as the query dataset, and the remaining data formed the database. Similarly, 500 images from each category were selected from the database to form the training dataset.

[0061] Target Models: Three classic deep hashing models, DPH, HashNet, and CSQ, are used as the target hashing models for evaluating adversarial robustness. These three models have been widely used in evaluating the adversarial robustness of deep hashing. Furthermore, the VGG-11 deep neural network structure is used by default as the feature extraction network for the deep hashing model.

[0062] Evaluation metrics: This study uses targeted mean average precision (t-mAP) and mean average precision (mAP) as evaluation metrics. Among them, t-mAP directly reflects the ability of adversarial samples with added target adversarial perturbations to retrieve target category data.

[0063] Experimental Results: To verify the effectiveness of this invention, we compared two anchor point generation methods: the PrototypeNet method in ProS-GAN and the component-voting scheme method in DHTA, which we abbreviate as PN and VT, respectively. We used the anchor points generated by these methods to guide the generation of general target adversarial perturbations. The experimental results are shown in Table 1. According to the results, we can find that (1) since the tolerance (Tol) of the anchor points generated by the PN method is the smallest and the tolerance of the anchor points generated by the VT method is the largest, the stability of the adversarial perturbation optimized under the guidance of the anchor points generated by the PN method is the worst (IS(δ)), while the stability of the adversarial sample optimized under the guidance of the anchor points generated by the VT method is the best. This phenomenon verifies the viewpoint of this invention that the anchor point tolerance and the stability of the adversarial sample are significantly correlated. (2) Since the semantic expressive power of the anchor (Anchor t-mAP) determines the upper limit of the adversarial perturbation attack capability, and the tolerance of the anchor determines the ability of the adversarial sample to approach this upper limit, the anchor generated by VT has high tolerance but low semantic expressive power; the anchor generated by the PN method has high anchor semantic expressive power and low tolerance. Therefore, both VT-TUAA and PN-TUAA methods achieved suboptimal results. (3) Since the CAE method takes into account both the semantic expressive power and tolerance of the anchor, the general target adversarial perturbation generated under its guidance achieved the best attack capability (Test t-mAP is the highest), thus significantly improving the accuracy of adversarial robustness assessment.

[0064] Tolerance and Stability Correlation Analysis Experiment: To directly verify the present invention's view that there is a significant correlation between anchor tolerance and adversarial example stability, the present invention conducted a correlation analysis experiment between the two. Specifically, the present invention uses the threshold T in CAE... a Values ​​were selected from {1, 20, 40, 60, 80, 100}, and then anchor points generated by the CAE method under different threshold values ​​were used to guide the generation of general-purpose adversarial perturbations. The changes in the stability of the adversarial perturbations were observed. Experimental results are as follows: Figure 3 As shown: (1) It can be seen from the first row of the figure that as the threshold increases, the anchor tolerance gradually increases, so the relationship between the threshold and stability can be approximately equivalent to the relationship between the anchor tolerance and stability. (2) As the anchor tolerance increases, the IS(δ) against perturbation gradually decreases, indicating that the stability gradually increases. This conclusion holds true for both ∈=0.1 and ∈=0.04, which verifies the correctness of the viewpoint of this invention. (3) It can be seen from the second row of the figure that the smaller the IS(δ), the better the stability, and the closer the attack capability of the adversarial sample is to the upper limit represented by the anchor. That is, the gap between Test t-mAP and Anchor t-mAP gradually decreases, which shows that stability reflects the ability to approach the upper limit of the adversarial attack capability.

[0065] Generality Analysis Experiment: To demonstrate the generality of the anchor point generation method proposed in this invention, we combined the CAE method with the currently proposed adversarial robustness evaluation method for deep hashing-based image-dependent adversarial sample generation, namely ProS-GAN and DHTA. Specifically, we replaced the PrototypeNet (PN) method in ProS-GAN with CAE and VT methods respectively to supervise the training of the GAN model, while keeping other experimental settings unchanged. In addition, we replaced the component-voting scheme (VT) method in DHTA with CAE and PN methods respectively, while keeping other experimental settings unchanged. The experimental results are shown in Table 2. It can be seen from the table that (1) the stability problem exists not only in the general target adversarial perturbation generation but also in the image-dependent target adversarial perturbation generation task, and the view of the correlation between tolerance and stability in this invention holds true in both ProS-GAN and DHTA methods. Specifically, it can be seen from the IS(δ) and Tol columns that the greater the tolerance, the better the stability, and thus the closer Testt-MAP is to Anchort-MAP. (2) The CAE method of the present invention takes into account both the semantic expressive power of anchor points and the tolerance of anchor points, and thus achieves the best results in both DHTA and ProS-GAN frameworks. That is, the test t-mAP is better than the other two methods by 1% to 3%.

[0066] Transferability Analysis: In order to verify the adversarial robustness evaluation capability (i.e., transferability) of the method of the present invention across models, the present invention conducted transferability experiments across hash bits, hash methods and network structures. The specific experimental setup and analysis are as follows.

[0067] (1) Cross-hash bit mobility: The source model and the target model for mobility evaluation in this invention use the DPH hash method and the VGG11 network structure, the difference being the hash bit. The experimental results are shown in Table 3. As can be seen from the table, the adversarial perturbation generated by the CAE-TUAA method of this invention achieves the best cross-hash bit mobility adversarial robustness evaluation effect.

[0068] (2) Transferability across hash methods: The source and target models have 16-bit network structures, both VGG11, differing only in the hash methods used: DPH, HashNet, and CSQ, respectively. Experimental results are shown in Table 4. The table demonstrates that the CAE-TUAA method of this invention achieves the best performance in evaluating cross-hash method transferability and robustness.

[0069] (3) Transferability across network structures: The source and target models use the DPH hash method with 16 hash bits, differing only in their network structures: AlexNet, ResNet18, and VGG11, respectively. Experimental results are shown in Table 5. The table shows that the transferability across network structures is significantly worse than the two aforementioned transfer methods. Therefore, it can be concluded that network structure is a major factor hindering transferability. Furthermore, the CAE-TUAA method of this invention still achieves the best transferability performance.

[0070] (4) Visualization Results: To visually demonstrate the effectiveness of the CAE-TUAA method of this invention in evaluating the robustness of a deep hash-based retrieval system, we provide visualization results. Specifically, we pre-set the retrieval target category to "Animal," then generated a general target adversarial perturbation map, added it to two randomly selected clean samples, and displayed their retrieval results as follows. Figure 4 As can be seen from the figure: (1) For clean, unperturbed samples, it can retrieve images that are semantically related to them. For example, for query data with the original labels "Flowers" and "Sky", it can correctly retrieve images of flowers and the sky. (2) However, after adding a generated perturbation invisible to the human eye to the clean samples, the model incorrectly returned images of the predefined animal categories. Therefore, the method of this invention makes a reliable judgment on the robustness of the model against adversarial attacks. That is, the current deep hash model faces serious robustness and security problems and is very vulnerable to attacks by adversarial samples. (3) In addition, it can be seen from the figure that after adding the same general target adversarial perturbation to two different query images, the same return result was obtained. This shows that the stability of the adversarial perturbation is strong, so that the two adversarial samples are predicted to be the same hash code, that is, the anchor point.

[0071]

[0072] Table 1. Adversarial robustness evaluation results on different depth hashing models.

[0073]

[0074]

[0075] Table 2 shows the results of the generality analysis using other methods.

[0076]

[0077] Table 3. Hashbit Migration

[0078]

[0079] Table 4. Portability across hash methods

[0080]

[0081] Table 5. Cross-network structure mobility

[0082] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Those skilled in the art can modify or make equivalent substitutions to the technical solutions of the present invention. The scope of protection of the present invention should be determined by the claims.

Claims

1. A method for evaluating the adversarial robustness of deep hashing models, characterized in that, The method includes: Calculate the tolerance of hash codes in the database Where Tol represents the tolerance function of hash code h, M represents the total number of hash codes in the database, and H u I(·) represents the set of all hash codes in the database with a tolerance of at least 1, and I(·) represents the indicator function, indicating that if h i If h = 1, then the indicator function value is 1; otherwise, the indicator function value is 0. Hash codes with a tolerance greater than the threshold are selected to form a candidate anchor set; Anchor points are obtained based on the semantic expressive power of the hash codes in the candidate anchor point set; wherein, obtaining anchor points based on the semantic expressive power of the hash codes in the candidate anchor point set includes: The semantic expressive power of all hash codes in the candidate anchor set is evaluated using the t-mAP evaluation method; Select the hash code with the strongest semantic expressive power as the anchor point; Guided by anchor points, general-purpose adversarial perturbations are iteratively generated using training data; The adversarial robustness of the deep hashing model is evaluated based on the attack effect of the general target adversarial perturbation on the test data.

2. The method as described in claim 1, characterized in that, The step of iteratively generating a general target adversarial perturbation using training data under the guidance of anchor points includes: Based on training data and the general objective adversarial perturbation δ obtained in round t. t Generate the input data for the (t+1)th round of the deep hash model; The input data of the t-th round is fed into the deep hash model; The pixel gradient is calculated using backpropagation based on the loss function, and the general objective adversarial perturbation δ is iteratively updated using gradient descent. t Then, let t = t + 1, and return to the general objective adversarial perturbation δ obtained based on the training data and the t-th round. t The loss function is designed in conjunction with anchor points. After reaching the preset number of updates, a general target counter-perturbation is obtained.

3. The method as described in claim 2, characterized in that, The loss function Where N represents the number of training data in the training dataset, and K represents the hash code length. The anchor point is represented by β, which is used to adjust the gradient magnitude during backpropagation during the update process. F represents the hash model, θ represents the parameters of the hash model, and x represents the setpoint. i Let F(x) represent the i-th training data, δ represent the general objective adversarial perturbation in any round, and F(x) represent the general objective adversarial perturbation in any round. i +δ;θ) represents the hash model F against the adversarial sample x i The prediction results for +δ.

4. The method as described in claim 2, characterized in that, The general objective adversarial perturbation δ is iteratively updated using gradient descent. t ,include: Obtaining Adversarial Perturbation Gradients and learning rate α; Based on general target counter-disturbance δ t and Obtain the general target counter-perturbation δ t+1 .

5. A device for evaluating the adversarial robustness of deep hashing models, characterized in that, The device includes: The anchor point calculation module is used to calculate the tolerance of hash codes in the database. Where Tol represents the tolerance function of hash code h, M represents the total number of hash codes in the database, and H u I(·) represents the set of all hash codes in the database with a tolerance of at least 1, and I(·) represents the indicator function, indicating that if h i =h, then the indicator function value is 1; otherwise, the indicator function value is 0; filter out hash codes with a tolerance greater than the threshold to form a candidate anchor set; obtain anchors based on the semantic expressive power of the hash codes in the candidate anchor set; wherein, obtaining anchors based on the semantic expressive power of the hash codes in the candidate anchor set includes: The semantic expressive power of all hash codes in the candidate anchor set is evaluated using the t-mAP evaluation method; Select the hash code with the strongest semantic expressive power as the anchor point; The perturbation generation module should be able to iteratively generate general-purpose adversarial perturbations using training data under the guidance of anchor points. The model testing module is used to evaluate the adversarial robustness of the deep hash model based on the attack effect of the general target adversarial perturbation on the test data.

6. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the method as described in any one of claims 1-4.

Citation Information

Patent Citations

  • Deep hash retrieval method and device and medium

    CN111522903A

  • Semi-supervised image retrieval method and device based on disturbance consistency self-integration

    CN112883216A