A robust neural network architecture search method for face verification

Abstract

The present invention discloses a robust neural network architecture search method for face verification, comprising the following steps: step 1, collecting face image data to form a sample set; step 2, initializing the neural network architecture population: selecting ArcFace as the basic model, and designing the search space based on this model; step 3, performing population iteration to form a new population; step 4, evaluating the architecture robustness; step 5, architecture selection: selecting a number of candidate architectures from the population in each cycle, and eliminating the candidate architectures with a disturbance amplitude greater than a preset threshold; step 6, outputting the optimal architecture: selecting the architecture with the lowest disturbance amplitude in the retained population and balanced performance in other indicators as the final output architecture, using the architecture for a face image recognition model, and using samples to train the face image recognition model. The present invention realizes the automated search of a robust neural network integrated architecture by introducing the concept of feature space disturbance quantization on the basis of an evolutionary algorithm.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and face recognition technology, and in particular, is a robust neural network architecture search method for face verification. The method aims to optimize and improve the face verification performance of neural networks in various complex and dynamic environments, and is widely used in security authentication, intelligent monitoring, identity authentication and other fields. Background Art

[0002] As an important means of identity authentication, face verification technology is widely used in various industries and scenarios, including but not limited to payment, bank cards, security monitoring, entry and exit management, social media, smart device unlocking, etc. These application scenarios require efficient, accurate and reliable identity authentication methods to ensure the authenticity of user identity and the security of the system. The core of face verification technology lies in face recognition, which confirms identity by extracting facial features from images and matching and classifying them. Early face recognition technology mainly relied on manually designed feature extraction methods and classifiers. These methods require manually designed feature extraction algorithms, such as SIFT, SURF, ORB, etc., to extract key points or features from face images, and then use classifiers such as support vector machines and decision trees to match and classify features. These methods require a lot of manual design work, poor adaptability to conditions such as lighting, expression, posture, etc., and limited generalization ability. In order to overcome these shortcomings, researchers began to explore face recognition technology based on deep neural networks. A neural network is a machine learning program or model that makes decisions in a similar way to the human brain, identifying phenomena, weighing pros and cons, and drawing conclusions by using processes that mimic the way biological neurons work together. Deep Neural Networks (DNN) is a neural network model composed of multiple hidden layers. Each hidden layer consists of multiple neurons, which transmit information and calculate through weights and activation functions. This technology uses deep learning models to automatically learn the complex feature representation of facial images without manually designing feature extraction methods, and has high accuracy, generalization ability and adaptability. For example, Convolutional Neural Network (CNN) can automatically learn the feature representation of facial images and extract discriminative features through a combination of multiple convolutional layers and pooling layers. These features can be used for identity recognition in face verification tasks, thereby improving the accuracy and robustness of recognition.

[0003] However, research has found that neural networks are significantly vulnerable to adversarial attacks. Common adversarial attacks add tiny perturbations that are imperceptible to the human eye to the input data, destroying the characteristic expression of the data during the forward propagation of the neural network, thereby inducing the model to output incorrect results. This vulnerability poses a major challenge to the safe application of artificial intelligence technology, especially in the fields of biometric recognition, autonomous driving, and intelligent computer-aided diagnosis, where any slight prediction error of the model may lead to catastrophic consequences.

[0004] Adversarial attack refers to the process of causing machine learning models to make wrong judgments by adding carefully designed, imperceptible small perturbations to the input data. These specially designed input data are called "adversarial samples". The principle of adversarial attack is based on the high nonlinearity of deep learning models and the irregularity of decision boundaries. Even for well-trained models, there are "holes" in the input space that attackers can exploit to mislead the model by adding perturbations. These perturbations are usually the difference between adversarial samples and original samples. They are designed to be so small that they are difficult for the human eye to detect, but they are enough to change the behavior of the model. Professor Goodfellow publicly demonstrated this in 2015. In recent years, a large number of adversarial attack methods for different task scenarios have been reported at home and abroad. These methods cover white-box attacks and black-box attacks, involving adversarial sample generation techniques based on model gradient information, inference scores, decision results, and geometric transformations, covering a variety of attack types such as non-directional attacks, directional attacks, reverse attacks, and prompt attacks, showing diverse attack capabilities.

[0005] For example, hackers can wear specially made "jamming" masks to deceive the face recognition system of mobile phones, thereby stealing the target user's money or secrets. In order to ensure the safe, reliable and controllable development of artificial intelligence, it is crucial to study the robust feature formation mechanism of neural networks and develop efficient adversarial defense measures to ensure that the model can maintain stable reasoning ability in complex adversarial environments.

[0006] Neural Architecture Search (NAS) technology was proposed by Google in 2016. It aims to realize automatic search for high-performance neural network architectures through algorithms. Traditional deep learning models usually require manual design of network structures, which requires a lot of professional knowledge and experience. The purpose of NAS is to reduce the manpower requirements in this process and find better and more efficient network architectures through automation. Figure 1As shown in the figure, the core of neural network architecture search lies in these three parts: search space, search strategy and performance evaluation. First, a search space containing various possible network structures is defined, and then a variety of search strategies such as genetic algorithms, reinforcement learning or gradient-based methods are used to effectively browse these structures. During the search process, performance evaluation is used to quantify the performance of each candidate structure on a specific task, and the search strategy is adjusted according to the evaluation results to further optimize the network structure. Finally, the optimal network structure is selected to achieve better task adaptability and performance level. In recent years, many popular NAS methods have emerged, such as DARTS, SPOS, etc. However, the performance of the architectures searched by these NAS methods is usually not good enough and the robustness is also poor. In order to solve the vulnerability of deep learning models under adversarial attacks and enhance the anti-interference ability in application scenarios with high security requirements such as face verification, Robust NAS (Robust Neural Architecture Search) technology needs to be focused on. The core idea of ​​Robust NAS is to focus on the robustness of the model on adversarial samples during the neural network architecture search process. To achieve this goal, the following key steps are usually included:

[0007] (1) Robustness metric: In the search process, in addition to considering the performance of the model on normal data, it is also necessary to evaluate the performance of the model on adversarial samples. This can be achieved by calculating the accuracy or loss of the model under adversarial attacks. The present invention quantifies the perturbation of the feature space of the neural network model by adversarial samples and uses a perturbation amplitude to evaluate the feature stability of the candidate architecture in the search space.

[0008] (2) Adversarial training: During the training phase, adversarial samples are used to train the model to improve the robustness of the model under adversarial attacks. This can be done by adding adversarial samples during training or using adversarial training methods such as PGD, which uses the projected gradient descent method when generating adversarial samples, making the perturbation more effective and powerful.

[0009] (3) Adjustment of search strategy: During the search process, the search strategy needs to be adjusted to better consider the robustness of the architecture. For example, the robustness metric can be used as a reward signal to guide the reinforcement learning algorithm to perform architecture search.

[0010] (4) Performance evaluation: In the evaluation stage, in addition to traditional accuracy indicators, it is also necessary to consider the performance of the model under adversarial attacks. This can be achieved by conducting adversarial attacks on the model and calculating the accuracy of the model after the attack. The present invention analyzes the impact mechanism of adversarial training on the evaluation model and designs a robustness evaluation indicator based on automatic adversarial attacks, and comprehensively conducts research on the robustness evaluation method of neural network architecture. Studies have shown that automatic adversarial attack technology can overcome the limitations of a single adversarial attack method and is a more comprehensive and accurate robustness evaluation indicator. Summary of the invention

[0011] The purpose of the present invention is to overcome the shortcomings of the prior art and to provide a robust neural network architecture search method for face verification that implements a robust neural network integrated architecture by introducing the concept of feature space perturbation quantization on the basis of an evolutionary algorithm.

[0012] The objective of the present invention is achieved through the following technical solution: a robust neural network architecture search method for face verification, comprising the following steps:

[0013] Step 1: Collect a dataset of face images, annotate each image, and form a labeled data sample set;

[0014] Step 2: Initialize the neural network architecture population: Select an existing high-performance face recognition neural network ArcFace as the basic model and design the search space based on this model; the specific process includes the following steps:

[0015] Step 21. Select the base model: Select ArcFace as the base model, download and load the pre-trained weights of the base model. ArcFace first applies a convolutional layer to extract features from the input data, then uses a normalization layer to stabilize and accelerate the training process; then applies an activation function to increase the nonlinear ability of the model; finally, uses a pooling layer to downsample the feature map;

[0016] Step 22: Design a specific search space based on ArcFace: Use the basic components of the ArcFace network as candidate operations in the search space to design a comprehensive search space; in this search space, the convolution layers are designed with different sizes of convolution kernels and different numbers of convolution layers; activation functions, pooling layers, and normalization layers include multiple different types;

[0017] Step 23, define a set of candidate architectures: use different combinations of convolutional layers, activation functions, pooling layers, and normalization layers to generate multiple candidate architectures; the candidate architecture uses the WideResNet network, which first extracts preliminary spatial features of the image through a convolutional layer, and then further processes the features through three residual blocks in sequence; the residual block passes the output to the normalization layer and activation function; and then inputs the average pooling layer for processing;

[0018] Step 24: Implement the search space: Use popular deep learning frameworks such as TensorFlow or PyTorch to implement the designed search space.

[0019] Step 25: Randomly select a certain number of architectures from the defined set of candidate network architectures as the initial population;

[0020] Step 3: Perform population iteration: randomly select two architectures from the initial population as parent architectures, then randomly select some components from the parent architectures to form child architectures, and finally add the generated child architectures to the population and merge them with the original population to form a new population;

[0021] Step 4: Evaluate the robustness of the architecture: For each candidate architecture, input the adversarial sample, extract the feature representation of the intermediate layer, and record the changes in the feature space; by calculating the distance between the original sample and the adversarial sample in the feature space, quantify the perturbation amplitude of the feature space to measure the feature anti-interference ability of the candidate architecture in the search space;

[0022] Step 5, architecture selection: adopt the elimination and retention strategy, select several candidate architectures from the population in each cycle, calculate the perturbation amplitude of each candidate architecture, and eliminate the candidate architectures whose perturbation amplitude is greater than the preset threshold;

[0023] Step 6: Output the optimal architecture: After the cycle ends, select the architecture with the lowest disturbance amplitude in the retained population and balanced performance in other indicators as the final output architecture, use this architecture in the face image recognition model, and use the samples to train the face image recognition model.

[0024] The step 4 specifically includes the following sub-steps:

[0025] Step 41, generation of adversarial samples: attack the original samples using FGSM, BIM, PGD, DeepFool, CW, PGDL2 and AutoAttack methods respectively to generate adversarial samples;

[0026] Step 42, feature representation extraction: extract feature representation from the middle layer of the candidate architecture, and record the changes of the adversarial sample in the feature space; the feature map is recorded as f(r,x′,θ(a)), where x represents the input adversarial sample, θ represents the neural network model parameters, and r represents the spatial coordinates of the feature map;

[0027] Step 43, calculate the feature perturbation amplitude: for each adversarial sample and each layer of feature map, calculate the difference between the feature map under the adversarial sample and the feature map under the original sample, and obtain the average perturbation amplitude of each layer of feature map by summing and averaging operations;

[0028] (1) Calculate the average value of the feature map: In each layer of the feature map, for each sample x′, calculate the average value τ of its feature map f(r, x′, θ(a)), where x′ represents the adversarial sample and θ(a) represents the neural network model parameters corresponding to architecture a;

[0029] (2) Calculate the part of the feature map of each layer that is higher than the average value and integrate it to obtain the perturbation capacity:

[0030] Volume(f,τ)=∫I[f(r)>τ](f(r)-τ)dr

[0031] τ is the average value of the feature map, I(·) represents the indicator function, and f(r) represents the feature map;

[0032] (3) Calculate the perturbation amplitude: For each sample and each layer of feature map, calculate the feature map perturbation amplitude of the original sample and the adversarial sample, take the average value, and calculate the difference in feature space perturbation capacity between the original sample and the adversarial sample as the perturbation amplitude:

[0033] Delta(f orig ,τ orig )=Volume(f orig ,τ orig )-Volume(f adv ,τ adv );

[0034] The subscript orig represents the original sample, and adv represents the adversarial sample; Delta represents the difference in perturbation capacity between the original sample and the adversarial sample in the feature space, and the lower the difference in perturbation capacity, the better.

[0035] Finally, the average of all the intermediate layer perturbation amplitudes is calculated as the final perturbation amplitude of the candidate architecture.

[0036] The beneficial effects of the present invention are as follows: the present invention realizes the automated search for robust neural network integrated architectures by introducing the concept of feature space perturbation quantization on the basis of evolutionary algorithms. Compared with traditional methods, the present invention significantly improves the robustness and design efficiency of the model under adversarial samples, and ensures the anti-interference ability of the searched architecture in practical applications. First, in terms of efficiency, the present invention can automatically search for robust architectures, which not only saves a lot of time and labor costs compared to manual design methods, but also can more systematically explore the design space and find the optimal architecture. Secondly, in terms of effectiveness, this automatic search method not only makes it more convenient to obtain models, but also significantly improves the robustness and anti-interference ability of the model under adversarial samples.

[0037] Through modular candidate sets and efficient evaluation indicators, the present invention provides a comprehensive and accurate robustness evaluation with wide applicability and significant technical advantages, improves the face verification performance of neural networks in various complex and dynamic environments, and provides an additional safeguard for social security. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the neural network architecture search technology framework;

[0039] Figure 2 It is the overall flow chart of the present invention;

[0040] Figure 3 is a flow chart of the population initialization in step 2 of the present invention;

[0041] Figure 4 It is a breeding flow chart of step 3 of the present invention;

[0042] Figure 5 is a flow chart of the feature space perturbation quantization method of step 4 of the present invention;

[0043] Figure 6 It is a line graph of characteristic graph changes under different counterattacks in step 4 of the present invention;

[0044] Figure 7 This is a flowchart of the architecture robustness evaluation of step 7 of the present invention. DETAILED DESCRIPTION

[0045] The present invention proposes a robust neural network architecture search method for face verification. By deeply analyzing the disturbance mechanism of adversarial attacks on the feature expression of neural networks, and combining evolutionary algorithms, an architecture with high robustness and other performance balance is selected. This method proposes a robustness evaluation method based on feature space perturbation quantification, which is used for the automated search of highly robust neural network architectures, so as to better resist the deception of adversarial attacks and improve the security of face verification systems. The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0046] like Figure 2 As shown, a robust neural network architecture search method for face verification of the present invention comprises the following steps:

[0047] Step 1: Collect a dataset of face images, annotate each image, and form a labeled data sample set; first collect a certain number of face images, the number depends on the specific image type, generally 10,000 images need to be collected, and then divided into 10 groups of images of different categories, each group of 1,000 images, annotate each image, and form a labeled dataset. The following is the specific method:

[0048] Step 11: Select a diverse sample of people: Make sure the selected sample covers different age groups including children, teenagers, adults, and the elderly, and the gender ratio should not be imbalanced. Recruit volunteers and ensure that they know and agree to the purpose of data collection.

[0049] Step 12. Use high-resolution camera equipment to shoot: Choose a high-resolution camera or smartphone camera to ensure the clarity and details of the captured image. Consider the real situation encountered in face recognition, shoot under different lighting conditions such as natural light, artificial light, cloudy days, direct sunlight, etc. to ensure the diversity of lighting in the dataset; shoot different expressions such as smile, frown, surprise, etc. and angles such as front, side, top view, top view, etc. to enhance the generalization ability of the model.

[0050] Step 13: Data labeling: Use labeling tools such as LabelImg to label the images. The labeling content includes the face area marked with a rectangular box, key points such as eyes, nose, mouth, and each volunteer's unique identity label. Each image contains at least one complete set of labeling information to ensure the accuracy of the data set.

[0051] Step 14: Dataset division: Divide the labeled image dataset into training set, validation set, and test set in a certain ratio of 70:20:10. Ensure that the number and distribution of samples in each subset are consistent to avoid data bias.

[0052] Step 2: Initialize the population of neural network architectures: Select an existing high-performance face recognition neural network ArcFace as the basic model, and design a search space based on this model; the search space includes a variety of convolutional layers, activation functions, pooling layers, normalization layers, and jump connections to form a large number of candidate network architectures for initializing the population of neural network architectures; the specific process is as follows: Figure 3 As shown, the following steps are included:

[0053] Step 21. Select the base model: Select ArcFace (AdditiveAngular Margin Loss) as the base model from the existing face recognition neural network, and download and load the pre-trained weights of the base model. ArcFace first applies a convolutional layer to extract features from the input data, then uses a normalization layer to stabilize and accelerate the training process; then applies an activation function to increase the nonlinear ability of the model; finally, uses a pooling layer to downsample the feature map, reducing the spatial dimension of the model while retaining the main features.

[0054] Step 22, design a specific search space based on ArcFace: Use the basic components of the ArcFace network as candidate operations for the search space and design a comprehensive search space; in this search space, different sizes of convolution kernels and different numbers of convolution layers are designed for the convolution layer; different sizes of convolution kernels are considered, such as 3*3, 5*5; different numbers of convolution layers, such as 3 layers, 5 layers, 7 layers, etc. The activation function includes a variety of different activation function options, such as ReLU, Leaky ReLU, Swish, etc.; in terms of pooling layers, different options such as maximum pooling, average pooling and Global Average Pooling are provided; the normalization layer includes Batch Normalization and Layer Normalization; ResNet-style jump connections are also considered; the selection of convolution kernel size and number, activation function, normalization layer and pooling layer, technical personnel in this field can also add other types according to actual needs, not limited to the above types.

[0055] Step 23, define a set of candidate architectures: Based on the designed search space, define a large-scale set of candidate network architectures; use different combinations of convolutional layers, activation functions, pooling layers, and normalization layers to generate multiple candidate architectures; the candidate architecture uses the WideResNet network, which first extracts the preliminary spatial features of the image through a convolutional layer, and then further processes the features through three residual blocks in sequence; the residual block (block) uses a skip connection mechanism to pass the input features directly to subsequent layers to better retain information and promote gradient propagation. The specific method is: block1 processes the output of the convolutional layer conv1 and passes its input to block2, block2 processes the output of block1 and passes its input to block3, block3 processes the output of block2 and passes the final output to the normalization layer (bn1) and activation function (relu); then it is input into the average pooling layer for processing.

[0056] Step 24. Implement the search space: Use popular deep learning frameworks such as TensorFlow or PyTorch to implement the designed search space. The functions of the PyTorch deep learning framework are needed to implement the designed search space. Convolutional layers, activation functions, normalization layers, and pooling layers can all be implemented through corresponding APIs or modules.

[0057] Step 25: Randomly select a certain number of architectures from the defined set of candidate network architectures as the initial population. Ensure that the initial population has sufficient diversity so that a better network architecture can be explored in the subsequent selection process;

[0058] Step 3, perform population iteration: Randomly select two architectures from the initial population as parent architectures to ensure the randomness and fairness of the selection process; then randomly select some components from the parent architecture to form a sub-architecture to ensure the effectiveness and diversity of the sub-architecture; finally, add the generated sub-architecture to the population and merge it with the original population to form a new population to ensure the number and diversity of the new population; in evolutionary algorithms (such as genetic algorithms), sub-architectures should be bred from parent architectures, and the parent architectures themselves are selected from the existing population. Therefore, the sub-architecture does not directly belong to the original population when it is generated. The detailed process is as follows Figure 4 As shown. For example, the parent architectures A and B contain different convolutional layers, activation functions, pooling layers, and normalization layers, respectively. The sub-architecture can select the convolutional layer and activation function from A, and the pooling layer and full normalization layer from B. The purpose of population iteration is to make the new population include not only the architectures in the original population, but also the newly generated sub-architectures. These sub-architectures may bring new features and improvements, ensuring the diversity of the population, so that there is a greater chance of finding a better architecture in subsequent iterations.

[0059] Step 4: Evaluate the robustness of the architecture: For each candidate architecture, input the adversarial sample, extract the feature representation of the intermediate layer, and record the changes in the feature space. By calculating the distance between the original sample and the adversarial sample in the feature space, quantify the degree of disturbance in the feature space, and use this disturbance amplitude to measure the anti-interference ability of the feature of the candidate architecture in the search space; the specific process is as follows: Figure 5 As shown, it includes the following sub-steps:

[0060] Step 41, Generation of adversarial samples: In order to evaluate the robustness of the neural network architecture, it is necessary to generate various types of adversarial samples. Descent), DeepFool, CW (Carlini & Wagner), PGDL2 and AutoAttack methods are used to attack the original samples and generate adversarial samples; the specific parameters of each attack method are set as follows: the perturbation size of the FGSM adversarial sample is 8 / 255; the perturbation size of the BIM adversarial sample is 8 / 255, the step size is 2 / 255, and the number of iterations is 10; the perturbation size of the PGD adversarial sample is 8 / 255, the step size is 2 / 255, and the number of iterations is 10; the number of iterations of the DeepFool adversarial sample is set to 50, and the overshoot factor is 0.02; the confidence level of the CW adversarial sample is set to 0, the number of iterations is 100, and the learning rate lr = 0.01; the perturbation size of the PGDL2 adversarial sample is set to 128 / 255, the step size is 15 / 255, and the number of iterations is 10; generating various types of adversarial samples can test the performance of the model in the face of various potential threats, thereby more comprehensively evaluating the robustness of the model. This can identify the vulnerabilities of the model. This can guide the improvement strategy of subsequent models and verify the universality of their defense capabilities. Samples generated by adversarial attacks on trained neural network models are adversarial samples. They are very similar to normal samples, but are carefully designed to make the model easily deceived or misclassified. These adversarial samples are used to evaluate the robustness of the model or improve the security of the model, rather than being used directly for training the model.

[0061] The difference between the feature maps of adversarial samples and clean samples under different attacks is different. The cal_delta_fm_volume function is used to calculate the delta mean of the difference between the feature map of the adversarial sample output by the model and the feature map of the original adversarial sample. The specific changes of the feature maps under different adversarial attacks are as follows: Figure 6 shown.

[0062] Delta mean is mainly used for analysis and evaluation. It can quantify the degree of feature map change. If delta mean is relatively large, it means that the model is more sensitive to this attack. You can use adversarial training methods (that is, use adversarial samples when training the model) to add adversarial samples with large delta mean to the training set to enhance the robustness of the model to the corresponding attack. If delta mean is small, it means that the model has a certain robustness to the attack method. It can also be used to analyze the relationship between delta mean and classification accuracy. The change in classification accuracy before and after the attack can help judge the robustness of the model. Generally, if the feature map of the model changes significantly (larger delta mean), the classification accuracy decreases, indicating that the model is less resistant to this attack. If there is indeed a strong correlation between high delta mean values ​​(increase in delta mean) and high attack success rate (decrease in classification accuracy), it means that the change in the feature map of the adversarial sample is an important reason for the model's misclassification. Through the above analysis (there may be different analyses depending on different situations), we can understand in which aspects the model is more vulnerable to attack, so as to improve the model more targeted. For example, the regularization term can be added to suppress the changes in the feature map and reduce the delta mean, thereby improving the model's resistance to adversarial attacks.

[0063] Step 42, feature representation extraction: Extract feature representation from the intermediate layers of the candidate architecture (referring to non-input and output layers, convolutional layers, pooling layers, and fully connected layers can all be used to extract and process features) to record the changes of adversarial samples in the feature space. The feature map is denoted as f(r,x′,θ(a)), where x represents the input adversarial sample, θ represents the neural network model parameters, and r represents the spatial coordinates of the feature map;

[0064] Step 43, calculate the feature perturbation amplitude: for each adversarial sample and each layer of feature map, calculate the difference between the feature map under the adversarial sample and the feature map under the original sample, and obtain the average perturbation amplitude of each layer of feature map by summing and averaging operations;

[0065] (1) Calculate the average value of the feature map: In each layer of the feature map, for each sample x′, calculate the average value τ of its feature map f(r, x′, θ(a)), where x′ represents the adversarial sample and θ(a) represents the neural network model parameters corresponding to architecture a;

[0066] (2) Calculate the part of the feature map of each layer that is higher than the average value and integrate it to obtain the perturbation capacity:

[0067] Volume(f,τ)=∫I[f(r)>τ](f(r)-τ)dr

[0068] τ is the average value of the feature map, I(·) represents the indicator function, and f(r) represents the feature map;

[0069] (3) Calculate the perturbation amplitude: For each sample and each layer of feature map, calculate the feature map perturbation amplitude of the original sample and the adversarial sample, take the average value, and calculate the difference in feature space perturbation capacity between the original sample and the adversarial sample as the perturbation amplitude:

[0070] Delta(f orig , τ orig )=Volume(f orig , τ orig )-Volume(f adv , τ adv );

[0071] The subscript orig represents the original sample (in step 1), and adv represents the adversarial sample (obtained by attacking the corresponding original sample). Delta represents the difference in perturbation capacity between the original sample and the adversarial sample in the feature space. Its main purpose is to quantify the perturbation of the feature space of the neural network model by the adversarial sample, and then evaluate the feature stability of the candidate architecture in the search space. The lower the difference in perturbation capacity, the better the robustness of the architecture.

[0072] Finally, the average of all the intermediate layer perturbation amplitudes is calculated as the final perturbation amplitude of the candidate architecture.

[0073] The pseudo code for calculating the quantitative index of feature space perturbation is as follows:

[0074]

[0075] Step 44, Visual Verification: By visualizing the perturbation changes in the feature space, the effectiveness of the designed quantitative indicators can be verified. By calculating the perturbation difference, a specific value can be obtained to indicate how much the feature map has changed before and after the adversarial attack. Through visualization, these changes can be observed more intuitively. For example, a feature map becomes more "flat" or "sharp" after the attack, and these visual changes may be related to the calculated perturbation difference value.

[0076] After generating adversarial samples, obtain their feature maps at different intermediate layers, store them in adv_fm_list, and define a color mapping function create_custom_cmap. Then use the plot_3d_featuremap function to draw a 3D visualization of the feature map. The function first creates a 3D coordinate axis, and then draws the surface of the feature map in 3D space through the ax.plot_surface method. The height (z-axis value) of each point in the feature map represents its activation value (these feature maps are two-dimensional matrices, representing the output activation values ​​of the intermediate layers of the neural network), and the color is colored according to the custom color map. This can better understand and explain the behavior and performance of neural networks in the face of adversarial attacks.

[0077] Step 5, architecture selection: adopt the elimination and retention strategy, select several candidate architectures from the population in each cycle, calculate the perturbation amplitude of each candidate architecture, and eliminate the candidate architectures with a perturbation amplitude greater than the preset threshold; retain the architectures with better performance to enter the next cycle, ensure that the quality of the architecture in the population continues to improve after each cycle, and at the same time retain diversity to avoid premature convergence. At the same time, methods such as crowding distance can also be used to ensure the diversity of individuals in the population and prevent falling into a local optimal solution.

[0078] Step 6: Output the optimal architecture: After several rounds of cycles, the quality of the architecture in the population will become higher and higher; after the cycle is over, the architecture with the lowest disturbance amplitude and balanced performance in other indicators in the remaining population is selected as the final output architecture, and the architecture is used in the face image recognition model, and the face image recognition model is trained using samples. The detailed process is as follows:

[0079] Step 61, set convergence criteria: Set the convergence criteria of the evolutionary algorithm, such as the maximum evolutionary generations or the disturbance amplitude index of individuals in the population no longer significantly improves. In this embodiment, the number of iterations is set to 100. After 100 cycles, all architectures in the population are comprehensively evaluated and ranked according to indicators such as disturbance amplitude, classification accuracy, and computational complexity.

[0080] Step 62, comprehensive index evaluation: Neural network architecture search uses the multi-objective evolutionary algorithm NSGA-II to integrate multiple indicators, which can be regarded as a multi-objective optimization problem. NSGA-II has performed well in many non-convex optimization problems. By calculating the crowding distance between individuals, it effectively handles multiple search targets with different value ranges, thereby improving search efficiency and result quality. For specific methods of selecting architectures, please refer to the paper: Deb K, Pratap A, Agarwal S, et al. A fast and elitist multiobjective genetic algorithm: NSGA-II [J]. IEEE transactions one evolutionary computation, 2002, 6 (2): 182-197.

[0081] Step 63: Select the final architecture: Select the architecture with the highest perturbation amplitude and balanced performance on other indicators as the final output architecture, which can ensure that the selected architecture is robust and efficient in different application scenarios.

[0082] Step 7: Evaluate model performance: The detailed process of model performance evaluation and evaluation index design is as follows: Figure 7 As shown, the specific method is:

[0083] Step 71, model performance evaluation: First, perform standard training and adversarial training on the final output architecture to generate two versions of the model. Standard training uses dataset D1 (such as CIFAR-10, MNIST), and the training parameters are set to learning rate: 0.01, batch size: 64, number of training rounds: 100, and record the loss value and classification accuracy during the training process. Adversarial training also uses dataset D1, and the adversarial sample generation method selects FGSM (Fast Gradient Sign Method). The training parameters are set to learning rate: 0.01, batch size: 64, number of training rounds: 100, FGSM perturbation size: 0.3, and the loss value and classification accuracy during the training process are also recorded. Then use a test dataset D2 that is the same as D1 but not included in the training set to evaluate the classification accuracy of the standard training model and the adversarial training model, respectively, and analyze the effect of adversarial training on improving the robustness of the model.

[0084] Step 72: Design evaluation indicators:

[0085] (1) Adversarial attack: Use the standard training model and the adversarial training model to generate adversarial samples using various attack methods. FGSM attack: Set the perturbation size to 0.3 to generate adversarial sample set A1. PGD attack: Set the number of iterations to 40 and the step size of each iteration to 0.01 to generate adversarial sample set A2. CW attack: Set the attack parameter to 10 to generate adversarial sample set A3. Record the time and resource consumption of generating adversarial samples.

[0086] (2) Adversarial sample testing: Use the generated adversarial sample sets (A1, A2, A3) to test the robustness of the model. Record the classification accuracy and adversarial attack success rate of the model under each adversarial sample. Calculate the perturbation degree of the adversarial sample and record it.

[0087] (3) Evaluation index analysis: Comprehensively analyze the classification accuracy, success rate of adversarial attacks, and the degree of perturbation of adversarial samples under different adversarial attacks. Design robustness evaluation indicators based on these evaluation results, such as Average Success Rate (ASR), Average Classification Accuracy (ACR), and Average Perturbation Degree (APD). Through these evaluation indicators, the comprehensive robustness and practicality of the model are evaluated to ensure that the model has high resistance to different types of adversarial attacks while maintaining high classification accuracy and low computational complexity.

[0088] Through comparison, it is found that the proposed method is superior to the traditional method in three indicators: average success rate (Average Success Rate, ASR), average classification accuracy (Average Classification Accuracy, ACR), and average perturbation degree (Average Perturbation Degree, APD).

[0089] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A robust neural network architecture search method for face verification, characterized in that: The steps include: Step 1: Collect a dataset of face images, annotate each image, and form a labeled data sample set; Step 2: Initialize the neural network architecture population: Select ArcFace as the base model and design the search space based on this model; the specific process includes the following steps: Step 21. Select the base model: Select ArcFace as the base model, download and load the pre-trained weights of the base model. ArcFace first applies a convolutional layer to extract features from the input data, then uses a normalization layer to stabilize and accelerate the training process; then applies an activation function to increase the nonlinear ability of the model; finally, uses a pooling layer to downsample the feature map; Step 22: Design a search space based on ArcFace: Use the basic components of the ArcFace network as candidate operations in the search space to design a comprehensive search space; in this search space, convolutional layers are designed with different sizes of convolutional kernels and different numbers of convolutional layers; activation functions, pooling layers, and normalization layers include multiple different types; Step 23, define a set of candidate architectures: use different combinations of convolutional layers, activation functions, pooling layers, and normalization layers to generate multiple candidate architectures; the candidate architecture uses the WideResNet network, which first extracts preliminary spatial features of the image through a convolutional layer, and then further processes the features through three residual blocks in sequence; the residual block passes the output to the normalization layer and activation function; and then inputs the average pooling layer for processing; Step 24: Implement the search space: Use the TensorFlow or PyTorch deep learning framework to implement the designed search space. Step 25: Randomly select multiple architectures from the defined set of candidate network architectures as the initial population; Step 3: Perform population iteration: randomly select two architectures from the initial population as parent architectures, then randomly select some components from the parent architectures to form child architectures, and finally add the generated child architectures to the population and merge them with the original population to form a new population; Step 4: Evaluate the robustness of the architecture: For each candidate architecture, input the adversarial sample, extract the feature representation of the intermediate layer, and record the changes in the feature space; by calculating the distance between the original sample and the adversarial sample in the feature space, quantify the perturbation amplitude of the feature space to measure the feature anti-interference ability of the candidate architecture in the search space; specifically, it includes the following sub-steps: Step 41, generation of adversarial samples: attack the original samples using FGSM, BIM, PGD, DeepFool, CW, PGDL2 and AutoAttack methods respectively to generate adversarial samples; Step 42, feature representation extraction: extract feature representation from the middle layer of the candidate architecture, and record the changes of the adversarial sample in the feature space; the feature map is recorded as f(r,x′,θ(a)), where x represents the input adversarial sample, θ represents the neural network model parameters, and r represents the spatial coordinates of the feature map; Step 43, calculate the feature perturbation amplitude: for each adversarial sample and each layer of feature map, calculate the difference between the feature map under the adversarial sample and the feature map under the original sample, and obtain the average perturbation amplitude of each layer of feature map by summing and averaging operations; (1) Calculate the average value of the feature map: In each layer of the feature map, for each sample x′, calculate the average value τ of its feature map f(r, x′, θ(a)), where x′ represents the adversarial sample and θ(a) represents the neural network model parameters corresponding to architecture a; (2) Calculate the part of the feature map of each layer that is higher than the average value and integrate it to obtain the perturbation capacity: Volume(f,τ)=∫I[f(r)>τ](f(r)-τ)dr τ is the average value of the feature map, I(·) represents the indicator function, and f(r) represents the feature map; (3) Calculate the perturbation amplitude: For each sample and each layer of feature map, calculate the perturbation amplitude of the feature map of the original sample and the adversarial sample, take the average value, and calculate the difference in the feature space perturbation capacity of the original sample and the adversarial sample as the perturbation amplitude: Delta(f orig ,t orig )=Volume(f orig ,t orig )-Volume(f adv ,t adv ); The subscript orig represents the original sample, and adv represents the adversarial sample; Delta represents the difference in perturbation capacity between the original sample and the adversarial sample in the feature space, and the lower the difference in perturbation capacity, the better; Finally, the average of all the middle layer perturbation amplitudes is calculated as the final perturbation amplitude of the candidate architecture; Step 5, architecture selection: adopt the elimination and retention strategy, select several candidate architectures from the population in each cycle, calculate the perturbation amplitude of each candidate architecture, and eliminate the candidate architectures whose perturbation amplitude is greater than the preset threshold; Step 6. Output the optimal architecture: After the cycle ends, select the architecture with the lowest disturbance amplitude in the retained population and balanced performance in classification accuracy and computational complexity as the final output architecture, use this architecture for the face image recognition model, and use the samples to train the face image recognition model.

Images