Image classification model anti-robustness tuning method and related device

Abstract

The invention belongs to the field of image processing, and discloses an image classification model anti-robustness tuning method and a related device, and the method comprises the steps: obtaining a source image sample and a plurality of guide image samples which are different from the source image sample in category; according to the source image sample and the plurality of guide image samples, constructing a feature search space by using a non-uniform feature level convex interpolation method with independent combination coefficients of different feature dimensions; taking maximization of classification loss of the image classification model as an optimization target, searching in a feature search space to obtain a key confrontation image sample; and constructing an enhanced adversarial data set based on the key adversarial image sample, and carrying out adversarial fine tuning training on the image classification model based on the enhanced adversarial data set to obtain an optimized image classification model. While the recognition accuracy is kept, the safety performance is effectively improved, and the problems that the improvement of the robustness of the model is limited and the image classification accuracy is reduced due to the limitation of the search space and the single feature processing in the existing adversarial training method are solved.

Description

Technical Field

[0001] This invention belongs to the field of image processing and relates to a method and related apparatus for optimizing the adversarial robustness of image classification models. Background Technology

[0002] Deep learning technology has made groundbreaking progress in fields such as image recognition, autonomous driving, and biometrics, becoming a core driving force for artificial intelligence applications. However, due to the high nonlinearity and complexity of deep neural networks, image classification models based on deep neural networks exhibit significant vulnerability to adversarial examples. This means that even minute perturbations imperceptible to the human eye can induce image classification models to output incorrect results with high confidence, posing a serious challenge to applications in security-sensitive fields. To defend against such attacks, adversarial training is considered one of the most effective defense mechanisms. This method introduces adversarial examples generated by attack algorithms during training, continuously refining model parameters through a game of "attack and defense," attempting to push the model's decision boundary away from the data points, thereby increasing the model's tolerance to perturbations. Existing mainstream solutions typically rely on gradient optimization or specific regularization constraints to find perturbations that maximize model loss within a small neighborhood of the input image, thus simulating potential attacks and strengthening the model's defense.

[0003] While existing methods have improved defense capabilities to some extent, significant technical shortcomings remain, limiting further improvements in the robustness of image classification models. Firstly, existing adversarial example generation methods typically restrict the search space to a very small area around the original samples. Within the norm sphere, this overly conservative constraint fails to adaptively detect the widespread "fuzzy regions" near the decision boundary, resulting in an overly limited distribution of generated adversarial examples that cannot fully expose the boundary defects of the image classification model. Secondly, when exploring potential adversarial regions, existing methods primarily employ sample-level linear interpolation strategies, applying the same mixing ratio to all feature dimensions of the input image. This approach severely ignores the differences in the contribution of different feature dimensions to the image classification model's decision, failing to flexibly construct effective paths across complex nonlinear decision boundaries. Furthermore, pure optimization-based attack methods often lack interpretable geometric constraints; the generated adversarial examples resemble "blind spots" in the feature space, lacking geometric guidance from other class samples. This makes it difficult for the image classification model to learn truly robust classification boundaries when fine-tuning using these samples, often resulting in a significant decrease in the accuracy of the original image sample recognition while improving robustness. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and related apparatus for optimizing the robustness of image classification models against adversarial forces.

[0005] To achieve the above objectives, the present invention employs the following technical solution: Compared with the prior art, the present invention has the following beneficial effects: This invention presents an image classification model adversarial robustness tuning method. By introducing a non-uniform feature-level convex interpolation mechanism between source image samples and guide image samples of disjoint categories, a flexible feature-level convex multicellular feature search space is constructed. This feature search space abandons the single-ratio limitation of traditional sample-level interpolation, allowing different feature dimensions to have independent mixing coefficients. This enables adaptive detection and localization of complex fuzzy regions and blind spots near the model's decision boundary. Therefore, key adversarial image samples generated based on this feature search space can provide accurate gradient information about decision boundary defects during model fine-tuning. Utilizing key adversarial image samples and their neighborhood distribution for adversarial fine-tuning not only significantly enhances the model's defense against various powerful adversarial attacks, but also avoids excessive distortion of the decision boundary due to the introduction of correct classification constraints from guide image samples. Furthermore, while maintaining high recognition accuracy for the original image samples, it effectively improves the model's overall security performance, solving the problem of limited robustness and decreased image classification accuracy caused by the limitations of the search space and the single feature processing in existing adversarial training methods. Attached Figure Description

[0006] Figure 1 This is a flowchart of the image classification model adversarial robustness tuning method according to an embodiment of the present invention.

[0007] Figure 2 This is a schematic diagram illustrating the principle of feature-level key adversarial image sample generation in an embodiment of the present invention.

[0008] Figure 3 This is a schematic diagram illustrating the adversarial fine-tuning principle based on neighborhood manifold sampling in an embodiment of the present invention.

[0009] Figure 4 This is a block diagram of the image classification model adversarial robustness tuning system according to an embodiment of the present invention. Detailed Implementation

[0010] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0011] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0012] The present invention will now be described in further detail with reference to the accompanying drawings: See Figure 1 In one embodiment of the present invention, an adversarial robustness tuning method for image classification models is provided, which helps to more reliably optimize the adversarial robustness of image classification models based on deep learning networks.

[0013] Specifically, the image classification model adversarial robustness tuning method of the present invention includes the following steps: S1: Obtain source image samples and several guide image samples that are of a different category from the source image samples.

[0014] S2: Based on the source image samples and several guide image samples, construct the feature search space using the non-uniform feature-level convex interpolation method with independent combination coefficients of different feature dimensions.

[0015] S3: With the goal of maximizing the classification loss of the image classification model, a search is performed in the feature search space to obtain key adversarial image samples.

[0016] S4: Construct an enhanced adversarial dataset based on key adversarial image samples, and perform adversarial fine-tuning training on the image classification model based on the enhanced adversarial dataset to obtain an optimized image classification model.

[0017] This invention presents an image classification model adversarial robustness tuning method. By introducing a non-uniform feature-level convex interpolation mechanism between source image samples and guide image samples of disjoint categories, a flexible feature-level convex multicellular feature search space is constructed. This feature search space abandons the single-ratio limitation of traditional sample-level interpolation, allowing different feature dimensions to have independent mixing coefficients. This enables adaptive detection and localization of complex fuzzy regions and blind spots near the model's decision boundary. Therefore, key adversarial image samples generated based on this feature search space can provide accurate gradient information about decision boundary defects during model fine-tuning. Utilizing key adversarial image samples and their neighborhood distribution for adversarial fine-tuning not only significantly enhances the model's defense against various powerful adversarial attacks, but also avoids excessive distortion of the decision boundary due to the introduction of correct classification constraints from guide image samples. Furthermore, while maintaining high recognition accuracy for the original image samples, it effectively improves the model's overall security performance, solving the problem of limited robustness and decreased image classification accuracy caused by the limitations of the search space and the single feature processing in existing adversarial training methods.

[0018] In one possible implementation, the guidance image sample has a model prediction confidence score greater than a preset prediction confidence score threshold based on the image classification model; the number of guidance image samples is not less than 4.

[0019] Explanatory, for target image classification models that need to improve robustness in adversarial environments, denoted as .in The parameters representing the model (including weights and biases). This model... Typically, it is a trained classifier that can classify the input image space. Mapping to tag space One image is selected from the original image dataset (e.g., training set or validation set) as the source image sample, denoted as . The corresponding tag category is Then the model prediction result This source image sample will serve as the reference point for subsequent feature space interpolation. Secondly, in order to construct a convex polytope that can span the decision boundary in the feature space, a set of guiding image samples, denoted as set, needs to be selected. , The number of image samples to guide the selection.

[0020] The selection of interpretive, guiding image samples should follow two key principles: Principle 1: Classes do not overlap. Each guiding image sample... Tag categories Must be compared with source image samples Tag categories They are all different. Criterion 2: Model prediction confidence. Guiding image samples. It should also be able to be modeled Correctly classified, high-confidence image samples. Selecting "correctly classified" image samples means they are located near the distribution center or typical region of their respective categories. Using these typical image samples as the basis of the feature space allows for the construction of more representative search directions, thus avoiding blind searches in irrelevant, noisy spaces.

[0021] Interpretively, the number of guide image samples determines the complexity and coverage of the subsequently constructed convex polytope. The value of is positively correlated with the diversity of the generated adversarial image samples. A larger A value of 4 indicates that the constructed convex multicell has more vertices, thus enabling the detection of more complex decision boundary shapes. It is generally recommended that the number of guiding image samples be no less than 4; for example, on the CIFAR-10 dataset, by setting... The final image classification model achieved a superior robustness gain. Through the above steps, source image samples were obtained. and a set of guide image samples This set of image samples defines a latent geometric region in the feature space that spans the area across which the model identifies the image as a category. Other categories The boundary line provides data preparation for accurately locating "blind spots" on the boundary using non-uniform feature-level convex interpolation.

[0022] In one possible implementation, the step of constructing the feature search space based on source image samples and several guide image samples using a non-uniform feature-level convex interpolation method with independent combination coefficients of different feature dimensions includes: Set combination coefficients for the source image samples and each guide image sample with dimensions consistent with the image dimensions.

[0023] The feature search space is constructed using the following formula. :

[0024] ,

[0025] in, for and each The combined coefficient vector, The combination coefficients of the source image samples, For element-wise multiplication, For source image samples, To guide the number of image samples, For the first The combination coefficients of the guide image samples For the first A sample of guiding images, for The Dimension value, for The Dimension value.

[0026] Explaining the core of this step, it involves constructing a convex multicell in the feature space through non-uniform feature-level convex interpolation. Specifically, to overcome the shortcomings of traditional sample-level interpolation, which uses a uniform mixing ratio for all feature dimensions and thus ignores the differences in the importance of different features to the model's decision, this step defines a set of combination coefficients consistent with the dimensions of the image samples. These are the combination coefficients corresponding to the source image samples. For the first The combination coefficients corresponding to each guide image sample, the shape of which is the same as that of the source image sample. They have the same shape.

[0027] Based on the above combination coefficients, this invention utilizes non-uniform feature-level convex interpolation to construct a feature search space. This construction process is achieved through element-wise weighted combination of source image samples and each guide image sample along the feature dimension, and its mathematical expression is shown in the following formula:

[0028] This means the feature search space Each pixel or feature value in the image is derived from the source image samples. and guide image samples The feature values ​​at corresponding positions are obtained by independently weighted summation. This feature-wise processing method gives the interpolation process a great deal of freedom, allowing the algorithm to assign different mixing weights to different regions of the image (such as foreground objects, background, edge textures, etc.), thereby simulating more complex and aggressive feature perturbations. Meanwhile, to ensure that the generated synthetic samples always remain within the geometric space enclosed by the source and guide samples, and do not diverge uncontrollably, this step involves combining the coefficient vectors. Strict convex combination constraints were imposed. The specific constraints are shown in the following equation: ,

[0029] in, This represents the index of any feature dimension of the input image (e.g., a pixel location and channel in the image). The constraint requires that for each dimension in the feature space... The coefficients of the source image samples in this dimension The coefficients of all guided image samples in this dimension The sum must be strictly equal to 1.

[0030] Through this constraint, all possible synthetic samples form a convex polytope in the feature search space. The vertices of this convex polytope are the source image samples. and various image guidance samples Since the guiding sample image is selected from categories that do not intersect with the source image sample, based on the manifold distribution characteristics in high-dimensional space, this convex polytope necessarily spans the fuzzy region of the model decision boundary. Compared to traditional methods that only search within a small norm sphere or along a line connecting two points around the source image sample, the convex polytope search space constructed in this invention covers a wider range of low-density regions containing potential adversarial examples, and can adaptively adjust the search path according to the importance of different features, providing a mathematical and geometric basis for the subsequent accurate localization of key adversarial examples.

[0031] In one possible implementation, the step of searching in the feature search space to obtain key adversarial image samples with the optimization objective of maximizing the classification loss of the image classification model includes: The objective function is constructed to maximize the classification loss of the image classification model. :

[0032] in, To guide the weight vector, For balance coefficient, for The guiding weight, For classification loss function, for and Classification loss, for Predicted category based on image classification model for The tag category.

[0033] Solve the objective function The optimal combination coefficient vector and guiding weight vector are obtained; based on the optimal combination coefficient vector and guiding weight vector, the source image samples and each guiding image sample are fused to obtain the key adversarial image samples.

[0034] Explaining this, this step first establishes a constrained optimization objective function aimed at maximizing the model's classification error rate. To comprehensively measure the deviation or attack effect of the synthetic image samples from the guide image samples of different categories, a learnable guide weight vector is introduced. And it satisfies the normalization constraint. The objective function takes the form shown below:

[0035] in, This indicates the difference between the synthesized image sample and the source image sample. Norm distance is used to constrain the perturbation amplitude, ensuring that the generated image samples are not excessively distorted visually. This is the balance coefficient; The classification loss function measures the model. Predicting the label category of the synthetic sample as the guide image sample The probability of misclassification. The physical meaning of the objective function is: to find an optimal combination of coefficient vectors and guiding weight vectors, so that the generated image samples are as close as possible to the source image samples, while being misclassified by the model into one or more target label categories highlighted by the guiding weights.

[0036] In one possible implementation, the objective function is solved. Obtaining the optimal combined coefficient vector and guiding weight vector includes: initializing the combined coefficient vector and guiding weight vector; iteratively executing the first fixed update step and the second fixed update step until the preset number of iterations or the objective function is reached. Convergence is achieved by using the current combined coefficient vector and guiding weight vector as the optimal combined coefficient vector and guiding weight vector. The first fixed update step includes: fixing the combined coefficient vector, and using the classification loss of each guiding image sample... The function normalizes and updates the guidance weights of each guidance image sample; the second fixed update step includes: fixing the guidance weight vector, updating the combined coefficient vector using the box constraint optimization algorithm, and normalizing the updated combined coefficient vector.

[0037] Explanatory, in order to effectively solve the above problem involving dual variables ( and For optimization problems with complex constraints, this step employs an alternating iterative optimization strategy. The specific execution process of this strategy is as follows: First, for... and Perform initialization. Typically initialized to make Approximates the state of the source image sample. Initialize to a uniform or random distribution. Then, enter an iterative loop, alternating between the following two sub-steps: First, fix the combination coefficients and update the guiding weights. Calculate the current synthetic sample for each... The classification loss value, using The function normalizes these loss values ​​to obtain new guiding weights. This step is for direction selection; it automatically identifies which direction of the guiding image sample is most likely to cause the model to make an error and assigns a larger guiding weight to that direction, thus guiding subsequent searches to focus on that weak point. The second step fixes the guiding weights and updates the combination coefficients. Guided by the weighted loss function with the current guiding weights, the box-constrained optimization algorithm is used to... Update. During the update process, an update needs to be performed after each iteration. Perform projection or normalization operations to strictly satisfy the defined convex combination constraints (i.e., This step is for location search, which involves finely adjusting the mixing ratio of each feature dimension along the selected attack direction to make the sample approach the fuzzy region of the decision boundary.

[0038] By alternately optimizing the combination coefficients and guiding weights, the method comprehensively considers both the perturbation strength and the directional orientation of the attack. Based on this dual-variable alternating optimization mechanism, the method of this invention can not only find the strongest attack image sample in a given guiding direction, but also automatically filter out the guiding category direction that poses the greatest threat to the current source image sample. Thus, even in black-box or gray-box scenarios where the specific boundary shape of the model is unknown, it can still adaptively discover the feature combination path that is most likely to cause the model to misjudge, enhancing the versatility and effectiveness of the generation method.

[0039] Through the aforementioned alternating optimization process until convergence or the preset number of iterations is reached, the optimal combined coefficient vector and guiding weight vector are finally obtained. These are then substituted into the interpolation formula to generate key adversarial image samples. These key adversarial image samples not only successfully cross the model's decision boundary but also reside in the blurred region at the boundary of different classes, carrying rich information about the model's robustness blind spots, providing invaluable data support for subsequent model fine-tuning.

[0040] In one possible implementation, constructing an enhanced adversarial dataset based on key adversarial image samples includes: constructing a neighborhood distribution of key adversarial image samples using a Gaussian mixture model, sampling in the neighborhood distribution to obtain several extended adversarial image samples, setting the label category of the extended adversarial image samples to the label category of the corresponding source image samples, and adding them to the training set of the image classification model to obtain the enhanced adversarial dataset.

[0041] Interpretively, although the actual distribution of adversarial image samples is unknown and highly complex, based on the manifold assumption, adversarial image samples tend to cluster near certain specific low-density manifolds. Since the generated key adversarial image sample has been located at a key point on this manifold, it is reasonable to assume that image samples in its neighborhood are also highly likely to be adversarial or belong to the transition region of the decision boundary.

[0042] Based on this, the present invention uses a Gaussian mixture model to approximate this local neighborhood distribution. Specifically, an adversarial distribution is defined. To generate the joint distribution of adversarial image samples and their labels, this step uses a Gaussian mixture distribution centered on key adversarial image samples to estimate this adversarial distribution. For the ... Corresponding key adversarial image samples generated from each source image sample Its neighborhood distribution probability density function can be expressed by the following formula:

[0043] in, The representation has a mean of zero and a covariance matrix of... The multivariate normal distribution, It is a standard deviation parameter that controls the neighborhood range or sampling diversity. It is the identity matrix; Represents relative to key adversarial image samples The offset; It was Dirac A function used to indicate the expanded image samples obtained from sampling. Inherit its source image samples Original authentic label category .

[0044] Based on the established approximate distribution model, this step performs a random sampling operation. Specifically, for each key adversarial image sample... From the corresponding distribution Several sample points are randomly selected from the dataset. These newly generated image samples are called extended adversarial image samples. In this way, the original single adversarial image sample is expanded into an extended adversarial image sample set containing rich variations. These extended adversarial image samples are densely distributed in the feature space near the manifold of the model decision boundary. They retain the core attack features of the key adversarial image samples while introducing diversity brought about by random perturbations. This allows for a more comprehensive depiction of the model's weak points in the current region, providing a sufficient and high-quality data foundation for subsequent fine-tuning to fill these blind spots.

[0045] In one possible implementation, the image classification model adversarial robustness tuning method of the present invention does not employ retraining from scratch, but rather performs adversarial fine-tuning based on pre-trained model parameters. This saves computational resources while preserving the feature extraction capabilities already learned by the model on the original task. The fine-tuning process employs a hybrid data training strategy, i.e., a hybrid training batch is constructed in each training iteration. This batch of data consists of two parts: one part is image samples from the original training set (used to maintain the model's classification accuracy for normal images); the other part is generated extended adversarial image samples, used to provide gradient information about weak points in the decision boundary. The data from the hybrid batch is input into the model to be tuned for forward propagation, and the model's predicted output is calculated. Subsequently, the difference between the predicted result and the true label is calculated using a loss function. For extended adversarial image samples, even though they are located in the ambiguous zone of the decision boundary or have been disturbed into the misclass region, the model is forced to classify them as the label category of the source image sample during the fine-tuning process. This constraint is crucial, essentially sending a strong corrective signal to the model: the decision boundary should be shifted in these ambiguous or confusing regions so that these samples fall back into the jurisdiction of the correct category.

[0046] Based on the calculated total loss, the gradient of the loss function with respect to the model parameters is calculated using the backpropagation algorithm. Subsequently, stochastic gradient descent or other optimizers are used to update the model parameters along the direction of gradient descent. The geometrically intuitive effect of this parameter update process is that it forces the model's decision boundary to deform, actively moving it away from the blurred regions detected by the feature-level convex polytope, thereby significantly increasing the margin between different categories. Through multiple rounds of iterative fine-tuning, the model gradually adapts to these generated complex adversarial image samples. The final optimized image classification model not only maintains competitive classification accuracy on the original test set, but more importantly, it demonstrates significantly improved defense capabilities against various powerful adversarial attacks. This method of actively detecting and repairing boundary blind spots fundamentally enhances the security and reliability of image classification models in complex, adversarial environments.

[0047] The principle of the image classification model robustness tuning method of the present invention will be further explained below.

[0048] See Figure 2This paper illustrates the fundamental principle of key adversarial image sample generation in the adversarial robustness tuning method of the image classification model of the present invention, including two parts: the construction of a feature-level convex polytope search space and the solution of constraint optimization. A feature-level convex polytope search space is constructed by introducing a non-uniform feature-level convex interpolation operation between the source image samples and the guide image samples of disjoint classes. This feature search space allows different feature dimensions to have independent interpolation combination coefficients during runtime, thereby flexibly probing the ambiguous region of the decision boundary within the geometric convex hull formed by the source image samples and the guide image samples. This avoids the limitation of traditional norm-based attack methods that search within a fixed small neighborhood, ultimately generating key adversarial image samples with higher relevance to the decision boundary. After the feature search space is constructed, a constraint optimization problem can be built. By iteratively solving for the optimal feature combination coefficients, adversarial samples located near the decision boundary are generated, providing a high-quality data foundation for subsequent model adversarial fine-tuning. Therefore, the present invention essentially improves the ability of generated adversarial image samples to detect the model decision boundary from the perspective of improving the construction of the adversarial image sample search space. Specifically, by using correctly predicted image samples of disjoint categories as guidance, non-uniform convex combination constraints are introduced in the feature dimension to construct a feature-level convex multicell. This allows the search path for adversarial image samples to adaptively extend to complex transition regions between different categories. Therefore, the key adversarial image samples generated based on this method can more accurately locate blind spots and vulnerabilities in the model's decision boundary. Compared to traditional gradient-based noise stacking methods, these image samples carry richer boundary distribution information and semantic features, maximizing the exposure of security risks of image classification models on complex decision manifolds. This significantly improves the robustness gain of model fine-tuning based on these image samples, assisting in the reliable deployment of image classification models in security-sensitive scenarios.

[0049] See Figure 3 To address the issue of improving the robustness of image classification models near complex decision boundaries, this invention employs an adversarial fine-tuning strategy based on neighborhood manifold sampling. Specifically, this strategy no longer limits itself to a single adversarial sample point but utilizes generated key adversarial image samples to estimate the adversarial distribution near the decision boundary. After obtaining the key adversarial image sample, it is treated as an anchor point in the current ambiguous region of the decision boundary, and a Gaussian mixture model is used to approximate the neighborhood distribution of this region. In practice, the mean is the perturbation direction of the key adversarial image sample relative to the source sample, and a preset variance is used. Construct a Gaussian distribution with standard deviation. Subsequently, random sampling is performed from this distribution to generate several extended adversarial image samples distributed near the decision boundary manifold. These extended adversarial image samples carry semantic information similar to the source samples, but are densely distributed in the feature space in ambiguous areas that are prone to model misjudgment.

[0050] During the model parameter update phase, the adversarial fine-tuning process includes mixing the extended adversarial image samples with normal image samples from the original training set and inputting them into the image classification model to be optimized. At this point, the model no longer simply fits the original data distribution but needs to simultaneously minimize the classification loss of both the normal image samples and the extended adversarial image samples. Since the extended adversarial image samples are labeled as the true categories of the source image samples, updating the model parameters through backpropagation forces the model's learned decision boundaries to actively move away from these ambiguous regions, thereby significantly increasing the inter-class margin.

[0051] In one possible implementation, the process of constructing and fine-tuning an enhanced adversarial dataset based on generated key adversarial image samples is a dynamic iterative process.

[0052] Specifically, this implementation uses counter-fine-tuning as an example to describe the process: S11: First, sample a batch of original image samples from the training dataset to ensure coverage of all categories and guarantee data diversity.

[0053] S12: For each original image sample, a non-uniform feature-level convex interpolation method is used to search for and generate the corresponding key adversarial image sample in the feature space, which is located near the current decision boundary of the model.

[0054] S13: Exploit Model the neighborhood of key adversarial image samples and sample to generate An extended adversarial image sample is added to the current training batch to form an enhanced adversarial dataset.

[0055] S14: Input the augmented adversarial dataset into the model and perform forward propagation, calculate the cross-entropy loss, and update the model parameters through stochastic gradient descent (SGD) or other optimizers to complete a round of fine-tuning.

[0056] S15: Iterate through S11~S14 until the model converges or reaches the predetermined number of training rounds.

[0057] As can be seen, the image classification model adversarial robustness tuning method of this invention essentially uses generated key samples as detectors to detect and repair defects on the model's decision boundary. Because a neighborhood manifold-based distribution sampling is introduced during fine-tuning, the model can access a wider range of potential attack samples near the decision boundary, rather than just a specific worst-case scenario. This mechanism gives the fine-tuned model better generalization ability and avoids the overfitting phenomenon that easily occurs in traditional adversarial training—that is, it is only effective against specific attacks but vulnerable to other attacks, and it severely impairs the recognition rate of normal samples.

[0058] In one possible implementation, experimental verification shows that even when facing various types of strong adversarial attacks, the optimized model of this invention maintains extremely high robustness and accuracy. Specifically, experiments on the classic datasets MNIST and CIFAR-10 demonstrate that the method of this invention improves adversarial robustness while having minimal impact on the classification accuracy of the original clean data. For example, on the CIFAR-10 dataset, compared to traditional PGD adversarial training, the method of this invention improves robustness against white-box PGD attacks by approximately 5.92%, while maintaining a high level of accuracy on normal samples. These experiments prove that this invention not only finds more meaningful adversarial image samples through feature-level interpolation, but also effectively optimizes and reconstructs the model's decision boundary through a distribution-based fine-tuning strategy, demonstrating significant practical value.

[0059] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the apparatus embodiments, please refer to the embodiments of the method of the present invention.

[0060] See Figure 4 In another embodiment of the present invention, an image classification model adversarial robustness tuning system is provided, which can be used to implement the above-mentioned image classification model adversarial robustness tuning method. Specifically, the image classification model adversarial robustness tuning system includes a sample acquisition module, a space construction module, a sample generation module, and a model tuning module.

[0061] The module consists of several components: a sample acquisition module for acquiring source image samples and several guide image samples of different categories; a space construction module for constructing a feature search space based on the source image samples and the guide image samples using a non-uniform feature-level convex interpolation method with independent combination coefficients of different feature dimensions; a sample generation module for searching the feature search space to obtain key adversarial image samples with the optimization objective of maximizing the classification loss of the image classification model; and a model tuning module for constructing an enhanced adversarial dataset based on the key adversarial image samples and performing adversarial fine-tuning training on the image classification model based on the enhanced adversarial dataset to obtain a tuned image classification model.

[0062] In one possible implementation, the guidance image sample has a model prediction confidence score greater than a preset prediction confidence score threshold based on the image classification model; the number of guidance image samples is not less than 4.

[0063] In one possible implementation, the step of constructing the feature search space using a non-uniform feature-level convex interpolation method with independent combination coefficients of different feature dimensions based on the source image samples and several guide image samples includes: setting combination coefficients for the source image samples and each guide image sample with dimensions consistent with the image dimensions; and constructing the feature search space using the following formula. :

[0064] ,

[0065] in, for and each The combined coefficient vector, The combination coefficients of the source image samples, For element-wise multiplication, For source image samples, To guide the number of image samples, For the first The combination coefficients of the guide image samples For the first A sample of guiding images, for The Dimension value, for The Dimension value.

[0066] In one possible implementation, the step of searching the feature search space to obtain key adversarial image samples with the goal of maximizing the classification loss of the image classification model includes: constructing an objective function with the goal of maximizing the classification loss of the image classification model. :

[0067] in, To guide the weight vector, For balance coefficient, for The guiding weight, For classification loss function, for and Classification loss, for Predicted category based on image classification model for The tag category.

[0068] Solve the objective function The optimal combination coefficient vector and guiding weight vector are obtained; based on the optimal combination coefficient vector and guiding weight vector, the source image samples and each guiding image sample are fused to obtain the key adversarial image samples.

[0069] In one possible implementation, the objective function is solved. Obtaining the optimal combined coefficient vector and guiding weight vector includes: initializing the combined coefficient vector and guiding weight vector; iteratively executing the first fixed update step and the second fixed update step until the preset number of iterations or the objective function is reached. Convergence is achieved by using the current combined coefficient vector and guiding weight vector as the optimal combined coefficient vector and guiding weight vector. The first fixed update step includes: fixing the combined coefficient vector, and using the classification loss of each guiding image sample... The function normalizes and updates the guidance weights of each guidance image sample; the second fixed update step includes: fixing the guidance weight vector, updating the combined coefficient vector using the box constraint optimization algorithm, and normalizing the updated combined coefficient vector.

[0070] In one possible implementation, the construction of the enhanced adversarial dataset based on key adversarial image samples includes: constructing a neighborhood distribution of key adversarial image samples using a Gaussian mixture model, sampling in the neighborhood distribution to obtain several extended adversarial image samples, setting the label category of the extended adversarial image samples to the label category of the corresponding source image samples, and adding them to the training set of the image classification model to obtain the enhanced adversarial dataset.

[0071] In one possible implementation, constructing the neighborhood distribution of key adversarial image samples using a Gaussian mixture model includes: using the perturbation direction of the key adversarial image samples relative to the source image samples as the mean, and a preset variance. A Gaussian distribution is constructed for the standard deviation as the neighborhood distribution of key adversarial image samples.

[0072] All relevant content of each step involved in the aforementioned embodiment of the image classification model adversarial robustness tuning method can be referenced in the functional description of the corresponding functional module of the image classification model adversarial robustness tuning system in the embodiment of the present invention, and will not be repeated here.

[0073] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0074] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of image classification model adversarial robustness tuning methods.

[0075] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the image classification model adversarial robustness tuning method in the above embodiments.

[0076] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for optimizing the adversarial robustness of an image classification model, characterized in that, include: Acquire source image samples and several guide image samples that are of different categories from the source image samples; Based on the source image samples and several guide image samples, a feature search space is constructed using a non-uniform feature-level convex interpolation method with independent combination coefficients of different feature dimensions. With the goal of maximizing the classification loss of the image classification model, a search is performed in the feature search space to obtain key adversarial image samples; An enhanced adversarial dataset is constructed based on key adversarial image samples, and the image classification model is fine-tuned and trained on the enhanced adversarial dataset to obtain an optimized image classification model.

2. The image classification model adversarial robustness tuning method according to claim 1, characterized in that, The guidance image samples have a model prediction confidence score greater than a preset prediction confidence score threshold based on the image classification model; the number of guidance image samples is not less than 4.

3. The image classification model adversarial robustness tuning method according to claim 1, characterized in that, The step of constructing the feature search space based on source image samples and several guide image samples using a non-uniform feature-level convex interpolation method with independent combination coefficients of different feature dimensions includes: Set combination coefficients for the source image samples and each guide image sample with dimensions consistent with the image dimensions; The feature search space is constructed using the following formula. : ， in, for and each The combined coefficient vector, The combination coefficients of the source image samples, For element-wise multiplication, For source image samples, To guide the number of image samples, For the first The combination coefficients of the guide image samples For the first A sample of guiding images, for The Dimension value, for The Dimension value.

4. The image classification model adversarial robustness tuning method according to claim 3, characterized in that, The optimization objective is to maximize the classification loss of the image classification model. The search in the feature search space yields key adversarial image samples, including: The objective function is constructed to maximize the classification loss of the image classification model. : in, To guide the weight vector, For balance coefficient, for The guiding weight, For classification loss function, for and Classification loss, for Predicted category based on image classification model for Tag categories; Solve the objective function Obtain the optimal combination coefficient vector and guiding weight vector; Based on the optimal combination coefficient vector and guiding weight vector, the source image samples and each guiding image sample are fused to obtain the key adversarial image samples.

5. The image classification model adversarial robustness tuning method according to claim 4, characterized in that, The objective function to be solved The optimal combination coefficient vector and guided weight vector are obtained by: Initialize the combined coefficient vector and the guiding weight vector; Iteratively execute the first fixed update step and the second fixed update step until the preset number of iterations or the objective function is reached. Convergence is achieved by using the current combined coefficient vector and guiding weight vector as the optimal combined coefficient vector and guiding weight vector. The first fixed update step includes: fixing the combined coefficient vector, and using the classification loss of each guiding image sample... The function normalizes and updates the guidance weights of each guidance image sample; the second fixed update step includes: fixing the guidance weight vector, updating the combined coefficient vector using the box constraint optimization algorithm, and normalizing the updated combined coefficient vector.

6. The image classification model adversarial robustness tuning method according to claim 1, characterized in that, The enhanced adversarial dataset constructed based on key adversarial image samples includes: A neighborhood distribution of key adversarial image samples is constructed using a Gaussian mixture model. Several extended adversarial image samples are obtained by sampling from the neighborhood distribution. The label categories of the extended adversarial image samples are set to the label categories of the corresponding source image samples and added to the training set of the image classification model to obtain the enhanced adversarial dataset.

7. The image classification model adversarial robustness tuning method according to claim 6, characterized in that, The neighborhood distribution of key adversarial image samples constructed using the Gaussian mixture model includes: The mean is the perturbation direction of the key adversarial image sample relative to the source image sample, and the variance is set to a preset value. A Gaussian distribution is constructed for the standard deviation as the neighborhood distribution of key adversarial image samples.

8. A system for optimizing the adversarial robustness of an image classification model, characterized in that, include: The sample acquisition module is used to acquire source image samples and several guide image samples that are of different categories from the source image samples. The space construction module is used to construct the feature search space based on the source image samples and several guide image samples, using a non-uniform feature-level convex interpolation method with independent combination coefficients of different feature dimensions; The sample generation module is used to search in the feature search space to obtain key adversarial image samples with the optimization objective of maximizing the classification loss of the image classification model. The model tuning module is used to construct an enhanced adversarial dataset based on key adversarial image samples, and to perform adversarial fine-tuning training on the image classification model based on the enhanced adversarial dataset to obtain a tuned image classification model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image classification model adversarial robustness tuning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the image classification model adversarial robustness tuning method as described in any one of claims 1 to 7.

Images