Neural network architecture for robust picture classification of mobile terminal and classification method thereof

Abstract

The invention relates to the technical field of neural network architecture for mobile terminal robust picture classification, and discloses a neural network architecture for mobile terminal robust picture classification and a classification method thereof, and the neural network architecture comprises a hierarchical structure of a mapping module, a reduction module, a feature extraction module and a classification module. An input picture can be efficiently processed, initial features are extracted and optimized layer by layer, a normal branch and a robust branch in a feature extraction module carry out operation on high-frequency information and redundant information respectively, and a feature fusion layer integrates output of the two branches. The method has the beneficial technical effects that the cleaning precision and the adversarial / natural disturbance robustness are taken into consideration on the premise of meeting the computing power constraint of the mobile terminal. The precision and robustness of a mobile terminal picture classification system are effectively improved, and the method is suitable for complex scenes in practical application and has good market prospects.

Description

Technical Field

[0001] This invention relates to the field of neural network architecture technology for robust image classification on mobile devices, and more particularly to a neural network architecture and classification method for robust image classification on mobile devices. Background Technology

[0002] Neural Architecture Search (NAS) technology is now widely used for automatically designing deep learning models, especially in mobile and edge device scenarios. By automatically searching for network structures under given computing power and latency constraints, it achieves high classification accuracy and low inference latency. Existing NAS methods typically use average accuracy on clean datasets as the primary optimization objective, and combine this with joint constraints such as model parameter count, floating-point operation cost, and terminal latency. It has already been applied in tasks such as image classification and object detection. For example, techniques such as gradient-based automatic network construction methods (e.g., DARTS: Differentiable Architecture Search) can efficiently search for image classification model structures.

[0003] However, in real-world applications, input images are often not ideally clean samples. On one hand, images captured by mobile devices are susceptible to factors such as changes in lighting, blurring, noise, and compression distortion. On the other hand, in scenarios such as content moderation, identity authentication, and security monitoring, attackers can significantly interfere with model predictions through carefully crafted adversarial perturbations, making them virtually imperceptible to the human eye. Such perturbations make the model highly sensitive to minute changes in the input space, causing predictions to fluctuate drastically within a reasonable perturbation range, severely impacting the system's security and reliability. Figure 1 As shown, the top row represents common weather erosion (clear, frost, snow, fog, rain), while the bottom row represents image distortion caused by typical adversarial attacks and implementation differences, all of which significantly affect the discrimination results of traditional models.

[0004] To improve model performance in the aforementioned complex environments, various adversarial training and robust optimization methods have been proposed. These methods introduce adversarial examples or robustness regularization terms onto a fixed network structure to enhance adversarial accuracy. However, numerous studies have shown that the network structure itself has a significant impact on robustness: different cell topologies, convolution types, and connection methods lead to fundamental differences in the smoothness of decision boundaries and classification margins. When using only traditional lightweight network structures (such as several hand-designed mobile models), even with the introduction of various robust training strategies, there remains a clear performance ceiling and stringent trade-offs between robust accuracy, clean accuracy, and computational cost.

[0005] Therefore, robustness is incorporated into the network architecture design goals from the outset. While considering terminal computing power and latency constraints, the system automatically searches for neural network structures more suitable for robust image classification scenarios. Leveraging the automation capabilities of Neural Architecture Search (NAS), by introducing robustness evaluation metrics and adversarial perturbation models during the search process, a class of network architectures that combine robust performance and deployment efficiency can be explored in the hypernetwork search space, thus providing an end-to-end structure generation method for robust image classification on mobile devices and edge computing. However, existing technologies have the following shortcomings when dealing with adversarial attack robustness: Evaluation metrics are susceptible to anomalous fluctuations: In adversarial training environments, gradient signals and performance metrics often exhibit heavy-tailed distributions, resulting in a small number of outlier extreme values. Traditional NAS directly uses average performance or gradients as evaluations of operation quality, making it vulnerable to interference from these outliers. For example, some candidate operations may perform normally in most cases, but cause a sharp drop in performance under individual adversarial examples. If only the average is taken, the true contribution of the operation may be underestimated, leading to incorrect architecture selection. Existing methods such as AdvRush (AdvRush: Searching for Adversarially Robust Neural Architectures) and LRNAS (LRNAS: Differentiable Searching for Adversarially Robust Lightweight Neural Architecture) do not adequately address this issue, resulting in an unstable search process: a few extreme samples may cause high-quality operations to be eliminated prematurely or low-quality operations to be incorrectly retained, affecting the robustness and accuracy of the target network configuration.

[0006] Balancing clean sample accuracy and adversarial robustness is challenging: Existing robust architecture search methods often sacrifice some clean sample accuracy while improving adversarial robustness. Traditional methods lack a unified measure of the "dual objectives," often failing to simultaneously guarantee high accuracy in attack-free scenarios and high robustness under attack. This is because current techniques typically optimize based on only a single metric (e.g., optimizing only the adversarial loss), leading to performance degradation of the resulting model in non-attack scenarios. Balancing these two performance metrics and designing a network that excels under both conditions is another challenge facing current technologies.

[0007] The search efficiency and stability are unsatisfactory: adversarial training itself is computationally expensive, and some methods require a large number of samples to average for stable evaluation, increasing search costs. Furthermore, existing methods lack theoretical guarantees and cannot quantitatively assess the relationship between the required sample size and evaluation error, relying solely on empirical parameter tuning. This makes the search process unreliable: developers struggle to determine in a timely manner whether the search has converged or whether the results are reliable, potentially leading to repeated trial and error and wasted computational resources. Summary of the Invention

[0008] Based on this, it is necessary to address the existing problem of searching neural network architectures for robust image classification on mobile devices, and propose a neural network architecture and classification method for robust image classification on mobile devices.

[0009] A neural network architecture for robust image classification on mobile devices, the neural network architecture comprising a mapping module, a reduction module, a feature extraction module and a classification module connected in sequence; The mapping module includes multiple first convolutional layers and normalized cells. The multiple first convolutional layers are connected in sequence, and the last first convolutional layer is connected to the normalized cells to extract the initial features of the input image. The reduction module includes multiple reduction cells connected in sequence. The first reduction cell is connected to the normalized cell. Each reduction cell processes the initial feature through a downsampling operation to obtain the reduced intermediate feature. The feature extraction module includes a normal branch, a robust branch, and a feature fusion layer. The normal branch and the robust branch are respectively connected to the last-ranked reduced cell. The normal branch and the robust branch are respectively connected to the feature fusion layer. The normal branch is used to extract high-frequency operators from the intermediate features to obtain precision-friendly operators. The robust branch is used to extract redundant operators from the intermediate features to obtain robust-friendly operators. The feature fusion layer is used to fuse the precision-friendly operators and the robust-friendly operators to obtain fused features. The classification module is used to classify the fused features.

[0010] Furthermore, both the normal branch and the robust branch in the feature extraction module include at least one bibranched cell; Each of the bi-branch cells is formed by multiple edges connected together, and each edge is configured with an image processing operator selected from a preset set of candidate operators; wherein, the set of candidate operators for bi-branch cells in the normal branch includes a first type of operator, which is configured to extract fine-grained high-frequency features; the set of candidate operators for bi-branch cells in the robust branch includes a second type of operator, which is configured to extract smooth and redundant features; The precision-friendly operator and the robust-friendly operator are obtained by combining image processing operators selected from the first type of operator and the second type of operator on each edge of the bi-branched cell, respectively.

[0011] Furthermore, the structure of the feature extraction module specifically includes: Multiple bi-branch cells are cascaded sequentially, and a feature fusion layer is disposed between every two adjacent bi-branch cells; wherein, the feature output by the previous bi-branch cell is adaptively fused by the feature fusion layer and then passed as input to the next bi-branch cell; Multiple reduced cells in the reduction module are inserted between sequences composed of multiple bi-branched cells to perform downsampling operations at multiple depths of the network, thereby achieving multi-scale feature extraction and obtaining the fused features.

[0012] Furthermore, it also includes a robust evaluator connected to each of the two-branch cells, used to collect and score the image processing operators selected for each of the two-branch cells in order to filter the corresponding target network configuration.

[0013] Furthermore, the robust evaluator is configured to execute an iterative screening strategy to search for the target network configuration, the screening strategy including the following steps: a) Based on the collected scores, for each edge in each of the two-branch cells, compare the comprehensive robust scores of each candidate image processing operator under it. b) Identify the candidate image processing operator with the lowest overall robustness score on each edge and use it as a candidate for elimination; c) When the score of a candidate image processing operator meets the preset significant disadvantage condition, it is marked as a significant disadvantage operation; d) In each round of search iteration, select several candidate image processing operators marked as elimination candidates or significant disadvantageous operations from each edge and remove them, thereby reducing the search space of the neural network architecture; e) Repeat steps a) to d) until the search space converges to determine the target network configuration.

[0014] A neural network-based image classification method, used to train the aforementioned neural network architecture for robust mobile image classification, includes: Get the images to be categorized; The image to be classified is input into a neural network architecture for robust image classification on mobile devices to obtain the classification result; The training steps for the neural network architecture designed for robust image classification on mobile devices include: Obtain the training dataset; Construct an initial super network; wherein the initial super network includes a mapping module, a reduction module, a feature extraction module, and a classification module, and each edge of each bi-branch cell in the feature extraction module is configured with a complete set of candidate image processing operators; The training dataset is input into the initial supernetwork for training, and the performance of each candidate image processing operator is evaluated. Based on the evaluation results, some image processing operators are iteratively filtered and removed from the candidate image processing operators of the initial super network to shrink the search space until the final target network configuration is determined. The network configuration is used as the trained neural network model.

[0015] Further, the step of iteratively filtering and removing some image processing operators from the candidate image processing operators of the initial supernetwork based on the evaluation results to narrow the search space until the final target network configuration is determined includes: Obtain the performance evaluation results of each candidate image processing operator on each edge of each bi-branch cell in the initial supernetwork based on the training dataset; For each edge of each bi-branch cell, compare the performance evaluation results of each candidate image processing operator; Based on the comparison results, identify the candidate image processing operators that meet the preset elimination conditions on each edge; Remove the identified candidate image processing operators that meet the preset elimination conditions to update the initial super network and shrink the search space; Determine whether the updated search space meets the preset convergence criteria; If not, then based on the updated supernetwork, return to the step of inputting the training dataset for training and evaluation; If so, the final target network configuration is determined based on the remaining candidate image processing operators on each edge of the initial supernetwork.

[0016] Further, the step of obtaining the performance evaluation results of each candidate image processing operator on each edge of each bi-branch cell in the initial supernetwork based on the training dataset includes: Based on the validation set, multiple rounds of performance contribution measurement are performed on each candidate image processing operator for each edge in the current super network; wherein, the performance contribution measurement is achieved by comparing the performance index difference of the super network on the image classification task when the candidate image processing operator is retained in the super network and when the candidate image processing operator is removed. Based on the results of the multi-round performance contribution measurement, a preset robust statistical algorithm is used to calculate a comprehensive robust score for each candidate image processing operator, which serves as the performance evaluation result.

[0017] Furthermore, in the step of identifying candidate image processing operators that meet the preset elimination conditions on each side based on the comparison results, the preset elimination conditions are: the comprehensive robustness score of the candidate image processing operator is lower than a preset threshold or is the lowest score among all candidate image processing operators on the same side.

[0018] Furthermore, the step of obtaining the training dataset includes: Obtain the original image dataset containing the target task labels; Preprocessing operations are performed on the original image dataset to obtain a clean image dataset; the preprocessing operations include resizing, cropping, and color normalization. The clean image dataset is input into a preset image degradation operator to generate multiple naturally perturbed version images to form a naturally perturbed image dataset; wherein each clean image and its corresponding naturally perturbed version images maintain the same target task label; An adversarial attack method based on gradient symbols is adopted to process the clean image dataset within a preset perturbation constraint to generate corresponding adversarial sample images to form an adversarial image dataset; wherein, each clean image and its corresponding adversarial sample image maintain the same target task label; Each original image is associated with a clean image, at least one naturally perturbed version image, and an adversarial example image, and thus a training dataset consisting of all the sample groups is obtained.

[0019] The beneficial effects of this invention are as follows: The architecture adopts a hierarchical structure of a mapping module, a reduction module, a feature extraction module, and a classification module. This allows for efficient and low-cost processing of input images, extraction of initial features, and layer-by-layer optimization, achieving end-to-end inference from the input image to the classification output. This design not only improves the model's processing speed but also enhances its adaptability to different image scales and complexities. The normal branch and robust branch in the feature extraction module respectively target high-frequency and redundant information, ensuring a good balance between accuracy and robustness. The normal branch focuses on extracting key features, while the robust branch enhances the model's resistance to noise and interference. This allows the model to maintain high classification accuracy in various environments. The feature fusion layer effectively integrates the outputs from the two branches, improving the overall performance of the classification module. In summary, this invention, while meeting the computational constraints of mobile devices, simultaneously considers cleanliness accuracy and robustness against adversarial / natural perturbations. It effectively improves the accuracy and robustness of mobile image classification systems, is suitable for complex scenarios in practical applications, and has good market prospects.

[0020] This invention innovatively introduces a robust branch and a normal branch for parallel optimization in the network structure. This dual-branch design ensures that both clean accuracy and adversarial robustness are considered during the search process, representing a significant innovation at the architectural level. The beneficial effects are that the final network architecture includes modules optimized for different scenarios, possessing both high expressive power and high resistance to perturbations. For example, the robust branch obtained by this invention tends to choose operations such as dilated convolutions and skip connections to form smoother, more redundant feature extraction paths, while the normal branch chooses operations such as separating convolutions to extract richer features. This differentiated architectural configuration makes the key feature extraction pathways more robust against adversarial attacks, less susceptible to disruption by single-point perturbations; simultaneously, it does not lose accuracy under normal input. Compared to existing single-structure models, the dual-branch architecture of this invention significantly improves accuracy under adversarial attacks (by several percentage points at the same attack intensity), while maintaining or even slightly improving accuracy on clean samples. In summary, the dual-branch architecture design achieves an excellent balance between security and practicality, a feature not found in traditional architectures. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] in: Figure 1Here is an example diagram of image perturbation in one embodiment; Figure 2 This is a comparison diagram of the overall structure of a super network and a regular network in one embodiment; Figure 3 Here is an example diagram of an image processing operator in one embodiment; Figure 4 This is a branching diagram of a bibranched cell structure in one embodiment; Figure 5 This is a schematic diagram of the feature fusion layer structure in one embodiment; Figure 6 This is a flowchart of a neural network architecture for robust image classification on mobile devices in one embodiment. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] A neural network architecture for robust image classification on mobile devices includes a mapping module, a reduction module, a feature extraction module, and a classification module connected in sequence. The mapping module comprises multiple first convolutional layers and normalized cells, with the first convolutional layers connected sequentially, and the last first convolutional layer connected to the normalized cell, for extracting initial features from the input image. The reduction module comprises multiple reduction cells connected in sequence, with the first-order reduction cell connected to the normalized cell. Each reduction cell processes the initial features through a downsampling operation to obtain reduced intermediate features. Features; the feature extraction module includes a normal branch, a robust branch, and a feature fusion layer. The normal branch and the robust branch are respectively connected to the last-ranked reduced cell. The normal branch and the robust branch are respectively connected to the feature fusion layer. The normal branch is used to extract high-frequency operators from the intermediate features to obtain precision-friendly operators. The robust branch is used to extract redundant operators from the intermediate features to obtain robust-friendly operators. The feature fusion layer is used to fuse the precision-friendly operators and the robust-friendly operators to obtain fused features. The classification module is used to classify the fused features.

[0025] In this embodiment, the architecture adopts a systematic design, aiming to collaboratively optimize the classification accuracy and robustness against interference / natural disturbances of the model under the constraints of limited computing resources on mobile devices, ultimately obtaining a lightweight yet reliable visual model. Specifically, the traditional single feature processing path is decoupled into two parallel and focused branches: "accuracy optimization" and "robustness enhancement." Through subsequent intelligent fusion, the model possesses both fine-grained discriminative power and anti-interference stability. The overall workflow follows the principle of coarse-to-fine and divide-and-conquer fusion, as follows: First, the mapping module, as the network entry point, performs a series of basic convolution and normalization operations on the input image to perform initial feature purification and standardization, providing a clean and standardized data foundation for subsequent complex processing. Subsequently, the features enter the reduction module. This module consists of multiple downsampling cells connected in series. Its core function is to perform feature abstraction and dimensionality management, gradually reducing the spatial size of the feature map to compress the computational load, while increasing the number of channels to enrich semantic information, thereby achieving a balance between efficiency and expressive power, producing intermediate features suitable for deep processing on mobile devices. The feature extraction module receives intermediate features from the reduction module and feeds them simultaneously into two structurally independent, functionally specialized parallel branches: The Normal Branch focuses on extracting high-frequency details and fine-grained features crucial for classification accuracy. Internally, it tends to configure or search for processing operators that capture subtle image textures and edge information (“accuracy-friendly operators”), aiming to improve the model's recognition accuracy on ideally clear images. The Robust Branch focuses on extracting smooth structures and redundant features that remain stable under perturbations. Internally, it tends to configure or search for processing operators that are insensitive to noise, blur, and pixel changes (“robust-friendly operators”), aiming to strengthen the model's performance retention capabilities in the face of natural degradation or adversarial attacks. The outputs of the two branches, carrying the complementary characteristics of “accuracy” and “robustness”, are fed into the feature fusion layer. This layer organically integrates the two types of features using specific algorithms (such as concatenation, weighted summation, or attention-based adaptive fusion) to generate fused features that simultaneously possess high discriminative power and strong stability. Finally, the classification module receives the fused features and uses operations such as global pooling and fully connected layers to aggregate the rich feature information into the final category probability distribution, thus completing the image classification task.

[0026] For the original image The images in the dataset undergo uniform preprocessing operations, including but not limited to: resizing, center cropping / random cropping, color normalization, and random flipping, to obtain a clean set of preprocessed images. These operations are used to simulate the image classification process under normal shooting and conventional image enhancement conditions.

[0027] To characterize degradation factors in real-world environments, clean images are input into preset image erosion and resampling operators to generate various versions with natural perturbations, including but not limited to: 1. Weather-related corrosion: frost, snow, fog, rain, etc.; 2. Degradational perturbations: blurring, noise, compression, and contrast changes; 3. Implement differences: Differences in interpolation and resampling methods between different libraries, etc.

[0028] Each clean image maintains consistent labels with its multiple perturbation versions, forming a collection of naturally perturbation images. (Show your opinion) Figure 1 ).

[0029] Based on the above definitions, to evaluate and improve the adversarial robustness of the network, a pre-defined adversarial attack method is used to generate adversarial examples within the constraint norm and perturbation radius for clean samples: The Fast Gradient Sign Method (FGSM) and Projected Gradient Descent are employed. For the current model and the input image, the gradient of the loss function with respect to the input is calculated; the sign of this gradient is then determined, and the gradient is applied at a preset step size. Each pixel is updated once along this direction; the resulting new image is still very close to the original image in pixel space, but it can significantly increase the probability of model prediction errors. FGSM has low computational cost and can quickly generate the first batch of adversarial examples for preliminary evaluation of the model's sensitivity to attacks. PGD is more aggressive and better reflects the model's robustness.

[0030] against The samples in the dataset are subjected to a pre-defined adversarial attack method with respect to the constraint norm and perturbation radius. Internally generated adversarial examples. And based on... A norm-constrained PGD attack, with 20 attack steps, a step size of 2 / 255, and a perturbation radius of 0.031, yields an adversarial image set. .

[0031] In this way, each original sample corresponds to a set of "cleaning graph – natural perturbation graph – adversarial graph", which provides a rich and controllable data foundation for the subsequent unified measurement of "cleaning accuracy – adversarial robustness".

[0032] like Figure 2 The left side shows the traditional architecture, and the right side shows the dual-branch architecture designed in this application. The architecture consists of a Stem module (mapping module), several dual-branch cells, several reduce cells, and fusion and classification heads. Robustness adjustments are made during the design of the super neural network using the DARTS supernet. An example diagram is shown below. Figure 2 .

[0033] Stem module: Maps the input 3-channel image to a higher-dimensional feature space, typically using one or more convolutional layers with normalization.

[0034] Multilayer cell cascade: The network contains L cells in cascade (L is a hyperparameter, for example, 10), of which about 1 / 3 and 2 / 3 of the layers are reduced cells, and the rest are bi-branched cells.

[0035] Reduce Cell: Reduces feature map resolution and expands the receptive field through stride convolution or pooling, enabling multi-scale representation.

[0036] DualBranchCell: It contains two sets of structures: Normal Cell and Robust Cell, and performs feature fusion at the tail through the Channel Attention Fusion layer.

[0037] Output Header: Performs global average pooling on the features of the last layer, then connects to a fully connected layer or other task head to output the classification result.

[0038] Construction of candidate operation set: Configure multiple image processing operators as candidate operations on each edge of each branch cell, including but not limited to: 3×3 / 5×5 depthwise separable convolution; 3×3 / 5×5 dilated convolution; 3×3 max / average pooling; Skip connections, etc.

[0039] The normal branch is more inclined towards fine-grained high-frequency operators, while the robust branch is more inclined towards smooth and redundant operators. The "precision-friendly operator" and the "robust-friendly operator" are combined with the edge as the granularity. That is, the "precision-friendly operator" and the "robust-friendly operator" each specifically include at least one of the above image processing operators, providing a finer search granularity while controlling the amount of computation.

[0040] In one embodiment, both the normal branch and the robust branch in the feature extraction module include at least one bi-branch cell; each bi-branch cell is formed by multiple edges connected together, and each edge is configured with an image processing operator selected from a preset set of candidate operators; wherein, the set of candidate operators for the bi-branch cells in the normal branch includes a first type of operator, which is configured to extract fine-grained high-frequency features; the set of candidate operators for the bi-branch cells in the robust branch includes a second type of operator, which is configured to extract smooth and redundant features; the precision-friendly operator and the robust-friendly operator are obtained by combining image processing operators selected from the first type of operator and the second type of operator respectively on each edge of the bi-branch cell.

[0041] In one embodiment, the feature extraction module specifically includes: multiple bi-branch cells cascaded sequentially, and a feature fusion layer disposed between every two adjacent bi-branch cells; wherein, the features output by the previous bi-branch cell are adaptively fused by the feature fusion layer and then passed as input to the next bi-branch cell; multiple reduction cells in the reduction module are inserted between the sequences formed by the multiple bi-branch cells to perform downsampling operations at multiple depths of the network, thereby achieving multi-scale feature extraction and obtaining the fused features.

[0042] like Figure 4 As shown, this invention designs a dual-branch structure, including a normal branch and a robust branch, based on the DARTS-type cell structure. The entire structure represents a Dual-BranchCell, with the upper half corresponding to the robust branch and the lower half to the normal branch. Image features from the previous layer first enter the left vertical bar, and then are simultaneously fed into the upper and lower sub-branches via the middle arrow. Feature transformation is performed in each sub-branch before fusion and output in subsequent layers. Because images exhibit two significantly different statistical types in real-world networks: Clean / Routine Enhanced Distribution (in Step 1) High-frequency textures and details are beneficial for improving accuracy; perturbation / adversarial distribution ( A larger receptive field, redundancy, and smoothness are needed to stabilize the decision boundary. If a single topology is forced to simultaneously accommodate both types of objectives, gradients will conflict, leading to unstable structural learning.

[0043] Therefore, this invention explicitly decouples at the Cell level: Robust branch ( Figure 4The upper dashed box indicates a bias towards a large receptive field and redundancy, improving stability against natural degradation and adversarial perturbations (such as dilated convolutions, residuals / skips, etc.). Input 1 / Input 2: The two input boxes located to the left of the robust branch represent the two input image features received by this cell from the previous layer of the network, generally corresponding to the outputs of the first two cells. They have the same spatial size, and the number of channels can be the same or aligned through 1×1 convolutions.

[0044] Intermediate Nodes 0–3: The four blue rectangles are labeled 0, 1, 2, and 3, representing the four intermediate computation nodes within the robust branch. Each node receives several directed edges from all preceding nodes and the two input nodes. Internally, the output image features of all input edges are added or concatenated to obtain the output image features of that node. The entire structure constitutes a small directed acyclic graph (DAG).

[0045] Output Nodes: The output box on the right represents the output nodes of this branch. Typically, the outputs of nodes 0–3 are aggregated in a preset manner (such as splicing) to form the total output image features of the robust branch. Then, this output is handed over to the subsequent fusion module or the next layer of the network.

[0046] Edges and colored dots: The black directed arrows connecting input 1, input 2, and each intermediate node, as well as the intermediate nodes themselves, represent the feature flow path. Multiple colored dots on the arrows represent the set of candidate operations on that edge; different colors indicate different operator types (e.g., ...). Figure 3 As shown), for example, a 3×3 / 5×5 depth separable convolution (corresponding to...). Figure 3 Examples of techniques include sequential convolution, 3×3 / 5×5 dilated convolution, 3×3 max / average pooling, and skip connections.

[0047] Some dots are circled with dashed lines to indicate operations that are currently selected or have high weight in the search iteration, while the remaining uncircled dots represent candidate operations that are still retained in the supernet. In this way, the diagram visually illustrates the structural feature that "multiple optional operators exist on the same edge, and the search algorithm decides which one to retain."

[0048] Through the above design, the robust branch tends to select operators with large receptive field, smoothness and redundancy to enhance robustness against natural erosion and disturbances.

[0049] Normal branch ( Figure 4(Lower dashed box): biased towards expressing details / high frequencies, improving the baseline accuracy of conventional image classification (classification / retrieval, etc.); Input 1 / Input 2: Similar to the robust branch, the normal branch also receives image features from two inputs, corresponding to the outputs of the first two cells respectively. Input 1 and Input 2 at the bottom of the figure maintain a one-to-one correspondence with those at the top, indicating that the two inputs will be fed into the normal branch and the robust branch simultaneously.

[0050] The middle nodes 0–3 and the output: The lower part also contains four blue rectangular nodes 0–3 and one output node. Its connection structure is basically isomorphic to that of the upper part, forming another independent directed acyclic graph.

[0051] Edges and candidate operations: The arrows and colored dots in the normal branch have the same meaning as the robust branch, except that the set of candidate operations is more inclined to fine convolution operators that perform better on clean images, such as 3×3 / 5×5 depth-separable convolutions; the dots circled in dashed circles are also used to indicate the currently more important or reserved operations.

[0052] In this way, the normal branch focuses on improving the characterization ability and classification accuracy on clean samples.

[0053] Each of the two branches completes node-level feature evolution, and then enters the lightweight channel attention feature fusion layer. Figure 5 The fusion layer employs a lightweight attention mechanism to adaptively adjust the proportion of the two features based on the "erosion / attack intensity" of the input image, thereby achieving adaptive routing and fusion of samples.

[0054] DARTS uses a single branch, single cell, and a single set of architecture parameters. Simultaneously serving all data; in robust training... The gradient is greatly affected by heavy tails and adversarial noise, and target conflict is obvious. The dual-branch Cell + fusion gating divides and conquers "clean targets" and "robust targets" at the structural level, and then the fusion gating performs sample-level weight allocation, which significantly reduces target conflict and gradient noise, and turns trade-off into a controllable structural parameterization problem.

[0055] Furthermore, this invention is linked with subsequent robust evaluation to estimate the contribution of "anomaly perception" to candidate operations on both branches, avoiding single noise-induced architectural decisions.

[0056] like Figure 5 As shown, this invention introduces a feature fusion layer (FusionLayer) after the dual-branch cell and inserts reduced cells (ReduceCell) at different network depths to jointly complete the adaptive fusion of multi-branch features and multi-scale feature extraction.

[0057] Feature fusion layer structure: 1. Concatenating dual-branch output with input in a normal branch: The vertical Dual-BranchCell on the left represents the overall structure of the preceding layer of bibranched cells, in which: The small orange block above is marked as , representing the output characteristics of the robust branch.

[0058] The small blue block below is marked as , representing the output characteristics of the normal branch.

[0059] The two diagonal arrows respectively and Enter "2×C" into the rectangle on the right. This input box indicates that the two features are concatenated along the channel dimension. That is, the number of channels changes from C to 2C, while the spatial size remains the same. .

[0060] 2. Lightweight Channel Attention (EfficientChannelAttention): The yellow box on the right side of the figure, labeled "EfficientChannelAttention," represents the lightweight channel attention module. The example uses an ECA (EfficientChannelAttention) structure. First, global average pooling is performed on the features after convolution / BN to obtain a global description for each channel. Then, weights for each channel are generated using one-dimensional convolution or a small MLP. Finally, weighted scaling is applied to the corresponding channels. In this way, different channels are automatically amplified or suppressed according to their importance in the current input image: for clean samples, more weights are allocated to those from... The normal branching channel; for corrosive or resistant samples, originating from The robust branch channel weights will be enhanced. "Convolutional / BN" refers to the cascaded structure of convolutional layers and batch normalization layers in a neural network.

[0061] After the ReLU→1×1Conv / BN→EfficientChannelAttention process described above, the fused feature map Hfusion is obtained, which serves as the final output of the current Dual-BranchCell and is fed into the next layer of the network. "Conv / BN" represents the cascaded module of the Convolutional Layer and the Batch Normalization Layer.

[0062] therefore, Figure 5Each box corresponds to a specific operator (activation, convolution, normalization, channel attention) within the FusionLayer (feature fusion layer). The arrows indicate the flow of features between operators. The upper and lower Hnorm / Hrob paths converge at "input 2×C" and are adaptively reweighted by the samples in the attention module, thereby achieving dynamic adjustment of the contribution of "normal branch vs. robust branch".

[0063] In one embodiment, a robust evaluator is also included, which is connected to each of the two-branch cells and is used to collect the image processing operators selected for each of the two-branch cells and perform a comprehensive robust score to screen the corresponding target network configuration.

[0064] In one embodiment, the robust evaluator is configured to perform an iterative screening strategy to search for the target network configuration, the screening strategy including the following steps: a) Based on the collected scores, for each edge in each of the two-branch cells, compare the comprehensive robust scores of each candidate image processing operator under it. b) Identify the candidate image processing operator with the lowest overall robustness score on each edge and use it as a candidate for elimination; c) When the score of a candidate image processing operator meets the preset significant disadvantage condition, it is marked as a significant disadvantage operation; d) In each round of search iteration, select several candidate image processing operators marked as elimination candidates or significant disadvantageous operations from each edge and remove them, thereby reducing the search space of the neural network architecture; e) Repeat steps a) to d) until the search space converges to determine the target network configuration.

[0065] In this embodiment, based on the supernetwork constructed in step 2, the present invention proposes the ROSE (Robus Outlier-aware Shapley Estimator) robust estimator. The robust estimator performs remote evaluation and scoring by reading the output features of each bi-branch cell and the candidate operator selection record. The robust estimator consists of a scoring module, a storage module, and a decision module, connected via a data bus, which are used to perform contribution measurement, scoring recording, and elimination decisions, respectively. A statistically robust score is given to the contribution of each candidate operation to the image classification task, which is used for subsequent architecture selection.

[0066] Contribution sample construction: Branch type An edge in a hypernet is One of its candidate operations is ; co-sampling There are 1 scenario (randomly arranged / removed order), numbered as follows: .

[0067] For each We need to obtain a set of standardized contribution samples. Then, robust estimates are made using MoM and IQR.

[0068] Clean / Adversarial accuracy gain definition: In each search iteration, from Extract a small batch of images to form a validation set containing both clean and perturbed images; for each candidate operation on each edge... Randomly generate multiple operation permutations or "remove / keep" combinations, and compute the performance metrics (such as classification accuracy or loss; if mobile devices have parameter limitations, device parameters can also be used as one of the evaluation metrics) of the feedforward computation network on the image validation set under both configurations that include and remove the operation. The performance difference between the two configurations is denoted as the value of the operation at the [number]th [position]. The marginal contributions under these scenarios are used to obtain contribution sequences for the clean scenario and the adversarial scenario, respectively. .

[0069] Cleaning accuracy gain: (Formula 1) Counteracting accuracy gains: (Formula 2) in: Under scenario s, with operation o retained, verify the accuracy of the cleaned images. This represents the scenario accuracy after removing operation 'o' under the same conditions (with the rest of the structure unchanged). adversarial scenarios , The definitions are similar.

[0070] To unify the measurement of the two objectives, "cleaning" and "adversarial," we can first perform a weighted synthesis (consistent with the composite loss during training): (Formula 3) in This is a performance weighting coefficient for clean samples, used to balance clean accuracy and adversarial accuracy.

[0071] Median-of-Means (MoM) Steady Average: make Divide into G subgroups in order. Each group is approximately [size missing] Calculate the arithmetic mean for each subgroup: (Formula 4) Then, take the median of these group means to obtain the MoM robust contribution value for this operation: (Formula 5) First, "divide the samples into buckets and then calculate the mean," and then take the median of these means. This can significantly reduce the impact of a few extreme samples on the overall estimate, and can still give an estimate that is close to the real Shapley contribution in a heavy-tailed environment such as adversarial training.

[0072] Outlier analysis and penalty factor based on IQR: In order to explicitly "record" those operations that perform extremely poorly or extremely well in very few scenarios (which are crucial for robustness), this invention uses IQR (Inter-Quartile Range) to score outlier samples in addition to MoM.

[0073] For sample set : Calculate the first and third quartiles: And order (Formula 6) Set upper and lower thresholds (γ>0 is a hyperparameter): (Formula 7) Count the number of samples that are "outstandingly good" and "outstandingly bad": (Formula 8) Define the exception penalty / reward factor: (Formula 9) like This indicates that the operation brings performance gains in most "extreme scenarios" and is an operation that contributes significantly to critical images.

[0074] like This indicates that it often lags behind on extreme images, representing a potential "weak link" in robustness, and should be penalized in searches.

[0075] Comprehensive robustness score calculation: Finally, this invention linearly combines the robust contribution of MoM (Overall Mean Method) with the outlier factor of IQR (Interquartile Range-based Outlier Detection) to obtain the comprehensive robustness score of each candidate operation on branch b and edge e: ,0< <1.(Formula 10) This reflects the stable average contribution of this operation across the "majority of images"; This reflects the criticality or risk of this operation on "a few extreme images"; This is the weighting coefficient for the anomaly factor, used to adjust the balance between steady-state performance and extreme robustness. It is typically set to 0.3–0.5, achieving a good trade-off between steady-state performance and extreme robustness (0.3 in this example). Ultimately, the contribution of each operation forms the corresponding architecture parameters. Each round outputs a set of architecture parameter matrices. The output is the network genotype code found during the search.

[0076] Candidate operation selection and architecture update based on robust scoring: After obtaining a comprehensive robust score for each candidate operation, this invention employs an iterative screening strategy to gradually shrink the search space and find a target network configuration suitable for the image classification task. For each edge in each cell, the contribution of its subordinate candidate operations is compared, and the operation with the lowest score is identified as a candidate to be eliminated. When the score of an operation is significantly lower than that of other operations on the same edge, it is marked as a "significantly inferior operation". In each round of search iteration, several operations with the lowest scores or significant inferior operations are selected from each edge and removed to reduce the search space. Eliminated candidate operations are removed from the supernetwork, and operations with higher scores are retained to continue to participate in training and evaluation. On the updated network, the ROSE evaluation process is reused to give a new round of robust scoring to the remaining candidate operations; this process is repeated until only the operation with the highest score remains on each edge, and the hypernetwork is discretized into a specific neural network architecture.

[0077] The robust image classification supernet of this invention can be summarized as follows: the input is connected to the Stem module (mapping module), which is the starting module, and the input is a constructed mini-batch of images (clean images, naturally perturbed images, or adversarial images). Figure 3The image (a hybrid of normalized and non-normalized) first passes through a Stem module consisting of one or more convolutional layers and normalization layers. This module maps the original 3-channel image to a higher-dimensional feature space, providing a unified initial representation for subsequent multi-layer Cells. The network consists of cascaded layers (layer is a hyperparameter, typically set to 10). At layers 1 / 3 and 2 / 3, ReduceCells are used to reduce the feature map resolution and expand the receptive field through stride convolutions or pooling. The remaining layers use DualBranchCells: each containing a normal branch (clean branch) and a robust branch (robust branch) structure, with feature fusion performed at the end using a channel attention fusion layer. After all Cells have been computed, the feature map from the last layer is taken, global average pooling is performed, and then a fully connected classification layer or task head is connected to obtain the prediction result for the input image. For each of the three cell types (Normal / Reduce / Robust), a separate set of edge-level architecture parameter matrices is maintained. Each row corresponds to an edge, and each column corresponds to a candidate operation. During forward computation, these... After softmax normalization, the weights of each candidate operation are obtained, which are used to perform weighted summation on multiple image processing operators to achieve "structure mixing in continuous space".

[0078] The flow of data within the supernet: Once any image enters the supernet, it will propagate forward along the path of "mapping module → dual-branch unit → feature fusion layer → multi-level reduced cell → global average pooling → classification head"; within different cells, multiple candidate operations are weighted and combined according to the current architecture parameters to form a "soft path" in the continuous structural space.

[0079] The process by which Supernet uses data to generate genotypes: Based on robust evaluation and screening of image data, this invention uses the clean graph-natural perturbation graph-adversarial graph constructed in step 1 as a validation set. Using the ROSE robust evaluation method described in step 3, multiple rounds of "marginal contribution" measurement are performed on each candidate operation of each edge in the supernet: comparing the performance difference (accuracy / loss) of the model on the image task when "keeping the operation" versus "removing the operation"; using Median-of-Means (MIQR) and IQR (interquartile range-based outlier detection) to obtain robust contribution values ​​and outlier penalty factors; these two are combined into the final contribution score, and low-scoring operations are iteratively eliminated accordingly. As the search iterates, the number of selectable operations for each edge gradually decreases, ultimately retaining only the most stable and optimal operations across various image scenarios.

[0080] When the search ends, each edge has only a few candidate operations remaining. The weights in the data already reflect the preference for each operation. This invention analyzes the three types of Cells separately: for each intermediate node i, the weight vectors of all its input edges are collected, and the maximum weight of each edge on all operations is calculated; the edges are sorted from largest to smallest according to this maximum weight, and the top two edges are selected as the actual inputs of the node; on each selected edge, the operation with the largest weight is selected from the candidate operations, and "operation type + source node number (op,j)" is recorded in the gene sequence. Through this operation, the network genotype searched for the corresponding data can be obtained. Based on the network genotype, a neural network can be constructed, retaining only the selected operations and connections, and trained from scratch or fine-tuned on the entire image training set to obtain a robust image classification model suitable for deployment; it can be transferred and used between different terminals / tasks as the final result of structure search.

[0081] Final Robust Image Classification Network Training and Usage Methods: After convergence, the operation with the highest score on each edge is fixed, resulting in the final network structure consisting of alternating stacks of normal and robust branches and the insertion of reduced cells. The network is trained on the entire training set (including clean and adversarial images). The training loss can be a weighted combination of clean loss and adversarial loss to continue balancing image classification accuracy and robustness.

[0082] During the deployment phase, the terminal or server invokes the model of this invention for image processing according to the following process: The image to be processed is acquired and scaled and normalized according to the same preprocessing procedure as in step 1; the preprocessed image is input into the trained robust network, sequentially undergoing bi-branch feature extraction and attention fusion through several normal and robust branches; at the output end, image classification results, detection results, or other image processing results are provided based on the specific task. Because the network structure has been jointly optimized for multiple scenarios ("clean image – perturbation image – adversarial image") during the search phase, it can maintain stable and reliable output even when the terminal encounters images with varying lighting, weather erosion, compressed noise, or adversarial attacks.

[0083] The innovations of this invention are as follows: Dual-branch robust architecture design: This invention innovatively introduces a robust branch and a normal branch for parallel optimization in the network structure. This dual-branch design ensures that the clean accuracy and adversarial robustness of the model are considered simultaneously during the search process, representing a significant innovation at the architectural level. The beneficial effects are: the final network architecture includes modules optimized for different scenarios, possessing both high expressive power and high resistance to perturbations. For example, the robust branch obtained by this invention tends to choose operations such as dilated convolutions and skip connections to form smoother, more redundant feature extraction paths, while the normal branch chooses operations such as separating convolutions to extract richer features. This differentiated architectural configuration makes the key feature extraction pathways more robust when facing adversarial attacks, less susceptible to disruption by single-point perturbations; at the same time, it does not lose accuracy under normal input. Compared with existing single-structure models, the dual-branch architecture of this invention significantly improves the accuracy under adversarial attacks (the improvement can reach several percentage points under the same attack intensity), while the accuracy on clean samples remains basically unchanged or even slightly improved. In summary, the dual-branch architecture design achieves an excellent balance between security and practicality, which is not available in traditional architectures.

[0084] Robust Anomaly Detection and Evaluation Mechanism: This invention introduces an evaluation method combining Median-of-Means and outlier detection (ROSE algorithm) in architecture search. This mechanism is the core innovation of this invention, enabling accurate evaluation of the true contribution of each candidate operation under adverse conditions such as adversarial attacks. The benefits include strong noise resistance and stable and reliable evaluation results. Compared with existing methods that rely on simple averaging or single-test evaluation, this scheme significantly reduces the impact of outliers on the evaluation, ensuring that the search process is not "misled." Theoretical analysis shows that the probability boundary of the deviation between the evaluated value and the actual contribution decreases exponentially (sub-Gaussian tail boundary), thus the required evaluation sample size is controllable and can be adjusted according to the expected confidence level, improving search efficiency. Simultaneously, the introduced anomaly penalty factor ensures that the evaluation metric focuses not only on average performance but also on worst-case performance, thereby guaranteeing that the selected architecture remains reliable under extreme conditions and improving the adversarial robustness of the final model.

[0085] A robust scoring-based iterative selection strategy: This invention proposes applying the robust scoring method to the elimination decision of candidate operations, gradually converging the architecture. This is another key improvement compared to traditional one-time selection schemes or selection schemes based on noisy gradient signals. Its beneficial effects are: through multiple iterations, each decision is based on more sufficient data, avoiding premature convergence and erroneous elimination in architecture search. Each iteration re-evaluates the operation, and this iterative calibration mechanism ensures that the ultimately retained operations contribute positively to performance under different scenarios. The final architecture performs excellently in both cleanliness and adversarial tests, avoiding overemphasis on one metric at the expense of another, achieving optimal performance balance. Furthermore, this strategy allows the search process to flexibly adapt to different resource conditions: if computation is ample, the number of samplings can be increased to improve evaluation accuracy; if resources are limited, the number of iterations or the size of the candidate set can be appropriately reduced to accelerate the search within an acceptable range. Therefore, this strategy improves the practicality and efficiency of NAS in robust scenarios.

[0086] In summary, this invention, through the synergistic effect of the above innovations, provides a novel scheme for searching neural network architectures in adversarial environments. Its significant advantages include: a robust and efficient search process, a model that combines accuracy and robustness, and a method with good scalability and reliability (it can be scaled in parallel to multi-GPU clusters while maintaining statistical properties). The scheme described in this technical disclosure provides ample support for subsequent patent applications.

[0087] Reference Figure 6 The present invention also provides an image classification method based on a neural network architecture for training the aforementioned neural network architecture for robust image classification on mobile devices, comprising: S001: Obtain the images to be categorized; S002 The image to be classified is input into a neural network architecture for robust image classification on mobile devices to obtain the classification result; The training steps for the neural network architecture designed for robust image classification on mobile devices include: S1: Obtain the training dataset; S2: Construct an initial super network; wherein the initial super network includes a mapping module, a reduction module, a feature extraction module, and a classification module, and each edge of each bi-branch cell in the feature extraction module is configured with a complete set of candidate image processing operators; S3: Input the training dataset into the initial super network for training, and evaluate the performance of each candidate image processing operator; S4: Based on the evaluation results, iteratively filter and remove some image processing operators from the candidate image processing operators of the initial super network to shrink the search space until the final target network configuration is determined; S5: Configure the network as a trained neural network model.

[0088] As described in steps S001-S002 above, image data is collected from various sources, such as photos taken with a smartphone, images downloaded from the internet, or images generated by an image sensor. The images to be classified should meet certain quality standards and be similar to the image types used during model training to ensure classification effectiveness. The acquired images to be classified are input into a neural network architecture designed for robust image classification on mobile devices. After inputting the images, the network undergoes a series of layered processing steps, including feature extraction, downsampling, and classification modules. Each layer analyzes different features of the image to extract category-related information. During processing, the network generates the final classification result by activating neurons between layers based on the knowledge gained during training.

[0089] As described in steps S1-S5 above, the first step is to obtain a training dataset containing the target task labels. This dataset can be obtained by collecting image data from a specific domain or by using publicly available datasets (such as ImageNet and CIFAR-10). Each image should be accompanied by a corresponding category label to ensure that the set used for training is of high quality and diversity. This step provides the necessary data foundation for subsequent training, ensuring that the network can learn features that support its efficient performance.

[0090] After determining the training dataset, the next step is to build the initial super network. The initial network design must include multiple functional modules, including a mapping module, a reduction module, a feature extraction module, and a classification module. This structured design allows the network to extract features layer by layer, as follows: Mapping module: Responsible for extracting initial features from the input image, typically achieved through convolutional layers and normalized cells. Reduction module: Aims to gradually reduce the spatial size of the feature map through downsampling operations, while enhancing the representational power of the features, making the network more efficient. Feature extraction module: This module is further subdivided into two independent bi-branch cells, each with a complete set of candidate image processing operators configured on each edge. This design allows the network to extract features from different perspectives, balancing classification accuracy and robustness. Classification module: Finally, it transforms the fused feature information into a specific classification result.

[0091] After the initial supernetwork is built, the acquired training dataset is input into the network for training. A key step in this process is evaluating the performance of each candidate image processing operator. Through forward propagation and loss function calculation, the network gradually adjusts its parameters, thereby continuously improving its ability to recognize training data. During training, it is necessary not only to continuously optimize the network weights but also to evaluate the effects of different operations, which provides a basis for subsequent selection operations.

[0092] An iterative approach is used to optimize the network structure. Based on the evaluation results of the previous step, candidate operations with poor performance are identified and eliminated. Specifically, the algorithm selects and removes corresponding operators according to certain conditions (such as performance below a certain threshold or significant disadvantage). Through this process, the network's search space is gradually reduced, thereby driving the network towards a better configuration and ensuring that a stable and accurate target network architecture is ultimately obtained.

[0093] Finally, after multiple iterations of selection and optimization, the network structure was finalized, and the output of this configuration can be used as the trained neural network model. This model, after complete training and evaluation, possesses the ability to perform robust image classification in specific mobile environments. The final model not only maintains high accuracy on clear input images but also demonstrates good stability in the face of various natural perturbations and adversarial attacks.

[0094] In one embodiment, step S5, which iteratively filters and removes some image processing operators from the candidate image processing operators of the initial supernetwork based on the evaluation results to narrow the search space until the final target network configuration is determined, includes: S501: Obtain the performance evaluation results of each candidate image processing operator on each edge of each bi-branch cell in the initial supernetwork based on the training dataset; S502: For each edge of each bi-branch cell, compare the performance evaluation results of each candidate image processing operator; S503: Based on the comparison results, identify candidate image processing operators on each side that meet the preset elimination conditions; S504: Remove the identified candidate image processing operators that meet the preset elimination conditions to update the initial super network and shrink the search space; S505: Determine whether the updated search space meets the preset convergence condition; S506: If not, then based on the updated supernetwork, return to the step of inputting the training dataset for training and evaluation; S507: If so, the final target network configuration is determined based on the remaining candidate image processing operators on each edge of the initial super network.

[0095] As described in steps S501-S507 above, the network obtains the performance evaluation results of each candidate image processing operator from the training dataset. A systematic measurement is performed on all candidate operators within each bi-branch cell of the initial supernetwork. Multiple rounds of evaluation using the training dataset ensure that the collected information is comprehensive and objective. In this way, the network has data support for subsequent decisions and can filter candidates based on their specific performance.

[0096] For each edge of each bi-branch cell, the performance evaluation results of all candidate operators are systematically compared. This comparison allows the network to identify which operations perform well and which perform poorly when processing the training dataset. By quantifying performance differences, such as key metrics like accuracy and loss, the network can prepare for more evidence-based elimination decisions. After the comparison, the system identifies candidate operators on each edge according to preset elimination criteria. These preset elimination criteria typically include, but are not limited to: candidate operator performance below a certain threshold, the lowest evaluation result among all candidates, and operators with a small overall contribution to the model. The importance of this step lies in clearly identifying those operators that play a small role in model training through this intelligent aggregation, thus preparing for subsequent removal.

[0097] After identification, the system will perform the actual removal operation. This operation helps update the structure of the initial supernetwork and narrow its search space. By removing candidate operators that underperform, the network can gradually eliminate redundant parts and refocus on the most promising operator combinations. This process not only improves computational efficiency but also enhances the accuracy of subsequent training. After updating the network structure, the system needs to check whether the current search space meets the convergence criteria. Convergence criteria may include the network performance ceasing to improve significantly or reaching a certain expected accuracy. This step confirms whether the network has enough new information to process, ensuring that subsequent training and evaluation are meaningful.

[0098] If the current search space does not meet the convergence criteria, the system will return to the training process, based on the updated supernetwork. Training data is then fed into the network for the next round of training and performance evaluation. This feedback mechanism is similar to iteration, progressively optimizing the network in each round, collecting more reliable and detailed performance data, allowing the network to continue to improve itself. If the current search space is confirmed to meet the convergence criteria, the final step is to determine the target network configuration. This configuration will be based on the valid candidate image processing operators remaining on each filtered edge. This final network configuration will serve as the optimized structure obtained after training, used for subsequent image classification tasks.

[0099] In one embodiment, step S501, which involves obtaining the performance evaluation results of each candidate image processing operator on each edge of each bi-branch cell in the initial supernetwork based on the training dataset, includes: S5011: Obtain and, based on the validation set, perform multiple rounds of performance contribution measurement on each candidate image processing operator for each edge in the current super network; wherein, the performance contribution measurement is achieved by comparing the performance index difference of the super network on the image classification task when the candidate image processing operator is retained in the super network and when the candidate image processing operator is removed. S5012: Based on the results of the multi-round performance contribution measurement, a preset robust statistical algorithm is used to calculate a comprehensive robust score for each candidate image processing operator, which is used as the performance evaluation result.

[0100] As described in steps S5011-S5012 above, by performing multiple rounds of performance contribution measurement on each candidate image processing operator for each edge, researchers can obtain the relative performance of each operator. This process is based on a validation set, which is a portion separated from the training dataset used to evaluate the model's generalization ability. In this step, the performance contribution measurement method used is to compare the performance difference of the current supernetwork when retaining a specific candidate image processing operator versus removing that operator. The core of this method is that it directly quantifies the impact of each operator on the performance of the image classification task through actual training and testing experiments. For example, assuming that a candidate image processing operator A performs well in the supernetwork, the system will first retain this operator in the supernetwork and calculate the model's classification accuracy on the validation set. Then, the system will remove the operator and recalculate the model's accuracy on the same validation set. If removing operator A significantly reduces the model's accuracy, it indicates that operator A makes a significant contribution to the model's overall performance, and the performance contribution metric will assign operator A a higher score. Conversely, if the accuracy does not change significantly, the contribution metric score for operator A will be lower. Multiple rounds of experiments can further reduce the impact of random factors on the evaluation results, making the obtained performance contribution metric more robust and reliable.

[0101] The results of multiple rounds of performance contribution measurements are combined into a robust score. In cases with multiple measurements, a simple average may be affected by outliers; therefore, a pre-defined robust statistical algorithm is used to calculate the overall robust score for each candidate operator. This robust statistical algorithm typically employs methods such as median, quantiles, or other statistical methods to reduce the impact of outliers and make the final score more representative. The calculation result of the robust score forms an important performance evaluation result, which directly affects the subsequent model architecture search and operator selection process. Through these two steps, we can evaluate and select the image processing operators that perform best in practical applications, thereby optimizing the architecture of the supernetwork and ultimately improving the accuracy and efficiency of image classification.

[0102] In one embodiment, in step S503 of identifying candidate image processing operators that meet preset elimination conditions on each edge based on the comparison results, the preset elimination conditions are: the comprehensive robustness score of the candidate image processing operator is lower than a preset threshold or is the lowest score among all candidate image processing operators on the same edge.

[0103] In this embodiment, the comprehensive robustness score is obtained by statistically analyzing the performance contribution of candidate operators in multiple experiments. This score reflects the stability and effectiveness of the operator. When the comprehensive robustness score of a candidate operator is lower than a preset threshold, it indicates that the operator performs poorly in multiple validation experiments and may not provide sufficient performance improvement in practical applications. Therefore, elimination based on this score can effectively exclude potentially "inefficient" operators, ensuring that the operators retained in the model architecture can play a good role in specific tasks. Secondly, the operator selection process is further improved based on the elimination condition of "the lowest score among all candidate image processing operators on the same edge". On the same edge, multiple candidate downstream processing operators may exhibit different efficiencies. If the score of an operator is the lowest among all operators on this edge, then the impact of this operator on the network performance is obviously minimal. Therefore, eliminating it is reasonable. This condition ensures that each edge of the super network retains as many high-performing operators as possible to optimize the overall network performance.

[0104] In one embodiment, step S1 of obtaining the training dataset includes: S101: Obtain the original image dataset containing the target task labels; S102: Perform preprocessing operations on the original image dataset to obtain a clean image dataset; the preprocessing operations include resizing, cropping, and color normalization. S103: Input the clean image dataset into a preset image degradation operator to generate multiple naturally perturbed version images to form a naturally perturbed image dataset; wherein, each clean image and its corresponding naturally perturbed version images maintain the same target task label; S104: An adversarial attack method based on gradient symbols is adopted to process the clean image dataset within a preset perturbation constraint to generate corresponding adversarial sample images to form an adversarial image dataset; wherein, each clean image and its corresponding adversarial sample image maintain the same target task label; S105: Extract the clean image, at least one naturally perturbed version image, and adversarial example image corresponding to each original image and associate them into a sample group to obtain the training dataset consisting of all the sample groups.

[0105] As described in steps S101-S105 above, obtaining the original image dataset containing the target task labels is the starting point for constructing the entire training dataset. These original images should be closely related to the required task (such as image classification, object detection, etc.), and each image should be accompanied by corresponding label information. This process ensures that subsequent operations have clear guiding objectives and provides the necessary baseline data for training.

[0106] After acquiring the raw dataset, preprocessing operations are performed. Preprocessing primarily includes resizing, cropping, and color normalization. These operations aim to eliminate inconsistencies or unnecessary complexity in the original data, making it more suitable for model training. Resizing and cropping ensure that all input images have the same size, conforming to the standard format required by deep learning models. Color normalization helps reduce the impact of factors such as lighting and contrast in the shooting environment, enhancing the model's robustness under different lighting conditions. This process ultimately yields a clean image dataset.

[0107] After obtaining the clean image dataset, each clean image is processed using a pre-defined image degradation operator to generate multiple naturally perturbed versions. These naturally perturbed versions may take into account various noises and interferences common in the real world (such as blurring, noise, compression distortion, etc.), enabling the model to learn richer features. During this process, each naturally perturbed image retains the same target task label as its original clean image. This data augmentation technique not only expands the diversity of the training dataset but also improves the model's generalization ability, allowing it to make accurate predictions even when faced with various interferences in real-world applications.

[0108] Next, a gradient-symmetric adversarial attack method is employed to process the clean image dataset to generate corresponding adversarial example images. Adversarial examples are generated by adding small perturbations to the input data. These perturbations are deliberately designed to cause the deep learning model to output incorrect results when processing this data. Similar to the naturally perturbated versions, each clean image and its corresponding adversarial example retain the same target task label. The goal of generating adversarial examples is to improve the model's robustness, making it more resistant to potential hostile attacks or irregular data.

[0109] Finally, each original image is associated with its corresponding clean image, at least one naturally perturbed version image, and adversarial example image to form a sample group. In this way, the constructed training dataset contains multiple sample groups, each representing the same task label, but enriching the data upon which the model is trained through different image versions. This construction method aims to maximize the utilization of data variations under different perturbations while ensuring label consistency, thereby effectively improving the quality and efficiency of model training.

Claims

1. A neural network architecture for robust image classification on mobile devices, characterized in that, The neural network architecture includes a mapping module, a reduction module, a feature extraction module, and a classification module connected in sequence. The mapping module includes multiple first convolutional layers and normalized cells. The multiple first convolutional layers are connected in sequence, and the last first convolutional layer is connected to the normalized cells to extract the initial features of the input image. The reduction module includes multiple reduction cells connected in sequence. The first reduction cell is connected to the normalized cell. Each reduction cell processes the initial feature through a downsampling operation to obtain the reduced intermediate feature. The feature extraction module includes a normal branch, a robust branch, and a feature fusion layer. The normal branch and the robust branch are respectively connected to the last-ranked reduced cell. The normal branch and the robust branch are respectively connected to the feature fusion layer. The normal branch is used to extract high-frequency operators from the intermediate features to obtain precision-friendly operators. The robust branch is used to extract redundant operators from the intermediate features to obtain robust-friendly operators. The feature fusion layer is used to fuse the precision-friendly operators and the robust-friendly operators to obtain fused features. The classification module is used to classify the fused features.

2. The neural network architecture for robust image classification on mobile devices according to claim 1, characterized in that, Both the normal branch and the robust branch in the feature extraction module include at least one bibranched cell; Each of the bi-branch cells is formed by multiple edges connected together, and each edge is configured with an image processing operator selected from a preset set of candidate operators; wherein, the set of candidate operators for bi-branch cells in the normal branch includes a first type of operator, which is configured to extract fine-grained high-frequency features; the set of candidate operators for bi-branch cells in the robust branch includes a second type of operator, which is configured to extract smooth and redundant features; The precision-friendly operator and the robust-friendly operator are obtained by combining image processing operators selected from the first type of operator and the second type of operator on each edge of the bi-branched cell, respectively.

3. The neural network architecture for robust image classification on mobile devices according to claim 2, characterized in that, The structure of the feature extraction module specifically includes: Multiple bi-branch cells are cascaded sequentially, and a feature fusion layer is disposed between every two adjacent bi-branch cells; wherein, the feature output by the previous bi-branch cell is adaptively fused by the feature fusion layer and then passed as input to the next bi-branch cell; Multiple reduced cells in the reduction module are inserted between sequences composed of multiple bi-branched cells to perform downsampling operations at multiple depths of the network, thereby achieving multi-scale feature extraction and obtaining the fused features.

4. The neural network architecture for robust image classification on mobile devices according to claim 2, characterized in that, It also includes a robust evaluator connected to each of the two-branch cells, which collects and scores the image processing operators selected for each of the two-branch cells in order to filter the corresponding target network configuration.

5. The neural network architecture for robust image classification on mobile devices according to claim 4, characterized in that, The robust evaluator is configured to perform an iterative screening strategy to search for the target network configuration, the screening strategy comprising the following steps: a) Based on the collected scores, for each edge in each of the two-branch cells, compare the comprehensive robust scores of each candidate image processing operator under it. b) Identify the candidate image processing operator with the lowest overall robustness score on each edge and use it as a candidate for elimination; c) When the score of a candidate image processing operator meets the preset significant disadvantage condition, it is marked as a significant disadvantage operation; d) In each round of search iteration, select several candidate image processing operators marked as elimination candidates or significant disadvantageous operations from each edge and remove them, thereby reducing the search space of the neural network architecture; e) Repeat steps a) to d) until the search space converges to determine the target network configuration.

6. A neural network-based image classification method, used to train the neural network architecture for robust mobile image classification as described in any one of claims 1-5, characterized in that, include: Get the images to be categorized; The image to be classified is input into a neural network architecture for robust image classification on mobile devices to obtain the classification result; The training steps for the neural network architecture designed for robust image classification on mobile devices include: Obtain the training dataset; Construct an initial super network; wherein the initial super network includes a mapping module, a reduction module, a feature extraction module, and a classification module, and each edge of each bi-branch cell in the feature extraction module is configured with a complete set of candidate image processing operators; The training dataset is input into the initial supernetwork for training, and the performance of each candidate image processing operator is evaluated. Based on the evaluation results, some image processing operators are iteratively filtered and removed from the candidate image processing operators of the initial super network to shrink the search space until the final target network configuration is determined. The network configuration is used as the trained neural network model.

7. The image classification method based on a neural network architecture according to claim 6, characterized in that, The step of iteratively filtering and removing some image processing operators from the candidate image processing operators of the initial supernetwork based on the evaluation results to narrow the search space until the final target network configuration is determined includes: Obtain the performance evaluation results of each candidate image processing operator on each edge of each bi-branch cell in the initial supernetwork based on the training dataset; For each edge of each bi-branch cell, compare the performance evaluation results of each candidate image processing operator; Based on the comparison results, identify the candidate image processing operators that meet the preset elimination conditions on each edge; Remove the identified candidate image processing operators that meet the preset elimination conditions to update the initial super network and shrink the search space; Determine whether the updated search space meets the preset convergence criteria; If not, then based on the updated supernetwork, return to the step of inputting the training dataset for training and evaluation; If so, the final target network configuration is determined based on the remaining candidate image processing operators on each edge of the initial supernetwork.

8. The image classification method based on a neural network architecture according to claim 7, characterized in that, The step of obtaining the performance evaluation results of each candidate image processing operator on each edge of each bi-branch cell in the initial supernetwork based on the training dataset includes: Based on the validation set, multiple rounds of performance contribution measurement are performed on each candidate image processing operator for each edge in the current super network; wherein, the performance contribution measurement is achieved by comparing the performance index difference of the super network on the image classification task when the candidate image processing operator is retained in the super network and when the candidate image processing operator is removed. Based on the results of the multi-round performance contribution measurement, a preset robust statistical algorithm is used to calculate a comprehensive robust score for each candidate image processing operator, which serves as the performance evaluation result.

9. The image classification method based on a neural network architecture according to claim 8, characterized in that, In the step of identifying candidate image processing operators that meet the preset elimination conditions on each side based on the comparison results, the preset elimination conditions are: the comprehensive robustness score of the candidate image processing operator is lower than a preset threshold or is the lowest score among all candidate image processing operators on the same side.

10. The image classification method based on a neural network architecture according to claim 6, characterized in that, The steps for obtaining the training dataset include: Obtain the original image dataset containing the target task labels; Preprocessing operations are performed on the original image dataset to obtain a clean image dataset; the preprocessing operations include resizing, cropping, and color normalization. The clean image dataset is input into a preset image degradation operator to generate multiple naturally perturbed version images to form a naturally perturbed image dataset; wherein each clean image and its corresponding naturally perturbed version images maintain the same target task label; An adversarial attack method based on gradient symbols is adopted to process the clean image dataset within a preset perturbation constraint to generate corresponding adversarial sample images to form an adversarial image dataset; wherein, each clean image and its corresponding adversarial sample image maintain the same target task label; Each original image is associated with a clean image, at least one naturally perturbed version image, and an adversarial example image, and thus a training dataset consisting of all the sample groups is obtained.

Images