Neural network architecture and classification method for mobile terminal robust picture classification

Abstract

The application relates to the technical field of neural network architecture for mobile terminal robust picture classification, and discloses a neural network architecture for mobile terminal robust picture classification and a classification method thereof, wherein the neural network architecture comprises a hierarchical structure of a mapping module, a reduction module, a feature extraction module and a classification module, can efficiently process an input picture, extract initial features and optimize layer by layer, normal branches and robust branches in the feature extraction module perform operations on high-frequency information and redundant information respectively, and a feature fusion layer integrates the outputs of the two branches. The application has the beneficial technical effects that, under the premise of meeting the mobile terminal computing power constraint, the clean precision and the anti / natural disturbance robustness are considered at the same time. The accuracy and the robustness of the mobile terminal picture classification system are effectively improved, the application is suitable for complex scenes in actual application, and the application has a good market prospect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural network architecture for mobile terminal robust picture classification, and particularly relates to a neural network architecture for mobile terminal robust picture classification and a classification method thereof. BACKGROUND

[0002] Current neural network architecture search (NAS) techniques have been widely used to automatically design deep learning models, especially in mobile terminal and edge device scenarios, by automatically searching network structures under given computing power and delay constraints, thereby obtaining higher classification accuracy and lower inference time delay. Existing NAS methods usually take the average accuracy on clean datasets as the main optimization target, and combine model parameter quantity, floating point operation quantity, and terminal delay indicators for joint constraint, and have been applied in image classification, target detection and other tasks. For example, the technology route represented by the gradient-based automatic network building method (such as DARTS: Differentiable Architecture Search) can efficiently search the structure of an image classification model.

[0003] However, in actual application environments, input images are often not ideal clean samples. On the one hand, images collected by mobile terminals are easily affected by factors such as light changes, blurring, noise, and compression distortion; on the other hand, in content review, identity authentication, security monitoring and other scenarios, attackers can significantly interfere with model prediction results under the premise that human eyes are almost difficult to detect by carefully constructing adversarial perturbations. Such perturbations make the model highly sensitive to small changes in the input space, resulting in dramatic fluctuations in the prediction results within a reasonable perturbation range, which seriously affects the safety and reliability of the system. As shown in FIG. 1, the upper row is the common weather erosion (clear, frost, snow, fog, rain), and the lower row is the image distortion caused by typical adversarial attacks and implementation differences, which will significantly affect the discrimination results of traditional models. Figure 1

[0004] To improve the performance of the model in the above complex environment, the industry has proposed various adversarial training and robust optimization methods to improve the adversarial accuracy by introducing adversarial samples or robustness regular terms on fixed network structures. However, a large number of studies have shown that the network structure itself has a significant impact on robustness: different cell topologies, convolution types, and connection methods will cause essential differences in the smoothness of the decision boundary and the classification interval. When only fixed traditional lightweight network structures (such as several hand-designed mobile terminal models) are used, even if various robust training strategies are introduced, there is still a clear performance upper limit and a strict trade-off relationship between the robust accuracy, clean accuracy, and computational overhead of the model.

[0005] ​Therefore, "robustness" is incorporated into the network architecture design goal from the beginning, while considering the terminal computing power and delay constraints, automatically searching for a neural network structure more suitable for robust picture classification scenarios. Based on the automation capability of neural network architecture search (NAS), by introducing robustness evaluation indicators and adversarial perturbation models in the search process, a class of network architectures with robust performance and deployment efficiency can be explored in the super network search space, thereby providing an end-to-end structure generation means for mobile and edge-side robust picture classification. However, in the case of adversarial attack robustness, the existing technology has the following shortcomings:

[0006] Evaluation indicators are easily affected by abnormal fluctuations: In the adversarial training environment, the gradient signal and performance indicators often show heavy-tailed distribution, and a small number of abnormal extreme values may occur. Traditional NAS directly uses average performance or gradient as operation evaluation, which is easily disturbed by these abnormal values. For example, some candidate operations perform normally in most cases, but cause a sharp drop in performance under individual adversarial samples. If only the average value is taken, the true contribution of this operation may be underestimated, causing incorrect architecture selection. Existing methods such as AdvRush (AdvRush: Searching for Adversarially Robust Neural Architectures, neural architecture search for adversarial robustness), LRNAS (LRNAS: Differentiable Searching for Adversarially Robust Lightweight Neural Architecture, differentiable search for adversarially robust lightweight neural architecture) do not fully solve this problem, resulting in unstable search process: a small number of extreme samples may cause high-quality operations to be eliminated too early or low-quality operations to be incorrectly retained, affecting the robustness and accuracy of the target network configuration.

[0007] It is difficult to balance clean accuracy and adversarial robustness: Existing robust architecture search often sacrifices part of the clean sample accuracy when improving adversarial robustness. Traditional methods lack a unified measure of "double targets" and often cannot guarantee high accuracy without attacks and high robustness with attacks. This is because existing technologies are usually based on single indicator optimization (e.g., only optimizing adversarial loss), resulting in a decline in model performance in non-attack scenarios. How to balance the two performances and design a network that performs well in both conditions is another challenge faced by existing technologies.

[0008] Search efficiency and stability are poor: Adversarial training itself has large computational overhead, and some methods require a large number of sample averages to obtain stable evaluation, increasing search costs. In addition, existing methods lack theoretical guarantees and cannot quantitatively evaluate the relationship between the required sample size and the evaluation error. Only empirical parameter tuning is available. This makes the search process lack reliability: developers have difficulty in determining whether the search has converged or the results are reliable in a timely manner, which may cause repeated trial and error and waste of computing resources. SUMMARY

[0009] Therefore, it is necessary to propose a neural network architecture and a classification method for mobile terminal robust picture classification for the existing mobile terminal robust picture classification neural network architecture search problem.

[0010] A neural network architecture for mobile terminal robust picture classification, the neural network architecture comprising a mapping module, a reduction module, a feature extraction module and a classification module connected in turn;

[0011] The mapping module comprises a plurality of first convolutional layers and a normalization cell, the plurality of first convolutional layers are connected in turn, and the last first convolutional layer is connected with the normalization cell, for extracting initial features of an input picture;

[0012] The reduction module comprises a plurality of reduction cells connected in turn, the first reduction cell is connected with the normalization cell through sorting, and each reduction cell processes the initial features through a downsampling operation to obtain reduced intermediate features;

[0013] The feature extraction module comprises a normal branch, a robust branch and a feature fusion layer, the normal branch and the robust branch are respectively connected with the last reduction cell through sorting, and the normal branch and the robust branch are respectively connected with the feature fusion layer, the normal branch is used for high-frequency operator extraction on the intermediate features to obtain precision-friendly operators, the robust branch is used for redundant operator extraction on the intermediate features to obtain robust-friendly operators, and the feature fusion layer is used for fusing the precision-friendly operators and the robust-friendly operators to obtain fused features;

[0014] The classification module is used for classifying the fused features.

[0015] Further, the normal branch and the robust branch in the feature extraction module each comprise at least one double-branch cell;

[0016] Each of the double-branch cells is formed by a plurality of edges, and each edge is configured with a picture processing operator selected from a preset candidate operator set; wherein the candidate operator set of the double-branch cell in the normal branch contains a first type of operator configured to extract fine-grained high-frequency features; and the candidate operator set of the double-branch cell in the robust branch contains a second type of operator configured to extract smooth and redundant features.

[0017] The precision-friendly operator and the robust-friendly operator are obtained by respectively combining the picture processing operators selected from the first type of operator and the second type of operator on each edge of the double-branch cell.

[0018] Further, the structure of the feature extraction module specifically includes:

[0019] The plurality of double-branch cells are sequentially cascaded, and a feature fusion layer is arranged between each two adjacent double-branch cells; wherein the features output by a previous double-branch cell are adaptively fused by the feature fusion layer and then input to a next double-branch cell.

[0020] The plurality of reduction cells in the reduction module are inserted between the sequence of the plurality of double-branch cells to perform down-sampling operations at multiple depths of the network, so as to realize multi-scale feature extraction and obtain the fused features.

[0021] Further, a robust evaluator is further included, which is connected with each of the double-branch cells and configured to collect the picture processing operators selected for each of the double-branch cells and score them to screen the target network configuration.

[0022] Further, the robust evaluator is configured to search for the target network configuration by executing an iterative screening strategy, and the screening strategy includes the following steps:

[0023] a) Based on the collected scores, for each edge in each of the double-branch cells, the comprehensive robust scores of the candidate picture processing operators under the edge are compared;

[0024] b) The candidate picture processing operator with the lowest comprehensive robust score on each edge is identified as a eliminated candidate;

[0025] c) When the score of a candidate picture processing operator meets a preset significant disadvantage condition, the candidate picture processing operator is marked as a significant disadvantage operation;

[0026] d) In each search iteration, a plurality of candidate picture processing operators marked as the eliminated candidate or the significant disadvantage operation are removed from each edge, so as to narrow the search space of the neural network architecture;

[0027] e) repeating steps a) to d) until the search space converges to determine the target network configuration.

[0028] A picture classification method based on a neural network architecture, for training the neural network architecture for mobile-oriented robust picture classification, comprising:

[0029] obtaining a picture to be classified;

[0030] inputting the picture to be classified into the neural network architecture for mobile-oriented robust picture classification to obtain a classification result;

[0031] wherein the training steps of the neural network architecture for mobile-oriented robust picture classification comprise:

[0032] obtaining a training data set;

[0033] constructing an initial super network; wherein the initial super network comprises a mapping module, a reduction module, a feature extraction module, and a classification module, and each edge of each double-branch cell in the feature extraction module is configured with a complete set of candidate picture processing operators;

[0034] inputting the training data set into the initial super network for training and evaluating the performance of each candidate picture processing operator;

[0035] based on the evaluation results, iteratively screening and removing part of the picture processing operators from the candidate picture processing operators of the initial super network to shrink the search space until the final target network configuration is determined;

[0036] the network configuration is used as the trained neural network model.

[0037] Further, the step of based on the evaluation results, iteratively screening and removing part of the picture processing operators from the candidate picture processing operators of the initial super network to shrink the search space until the final target network configuration is determined, comprises:

[0038] obtaining the performance evaluation results of each candidate picture processing operator on each edge of each double-branch cell in the initial super network based on the training data set;

[0039] for each edge of each double-branch cell, comparing the performance evaluation results of each candidate picture processing operator thereon;

[0040] according to the comparison results, identifying the candidate picture processing operators on each edge that meet the preset elimination conditions;

[0041] removing the identified candidate picture processing operators that meet the preset elimination conditions to update the initial super network and shrink the search space;

[0042] determining whether the updated search space satisfies a preset convergence condition;

[0043] If not, returning to the step of inputting the training data set into the training and evaluation based on the updated super network;

[0044] If yes, determining the final target network configuration according to the remaining candidate image processing operators on each edge in the initial super network.

[0045] Further, the step of obtaining each candidate image processing operator on each edge of each double-branch cell in the initial super network based on the performance evaluation result of the training data set comprises:

[0046] obtaining and performing multi-round performance contribution measurement on each candidate image processing operator of each edge in the current super network based on the validation set; wherein the performance contribution measurement is realized by comparing the performance index difference of the super network on the image classification task when the candidate image processing operator is retained and removed in the super network;

[0047] Based on the results of the multi-round performance contribution measurement, a comprehensive robust score is calculated for each candidate image processing operator using a preset robust statistical algorithm, which is used as the performance evaluation result.

[0048] Further, in the step of identifying the candidate image processing operators on each edge that meet the preset elimination condition according to the comparison result, the preset elimination condition is that the comprehensive robust score of the candidate image processing operator is lower than a preset threshold or is the lowest among all candidate image processing operators on the same edge.

[0049] Further, the step of obtaining the training data set comprises:

[0050] obtaining an original image data set containing target task labels;

[0051] performing a preprocessing operation on the original image data set to obtain a clean image data set; the preprocessing operation includes size adjustment, cropping and color normalization;

[0052] inputting the clean image data set into a preset image degradation operator to generate corresponding multiple natural disturbance version images to constitute a natural disturbance image data set; wherein each clean image and its corresponding natural disturbance version images maintain the same target task label;

[0053] The gradient sign-based adversarial attack method is used to process the clean picture dataset under a preset perturbation constraint, and corresponding adversarial sample pictures are generated to constitute an adversarial picture dataset; wherein each clean picture and its corresponding adversarial sample picture maintain the same target task label.

[0054] The clean picture, at least one natural perturbation version picture and the adversarial sample picture corresponding to each original picture are associated as a sample group, so as to obtain the training dataset composed of all the sample groups.

[0055] The architecture adopts a hierarchical structure of a mapping module, a reduction module, a feature extraction module and a classification module, can efficiently process input pictures, extract initial features and optimize layer by layer, and realizes end-to-end inference from input pictures to classification output. This design not only improves the processing speed of the model, but also enhances its adaptability to different image scales and complexities. The normal branch and the robust branch in the feature extraction module respectively perform operations on high-frequency information and redundant information, ensuring a good balance between accuracy and robustness of the model. The normal branch focuses on extracting key features, while the robust branch enhances the anti-interference ability of the model to noise and interference, which enables the model to maintain high classification accuracy in various environments. The feature fusion layer effectively integrates the outputs from the two branches, improving the overall performance of the classification module. In summary, the present application meets the mobile terminal computing power constraint, and simultaneously considers clean accuracy and adversarial / natural perturbation robustness. It effectively improves the precision and robustness of the mobile terminal picture classification system, is suitable for complex scenes in practical applications, and has good market prospects.

[0056] The present application innovatively introduces a robust branch and a normal branch in parallel optimization in the network structure. This dual-branch design ensures that the clean accuracy and adversarial robustness of the model are considered simultaneously during the search process, which is an important innovation at the architecture level. The beneficial effects are that the final output network architecture contains modules optimized for different scenarios, with high expression ability and high anti-interference ability. For example, the robust branch searched by the present application tends to select operations such as dilated convolution and skip connection to form a smoother and more redundant feature extraction path, while the normal branch selects operations such as separable convolution to extract rich features. This differentiated architecture configuration makes the model more robust in key feature extraction paths when facing adversarial attacks, and is not easily damaged by single-point perturbations. At the same time, the accuracy is not lost under normal input. Compared with existing single-structure models, the dual-branch architecture of the present application significantly improves the accuracy index under adversarial attacks (the improvement amplitude can reach several percentage points under the same attack intensity), while the accuracy on clean samples remains basically unchanged or even slightly improves. In summary, the dual-branch architecture design enables the model to achieve an excellent balance between security and practicality, which is not possessed by traditional architectures. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0058] Wherein:

[0059] Figure 1 An example diagram of image disturbance in an embodiment;

[0060] Figure 2 A comparison diagram of the overall structure of a super network and a normal network in an embodiment;

[0061] Figure 3 An example diagram of a picture processing operator in an embodiment;

[0062] Figure 4 A branch diagram of a double-branch cell structure in an embodiment;

[0063] Figure 5 A schematic diagram of a feature fusion layer structure in an embodiment;

[0064] Figure 6 A flowchart of a neural network architecture for mobile terminal-oriented robust picture classification in an embodiment. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0066] The neural network architecture for mobile terminal-oriented robust picture classification comprises a mapping module, a reduction module, a feature extraction module and a classification module connected in sequence; the mapping module comprises a plurality of first convolutional layers and a normalization cell, the plurality of first convolutional layers are connected in sequence, the last first convolutional layer is connected with the normalization cell, and the normalization cell is used for extracting initial features of an input picture; the reduction module comprises a plurality of reduction cells connected in sequence, a first reduction cell is connected with the normalization cell through sequencing, each reduction cell processes the initial features through a downsampling operation to obtain intermediate features after reduction; the feature extraction module comprises a normal branch, a robust branch and a feature fusion layer, the normal branch and the robust branch are connected with the last reduction cell through sequencing, the normal branch and the robust branch are connected with the feature fusion layer, the normal branch is used for extracting high-frequency operators from the intermediate features to obtain precision-friendly operators, the robust branch is used for extracting redundant operators from the intermediate features to obtain robust-friendly operators, and the feature fusion layer is used for fusing the precision-friendly operators and the robust-friendly operators to obtain fused features; and the classification module is used for classifying the fused features.

[0067] In this embodiment, the architecture adopts a systematic design, aiming to collaboratively optimize the classification accuracy and robustness against interference / natural disturbances of the model under the constraints of limited computing resources on mobile devices, ultimately obtaining a lightweight yet reliable visual model. Specifically, the traditional single feature processing path is decoupled into two parallel and focused branches: "accuracy optimization" and "robustness enhancement." Through subsequent intelligent fusion, the model possesses both fine-grained discriminative power and anti-interference stability. The overall workflow follows the principle of coarse-to-fine and divide-and-conquer fusion, as follows: First, the mapping module, as the network entry point, performs a series of basic convolution and normalization operations on the input image to perform initial feature purification and standardization, providing a clean and standardized data foundation for subsequent complex processing. Subsequently, the features enter the reduction module. This module consists of multiple downsampling cells connected in series. Its core function is to perform feature abstraction and dimensionality management, gradually reducing the spatial size of the feature map to compress the computational load, while increasing the number of channels to enrich semantic information, thereby achieving a balance between efficiency and expressive power, producing intermediate features suitable for deep processing on mobile devices. The feature extraction module receives intermediate features from the reduction module and feeds them simultaneously into two structurally independent, functionally specialized parallel branches: The Normal Branch focuses on extracting high-frequency details and fine-grained features crucial for classification accuracy. Internally, it tends to configure or search for processing operators that capture subtle image textures and edge information (“accuracy-friendly operators”), aiming to improve the model's recognition accuracy on ideally clear images. The Robust Branch focuses on extracting smooth structures and redundant features that remain stable under perturbations. Internally, it tends to configure or search for processing operators that are insensitive to noise, blur, and pixel changes (“robust-friendly operators”), aiming to strengthen the model's performance retention capabilities in the face of natural degradation or adversarial attacks. The outputs of the two branches, carrying the complementary characteristics of “accuracy” and “robustness”, are fed into the feature fusion layer. This layer organically integrates the two types of features using specific algorithms (such as concatenation, weighted summation, or attention-based adaptive fusion) to generate fused features that simultaneously possess high discriminative power and strong stability. Finally, the classification module receives the fused features and uses operations such as global pooling and fully connected layers to aggregate the rich feature information into the final category probability distribution, thus completing the image classification task.

[0068] For the original image The images in the dataset undergo uniform preprocessing operations, including but not limited to: resizing, center cropping / random cropping, color normalization, and random flipping, to obtain a clean set of preprocessed images. These operations are used to simulate the image classification process under normal shooting and conventional image enhancement conditions.

[0069] To characterize degradation factors in real-world environments, clean images are input into preset image erosion and resampling operators to generate various versions with natural perturbations, including but not limited to:

[0070] 1. Weather type of corrosion: frost, snow, fog, rain, etc.

[0071] 2. Degradation type of disturbance: blur, noise, compression, contrast change, etc.

[0072] 3. Implementation difference: interpolation, resampling method difference between different libraries, etc.

[0073] Each clean picture and its multiple perturbation versions are labeled consistently to form a natural perturbation picture set . (Show the idea Figure 1 ).

[0074] Based on the above definition, in order to evaluate and improve the adversarial robustness of the network, the preset adversarial attack method is used for clean samples to generate adversarial samples within the constraint norm and perturbation radius:

[0075] Fast Gradient Sign Method (FGSM) and Projected Gradient Descent are used. For the current model and input picture, calculate the gradient of the loss function to the input; take the sign of the gradient, and update each pixel in the direction by a preset step . The new picture obtained is still very close to the original picture in pixel space distance, but can significantly increase the probability of model prediction error. FGSM has low computational overhead and can quickly generate the first batch of adversarial samples for preliminary evaluation of model sensitivity to attacks. PGD attack is more powerful and can better reflect the robustness of the model.

[0076] For the samples in , the preset adversarial attack method is used to generate adversarial samples within the constraint norm and perturbation radius . And use PGD attack based on norm constraint, attack step is 20 steps, step is 2 / 255, perturbation radius is 0.031 to get the adversarial picture set .

[0077] In this way, each original sample corresponds to a group of "clean picture-natural perturbation picture-adversarial picture", which provides a rich and controllable data basis for subsequent unified metrics "clean accuracy-adversarial robustness".

[0078] As Figure 2The left side is a traditional architecture, and the right side is a dual-branch architecture designed by the present application. The whole is composed of a Stem module (mapping module), a plurality of dual-branch cells (Dual Branch Cell), a plurality of reduction cells (Reduce Cell), and a fusion and classification head. When designing a super neural network, robustness adjustment design is performed on the DARTS super network. The example diagram is as follows Figure 2 .

[0079] Stem module: map the input 3-channel picture to a higher-dimensional feature space, generally using one or more convolution + normalization structures.

[0080] Multi-layer cell series: a total of L cells (L is a hyperparameter, for example, 10) are connected in series in the network, of which about 1 / 3 and 2 / 3 are reduction cells, and the rest are dual-branch cells.

[0081] Reduce Cell (reduction cell): reduce the feature map resolution and expand the receptive field through step convolution or pooling to achieve multi-scale representation.

[0082] DualBranchCell (dual-branch cell): contains two sets of Normal Cell (normal branch) and Robust Cell (robust branch) structures inside, and performs feature fusion through a channel attention Fusion layer (feature fusion layer) at the tail.

[0083] Output head: perform global average pooling on the last layer of features, and then connect a fully connected layer or other task head to output the classification result.

[0084] Candidate operation set construction:

[0085] A variety of picture processing operators are configured as candidate operations on each edge of each branch Cell, including but not limited to:

[0086] 3x3 / 5x5 depthwise separable convolution;

[0087] 3x3 / 5x5 hollow convolution;

[0088] 3x3 max / average pooling;

[0089] Skip connection (skip) and the like.

[0090] The normal branch is more inclined to fine-grained high-frequency operators, and the robust branch is more inclined to smooth and redundant operators. The "precision-friendly operators" and "robust-friendly operators" are combined at the edge granularity, that is, the "precision-friendly operators" and "robust-friendly operators" respectively include at least one of the above picture processing operators, providing finer search granularity under the premise of controlling the amount of calculation.

[0091] In one embodiment, the normal branch and the robust branch in the feature extraction module each comprises at least one double-branch cell; each double-branch cell is formed by a plurality of edges, and each edge is configured with a picture processing operator selected from a preset candidate operator set; wherein the candidate operator set of the double-branch cell in the normal branch contains a first type of operator configured to extract fine-grained high-frequency features; the candidate operator set of the double-branch cell in the robust branch contains a second type of operator configured to extract smooth and redundant features; the precision-friendly operator and the robust-friendly operator are obtained by respectively combining the picture processing operators selected from the first type of operator and the second type of operator on each edge of the double-branch cell.

[0092] In one embodiment, the structure of the feature extraction module specifically comprises: a plurality of double-branch cells are sequentially cascaded, and a feature fusion layer is arranged between each two adjacent double-branch cells; wherein the features output by a previous double-branch cell are adaptively fused by the feature fusion layer and then transmitted as input to a next double-branch cell; a plurality of reduction cells in the reduction module are inserted between the sequence of the plurality of double-branch cells to perform down-sampling operation at multiple depths of the network, so as to realize multi-scale feature extraction and obtain the fused features.

[0093] As shown in Figure 4 , the present application designs a double branch containing a normal branch and a robust branch on the basis of the DARTS type cell structure. The whole represents a Dual-BranchCell, the upper half of which corresponds to the robust branch (RobustCell), and the lower half of which corresponds to the normal branch (NormalCell). The picture features from the previous layer network first enter the left vertical bar, and then are sent to the upper and lower two sub-branches through the middle arrow, and after the feature transformation is completed in the two sub-branches, the fusion and output are performed in the subsequent layer. Since the pictures in the real link will appear two types of significantly different statistics:

[0094] The clean / regular enhancement distribution (in step 1 ), high-frequency textures and details are conducive to improving the precision; the perturbation / adversarial distribution ( ) requires a larger receptive field, redundancy and smoothness to stabilize the decision boundary. If a single topology is forcibly used to simultaneously consider both types of targets, the gradients will conflict with each other, resulting in unstable structure learning.

[0095] Therefore, the present application explicitly decouples at the Cell level:

[0096] The robust branch ( Figure 4Upper half part of dashed box): Bias towards large receptive field and redundancy, improve stability against natural degradation and adversarial perturbation (e.g. dilated convolution, residual / Skip, etc.). Input 1 / 2: Two input boxes on the left side of the robust branch represent two input picture features received by the cell from the previous layer network, generally corresponding to the output of the first two cells. Their spatial dimensions are the same, and the number of channels can be the same or aligned through 1x1 convolution.

[0097] Intermediate nodes 0-3: Four blue rectangular boxes are labeled 0, 1, 2, and 3, respectively, representing the four intermediate computing nodes inside the robust branch. Each node receives several directed edges from all previous nodes and two input nodes. Inside the node, the output picture features of all input edges are added or concatenated to obtain the output picture features of the node. The entire structure forms a small directed acyclic graph (DAG).

[0098] Output node: The output box on the right represents the output node of the branch, which generally aggregates the outputs of nodes 0-3 in a predetermined manner (e.g. concatenation) to form the total output picture features of the robust branch, which is then passed to the subsequent fusion module or the next layer network.

[0099] Edges and colored dots:

[0100] Black directed arrows connecting input 1, input 2, and each intermediate node, as well as between intermediate nodes, represent the path of feature flow. Multiple colored dots on the arrow represent the candidate operation set on this edge, with different colors representing different operator types (e.g. Figure 3 as shown), such as 3x3 / 5x5 depthwise separable convolution (corresponding to sequential convolution in Figure 3 ), 3x3 / 5x5 dilated convolution, 3x3 max / average pooling, skip connection, etc.

[0101] Some dots are circled with dashed lines to represent the operations selected in the current search iteration or the operations with larger weights. The remaining uncircled dots represent alternative operations still retained in the super network. In this way, the structure feature of "multiple selectable operators on the same edge, with the final one determined by the search algorithm" is visually displayed in the figure.

[0102] Through the above design, the robust branch is more inclined to select operators with large receptive field, smoothing, and redundancy characteristics to enhance robustness against natural degradation and adversarial perturbation.

[0103] Normal branch ( Figure 4Bottom half of the dashed box): bias towards expressing details / high frequencies, improve the baseline accuracy of regular picture classification (classification / retrieval, etc.); input 1 / input 2: similar to the robust branch, the normal branch also receives two input picture features, corresponding to the outputs of the first two cells respectively. The input 1 and input 2 at the bottom of the figure are one-to-one corresponding to the top, indicating that the two inputs will be sent into the normal branch and the robust branch at the same time.

[0104] Middle nodes 0-3 and output: the lower half also contains four blue rectangular nodes 0-3 and an output node, and the connection structure is basically the same as the upper half, forming another set of independent directed acyclic graphs.

[0105] Edge and candidate operation: the arrow and colored dot in the normal branch have the same meaning as the robust branch, except that the candidate operation set is more biased towards fine convolution operators that perform better on clean pictures, such as 3x3 / 5x5 depth separable convolution, etc. The dotted circle also indicates the current more important or retained operation.

[0106] In this way, the normal branch focuses on improving the representation ability and classification accuracy on clean samples.

[0107] Two branches respectively complete node-level feature evolution, and then enter the lightweight channel attention feature fusion layer (Fusion Layer) Figure 5 , which implements a lightweight attention mechanism, and adaptively adjusts the proportion of two features according to the "erosion / attack intensity" of the input image, realizing sample adaptive routing and fusion.

[0108] DARTS is a single-branch single Cell, and a set of architecture parameters serve all data; in robust training, The gradient is greatly affected by the heavy tail and adversarial noise, and the target conflict is obvious. The dual-branch Cell+fusion gate divides the "clean target" and "robust target" at the structural level, and then allocates the sample-level weight by the fusion gate, which significantly reduces the target conflict and gradient noise, and changes the trade-off into a controllable structural parameterization problem.

[0109] Further, the present application and subsequent robust evaluation linkage estimate the contribution of the candidate operations on the two branches respectively "abnormal perception", avoiding single noise misleading architecture decision.

[0110] As Figure 5 shown, the present application introduces a feature fusion layer (Fusion Layer) after the dual-branch cell (Dual-Branch Cell), and inserts a reduction cell (Reduce Cell) at different depths of the network, which together completes the adaptive fusion of multi-branch features and multi-scale feature extraction.

[0111] Feature fusion layer structure:

[0112] 1. Dual-branch output and input concatenation normal branch:

[0113] The left vertical Dual-BranchCell represents the overall structure of the previous layer of dual-branch cells, where:

[0114] The upper orange small block is marked as , representing the output features of the robust branch.

[0115] The lower blue small block is marked as , representing the output features of the normal branch.

[0116] The two diagonal arrows send and to the right rectangular box "Input 2xC". This input box represents the concatenation of the two feature channels in the channel dimension. That is, the number of channels changes from C to 2C, and the spatial size remains .

[0117] 2. Efficient Channel Attention (Efficient Channel Attention):

[0118] The yellow square "Efficient Channel Attention" on the right side of the figure represents the lightweight channel attention module, and the embodiment adopts the ECA (Efficient Channel Attention) structure:

[0119] First, the global average pooling is performed on the features after convolution / BN to obtain the global description of each channel. Then, one-dimensional convolution or small MLP is used to generate the weight of each channel. Finally, the corresponding channel is scaled by weighting. In this way, different channels will be automatically amplified or suppressed according to the importance of the current input image: for clean samples, more weight is allocated to the normal branch channel from ; for corrosion or adversarial samples, the weight of the robust branch channel from will be enhanced. "Convolution / BN" represents the cascade structure of the convolution layer (Convolutional Layer) and the batch normalization layer (Batch Normalization Layer) in the neural network.

[0120] After the above ReLU→1×1Conv / BN→EfficientChannelAttention processing, the fused feature map Hfusion is obtained, which is the final output of the current Dual-BranchCell and is sent to the next layer network. "Conv / BN" represents the cascade module of the convolutional layer (Convolutional Layer) and the batch normalization layer (Batch Normalization Layer).

[0121] Therefore, Figure 5 Each box in the figure corresponds to a specific operator (activation, convolution, normalization, channel attention) in the FusionLayer, and the arrows represent the flow of features between operators. The upper and lower paths Hnorm / Hrob converge at "input 2xC" and are adaptively reweighted in the attention module, thereby achieving dynamic adjustment of the contributions of the "normal branch vs robust branch".

[0122] In one embodiment, a robust evaluator is further included, which is connected with each of the dual-branch cells, for collecting the selected picture processing operators corresponding to each of the dual-branch cells and performing comprehensive robust scoring to screen the target network configuration.

[0123] In one embodiment, the robust evaluator is configured to perform an iterative screening strategy to search for the target network configuration, and the screening strategy includes the following steps:

[0124] a) Based on the collected scores, for each edge in each of the dual-branch cells, compare the comprehensive robust scores of the candidate picture processing operators under it;

[0125] b) Identify the candidate picture processing operator with the lowest comprehensive robust score on each edge as the eliminated candidate;

[0126] c) When the score of a candidate picture processing operator meets a preset significant disadvantage condition, mark it as a significant disadvantage operation;

[0127] d) In each search iteration, select a number of candidate picture processing operators marked as the eliminated candidate or the significant disadvantage operation from each edge to remove, thereby narrowing the search space of the neural network architecture;

[0128] e) Repeat steps a) to d) until the search space converges to determine the target network configuration.

[0129] In this embodiment, on the basis of the super network constructed in step 2, the application proposes a ROSE (Robust Outlier-aware Shapley Estimator) robust evaluator. The robust evaluator reads the output features of each double-branch cell and the candidate operator selection record, performs remote evaluation and scoring, and is composed of a scoring module, a storage module and a decision module, which are connected through a data bus and are respectively used to complete contribution measurement, scoring record and elimination decision. The contribution of each candidate operation in the picture classification task is statistically robustly scored for subsequent architecture screening.

[0130] Contribution sample construction

[0131] Score branch type An edge in the super network is , and a candidate operation on the edge is .

[0132] Common sampling scenarios (random arrangement / removal order), numbered .

[0133] For each , we get a set of normalized contribution samples , and then make a robust estimation through MoM and IQR.

[0134] Definition of clean / against accuracy gain

[0135] In each search iteration, a small batch of pictures is extracted from to form a validation set containing clean and perturbed pictures; for each candidate operation on each edge , a variety of operation arrangement orders or "remove / keep" combinations are randomly generated, and the performance indicators (such as classification accuracy or loss) of the network are calculated forwardly under two configurations containing the operation and removing the operation respectively; the performance difference between the two configurations is recorded as the marginal contribution of the operation under the th scenario, and the contribution sequences of clean and against scenarios are obtained respectively .

[0136] Clean accuracy gain

[0137] (Formula 1)

[0138] Against accuracy gain

[0139] (Formula 2)

[0140] Wherein:

[0141] For the scenario s, the accuracy of the validation set clean picture for the reserved operation o;

[0142] For the scenario accuracy after the removal operation o in the same scenario (the rest of the structure remains unchanged)

[0143] Against the scenario ,Definition similar.

[0144] In order to unify the two targets of "cleaning + confrontation", a weighted synthesis can be done first (consistent with the composite loss during training):

[0145] (Formula 3)

[0146] Where is the performance weight coefficient of the clean sample, which is used to balance the clean accuracy and the confrontation accuracy.

[0147] Median-of-Means (MoM) robust average:

[0148] Let Divide it into G subgroups in order , each group size is about . Calculate the arithmetic mean of each subgroup:

[0149] (Formula 4)

[0150] Then take the median of these group means to get the MoM robust contribution value of the operation:

[0151] (Formula 5)

[0152] First "bucketing and then calculating the average", and then taking the median of these means, can significantly weaken the influence of a small number of extreme samples on the overall estimate, and still give an estimate close to the true Shapley contribution in the heavy-tailed environment of confrontation training.

[0153] IQR-based outlier analysis and penalty factor:

[0154] In order to explicitly "account" those operations that perform very poorly or very well in a small number of scenarios (very critical to robustness), the present invention uses IQR (Inter-Quartile Range, Quartile Range) to score abnormal samples in addition to MoM.

[0155] For the sample set :

[0156] Calculate the first and third quartiles:

[0157]

[0158] And let

[0159] (Formula 6)

[0160] Set the upper and lower thresholds (γ>0 is a hyperparameter):

[0161] (Formula 7)

[0162] Statistics of the number of samples of "abnormal good" and "abnormal bad":

[0163] (Formula 8)

[0164] Define the abnormal penalty / reward factor:

[0165] (Formula 9)

[0166] If , it means that the operation brings performance gain in most "extreme scenarios", and is a larger operation on key images.

[0167] If , it means that it often lags behind in extreme images, which is a potential "short board" in robustness and needs to be punished in the search.

[0168] Comprehensive robust score calculation:

[0169] Finally, the present application linearly combines the MoM (overall mean method) robust contribution with the IQR (outlier detection method based on quartile distance) abnormal factor to obtain the comprehensive robust score of each candidate operation on branch b and edge e:

[0170] ,0< <1. (Formula 10)

[0171] Reflect the stable average contribution of the operation on "most pictures";

[0172] Reflect the criticality or risk of the operation on "a few extreme pictures";

[0173] is the abnormal factor weight coefficient, used to adjust the balance between steady-state performance and extreme robustness, generally taken as 0.3~0.5, which can achieve a good compromise between "steady-state performance" and "extreme robustness". (In the example, 0.3 is taken). Finally, the contribution degree of each operation forms the corresponding architecture parameter . Output a set of architecture parameter matrix every round Output the searched network genotype code.

[0174] Candidate operation filtering and architecture updating based on robust score:

[0175] After obtaining the comprehensive robust score of each candidate operation, the present application adopts an iterative filtering strategy to gradually shrink the search space and search for the target network configuration suitable for the picture classification task. For each edge in each Cell, the contribution of the candidate operations thereunder is compared, and the operation with the lowest score is identified as the eliminated candidate. When the score of an operation is significantly lower than that of other operations on the same edge, the operation is marked as a "significantly disadvantaged operation". In each search iteration, a number of operations with the lowest score or significant disadvantage on each edge are removed to reduce the search space. The eliminated candidate operations are deleted from the super network, and the operations with higher scores are retained to continue participating in training and evaluation.

[0176] On the updated network, the ROSE evaluation process in use is reused to perform a new round of robust scoring on the remaining candidate operations. This iterative process continues until only the operation with the highest score remains on each edge, and the super network is discretized into a specific neural network architecture.

[0177] The robust picture classification super network (Super Net) of the present application can be summarized as the following series structure: input and Stem module (mapping module), which is the starting module, the input is the constructed picture mini-batch (clean picture, natural disturbance picture or adversarial Figure Three mixing) first passes through the Stem module composed of one or more convolution + normalization layers to map the original 3-channel picture to a higher-dimensional feature space, providing a unified initial representation for the subsequent multi-layer Cell. There are layers (layer is a hyperparameter, generally set to 10) of Cells in series in the network. At 1 / 3 and 2 / 3 layer positions, ReduceCell is used to reduce feature map resolution and expand receptive field through stride convolution or pooling. The remaining positions use DualBranchCell: which contains two sets of structures, normal branch (clean branch) and robust branch (robust branch), and uses channel attention Fusion layer at the tail for feature fusion. After all Cells are calculated, the feature map in the last layer state is taken, global average pooling is performed, and a fully connected classification layer or task head is connected to obtain the prediction result of the input picture. For the three types of Cells (Normal / Reduce / Robust), a set of edge-level architecture parameter matrices is maintained for each type of Cell, each row corresponds to an edge, and each column corresponds to a candidate operation. During forward calculation, these After softmax normalization, the weight weights of each candidate operation is obtained, which is used for weighted summation of multiple picture processing operators to realize "structural mixing in continuous space".

[0178] The flow process of data in the super network:

[0179] After any picture enters the super network, it will be forward propagated along the path of "mapping module → double-branch unit → feature fusion layer → multi-level reduction cell → global average pooling → classification head"; different cells form a "soft path" in the continuous structure space by weighting and combining multiple candidate operations according to the current architecture parameters.

[0180] The process of generating genotype by the super network using data:

[0181] Based on the robust evaluation and screening of picture data, the clean picture-natural disturbance picture-adversarial picture constructed by step 1 is used as a verification set, and the ROSE robust evaluation method in step 3 is used to measure the marginal contribution of each candidate operation on each edge in the super network for multiple rounds: the performance difference (accuracy / loss) of the model on the picture task is compared between "keeping the operation" and "removing the operation"; the Median-of-Means (Mean of Means) + IQR (Interquartile Range-based outlier detection method) are used for detection to obtain the robust contribution value and the abnormal penalty factor; the two are combined into the final contribution degree, and the low-score operations are iteratively eliminated accordingly. As the search iteration proceeds, the number of optional operations on each edge gradually decreases, and finally only the operations that perform most stably and optimally in multiple picture scenarios are retained.

[0182] When the search is completed, only a few candidate operations are left on each edge, and the weights in the formula reflect the preferences for each operation. The present application analyzes the three types of cells respectively: for each intermediate node i, collect the weight vectors of all input edges, calculate the maximum weight of each edge on all operations; sort the maximum weights from large to small, select the top 2 edges as the actual input of the node; on each selected edge, select the operation with the maximum weight from the candidate operations, and record "operation type + source node number (op, j)" to the genotype sequence. Through this operation, the searched network genotype corresponding to the data can be obtained, and based on the network genotype, a neural network can be constructed, which only retains the selected operations and connections, and is trained from scratch or fine-tuned on the entire picture training set to obtain a robust picture classification model suitable for deployment; the model is used for migration between different terminals / tasks as the final result of structural search.

[0183] Final robust picture classification network training and use method:

[0184] After convergence, the operation with the highest score on each edge is fixed to obtain the final network structure stacked by normal branches and robust branches alternately and inserted with reduced cells;

[0185] Based on the entire training set (including clean pictures and adversarial pictures), the network is completely trained, and the training loss can be a weighted combination of clean loss and adversarial loss to continue balancing the picture classification accuracy and robustness.

[0186] In the deployment stage, the terminal or server calls the model of the application to process pictures according to the following process: obtaining pictures to be processed, and performing scale and normalization operations according to the same preprocessing process in step 1; inputting the preprocessed picture into the trained robust network, and sequentially passing through the feature extraction and attention fusion of a plurality of normal branches and robust branches; and outputting picture classification results, detection results or other picture processing results according to specific tasks. Since the network structure has been jointly optimized for the "clean picture-disturbance picture-adversarial picture" multiple scenes in the search stage, the terminal can still maintain stable and reliable output when facing pictures with light changes, weather corrosion, compression noise or adversarial attacks.

[0187] The innovations of the application are as follows: dual-branch robust architecture design: the application innovatively introduces parallel optimization of robust branches and normal branches in the network structure. This dual-branch design ensures that the clean accuracy and adversarial robustness of the model are considered simultaneously during the search process, which is an important innovation at the architecture level. The beneficial effects are that the final output network architecture contains modules optimized for different scenes, with high expression ability and high anti-disturbance ability. For example, the robust branch searched by the application tends to select operations such as dilated convolution and skip connection to form a more smooth and redundant feature extraction path, while the normal branch selects operations such as separable convolution to extract rich features. This differentiated architecture configuration makes the model more robust in the face of adversarial attacks, and the key feature extraction path is not easily damaged by single-point disturbance; at the same time, the accuracy on clean samples is not lost. Compared with existing single-structure models, the dual-branch architecture of the application significantly improves the accuracy index under adversarial attacks (the improvement amplitude can reach several percentage points under the same attack intensity), while the accuracy on clean samples remains basically unchanged or even slightly improves. In summary, the dual-branch architecture design makes the model achieve an excellent balance between security and practicality, which is not possessed by traditional architectures.

[0188] Robust anomaly-aware evaluation mechanism: the present application introduces the evaluation method of Median-of-Means combined with anomaly detection (ROSE algorithm) in architecture search. This mechanism is the core innovation of the present application, which can accurately evaluate the true contribution of each candidate operation in the presence of adversarial attacks and other harsh conditions. The beneficial effects are: strong anti-noise ability, stable and reliable evaluation results. Compared with the existing methods that rely on simple average or single evaluation, the present scheme greatly reduces the influence of abnormal samples on evaluation, ensuring that the search process is not "misguided". Theoretical analysis shows that the deviation probability limit of the evaluation value and the actual contribution decreases exponentially (sub-Gaussian tail bound), so the required evaluation sample size is controllable and can be adjusted according to the desired confidence, improving the search efficiency. At the same time, the introduction of the anomaly penalty factor makes the evaluation index not only focus on the average performance, but also focus on the worst-case performance, so as to ensure that the selected architecture is still reliable in extreme cases, and the adversarial robustness of the final model is improved.

[0189] Robust score-based iterative selection strategy: the present application proposes to apply the above robust score to the elimination decision of candidate operations, and gradually converges the architecture. Compared with the traditional one-time or noise gradient signal-based selection scheme, this is another key improvement. The beneficial effects are: through multiple cycles, each decision is based on more sufficient data, avoiding premature convergence and false elimination of architecture search. Each round is re-evaluated during the iterative process, and this repeated calibration mechanism ensures that the operations finally retained have positive contributions to the performance under different scenarios. The final obtained architecture performs well in both clean and adversarial tests, and will not be biased to another index because of excessive focus on one index, achieving optimal performance balance. In addition, this strategy makes the search process flexible to adapt to different resource conditions: if the computation is sufficient, the number of samples can be increased to improve the evaluation accuracy; if the resources are limited, the number of iterations or the size of the candidate set can also be appropriately reduced, speeding up the search within an acceptable range. Therefore, this strategy improves the practicality and efficiency of NAS in the robustness scenario.

[0190] In summary, the present application provides a new scheme for neural network architecture search in an adversarial environment through the synergistic effect of the above innovations. The significant beneficial effects include: robust and efficient search process, high precision and robustness of the obtained model, good scalability and reliability of the method (parallel expansion to multiple GPU clusters without changing the statistical properties). The scheme described in the present technical disclosure provides sufficient support for subsequent patent applications.

[0191] Reference Figure 6 The present application also provides a picture classification method based on a neural network architecture, which is used to train the mobile terminal robust picture classification neural network architecture described above, comprising:

[0192] S001: obtaining a picture to be classified;

[0193] S002 inputting the picture to be classified into a neural network architecture for mobile terminal-oriented robust picture classification to obtain a classification result;

[0194] The training step of the neural network architecture for mobile terminal-oriented robust picture classification comprises:

[0195] S1: obtaining a training data set;

[0196] S2: constructing an initial super network; wherein the initial super network comprises a mapping module, a reduction module, a feature extraction module, and a classification module, and each edge of each double-branch cell in the feature extraction module is configured with a complete set of candidate picture processing operators;

[0197] S3: inputting the training data set into the initial super network for training, and evaluating the performance of each candidate picture processing operator;

[0198] S4: based on the evaluation result, iteratively screening and removing part of the picture processing operators from the candidate picture processing operators of the initial super network to shrink the search space, until the final target network configuration is determined;

[0199] S5: taking the network configuration as a trained neural network model.

[0200] As described in steps S001-S002 above, image data is collected from different sources, such as photos taken with a smartphone, pictures downloaded from the Internet, or images generated by image sensors. The pictures to be classified should meet certain quality standards and be similar to the types of images used in the model training process to ensure the classification effect. The obtained pictures to be classified are input into the neural network architecture for mobile terminal-oriented robust picture classification. After inputting the image, the network will go through a series of hierarchical processing, including feature extraction, down-sampling, and classification module steps. Each layer analyzes different features of the image to extract information related to the category. During processing, the network will generate the final classification result by completing the neuron activation between layers based on the knowledge obtained during the training phase.

[0201] As described in steps S1-S5 above, a training data set containing target task labels is first needed. This data set can be obtained by collecting picture data in a specific field, or using publicly available data sets (such as ImageNet and CIFAR-10, etc.). Each picture should be accompanied by the corresponding category label to ensure that the set used for training has high quality and diversity. This step provides the necessary data basis for subsequent training, ensuring that the network can learn features that support its high performance.

[0202] After determining the training dataset, the next step is to build the initial super network. The design of the initial network must include multiple functional modules, including mapping modules, reduction modules, feature extraction modules, and classification modules. This structured design idea allows the network to extract features layer by layer, as follows: Mapping module: responsible for extracting the initial features of the input picture, usually through convolutional layers and normalization cells. Reduction module: aims to gradually reduce the spatial size of the feature map through downsampling operations, while enhancing the representation ability of the features, making the network more efficient. Feature extraction module: this module is further divided into two independent double-branch cells, and each edge in each cell is configured with a complete set of candidate picture processing operators. This design allows the network to extract features from different angles, balancing classification accuracy and robustness. Classification module: finally converts the fused feature information into specific classification results.

[0203] After the initial super network is built, the training dataset obtained is then input into the network for training. The key step in this process is to evaluate the performance of each candidate picture processing operator. Through forward propagation and loss function calculation, the network will gradually adjust its parameters, so that the recognition ability of the training data is continuously enhanced. During the training process, not only the weights of the network need to be continuously optimized, but also the effects of different operations need to be evaluated, which also provides a basis for subsequent operation selection.

[0204] The network structure is optimized in an iterative manner. According to the results of the previous step evaluation, the candidate operations with poor performance are identified and eliminated. Specifically, the algorithm will select and remove the corresponding operators according to certain conditions (such as performance below a certain threshold or significant disadvantage). Through this process, the search space of the network will gradually shrink, thus promoting the network to develop towards a more optimal configuration, ensuring that the final target network architecture has stable performance and superior accuracy.

[0205] Finally, after multiple iterations of screening and optimization, the network structure is finally determined, and the configuration can be output as a trained neural network model. This model, after complete training and evaluation, has the ability to perform robust picture classification in a specific mobile environment. The final model not only maintains high accuracy on clear input images, but also exhibits good stability when faced with various natural disturbances and adversarial attacks.

[0206] In one embodiment, the step S5 of iteratively selecting and removing part of the picture processing operators from the candidate picture processing operators of the initial super network based on the evaluation results to shrink the search space until a final target network configuration is determined, comprises:

[0207] S501: Obtain the performance evaluation results of each candidate picture processing operator on each edge of each double-branch cell in the initial super network based on the training data set;

[0208] S502: Compare the performance evaluation results of each candidate picture processing operator on each edge of each double-branch cell;

[0209] S503: Identify the candidate picture processing operators on each edge that meet the preset elimination condition according to the comparison results;

[0210] S504: Remove the identified candidate picture processing operators that meet the preset elimination condition to update the initial super network and shrink the search space;

[0211] S505: Determine whether the updated search space meets the preset convergence condition;

[0212] S506: If not, return to the step of inputting the training data set into the training and evaluation based on the updated super network;

[0213] S507: If yes, determine the final target network configuration according to the remaining candidate picture processing operators on each edge of the initial super network.

[0214] As described in steps S501-S507 above, the network obtains the performance evaluation results of each candidate picture processing operator from the training data set. All candidate operators in each double-branch cell of the initial super network are systematically measured. Multiple rounds of evaluation using the training data set ensure that the collected information is comprehensive and objective. In this way, the network has data support in subsequent decision-making and can select candidate operators based on their specific performance.

[0215] For each edge of each double-branch cell, the performance evaluation results of all candidate operators are systematically compared. This comparison allows the network to identify which operations perform well and which operations perform poorly when processing the training data set. By quantifying performance differences such as accuracy, loss value, and other key indicators, the network can make more informed elimination decisions. After comparison, the system will identify the candidate operators on each edge based on the preset elimination criteria. These preset elimination conditions usually include but are not limited to: the performance of the candidate operator is lower than a certain threshold, the lowest evaluation result among all candidates, the operator that contributes less to the overall model, etc. The importance of this step lies in clearly identifying those operators that play a smaller role in model training through this intelligent summary, thereby preparing for subsequent removal work.

[0216] After the identification, the system will perform the actual operation removal. This operation helps to update the structure of the initial super network, narrowing its search space. By removing those underperforming candidate operators, the network is able to progressively eliminate redundant parts and refocus on the most promising operator combinations. This process not only improves computational efficiency but also increases the accuracy of subsequent training. After updating the network structure, the system needs to detect whether the current search space meets the convergence condition. The convergence condition can be that the network performance stops improving significantly, or reaches a certain expected accuracy, etc. This step confirms whether the network has enough new information to process, ensuring that the subsequent training and evaluation are meaningful.

[0217] If the current search space does not meet the convergence condition, the system will return to the training process based on the updated super network. By inputting the training data into the network, the next round of training and performance evaluation is performed. This feedback mechanism is similar to iteration, gradually optimizing the network in each round, collecting more reliable and detailed performance data, so that the network can continue to improve itself. If it is confirmed that the current search space meets the convergence condition, the final step is to determine the target network configuration. This configuration will be based on the effective candidate image processing operators remaining on each edge of the filtered super network. This final network configuration will be used as the optimized structure obtained after training, for subsequent image classification tasks.

[0218] In one embodiment, the step S501 of obtaining each candidate image processing operator on each edge of each double-branch cell in the initial super network based on the performance evaluation result of the training data set comprises:

[0219] S5011: Obtain and perform multiple rounds of performance contribution measurement on each candidate image processing operator on each edge of the current super network based on the validation set; wherein the performance contribution measurement is achieved by comparing the difference in performance indicators of the super network on the image classification task when the candidate image processing operator is retained and removed in the super network;

[0220] S5012: Based on the results of the multiple rounds of performance contribution measurement, a comprehensive robust score is calculated for each candidate image processing operator using a pre-set robust statistical algorithm as the performance evaluation result.

[0221] As described in steps S5011-S5012 above, by performing multiple rounds of performance contribution measurement on each candidate image processing operator of each edge, the researcher can obtain the relative performance of each operator. This process is based on a validation set, which is a part of the training dataset separated for evaluating the generalization ability of the model. In this step, the performance contribution measurement method used is by comparing the performance difference of the current super network when retaining a specific candidate image processing operator and removing that operator. The core of this method is that it directly quantifies the impact of each operator on the performance of the image classification task through actual training and testing experiments. For example, assuming that a certain candidate image processing operator A performs well in the super network, the system will first retain this operator in the super network and calculate the classification accuracy of the model on the validation set. Then, the system will remove the operator and recalculate the accuracy of the model on the same validation set. If the accuracy of the model decreases significantly after removing operator A, it indicates that operator A has a significant contribution to the overall performance of the model, and the performance contribution measurement will give operator A a higher score; conversely, if the accuracy does not change much, the contribution measurement score of operator A will be lower. Through multiple rounds of experiments, the influence of accidental factors on the evaluation results can be further reduced, making the performance contribution measurement more stable and reliable.

[0222] The results of multiple rounds of performance contribution measurement are integrated into a robust score. In the case of multiple measurements, a simple average may be affected by outliers, so a pre-set robust statistical method is used to calculate the comprehensive robust score of each candidate operator. This robust statistical method usually uses methods such as median, quantile or other statistical methods to reduce the influence of outliers on the results, making the final score more representative. The calculation result of the robust score will form an important performance evaluation result, which will directly affect the subsequent model architecture search and operator selection process. Through the above two steps, the best-performing image processing operators in actual applications can be evaluated and selected, thereby optimizing the architecture of the super network and ultimately improving the accuracy and efficiency of image classification.

[0223] In one embodiment, in step S503 of identifying candidate image processing operators on each edge that meet the pre-set elimination condition according to the comparison result, the pre-set elimination condition is that the comprehensive robust score of the candidate image processing operator is lower than a pre-set threshold or is the lowest among all candidate image processing operators on the same edge.

[0224] In this embodiment, the comprehensive robust score is obtained by statistically analyzing the performance contribution of the candidate operator in multiple experiments. This score reflects the stability and effectiveness of the operator. When the comprehensive robust score of a candidate operator is lower than a preset threshold, it indicates that the operator performs poorly in multiple validation experiments and may not provide sufficient performance improvement in practical applications. Therefore, based on this score, the elimination can effectively exclude those potential "inefficient" operators, ensuring that the operators retained in the model architecture can play a good role in specific tasks. Secondly, based on the elimination condition of "the lowest score among all candidate picture processing operators on the same side", the operator selection process is further improved. On the same side, multiple candidate downstream processing operators may exhibit different performance. If the score of an operator is the lowest among all operators on this side, the performance impact of this operator on the network is obviously the smallest. Therefore, it is reasonable to eliminate it. This condition ensures that each side of the super network retains as many high-performance operators as possible to optimize the overall performance of the network.

[0225] In one embodiment, the step S1 of obtaining the training data set comprises:

[0226] S101: obtaining an original picture data set containing target task labels;

[0227] S102: performing a preprocessing operation on the original picture data set to obtain a clean picture data set; the preprocessing operation includes size adjustment, cropping and color normalization;

[0228] S103: inputting the clean picture data set into a preset picture degradation operator to generate corresponding multiple natural perturbation version pictures to constitute a natural perturbation picture data set; wherein each clean picture and its corresponding natural perturbation version pictures maintain the same target task label;

[0229] S104: using a gradient sign-based adversarial attack method to process the clean picture data set within a preset perturbation constraint to generate corresponding adversarial sample pictures to constitute an adversarial picture data set; wherein each clean picture and its corresponding adversarial sample picture maintain the same target task label;

[0230] S105: extracting the clean picture, at least one natural perturbation version picture and adversarial sample picture corresponding to each original picture to form a sample group, thereby obtaining the training data set composed of all sample groups.

[0231] As described in steps S101-S105 above, obtaining the original picture dataset containing the target task label is the starting point of the entire training dataset construction. These original pictures should be closely related to the required task (such as image classification, object detection, etc.), and each picture is attached with corresponding label information. This process ensures that the subsequent operations have a clear guidance target and provides the necessary baseline data for training.

[0232] After obtaining the original dataset, preprocessing operations are performed. Preprocessing mainly includes size adjustment, cropping, and color normalization, etc. These operations aim to eliminate inconsistencies or unnecessary complexities in the original data, making the data more suitable for model training. Size adjustment and cropping ensure that all input pictures have the same size, suitable for the standard format required by deep learning models. Color normalization helps to reduce the impact of factors such as light, contrast, etc. in the shooting environment, enhancing the robustness of the model under different lighting conditions. This process ultimately results in a clean picture dataset.

[0233] After obtaining the clean picture dataset, each clean picture is processed through a pre-set picture degradation operator to generate corresponding multiple natural perturbation versions. These natural perturbation versions may consider various noises and disturbances commonly found in the real world (such as blur, noise, compression distortion, etc.), enabling the model to learn more rich features. In this process, each natural perturbation picture still maintains the same target task label as its original clean picture. This data augmentation method not only expands the diversity of the training dataset, but also improves the generalization ability of the model, allowing it to make accurate predictions when faced with various disturbances in actual applications.

[0234] Next, the clean picture dataset is processed using a gradient sign-based adversarial attack method to generate corresponding adversarial sample pictures. Adversarial samples are generated by adding small perturbations to the input data, and these perturbations are specifically designed to cause the deep learning model to output incorrect results when processing these data. Similar to natural perturbation version pictures, each clean picture and its corresponding adversarial sample still maintain the same target task label. The goal of generating adversarial samples is to improve the robustness of the model, making it more resistant to potential hostile attacks or irregular data.

[0235] Finally, each original picture is associated with its corresponding clean picture, at least one natural perturbation version picture, and adversarial sample picture to form a sample group. In this way, the constructed training dataset will contain multiple sample groups, so that each sample group presents the same task label, but through different image versions, the data for model training is enriched. This construction method strives to maximize the use of data changes under different perturbations while ensuring consistent labels, thereby effectively improving the quality and efficiency of model training.

Claims

1. An image classification system based on a neural network architecture for robust image classification on mobile devices, characterized in that, The image classification system includes a mapping module, a reduction module, a feature extraction module, and a classification module connected in sequence. The mapping module includes multiple first convolutional layers and normalized cells. The multiple first convolutional layers are connected sequentially, and the last first convolutional layer is connected to the normalized cells to extract the initial features of the input image. The reduction module includes multiple reduction cells connected in sequence. The first reduction cell is connected to the normalized cell. Each reduction cell processes the initial feature through a downsampling operation to obtain the reduced intermediate feature. The feature extraction module includes a normal branch, a robust branch, and a feature fusion layer. The normal branch and the robust branch are respectively connected to the last-ranked reduced cell. The normal branch and the robust branch are respectively connected to the feature fusion layer. The normal branch is used to extract high-frequency operators from the intermediate features to obtain precision-friendly operators. The robust branch is used to extract redundant operators from the intermediate features to obtain robust-friendly operators. The feature fusion layer is used to fuse the precision-friendly operators and the robust-friendly operators to obtain fused features. The classification module is used to classify the fused features; Both the normal branch and the robust branch in the feature extraction module include at least one bibranched cell; Each of the bi-branch cells is formed by multiple edges connected together, and each edge is configured with an image processing operator selected from a preset set of candidate operators; wherein, the set of candidate operators for bi-branch cells in the normal branch includes a first type of operator, which is configured to extract fine-grained high-frequency features; the set of candidate operators for bi-branch cells in the robust branch includes a second type of operator, which is configured to extract smooth and redundant features; The precision-friendly operator and the robust-friendly operator are obtained by combining image processing operators selected from the first type of operator and the second type of operator on each edge of the bi-branch cell, respectively. The feature extraction module specifically includes: Multiple bi-branch cells are cascaded sequentially, and a feature fusion layer is disposed between every two adjacent bi-branch cells; wherein, the feature output by the previous bi-branch cell is adaptively fused by the feature fusion layer and then passed as input to the next bi-branch cell; Multiple reduced cells in the reduction module are inserted between sequences composed of multiple bi-branched cells to perform downsampling operations at multiple depths of the network, thereby achieving multi-scale feature extraction and obtaining the fused features.

2. The image classification system according to claim 1, characterized in that, It also includes a robust evaluator connected to each of the two-branch cells, which collects and scores the image processing operators selected for each of the two-branch cells in order to filter the corresponding target network configuration.

3. The image classification system according to claim 2, characterized in that, The robust evaluator is configured to perform an iterative screening strategy to search for the target network configuration, the screening strategy comprising the following steps: a) Based on the collected scores, for each edge in each of the two-branch cells, compare the comprehensive robust scores of each candidate image processing operator under it. b) Identify the candidate image processing operator with the lowest overall robustness score on each edge and use it as a candidate for elimination; c) When the score of a candidate image processing operator meets the preset significant disadvantage condition, it is marked as a significant disadvantage operation; d) In each round of search iteration, select several candidate image processing operators marked as elimination candidates or significant disadvantage operations from each edge and remove them, thereby reducing the search space of the image classification system; e) Repeat steps a) to d) until the search space converges to determine the target network configuration.

4. A neural network-based image classification method for training the image classification system according to any one of claims 1-3, characterized in that, include: Get the images to be categorized; The image to be classified is input into an image classification system for robust image classification on mobile devices to obtain the classification result; The training steps of the image classification system based on a neural network architecture for robust image classification on mobile devices include: Obtain the training dataset; Construct an initial super network; wherein the initial super network includes a mapping module, a reduction module, a feature extraction module, and a classification module, and each edge of each bi-branch cell in the feature extraction module is configured with a complete set of candidate image processing operators; The training dataset is input into the initial supernetwork for training, and the performance of each candidate image processing operator is evaluated. Based on the evaluation results, some image processing operators are iteratively filtered and removed from the candidate image processing operators of the initial super network to shrink the search space until the final target network configuration is determined. The network configuration is used as the trained neural network model.

5. The image classification method based on a neural network architecture according to claim 4, characterized in that, The step of iteratively filtering and removing some image processing operators from the candidate image processing operators of the initial supernetwork based on the evaluation results to narrow the search space until the final target network configuration is determined includes: Obtain the performance evaluation results of each candidate image processing operator on each edge of each bi-branch cell in the initial supernetwork based on the training dataset; For each edge of each bi-branch cell, compare the performance evaluation results of each candidate image processing operator; Based on the comparison results, identify the candidate image processing operators that meet the preset elimination conditions on each edge; Remove the identified candidate image processing operators that meet the preset elimination conditions to update the initial super network and shrink the search space; Determine whether the updated search space meets the preset convergence criteria; If not, then based on the updated supernetwork, return to the step of inputting the training dataset for training and evaluation; If so, the final target network configuration is determined based on the remaining candidate image processing operators on each edge of the initial supernetwork.

6. The image classification method based on a neural network architecture according to claim 5, characterized in that, The step of obtaining the performance evaluation results of each candidate image processing operator on each edge of each bi-branch cell in the initial supernetwork based on the training dataset includes: Based on the validation set, multiple rounds of performance contribution measurement are performed on each candidate image processing operator for each edge in the current super network; wherein, the performance contribution measurement is achieved by comparing the performance index difference of the super network on the image classification task when the candidate image processing operator is retained in the super network and when the candidate image processing operator is removed. Based on the results of the multi-round performance contribution measurement, a preset robust statistical algorithm is used to calculate a comprehensive robust score for each candidate image processing operator, which serves as the performance evaluation result.

7. The image classification method based on a neural network architecture according to claim 6, characterized in that, In the step of identifying candidate image processing operators that meet the preset elimination conditions on each side based on the comparison results, the preset elimination conditions are: the comprehensive robustness score of the candidate image processing operator is lower than a preset threshold or is the lowest score among all candidate image processing operators on the same side.

8. The image classification method based on a neural network architecture according to claim 4, characterized in that, The steps for obtaining the training dataset include: Obtain the original image dataset containing the target task labels; Preprocessing operations are performed on the original image dataset to obtain a clean image dataset; the preprocessing operations include resizing, cropping, and color normalization. The clean image dataset is input into a preset image degradation operator to generate multiple naturally perturbed version images to form a naturally perturbed image dataset; wherein each clean image and its corresponding naturally perturbed version images maintain the same target task label; An adversarial attack method based on gradient symbols is adopted to process the clean image dataset within a preset perturbation constraint to generate corresponding adversarial sample images to form an adversarial image dataset; wherein, each clean image and its corresponding adversarial sample image maintain the same target task label; Each original image is associated with a clean image, at least one naturally perturbed version image, and an adversarial example image, and thus a training dataset consisting of all the sample groups is obtained.

Images