Image classification method and device applied to intelligent terminal

Abstract

The application provides an image classification method and device applied to a smart terminal. The method comprises the following steps: in response to an image classification task, randomly selecting a candidate model from a candidate model set to obtain a target model; and using the target model to perform the image classification task to obtain a classification result. The candidate model set comprises a plurality of candidate models obtained by structurally sparsing an original model according to different sparse thresholds by a cloud GPU. By using the image classification method provided by the application, even if an attacker obtains all information used for reasoning, including a model structure, parameters, a random strategy and a candidate model set used for randomness, the attacker cannot know which model in the candidate model set used for reasoning at present. Therefore, regardless of whether the attacker uses the same random strategy or generates attack samples by using a fixed model, the image classification method provided by the application can balance the robustness and high efficiency of the neural network model used for reasoning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent terminals, and in particular to an image classification method and device applied to an intelligent terminal. BACKGROUND

[0002] In order to accelerate the convolutional neural network at the terminal side and thus quickly obtain the classification result of an image, existing researches choose to add a convolutional neural network accelerator in an intelligent terminal to realize inference of the neural network at the terminal side. For example, a neural-network processing unit (NPU) is provided in a smart phone to accelerate neural network calculation; for another example, a special chip EyeRiss designed for an Internet of Things intelligent terminal device is used to accelerate the convolutional neural network. However, since the above-mentioned chip is exposed in the intelligent terminal of the user side, the neural network model used for inference, including the structure and parameters of the neural network model, can be easily obtained by an attacker. Using the structure and parameters of the neural network model, the attacker can launch a white-box attack on the image to be classified in the intelligent terminal, so that the classification result output by the neural network is wrong.

[0003] At present, existing researches use gradient information of the neural network to generate an adversarial sample, and use the adversarial sample to train the neural network model to obtain a structure and parameters with high robustness. However, the structure and model with high robustness often have more non-zero values, which leads to very low efficiency when the neural network is used for inference. However, if the neural network is compressed at a large compression rate, the ability of the model to resist attacks will be reduced. That is, there is a certain contradiction between the robustness and efficiency of the neural network model.

[0004] The image classification method applied to the intelligent terminal in the existing researches is mostly a trade-off between robustness and efficiency, and cannot take into account the robustness and efficiency of the neural network model used for inference. SUMMARY

[0005] In order to solve the problem that the image classification method applied to the intelligent terminal in the existing researches is mostly a trade-off between robustness and efficiency, and cannot take into account the robustness and efficiency of the neural network model used for inference, the present application provides an image classification method and device for an intelligent terminal through the following aspects.

[0006] In a first aspect, the present application provides an image classification method applied to an intelligent terminal. The image classification method comprises:

[0007] In response to the image classification task, a candidate model is randomly selected from the candidate model set to obtain a target model; wherein the candidate model set includes a plurality of candidate models, and the plurality of candidate models are obtained by the cloud GPU from the same original model according to different sparse thresholds and structured sparse;

[0008] The image classification task is performed using the target model to obtain a classification result.

[0009] In a possible implementation, the classification accuracy of the candidate model is greater than or equal to 80% of the classification accuracy of the original model.

[0010] In a possible implementation, before the image classification method responds to the image classification task, the image classification method further includes:

[0011] The plurality of candidate models are sequentially coded from small to large according to the number of model parameters by the cloud GPU to obtain a coding set;

[0012] Wherein, the coding of the candidate model with the least number of parameters is 0, and the coding of the candidate model with the most number of parameters is a preset value, and the preset value is the number of the plurality of candidate models minus one.

[0013] In a possible implementation, the candidate model is randomly selected from the candidate model set to obtain the target model, including:

[0014] A number is randomly selected from 0 to the preset value to obtain a positioning number;

[0015] The target model is obtained according to the positioning number, the coding set and the candidate model set.

[0016] In a possible implementation, the original model is obtained by the cloud GPU according to the following method:

[0017] Obtain a training data set;

[0018] According to the training data set and the neural network model, an adversarial sample is generated using a preset attack method;

[0019] The neural network model is trained using the adversarial sample to obtain an updated neural network model;

[0020] The above process of generating an adversarial sample and training is repeated until convergence to obtain a trained neural network model;

[0021] The trained neural network model is quantized according to the data calculation bit width of the intelligent terminal to obtain the original model.

[0022] In a possible implementation, the neural network model is trained using the adversarial sample to obtain an updated neural network model, including:

[0023] The neural network model is trained using full-rank adversarial examples to obtain the first-stage neural network model.

[0024] Singular value pruning is used to structure the sparse first-stage neural network model, resulting in an updated neural network model.

[0025] The total loss function for training includes the training loss function and the orthogonal loss function.

[0026] In one possible implementation, the default attack method is either FGSM or PGD.

[0027] Secondly, this application provides an image classification device for use in a smart terminal. The image classification device is used to implement the image classification method for use in a smart terminal as provided in the first aspect of this application, and the image classification device includes: a model storage module, a random selection module, and a classification execution module;

[0028] The model storage module is used to store a set of candidate models. The set of candidate models includes multiple candidate models, which are obtained by the cloud GPU by sparsening the same original model according to different sparsity thresholds.

[0029] The random selection module is used to randomly select a candidate model from the candidate model set in response to the image classification task, and obtain the target model;

[0030] The classification execution module is used to perform image classification tasks using the target model and obtain classification results.

[0031] In one possible implementation, the random selection module includes a random number generation unit and a decoding unit; wherein,

[0032] The random number generation unit is used to respond to the image classification task by randomly selecting a number from 0 to a preset value to obtain the location number;

[0033] The decoding unit is used to obtain the target model based on the positioning number, the encoding set, and the candidate model set. The encoding set is obtained by the cloud GPU encoding multiple candidate models in ascending order of the number of model parameters. The candidate model with the fewest parameters is encoded as 0, and the candidate model with the most parameters is encoded as a preset value, which is the number of multiple candidate models minus one.

[0034] In one possible implementation, the image classification device further includes a communication module;

[0035] The communication module is used to obtain a set of candidate models from the cloud GPU.

[0036] The original model is obtained by a cloud GPU according to the following method:

[0037] obtain a training data set;

[0038] According to the training data set and the neural network model, an adversarial sample is generated using a preset attack method;

[0039] The neural network model is trained using the adversarial sample to obtain an updated neural network model;

[0040] The above process of generating an adversarial sample and training is repeated until convergence, and a trained neural network model is obtained;

[0041] According to the data bit width of the intelligent terminal, the trained neural network model is quantized to obtain the original model.

[0042] The image classification method provided by the application comprises: in response to an image classification task, a candidate model is randomly selected from a candidate model set to obtain a target model; and the target model is used to perform the image classification task to obtain a classification result. The candidate model set comprises a plurality of candidate models, and the plurality of candidate models are obtained by structuring a sparse original model according to different sparse thresholds by a cloud GPU. Even if an attacker obtains all information used for reasoning, including a model structure, parameters, a random strategy and a candidate model set used for random selection, the image classification method provided by the application cannot know which model in the candidate model set used for reasoning is currently used. Therefore, regardless of whether an attacker generates an adversarial sample using the same random strategy or generates an attack sample using a fixed model, the image classification method provided by the application can balance the robustness and efficiency of the neural network model used for reasoning. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the application, the drawings required in the embodiments will be briefly introduced below. Obviously, other drawings can also be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0044] Figure 1 A workflow diagram of an image classification method applied to an intelligent terminal provided by an embodiment of the application;

[0045] Figure 2 A workflow diagram of obtaining the original model in an image classification method applied to an intelligent terminal provided by an embodiment of the application;

[0046] Figure 3A training process schematic diagram for obtaining the updated neural network model in the image classification method applied to the intelligent terminal provided by an embodiment of the present application is provided.

[0047] Figure 4 A training process code schematic diagram for obtaining the trained neural network model in the image classification method applied to the intelligent terminal provided by an embodiment of the present application is provided.

[0048] Figure 5 A structure schematic diagram of the candidate model set in the image classification method applied to the intelligent terminal provided by an embodiment of the present application is provided.

[0049] Figure 6 A classification accuracy transfer schematic diagram for not passing pruning energy when attacking and using different pruning energies when reasoning in the simulation experiment is provided.

[0050] Figure 7 A structure schematic diagram of the image classification device applied to the intelligent terminal provided by an embodiment of the present application is provided.

[0051] Figure 8 A structure schematic diagram of the image classification device applied to the intelligent terminal provided by another embodiment of the present application is provided. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0053] In order to solve the problem that the current research on the image classification method applied to the intelligent terminal is mainly between robustness and high efficiency, and cannot balance the robustness and high efficiency of the neural network model for reasoning, the present application provides an image classification method and device for intelligent terminal through the following embodiments.

[0054] Referring to Figure 1 The first embodiment of the present application provides an image classification method applied to the intelligent terminal. The image classification method comprises:

[0055] Step 11, in response to an image classification task, randomly selecting a candidate model from a candidate model set to obtain a target model; wherein the candidate model set comprises a plurality of candidate models, and the plurality of candidate models are obtained by a cloud GPU (Graphics Processing Unit, graphics processing unit) from a same original model according to different sparse thresholds and structured sparse.

[0056] Step 12, using the target model to perform the image classification task, and obtaining a classification result. When the intelligent terminal needs to perform the next classification task, steps 11-12 are re-executed, and the cycle continues.

[0057] In some embodiments, referring to Figure 2 , the original model is obtained by a cloud GPU according to the method shown in steps 21-25.

[0058] Step 21, obtaining a training data set.

[0059] Step 22, generating an adversarial sample using a preset attack method according to the training data set and a neural network model.

[0060] A neural network is easily affected by a certain degree of noise (perturbation) and causes classification errors. In actual applications, an attacker can generate adversarial noise specifically for the neural network, so that the classification accuracy of the neural network becomes 0. These adversarial noises can be generated by attack methods including but not limited to FGSM (Fast Gradient Sign Method), PGD (Project Gradient Descent), and AutoAttack. Due to the existence of adversarial attacks, the use of neural networks in actual scenarios will face very serious security problems, for example, in autonomous driving, if an attacker uses adversarial noise to attack, the erroneous classification of the neural network can cause serious traffic accidents.

[0061] In this embodiment, an adversarial sample is generated using a preset attack method according to the training data set and the neural network model. The subsequent training process is performed using the adversarial sample, so that the trained neural network model has higher robustness. For example, the preset attack method uses FSGM or PGD. It should be noted that the generation method of the adversarial sample is not limited to FGSM and PGD, and different attack methods can be selected according to specific application scenarios.

[0062] Step 23, training the neural network model using the adversarial sample, and obtaining an updated neural network model.

[0063] Step 24, repeating the above generation of adversarial samples and training process until convergence, and obtaining a trained neural network model. Using the updated neural network model and the training data set, an adversarial sample is generated using a preset attack method, and then training is performed. Such iteration is performed until the training process converges, that is, the classification accuracy is stable at a high level, the training is stopped, and a trained neural network model is obtained. The trained neural network model can be further structured and sparse by some parameters, such as rank energy.

[0064] Step 25, calculating the bit width of the data of the intelligent terminal, quantizing the trained neural network model to obtain the original model.

[0065] The trained neural network model obtained by the cloud GPU is generally a 32-bit floating point number, while the data in the intelligent terminal device is more diverse, which can be a 32-bit, 16-bit, 8-bit floating point number, or a 32-bit, 16-bit, 8-bit fixed point number.

[0066] For example, if the data calculation bit width of the intelligent terminal is 8-bit fixed point number, the trained neural network model is quantized according to 8-bit fixed point to obtain the original model; if the data calculation bit width of the intelligent terminal is 32-bit floating point number, the trained neural network model is the original model.

[0067] In order to improve the execution efficiency of the neural network, the neural network is trained in a structured sparse manner to obtain an updated neural network model. In one possible implementation, step 23 includes step 231 and step 232.

[0068] Step 231, training the neural network model using the full rank of the adversarial sample to obtain a first stage neural network model.

[0069] Step 232, structuring and sparsifying the first stage neural network model in a singular value pruning manner to obtain the updated neural network model.

[0070] As shown in Figure 3 , the structure of the neural network includes multiple layers of weight data (each cube in the upper left represents the weight of a layer), batch normalization operation and nonlinear operation, etc. It should be noted that because the batch normalization and nonlinear operation are not the focus of the present implementation, and the calculation method is the same as that in other training methods, the batch normalization and nonlinear operation are omitted in Figure 3 . In addition, it should be noted that the connection method of each layer of weight data is not limited to the above method. In some network structures, there may be some bypass connections, but no matter what the structure of the network is, the method described below can be used to decompose the weight part to obtain an updated neural network model. Figure 3

[0071] Referring to Figure 3 ​, the weight matrix includes 4 dimensions, the length and width of the convolution kernel are k, the number of channels is c, and the number of convolution kernels is m, denoted as K. Because any matrix can be decomposed into two smaller orthogonal matrices U and V and a singular value matrix s, in the present implementation, the decomposition is used to reduce the calculation amount of the neural network. First, the weight matrix K is transformed into three parts, that is, U ∈ R m×r , V ∈ R r×ckk and s ∈ R r , where U and V are orthogonal matrices, and r is the rank of the weight matrix K, r = min(m, ckk). Then, the three parts are transformed back into the form of a 4-dimensional matrix, and two sub-weight matrices are obtained, that is, and where diag(·) represents a diagonal matrix. K1 and K2 are the full-rank decomposition forms of the weight matrix K, corresponding to the two sub-weight matrices in the first step of the full-rank training in Figure 3

[0072] Because the full-rank form can express more information of the original weight matrix, in the present implementation, the weight matrix K1 and K2 in the full-rank decomposition form are used for training. In this process, K1 and K2 are trainable parameters.

[0073] However, the first-stage neural network model obtained in this training process is not a lightweight model. In order to improve the inference efficiency, a second stage is added in the training process, that is, the singular value pruning method is used to structure and sparsify the first-stage neural network model, and the updated neural network model is obtained. This process is also called low-rank fine-tuning training. In the low-rank fine-tuning training process, the first-stage neural network model is structured and sparsified according to the following singular value pruning formula: where S is the set of all singular values, and e ∈ [0, 1] is a predefined threshold. That is, the singular values that satisfy the above formula are set to 0, and therefore the corresponding calculation process can also be skipped. By using the low-rank fine-tuning training method, the smaller singular values in the S set can be removed, and the classification accuracy loss of the updated neural network model is very small.

[0074] It should be noted that, by the low-rank fine-tuning training method, K1 and K2 cannot be guaranteed to be orthogonal, because they are free in the training process. Therefore, after the smaller singular values are pruned, the updated neural network model will suffer a very serious accuracy loss. In order to avoid this problem, in the present implementation, an orthogonal regularization term is additionally introduced into the total loss function of the training, that is, the total loss function of the training is: L = L N [U, s, V] + λL υ ​(U,V); where L N Consistent with the loss function used during training of a typical neural network, L υ It is an orthogonal loss function, and λ is a hyperparameter that can adjust the impact of the orthogonal loss function on the entire training process.

[0075] Among them, L υ Calculate using the following formula: Among them, ||·|| F is the Forbes regularization of the matrix, and L is the total number of layers in the neural network.

[0076] When training the weight parameters in the neural network model, backpropagation is performed using the total loss function L to continuously update the decomposed weight matrices K1 and K2. In this embodiment, adversarial examples are generated using FGSM or PGD methods and used as input to the neural network training process. When the training process converges, i.e., the classification accuracy stabilizes at a high level, training is stopped, resulting in a robust and efficient trained neural network model.

[0077] The algorithm code for steps 21-25 of the above training process is described as follows: Figure 4 As shown. (Through) Figure 4 The algorithm code shown can train a lightweight and robust model f0.

[0078] It should be noted that other training methods can also be used to obtain the trained neural network model, as long as it is ensured that the trained neural network model can be further structured and sparsed through some parameters.

[0079] Before performing further structured sparsity to obtain candidate models, the bit width needs to be calculated based on the data from the smart terminal device, and the trained neural network model f0 needs to be quantized to obtain the original model.

[0080] The original model The structured sparsity method, as shown in the singular value pruning formula above, can be used to obtain candidate models with fewer parameters. No retraining is required. See also Figure 5 This series of candidate models is referred to as the candidate model set. It should be noted that, in this embodiment, the original model... It is also used as a candidate model in the subsequent random selection process.

[0081] In one implementation, to ensure the robustness of the image classification method, the classification accuracy of the candidate models needs to be guaranteed. Therefore, on a cloud GPU, the classification accuracy of each candidate model is evaluated using a test dataset. If the classification accuracy of a candidate model is higher than that of the original model... If the classification accuracy of the model drops by more than 20% compared to the original model's accuracy, the corresponding model is discarded. This approach also ensures that the number of candidate models in the candidate model set is limited, making it easier to randomly select a model for inference in subsequent iterations.

[0082] In one implementation, the number of candidate models used for random switching can be adjusted according to actual needs, thereby obtaining a set of candidate models with different robustness and efficiency.

[0083] like Figure 5 As shown, since the candidate models share weight parameters and each candidate model is incremental, that is, the original model... It contains the largest number of parameters. yes a subset of yes A subset of the candidate models, and so on. Therefore, the candidate model set is encoded, and the model parameters are stored according to the encoding, so that the smart terminal can quickly locate the parameters of the candidate models based on the encoding.

[0084] In one implementation, prior to responding to the image classification task, the image classification method further includes: encoding the plurality of candidate models sequentially from smallest to largest number of model parameters using a cloud GPU to obtain an encoding set; wherein the candidate model with the fewest parameters is encoded as 0, and the candidate model with the most parameters is encoded as a preset value n, wherein the preset value n is the number of the plurality of candidate models minus one.

[0085] Furthermore, if the candidate model set contains 4 candidate models, one encoding method is: [The code snippet is missing from the original text]. All parameters are encoded as 0, which will belong to the candidate model. Not a candidate model The model parameters are encoded as 1, which will belong to the candidate model. Not a candidate model The model parameters are encoded as 2, which will belong to the candidate model. Not a candidate model The model parameters are encoded as 3, and all the encodings are called the encoding set.

[0086] Correspondingly, in this implementation, the candidate models in the candidate model set are determined by encoding, replacing the original model. all parameters of the original model are stored. In this way, the hardware area of the storage resource on the intelligent terminal can be saved. When performing the classification task, the intelligent terminal randomly selects a number from 0 to a preset value to obtain a positioning number; and obtains the target model according to the positioning number, the code set and the candidate model set. For example, there are 4 candidate models in the candidate model set, and the corresponding code set is {0, 1, 2, 3}; the intelligent terminal generates a random number 2, and then takes out the parameters corresponding to all numbers less than or equal to 2 from the candidate model set, that is, takes out the parameters corresponding to 0, 1 and 2 to obtain the target model.

[0087] That is, when the multiple candidate models in the candidate model set are stored in the intelligent terminal, all parameters of the original model are stored, and then all parameters of the original model are divided according to the code set. Each time the image classification task is performed, the target model is determined according to the randomly generated positioning number. By using the above code set form, the hardware resources occupied by the candidate model set can be greatly reduced.

[0088] The embodiment provides an image classification method applied to an intelligent terminal. The image classification method comprises the following steps: in response to an image classification task, randomly selecting a candidate model from a candidate model set to obtain a target model; and performing the image classification task by using the target model to obtain a classification result. The candidate model set comprises multiple candidate models, and the multiple candidate models are obtained by structurally sparsifying an original model according to different sparse thresholds by a cloud GPU. By using the image classification method provided in the embodiment, even if an attacker obtains all information used for reasoning, including a model structure, parameters, a random strategy and a candidate model set used for randomness, the attacker cannot know which model in the candidate model set used for reasoning is currently used. Therefore, no matter whether the attacker generates an adversarial sample by using the same random strategy or generates an attack sample by using a fixed model, the image classification method provided in the embodiment can balance the robustness and high efficiency of the neural network model used for reasoning.

[0089] In order to illustrate the beneficial effects of the image classification method applied to the intelligent terminal provided in the embodiment, the PreActResNet18 network is used as a neural network model for simulation experiment. In the simulation experiment, the PGD-7 method is used to generate an attack sample, adversarial training is performed, a trained neural network model is obtained, and then the PGD-20 method is used to attack the intelligent terminal. Referring to FIG. 6, the classification accuracy transfer conditions of the simulation experiment under different attack models and reasoning models are shown. Figure 6 Figure 6 ​​It can be seen from the results shown that the classification accuracy when the pruning energy during reasoning and the pruning energy during attack are different is greater than the classification accuracy when the pruning energy during reasoning and the pruning energy during attack are the same.

[0090] That is, when the random strategy is adopted, even if the attacker obtains the candidate model set, the encoding set and the random strategy used for reasoning, because the selection of the target model is random, the attacker cannot select the same target model as the intelligent terminal with high probability, thereby greatly improving the robustness of the neural network used for reasoning by the intelligent terminal. Further, if any one of the candidate model set, the encoding set and the random strategy used for reasoning is not obtained by the attacker, the robustness of the neural network used for reasoning by the intelligent terminal can be further increased. At the same time, in the embodiment, only one training is needed to obtain a candidate model set, and the calculation amount of the candidate model in the candidate model set is less than that of the original model, and the efficiency is not reduced on the basis of improving the robustness.

[0091] Corresponding to the foregoing one embodiment of the image classification method applied to the intelligent terminal, the second embodiment of the present application provides an image classification device applied to the intelligent terminal. Referring to Figure 7 , the image classification device comprises a model storage module, a random selection module and a classification execution module.

[0092] The model storage module is configured to store a candidate model set, the candidate model set comprising a plurality of candidate models, the plurality of candidate models being obtained by a cloud GPU from one original model according to different sparse thresholds and structured sparse.

[0093] The random selection module is configured to randomly select one candidate model from the candidate model set to obtain a target model in response to an image classification task.

[0094] The classification execution module is configured to execute the image classification task using the target model to obtain a classification result.

[0095] Referring to Figure 8 , in one implementation manner, the random selection module comprises a random number generation unit and a decoding unit; wherein,

[0096] The random number generation unit is configured to randomly select a number from 0 to a preset value to obtain a positioning number in response to an image classification task.

[0097] The decoding unit is configured to obtain the target model according to the positioning number, the encoding set and the candidate model set; wherein the encoding set is obtained by sequentially encoding the plurality of candidate models from small to large according to the number of model parameters by the cloud GPU; wherein the encoding of the candidate model with the least number of parameters is 0, and the encoding of the candidate model with the most number of parameters is a preset value, and the preset value is the number of the plurality of candidate models minus one. That is, the original model is stored in the form of cloud GPU encoding in the model storage module When the random number generation unit randomly generates the positioning number, the decoding unit determines which parameters participate in the formation of the target model from the encoding set according to the positioning number, so as to obtain the target model by taking out the corresponding parameters from the model storage module.

[0098] As shown in Figure 8 In an implementation manner, the image classification device further includes a communication module. The communication module is configured to obtain the candidate model set from the cloud GPU.

[0099] The original model is obtained by the cloud GPU according to the following method: obtaining a training data set; generating an adversarial sample using a preset attack method according to the training data set and a neural network model; training the neural network model using the adversarial sample to obtain an updated neural network model; repeating the above processes of generating an adversarial sample and training until convergence to obtain a trained neural network model; and quantizing the trained neural network model according to the data calculation bit width of the intelligent terminal to obtain the original model.

[0100] Further, the communication module is further configured to obtain the encoding set from the cloud GPU and transmit to the decoding unit.

[0101] Further, in actual application, the image classification device of the intelligent terminal can also transmit the image collected in the image classification task to the cloud GPU through the communication module. The cloud GPU generates a new training data set according to the collected image. According to the new training data set and the previously trained neural network model, the adversarial sample is regenerated, and the adversarial training is further performed to update the original model.

[0102] The image classification device provided in the embodiment only needs to add a random selection module in the existing neural network hardware acceleration in the intelligent terminal, such as EyeRiss accelerator or NPU. Other effects of the image classification device provided in the embodiment when the foregoing method is applied can be referred to the description in the foregoing method embodiments, which will not be described herein.

[0103] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0104] Similar parts in various embodiments of the specification are cross-referenced.

Claims

1. An image classification method applied to a smart terminal, characterized in that, The method comprises the following steps: In response to an image classification task, a candidate model is randomly selected from a candidate model set to obtain a target model; wherein the candidate model set comprises a plurality of candidate models, and the plurality of candidate models are obtained by structurally sparsifying one original model by a cloud GPU according to different sparse thresholds; Original model contains the largest number of parameters, is a subset of is a subset of, and so on; When the plurality of candidate models in the candidate model set are stored in the intelligent terminal, all parameters of the original model are stored, and then the all parameters of the original model are divided according to an encoding set; The image classification task is executed using the target model to obtain a classification result; Before responding to the image classification task, the image classification method further comprises: The plurality of candidate models are sequentially encoded from small to large according to the number of model parameters by a cloud GPU to obtain an encoding set; Will belong to the candidate model All parameters are encoded as 0, which will belong to the candidate model. Not a candidate model The model parameters are encoded as 1, which will belong to the candidate model. Not a candidate model The model parameters are encoded as 2, which will belong to the candidate model. Not a candidate model The model parameters are encoded as 3, and all the encodings are called the encoding set; Wherein the encoding of the candidate model with the least number of parameters is 0, and the encoding of the candidate model with the most number of parameters is a preset value n, and the preset value is the number of the plurality of candidate models minus one; The plurality of candidate models in the candidate model set are stored in the intelligent terminal as original models having all parameters, and then all parameters of the original models are divided according to the coding set. The target model is obtained by randomly selecting one candidate model from the candidate model set, comprising: A positioning number is randomly selected from 0 to the preset value to obtain the positioning number; The target model is obtained according to the positioning number, the encoding set and the candidate model set. 2.The image classification method applied to a smart terminal according to claim 1, characterized in that, The classification accuracy of the candidate model is greater than or equal to 80% of the classification accuracy of the original model. 3.The image classification method applied to a smart terminal according to claim 1, characterized in that, The original model is obtained by a cloud GPU according to the following method: Obtain a training data set; Generate an adversarial sample using a preset attack method according to the training data set and a neural network model; Train the neural network model using the adversarial sample to obtain an updated neural network model; Repeat the process of generating an adversarial sample and training until convergence to obtain a trained neural network model; Quantize the trained neural network model according to the data calculation bit width of the intelligent terminal to obtain the original model. 4.The image classification method applied to a smart terminal according to claim 3, characterized in that, Training the neural network model using the adversarial sample to obtain an updated neural network model comprises: Training the neural network model using the adversarial sample to obtain a first-stage neural network model; Structurally sparse the first-stage neural network model by singular value pruning to obtain the updated neural network model; Wherein, the total loss function of training includes training loss function and orthogonal loss function. 5.The image classification method applied to a smart terminal according to claim 3, characterized in that, The preset attack method is FGSM method or PGD method.

6. An image classification apparatus applied to a smart terminal, characterized by, The image classification device is used to implement the image classification method applied to the intelligent terminal according to any one of claims 1-5, and the image classification device comprises a model storage module, a random selection module and a classification execution module; Wherein, the model storage module is used to store a candidate model set, and the candidate model set comprises a plurality of candidate models, and the plurality of candidate models are obtained by structurally sparsifying one original model by a cloud GPU according to different sparse thresholds; The random selection module is used to randomly select a candidate model from the candidate model set to obtain a target model in response to an image classification task; The classification execution module is configured to execute the image classification task using the target model to obtain a classification result. The random selection module comprises a random number generation unit and a decoding unit. The random number generation unit is configured to randomly select a number from 0 to a preset value in response to an image classification task to obtain a positioning number. The decoding unit is configured to obtain the target model according to the positioning number, an encoding set, and the candidate model set, wherein the encoding set is obtained by sequentially encoding the plurality of candidate models from small to large according to the number of model parameters by the cloud GPU, wherein the encoding of the candidate model with the least number of parameters is 0, the encoding of the candidate model with the most number of parameters is the preset value, and the preset value is the number of the plurality of candidate models minus one. 7.The image classification apparatus for a smart terminal according to claim 6, wherein, The communication module is further included. The communication module is configured to obtain the candidate model set from the cloud GPU. The original model is obtained by the cloud GPU according to the following method: Obtain a training data set; Generate an adversarial sample using a preset attack method according to the training data set and a neural network model; Train the neural network model using the adversarial sample to obtain an updated neural network model; Repeat the above process of generating an adversarial sample and training until convergence to obtain a trained neural network model; Quantize the trained neural network model according to the data calculation bit width of the intelligent terminal to obtain the original model.

Images