Image classification model training method, image processing method and related device

Abstract

The invention belongs to the field of image processing, and discloses an image classification model training method, an image processing method and a related device, and the method comprises the steps: obtaining and inputting a training image into an image classification model for forward propagation, and determining an enhancement state of a current to-be-enhanced network layer of the image classification model through a random Batch scheduler; when the enhancement state of the current to-be-enhanced network layer is enhancement, performing data enhancement on the feature data of the training image in the current to-be-enhanced network layer, and continuing forward propagation; and when data enhancement is performed, adding a self-distillation dynamic soft label regular term into a loss function of the image classification model to obtain an optimized loss function, and optimizing model parameters by optimizing the loss function according to a classification result of the training image and a real label. The training efficiency of the image classification model is improved, the noise resistance of the image classification model is improved, and the stability and credibility of the image classification model in practical application are effectively improved.

Description

Technical Field

[0001] The invention belongs to the field of image processing and relates to an image classification model training method, an image processing method and related devices. Background Art

[0002] In recent years, deep learning has made significant progress in many application areas, including vision (such as object detection and semantic segmentation), speech (such as sound source localization and tracking and speech recognition), and text (such as entity recognition and automatic language translation), and has continuously refreshed accuracy and other key performance indicators. In machine learning, the sample average error is minimized as the optimization objective function for model training to reach convergence or appropriate accuracy. It has also been widely adopted in the training of deep image classification models. This learning rule is also called empirical risk minimization (ERM). However, when there is severe label noise or even when the labels are randomly assigned in the classification task, ERM can still make the model have high accuracy on the training data set. This memory behavior of the neural network indicates that the model has not learned the correct invariant features of the category and there is a serious overfitting phenomenon. Even if a regularization term (such as Lasso regularization) is added to the objective function, this network memory behavior cannot be significantly alleviated. In addition, image classification models trained with ERM will only perform well when the test data is very similar to the data distribution in the training set. When evaluated with slightly different data distributions, such as adversarial samples, they often give very obvious wrong predictions with very high confidence. Indirectly, this shows that ERM cannot explain or provide generalization capabilities when the test distribution is inconsistent with the training data distribution. In the noisy environment of actual deployment, it is extremely common for the inferred data to be inconsistent with the training data distribution. The vulnerability of this model can easily cause malicious attacks, which has caused serious concerns about the security of image classification models in many real-world applications, especially in safety-critical scenarios such as unmanned driving and biometrics.

[0003] The above challenges have triggered the exploration of the robustness of image classification models in academia and industry from multiple aspects such as training algorithms, data enhancement, learning mechanisms and model architecture design, including the study and analysis of the reasons for the existence of image adversarial samples. Based on the defects of ERM in training image classification models with many parameters, the robustness of image classification models is generally improved from the perspective of data enhancement, and supervised learning signals are constructed through derived enhanced image data. First, new training data is synthesized based on the original image data through the neighboring risk minimization (VRM) principle. Specifically, data enhancement describes the distribution of neighborhood training data with the same semantics as the original image sample, which often requires prior knowledge. For example, when performing image classification, the neighborhood of an image can be defined as its horizontal inversion, rotation and scaling. Then, the support of the original training distribution is expanded by extracting additional virtual samples from the neighborhood distribution of training samples. It is worth noting that recent studies have shown that image data enhancement can improve the generalization ability and robustness of the model, whether in the original space or the representation space, regardless of whether the label preservation strategy is used.

[0004] Nevertheless, existing image data augmentation schemes have two limitations: first, the augmentation process depends on the dataset, and therefore requires the use of expert knowledge; second, the learning process largely ignores the similarity of the neighborhood distribution of the synthetic image data and the original image data, and does not take into account the prior distribution information of the synthetic image data around the original image data, so that the label consistency of the image classification model is not guaranteed during the training process. Therefore, the subsequent prediction behavior of the image classification model will not be explicitly optimized for this, but this stable behavior is crucial for the security of the image classification model in adversarial environments. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide an image classification model training method, an image processing method and related devices.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides an image classification model training method, comprising: obtaining and inputting a training image into an image classification model for forward propagation; wherein, during the forward propagation process, determining the enhancement state of the current network layer to be enhanced of the image classification model through a random batch scheduler; when the enhancement state of the current network layer to be enhanced is enhancement, performing data enhancement on the feature data of the training image in the current network layer to be enhanced and obtaining the classification soft label of the training image after the data enhancement, and continuing the forward propagation; when the enhancement state of the current network layer to be enhanced is no enhancement, continuing the forward propagation; wherein the network layer to be enhanced is a convolutional layer and a fully connected layer in the image classification model; after the forward propagation is completed, performing a parameter optimization step of the image classification model; wherein, when data enhancement is performed, the parameter optimization step comprises: adding a self-distillation dynamic soft label regularization term to the loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameters by optimizing the loss function according to the classification results and true labels of the training images; when data enhancement is not performed, the parameter optimization step comprises: optimizing the model parameters by the loss function of the image classification model according to the classification results and true labels of the training images.

[0008] Optionally, the method of determining the enhancement state of the current network layer to be enhanced of the image classification model through the random Batch scheduler includes: for the current training image, using a Bernoulli process to generate a random vector, and each component in the random vector matches each network layer to be enhanced of the image classification model to be trained one by one; using each component in the random vector as the observation value of each matched network layer to be enhanced, when the observation value of the current network layer to be enhanced is 0, the enhancement state of the current network layer to be enhanced is enhancement; when the observation value of the current network layer to be enhanced is 1, the enhancement state of the current network layer to be enhanced is no enhancement.

[0009] Optionally, the data enhancement of the feature data of the training image in the current network layer to be enhanced includes: when the current network layer to be enhanced is a fully connected layer, neighborhood sampling of the feature data of the training image is performed by combining truncated Gaussian noise and Dropout technology.

[0010] Optionally, performing data enhancement on the feature data of the training image in the current network layer to be enhanced includes: when the current network layer to be enhanced is a convolutional layer, adding Gaussian noise to the feature data of the training image.

[0011] Optionally, the optimization loss function for:

[0012]

[0013] Among them, θ is the model parameter of the image classification model, B is the batch size of training images, η is the balance parameter, M is the total number of enhancements, and D KL is the KL distance, is the true label of the i-th training image, Y i is the soft label of the i-th training image, Y ij is the classification soft label of the i-th training image after the j-th data augmentation.

[0014] Optionally, the loss function of the image classification model is a standard classification cross entropy function; when optimizing the model parameters by optimizing the loss function or optimizing the model parameters by the loss function of the image classification model, the loss function value is passed to each network layer of the image classification model according to the chain rule, and the gradient descent method is used to update the weights and bias parameters of each network layer.

[0015] A second aspect of the present invention provides an image classification method, comprising: obtaining an image to be classified; inputting the image to be classified into a preset image classification model to obtain a classification result; wherein the image classification model is trained using the above-mentioned image classification model training method.

[0016] In a third aspect, the present invention provides an image classification model training system, comprising: a data enhancement module, for acquiring and inputting a training image into an image classification model for forward propagation, and obtaining a classification result of the training image after the forward propagation is completed; wherein, during the forward propagation process, the enhancement state of the current network layer to be enhanced of the image classification model is determined by a random batch scheduler; when the enhancement state of the current network layer to be enhanced is to perform enhancement, data enhancement is performed on the feature data of the training image in the current network layer to be enhanced and a classification soft label of the training image after the data enhancement is obtained, and forward propagation is continued; when the enhancement state of the current network layer to be enhanced is not to perform enhancement, When enhancing, forward propagation continues; wherein, the network layer to be enhanced is the convolutional layer and the fully connected layer in the image classification model; the model parameter optimization module is used to perform the parameter optimization step of the image classification model after the forward propagation is completed; wherein, when data enhancement has been performed, the parameter optimization step includes: adding the self-distillation dynamic soft label regularization term to the loss function of the image classification model to obtain the optimized loss function, and optimizing the model parameters by optimizing the loss function according to the classification results and the true labels of the training images; when data enhancement has not been performed, the parameter optimization step includes: optimizing the model parameters by the loss function of the image classification model according to the classification results and the true labels of the training images.

[0017] In a fourth aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned image classification model training method or the steps of the above-mentioned image classification method when executing the computer program.

[0018] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned image classification model training method or the steps of the above-mentioned image classification method are implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The image classification model training method of the present invention determines the enhancement state of the current network layer to be enhanced by a random batch scheduler, realizes random induction of the neighborhood of the representation space sample, thereby reducing the redundant calculation cost and improving the training efficiency of the image classification model. The self-distillation dynamic soft label regularization term is added to the loss function of the image classification model to obtain an optimized loss function, and the model parameters are optimized by optimizing the loss function. The self-distillation dynamic soft label guides the image classification model to fit the neighborhood of the original image sample in multiple semantic levels by connecting the predicted distribution between the original image sample and the enhanced image sample, thereby improving the image classification model's ability to resist noise, so that the final image processing model has a stronger resistance to subtle disturbances of the image, especially when the inferred image is inconsistent with the training image distribution, such as image distribution drift and image adversarial attack, the image classification model trained by the image classification model training method can still provide a relatively accurate classification result in a high confidence manner, which not only enhances the adaptability of the image classification model to unknown or abnormal image data, but also effectively improves the stability and credibility of the image classification model in practical applications, thereby laying a solid foundation for realizing more extensive and safer image recognition and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the image classification model training method according to an embodiment of the present invention.

[0022] Figure 2 This is a structural block diagram of an image classification model training system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0026] See also Figure 1 In one embodiment of the present invention, an image classification model training method is provided. Based on data enhancement, an adversarial defense route based on random induction of the neighborhood of characterizing spatial data is designed to improve the robustness of the image classification model, thereby ensuring the classification results of the image classification model on actual image data.

[0027] Specifically, the image classification model training method of the present invention comprises the following steps:

[0028] S1: Obtain and input the training image into the image classification model for forward propagation, and obtain the classification result of the training image after the forward propagation is completed.

[0029] Among them, during the forward propagation process, the enhancement state of the current network layer to be enhanced of the image classification model is determined by a random batch scheduler; when the enhancement state of the current network layer to be enhanced is enhancement, data enhancement is performed on the feature data of the training image in the current network layer to be enhanced, and forward propagation continues; when the enhancement state of the current network layer to be enhanced is no enhancement, forward propagation continues; among them, the network layer to be enhanced is the convolutional layer and the fully connected layer in the image classification model.

[0030] S2: Perform parameter optimization steps for the image classification model.

[0031] Among them, when data enhancement has been performed, the parameter optimization step includes: adding the self-distillation dynamic soft label regularization term to the loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameters by optimizing the loss function according to the classification results and true labels of the training images; when data enhancement has not been performed, the parameter optimization step includes: optimizing the model parameters by the loss function of the image classification model according to the classification results and true labels of the training images.

[0032] The image classification model training method of the present invention determines the enhancement state of the current network layer to be enhanced by a random batch scheduler, realizes random induction of the neighborhood of the representation space sample, thereby reducing the redundant calculation cost and improving the training efficiency of the image classification model. The self-distillation dynamic soft label regularization term is added to the loss function of the image classification model to obtain an optimized loss function, and the model parameters are optimized by optimizing the loss function. The self-distillation dynamic soft label guides the image classification model to fit the neighborhood of the original image sample in multiple semantic levels by connecting the predicted distribution between the original image sample and the enhanced image sample, thereby improving the image classification model's ability to resist noise, so that the final image processing model has a stronger resistance to subtle disturbances of the image, especially when the inferred image is inconsistent with the training image distribution, such as image distribution drift and image adversarial attack, the image classification model trained by the image classification model training method can still provide a relatively accurate classification result in a high confidence manner, which not only enhances the adaptability of the image classification model to unknown or abnormal image data, but also effectively improves the stability and credibility of the image classification model in practical applications, thereby laying a solid foundation for realizing more extensive and safer image recognition and application.

[0033] In a possible implementation, determining the enhancement state of the current network layer to be enhanced by a random Batch scheduler includes: for the current training image, using a Bernoulli process to generate a random vector λ, and each component in the random vector λ matches each network layer to be enhanced of the image classification model to be trained one by one; using each component in the random vector λ as the observation value of each matched network layer to be enhanced, when the observation value of the network layer to be enhanced is 0, the enhancement state of the network layer to be enhanced is enhancement; when the observation value of the network layer to be enhanced is 1, the enhancement state of the network layer to be enhanced is no enhancement.

[0034] Specifically, the convolutional layer and the fully connected layer in the image classification model are extracted as a set of network layers to be enhanced s. The random batch scheduler uses the Bernoulli process to generate a random vector λ, in which each component is independent, that is, the Layer-Wise method can be adopted to achieve fine adjustment, which can be expressed as:

[0035]

[0036] Where λ(t) represents the specific state of the random batch scheduler at time t, and the superscript i j Where i represents the i-th training image, j∈[s] is the index of the selected network layer s to be enhanced according to its order in the set of network layers to be enhanced s. That is, if the observation value of the jth layer is 1, then the layer is enhanced; if That is, if the observed value of the jth layer is 0, then the layer is not enhanced. Each item on the right is a random batch scheduling strategy for each layer in the image classification model training, which is determined by the corresponding observed value on the left.

[0037] In a possible implementation, the data enhancement of the feature data of the training image in the current network layer to be enhanced includes: when the current network layer to be enhanced is a fully connected layer, neighborhood sampling of the feature data of the training image is performed by combining truncated Gaussian noise and Dropout technology.

[0038] Specifically, the image data is subjected to neighborhood sampling of the feature data of the training image by combining truncated Gaussian noise and Dropout technology at the fully connected layer, that is, adjacent pixels in the feature data of the training image are sampled to obtain enhanced image data.

[0039] Explanatory, truncated Gaussian noise is a specific type of random noise that conforms to the Gaussian distribution (also known as the normal distribution), but restricts its value range. In machine learning, Gaussian noise often appears in various data processing and model training stages. There are two ways to deal with Gaussian noise: denoising and robust learning. The goal of denoising is to extract clean signals from noise-contaminated data, while the goal of robust learning is to make the model robust to noise, that is, to achieve good performance in the presence of noise. Dropout technology is a commonly used regularization technique designed to prevent neural networks from overfitting. Its basic idea is to randomly drop some neurons with a certain probability during the training process of the neural network. Specifically, the Dropout technology randomly selects some neurons in each training batch and sets their output to 0. This makes the model more robust, reduces dependence on specific neurons, and thus improves the generalization ability of the model. At test time, all neurons are involved in the calculation, but the output of each neuron is multiplied by a scaling factor to ensure that the output of the model at test time is the same as that at training time.

[0040] In a possible implementation, performing data enhancement on the feature data of the training image in the current network layer to be enhanced includes: when the current network layer to be enhanced is a convolutional layer, adding Gaussian noise to the feature data of the training image.

[0041] Specifically, in the convolution layer, Gaussian noise is only used for the feature data of the training image, that is, noise obeying the normal distribution is added to the feature data of the training image to obtain enhanced image data.

[0042] In a possible implementation, the loss function of the image classification model is a standard classification cross entropy function. Specifically, the standard classification cross entropy function is used as the optimization target to ensure that the image classification model learns on the original training image and quickly reaches a certain recognition accuracy.

[0043] In a possible implementation, the optimization loss function for:

[0044]

[0045] Among them, θ is the model parameter of the image classification model, B is the batch size of training images, η is the balance parameter, M is the total number of enhancements, and D KL is the KL distance, is the true label of the i-th training image, Y i is the soft label of the i-th training image, Y ij is the classification soft label of the i-th training image after the j-th data augmentation.

[0046] The above optimization loss function middle, is the standard categorical cross entropy function, is the self-distillation dynamic soft label regularization term.

[0047] In one possible implementation, the loss function for:

[0048]

[0049] Specifically, the training image is input into the image classification model to obtain the SoftMax normalized probability classification result, i.e., the classification soft label of the training image, and then the classification result is used as the soft label of the training image after data enhancement, and then the enhanced image data is sent into the image classification model to obtain the classification soft label of the training image after data enhancement, and during the training process, the KL distance between the training image and the classification soft label of the training image after data enhancement is added as an additional regularization term, i.e., the self-distillation dynamic soft label regularization term, to the loss function of the image classification model, and the regularization term is used to provide a supervised learning signal in the loss function of the image classification model.

[0050] Explanatory, KL distance (Kullback-Leibler Divergence), also known as relative entropy, is an asymmetric indicator to measure the difference between two probability distributions P and Q. It reflects the additional information loss caused by using distribution Q for encoding when distribution P is known. KL distance is widely used in fields such as machine learning, information theory, and statistics, such as evaluating the difference between the model's predicted distribution and the true distribution, optimizing model parameters, etc. By minimizing the KL distance, the distribution learned by the model can be made closer to the distribution of the real data, thereby improving the accuracy and generalization ability of the model.

[0051] In general, the image classification model training method of the present invention characterizes the random induction of the spatial sample neighborhood and is modeled as a Bernoulli process, and the observation value of the t-th iteration determines whether to perform data enhancement on a certain network layer to be enhanced in the t-th iteration. The self-distillation dynamic soft label regularization term guides the image classification model to fit the original sample neighborhood in multiple semantic levels by connecting the predicted distribution between the original training image and the enhanced image data, thereby improving the image classification model's ability to resist noise. The standard classification cross entropy function is used as the main optimization target to ensure the learning of the image classification model on the original sample and quickly achieve a certain recognition accuracy. At the same time, the intermediate output distribution of the original training image is used to construct a self-supervisory signal to learn the model parameters, that is, the KL divergence between the predicted distribution of the enhanced image data and the predicted distribution of the original training image is considered, and the complementary information of the prediction is considered. The forced matching of the two inferred distributions makes the image classification model more consistent in the prediction behavior in the local neighborhood, so that the final image classification model has a stronger resistance to subtle disturbances. This method can also reduce redundant computing costs and improve the training efficiency of the image classification model.

[0052] In a possible implementation, after experimental verification, the image classification model training method of the present invention, even if different image classification models, such as VGG16 or ResNet44, are used, the error rate on CIFAR-10, CIFAR-100 and ImageNet data sets is significantly reduced, indicating that the image classification model training method of the present invention has significantly improved the robustness of the image classification model.

[0053] In another embodiment of the present invention, there is provided an image classification method, comprising the following steps:

[0054] Obtain an image to be classified; input the image to be classified into a preset image classification model to obtain a classification result; wherein the image classification model is trained using the above-mentioned image classification model training method.

[0055] Specifically, by using the image classification model trained by the above-mentioned image classification model training method to process the image to be classified, the classification accuracy of the image to be classified can be guaranteed when the data distribution is inconsistent.

[0056] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0057] See also Figure 2 In another embodiment of the present invention, an image classification model training system is provided, which can be used to implement the above-mentioned image classification model training method. Specifically, the image classification model training system includes a data enhancement module and a model parameter optimization module.

[0058] The data enhancement module is used to obtain and input the training image into the image classification model for forward propagation, and obtain the classification result of the training image after the forward propagation is completed; wherein, during the forward propagation process, the enhancement state of the current network layer to be enhanced of the image classification model is determined by the random batch scheduler; when the enhancement state of the current network layer to be enhanced is to be enhanced, the feature data of the training image is enhanced in the current network layer to be enhanced, and the forward propagation continues; when the enhancement state of the current network layer to be enhanced is not to be enhanced, the forward propagation continues; wherein the network layer to be enhanced is the convolution layer and the fully connected layer in the image classification model. The model parameter optimization module is used to perform the parameter optimization step of the image classification model; wherein, when data enhancement is performed, the parameter optimization step includes: adding the self-distillation dynamic soft label regularization term to the loss function of the image classification model to obtain the optimized loss function, and optimizing the model parameters by optimizing the loss function according to the classification results and true labels of the training images; when data enhancement is not performed, the parameter optimization step includes: optimizing the model parameters by the loss function of the image classification model according to the classification results and true labels of the training images.

[0059] All relevant contents of each step involved in the embodiment of the aforementioned image classification model training method can be referred to the functional description of the functional modules corresponding to the image classification model training system in the embodiment of the present invention, and will not be repeated here.

[0060] In yet another embodiment of the present invention, an image classification system is provided, which can be used to implement the above-mentioned image classification method. Specifically, the image classification system includes an image acquisition module and an image classification module.

[0061] Among them, the image acquisition module is used to acquire the image to be classified; the image classification module is used to input the image to be classified into a preset image classification model to obtain a classification result; wherein the image classification model is trained using the above-mentioned image classification model training method.

[0062] All relevant contents of each step involved in the embodiment of the aforementioned image classification method can be referred to the functional description of the functional modules corresponding to the image classification system in the embodiment of the present invention, and will not be repeated here.

[0063] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0064] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the image classification model training method or the image classification method.

[0065] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the image classification model training method or the image classification method in the above embodiment.

[0066] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for training an image classification model, characterized in that: include: The training image is obtained and input into the image classification model for forward propagation; wherein, during the forward propagation process, the enhancement state of the current network layer to be enhanced of the image classification model is determined by a random batch scheduler; when the enhancement state of the current network layer to be enhanced is enhancement, data enhancement is performed on the feature data of the training image in the current network layer to be enhanced and the classification soft label of the training image after data enhancement is obtained, and forward propagation is continued; when the enhancement state of the current network layer to be enhanced is no enhancement, forward propagation is continued; wherein the network layer to be enhanced is the convolutional layer and the fully connected layer in the image classification model; After the forward propagation is completed, the parameter optimization step of the image classification model is performed; wherein, when data enhancement is performed, the parameter optimization step includes: adding the self-distillation dynamic soft label regularization term to the loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameters by optimizing the loss function according to the classification results and true labels of the training images; when data enhancement is not performed, the parameter optimization step includes: optimizing the model parameters by the loss function of the image classification model according to the classification results and true labels of the training images.

2. The image classification model training method according to claim 1, characterized in that: Determining the enhancement state of the current to-be-enhanced network layer of the image classification model by the random Batch scheduler includes: For the current training image, a random vector is generated using the Bernoulli process, and each component in the random vector corresponds to each network layer to be enhanced in the image classification model to be trained; Each component in the random vector is used as the observation value of each matching network layer to be enhanced. When the observation value of the current network layer to be enhanced is 0, the enhancement state of the current network layer to be enhanced is enhancement; when the observation value of the current network layer to be enhanced is 1, the enhancement state of the current network layer to be enhanced is not enhancement.

3. The image classification model training method according to claim 1, characterized in that: The data enhancement of the feature data of the training image at the current network layer to be enhanced includes: When the current network layer to be enhanced is a fully connected layer, the feature data of the training image is sampled in the neighborhood by combining truncated Gaussian noise and Dropout technology.

4. The image classification model training method according to claim 1 or 3, characterized in that: The data enhancement of the feature data of the training image at the current network layer to be enhanced includes: When the current network layer to be enhanced is a convolutional layer, Gaussian noise is added to the feature data of the training image.

5. The image classification model training method according to claim 1, characterized in that: The optimization loss function for: Among them, θ is the model parameter of the image classification model, B is the batch size of the training image, η is the balance parameter, M is the total number of enhancements, and D KL is the KL distance, is the true label of the i-th training image, Y i is the soft label of the i-th training image, Y ij is the classification soft label of the i-th training image after the j-th data augmentation.

6. The image classification model training method according to claim 1, characterized in that: The loss function of the image classification model is a standard classification cross entropy function; When optimizing the model parameters by optimizing the loss function or optimizing the model parameters by the loss function of the image classification model, the loss function value is transferred to each network layer of the image classification model according to the chain rule, and the weight and bias parameters of each network layer are updated by the gradient descent method.

7. An image classification method, characterized in that: include: Get the image to be classified; The image to be classified is input into a preset image classification model to obtain a classification result; wherein the image classification model is trained using the image classification model training method described in any one of claims 1 to 6.

8. An image classification model training system, characterized in that: include: A data enhancement module is used to obtain and input the training image into the image classification model for forward propagation, and obtain the classification result of the training image after the forward propagation is completed; wherein, during the forward propagation process, the enhancement state of the current network layer to be enhanced of the image classification model is determined by a random batch scheduler; when the enhancement state of the current network layer to be enhanced is to perform enhancement, the feature data of the training image is enhanced in the current network layer to be enhanced and the classification soft label of the training image after the data enhancement is obtained, and the forward propagation is continued; when the enhancement state of the current network layer to be enhanced is not to perform enhancement, the forward propagation is continued; wherein the network layer to be enhanced is the convolution layer and the fully connected layer in the image classification model; A model parameter optimization module is used to perform parameter optimization steps of the image classification model after the forward propagation is completed; wherein, when data enhancement has been performed, the parameter optimization step includes: adding the self-distillation dynamic soft label regularization term to the loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameters by optimizing the loss function according to the classification results and true labels of the training images; when data enhancement has not been performed, the parameter optimization step includes: optimizing the model parameters by the loss function of the image classification model according to the classification results and true labels of the training images.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the image classification model training method as described in any one of claims 1 to 6, or implements the steps of the image classification method as described in claim 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the image classification model training method as described in any one of claims 1 to 6 are implemented, or the steps of the image classification method as described in claim 7 are implemented.

Images