Image classification model training method, image processing method, and related apparatuses

Abstract

The application belongs to the field of image processing, and discloses an image classification model training method, an image processing method and related devices, comprising: acquiring and inputting a training image to an image classification model for forward propagation, and determining the enhancement state of the current network layer to be enhanced of the image classification model through a random Batch scheduler; when the enhancement state of the current network layer to be enhanced is enhancement, data enhancement is performed on the feature data of the training image in the current network layer to be enhanced, and forward propagation is continued; when data enhancement is performed, a self-distillation dynamic soft label regularization term is added to the loss function of the image classification model to obtain an optimized loss function, and the model parameters are optimized through the optimized loss function according to the classification result and the real label of the training image. The training efficiency of the image classification model is improved, the resistance of the image classification model to noise is improved, and the stability and reliability of the image classification model in actual application are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and relates to an image classification model training method, an image processing method and related devices. BACKGROUND

[0002] In recent years, deep learning has made significant progress in many application fields, including vision (such as object detection and semantic segmentation), speech (such as sound source localization and tracking and speech recognition), and text (such as entity recognition and automated language translation), and is constantly refreshing the accuracy and other key performance indicators. In machine learning, the optimization objective function is achieved by minimizing the sample average error as the model training reaches convergence or appropriate accuracy, which is widely used in the training of deep image classification models, and this learning rule is also known as empirical risk minimization (ERM). However, when there is serious label noise or even labels are randomly assigned in the classification task, ERM can still make the model have high accuracy on the training data set, and this neural network memory behavior shows that the model has not learned the correct class-invariant features, and there is a serious overfitting phenomenon. Even if the regularization term (such as Lasso regularization) is added to the objective function, it cannot significantly alleviate this network memory behavior. In addition, the image classification model trained using ERM will only perform well when the test data and the training data have very similar distributions. When evaluated with slightly different data distributions, such as adversarial samples, they often give very obvious wrong predictions with very high confidence. Indirectly, this shows that ERM cannot explain or provide the generalization ability when the test distribution is inconsistent with the training data distribution, and it is extremely common to speculate that the data is inconsistent with the training data distribution in the noisy environment of actual deployment. The vulnerability of such models is easy to cause malicious attack behavior, which raises serious concerns about the security of image classification models in many real-world applications, especially in safety-critical scenarios such as autonomous driving and biometric identification.

[0003] The above challenges have led to exploration of the robustness of image classification models from various aspects such as training algorithms, data augmentation, learning mechanisms, and model architecture design in academia and industry, including research and analysis of the causes of the existence of image adversarial samples. Based on the defects of ERM in training image classification models with numerous parameters, the robustness of image classification models is generally improved from the perspective of data augmentation, and the derived augmented image data is used to construct supervised learning signals. First, new training data is synthesized based on the original image data through the VRM principle. Specifically, data augmentation describes the neighborhood training data distribution with the same semantics as the original image sample, which often requires prior knowledge. For example, in image classification, the neighborhood of an image can be defined as the set of its horizontally inverted, rotated, and scaled images. Then, the support of the original training distribution is expanded by extracting additional virtual samples from the neighborhood distribution of the training samples. It is worth noting that recent research shows that image data augmentation can improve the generalization ability and robustness of the model, whether in the original space or in the representation space, and whether a label preservation strategy is used or not.

[0004] However, the existing image data augmentation scheme has two limitations: first, the augmentation process relies on the dataset, so expert knowledge is needed; second, the similarity of synthetic image data and original image data in the neighborhood distribution is largely ignored in the learning process, and the prior distribution information of synthetic image data around the original image data is not considered, so the label consistency of the image classification model is not guaranteed in the training process. Therefore, the subsequent prediction behavior of the image classification model does not explicitly optimize this behavior, but this stable behavior is crucial for the safety of the image classification model in the adversarial environment. SUMMARY

[0005] The purpose of the present application is to overcome the above-mentioned shortcomings of the prior art and provide an image classification model training method, an image processing method and related devices.

[0006] To achieve the above-mentioned purpose, the following technical solutions are adopted:

[0007] In a first aspect, the present application provides a method for training an image classification model, comprising: obtaining and inputting a training image into the image classification model for forward propagation; wherein the enhancement state of a current network layer to be enhanced of the image classification model is determined by a random Batch scheduler during the forward propagation; when the enhancement state of the current network layer to be enhanced is enhancement, the feature data of the training image is data enhanced at the current network layer to be enhanced, and the classification soft label of the training image after data enhancement is obtained, and the forward propagation is continued; when the enhancement state of the current network layer to be enhanced is no enhancement, the forward propagation is continued; wherein the network layer to be enhanced is a convolutional layer and a fully connected layer in the image classification model; after the forward propagation is completed, a parameter optimization step of the image classification model is performed; wherein when data enhancement is performed, the parameter optimization step comprises: adding a self-distillation dynamic soft label regularization term into a loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameters by the optimized loss function according to the classification result and the real label of the training image; when no data enhancement is performed, the parameter optimization step comprises: optimizing the model parameters by the loss function of the image classification model according to the classification result and the real label of the training image.

[0008] Optionally, the determination of the enhancement state of the current network layer to be enhanced of the image classification model by the random Batch scheduler comprises: generating a random vector by a Bernoulli process for the current training image, each component in the random vector one-to-one matches each network layer to be enhanced of the image classification model to be trained; taking each component in the random vector as the observation value of the matched network layer to be enhanced, when the observation value of the current network layer to be enhanced is 0, the enhancement state of the current network layer to be enhanced is enhancement; when the observation value of the current network layer to be enhanced is 1, the enhancement state of the current network layer to be enhanced is no enhancement.

[0009] Optionally, the data enhancement of the feature data of the training image at the current network layer to be enhanced comprises: when the current network layer to be enhanced is a fully connected layer, the feature data of the training image is neighborhood sampling by combining truncated Gaussian noise and Dropout technology.

[0010] Optionally, the data enhancement of the feature data of the training image at the current network layer to be enhanced comprises: when the current network layer to be enhanced is a convolutional layer, Gaussian noise is added to the feature data of the training image.

[0011] Optionally, the optimized loss function is:

[0012]

[0013] wherein, is the model parameter of the image classification model, B is the batch size of the training image, is the balance parameter, M is the total number of enhancement, D KL is the KL distance, is the real label of the i-th training image, Y i is the classification soft label of the i-th training image, Y ij is the classification soft label of the i-th training image after the j-th data enhancement.

[0014] Optionally, the loss function of the image classification model is a standard classification cross-entropy function; when the model parameter is optimized by the loss function or by the loss function of the image classification model, the loss function value is transmitted to each network layer of the image classification model according to the chain rule, and the weight and bias parameters of each network layer are updated by using the gradient descent method.

[0015] In the second aspect of the present application, an image classification method is provided, comprising: obtaining an image to be classified; inputting the image to be classified into a preset image classification model to obtain a classification result; wherein the image classification model is trained by using the image classification model training method described above.

[0016] In the third aspect of the present application, an image classification model training system is provided, comprising: a data enhancement module, configured to obtain and input a training image into an image classification model for forward propagation, and obtain a classification result of the training image after the forward propagation is completed; wherein the enhancement state of a current network layer to be enhanced of the image classification model is determined by a random Batch scheduler during the forward propagation; when the enhancement state of the current network layer to be enhanced is enhancement, the feature data of the training image is enhanced at the current network layer to be enhanced, and the classification soft label of the training image after the data enhancement is obtained, and the forward propagation is continued; when the enhancement state of the current network layer to be enhanced is no enhancement, the forward propagation is continued; wherein the network layer to be enhanced is a convolutional layer and a fully connected layer in the image classification model; a model parameter optimization module, configured to perform a parameter optimization step of the image classification model after the forward propagation is completed; wherein when the data enhancement is performed, the parameter optimization step comprises: adding a self-distillation dynamic soft label regularization term into a loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameter by the optimized loss function according to the classification result and the real label of the training image; when the data enhancement is not performed, the parameter optimization step comprises: optimizing the model parameter by the loss function of the image classification model according to the classification result and the real label of the training image.

[0017] In a fourth aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image classification model training method or the steps of the image classification method when executing the computer program.

[0018] In a fifth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the image classification model training method or the steps of the image classification method.

[0019] Compared with the prior art, the present application has the following beneficial effects:

[0020] The image classification model training method of the present application determines the enhancement state of the current network layer to be enhanced through the random Batch scheduler, realizes random induction of the representation space sample neighborhood, reduces the redundant calculation cost, and improves the training efficiency of the image classification model. The self-distillation dynamic soft label regularization term is added to the loss function of the image classification model to obtain an optimized loss function, and the model parameters are optimized through the optimized loss function. The self-distillation dynamic soft label connects the prediction distribution between the original image sample and the enhanced image sample, guides the image classification model to fit the original image sample neighborhood in multiple semantic levels, and further improves the resistance of the image classification model to noise, so that the final image processing model has stronger resistance to subtle disturbances of images. Especially when the distribution of the prediction image is inconsistent with the training image, such as image distribution drift and image adversarial attack, the image classification model trained by the image classification model training method can still provide more accurate classification results in a high confidence manner. Not only does it enhance the adaptability of the image classification model to unknown or abnormal image data, but also effectively improves the stability and credibility of the image classification model in actual application, thereby laying a solid foundation for more extensive and safer image recognition and application. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The image classification model training method flowchart of the embodiment of the present application.

[0022] Figure 2 The image classification model training system structure block diagram of the embodiment of the present application. DETAILED DESCRIPTION

[0023] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work should belong to the protection scope of the present application.

[0024] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device including a series of steps or units does not necessarily have to be limited to the clearly listed steps or units, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0025] The present application will be described in further detail below with reference to the drawings:

[0026] Referring to Figure 1 In an embodiment of the present application, a method for training an image classification model is provided. Based on data enhancement, an adversarial defense route based on random induction of a neighborhood in a feature space is designed to improve the robustness of the image classification model, thereby ensuring the classification result of the image classification model on actual image data.

[0027] Specifically, the method for training an image classification model comprises the following steps:

[0028] S1: acquiring and inputting a training image into an image classification model for forward propagation, and acquiring a classification result of the training image after the forward propagation is completed.

[0029] In the forward propagation process, the enhancement state of a current network layer to be enhanced of the image classification model is determined by a random Batch scheduler; when the enhancement state of the current network layer to be enhanced is enhancement, the feature data of the training image is enhanced at the current network layer to be enhanced, and the forward propagation is continued; when the enhancement state of the current network layer to be enhanced is no enhancement, the forward propagation is continued; wherein the network layer to be enhanced is a convolutional layer and a fully connected layer in the image classification model.

[0030] S2: performing a parameter optimization step of the image classification model.

[0031] When data augmentation is performed, the parameter optimization step includes: adding a self-distillation dynamic soft label regularization term to a loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameters by the optimized loss function according to the classification result and the real label of the training image; when no data augmentation is performed, the parameter optimization step includes: optimizing the model parameters by the loss function of the image classification model according to the classification result and the real label of the training image.

[0032] The image classification model training method of the application determines the enhancement state of the current network layer to be enhanced through a random Batch scheduler, realizes random induction of a representation space sample neighborhood, reduces redundant calculation cost, and improves the training efficiency of the image classification model. A self-distillation dynamic soft label regularization term is added to the loss function of the image classification model to obtain an optimized loss function, and the model parameters are optimized by the optimized loss function. The self-distillation dynamic soft label guides the image classification model to fit the neighborhood of the original image sample in multiple semantic levels by connecting the prediction distribution between the original image sample and the enhanced image sample, thereby improving the resistance of the image classification model to noise, making the final image processing model have stronger resistance to subtle disturbances of images, and especially when the distribution of the prediction image is inconsistent with the distribution of the training image, such as image distribution drift and image adversarial attack, the image classification model trained by the image classification model training method can still provide more accurate classification results with high confidence. Not only does it enhance the adaptability of the image classification model to unknown or abnormal image data, but it also effectively improves the stability and reliability of the image classification model in practical applications, thereby laying a solid foundation for more extensive and safer image recognition and application.

[0033] In a possible implementation, the determination of the enhancement state of the current network layer to be enhanced by the random Batch scheduler includes: generating a random vector λ by using a Bernoulli process for the current training image, each component in the random vector λ one-to-one matches each network layer to be enhanced of the image classification model to be trained; taking each component in the random vector λ as an observation value of the matched network layer to be enhanced, when the observation value of the network layer to be enhanced is 0, the enhancement state of the network layer to be enhanced is to be enhanced; when the observation value of the network layer to be enhanced is 1, the enhancement state of the network layer to be enhanced is not to be enhanced.

[0034] Specifically, the convolutional layer and the fully connected layer in the image classification model are extracted as a network layer set s to be enhanced, and the random Batch scheduler is a random vector λ generated by using a Bernoulli process, wherein each component is independent, that is, a Layer-Wise manner can be used to realize fine adjustment, which can be represented as:

[0035]

[0036] where λ(t) represents the specific state of the random Batch scheduler at time t, and i is the index of the i-th training image j where i represents the i-th training image, and j ∈ [s] is the index of the selected network layer s to be enhanced according to its order in the set of network layers s to be enhanced. If i.e., if the observation value of the j-th layer is 1, the layer is enhanced; if i.e., if the observation value of the j-th layer is 0, the layer is not enhanced. Each term on the right side is a random batch scheduling strategy for each layer during the training of the image classification model, which is determined by the corresponding observation value on the left side.

[0037] In a possible implementation, the data augmentation of the feature data of the training image by the current network layer to be enhanced includes: when the current network layer to be enhanced is a fully connected layer, performing neighborhood sampling on the feature data of the training image by combining truncated Gaussian noise and Dropout technology.

[0038] Specifically, the feature data of the training image is sampled by combining truncated Gaussian noise and Dropout technology to obtain enhanced image data.

[0039] Explanatorily, truncated Gaussian noise is a specific random noise that conforms to Gaussian distribution (also known as normal distribution), but its value range is limited. In machine learning, Gaussian noise often appears in various data processing and model training stages. The methods for processing Gaussian noise can be divided into two categories: noise reduction and robust learning. The goal of noise reduction is to extract clean signals from noisy data, while the goal of robust learning is to make the model have certain robustness to noise, i.e., to obtain good performance in the presence of noise. Dropout technology is a commonly used regularization technique, which aims to prevent overfitting of neural networks. The basic idea is to randomly discard some neurons during the training process of the neural network with a certain probability. Specifically, Dropout technology will randomly select some neurons in each training batch and set their outputs to 0. In this way, the model becomes more robust and reduces the dependence on specific neurons, thereby improving the generalization ability of the model. During testing, all neurons will participate in the calculation, but the output of each neuron will be multiplied by a scaling factor to ensure that the output of the model during testing is the same as during training.

[0040] In a possible implementation, the data augmentation of the feature data of the training image by the current network layer to be enhanced includes: when the current network layer to be enhanced is a convolutional layer, adding Gaussian noise to the feature data of the training image.

[0041] Specifically, only the feature data of the training image is added with Gaussian noise at the convolutional layer, that is, noise obeying normal distribution is added in the feature data of the training image to obtain the enhanced image data.

[0042] In a possible implementation, the loss function of the image classification model is a standard classification cross-entropy function. Specifically, the standard classification cross-entropy function is used as an optimization objective, so as to ensure that the image classification model learns on the original training image and quickly reaches a certain recognition accuracy.

[0043] In a possible implementation, the optimization loss function is as follows:

[0044]

[0045] wherein, θ is a model parameter of the image classification model, B is a batch size of the training image, η is a balance parameter, M is a total number of times of enhancement, D KL is a KL distance, Y is a true label of the i th training image, Y i is a classification soft label of the i th training image, Y ij is a classification soft label of the i th training image after the j th time of data enhancement.

[0046] The optimization loss function is as follows: wherein, H is a standard classification cross-entropy function, and H is a self-distillation dynamic soft label regularization term.

[0047] In a possible implementation, the loss function is as follows:

[0048]

[0049] Specifically, the training image is input into the image classification model to obtain a SoftMax normalized probability classification result, that is, a classification soft label of the training image, and then the classification result is used as a soft label of the training image after data enhancement, and the enhanced image data is input into the image classification model to obtain a classification soft label of the training image after data enhancement, and the KL distance between the training image and the classification soft label of the training image after data enhancement is used as an additional regularization term, that is, a self-distillation dynamic soft label regularization term, which is added to the loss function of the image classification model, so as to provide a supervised learning signal in the loss function of the image classification model through the regularization term.

[0050] Explanatory, KL distance (Kullback-Leibler Divergence), also known as relative entropy, is a non-symmetrical index to measure the difference between two probability distributions P and Q. It reflects the additional information loss caused by using distribution Q for encoding under the condition of knowing distribution P. KL distance has wide applications in machine learning, information theory and statistics, such as evaluating the difference between model prediction distribution and real distribution, optimizing model parameters, etc. By minimizing KL distance, the distribution learned by the model can be closer to the distribution of real data, thereby improving the accuracy and generalization ability of the model.

[0051] In summary, the image classification model training method of the present application, the representation space sample neighborhood is randomly induced to be modeled as a Bernoulli process, and the observation value of the tth iteration determines whether to perform data enhancement on a certain network layer to be enhanced at the tth iteration. The self-distillation dynamic soft label regularization term guides the image classification model to fit the original sample neighborhood in multiple semantic levels by connecting the prediction distribution between the original training image and the enhanced image data, thereby improving the noise resistance of the image classification model. Using the standard classification cross-entropy function as the main optimization target ensures the learning of the image classification model on the original sample, and quickly reaches a certain recognition accuracy. At the same time, the intermediate output distribution of the original training image is used to construct a self-supervised signal for model parameter learning, i.e. the KL divergence between the prediction distribution of the enhanced image data and the prediction distribution of the original training image, which considers the complementary information of the prediction. The forced matching of the two inference distributions makes the prediction behavior of the image classification model more consistent within the local neighborhood, and makes the final image classification model have stronger resistance to subtle disturbances. This method can also reduce the redundant calculation cost and improve the training efficiency of the image classification model.

[0052] In one possible implementation, through experimental demonstration, the image classification model training method of the present application can significantly reduce the error rate on CIFAR-10, CIFAR-100 and ImageNet data sets even if different image classification models such as VGG16 or ResNet44 are used, indicating that the robustness of the image classification model is significantly improved.

[0053] In another embodiment of the present application, an image classification method is provided, comprising the following steps:

[0054] An image to be classified is obtained, and the image to be classified is input into a preset image classification model to obtain a classification result. The image classification model is trained by the image classification model training method described above.

[0055] Specifically, the image classification model trained by the image classification model training method can ensure the classification accuracy of the to-be-classified image in the case of inconsistent data distribution.

[0056] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the apparatus embodiment, please refer to the method embodiment of the present application.

[0057] Referring to Figure 2 In another embodiment of the present application, an image classification model training system is provided, which can be used to implement the image classification model training method described above. Specifically, the image classification model training system comprises a data enhancement module and a model parameter optimization module.

[0058] The data enhancement module is configured to obtain and input the training image into the image classification model for forward propagation, and obtain the classification result of the training image after the forward propagation is completed. In the forward propagation process, the enhancement state of the current to-be-enhanced network layer of the image classification model is determined by a random Batch scheduler. When the enhancement state of the current to-be-enhanced network layer is enhancement, the feature data of the training image is enhanced at the current to-be-enhanced network layer, and the forward propagation is continued. When the enhancement state of the current to-be-enhanced network layer is no enhancement, the forward propagation is continued. The to-be-enhanced network layer is a convolutional layer and a fully connected layer in the image classification model. The model parameter optimization module is configured to perform the parameter optimization step of the image classification model. When the data enhancement is performed, the parameter optimization step comprises: adding a self-distillation dynamic soft label regularization term to the loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameters by the optimized loss function according to the classification result of the training image and the real label. When the data enhancement is not performed, the parameter optimization step comprises: optimizing the model parameters by the loss function of the image classification model according to the classification result of the training image and the real label.

[0059] The foregoing embodiments of the image classification model training method involve all related contents of each step, which can be referred to the function description of the corresponding function module of the image classification model training system in the embodiments of the present application, and will not be repeated here.

[0060] In another embodiment of the present application, an image classification system is provided, which can be used to implement the image classification method described above. Specifically, the image classification system comprises an image acquisition module and an image classification module.

[0061] The image acquisition module is configured to obtain the to-be-classified image. The image classification module is configured to input the to-be-classified image into a preset image classification model to obtain a classification result. The image classification model is trained by the image classification model training method described above.

[0062] All the related contents of the steps involved in the foregoing embodiment of the image classification method can be cited to the function description of the function modules corresponding to the image classification system in the embodiments of the present application, and will not be repeated here.

[0063] The division of the modules in the embodiments of the present application is illustrative, and is only a logical function division. In actual implementation, there can be another division manner. In addition, each function module in each embodiment of the present application can be integrated in one processor, or can be a separate physical existence, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module.

[0064] In another embodiment of the present application, a computer device is provided, which includes a processor and a memory. The memory is configured to store a computer program, and the computer program includes program instructions. The processor is configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor is the computing core and control core of the terminal, and is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in the computer storage medium to implement a corresponding method flow or a corresponding function. The processor in the embodiments of the present application can be used for the operation of the image classification model training method or the image classification method.

[0065] In another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in a computer device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include a built-in storage medium in the computer device, and of course can also include an extended storage medium supported by the computer device. The computer readable storage medium provides a storage space, which stores an operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. One or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the image classification model training method or the image classification method in the above embodiments.

[0066] Those skilled in the art should understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0067] The present application is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified in the flow or flows and / or block or blocks.

[0068] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocksFigure 1 the function specified in the one or more blocks.

[0069] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processing, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flow Figure 1 the flow or flows and / or blocks Figure 1 the steps of the function specified in the one or more blocks.

[0070] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit it, although the above embodiments of the present application have been described in detail, those skilled in the art should understand: the specific embodiments of the present application can be modified or replaced by the same, without departing from the spirit and scope of the present application, any modification or equivalent replacement, which should be covered within the scope of protection of the claims of the present application.

Claims

1. An image classification model training method, characterized in that, The method comprises: obtaining and inputting a training image into an image classification model for forward propagation; wherein the enhancement state of a current network layer to be enhanced of the image classification model is determined by a random Batch scheduler during the forward propagation; when the enhancement state of the current network layer to be enhanced is enhancement, the feature data of the training image is data enhanced at the current network layer to be enhanced, and the classification soft label of the training image after data enhancement is obtained, and the forward propagation is continued; when the enhancement state of the current network layer to be enhanced is no enhancement, the forward propagation is continued; wherein the network layer to be enhanced is a convolutional layer and a fully connected layer in the image classification model; after the forward propagation is completed, a parameter optimization step of the image classification model is performed; wherein when data enhancement is performed, the parameter optimization step comprises: adding a self-distillation dynamic soft label regularization term into a loss function of the image classification model to obtain an optimized loss function, and optimizing the model parameters by the optimized loss function according to the classification result of the training image and the real label; when no data enhancement is performed, the parameter optimization step comprises: optimizing the model parameters by the loss function of the image classification model according to the classification result of the training image and the real label.

2. The image classification model training method of claim 1, wherein, The determination of the enhancement state of the current network layer to be enhanced of the image classification model by the random Batch scheduler comprises: for the current training image, a random vector is generated by using a Bernoulli process, and each component in the random vector one-to-one matches each network layer to be enhanced of the training image classification model; each component in the random vector is taken as an observation value of the matched network layer to be enhanced, when the observation value of the current network layer to be enhanced is 0, the enhancement state of the current network layer to be enhanced is enhancement; when the observation value of the current network layer to be enhanced is 1, the enhancement state of the current network layer to be enhanced is no enhancement.

3. The image classification model training method of claim 1, wherein, The data enhancement of the feature data of the training image at the current network layer to be enhanced comprises: when the current network layer to be enhanced is a fully connected layer, neighborhood sampling is performed on the feature data of the training image by combining truncated Gaussian noise and Dropout technology.

4. The image classification model training method of claim 1 or 3, characterized in that, The data enhancement of the feature data of the training image at the current network layer to be enhanced comprises: when the current network layer to be enhanced is a convolutional layer, Gaussian noise is added in the feature data of the training image.

5. The image classification model training method of claim 1, wherein, The optimization loss function is: where θ is the model parameter of the image classification model, B is the training image batch size, η is the balance parameter, M is the total number of times of enhancement, D KL is the KL distance, is the real label of the i-th training image, Y i is the classification soft label of the i-th training image, Y ij is the classification soft label of the i-th training image after the j-th time of data enhancement.

6. The image classification model training method of claim 1, wherein, The loss function of the image classification model is a standard classification cross-entropy function; when the model parameters are optimized by the optimized loss function or the model parameters are optimized by the loss function of the image classification model, the loss function value is transmitted to each network layer of the image classification model according to the chain rule, and the weight and bias parameters of each network layer are updated by using the gradient descent method.

7. An image classification method characterized by, The method comprises: obtaining an image to be classified; inputting the image to be classified into a preset image classification model to obtain a classification result; wherein the image classification model is trained by using the image classification model training method in any one of claims 1 to 6.

8. An image classification model training system, comprising: The method comprises: The data enhancement module is configured to input the training image into the image classification model for forward propagation, and obtain a classification result of the training image after the forward propagation is completed; wherein, an enhancement state of a current network layer to be enhanced of the image classification model is determined by a random Batch scheduler during the forward propagation; when the enhancement state of the current network layer to be enhanced is enhancement, the feature data of the training image is enhanced by the current network layer to be enhanced, and a classification soft label of the training image after the data enhancement is obtained, and the forward propagation is continued; when the enhancement state of the current network layer to be enhanced is no enhancement, the forward propagation is continued; wherein, the network layer to be enhanced is a convolution layer and a full connection layer in the image classification model; The model parameter optimization module is configured to perform a parameter optimization step of the image classification model after the forward propagation is completed; wherein, when the data enhancement is performed, the parameter optimization step includes: adding a self-distillation dynamic soft label regularization term into a loss function of the image classification model to obtain an optimization loss function, and optimizing the model parameters by the optimization loss function according to the classification result of the training image and a real label; when the data enhancement is not performed, the parameter optimization step includes: optimizing the model parameters by a loss function of the image classification model according to the classification result of the training image and the real label.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the image classification model training method according to any one of claims 1 to 6, or implement the steps of the image classification method according to claim 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the image classification model training method according to any one of claims 1 to 6, or implement the steps of the image classification method according to claim 7.

Images