Image Classification Method, System and Device Based on Dynamic Semi-Supervised Deep Learning

Abstract

The present invention provides an image classification method, system and device based on dynamic semi-supervised deep learning. The method includes: constructing an image classification model, obtaining an image data set, and based on the image data set, training and optimizing the image classification model by using a dynamic semi-supervised deep learning method based on pseudo-labels and consistency regularization to obtain a trained image classification model; an image classification module, obtaining an image to be classified, and classifying the image to be classified by using the trained image classification model to obtain the label of the image to be classified. The present invention adopts an image classification method based on dynamic semi-supervised deep learning. Based on pseudo-labels and consistency regularization, through the dynamic adjustment of the confidence threshold, the utilization rate of pseudo-label data is improved, thereby fundamentally solving the problems of the existing situation of fixed confidence threshold and insufficient use of image samples, resulting in a decrease in the accuracy of the image classification model and low image classification accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of image classification, and particularly relates to an image classification method, system and device based on dynamic semi-supervised deep learning. Background Art

[0002] In recent years, neural networks have become important processing and analysis tools in the field of image classification and have been widely used in image classification technology. The image classification accuracy of neural networks is closely related to the number of labeled image samples and the annotation accuracy. If there is a sufficient number of accurately labeled image datasets, an image classification model with better classification effect can be trained. However, the labor cost of labeling a large number of image datasets is very high.

[0003] Therefore, due to the limited number of labeled images, semi-supervised learning methods have been more and more widely used. Semi-supervised learning enables only a small number of labeled images and a certain number of unlabeled images to be required in the training process of the image classification model, greatly reducing the demand for labeled images. In the prior art, under the premise of limited labeled data, pseudo-labels are generated from unlabeled data, such as MixMatch, FixMatch, etc. However, in the existing methods, only a fixed number of unlabeled images with high confidence are used for training, so that many image samples are not used, and the scale of the training set is reduced, greatly affecting the classification accuracy of the image classification model.

[0004] Therefore, how to improve the precision of the image classification model and the accuracy of image classification is an urgent problem to be solved in this field. Summary of the Invention

[0005] The purpose of the present invention is to provide an image classification method, system and device based on dynamic semi-supervised deep learning for the defects of the prior art. The image classification method based on dynamic semi-supervised deep learning of the present invention, based on pseudo-labels and consistency regularization, improves the utilization rate of pseudo-label data through dynamic adjustment of the confidence threshold, thus fundamentally solving the problem that the accuracy of the image classification model decreases and the image classification accuracy is low due to the existing situation of fixed confidence threshold and insufficient use of image samples.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions:

[0007] The present invention provides an image classification method based on dynamic semi-supervised deep learning, including the steps of:

[0008] S1. Construct an image classification model, and the image classification model adopts a WRN model;

[0009] S2. Obtain an image dataset, and use the image dataset to train and optimize the image classification model by adopting a dynamic semi-supervised deep learning method based on pseudo-labels and consistency regularization to obtain a trained image classification model. Specifically, S2 includes the following steps:

[0010] S21. Obtain an image dataset and perform hyperparameter settings. Among them, the image dataset includes a labeled image set X = (x b , p b ): b ∈ (1, …, B) and an unlabeled image set U = u b : b ∈ (1, …, μB). The hyperparameters at least include a confidence threshold τ, an augmentation amount aug times , the proportion μ of unlabeled images, and the weight λ of the loss corresponding to unlabeled images u . Among them, x b represents the b-th image, p b represents the label corresponding to the b-th image, B represents the number of labeled images, and μ b represents the b-th unlabeled image;

[0011] S22. Input the labeled image set X into the image classification model for supervised training. During the training process, the loss function of the labeled images adopted is:

[0012]

[0013] Among them, H() represents cross-entropy, p m () represents the probability distribution predicted by the model, y is the probability, and α(x b ) represents weak augmentation of the image x b ;

[0014] S23. After weakly augmenting the unlabeled image set, input it into the image classification model for prediction, and label the unlabeled images based on the prediction values to form pseudo-labeled images. The pseudo-label is:

[0015] q b = p model (y ∣ α(u b ); θ)

[0016] Among them, q b represents the pseudo-label, p model () represents the probability distribution predicted by the model, y is the probability, α(μ b ) is the weak augmentation of the unlabeled image μ b , and θ is the hyperparameter of the model;

[0017] S24. After strongly augmenting the unlabeled image set, input it into the image classification model for prediction;

[0018] S25. Input the pseudo-labeled images with confidence higher than the threshold τ into the image classification model for training, calculate the cross-entropy loss between the prediction values corresponding to strong augmentation and the pseudo-labels corresponding to weak augmentation. The loss function for unlabeled images is as follows:

[0019]

[0020] where max(q b ) is the maximum value in the pseudo-label probability distribution, argmax(q b ) also represents the maximum value in the pseudo-label probability distribution, p model (y∣∣A(u b )) represents the probability distribution of the pseudo-label, and A(u b ) represents the strong augmentation of the unlabeled image μ b ;

[0021] S26. Dynamically adjust the confidence threshold τ and the augmentation amount aug times ; The augmentation amount aug times is used to control the number of image augmentation times;

[0022] S27. Repeat steps S23 - S26 until the minimization of the loss function minimizer(l s +λ u l u ) is satisfied; Among them, the specific update process of the confidence threshold τ is as follows:

[0023]

[0024] τ now =τ start +(τ end -τ start )×factor

[0025] where factor is the weight factor for adjusting the threshold, step now represents the current training step, step cycle represents the number of training steps required for the threshold to be updated to τ end , τ start is the starting threshold for training, and τ end is the threshold to be finally updated to;

[0026] S3. Obtain the image to be classified, and use the trained image classification model to classify the image to be classified to obtain the label of the image to be classified.

[0027] Furthermore, the image classification model is a WRN-28-2 model.

[0028] The present invention provides an image classification system based on dynamic semi-supervised deep learning. The image classification system based on dynamic semi-supervised deep learning executes the image classification method based on dynamic semi-supervised deep learning, and includes: an image classification model construction module, an image classification model training module, and an image classification module;

[0029] The image classification model construction module constructs an image classification model, and the image classification model is a WRN model;

[0030] The image classification model training module obtains an image data set, and uses the image data set to train and optimize the image classification model by using a dynamic semi-supervised deep learning method based on pseudo-labels and consistency regularization to obtain a trained image classification model;

[0031] The image classification module obtains an image to be classified, and uses the trained image classification model to classify the image to be classified to obtain a label of the image to be classified.

[0032] The present invention also provides a computer device, which includes a memory and a processor. A computer program is stored on the memory, and when the processor executes the computer program, the above method is implemented.

[0033] An image classification method, system and device based on dynamic semi-supervised deep learning disclosed by the present invention. The image classification method based on dynamic semi-supervised deep learning of the present invention is based on pseudo-labels and consistency regularization, and through dynamic adjustment of the confidence threshold, the utilization rate of pseudo-label data is improved, thereby fundamentally solving the problem that the existing fixed confidence threshold and insufficient use of image samples lead to a decrease in the accuracy of the image classification model and low image classification accuracy.

[0034] Compared with the prior art, it has the following beneficial effects:

[0035] 1. The image classification method based on dynamic semi-supervised deep learning of the present invention is based on pseudo-labels and consistency regularization. Through dynamic adjustment of the confidence threshold, the utilization rate of pseudo-label data is improved, the accuracy of the image classification model is improved, and thus the image classification accuracy is improved.

[0036] 2. The image classification method based on dynamic semi-supervised deep learning of the present invention trains the image classification model progressively through dynamic adjustment of parameters, gradually improves the pseudo-label marking ability of the image classification model, and also reduces the error rate of each round of pseudo-label marking, improves the accuracy of the image classification model, and thus improves the image classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0038] Figure 1 It is a schematic diagram of an image classification method based on dynamic semi-supervised deep learning provided by an embodiment of the present invention.

[0039] Figure 2 It is the basic structure diagram of the WRN model provided by an embodiment of the present invention.

[0040] Figure 3 It is a schematic diagram of an image classification system based on dynamic semi-supervised deep learning provided by an embodiment of the present invention. Detailed implementation manners

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0042] It should be understood that when used in this specification and the appended claims, the terms "comprises" and "comprising" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0043] The following further illustrates the present invention with reference to the drawings and specific embodiments, but does not limit the present invention.

[0044] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in this specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0045] It should be further understood that the term " / and / " used in this specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0046] An image classification method based on dynamic semi-supervised deep learning according to the present invention. As Figure 1As shown in the figure, the image classification method based on dynamic semi-supervised deep learning includes the following steps S1 to S3.

[0047] S1. Construct an image classification model, and the image classification model adopts the WRN model.

[0048] In a specific embodiment, the image classification model adopts the WRN-28-2 model. The basic structure BasicBlock of the WRN model is as Figure 2 shown. While inheriting the traditional residual structure, this structure introduces convolutional dropout regularization in the backbone structure. The principle of this design is that WRN believes that during model training, since the gradient can directly pass through the skip connection during backpropagation without being forced to pass through the residual block, this may result in only a very limited number of residual network layers learning useful knowledge, while more residual network layers make only a small contribution to the final result. Therefore, WRN chooses to add dropout regularization between two convolutional layers.

[0049] S2. Obtain an image dataset, and use the image dataset to train and optimize the image classification model by using the dynamic semi-supervised deep learning method based on pseudo-labels and consistency regularization to obtain a trained image classification model; specifically, S2 includes the following steps:

[0050] S21. Obtain an image dataset and perform hyperparameter setting; among them, the image dataset includes a labeled image set X = (x b , p b ): b ∈ (1,..., B) and an unlabeled image set U = u b : b ∈ (1,..., μB), and the hyperparameters at least include a confidence threshold τ, an augmentation amount aug times , the proportion μ of unlabeled images, and the weight λ of the loss corresponding to unlabeled images u ; among them, x b represents the b-th image, p b represents the label corresponding to the b-th image, B represents the number of labeled images, and μ b represents the b-th unlabeled image.

[0051] In a specific embodiment, p b represents the label corresponding to the b-th image and uses one-hot encoding.

[0052] In a specific embodiment, μ b refers to the unlabeled images of each category. Because cifar10 is used, it is artificially transformed into a semi-supervised dataset for use.

[0053] S22. Input the labeled image set X into the image classification model for supervised training. During the training process, the loss function of the labeled images used is:

[0054]

[0055] where H() represents the cross-entropy, p m () represents the probability distribution predicted by the model, y is the probability, and α(x b ) represents weak augmentation of the image x b .

[0056] In a specific embodiment, p m () represents the probability distribution predicted by the model, which is a conditional probability. α(x b ) performs weak augmentation, blurring, etc. on the image x b .

[0057] S23. After weakly augmenting the unlabeled image set, input it into the image classification model for prediction. Based on the prediction value, label the unlabeled images to form pseudo-labeled images. The pseudo-label is:

[0058] q b = p model (y | α(u b )); θ

[0059] where q b represents the pseudo-label, p model () represents the probability distribution predicted by the model, y is the probability, α(μ b ) is the weak augmentation of the unlabeled image μ b , and θ is the hyperparameter of the model.

[0060] S24. After strongly augmenting the unlabeled image set, input it into the image classification model for prediction.

[0061] S25. Input the pseudo-labeled images with a confidence higher than the threshold τ into the image classification model for training. Calculate the cross-entropy loss between the prediction value corresponding to the strong augmentation and the pseudo-label corresponding to the weak augmentation. The loss function of the unlabeled images is:

[0062]

[0063] where max(q b ) is the maximum value in the pseudo-label probability distribution, argmax(q b ) also represents the maximum value in the pseudo-label probability distribution, p model (y || A(u b )) represents the probability distribution of the pseudo-label, A(ub ) represents the unlabeled image μ b with strong augmentation.

[0064] Specifically, for the set of unlabeled images, a pseudo-label is calculated for each image and then used in the loss. To obtain a pseudo-label, we first generate a weakly augmented version α(μ b ) of the given unlabeled image μ b and calculate the predicted class distribution q b = p m (y | α(μ b )) of the model for this version. Then, we use q b = argmax(q b ) as the pseudo-label for this sample and generate strongly augmented data for μ b as the training sample corresponding to the pseudo-label.

[0065] S26. Dynamically adjust the confidence threshold τ and the augmentation amount aug times ; the augmentation amount aug times is used to control the number of image augmentations;

[0066] Among them, the augmentation amount aug times is used to control the number of image augmentations and is a discrete variable that is incremented by one when the training progress epoch of the model reaches a certain progress (the specific progress is adjusted manually). After the update, it will affect the next image augmentation method. For example, if only blurring was used before, multiple methods such as grayscale will be used later.

[0067] Specifically, by adjusting the threshold τ of the pseudo-label, the scale of the image set that can be used is dynamically adjusted. That is, first, the pseudo-label images with higher confidence are used, and then a new round of pseudo-labeling process is carried out using the updated neural network. Then, the required confidence threshold is lowered, and more image sets are added, thereby dynamically adjusting the scale of the image data set used and improving the accuracy of the neural network and the image classification accuracy.

[0068] S27. Repeat steps S23 - S26 until the minimum loss function minimizer(l s + λ u l u ) is satisfied; among them, the specific update process of the confidence threshold τ is as follows:

[0069]

[0070] τ now = τ start + (τ end - τ start ) × factor

[0071] Among them, factor is the weight factor for adjusting the threshold, and step now represents the number of steps of the current training, step cycle represents that the threshold is updated to τ end the number of training steps required to reach τ start is the starting threshold of the training, τ end is the threshold to be finally updated to.

[0072] By repeating steps S23 - S26, the trained image classification model is used to label. This process is progressive, and the threshold will also be adjusted. Not all the data with pseudo - labels will be used, and this threshold will control the number of pseudo - label images used.

[0073] S3. Obtain the image to be classified, and use the trained image classification model to classify the image to be classified, so as to obtain the label of the image to be classified.

[0074] Figure 3 is an image classification system based on dynamic semi - supervised deep learning provided by an embodiment of the present invention. As Figure 3 shown, the image classification system based on dynamic semi - supervised deep learning includes: an image classification model construction module, an image classification model training module, and an image classification module;

[0075] The image classification model construction module constructs an image classification model, and the image classification model is a WRN model;

[0076] The image classification model training module obtains an image data set, and uses the image data set to train and optimize the image classification model by using a dynamic semi - supervised deep learning method based on pseudo - labels and consistency regularization, so as to obtain a trained image classification model;

[0077] The image classification module obtains the image to be classified, and uses the trained image classification model to classify the image to be classified, so as to obtain the label of the image to be classified.

[0078] The above - mentioned image classification system based on dynamic semi - supervised deep learning can be implemented in the form of a computer program, and the computer program can run on a computer device.

[0079] The computer device can be a server. Among them, the server can be an independent server or a server cluster composed of multiple servers.

[0080] The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the memory can include a non - volatile storage medium and an internal memory.

[0081] The non-volatile storage medium can store an operating system and computer programs. The computer programs include program instructions which, when executed, can cause a processor to execute an image classification method based on dynamic semi-supervised deep learning.

[0082] The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

[0083] The internal memory provides an environment for the operation of the computer programs in the non-volatile storage medium. When the computer programs are executed by the processor, the processor can be caused to execute an image classification method based on dynamic semi-supervised deep learning.

[0084] The network interface is used for network communication with other devices. Those skilled in the art can understand that the above computer device structure is only a part of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0085] Among them, the processor is used to run the computer program stored in the memory, and this program implements the image classification method based on dynamic semi-supervised deep learning described in the first embodiment.

[0086] It should be understood that in the embodiments of this application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0087] Those of ordinary skill in the art can understand that all or part of the processes in the methods of implementing the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, and this storage medium is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above method.

[0088] The present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the image classification method based on dynamic semi-supervised deep learning described in Embodiment 1.

[0089] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, an optical disc, or other various computer-readable storage media that can store program codes.

[0090] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of the examples have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional person can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0091] In several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0092] The steps in the method embodiments of the present invention can be adjusted, combined, and deleted according to actual needs. The units in the apparatus embodiments of the present invention can be combined, divided, and deleted according to actual needs. In addition, the functional units in each embodiment of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0094] Note that the above is only the preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An image classification method based on dynamic semi-supervised deep learning, characterized in that, Including the steps: S1. Construct an image classification model, and the image classification model adopts the WRN model; S2. Obtain an image dataset, and use the image dataset to train and optimize the image classification model by using a dynamic semi-supervised deep learning method based on pseudo-labels and consistency regularization to obtain a trained image classification model. Specifically, S2 includes the following steps: S21. Obtain an image dataset and perform hyperparameter settings; wherein, the image dataset includes a labeled image set X = (x b , p b ): b ∈ (1,..., B) and an unlabeled image set u = u b : b ∈ (1,..., μB), and the hyperparameters at least include a confidence threshold τ, an augmentation amount aug times , the proportion μ of unlabeled images, and the weight λ u of the loss corresponding to the unlabeled images; wherein, x b represents the b-th image, p b represents the label corresponding to the b-th image, B represents the number of labeled images, and μ b represents the b-th unlabeled image; S22. Input the labeled image set X into the image classification model for supervised training. During the training process, the loss function of the labeled images used is: Among them, H() represents the cross-entropy, P m () represents the probability distribution predicted by the model, y is the probability, and α(x b ) represents weak augmentation of the image x b ; S23. Weakly augment the unlabeled image set and then input it into the image classification model for prediction. Based on the prediction values, label the unlabeled images to form pseudo-labeled images. The pseudo-label is: q b = p model (y|α(u b )); θ) Among them, q b represents the pseudo-label, p model () represents the probability distribution predicted by the model, y is the probability, and α(μ b ) is the weak augmentation of the unlabeled image μ b , and θ is the hyperparameter of the model; S24. Strongly augment the unlabeled image set and then input it into the image classification model for prediction; S25. Input the pseudo-labeled images with confidence higher than the threshold τ into the image classification model for training, calculate the cross-entropy loss between the prediction values corresponding to the strong augmentation and the pseudo-labels corresponding to the weak augmentation. The loss function of the unlabeled images is: where, max(q b ) is the maximum value in the pseudo-label probability distribution, argmax(q b ) also represents the maximum value in the pseudo-label probability distribution, P model (y|A(u b )) represents the probability distribution of the pseudo-label, and A(u b ) represents the strong augmentation of the unlabeled image μ b ; S26. Dynamically adjust the confidence threshold τ and the enhancement amount aug times ; The enhancement amount aug times is used to control the number of image enhancement times; S27. Repeat steps S23 - S26 until the loss function minimizer( l s + λ u l u ) is satisfied; wherein, the specific update process of the confidence threshold τ is as follows: τ now = τ start + (τ end - τ start ) × factor Among them, factor is the weight factor for adjusting the threshold, step now represents the number of steps of the current training, step cycle indicates that the threshold is updated to τ end the number of training steps required, τ start is the starting threshold of the training, τ end is the threshold to be finally updated to; S3. Obtain the image to be classified, and use the trained image classification model to classify the image to be classified to obtain the label of the image to be classified.

2. The method according to claim 1, wherein The image classification model is a WRN-28-2 model.

3. An image classification system based on dynamic semi-supervised deep learning, characterized in that, The image classification system based on dynamic semi-supervised deep learning executes the image classification method based on dynamic semi-supervised deep learning as described in claim 1, including: an image classification model construction module, an image classification model training module, and an image classification module; The image classification model construction module constructs an image classification model, and the image classification model is a WRN model; The image classification model training module obtains an image dataset, and uses the image dataset to train and optimize the image classification model by using a dynamic semi-supervised deep learning method based on pseudo-labels and consistency regularization to obtain a trained image classification model; The image classification module obtains the image to be classified, and uses the trained image classification model to classify the image to be classified to obtain the label of the image to be classified.

4. A computer device, characterized in that, The device includes a memory and a processor. A computer program is stored on the memory. When the processor executes the computer program, the method described in any one of claims 1 to 2 is implemented.

Images