An image classification method, computer device and storage medium

Abstract

The application provides an image classification method, computer equipment and a storage medium, and the classification method comprises the following steps: a plurality of convolution layers and a plurality of pooling layers are respectively constructed, each convolution layer is used for convolution of initial image data, and each pooling layer is used for pooling of the initial image data after convolution; a full connection layer is constructed, which is used for classifying the initial image data after multiple times of convolution and pooling, and outputting target classification result data; wherein the full connection layer is composed of a plurality of pulse neurons. The problems that a convolutional neural network needs huge computing resources in a training process and is prone to insufficient memory in use in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an image classification method, a computer device and a storage medium. BACKGROUND

[0002] Artificial neural network is an important algorithm of machine learning, and is also an important part of artificial intelligence technology. The inspiration of constructing artificial neural network comes from human brain. Human brain is composed of a large number of neurons connected to each other to form a complex network. People imitate this working mechanism to construct artificial neural network. In artificial neural network, neurons are divided into different layers, each layer is composed of multiple neurons, each layer is connected to the previous layer, and after performing corresponding calculation, the result of the layer is output to the next layer. With the progress of computer science and data science, more and more computing resources and data can be used, and artificial neural network can therefore stack more layers to obtain more effects. Such neural network can also be called deep neural network.

[0003] Convolutional neural network is a kind of artificial neural network. In the task of image classification, convolutional neural network is a commonly used deep neural network algorithm. Its special convolutional layer can be used to extract local features of images, and good results can be obtained in the field of image classification. However, convolutional neural network requires huge computing resources in the training process, and it is easy to run out of memory in use. SUMMARY

[0004] In view of the problems in the prior art, the present application provides an image classification method, a computer medium and a storage device, which solves the problem that the convolutional neural network in the prior art requires huge computing resources in the training process and is easy to run out of memory in use.

[0005] In a first aspect, the present application provides an image classification method, which comprises: constructing a plurality of convolutional layers and a plurality of pooling layers respectively, each of the convolutional layers being used for convolution of initial image data, and each of the pooling layers being used for pooling of the initial image data after convolution; constructing a fully connected layer, which is used for classifying the initial image data after multiple convolution and pooling, and outputting target classification result data; wherein the fully connected layer is composed of a plurality of spiking neurons.

[0006] Optionally, the spiking neuron is a LIF neuron, and the working mode of the LIF neuron is:

[0007]

[0008]

[0009] s(t) = 0, if u(t) < uth

[0010] wherein, τ represents a constant, u represents a membrane potential, t represents a time step, represents a resting potential, ω j represents a weight of the jth input synapse, K(·) represents a kernel function of a delay effect, and s represents an output spike, represents a reset potential, u th represents a threshold value for determining whether to ignite, represents that when the kth pulse of the jth input synapse is at T ω represents the moment of activation within the integration time, T ω represents an integration time window.

[0011] Optionally, the plurality of convolutional layers and the plurality of pooling layers are respectively constructed, each of the convolutional layers is used for convolution on the initial image data, and each of the pooling layers is used for pooling on the initial image data after convolution, and the method comprises: constructing a first convolutional layer, the convolutional kernel size of the first convolutional layer is 5*5, and the first convolutional layer is used for expanding the single-channel initial image data to 64 channels; wherein, the stride of the first convolutional layer is 2, and the padding is 1; constructing a first maximum pooling layer, the window size of the first maximum pooling layer is 3*3, and the first maximum pooling layer is used for reducing the calculation amount; wherein, the stride of the first maximum pooling layer is 2, and the padding is 1; sequentially constructing a second convolutional layer, a third convolutional layer and a fourth convolutional layer, which are used for respectively expanding the 64-channel image data output by the first convolutional layer to 128 channels, 128 channels and 96 channels; wherein, the convolutional kernel size of the second convolutional layer, the third convolutional layer and the fourth convolutional layer is all 3*3, the stride is 1, and the padding is 1; constructing a second maximum pooling layer, the window size of the second maximum pooling layer is 3*3, and the stride is 2.

[0012] Optionally, the fully connected layer is composed of 3 pulse neurons.

[0013] Optionally, the original activation function in the snntorch learning framework is replaced with a sigmoid function to obtain a target training model of the image classification method; and the training data set is input into the training model to train the image classification method.

[0014] Optionally, the Fashion-MNIST data set is used as the training data set of the training model.

[0015] Optionally, the accuracy of the classification method is verified by using a confusion matrix, and the verification method of the accuracy comprises: comparing the original label of the training data set with the predicted label obtained by the classification method to obtain a class confusion result, and visualizing and displaying the class confusion result.

[0016] Optionally, the evaluation index of the image classification method comprises Micro-F1 and Macro-F1.

[0017] In a second aspect, the present application provides a computer device comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor to make the processor perform the following steps of the image classification method: constructing a plurality of convolution layers and a plurality of pooling layers respectively, each of the convolution layers is used for convolution of initial image data, and each of the pooling layers is used for pooling of the initial image data after convolution; constructing a fully connected layer, which is used for classification of the initial image data after multiple convolution and pooling, and outputs target classification result data; wherein the fully connected layer is composed of a plurality of spiking neurons.

[0018] In a third aspect, the present application provides a storage medium storing a computer program, and the computer program is executed by a processor to make the processor perform the following steps of the image classification method: constructing a plurality of convolution layers and a plurality of pooling layers respectively, each of the convolution layers is used for convolution of initial image data, and each of the pooling layers is used for pooling of the initial image data after convolution; constructing a fully connected layer, which is used for classification of the initial image data after multiple convolution and pooling, and outputs target classification result data; wherein the fully connected layer is composed of a plurality of spiking neurons.

[0019] Compared with the prior art, the present application has the following beneficial effects:

[0020] By constructing a plurality of convolution layers to perform convolution processing on the input initial image data, constructing a plurality of pooling layers to perform pooling operation on the initial image data after convolution, and then constructing a fully connected layer and adopting spiking neurons to constitute the fully connected layer, the fully connected layer composed of spiking neurons is used to replace the fully connected layer of the convolutional neural network in the prior art, so that the fully connected layer composed of spiking neurons is used to classify the initial image data after multiple convolution and pooling, and output target classification result data, thereby realizing classification processing of the image. By adopting spiking neurons to constitute the fully connected layer, the spiking neurons have a unique working mechanism similar to the biological neural network, which can reduce the required computing resources, and solve the problem that the convolutional neural network in the prior art requires huge computing resources in the training process and is prone to insufficient memory in use. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of an image classification method provided by an embodiment of the present application;

[0022] Figure 2 A structure principle diagram of an image processing process provided by an embodiment of the present application;

[0023] Figure 3Another structural principle diagram of an image processing process provided by the embodiment of the present application is shown in the figure;

[0024] Figure 4 A training curve diagram provided by the embodiment of the present application is shown in the figure;

[0025] Figure 5 An evaluation of a confusion matrix provided by the embodiment of the present application is shown in the figure;

[0026] Figure 6 Two evaluation index result diagrams provided by the embodiment of the present application are shown in the figures. DETAILED DESCRIPTION

[0027] The technical solutions in the present application are further described below with reference to the accompanying drawings and embodiments.

[0028] Figure 1 A flowchart of an image classification method provided by the embodiment of the present application is shown in the figure, Figure 1 The classification method includes the following steps:

[0029] Step S101, a plurality of convolution layers and a plurality of pooling layers are respectively constructed, each of the convolution layers is used for convolution of initial image data, and each of the pooling layers is used for pooling of the initial image data after convolution;

[0030] Step S102, a fully connected layer is constructed, which is used for classification of the initial image data after multiple convolution and pooling, and outputs target classification result data; wherein the fully connected layer is composed of a plurality of pulse neurons.

[0031] In the embodiment, the initial image data is processed by convolution through the construction of a plurality of convolution layers, the initial image data after convolution is operated by pooling through the construction of a plurality of pooling layers, and the fully connected layer is constructed and composed of pulse neurons, replacing the fully connected layer of the convolution neural network in the prior art, so that the fully connected layer composed of pulse neurons classifies the initial image data after multiple convolution and pooling, and outputs target classification result data, realizing the classification processing of the image. By using pulse neurons to constitute the fully connected layer, the pulse neurons have a unique working mechanism similar to the biological neural network, which can reduce the required computing resources, and solve the problem that the convolution neural network in the prior art requires huge computing resources in the training process and is prone to insufficient memory in use.

[0032] It should be noted that the spiking neural network has many differences compared with the deep neural network, simulates the information transmission form of the biological neural network, and transmits the sequence data of pulses in the spiking neural network, so that the original information data needs to be encoded into the pulse sequence in the form of rate coding, time coding or population coding when using the spiking neural network; the neuron in the spiking neural network is also different from the working mechanism of the ordinary neuron, and is similar to the neuron in the recurrent neural network, and contains the state information of the neuron; the spiking neural network is similar to the working mechanism of the biological neural network, and has the advantages of low power consumption and good processing of space-time information compared with the deep neural network. Figure 2 A structure principle diagram of an image processing process provided by an embodiment of the present application is shown in Figure 2 As shown in the figure, the image processing process includes a convolution layer, a pooling layer and a full connection layer, the initial picture processing data is convolved through the convolution layer, the image processing data after convolution is pooled through the pooling layer, and finally the initial image data after convolution and pooling is classified through the full connection layer, and target classification result data is output. In the figure, Image represents initial image data, Convolution layer represents convolution layer, Maximum pooling layer represents pooling layer, and Full connected layer represents full connection layer.

[0033] In another embodiment of the present application, the spiking neuron is an LIF neuron, and the working mode of the LIF neuron is:

[0034]

[0035]

[0036] s(t)=0,if u(t)<u th

[0037] Wherein, τ represents a constant, u represents a membrane potential, t represents a time step, represents a resting potential, ω j represents the weight of the jth input synapse, K(·) represents a kernel function of delay effect, and s represents an output peak, represents a reset potential, u th represents a threshold value for determining whether to ignite, represents the time when the kth pulse of the jth input synapse activates in T ω This integral time window. ω represents the integral time window.

[0038] In another embodiment of the present application, Figure 3 A structure principle diagram of another image processing process provided by an embodiment of the present application is shown inFigure 3 As shown, a plurality of convolution layers and a plurality of pooling layers are constructed respectively, each of the convolution layers is used for convolution of initial image data, and each of the pooling layers is used for pooling of the initial image data after convolution, comprising: constructing a first convolution layer, a convolution kernel size of the first convolution layer is 5*5, and the first convolution layer is used for expanding the initial image data of a single channel to 64 channels; wherein, a stride of the first convolution layer is 2, and padding is 1; constructing a first maximum pooling layer, a window size of the first maximum pooling layer is 3*3, and the first maximum pooling layer is used for reducing the amount of calculation; wherein, a stride of the first maximum pooling layer is 2, and padding is 1; sequentially constructing a second convolution layer, a third convolution layer and a fourth convolution layer, which are used for expanding the 64-channel image data output by the first convolution layer to 128 channels, 128 channels and 96 channels respectively; wherein, the convolution kernel size of the second convolution layer, the third convolution layer and the fourth convolution layer is all 3*3, the stride is 1, and the padding is 1; constructing a second maximum pooling layer, a window size of the second maximum pooling layer is 3*3, and the stride is 2. The full connection layer is composed of 3 pulse neurons.

[0039] In the embodiment, the initial data of a single channel is expanded to 64 channels by constructing a first convolution layer, and then a second first maximum pooling layer is constructed to pool the initial image data after convolution, and then a second convolution layer, a third convolution layer and a fourth convolution layer are constructed again to expand the 64-channel image data output by the first convolution layer to 128 channels, 128 channels and 96 channels respectively; then a second maximum pooling layer is constructed to perform a pooling operation on the initial image data after convolution, and finally a full connection layer is constructed, which adopts 3 pulse units to constitute the full connection layer, classifies the initial image data after multiple convolution and pooling, and outputs target classification result data. In the figure, Image represents initial image data, Convolution layer represents a convolution layer, Maximum pooling layer represents a maximum pooling layer, and Full connected layer represents a full connection layer.

[0040] In another embodiment of the present application, the original activation function in the snntorch learning framework is replaced with a sigmoid function to obtain a target training model of the image classification method; and the training data set is input into the training model to train the image classification method.

[0041] It should be noted that after the image classification method is constructed, the image classification method needs to be trained, in this embodiment, the original activation function in the snntorch learning framework is replaced with a sigmoid function to obtain a target training model of the image classification method, and a training data set is input into the training model to train the image classification method. The snntorch learning framework can be compatible with the pytorch framework in the deep neural network, and the convolutional layer can be established by using the tool class provided by pytorch, and the pulse neuron model in the fully connected layer adopts the LIF neuron model provided in snntorch, and snntorch also provides automatic differentiation and back propagation tools, which can be used to train the model, Figure 4 A training curve provided by an embodiment of the present application is shown in FIG. 8. Figure 4

[0042] In another embodiment of the present application, the Fashion-MNIST data set is used as the training data set of the training model.

[0043] It should be noted that the Fashion-MNIST data set covers 70,000 positive pictures of different goods from 10 categories, and has the same format as the MNIST data set, that is, 28*28 grayscale pictures, and the training and testing data are divided into 60000 / 10000.

[0044] In another embodiment of the present application, the accuracy of the classification method is verified by using a confusion matrix, and the verification method of the accuracy includes: comparing the original label of the training data set with the predicted label obtained by the classification method to obtain a class confusion result, and visualizing the class confusion result.

[0045] In this embodiment, the accuracy of the classification method is verified by using a confusion matrix, the original label of the training data set is compared with the predicted label obtained by the classification method to obtain a class confusion result, and specifically, Figure 5 The evaluation of the confusion matrix provided by the embodiment of the present application, each column of the confusion matrix represents the prediction of an instance of a class, and each row represents an actual instance of a class. Through the confusion matrix, it can be easily seen whether the model confuses two different classes. The confusion matrix is displayed in a visual manner, and the user directly views the accuracy of the image classification method. When the value on the diagonal line is larger, the accuracy of the image classification method is higher.

[0046] In another embodiment of the present application, the evaluation index of the image classification method includes Micro-F1 and Macro-F1. ​

[0047] In the embodiment, the data set used is the Fashion-MNIST data set, which contains more than 70,000 pictures, a total of 10 categories, but the number of pictures of each category is different. If the data quantity of one category is much larger than that of other categories, the overall prediction of the category with a large proportion can also obtain a high accuracy, and the F1 value considers the proportion of each category in the calculation process. Therefore, the F1 value is calculated to evaluate the model. The Micro-F1 and Macro-F1 indexes are slightly different in the calculation process, and the two indexes of the model are calculated to evaluate the model, Figure 6 The two evaluation index result graphs provided by the embodiment of the application are as shown in Figure 6 The F1 value of Micro-F1 is 0.9056, and the F1 value of Macro-F1 is 0.9055. Generally, the value of the index is distributed between 0 and 1, and the closer the value is to 1, the better. From Figure 5 It can be seen that the indexes of the image classification method have achieved good results.

[0048] In another embodiment of the application, a computer device is provided, comprising a processor and a memory, the memory storing a computer program, and the computer program being executed by the processor to make the processor execute the following steps of the image classification method: respectively constructing a plurality of convolution layers and a plurality of pooling layers, each convolution layer being used for convolution of initial image data, and each pooling layer being used for pooling of the initial image data after convolution; constructing a fully connected layer, which is used for classifying the initial image data after multiple convolution and pooling, and outputting target classification result data; wherein the fully connected layer is composed of a plurality of spiking neurons.

[0049] In another embodiment of the application, a storage medium is provided, which stores a computer program, and the computer program is executed by a processor to make the processor execute the following steps of the image classification method: respectively constructing a plurality of convolution layers and a plurality of pooling layers, each convolution layer being used for convolution of initial image data, and each pooling layer being used for pooling of the initial image data after convolution; constructing a fully connected layer, which is used for classifying the initial image data after multiple convolution and pooling, and outputting target classification result data; wherein the fully connected layer is composed of a plurality of spiking neurons.

[0050] It is to be noted that, in the present text, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0051] Finally, it has to be noted that all details of the above-described embodiments can be modified or substituted by other technically equivalent parts, and that "technically equivalent" means that the technical function of the parts is the same, and that the technical function is achieved in the same way.

Claims

1. An image classification method, characterized by, The classification method comprises: respectively constructing a plurality of convolution layers and a plurality of pooling layers, each of the convolution layers being used for convolution of initial image data, and each of the pooling layers being used for pooling of the initial image data after convolution; constructing a fully connected layer, which is used for classifying the initial image data after multiple convolution and pooling, and outputting target classification result data; wherein the fully connected layer is composed of a plurality of spiking neurons; the spiking neuron is an LIF neuron, and the working mode of the LIF neuron is: wherein, represents a constant, represents a membrane potential, represents a time step, represents a resting potential, represents a weight of the input synapse, represents a kernel function for a delay effect, represents an output spike, represents a reset potential, represents a threshold for deciding whether to fire, represents a time when the input synapse, is activated within the integration time, represents an integration time window; the respectively constructing a plurality of convolution layers and a plurality of pooling layers, each of the convolution layers being used for convolution of initial image data, and each of the pooling layers being used for pooling of the initial image data after convolution, comprises: constructing a first convolution layer, the convolution kernel size of the first convolution layer being 5*5, which is used for expanding the initial image data of a single channel to 64 channels; wherein the stride of the first convolution layer is 2, and the padding is 1; constructing a first maximum pooling layer, the window size of the first maximum pooling layer being 3*3, which is used for reducing the amount of calculation; wherein the stride of the first maximum pooling layer is 2, and the padding is 1; sequentially constructing a second convolution layer, a third convolution layer and a fourth convolution layer, which are used for expanding the 64-channel image data output by the first convolution layer to 128 channels, 128 channels and 96 channels respectively; wherein the convolution kernel size of the second convolution layer, the third convolution layer and the fourth convolution layer is all 3*3, the stride is 1, and the padding is 1; constructing a second maximum pooling layer, the window size of the second maximum pooling layer being 3*3, and the stride being 2.

2. The image classification method of claim 1, wherein, The fully connected layer is composed of 3 spiking neurons.

3. A method of image classification according to any one of claims 1-2, wherein, The original activation function in the snntorch learning framework is replaced with a sigmoid function to obtain a target training model of the image classification method; The training data set is input into the training model to train the image classification method.

4. The image classification method of claim 3, wherein, The Fashion-MNIST data set is used as the training data set of the training model.

5. A method of image classification as claimed in claim 4, wherein, The accuracy of the classification method is verified by using a confusion matrix, and the verification method of the accuracy comprises: comparing the original label of the training data set with the predicted label obtained by the classification method to obtain a class confusion result, and visualizing the class confusion result.

6. The image classification method of claim 4, wherein, The evaluation indexes of the image classification method include Micro-F1 and Macro-F1.

7. A computer device comprising a processor and a memory, the memory having stored therein a computer program, characterized in that, The computer program is executed by the processor to make the processor execute the steps of the image classification method according to any one of claims 1-6.

8. A storage medium storing a computer program, characterized in that, The computer program is executed by the processor to make the processor execute the steps of the image classification method according to any one of claims 1-6.

Images