A lightweight convolutional neural network construction method based on SqueezeNet

Abstract

This invention relates to the field of deep learning algorithm design technology, specifically to a method for constructing a lightweight convolutional neural network based on SqueezeNet. The method includes: improving the SqueezeNet model to obtain a Slight-SqueezeNet model; fusing InstanceNormalization and GroupNormalization to obtain an IG-module normalization method; and integrating the IG-module into the Slight-SqueezeNet model to obtain a Slight-SqueezeNet-IGN lightweight convolutional neural network. This invention constructs a novel lightweight convolutional neural network by improving SqueezeNet and fusing IG-module. Under the same task, compared to large convolutional neural networks such as AlexNet and VGG16, it can improve recognition accuracy without increasing the number of parameters or model complexity. It has good recognition accuracy and generalization ability, and has stronger practical value and significance. It can be better embedded into hardware devices to meet practical industrial applications.

Description

Technical Field

[0001] This invention relates to the field of deep learning algorithm design technology, and in particular to a method for constructing a lightweight convolutional neural network based on SqueezeNet. Background Technology

[0002] In recent years, deep learning technology has ushered in a period of rapid development, achieving remarkable results in the field of image recognition, with a plethora of deep learning models emerging. However, while accuracy has increased, the inference complexity and computation time of these models have also surged. Therefore, the industry urgently needs a deep learning algorithm that can simultaneously achieve high accuracy and fast computation, to be embedded into hardware devices for application in various industrial fields and to promote national industrial development.

[0003] Conventional convolutional neural networks (CNNs) often achieve high model accuracy by simply stacking network layers, such as AlexNet, VGG16Net, and GoogleNet. However, this approach results in complex networks with a large number of parameters, making them prone to overfitting on small datasets and causing a decrease in model accuracy instead of an increase. As the number of network layers increases, the vanishing gradient problem also becomes severe, and the training process consumes a significant amount of time, hindering network optimization and subsequent hardware deployment. Therefore, designing a lightweight CNN with fewer parameters, a smaller model size, and high accuracy—and improving model recognition accuracy while reducing parameter and model complexity—is a pressing issue that needs to be addressed. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight convolutional neural network construction method based on SqueezeNet, which can construct a lightweight convolutional neural network that reduces the number of parameters and model complexity while improving model accuracy.

[0005] To achieve the above objectives, this invention provides a lightweight convolutional neural network construction method based on SqueezeNet, comprising:

[0006] The SqueezeNet model was improved to obtain the Slight-SqueezeNet model;

[0007] By merging Instance Normalization and Group Normalization, we obtain the IG-module normalization method;

[0008] The IG-module is integrated into the Slight-SqueezeNet model to obtain the Slight-SqueezeNet-IGN lightweight convolutional neural network.

[0009] The SqueezeNet model includes two convolutional layers 1-2, three max pooling layers 1-3, a global average pooling layer, eight fire modules 1-8, and a softmax activation layer. The convolutional layer 1, the max pooling layer 1, the fire modules 1-3, the max pooling layer 2, the fire modules 4-7, the max pooling layer 3, the fire module 8, the global average pooling layer, and the softmax activation layer are connected sequentially.

[0010] The fire module includes a compression layer and an expansion layer. The compression layer consists of 1x1 convolutional kernels, and the expansion layer includes a left branch and a right branch. The left branch consists of 1x1 convolutional kernels, and the right branch consists of 3x3 convolutional kernels.

[0011] The specific steps for improving the SqueezeNet model include: changing the stride parameter in the convolutional layer 1 of the SqueezeNet model to 1, and deleting the pooling layer 1 of the SqueezeNet model.

[0012] The specific steps of integrating the IG-module into the Slight-SqueezeNet model include: integrating the IG-module into the right branch of the fire module, using the feature image after feature extraction by the 3x3 convolution kernel as input, and inputting the feature image into the InstanceNormalization and Group Normalization respectively, so as to learn new features while preserving as many original features as possible.

[0013] This invention presents a lightweight convolutional neural network construction method based on SqueezeNet. By improving the network structure of SqueezeNet, it reduces the number of model parameters and lowers its complexity. Furthermore, the improved model enhances feature extraction capabilities and improves generalization ability. This invention also proposes a plug-and-play image normalization module (IG-module), which can improve recognition accuracy and feature extraction capabilities without increasing the number of parameters. This plug-and-play module can be embedded into different deep learning networks, exhibiting a certain degree of universality. This invention successfully proposes a novel lightweight convolutional neural network (Slight-SqueezeNet-IGN) by improving SqueezeNet and fusing the IG-module. Under the same task, compared to large convolutional neural networks such as AlexNet and VGG16, it can improve recognition accuracy without increasing the number of parameters and model complexity, demonstrating good recognition accuracy and generalization ability. It has greater practical value and significance, and can be better embedded into hardware devices to meet practical industrial applications. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a diagram showing the components of the SqueezeNet model and the locations where improvements are made to the model.

[0016] Figure 2 This is a schematic diagram of the components of the fire module.

[0017] Figure 3 This is a schematic diagram of the components of the standardized IG-module.

[0018] Figure 4 This is a schematic diagram of the fire module integrated into the IG-module.

[0019] Figure 5 This is a flowchart of a lightweight convolutional neural network construction method based on SqueezeNet according to the present invention. Detailed Implementation

[0020] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0021] Please see Figures 1-5 This invention provides a lightweight convolutional neural network construction method based on SqueezeNet, comprising:

[0022] S1 improves the SqueezeNet model to obtain the Slight-SqueezeNet model;

[0023] The SqueezeNet model comprises two convolutional layers (1-2), three max-pooling layers (1-3), a global average pooling layer, eight fire modules (1-8), and a softmax activation layer. These convolutional layers are sequentially connected: convolutional layer 1, max-pooling layer 1, fire modules 1-3, max-pooling layer 2, fire modules 4-7, max-pooling layer 3, fire module 8, the global average pooling layer, and the softmax activation layer. Each fire module includes a compression layer and a dilation layer. The compression layer consists of 1x1 convolutional kernels, and the dilation layer includes a left branch and a right branch. The left branch consists of 1x1 convolutional kernels, and the right branch consists of 3x3 convolutional kernels. When using fire modules, the number of convolutional kernels in the compression layer is generally less than the number of convolutional kernels in the dilation layer. This significantly reduces the number of parameters and the number of channels in the feature map. The use of fire modules allows SqueezeNet to improve accuracy while significantly reducing the number of parameters. Then, SqueezeNet is improved. The first operation is to reduce the number of convolutional layers 1 (…) of the SqueezeNet network. Figure 1 The step size parameter in the middle (indicated by point 1) is changed to 1. Operation two is to change the pooling layer 1 of the SqueezeNet network ( Figure 1 The two operations (deleting at point 2) not only reduce the number of parameters in the SqueezeNet model, but also increase the image dimension of the network feature extraction layer, thereby preserving image feature information and improving the model's recognition accuracy. Next, there are improvements to the fire module; the specific fire module structure diagram is shown below. Figure 2 As shown, Slight-SqueezeNet is proposed after making minor modifications and reductions to the SqueezeNet model. Since the use of some convolutional layers in the SqueezeNet model can lead to excessive loss of information in the input image, increasing the error rate of the model, Slight-SqueezeNet is proposed to reduce the loss of image information and improve the generalization performance of the model.

[0024] S2 combines Instance Normalization and Group Normalization to obtain the IG-module normalization method;

[0025] By combining instance normalization and group normalization, the input image is normalized by instance normalization and group normalization respectively. Then, the feature image is overlaid by adding features. The fused feature image not only retains the feature information of the original image as much as possible, but also incorporates some new feature information. This normalization method does not increase the number of network parameters, and at the same time, it is beneficial to improve the recognition accuracy and generalization ability of the model.

[0026] S3 integrates the IG-module into the Slight-SqueezeNet model to obtain the Slight-SqueezeNet-IGN lightweight convolutional neural network;

[0027] Integrating IG-module into the fire module, that is Figure 1 The third improvement measure involves integrating the IG-module into the right branch of the fire module. The feature image extracted by the 3×3 convolution kernel is used as input, and the feature image is input into instance normalization and group normalization respectively. While preserving as many original features as possible, new features are learned. This enhances the feature extraction capability of the fire module and ultimately improves the feature extraction capability of the overall network. It also improves the recognition accuracy and generalization ability of the model while reducing parameters, which is more conducive to the industrial embedded application of deep learning classification algorithms.

[0028] This invention presents a lightweight convolutional neural network construction method based on SqueezeNet. By improving the network structure of SqueezeNet, it reduces the number of model parameters and lowers its complexity. Furthermore, the improved model enhances feature extraction capabilities and improves generalization ability. This invention also proposes a plug-and-play image normalization module (IG-module), which can improve recognition accuracy and feature extraction capabilities without increasing the number of parameters. This plug-and-play module can be embedded into different deep learning networks, exhibiting a certain degree of universality. This invention successfully proposes a novel lightweight convolutional neural network (Slight-SqueezeNet-IGN) by improving SqueezeNet and fusing the IG-module. Under the same task, compared to large convolutional neural networks such as AlexNet and VGG16, it can improve recognition accuracy without increasing the number of parameters and model complexity, demonstrating good recognition accuracy and generalization ability. It has greater practical value and significance, and can be better embedded into hardware devices to meet practical industrial applications.

[0029] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A method for constructing a light-weight convolutional neural network based on SqueezeNet, characterized in that, The application relates to a Slight-SqueezeNet model obtained by improving a SqueezeNet model; the SqueezeNet model comprises two convolution layers 1-2, three maximum pooling layers 1-3, a global average pooling layer, eight fire modules 1-8 and a softmax function activation layer; the convolution layer 1, the maximum pooling layer 1, the fire modules 1-3, the maximum pooling layer 2, the fire modules 4-7, the maximum pooling layer 3, the fire module 8, the global average pooling layer and the softmax function activation layer are sequentially connected; the fire module comprises a compression layer and a dilation layer; the compression layer is composed of a 1x1 size convolution kernel; the dilation layer comprises a left branch and a right branch; the left branch is composed of a 1x1 convolution kernel; and the right branch is composed of a 3x3 size convolution kernel. An IG-module normalization mode is obtained by fusing an Instance Normalization and a Group Normalization; the specific process of the IG-module normalization mode comprises the following steps: input images are subjected to Instance Normalization and Group Normalization respectively, and then subjected to an add feature image superposition operation. The IG-module is integrated into the Slight-SqueezeNet model to obtain a Slight-SqueezeNet-IGN light-weight convolutional neural network; the specific steps comprise the following steps: the IG-module is integrated into the right branch of the fire module; the feature image obtained by performing feature extraction on a 3x3 convolution kernel is taken as input; the feature images are input into the Instance Normalization and the Group Normalization respectively; and new features are learned under the premise of maintaining original features as much as possible.

2. The SqueezeNet-based light-weight convolutional neural network construction method according to claim 1, wherein the specific steps of improving the SqueezeNet model comprise the following steps: the step parameter in the convolution layer 1 of the SqueezeNet model is changed to 1; and the pooling layer 1 of the SqueezeNet model is deleted. ​ ​

Images