Lightweight classification method for multispectral images based on binary neural network

Abstract

The present invention discloses a multispectral image lightweight classification method based on a binary neural network, comprising the following steps: step S1: modifying MobileNetV1, replacing the 3*3 deep convolution and 1*1 point-by-point convolution in the original network with 1-bit 3*3 and 1-bit 1*1 convolution respectively; step S2: activation and weight are binary quantized through a sign function; step S3: proposing to add a shortcut to transmit these real values ​​before they are binarized, retaining information, and improving the expression ability of the network; step S4: since the binary neural network is more sensitive to the activation distribution, we propose to use the RPReLU function to move the activation distribution; the problem that the traditional multispectral classification model has low classification accuracy and large result error, and the existing multispectral classification model based on deep learning cannot meet the real-time requirements of industry due to its huge amount of calculation and data storage requirements, is solved.

Description

Technical Field

[0001] The invention relates to the field of multispectral image classification, in particular to a multispectral image lightweight classification method based on a binary neural network. Background Art

[0002] Multispectral images (MSI) contain rich spectral and spatial information. Different from traditional images, multispectral images can identify objects not only based on the differences in image morphology and structure, but also based on the differences in spectral characteristics, thus expanding the amount of remote sensing information and having a wide range of applications, mainly including national defense, environmental science, forestry, etc.

[0003] Traditional machine learning methods cannot achieve ideal results in the field of remote sensing. In recent years, deep learning has achieved remarkable results in the field of computer vision, and remote sensing research has also shifted from traditional machine learning to flexible neural network structures. However, deep learning models often have too high computational complexity, and their large number of weight parameters will lead to considerable resource consumption. For multispectral images, they are rich in a large amount of ground object information, and there are many spectral bands. The data has a higher dimension and is more time-consuming to train. Therefore, it is necessary to compress the network model to obtain a lightweight model. Therefore, this patent analyzes the application of multispectral in actual industry and commerce from the perspective of lightweight models.

[0004] Prior art 1

[0005] Traditional multispectral image classification algorithms are usually divided into unsupervised and supervised, such as unsupervised clustering algorithms and supervised support vector machines. Senthilnath J et al. proposed a new clustering method based on the bat algorithm to solve the problem of multispectral image classification. They proposed a partition clustering algorithm to extract information in the form of optimal cluster centers from training samples. Zhu Q et al. proposed a BoVW-based method, whose visual dictionary is obtained by performing k-means clustering on features. Tuia D et al. proposed to encode the output spatial structure using a hierarchical tree, and this information will be added to the model in the kernel and loss function.

[0006] Disadvantages of the prior art 1

[0007] They only focus on the spectral information of multispectral images and ignore the importance of their spatial information, resulting in low final classification accuracy and large result errors.

[0008] Prior art 2

[0009] In recent years, deep learning has achieved remarkable results in the field of computer vision, and more and more deep learning models have been applied to remote sensing image processing. WZhao et al. proposed a multi-local convolutional neural network model based on superpixels, which extended the simple linear iterative clustering algorithm to segment multispectral images and generate superpixels. In order to make full use of spatial spectrum and environmental information, a multi-local region joint representation method based on superpixels was proposed to extract effective joint feature representation. M Lin et al. proposed a context-aware graph neural network to enhance feature expression. The network extracts intra-class embedding in non-Euclidean space by combining similarly distributed feature vertices, and uses gated recurrent units to capture context-aware information. Sargent et al. proposed to effectively integrate the convolutional neural network based on deep spatial feature representation and the multi-layer perceptron based on spectral recognition to achieve the classification of remote sensing images with extremely fine spatial resolution.

[0010] Disadvantages of the second prior art

[0011] Although deep learning models perform well in multispectral classification, they require a lot of computing resources and space storage, which cannot meet the real-time requirements of industry. In order to solve the problem of resource occupation, we propose to use binary neural network (BNN) to obtain a lightweight multispectral classification model. Binary neural network is an important representative of network quantization technology. It uses the sign function to binarize parameters such as weights and activations into two values ​​of -1 and +1 to achieve model compression. Summary of the invention

[0012] In order to solve the problems existing in the prior art, the present invention provides a lightweight classification method for multispectral images based on binary neural networks, which solves the problem that the traditional multispectral classification model has low classification accuracy and large result errors, and the existing multispectral classification model based on deep learning cannot meet the real-time requirements of the industry due to its huge amount of calculation and data storage requirements.

[0013] The specific technical solutions are as follows:

[0014] The lightweight classification method of multispectral images based on binary neural network includes the following steps:

[0015] Step S1: Modify MobileNetV1 and replace the 3*3 depth convolution and 1*1 point-by-point convolution in the original network with 1-bit 3*3 and 1-bit 1*1 convolution respectively;

[0016] Step S2: Activation and weights are binary quantized using the sign function;

[0017] Step S3: It is proposed to add a shortcut to transmit these real values ​​before they are binarized, so as to retain information and improve the expression ability of the network;

[0018] Step S4: Since binary neural networks are sensitive to activation distribution, we propose to use the RPReLU function to shift the activation distribution;

[0019] Step S5: For the channel attention mechanism, a lightweight ECA module is used, which uses a local cross-channel interaction method without dimensionality reduction to reduce the model complexity while maintaining performance. This method can be effectively implemented through one-dimensional convolution;

[0020] Step S6: select k=3, perform global average pooling on each channel, use a one-dimensional convolution with a receptive field of 3 to obtain the weight corresponding to each channel, and then use the weight to weight to obtain a new weighted feature;

[0021] Step S7: Using the lightweight channel attention module only adds a small number of parameters, but can achieve significant performance gains;

[0022] Step S8: Experiments are conducted on three widely adopted public datasets.

[0023] Preferably, the specific formula of RPReLU in step S4 is:

[0024]

[0025] Here, γ and δ are learnable coefficients used to shift the distribution, and β is a learnable coefficient used to control the slope of the negative part.

[0026] Preferably, the specific formula of step S5 is:

[0027] w=σ(C1D k (y))

[0028] Among them, for the weight y, the k neighbors near the channel are used to participate in the attention prediction of this channel, and σ is the Sigmoid function.

[0029] Preferably, the experimental conclusion of step S8 is: CABNN is superior to the most advanced methods in all aspects. With only a small amount of computation, the accuracy is improved by 0.37% on the NaSC-TG2 dataset, 0.35% on the EuroSAT dataset, and 2.18% on the GID Fine Land-cover Classification dataset.

[0030] The beneficial effects of the multispectral image lightweight classification method based on binary neural network of the present invention are as follows:

[0031] 1. In order to solve the defects of the huge amount of calculation and data storage requirements of the existing model, the present invention proposes to use a lightweight binary neural network to classify multispectral images. A binary neural network refers to a deep network model with binary weights and binary activations, that is, the floating point 32-bit data in the traditional convolutional neural network is converted into 1-bit data. After such a binarization process, the parameter bit width is reduced to 1 / 32 of the original, and the required storage space is greatly reduced. The convolution process can be replaced by XOR operations and popcount operations, which is faster and reduces the computational cost of the network.

[0032] 2. In view of the characteristics of multispectral remote sensing images such as high dimensionality and richer information, this patent introduces a lightweight channel attention mechanism ECA module to assign different weights to different channels, focus on important information and suppress unimportant information, so as to better handle multispectral image classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is the ECA module diagram of the present invention.

[0034] Figure 2 It is a flow chart of the present invention.

[0035] Figure 3 This is the confusion matrix diagram of the NaSC-TG2 data set of the present invention.

[0036] Figure 4 This is the confusion matrix diagram of the EuroSAT data set of the present invention.

[0037] Figure 5 This is the confusion matrix diagram of the GID Fine Land-cover Classification dataset of the present invention. DETAILED DESCRIPTION

[0038] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0039] The present invention is implemented through the existing deep learning framework Pytorch and the corresponding programming library. Pytorch mainly uses deep learning models, including linear modules, convolution modules, etc.

[0040] The specific implementation principle of the scheme is as follows:

[0041] like Figure 2 As shown in the figure, first, MobileNetV1 is modified by replacing the 3*3 depth convolution and 1*1 point-by-point convolution in the original network with 1-bit 3*3 and 1-bit 1*1 convolutions respectively.

[0042] In addition, activations and weights are binary quantized using the sign function.

[0043] We found that real values ​​are generated after convolution operation in binary neural network, and these real values ​​will be binarized in the next convolution operation, which leads to a large amount of information loss and affects the final classification accuracy. Therefore, we propose to add a shortcut to transfer these real values ​​before they are binarized, retain information, and improve the network's expressiveness.

[0044] Since binary neural networks are sensitive to activation distribution, we propose to use the RPReLU function to move the activation distribution. RPReLU is defined as follows:

[0045]

[0046] Here, γ and δ are learnable coefficients used to shift the distribution, and β is a learnable coefficient used to control the slope of the negative part.

[0047] Secondly, for the channel attention mechanism, such as Figure 1 As shown in the figure, a lightweight ECA module is used, which uses a local cross-channel interaction method without dimensionality reduction to reduce the complexity of the model while maintaining performance. This method can be effectively implemented through one-dimensional convolution. The specific formula is as follows:

[0048] w=σ(C1D k (y))

[0049] Among them, for the weight y, the k neighbors near the channel are used to participate in the attention prediction of this channel, and σ is the Sigmoid function.

[0050] We choose k=3, and after global average pooling for each channel, use a one-dimensional convolution with a receptive field of 3 to obtain the weight corresponding to each channel, and then use the weight to weight to obtain a new weighted feature. Using this lightweight channel attention module only adds a small number of parameters, but can achieve significant performance gains.

[0051] Finally, this patent conducted experiments on three widely used public datasets. The experiments show that the CABNN we proposed is superior to the most advanced methods in all aspects. With only a small amount of computation, the accuracy is improved by 0.37% on the NaSC-TG2 dataset, 0.35% on the EuroSAT dataset, and 2.18% on the GID Fine Land-cover Classification dataset. The confusion matrix of each dataset is as follows: Figures 3 to 5 shown.

[0052] The network architecture is implemented as a key protection point, and the specific protection technology focuses are as follows:

[0053] Binarized Baseline Network

[0054] Modifications are made to the MobileNetV1 network structure. The initial depthwise separable convolution is replaced with a 1-bit convolution layer. The weights and activations in the network are binarized into +1 and -1 using the sign function. A shortcut is added to reduce information loss. The RPReLU function is then used to reshape the distribution of the activation function to improve the network's expressiveness.

[0055] ECA Module

[0056] By utilizing the local cross-channel interaction method without dimensionality reduction, corresponding weights are assigned to different channels of the multispectral image to enhance the feature extraction capability of the network and improve the model performance.

Claims

1. A lightweight classification method for multispectral images based on binary neural network, characterized in that: The following steps are involved: Step S1: Modify MobileNetV1 and replace the 3*3 depth convolution and 1*1 point-by-point convolution in the original network with 1-bit 3*3 and 1-bit 1*1 convolution respectively; Step S2: Activation and weights are binary quantized using the sign function; Step S3: It is proposed to add a shortcut to transmit these real values ​​before they are binarized, so as to retain information and improve the expression ability of the network; Step S4: Since the binary neural network is sensitive to the activation distribution, the RPReLU function is used to move the activation distribution; Step S5: For the channel attention mechanism, a lightweight ECA module is used, which uses a local cross-channel interaction method without dimensionality reduction to reduce the model complexity while maintaining performance. This method can be effectively implemented through one-dimensional convolution; Step S6: select k=3, perform global average pooling on each channel, use a one-dimensional convolution with a receptive field of 3 to obtain the weight corresponding to each channel, and then use the weight to weight to obtain a new weighted feature; Step S7: Experiments were conducted on three widely adopted public datasets; The specific formula of RPReLU in step S4 is: Among them, γ and δ are learnable coefficients used to shift the distribution, and β is a learnable coefficient used to control the slope of the negative part; The specific formula of step S5 is: in=σ(C1D k (y)) Among them, for the weight y, the k neighbors near the channel are used to participate in the attention prediction of this channel, and σ is the Sigmoid function.

2. The multispectral image lightweight classification method based on binary neural network according to claim 1 is characterized in that: The experimental conclusion of step S7 is that CABNN is superior to the most advanced methods in all aspects. With only a small amount of computation, the accuracy is improved by 0.37% on the NaSC-TG2 dataset, 0.35% on the EuroSAT dataset, and 2.18% on the GIDFine Land-cover Classification dataset.

Images