Image classification method, system, device and medium based on deep binary neural network model
By improving the binary convolution module, attention mechanism, and adaptive linear scaling factor, the deep binary neural network structure is reshaped, solving the problems of high model complexity and unstable training, and achieving higher image classification accuracy and speed.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2023-12-28
- Publication Date
- 2026-07-03
AI Technical Summary
Existing binary neural network models are complex in structure, have high computational and storage costs, are unstable during training, and suffer from low accuracy due to information loss.
By improving the binary convolution module, adding an attention mechanism module, and giving backpropagation an adaptive linear scaling factor, the deep binary neural network structure is reshaped, and the model is trained using the Adam algorithm.
This reduces model complexity and improves training stability, as well as the accuracy and speed of image classification.
Smart Images
Figure CN117765334B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, and in particular to an image classification method, system, device and medium based on a deep binary neural network model. Background Technology
[0002] With the rapid development of neural networks, their application in mobile devices and edge computing platforms has attracted widespread attention from industry and academia, such as image classification, object detection, and autonomous driving. However, convolutional neural network (CNN) architectures require more computing power, and addressing the challenges of computational and network requirements associated with CNNs is difficult, especially when resource constraints are encountered and implementation on low-power platforms is necessary.
[0003] Simons et al. restricted the weights and activations in neural networks to two fixed values, typically -1 and 1 (or 0 and 1) (T. Simons, and D.J. Lee, "A review of binarized neural networks," Electronics, vol. 8, no. 6, pp. 661-687, Jun. 2019). This reduction in computational and storage requirements improved the efficiency and speed of neural networks, leading to the widespread application of binary neural networks (BNNs) in computer vision, speech recognition, and other fields. Compared to traditional floating-point computation, BNNs improve efficiency on the same hardware and enable fast inference in resource-constrained environments, such as mobile devices. During the development of BNNs, researchers proposed various binary methods and algorithms, including binary networks, XNOR-Net, and DoReFa-Ne.
[0004] As a pioneering work, binary neural networks used sign functions for binarizing weights and activations and employed direct estimators (STEs) to approximate the quantization error. With further research, many schemes for binary networks have been developed to reduce the gap between full-precision and binary networks. Recent advances in binary neural networks can be categorized into three main types:
[0005] Minimizing Quantization Error: A common approach to optimizing binary neural networks is to minimize the quantization error in the weights and activations. This requires ensuring that the quantization parameters closely match the full-precision parameters, and it is expected that the performance of the binary network will be comparable to that of the full-precision network. Rastegari et al. proposed XNOR-Net (M. Rastegari et al., "Xnor-net: Imagenet classification using binary convolutional neural networks," in Proc. Eur. Conf. Comput. Vis. (ECCV), 2016, pp. 525-542), which addresses quantization error by introducing a scaling factor to approximate floating-point weights and binarizing the weights and activations. Similarly, XNOR-Net++ combines the weight and activation scaling factors into a single factor, enabling fine-tuning of the convolution output.
[0006] Improving Network Loss Functionality: Hou et al. introduced a method called Loss-Aware Binarization (LAB) (L. Hou, Q. Yao, and JT Kwok, "Loss-aware binarization of deep networks," 2016, arXiv: 1611.01600.), which aims to minimize the overall loss associated with binary weights by using a quasi-Newton algorithm. In addition to quantitatively considering the task-related loss, an additional quantization-aware loss is added during the optimization process. Ding et al. highlighted some challenges faced by binary neural networks due to forward binarization and backpropagation, including issues such as "degeneration," "saturation," and "gradient mismatch," which can affect the performance and training process of binary neural networks (R. Ding et al., "Regularizing activation distribution for training binarized deep networks," in Proc. IEEE / CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 11408-11417.). To address these challenges, we can obtain guidance information from large-scale, full-precision models. The apprenticeship method proposed by Ding et al. uses a pre-trained, full-precision, large-scale teacher network to train a low-precision student network. This method allows knowledge captured by the teacher network to guide the training process of the binary network model, improving its performance and generalization ability (A. Mishra, and D. Marr, "Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy," 2017, arXiv:1711.05852.).
[0007] Reducing Gradient Error: Traditionally, training binary networks relies on backpropagation algorithms, but this can lead to gradient mismatch between the sign function and the pass-through estimator (STE). To address this issue, researchers have proposed various techniques to mitigate gradient mismatch in backpropagation. Liu et al. proposed a custom method, the sign function, to replace the sign function in the gradient estimator used in Bi-real networks. By designing an approximation function, they improved the compatibility between the sign function and backpropagation (Z. Liu et al., "Bi-real net: Enhancing the performance of 1-bit CNNS with improved representational capability and advanced training algorithm," in Proc. Eur. Conf. Comput. Vis. (ECCV), Sep. 2018, pp. 722-737.). Binary Neural Networks+ employs a different approach, proposing a better approximation of the derivative of the sign function and a regularization function to encourage learnable weights to be closer to binary values, further reducing gradient mismatch. Gong et al. proposed a method called Soft Quantization (DSQ), which makes the quantization operation differentiable, allowing for smoother gradient updates during training (R. Gong et al., "Differentiable soft quantization: Bridging full-precision and low-bit neural networks," in Proc. IEEE / CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2019, pp. 4852-4861.). Qin et al.
[34] developed IR-Net, which reduces gradient error during training by using an adaptive error decay estimator (EDE). EDE balances the ability to update weights with the goal of reducing gradient error, taking into account different requirements at different stages of the training process (H. Qin et al., "Forward and backward information retention for accurate binary neural networks," in Proc. IEEE / CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 2250-2259.).
[0008] The manually designed binarization networks VGG and ResNet are widely considered benchmark models in the binarization field due to their strong performance on various datasets. On one hand, VGG is known for its deep network structure containing 16 or 19 layers, consisting of simple convolutional and pooling layers. Therefore, using VGG networks as binary benchmark networks is a common and straightforward approach. On the other hand, ResNet introduces the concept of residual learning, effectively addressing the challenges of vanishing and exploding gradients in deep neural networks through cross-layer connections. Compared to VGG, ResNet has a deeper network structure and achieves significant performance across various image tasks.
[0009] These benchmark networks are typically directly binarized to obtain binary networks, with the expectation of high performance. In reality, directly binarized benchmark networks perform poorly on various datasets. The state-of-the-art binary network, ReActNet, takes a different approach, using the lightweight MobileNet as its benchmark network, resulting in better performance. MobileNet is a compact convolutional neural network renowned for its efficiency and accuracy, particularly well-suited for deployment on mobile devices. By employing depthwise separable convolutions, MobileNet reduces computational complexity and model size while maintaining high accuracy. Research indicates that MobileNet's more compact network architecture is better suited for training binary networks.
[0010] Neural Architecture Search (NAS) has garnered significant attention for its ability to improve network performance and generalization by automatically searching for optimal network structures. NAS methods are broadly categorized into three types: evolutionary-based methods, reinforcement learning-based methods, and gradient-based methods. Specifically, evolutionary-based methods typically require substantial computational resources and expertise to find high-performance encodings for network structures; reinforcement learning-based methods generally require even more computational resources, potentially taking days even on medium-sized datasets using thousands of GPUs; gradient-based methods are more efficient than both evolutionary and reinforcement learning-based methods, however, researchers often discover pathological structures due to inappropriate adaptation to gradient optimization, a relationship that has not yet been mathematically proven. Furthermore, gradient-based methods require the pre-construction of a supernet, which also demands a high level of expertise. Therefore, combining binary networks with NAS is a promising approach, but it also presents greater challenges.
[0011] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:
[0012] 1. High requirements for model structure: In order to maintain the performance of the model, binary neural networks may require more complex structures and more parameters, which increases the computation and storage costs of the model.
[0013] 2. Difficult and unstable training: During the training process, since binary neural networks use binary weights and activation values, they may face convergence difficulties and require more training time and resources.
[0014] 3. Information loss and low precision: Restricting weights and activation values to binary may lead to information loss, reduce the model's representational power, and thus affect the model's performance. Summary of the Invention
[0015] In order to overcome the shortcomings of the prior art, the present invention aims to propose an image classification method, system, device and medium based on a deep binary neural network model. The improved deep binary neural network model is used for image classification, which reduces the model complexity and has better stability. The accuracy and speed of image classification results are improved to a certain extent.
[0016] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0017] Image classification methods based on deep binary neural network models include the following steps:
[0018] Obtain the image to be classified;
[0019] The image to be classified is input into a preset deep binary neural network model to obtain the classification prediction result corresponding to the image to be classified through the deep binary neural network model.
[0020] The deep binary neural network model is obtained through the following steps:
[0021] Step 1: Improve the binary convolution module by fusing the improved binary convolution module with the remodeling activation module to reconstruct the structure of the deep binary neural network model.
[0022] Step 2: Based on the deep binary neural network model structure obtained in Step 1, add an attention mechanism module to each layer of binary convolution operation;
[0023] Step 3: Based on the deep binary neural network model structure obtained in Step 1, assign an adaptive linear scaling factor for backpropagation to the binary convolution operation.
[0024] Step 4: Based on the deep binary neural network model structure obtained in Steps 1-3, build the baseline network of the deep binary neural network model and initialize the floating-point parameters of the baseline network.
[0025] Step 5: Train the deep binary neural network model obtained in step 4 using the Adam algorithm to obtain the deep binary neural network model.
[0026] The specific process of step 1 is as follows:
[0027] Step 1.1: Improve the two point convolutions in the binary convolution module to grouped point convolutions. One grouped point convolution operation is followed by a channel blending module. The channel blending module allows image features to be fused and communicated between different channels by changing the order of different channels. The other grouped point convolution operation is followed by nothing, which is used to retain more information from the original channels.
[0028] Step 1.2: Use the Reshape Activation module to change the distribution of activation by replacing the Sign function, as shown below:
[0029]
[0030]
[0031] Where x represents the real-valued activation before binarization, x b These are their binarized values, β is a trainable coefficient that affects the slope of the negative half-axis, and α and ξ are learnable offsets that reshape the distribution on the horizontal and vertical axes.
[0032] The specific process of step 2 is as follows:
[0033] Step 2.1: By using a weight matrix, different weights are assigned to different positions in the image from the perspective of channels, and more important feature information is obtained. The importance of each feature channel is automatically learned through the weight matrix.
[0034] Step 2.2: For a given feature map in a certain layer of the network, first perform global average pooling on the feature map to generate a one-dimensional vector. Each channel can be represented by a numerical value. Then pass the vector through two fully connected layers to generate the required weight matrix.
[0035] Step 2.3: Then, assign weights to the weight matrix generated in Step 2.2 and the original feature map to obtain the final feature map, which has the same size as the original feature map. The attention mechanism assigns corresponding weights to each feature channel according to the importance of each feature channel, allowing the neural network to focus on the channel most relevant to the current task.
[0036] Step 3 assigns an adaptive linear scaling factor that can be backpropagated, specifically:
[0037] w~α·bin(w)~(a·α+b)·bin(w)
[0038] Where w is the weight of the full-precision convolution kernel, b w α represents the weights for binarization, and α is the scaling factor.
[0039] The backpropagation of the adaptive linear scaling factor in step 3 is specifically as follows:
[0040] Let x be the node value of layer N-1 of the network, h be the node value of layer N of the network, and y be the node value of layer N+1 of the network (N>0). The original weights α·bin(w) are replaced with improved weights (a·α+b)·bin(w). The linear adaptive scaling factor a·α+b is updated via gradient descent. Given a training dataset D, assuming E is the loss of layer N+2 of the network, w1, a1, b1, and α1 are the parameters from layer N-1 to layer N, and w2, a2, b2, and α2 are the parameters from layer N to layer N+1, the backpropagation process is as follows:
[0041]
[0042] The partial derivatives of the variable loss can be derived using the chain rule:
[0043]
[0044] The partial derivatives output to the variable can also be derived using the chain rule:
[0045]
[0046]
[0047] Use an approximation sign function to update the gradient:
[0048]
[0049]
[0050]
[0051] During training, the true numerical weights are retained, and backpropagation is performed using the chain rule of the approximation function to approximate the approximation function, continuously updating the weights of each layer according to the equation.
[0052] The specific process of step 4 is as follows:
[0053] Step 4.1: Initialize the parameters of the baseline network, namely the weights w and parameters a and b of the full-precision convolutional kernel from Step 3.
[0054] Step 4.2: Select an approximate gradient function for backpropagation in the binary network to update the weights of the deep binary neural network model, preparing the binary neural network for normal training. The approximate gradient function is as follows:
[0055]
[0056] The specific process of step 5 is as follows:
[0057] The Adam algorithm is used for training, employing a learning rate decreasing with a cosine function scheduler. The Adam algorithm dynamically adjusts the learning rate based on the first and second estimates of the gradient.
[0058]
[0059]
[0060]
[0061] Where, m t v is the average value of the gradient at the first time step. t Let be the non-central variance of the gradient at the second time step, where β1 is set to 0.9, β2 to 0.9999, and ε to 10⁻⁸.
[0062] Image classification systems based on deep binary neural network models include:
[0063] The image acquisition module is used to acquire images to be classified.
[0064] The image input module inputs the image to be classified into a preset deep binary neural network model so as to obtain the classification prediction result corresponding to the image to be classified through the deep binary neural network model.
[0065] The deep binary neural network model includes:
[0066] The attention mechanism module first enters the image, which dynamically adjusts the weights of different channels, then enhances the features of important channels and suppresses the features of unimportant channels to obtain a new feature map.
[0067] Reshape the activation module to incorporate the features obtained from the attention mechanism module. Figure 2 Value-based processing and dynamic compression produce different binary feature maps;
[0068] The binary convolution module is used to perform binary convolution operations on the binary feature maps obtained by the re-activation module, extract features, and output the feature maps after XOR operation.
[0069] The channel mixing module is used to mix feature information from different channels to obtain richer and more diverse feature maps.
[0070] Image classification devices based on deep binary neural network models include:
[0071] Memory: Used to store the computer program for the image classification method based on the deep binary neural network model;
[0072] Processor: Used to implement the image classification method based on a deep binary neural network model when executing the computer program.
[0073] A computer-readable storage medium storing a computer program that, when executed by a processor, enables an image classification method based on a deep binary neural network model.
[0074] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0075] 1. In step 1 of this invention, the structure of the deep binary neural network model is reconstructed. By improving the binary convolution module, new connections are made between different parts of the channel mixing module and the original channel module, which increases the diversity of feature maps and reduces computational complexity. The channel mixing module reduces redundant information and increases effective information, thus realizing richer feature map information between different channels.
[0076] 2. In step 2 of this invention, by adding an attention mechanism module to each layer of binary convolution operation, more image feature information can be obtained, useful feature channels are enhanced, unimportant feature channels are suppressed, and the sensitivity of the deep binary neural network model to channel features is improved.
[0077] 3. In step 3 of this invention, an adaptive linear scaling factor that can be backpropagated is assigned, which enables stable training of the binary neural network and improves accuracy without increasing computational load.
[0078] 4. In step 4 of this invention, by building a baseline network for a deep binary neural network model and initializing floating-point parameters, the techniques of steps 1-3 can be integrated, preparing for subsequent training.
[0079] In summary, this invention utilizes multiple modules to reshape and fuse the deep binary neural network model, namely, reshaping the channel mixing module, adding an attention mechanism module, and assigning an adaptive linear scaling factor. This improves the deep binary neural network model, and the improved deep binary neural network model is used for image classification, reducing model complexity. At the same time, the training process has better stability, and the accuracy and speed of classification results on different datasets are improved to a certain extent. Attached Figure Description
[0080] Figure 1 This is a flowchart of the method of the present invention.
[0081] Figure 2 These are feature maps processed by binary neural network modules of different depths according to embodiments of the present invention. The yellow line represents the real value propagated within the path, and the blue line represents the binary value.
[0082] Figure 3 This is a structural diagram of the improved binary convolution module and remodeling activation module fusion provided in the embodiments of the present invention.
[0083] Figure 4 This is an intermediate feature map provided in an embodiment of the present invention, wherein, Figure 4 (a) is the input image. Figure 4 (b) is the feature map of the original point-to-convolution. Figure 4 (c) is a feature diagram of the present invention.
[0084] Figure 5 This is a detailed operational diagram of the attention mechanism provided in the embodiments of the present invention.
[0085] Figure 6 This is a diagram illustrating the adaptive linear scaling factor provided in an embodiment of the present invention.
[0086] Figure 7 The diagram shows the results of three improved deep binary neural networks provided in this embodiment of the invention. Vanilla represents the baseline network, and the curves of different colors represent the effects of combining the Channel Mixing (CS) module, the Attention Mechanism (SE) block, and the Adaptive Linear Scale Factor (ALSF) with Vanilla, respectively. Detailed Implementation
[0087] The present invention will now be described in further detail with reference to the accompanying drawings.
[0088] like Figure 1 As shown, the image classification method based on a deep binary neural network model includes the following steps:
[0089] Obtain the image to be classified;
[0090] The image to be classified is input into a preset deep binary neural network model to obtain the classification prediction result corresponding to the image to be classified through the deep binary neural network model.
[0091] The deep binary neural network model is obtained through the following steps:
[0092] Step 1: Improve the binary convolution module by fusing the improved binary convolution module with the remodeling activation module to reconstruct the structure of the deep binary neural network model.
[0093] Step 2: Based on the deep binary neural network model structure obtained in Step 1, add an attention mechanism module to each layer of binary convolution operation;
[0094] Step 3: Based on the deep binary neural network model structure obtained in Step 1, assign an adaptive linear scaling factor to the binary convolution operation through backpropagation, so that the scaling factor of the binary convolution kernel can be obtained through backpropagation training.
[0095] Step 4: Based on the deep binary neural network model structure obtained in Steps 1-3, build the baseline network of the deep binary neural network model and initialize the floating-point parameters of the baseline network.
[0096] Step 5: Train the deep binary neural network model obtained in step 4 using the Adam algorithm to obtain the deep binary neural network model.
[0097] like Figure 2 As shown, the image classification method based on a deep binary neural network model, after processing by different modules, yields the following feature maps:
[0098] The yellow line represents the real value propagating within the path, and the blue line represents the binary value. The first feature map is the real value representation of the input, the second feature map is the representation after the attention mechanism, the third feature map is the binary representation after re-activation, the fourth feature map is the representation after binary convolution, the fifth feature map is the binary representation after channel mixing, and the sixth feature map is the residual real value representation after adding the original input features.
[0099] The technical solution of the present invention will be further described below with reference to the embodiments.
[0100] Example
[0101] This invention conducted image classification experiments on the CIFAR10
[38] dataset, which consists of 60K images with a spatial resolution of 32×32, evenly distributed across 10 common categories in daily life, with a training set to test set ratio of 5:1. Data augmentation methods were also used in the experiment, including random cropping, image lighting, and random horizontal flipping for CIFAR10.
[0102] The specific process of step 1 is as follows: The module that reshapes activation changes the distribution of activation by replacing the traditional Sign function, as shown below:
[0103]
[0104]
[0105] Where x represents the real-valued activation before binarization, x b α is a binary value, β is a trainable coefficient that affects the slope of the negative half-axis, α and ξ are learnable offsets that reshape the distribution on the horizontal and vertical axes, and RSign and RPReLU functions are used in this invention.
[0106] like Figure 3 As shown, this invention proposes a structure that reduces computation and increases information exchange between different channels. First, grouped point convolutions are used instead of ordinary point convolutions, which greatly reduces the number of parameters. Two groups of point convolutions are designed to be operated in parallel. Finally, the channels are aligned by concatenation. One group of point convolutions is followed by a channel mixing module. By changing the order of different channels, image features can be fused and communicated between different channels. The other group of point convolutions is not followed by any operation to retain more feature information from the original channels. The new connection between these two different parts increases the diversity of feature maps and reduces computational complexity.
[0107] like Figure 4 As shown, the intermediate feature maps are visualized respectively. Figure 4 (a) The input is fed into the original point-wise convolution, and the output result is shown in Figure (b). Figure 4 (a) Input to the present invention, the output result is shown in Figure (c). The intermediate feature map information of the present invention is richer and more diverse.
[0108] The specific process of step 2 is as follows:
[0109] Attention mechanisms are used to capture the interdependencies between feature channels. By using a weight matrix, different weights are assigned to different positions in the image from the perspective of channels, more important feature information is obtained, and the importance of each feature channel is automatically learned.
[0110] For a given feature map in a certain layer of the network, the feature map is first subjected to global average pooling to generate a one-dimensional vector, so that each channel can be represented by a numerical value. Then, the vector is passed through two fully connected layers to generate the required weight information, where the weights are obtained through learning.
[0111] The generated weight vector and the original feature map are then weighted and assigned to obtain the final feature map, which has the same size as the original feature map. The attention mechanism assigns corresponding weights to each feature channel according to the importance of each feature channel, allowing the neural network to focus on the channel most relevant to the current task.
[0112] Therefore, the attention mechanism, also known as the channel attention mechanism, enhances useful feature channels, suppresses unimportant feature channels, and improves the network's sensitivity to channel features. Furthermore, the attention mechanism can be easily integrated into existing network structures, typically in the first and penultimate layers. Since binarization leads to the loss of a significant amount of information in each layer, it is crucial to focus on the useful feature information of different channels. Specific details are as follows... Figure 5 As shown.
[0113] Step 3 assigns an adaptive linear scaling factor that can be backpropagated, specifically:
[0114] w~α·bin(w)~(a·α+b)·bin(w)
[0115] Where w is the weight of the full-precision convolution kernel, b w α represents the weights for binarization, and α is the scaling factor.
[0116] like Figure 6 The diagram illustrates a three-layer network to explain how parameters are updated via backpropagation. x represents the network's input, h represents intermediate nodes, and y represents the network's output. The original weights α·bin(w) are replaced with improved weights (a·α+b)·bin(w). The linear adaptive scaling factor a·α+b is updated via gradient descent. Given a training dataset D, the framework's learning objective is to find w, a, and b. Assuming E is the network's loss, the backpropagation process is as follows:
[0117]
[0118] The partial derivatives of the variable loss can be derived using the chain rule:
[0119]
[0120] The partial derivatives output to the variable can also be derived using the chain rule:
[0121]
[0122]
[0123] It is important to note that this is not differentiable, therefore an approximation sign function is used to update the gradient, given:
[0124]
[0125]
[0126]
[0127] During training, the true numerical weights are retained, and backpropagation is performed using the chain rule of approximation functions. The weights of each layer are continuously updated according to the equation. The method of this invention is not only easy to guide, but also improves accuracy without increasing the amount of computation.
[0128] The specific process of step 4 is as follows: after integrating the improved parts of steps 1-3 above, a baseline network of the deep binary neural network model is built, and various parameters of the baseline network are initialized, namely the weight w and parameters a and b of the full-precision convolution kernel in step 3. An approximate gradient function for backpropagation of the binary network is selected to prepare for the binary neural network to be trained normally.
[0129] The approximate gradient function is:
[0130]
[0131] In step 5, the Adam algorithm is used for training, employing a method where the learning rate decreases with a cosine function scheduler. The Adam algorithm dynamically adjusts the learning rate based on the first and second estimates of the gradient.
[0132]
[0133]
[0134]
[0135] Where, m t v is the average value of the gradient at the first time step. t Let be the non-central variance value of the gradient at the second time step, where β1 is generally set to 0.9, β2 is generally set to 0.9999, and ε is generally set to 10⁻⁸.
[0136] Experimental Results and Analysis
[0137] For images with lower resolution in the CIFAR10 dataset, excessive downsampling can lead to excessive information loss. Therefore, in the original network, the stride of the first real-valued downsampling convolution is set to 1, and the new baseline is called "Vanilla". Ablation studies were conducted on Vanilla networks with different combinations of channel mixing CS module, attention mechanism SE module, and adaptive linear scaling factor ALSF on the CIFAR10 dataset, as shown in Table 1. The calculation results show that the present invention can achieve high computational accuracy. Figure 7 As shown, in the early stages of Vanilla network training, there are oscillations and instabilities. The three modules in this invention improve the training anomalies, making the training process smoother until convergence, and also improving the network accuracy. The accuracy of Vanilla+CS reaches 89.48%, which is 1.02% higher than Vanilla's accuracy; the accuracy of Vanilla+CS+SE reaches 90.07%, which is 1.59% higher than Vanilla's accuracy; and the accuracy of Vanilla+CS+SE+ALSF reaches 90.17%, which is 4.69% higher than Vanilla's accuracy.
[0138] Table 1. Vanilla and the CS module, SE block and ALSF of the present invention are respectively used as different combinations.
[0139]
[0140]
[0141] The evaluation results on the CIFAR10 dataset are shown in Table 2. Compared with manually designed binary neural networks such as XNOR-Net, this invention achieves similar accuracy but requires only 7% of the parameters. Compared with ReActNet, this invention not only significantly reduces the number of parameters but also improves the model's accuracy by 4.58%. This invention demonstrates strong performance on the small dataset CIFAR10. Moreover, the full-precision network requires 32 bits to store weights and activations, while the binary network only requires 1 bit. Therefore, when considering the equivalent parameter count, the binary network has higher computational efficiency.
[0142] Table 2 shows the error and parameter comparison between the present invention and state-of-the-art networks on the CIFAR10 dataset. The comparison focuses on networks with activation and weight binarization.
[0143] Table 2
[0144]
[0145]
[0146] Example 2
[0147] Image classification experiments were conducted on the ImageNet dataset, which contains 1.3 million training images and 50 K test images, roughly evenly distributed across 1K classes. The images are high-resolution images of varying sizes; before entering the network, they need to be preprocessed to uniformly convert them to a resolution of 224×224. Similar to most architecture search methods, to reduce the search and training time on large datasets and improve search efficiency, this invention first performs architecture search on CIFAR10, then transfers the optimal architecture to the ImageNet dataset for training, evaluating the algorithm's portability.
[0148] To ensure the comparability of the experiments, this invention also used data augmentation methods in the experiments. For ImageNet, label smoothing and lighting data augmentation operations were added.
[0149] Experimental Results and Analysis
[0150] CSA-Net was further evaluated on ImageNet, and the overall results are shown in Table 3. Furthermore, the computational cost of this invention is significantly lower than almost all binary neural network models. While Bi-Real-Net achieves an accuracy of 64.5%, this invention surpasses it by 0.85%. Compared to ReActNet, which has similar model parameters, the performance of this invention, although somewhat lower, is significantly reduced in computation. Therefore, the model proposed in this invention can greatly reduce the computational cost of binary networks, stably improve model performance on small datasets, and, more importantly, reduce computational cost, thus preventing performance degradation on large datasets like ImageNet. The lighter and more accurate network structure produced by this invention allows for a trade-off between complexity and performance when facing specific problems.
[0151] Table 3 compares the accuracy, computational cost, and parameters of the present invention with those of a binary network, specifically for networks with activation and weight binarization.
[0152] Table 3
[0153]
[0154] In summary, this invention utilizes multiple modules to reshape and fuse the deep binary neural network model, namely, reshaping the channel mixing module, adding an attention mechanism module, and assigning an adaptive linear scaling factor. This improves the deep binary neural network model, and the improved deep binary neural network model is used for image classification, reducing model complexity. At the same time, the training process has better stability, and the accuracy and speed of classification results on different datasets are improved to a certain extent.
[0155] Image classification systems based on deep binary neural network models include:
[0156] The image acquisition module is used to acquire images to be classified.
[0157] The image input module inputs the image to be classified into a preset deep binary neural network model so as to obtain the classification prediction result corresponding to the image to be classified through the deep binary neural network model.
[0158] The deep binary neural network model includes:
[0159] The attention mechanism module first enters the image, which dynamically adjusts the weights of different channels, then enhances the features of important channels and suppresses the features of unimportant channels to obtain a new feature map.
[0160] Reshape the activation module to incorporate the features obtained from the attention mechanism module. Figure 2 Value-based processing and dynamic compression produce different binary feature maps;
[0161] The binary convolution module is used to perform binary convolution operations on the binary feature maps obtained by the re-activation module, extract features, and output the feature maps after XOR operation.
[0162] The channel mixing module is used to mix feature information from different channels to obtain richer and more diverse feature maps.
[0163] Image classification devices based on deep binary neural network models include:
[0164] Memory: Used to store the computer program for the image classification method based on the deep binary neural network model;
[0165] Processor: Used to implement the image classification method based on a deep binary neural network model when executing the computer program.
[0166] A computer-readable storage medium storing a computer program that, when executed by a processor, enables an image classification method based on a deep binary neural network model.
Claims
1. An image classification method based on a deep binary neural network model, comprising the following steps: Obtain the image to be classified; The image to be classified is input into a preset deep binary neural network model to obtain the classification prediction result corresponding to the image to be classified through the deep binary neural network model. The deep binary neural network model is characterized by being obtained through the following steps: Step 1: Improve the binary convolution module by fusing the improved binary convolution module with the remodeling activation module to reconstruct the structure of the deep binary neural network model. Step 2: Based on the deep binary neural network model structure obtained in Step 1, add an attention mechanism module to each layer of binary convolution operation; Step 3: Based on the deep binary neural network model structure obtained in Step 1, assign an adaptive linear scaling factor for backpropagation to the binary convolution operation. Step 4: Based on the deep binary neural network model structure obtained in Steps 1-3, build the baseline network of the deep binary neural network model and initialize the floating-point parameters of the baseline network. Step 5: Train the deep binary neural network model obtained in step 4 using the Adam algorithm to obtain the deep binary neural network model. The specific process of step 1 is as follows: Step 1.1: Improve the two point convolutions in the binary convolution module to grouped point convolutions. One grouped point convolution operation is followed by a channel blending module. The channel blending module allows image features to be fused and communicated between different channels by changing the order of different channels. The other grouped point convolution operation is followed by nothing, which is used to retain more information from the original channels. Step 1.2: Use the Reshape Activation module to change the distribution of activation by replacing the Sign function, as shown below: wherein, denotes the real-valued activations before binarization, are their binarized values, is a trainable coefficient that influences the slope of the negative half-axis, and are learnable offsets that reshape the distribution on the horizontal and vertical axes; Step 3 assigns an adaptive linear scaling factor through backpropagation, specifically as follows: wherein, is a weight of a full-precision convolution kernel, is a binarized weight, is a scale factor.
2. The image classification method based on a deep binary neural network model according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1: By using a weight matrix, different weights are assigned to different positions in the image from the perspective of channels, and more important feature information is obtained. The importance of each feature channel is automatically learned through the weight matrix. Step 2.2: For a given feature map in a certain layer of the network, first perform global average pooling on the feature map to generate a one-dimensional vector, with each channel represented by a numerical value. Then, pass the vector through two fully connected layers to generate the required weight matrix. Step 2.3: Then, assign weights to the weight matrix generated in Step 2.2 and the original feature map to obtain the final feature map, which has the same size as the original feature map. The attention mechanism assigns corresponding weights to each feature channel according to the importance of each feature channel, allowing the neural network to focus on the channel most relevant to the current task.
3. The image classification method based on a deep binary neural network model according to claim 1, characterized in that, The backpropagation of the adaptive linear scaling factor in step 3 is specifically as follows: The node value of layer N-1 of the network. For the node values of layer N in the network, For the node values of layer N+1 of the network (N>0), use the improved weights. Replace the original weights The linear adaptive scaling factor is updated via gradient descent. Given a training dataset D, assume It is the loss of the N+2 layers of the network. , , and For parameters from layer N-1 to layer N, , , and For parameters from layer N to layer N+1, the backpropagation process is as follows: The partial derivatives of the variable loss develop through the chain rule: The partial derivatives output to the variable are developed using the chain rule: Use an approximation sign function to update the gradient: During training, the true numerical weights are retained, and backpropagation is performed using the chain rule of the approximation function to approximate the approximation function, continuously updating the weights of each layer according to the equation.
4. The image classification method based on a deep binary neural network model according to claim 1, characterized in that, The specific process of step 4 is as follows: Step 4.1: Initialize the parameters of the baseline network, i.e., the weights of the full-precision convolutional kernels in Step 3. and parameters and , Step 4.2: Select an approximate gradient function for backpropagation in the binary network to update the weights of the deep binary neural network model, preparing the binary neural network for normal training. The approximate gradient function is as follows: 。 5. The image classification method based on a deep binary neural network model according to claim 1, characterized in that, The specific process of step 5 is as follows: The Adam algorithm is used for training, employing a learning rate decreasing with a cosine function scheduler. The Adam algorithm dynamically adjusts the learning rate based on the first and second estimates of the gradient. in, The gradient is the average value at the first time step. Let be the non-central variance of the gradient at the second time step, where Set it to 0.
9. Set it to 0.9999. Set as .
6. An image classification system based on a deep binary neural network model, used to implement the method of claim 1, characterized in that, include: The image acquisition module is used to acquire images to be classified. The image input module inputs the image to be classified into a preset deep binary neural network model so as to obtain the classification prediction result corresponding to the image to be classified through the deep binary neural network model. The deep binary neural network model includes: The attention mechanism module first enters the image, which is used to dynamically adjust the weights of different channels. Then, it enhances the features of important channels and suppresses the features of unimportant channels to obtain a new feature map. The re-activation module is used to binarize the feature maps obtained by the attention mechanism module, dynamically compress them, and generate different binary feature maps. The binary convolution module is used to perform binary convolution operations on the binary feature maps obtained by the re-activation module, extract features, and output the feature maps after XOR operation. The channel mixing module is used to mix feature information from different channels to obtain richer and more diverse feature maps.
7. An image classification device based on a deep binary neural network model, characterized in that, include: Memory: for storing the computer program of the image classification method based on the deep binary neural network model as described in any one of claims 1 to 5; Processor: for implementing an image classification method based on a deep binary neural network model when executing the computer program of any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the image classification method based on a deep binary neural network model as described in any one of claims 1 to 5.