A training method, device and computer equipment for a binary neural network model

Abstract

The present invention relates to a training method, device and computer equipment for a binary neural network model. The method comprises: introducing enhanced thermometer coding into the input layer of a small sample image recognition model, adopting a convolution layer without batch normalization, and designing a binary neural network model. Inputting a picture to be predicted into the binary neural network model to extract features to be predicted, performing a one-stage training on the features to be predicted using the original activation function and weights, and then performing a two-stage training on the prediction picture input into the trained binary neural network model using the binarized original activation function and weights to obtain an optimal binary neural network model. The optimal binary neural network model is deployed into a C language inference framework to complete system module development and obtain edge device adaptation information. Based on this, the register transfer level process of the target edge device is designed. The use of this method can improve the efficiency and lightweight performance of deep learning of resource-constrained edge devices.

Description

Technical Field

[0001] The present application relates to the field of edge computing and deep learning technology, and in particular to a training method, apparatus, and computer equipment for a binary neural network model. Background Art

[0002] Deep convolutional neural networks (CNNs) have become a key research area in the field of artificial intelligence due to their powerful learning capabilities and broad application prospects. However, traditional deep CNN models, due to their complex structure and large number of parameters, typically require significant storage and computing resources. For example, the VGG-16 network contains approximately 140 million 32-bit floating-point parameters, requiring over 500MB of storage, and each forward pass requires approximately 1.6×10^10 floating-point operations. This enormous computational and storage demand makes deep CNNs heavily dependent on high-performance hardware (such as GPUs), making them difficult to run efficiently on edge devices with limited computing resources, such as mobile devices and IoT devices.

[0003] To address this issue, researchers have proposed a variety of model compression and optimization techniques, among which binary neural networks (BNNs) are a particularly promising solution. By quantizing weights and activation values ​​into binary form (typically +1 and -1), BNNs significantly reduce the model's computational complexity and storage requirements. For example, compared to full-precision networks, BNNs can reduce model size by 32 times while optimizing convolution operations into efficient bitwise operations (such as XNOR and popcount), significantly improving computational efficiency.

[0004] Although BNN has made significant progress in model compression and acceleration, it still faces some challenges in practical applications. Existing BNN technology still has the following key issues:

[0005] 1. Input information loss: ResNet, the most widely used baseline network for binarized neural networks, does not achieve full binarization in the true sense; the first convolution layer still uses full-precision convolution. Directly binarizing the input layer results in insufficient representation of shallow features, resulting in a significant loss of input information and a significant decrease in network accuracy.

[0006] 2. Edge distortion: However, since weights and activation values ​​are usually binarized to 1 and -1 during binary convolution, 1 is used to represent 1 and 0 is used to represent -1 when storing weights. The traditional zero-padding strategy introduces -1 value padding in the binary convolution, resulting in edge feature distortion.

[0007] 3. Unstable training: BNN relies on the Batch Normalization (BN) layer for stable training. However, the BN layer requires full-precision calculations, which introduces full-precision floating-point calculations. The computational cost is very high and is not friendly to low-precision hardware. In addition, it is usually implemented using non-binary parameters, which leaves obstacles for the effective implementation of BNN training.

[0008] 4. Poor hardware adaptability: Existing BNN deployment solutions lack end-to-end optimization for low-power processors, making it difficult to balance model size and computational efficiency.

[0009] Therefore, the design and deployment of BNNs for resource-constrained edge devices has important research significance and application value. To address the above issues, this application proposes a lightweight VS-BNN design and deployment method that achieves a balance between accuracy and performance through enhanced thermometer encoding, batch normalization-free (BN-Free) layer training optimization, and a hardware-friendly inference framework. Summary of the Invention

[0010] Based on this, it is necessary to provide a training method, device and computer equipment for a binary neural network model that can achieve high precision, low computational complexity, low storage requirements and is hardware-friendly to address the above technical problems.

[0011] A training method for a binarized neural network model, the method comprising:

[0012] Enhanced thermometer coding is introduced into the input layer of the small sample image recognition model, and a binarized neural network model is designed using convolutional layers without batch normalization.

[0013] Input the image to be predicted into the binary neural network model to extract the features to be predicted, and use the original activation function and weights to perform one-stage training on the features to be predicted to obtain the first predicted image and the trained binary neural network model.

[0014] The first prediction image input into the trained binary neural network model is trained in two stages using the binarized original activation function and weights to obtain the weight matrix, model parameters and optimal binarized neural network model.

[0015] Deploy the optimal binary neural network model into the C language inference framework, complete the system module development, and obtain edge device adaptation information.

[0016] Design the register transfer level flow of the target edge device based on the edge device adaptation information.

[0017] A training device for a binary neural network model, comprising:

[0018] A model design module is used to introduce enhanced thermometer coding into the input layer of the small-shot image recognition model, use convolutional layers without batch normalization, and design a binarized neural network model.

[0019] The one-stage training module is used to input the image to be predicted into the binary neural network model to extract the features to be predicted, and use the original activation function and weights to perform one-stage training on the features to be predicted to obtain the first predicted image and the trained binary neural network model.

[0020] The two-stage training module is used to perform two-stage training on the first prediction image input into the trained binary neural network model using the binarized original activation function and the weight to obtain the weight matrix, model parameters and the optimal binarized neural network model.

[0021] The edge device configuration module is used to deploy the optimal binary neural network model into the C language inference framework, complete the system module development, and obtain edge device adaptation information.

[0022] The deployment module is used to design the register transfer level process of the target edge device according to the edge device adaptation information.

[0023] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0024] Enhanced thermometer coding is introduced into the input layer of the small sample image recognition model, and a binarized neural network model is designed using convolutional layers without batch normalization.

[0025] Input the image to be predicted into the binary neural network model to extract the features to be predicted, and use the original activation function and weights to perform one-stage training on the features to be predicted to obtain the first predicted image and the trained binary neural network model.

[0026] The first prediction image input into the trained binary neural network model is trained in two stages using the binarized original activation function and weights to obtain the weight matrix, model parameters and optimal binarized neural network model.

[0027] Deploy the optimal binary neural network model into the C language inference framework, complete the system module development, and obtain edge device adaptation information.

[0028] Design the register transfer level flow of the target edge device based on the edge device adaptation information.

[0029] The beneficial effects of this program are as follows:

[0030] The training method, apparatus, and computer equipment for the aforementioned binary neural network model utilize enhanced thermometer coding technology to convert input features into a high-dimensional sparse representation, significantly reducing input information loss. While traditional binary coding uses only a single bit to represent features, enhanced thermometer coding utilizes multiple bits, preserving more detail from the original data and improving feature representation. The even-odd padding strategy effectively mitigates edge distortion by applying special processing to the edges of the input image. While traditional padding methods can blur edge information, even-odd padding differentiates the padding based on the parity of pixel values, preserving the clarity of edge features. The batch normalization-free (BN-Free) design eliminates the computationally expensive batch normalization layer and instead utilizes weight scale normalization (SWS) technology to normalize the weight matrix. This maintains model training stability without batch normalization, reducing computational overhead and making it particularly suitable for resource-constrained edge devices. Furthermore, the training implementation utilizes a two-stage training strategy consisting of real-valued weight pre-training followed by binary weight fine-tuning. In the first stage, training uses the original activation functions and weights, leveraging the strong expressiveness of real-valued parameters to capture complex image features and provide a solid foundation for the model. In the second stage, the activation functions and weights are binarized, significantly reducing model storage and computational requirements. Adaptive Gradient Clipping (AGC) technology dynamically adjusts the gradient range to effectively address training instability. When gradients fluctuate drastically, AGC automatically clips the gradient to prevent gradient explosion; when gradients are small, it maintains normal gradient updates to ensure model convergence. Finally, at the hardware adaptation level, the optimized binarized neural network model is deployed to a C language inference framework, leveraging C's high performance to optimize model execution efficiency. A register transfer level (RTL) flow is designed based on edge device adaptation information, customized and optimized for the target hardware architecture. For example, data transfer paths and computational order are adjusted to the memory access patterns and compute unit structures of specific edge devices, ensuring a perfect match between model operations and hardware resources.

[0031] In summary, this method improves input quality through enhanced thermometer encoding and parity padding, ensures training stability through batch-free normalization and weight scale standardization, balances accuracy and performance through real-valued pre-training and binary fine-tuning, and finally achieves hardware-friendly deployment through a C language inference framework and RTL process design, providing an efficient, lightweight, and high-performance solution for deep learning applications on edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A network architecture diagram of a training method for a binarized neural network model in one embodiment;

[0033] Figure 2 1 is a flow chart of a training method for a binarized neural network model in one embodiment;

[0034] Figure 3 A schematic diagram of a development process for implementing a VS-BNN reasoning framework using C language in one embodiment;

[0035] Figure 4 A schematic diagram of the C language program execution flow for VS-BNN in one embodiment;

[0036] Figure 5 is a structural block diagram of a training device for a binarized neural network model in one embodiment;

[0037] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0039] The present invention provides a training method for a binary neural network model, which can be applied to Figure 1 The network architecture shown in Figure 1 uses a lightweight binarized neural network (VS-BNN) design. The VS-BNN model is optimized based on the VGG-Small (small sample image recognition model) architecture and consists of six convolutional layers and one fully connected layer. The input image size is 3×32×32, and the output has 10 classifications. The detailed structure of the model is shown in Table 1:

[0040] Table 1 VS-BNN network structure

[0041]

[0042] In one embodiment, Figure 2 As shown in the figure, a training method for a binary neural network model is provided. Figure 1 The network architecture in the example is used to illustrate the following steps:

[0043] Step 202: Introduce enhanced thermometer coding into the input layer of the small sample image recognition model, adopt a convolutional layer without batch normalization, and design a binarized neural network model.

[0044] Specifically, enhanced thermometer coding is introduced into the input layer of the small-sample image recognition model. The input binarization layer implements enhanced thermometer coding, combined with LBP and thermometer coding. A 32×32 RGB image is fed into the enhanced thermometer coding module to generate 48-channel binary features. The encoding process involves dynamic resolution thermometer coding (R=32) and LBP coding (3×3 neighborhood window), ultimately concatenating the resulting 48×32×32 feature map.

[0045] Furthermore, the Bconv_3×3 convolution layer implements: padding according to the even-odd padding strategy (Formula 1); weight scale normalization (SWS) is performed to normalize the weight of each convolution kernel according to (Formula 2) to eliminate distribution offset; adaptive gradient clipping (AGC) is performed to dynamically clip the gradient according to the ratio of the gradient to the weight norm (Formula 3):

[0046] (1)

[0047] in, is the fill value, is the channel index, (h, w) is the spatial coordinate; if the value of c+h+w is odd, the padding is 1, otherwise the padding is 0;

[0048] (2)

[0049] in, is the standard weight scale, is a fixed scalar used to keep the variance unchanged, and its value depends on the activation function used. is the mean of each channel, is the original weight before normalization, is the number of channels, is the variance. And adaptive gradient clipping is used to adjust the gradient norm:

[0050] (3)

[0051] in, is the gradient matrix of the lth layer, is the clipping threshold, a hyperparameter that controls the degree of gradient clipping. is the weight matrix of the lth layer, is the index of the neural network layer, is the Frobenius norm.

[0052] Furthermore, the BN-Free dimensionality reduction convolution layer and the BN-Free convolution layer are implemented: Rsign (Formula 4) is used as the binarization function and RPreLU (Formula 5) is used as the activation function; the fully connected layer is implemented, and the 512×4×4 feature map is flattened after adaptive maximum pooling to output a 10-dimensional classification result;

[0053] (4)

[0054] (5)

[0055] in, is the activation value after binarization, is the Rsign function, is the real-valued input of the Rsign function h on the i-th channel, For binarization, is a real value, is the learnable coefficient for controlling the threshold, is the input of the RPreLU function f on the i-th channel, is the RPReLU (Randomized Parametric Rectified Linear Unit) function, is the learnable coefficient that controls the slope of the negative part, and A learnable shift for mobile distributions.

[0056] Step 204 , input the image to be predicted into the binary neural network model to extract features to be predicted, and perform a one-stage training on the features to be predicted using the original activation function and weights to obtain a first predicted image and a trained binary neural network model.

[0057] Specifically, we use the CIFAR-10 dataset for training and testing. This dataset contains 60,000 32×32 color images divided into 10 categories, with 6,000 images in each category, of which 50,000 images are used as training sets and 10,000 images are used as test sets.

[0058] Step 206 , using the binarized original activation function and weights to perform two-stage training on the first prediction image input to the trained binarized neural network model, to obtain a weight matrix, model parameters, and an optimal binarized neural network model.

[0059] Specifically, a two-stage training strategy is adopted. The first stage uses binary activations and floating-point weights for training; the second stage binarizes the weights and continues training. Both stages use the Adam optimizer, with a linear decrease learning rate adjustment strategy.

[0060] Furthermore, the loss function adopts the distribution loss function (Formula 6):

[0061] (6)

[0062] in, is the distribution loss function, For categories, is the batch size, is the softmax output of the full-precision model, is a full-precision model, is the input image, It is a binary neural network.

[0063] The model parameters obtained from the two-stage training are preprocessed. The weights are binarized and compressed. The trained 32-bit floating-point weights are binarized to ±1 after weight scale normalization (SWS). These weights are then packaged into 8-bit groups and stored as uint8_t. The C header file weight_data.h is generated, containing the compressed weight matrix and network structure parameters, in preparation for subsequent deployment.

[0064] In step 208 , the optimal binary neural network model is deployed into the C language reasoning framework to complete the system module development and obtain the edge device adaptation information.

[0065] Specifically, we will complete the implementation of the VS-BNN inference framework in C language. The development route is as follows: Figure 3 As shown, the basic computational modules required for BNN inference are implemented, including the binarization function RSign, the activation function Q_RPReLU, the binarized convolution Bconv_3×3, the maximum pooling Max_pool, and the channel concatenation operation Concatenate. The network layer is modularized, with each layer of the VS-BNN model (input binarization layer, BN-Free convolution layer, fully connected layer, etc.) independently developed and implemented, and forward propagation is implemented through pointer chaining. The feature map output of each layer of the PyTorch model is compared with that of C language inference to verify the accuracy of the inference logic. The experimental results show that the expected data is consistent with the PyTorch model.

[0066] Step 210 : Design a register transfer level process of the target edge device according to the edge device adaptation information.

[0067] Specifically, the target edge device is adapted, and the storage capacity of the target edge device is modified according to the size of the C program implemented in step 208; the linker script (linker.ld) is reconstructed to divide the size of the code segment (.text) and the data segment (.data); the C program is compiled using the GNU Toolchain to generate a Verilog memory initialization file for hardware deployment.

[0068] We then performed RTL-level simulation using Synopsys VCS and loaded the generated Verilog memory initialization file. The simulation results were consistent with expectations. This included implementing the RTL design on the target edge device using Vivado 2019.3, including hardware design, clock configuration, reset signal setting, synthesis and implementation, and bitstream generation. The verified C program was burned into the target edge device, and the program's execution status was monitored using a serial debugging tool.

[0069] Through the above steps, the VS-BNN model was successfully deployed and verified on the target edge device. Through algorithm-hardware co-optimization, efficient deployment of lightweight BNNs was achieved. From model design and training strategy to hardware adaptation, each step was customized to the resource constraints of the edge device, ultimately achieving a balance between accuracy, storage, and computational efficiency.

[0070] In the aforementioned binary neural network model training method, enhanced thermometer coding technology, at the model design level, transforms input features into a high-dimensional sparse representation, significantly reducing input information loss. Traditional binary coding uses only a single bit to represent features, while enhanced thermometer coding uses multiple bits, preserving more detail from the original data and improving feature representation. The even-odd padding strategy effectively mitigates edge distortion by applying special processing to the edges of the input image. Traditional padding methods can blur edge information, while even-odd padding differentiates the padding based on the parity of pixel values, preserving the clarity of edge features. The batch normalization-free (BN-Free) design eliminates the computationally expensive batch normalization layer and instead utilizes weight scale normalization (SWS) technology to normalize the weight matrix. This maintains model training stability without batch normalization, reducing computational overhead and making it particularly suitable for resource-constrained edge devices. Secondly, at the training implementation level, a two-stage training strategy is employed: real-valued weight pre-training followed by binary weight fine-tuning. In the first stage, training uses the original activation functions and weights, leveraging the strong expressiveness of real-valued parameters to capture complex image features and provide a solid foundation for the model. In the second stage, the activation functions and weights are binarized, significantly reducing model storage and computational requirements. Adaptive Gradient Clipping (AGC) technology dynamically adjusts the gradient range to effectively address training instability. When gradients fluctuate drastically, AGC automatically clips the gradient to prevent gradient explosion; when gradients are small, it maintains normal gradient updates to ensure model convergence. Finally, at the hardware adaptation level, the optimized binarized neural network model is deployed to a C language inference framework, leveraging C's high performance to optimize model execution efficiency. A register transfer level (RTL) flow is designed based on edge device adaptation information, customized and optimized for the target hardware architecture. For example, data transfer paths and computational order are adjusted to the memory access patterns and compute unit structures of specific edge devices, ensuring a perfect match between model operations and hardware resources.

[0071] In summary, this method improves input quality through enhanced thermometer encoding and parity padding, ensures training stability through batch-free normalization and weight scale standardization, balances accuracy and performance through real-valued pre-training and binary fine-tuning, and finally achieves hardware-friendly deployment through a C language inference framework and RTL process design, providing an efficient, lightweight, and high-performance solution for deep learning applications on edge devices.

[0072] In one embodiment, the image to be predicted is input into a binary neural network model. After the pixel values ​​of the image to be predicted are thermometer-encoded and LBP-encoded respectively in the input layer, the two encoding results are concatenated based on the channel information to extract the features to be predicted. The features to be predicted are dynamically filled with an even-odd padding strategy in the convolutional layer without batch normalization:

[0073] ;

[0074] in, is the fill value, is the channel index, (h, w) is the spatial coordinate; if the value of c+h+w is odd, the padding is 1, otherwise the padding is 0. The original convolution weight is scaled normalized according to the channel to obtain the standard weight scale:

[0075] ;

[0076] in, is the standard weight scale, is a fixed scalar used to keep the variance constant, and its value depends on the activation function used. is the mean of each channel, is the original weight before normalization, is the number of channels, is the variance. And adaptive gradient clipping is used to adjust the gradient norm:

[0077] ;

[0078] in, is the gradient matrix of the lth layer, is the clipping threshold, a hyperparameter that controls the degree of gradient clipping. is the weight matrix of the lth layer, is the index of the neural network layer, is the Frobenius norm. The first stage features of the features to be predicted are extracted to obtain the first predicted image and the trained binary neural network model.

[0079] In one embodiment, the original activation function after binarization and the weighted activation function are used to improve the activation function:

[0080] ;

[0081] ;

[0082] in is the activation value after binarization, is the Rsign function, is the real-valued input of the Rsign function h on the i-th channel, For binarization, is a real value, is the learnable coefficient for controlling the threshold; is the input of the RPreLU function f on the i-th channel, is the RPreLU function, is the learnable coefficient that controls the slope of the negative part, and is a learnable shift of the mobile distribution. Based on the improved activation function, a distributed loss function is used to perform two-stage training on the first prediction image input to the trained binary neural network model to obtain the weight matrix, model parameters and optimal binary neural network model:

[0083] ;

[0084] in, is the distributed loss function, For categories, is the batch size, is the softmax output of the full-precision model, is a full-precision model, is the input image, It is a binary neural network.

[0085] In one embodiment, a C language inference framework includes a computational module and a network layer module. The binarization function, improved activation function, binarized convolution, maximum pooling, and channel connection operations of the optimal binarized neural network model are deployed to the computational module of the C language inference framework for feature inference. Each network layer of the optimal binarized neural network model is independently developed, and forward propagation is implemented by calling a pointer chain. This completes system module development, outputs a target prediction image, and obtains edge device adaptation information corresponding to the prediction process.

[0086] In one embodiment, the weight matrix is ​​normalized in terms of weight scale and stored according to standard units, and the weight matrix is ​​binarized and compressed to generate a C language header file.

[0087] In one embodiment, an automated software suite is used to configure hardware, clocks, reset signals, synthesis and implementation, and bitstream generation information on a target edge device based on the edge device adaptation information. The C program developed after the system module is developed is burned to the target edge device. The program running status is monitored through a serial port debugging tool to complete the design of the register transfer level process.

[0088] In one embodiment, Figure 3 As shown, a C language inference program running process for VS-BNN is provided. The specific steps are as follows:

[0089] 1. Design a lightweight BNN model, VS-BNN, which can significantly reduce computational complexity and storage requirements while maintaining high accuracy. The network design includes:

[0090] (1) Optimization based on VGG-Small to reduce the number of parameters and computational complexity: Due to the limited hardware resources of edge devices, it is necessary to design a suitable and lightweight network structure. The most widely used baseline network for binary neural networks is ResNet, etc., which has a large network scale. In the experiment, the present invention mainly verifies the accuracy performance of the binary neural network BNN on the small-scale data set CIFAR-10. Therefore, the binary neural network architecture of the present invention is mainly optimized based on VGG-Small (small sample image recognition model) to simplify the network structure.

[0091] (2) Enhanced thermometer coding (combining LBP coding and thermometer coding) technology is introduced to improve the expressiveness of input features, reduce the information loss problem caused by insufficient input channels, and realize the binarization of the input layer: the input image is fused through thermometer coding and LBP coding to generate high-dimensional binary features. The specific steps include:

[0092] Step 1: Perform thermometer encoding on the pixel values. Quantize the input pixel values ​​into a binary vector of length L / R (L=255, R=32). For example, the pixel value 109 is mapped to 1,1,1,0,0,0,0,0.

[0093] Step 2: Perform LBP encoding on the pixel value, centering on a 3×3 neighborhood window, and compare the surrounding pixels to generate an 8-bit binary code;

[0094] Step 3: Perform feature concatenation and concatenate the two encoding results by channel to generate 48-channel input features. The number of channels is reduced by 50% compared with the traditional method while retaining detailed information.

[0095] (3) An even-odd padding strategy is used in binary convolution to alleviate the edge distortion problem in binary convolution: the padding value is dynamically generated according to the parity of the channel index (c) and the spatial coordinates (h, w) (Formula 1) to avoid edge feature distortion caused by zero padding;

[0096] (1)

[0097] (4) Add a batch normalization-free (BN-Free) design, remove the BN layer in VS-BNN to reduce the amount of floating-point calculations, and introduce the following technical replacements to maintain training stability:

[0098] ① Scaled Weight Standardization (SWS): The convolution weights are normalized by mean and variance per channel (Formula 2) to correct the shift in activation distribution;

[0099] (2)

[0100] Adaptive Gradient Clipping (AGC): Dynamically adjusts the gradient norm (Formula 3) to prevent gradient explosion during training.

[0101] (3)

[0102] 2. Using efficient BNN training methods, the classification accuracy of VS-BNN on the small-scale CIFAR-10 dataset is close to that of the full-precision network. The methods used include:

[0103] (1) A two-stage training strategy is adopted. The first stage of training uses binary activations and floating-point weights. The second stage of training inherits the weights of the first stage and uses binary activations and weights to reduce the difficulty of training binary neural networks directly from scratch.

[0104] (2) Using the distribution loss function (Formula 4) to force the output distribution of the binary network to approach the distribution of the full-precision network to improve the generalization ability of the model;

[0105] (4)

[0106] (3) Using improved activation functions such as RSign (Formula 5) and RPReLU (Formula 6) to enhance the model's ability to fit nonlinear features;

[0107] (5)

[0108] (6)

[0109] 3. Propose a complete technical link from model training to hardware deployment, and establish a seamless connection from PyTorch model training to edge device deployment, including:

[0110] (1) Complete the two-stage training of VS-BNN (real-valued weight pre-training + binary weight fine-tuning) in the PyTorch framework, and use the distribution loss function to improve the output distribution matching degree of the binary neural network;

[0111] (2) VS-BNN reasoning logic is implemented in C language, completing model parameter preprocessing (binarization, compression and packaging) and network layer modular development, supporting weight quantization compression and buffer reuse mechanisms. By packaging model parameters into binary form and storing them as header files, the storage requirements of the model are significantly reduced.

[0112] Complete the deployment of VS-BNN on the target edge device and optimize the hardware characteristics of the edge device, including expanding storage capacity and optimizing program memory layout. Through these optimizations, ensure that the model can run efficiently on the target edge device.

[0113] It should be understood that although Figure 2-Figure 4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2-Figure 4 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0114] In one embodiment, Figure 5 As shown, a training device for a binary neural network model is provided, including: a model design module 502, a first-stage training module 504, a second-stage training module 506, an edge device configuration module 508 and a deployment module 510, wherein:

[0115] The model design module 502 is used to introduce enhanced thermometer coding into the input layer of the small sample image recognition model, adopt a convolutional layer without batch normalization, and design a binary neural network model.

[0116] The one-stage training module 504 is used to input the image to be predicted into the binary neural network model to extract the features to be predicted, and use the original activation function and weights to perform one-stage training on the features to be predicted to obtain the first predicted image and the trained binary neural network model.

[0117] The two-stage training module 506 is used to perform two-stage training on the first prediction image input into the trained binary neural network model using the binarized original activation function and the weight to obtain the weight matrix, model parameters and the optimal binarized neural network model.

[0118] The edge device configuration module 508 is used to deploy the optimal binary neural network model into the C language reasoning framework, complete the system module development, and obtain the edge device adaptation information.

[0119] The deployment module 510 is configured to design a register transfer level process of a target edge device according to the edge device adaptation information.

[0120] Regarding the specific definition of a training device for a binary neural network model, please refer to the definition of a training method for a binary neural network model above, which will not be repeated here. The various modules in the above-mentioned training device for a binary neural network model can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0121] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a training method for a binary neural network model is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0122] Those skilled in the art will understand that Figure 5-Figure 6 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0123] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0124] Enhanced thermometer coding is introduced into the input layer of the small sample image recognition model, and a binarized neural network model is designed using convolutional layers without batch normalization.

[0125] Input the image to be predicted into the binary neural network model to extract the features to be predicted, and use the original activation function and weights to perform one-stage training on the features to be predicted to obtain the first predicted image and the trained binary neural network model.

[0126] The first prediction image input into the trained binary neural network model is trained in two stages using the binarized original activation function and weights to obtain the weight matrix, model parameters and optimal binarized neural network model.

[0127] Deploy the optimal binary neural network model into the C language inference framework, complete the system module development, and obtain edge device adaptation information.

[0128] Design the register transfer level flow of the target edge device based on the edge device adaptation information.

[0129] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware using a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0130] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A training method for a binary neural network model, characterized in that: The method comprises: We introduce enhanced thermometer coding into the input layer of the small sample image recognition model, use convolutional layers without batch normalization, and design a binary neural network model. Inputting the image to be predicted into the binary neural network model to extract features to be predicted, and performing a one-stage training on the features to be predicted using the original activation function and weights to obtain a first predicted image and a trained binary neural network model; Performing two-stage training on the first prediction image input into the trained binarized neural network model using the binarized original activation function and the weight to obtain a weight matrix, model parameters, and an optimal binarized neural network model; Deploy the optimal binarized neural network model into the C language reasoning framework, complete the system module development, and obtain edge device adaptation information; A register transfer level process of a target edge device is designed according to the edge device adaptation information.

2. The training method of a binary neural network model according to claim 1, characterized in that: Inputting the image to be predicted into the binary neural network model to extract the features to be predicted, and performing a one-stage training on the features to be predicted using the original activation function and weights to obtain a first predicted image and a trained binary neural network model, including: Input the image to be predicted into the binarized neural network model, perform thermometer coding and LBP coding on the pixel values ​​of the image to be predicted respectively through the input layer, splice the two coding results according to channel information, and extract the features to be predicted; The feature to be predicted uses an even-odd padding strategy to dynamically generate padding values ​​in the convolutional layer without batch normalization: in, is the fill value, is the channel index, (h, w) is the spatial coordinate; if the value of c+h+w is an odd number, the padding is 1, otherwise the padding is 0; And the original convolution weights are scaled and normalized according to the channel to obtain the standard weight scale: in, is the standard weight scale, is a fixed scalar used to keep the variance unchanged, and its value depends on the activation function used. is the mean of each channel, is the original weight before normalization, is the number of channels, is the variance; And use adaptive gradient clipping to adjust the gradient norm: in, is the gradient matrix of the lth layer, is the clipping threshold, which is used to control the degree of gradient clipping. is the weight matrix of the lth layer, is the index of the neural network layer, is the Frobenius norm; The first stage features of the features to be predicted are extracted to obtain a first predicted image and a trained binarized neural network model.

3. The training method of a binary neural network model according to claim 2, characterized in that: The first prediction image input into the trained binary neural network model is trained in two stages using the binarized original activation function and the weight to obtain a weight matrix, model parameters, and an optimal binarized neural network model, including: The original activation function after binarization and the weighted improved activation function are used: in, is the activation value after binarization, is the Rsign function, is the real-valued input of the Rsign function h on the i-th channel, For binarization, is a real value, is the learnable coefficient for controlling the threshold, is the input of the random parameterized rectified linear unit function f on the i-th channel, is the random parameterized rectified linear unit function, is the learnable coefficient that controls the slope of the negative part, and Learnable shift for mobile distribution; According to the improved activation function, the distributed loss function is used to perform two-stage training on the first prediction image input to the trained binary neural network model to obtain the weight matrix, model parameters and optimal binary neural network model: in, is the distributed loss function, For categories, is the batch size, is the softmax output of the full-precision model, is a full-precision model, is the input image, It is a binary neural network.

4. The training method of a binary neural network model according to any one of claims 1 to 3, characterized in that: The C language reasoning framework includes a computing module and a network layer module; Deploy the optimal binarized neural network model into the C language inference framework, complete system module development, and obtain edge device adaptation information, including: The binarization function, improved activation function, binarization convolution, maximum pooling and channel connection operations of the optimal binarization neural network model are deployed to the computing module of the C language inference framework for feature inference, and each network layer of the optimal binarization neural network model is independently developed. The forward propagation is realized by calling the pointer chain to complete the system module development, output the target prediction image, and obtain the edge device adaptation information corresponding to the prediction process.

5. The training method of a binary neural network model according to claim 4, characterized in that: Before deploying the optimal binarized neural network model into the C language reasoning framework, completing the system module development, and obtaining the edge device adaptation information, the following steps are included: The weight matrix is ​​normalized in terms of weight scale and stored according to standard units, and the weight matrix is ​​binarized and compressed to generate a C language header file.

6. The training method of a binary neural network model according to claim 5, characterized in that: Designing a register transfer level process of a target edge device according to the edge device adaptation information includes: Through the automated software suite, the hardware, clock, reset signal, synthesis and implementation, and bit stream generation information are configured on the target edge device according to the edge device adaptation information, and the C program obtained after the system module development is burned to the target edge device. The program running status is monitored through the serial port debugging tool to complete the design of the register transfer level process.

7. A training device for a binary neural network model, characterized in that: The device comprises: A model design module, which introduces enhanced thermometer encoding into the input layer of a small-shot image recognition model, uses convolutional layers without batch normalization, and designs a binarized neural network model. A one-stage training module is used to input the image to be predicted into the binarized neural network model to extract the features to be predicted, and perform one-stage training on the features to be predicted using the original activation function and weights to obtain a first predicted image and a trained binarized neural network model; A two-stage training module is used to perform two-stage training on the first prediction image input into the trained binary neural network model using the binarized original activation function and the weight to obtain a weight matrix, model parameters and an optimal binarized neural network model; An edge device configuration module is used to deploy the optimal binarized neural network model into the C language reasoning framework, complete system module development, and obtain edge device adaptation information; A deployment module is used to design a register transfer level process of a target edge device according to the edge device adaptation information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images