Low-power convolutional neural network hardware acceleration method based on stochastic computing

Abstract

This invention discloses a low-power convolutional neural network hardware acceleration method based on random computation. First, a convolutional neural network model is constructed and trained. Then, during the inference phase, the trained model is deployed to a hardware computing unit, and random computation is used to partially or completely replace the convolutional computations in the model. The random computation includes three stages: encoding, computation, and decoding. In the encoding stage, the input feature map is scaled, and the convolutional kernel is quantized. The gray values ​​of the convolutional kernel and each point in the input feature map are normalized into probability values, and an encoding sequence is generated for each probability value. In the computation stage, the two encoding sequences of the convolutional kernel and corresponding points in the input feature map are subjected to a traversal AND operation to generate an AND operation sequence. In the decoding stage, the AND operation sequence is decoded to obtain the computation result. Without significantly affecting model performance, this method reduces computational complexity and hardware resource consumption, making it suitable for scenarios with limited power consumption and resources.

Description

Technical Field

[0001] This invention belongs to the field of neural network image processing technology, and in particular to a low-power convolutional neural network hardware acceleration method based on random computation. Background Technology

[0002] Convolutional Neural Networks (CNNs) are deep learning models inspired by the perceptual mechanisms of the biological visual cortex. They can automatically learn abstract features from low to high levels from grid-like input data (such as images, speech spectra, and remote sensing data), possessing extremely strong feature representation capabilities and generalization performance. Therefore, they have become one of the core technologies in pattern recognition and intelligent processing within the field of artificial intelligence. Convolution, as the core of CNN technology, solves the pain points of traditional networks in processing grid-like data through its logic of local perception and weight sharing. It also endows CNNs with the ability to automatically extract hierarchical features, which is the fundamental guarantee for the high efficiency, accuracy, and strong generalization of CNNs. Furthermore, it supports CNNs as a core technology in fields such as computer vision, image processing, and medical image analysis.

[0003] However, with the rapid development and progress of convolutional neural networks (CNNs), their model parameters and computational load have exploded. Traditional CPUs can no longer meet their high computing power requirements, thus posing challenges and limitations to the application of deep learning. Specifically, in scenarios with high response efficiency requirements, such as autonomous driving, facial recognition, and intelligent monitoring, massive amounts of image data need to be processed in millisecond-level real-time. However, the current CPU's computing power output is far from meeting the speed threshold for image processing in such scenarios. In special scenarios such as on-board remote sensing processing platforms, industrial edge terminals, and portable smart devices, the devices are limited by factors such as power supply conditions, size specifications, and operating environment, placing extreme demands on power consumption control. However, the core convolution operation of CNNs is essentially a large number of inner product operations, and a large amount of numerical calculation will inevitably generate high power consumption. This will not only significantly shorten the device's battery life but may also cause severe overheating and decreased operational stability due to excessive power consumption, greatly limiting the application of neural networks in low-power scenarios. Therefore, controlling the low-power, low-latency operation of neural networks while maintaining or even improving the high accuracy of image processing has become the core research direction and key breakthrough goal in the current field of neural network image processing.

[0004] Existing hardware acceleration methods for convolutional neural networks primarily focus on three major hardware platforms: GPUs (Graphics Processing Units), ASICs (Application-Specific Integrated Circuits), and FPGAs (Field-Programmable Gate Arrays). Acceleration is mostly achieved through the following two approaches: One approach is to accelerate the process by reducing model complexity through model compression techniques such as pruning and quantization. This is mainly achieved by removing redundant parameters from the model (pruning) and reducing the data precision of weight parameters (quantization). While ensuring that the decline in the recognition accuracy of the CNN model is within an acceptable range, the model complexity is significantly reduced, thereby improving the image processing speed and reducing the power consumption during hardware operation.

[0005] Another approach is to leverage the strong parallel processing capabilities of hardware platforms to improve computational efficiency. GPUs, ASICs, and FPGAs all possess strong parallel computing capabilities and resources. By mapping highly parallel computational tasks such as convolution and pooling in CNNs to the parallel computing units of these hardware platforms, the parallel execution efficiency of computational instructions can be significantly improved, thereby increasing the model processing speed.

[0006] However, both of these methods have obvious limitations. The acceleration effect of model compression technology depends on the degree of compression, but excessive compression will lead to a sharp decline in model performance, making it difficult to meet the needs of application scenarios with extremely high processing speed requirements, such as autonomous driving and facial recognition. While the strong parallel computing capabilities of the hardware platform can avoid the problem of a sharp decline in model accuracy, the high-density operation of its parallel computing units will generate significant power consumption, leading to a surge in power consumption of the hardware platform, which is difficult to meet the needs of power-sensitive application scenarios such as on-board platforms and edge devices. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to propose a low-power convolutional neural network hardware acceleration method based on random computation.

[0008] The present invention solves the aforementioned technical problem by adopting the following technical solution: A low-power convolutional neural network hardware acceleration method based on random computation, characterized by comprising the following steps: Step 1: Build a convolutional neural network model and train the model; The second step is to deploy the trained model to the hardware computing unit during the inference phase, and use random computation to partially or completely replace the convolutional computations in the model. The random computation includes three stages: encoding, computation, and decoding. In the encoding stage, the input feature map is scaled, and the convolution kernel is quantized. The grayscale values ​​of the convolution kernel and each point in the input feature map are normalized to a probability value, the magnitude of which is... , Indicates the length of the encoded sequence. Take a positive integer; Generate a coding sequence for each probability value. First, generate an initial coding sequence based on the probability values, ensuring that 1s are evenly distributed throughout the initial coding sequence; then shift the initial coding sequence to the right. The bits are used to obtain the encoded sequence corresponding to the probability value; During the computation phase, the two encoding sequences of the convolution kernel and the corresponding points of the input feature map are used as the upper encoding sequence and the lower encoding sequence. The first bit of the upper encoding sequence and the lower encoding sequence are ANDed. The lower encoding sequence automatically moves to the next bit. The value of the current bit of the lower encoding sequence determines whether the upper encoding sequence should move to the next bit. If the value of the current bit of the lower encoding sequence is 1, the upper encoding sequence moves to the next bit, and the next bit of the upper encoding sequence is ANDed with the current bit of the lower encoding sequence. If the value of the current bit of the lower encoding sequence is 0, the upper encoding sequence stays at the current bit, and the current bits of the upper and lower encoding sequences are ANDed. The above process is repeated to traverse all bits of the lower encoding sequence and generate the AND operation sequence. In the decoding stage, the AND operation sequence is decoded to obtain the calculation result.

[0009] Furthermore, during the encoding phase, the probability values ​​0 and 1 are set to 0. As a base probability value, an encoding sequence is generated for the base probability value; if the current probability value is the sum of multiple base probability values, then the encoding sequences of multiple base probability values ​​are ORed by bit to obtain the encoding sequence of the current probability value.

[0010] Compared with the prior art, the present invention has the following beneficial effects: 1. Reduced computational complexity and hardware resource consumption. This invention maps the multiplication and addition operations of convolution to logical operations and statistical counting operations based on deterministic random computation coding. It utilizes simple logic units such as AND gates, OR gates, and counters in hardware resources to complete the multiplication and addition operations, effectively reducing the large-scale requirement for multipliers and adders in the hardware implementation process, thereby reducing the chip area and power consumption. It is suitable for embedded or edge computing platforms with limited power consumption and resources.

[0011] 2. It has high detection accuracy. This invention mainly optimizes the algorithm by adopting a differentiated, hierarchical random computation replacement strategy to replace the traditional convolution operation in the original model with random computation, thereby improving the overall computational efficiency without significantly affecting the model performance.

[0012] 3. It has good fault tolerance. The encoding sequence generated by random computation in this invention essentially determines the probability value based on the number of 1s in the sequence, and each bit in the sequence has the same weight. This differs from traditional binary weighted representation, where each bit has a different weight. In traditional binary representation, flipping a high-order bit often causes a significant deviation in the numerical result, thus significantly affecting computational accuracy and system stability. However, in the encoding sequence of this invention, flipping a single bit only causes a minor change in the number of "1s" in the encoding sequence. Furthermore, if multiple bits are flipped, they may cancel each other out, allowing the system to maintain high computational stability and robustness.

[0013] 4. It has good versatility and scalability. The hardware acceleration method based on random computation proposed in this invention does not depend on specific network models, parameter settings, and training datasets. It can adapt to network model structures of different sizes and complexities by adjusting the length and alternating positions of the randomly computed encoding sequence, and has good versatility and scalability. Attached Figure Description

[0014] Figure 1 This is an overall flowchart of the present invention; Figure 2 This is a schematic diagram of the traversal AND operation of the present invention; Figure 3 This is a structural diagram of the target detection model according to an embodiment of the present invention. Detailed Implementation

[0015] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of protection of this application.

[0016] This invention provides a low-power hardware acceleration method for convolutional neural networks based on random computation (hereinafter referred to as the method, see [link]). Figure 1-2 ), including the following steps: Step 1: Build a convolutional neural network model and train the model; The second step is to deploy the trained model to a hardware computing unit (such as an FPGA) during the inference phase, and use random computation to partially or completely replace the traditional convolutional computation in the model to achieve hardware acceleration of low-power convolutional neural networks. Randomized computation, as an emerging computing method, relies on probabilistic randomness to approximate precise computation results. By utilizing AND-OR circuit structures in hardware to implement traditional multiplication-accumulation operations, it can significantly reduce the computational complexity and resource consumption of deploying convolutional neural networks in hardware, achieving low-power hardware acceleration. The randomized computation includes three stages: encoding, computation, and decoding. Random computation encoding is mainly divided into two types: encoding based on random number sources and encoding based on deterministic distributions. Encoding based on random number sources uses random number generators such as linear feedback shift registers (LFSRs) to compare the input value with random numbers and generate a bit stream that satisfies the target probability distribution. Although this method is simple to implement and highly versatile, it is difficult to decode because it has multiple binary sequence representations for a single value. It cannot be decoded with a small amount of data and requires a large amount of statistical results to gradually approximate the true value, which limits the ability to accurately analyze and reproduce the data and makes it difficult to use in scenarios with high deterministic requirements.

[0017] Therefore, this invention employs a deterministic distribution-based random computation encoding method in the random computation encoding stage, encoding both the convolutional kernel and the input feature map separately, generating a binary encoding sequence for each point in both the convolutional kernel and the input feature map. First, the input feature map is scaled, and the convolutional kernel is quantized to -128 to 127. Then, the grayscale values ​​of the convolutional kernel and each point in the input feature map are normalized to a probability value between 0 and 1, with the probability value being... , Indicates the length of the encoded sequence. Take positive integers; generate convolution kernels and corresponding encoding sequences for points in the input feature map based on the probability values; specifically, each probability value corresponds to an encoding method, and the probability value represents the proportion of 1 in the encoding sequence, with 1 evenly distributed in the initial encoding sequence. The initial encoding sequence is shifted to generate the final encoding sequence, and the shift amount is determined by the probability value, i.e., the initial encoding sequence is shifted to the right. This process involves dividing the bits into 10 bits to obtain the final encoded sequence.

[0018] Set the probability value to 0, As the base probability value, only the base probability value needs to be encoded. The other probability values ​​can be obtained by adding multiple base probability values. If the current probability value is the sum of multiple base probability values, then the encoded sequences of multiple base probability values ​​are ORed by bit to obtain the encoded sequence of the current probability value. Taking a 16-bit encoded sequence as an example, the encoded sequences corresponding to different probability values ​​are shown in Table 1.

[0019] Table 1. Encoding table for 16-bit encoded sequences.

[0020] The length of the encoded sequence can be determined according to the actual situation. Since the probability values ​​are discrete, it is impossible to cover all probability values, so a certain error will be generated. The error decreases as the sequence length increases, so the longer the sequence length, the higher the accuracy.

[0021] By mapping the convolution kernel and each point of the input feature map to an encoding sequence with specific probability distribution characteristics, this encoding method can effectively reduce the correlation between different encoding sequences under a finite bit stream length, thereby mitigating the adverse effects of correlation on the accuracy of random computation and ensuring the statistical accuracy and stability of subsequent logical operation results. At the same time, this encoding method avoids dependence on high-quality random number sources, improves the controllability and reproducibility of the encoding process, and further enhances the reliability of the model at the hardware implementation level.

[0022] In the random computation phase, the two encoded sequences of the convolution kernel and corresponding points of the input feature map are subjected to AND operations according to preset rules to obtain the AND operation sequence; specifically, as follows: Figure 2 As shown, the two encoded sequences are designated as the upper and lower encoded sequences, respectively. First, the first bit of the upper and lower encoded sequences are ANDed. The lower encoded sequence automatically moves to the next bit. The upper encoded sequence, however, needs to determine whether to move to the next bit based on the value of the current bit of the lower encoded sequence. If the value of the current bit of the lower encoded sequence is 1, the upper encoded sequence moves to the next bit, and the next bit of the upper encoded sequence is ANDed with the current bit of the lower encoded sequence. If the value of the current bit of the lower encoded sequence is 0, the upper encoded sequence remains at the current bit, and the current bits of the upper and lower encoded sequences are ANDed. This process is repeated to traverse all bits of the lower encoded sequence, generating the AND operation sequence. The above-described traversal AND operation solves the problem in traditional bitwise AND operations where strong correlation occurs when the two input encoded sequences are the same, leading to errors in the calculation results.

[0023] In the decoding stage of random computation, the AND operation sequence is decoded to obtain the computation result.

[0024] Example This embodiment uses the YOLOv5 object detection model as an example, such as Figure 3As shown, the YOLOv5 object detection model includes a backbone network, a neck network, and a detection head. Different degrees of random computation are introduced to replace traditional convolution operations at different levels of different network structures in the model. For the backbone network, considering that the shallow features extracted by the backbone network contain rich texture, edge, and local structural information and are highly sensitive to numerical accuracy, while deep features have a strong tolerance for local numerical perturbations and accuracy, the backbone network retains traditional high-precision convolution calculations when extracting shallow features, and only introduces random calculations in deep feature extraction to avoid significantly affecting the expression of shallow features. That is, the convolutions from the focus layer to CSP1-2 layers of the backbone network retain the traditional convolution calculation method, while the convolution calculations of other layers are replaced by random calculations. For the neck network, considering that the 1×1 and 3×3 convolutions involved are essentially weighted sums, and that neither type of convolution undertakes the task of extracting low-level texture and edges, but is mainly used for feature channel alignment and fusing features of different scales, random calculations are used to completely replace convolution calculations. For the detection head, since the detection head mainly uses linear mapping and dense prediction, it has a strong tolerance for numerical approximation errors, so random calculations are used to completely replace convolution calculations.

[0025] In summary, this embodiment employs a differentiated, hierarchical random computation replacement strategy in the backbone network, neck network, and detection head of the YOLOv5 target detection model, which can significantly reduce computational complexity and hardware resource consumption with almost no impact on target detection quality.

[0026] To verify the effectiveness of this method, the random computation of this invention was compared with traditional convolution operations. The evaluation was mainly based on aspects such as detection accuracy, power consumption, detection speed, and resource area. The experimental results are shown in Table 2.

[0027] Table 2. Comparison of experimental results between traditional convolution calculation and random calculation of the present invention.

[0028] As shown in Table 2, the resource area, power consumption, detection speed, and energy efficiency ratio of the method of the present invention are superior to those of traditional convolutional computation. However, the detection accuracy is slightly lower than that of traditional convolutional computation. This is because the random computation unit used in the method of the present invention is simpler and more efficient, and can be deployed on a large scale to improve parallelism, thereby improving the inference speed. However, due to the lower precision of the shorter random sequence, the accuracy of inference is slightly reduced, but the decrease can be controlled within 1%. Therefore, the method of the present invention can reduce the resource area by about 32.31% and the power consumption by 30% while maintaining the model inference accuracy by about 27.66% and the energy efficiency ratio by 82.33%, respectively.

[0029] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A low-power convolutional neural network hardware acceleration method based on random computation, characterized in that, Includes the following steps: Step 1: Build a convolutional neural network model and train the model; The second step is to deploy the trained model to the hardware computing unit during the inference phase, and use random computation to partially or completely replace the convolutional computations in the model. The random computation includes three stages: encoding, computation, and decoding. During the encoding stage, the input feature map is scaled, and the convolution kernel is quantized. The grayscale values ​​of the convolution kernel and each point in the input feature map are normalized to a probability value, with a probability value of [value to be filled in]. , Indicates the length of the encoded sequence. Take a positive integer; Generate a coding sequence for each probability value. First, generate an initial coding sequence based on the probability values, ensuring that 1s are evenly distributed throughout the initial coding sequence; then shift the initial coding sequence to the right. The bits are used to obtain the encoded sequence corresponding to the probability value; During the computation phase, the two encoding sequences of the convolution kernel and the corresponding points of the input feature map are used as the upper encoding sequence and the lower encoding sequence. The first bit of the upper encoding sequence and the lower encoding sequence are ANDed. The lower encoding sequence automatically moves to the next bit. The value of the current bit of the lower encoding sequence determines whether the upper encoding sequence should move to the next bit. If the value of the current bit of the lower encoding sequence is 1, the upper encoding sequence moves to the next bit, and the next bit of the upper encoding sequence is ANDed with the current bit of the lower encoding sequence. If the value of the current bit of the lower encoding sequence is 0, the upper encoding sequence stays at the current bit, and the current bits of the upper and lower encoding sequences are ANDed. The above process is repeated to traverse all bits of the lower encoding sequence and generate the AND operation sequence. In the decoding stage, the AND operation sequence is decoded to obtain the calculation result.

2. The hardware acceleration method for low-power convolutional neural networks based on random computation according to claim 1, characterized in that, During the encoding phase, the probability values ​​0 and 1 are set to 0. As a base probability value, an encoding sequence is generated for the base probability value; if the current probability value is the sum of multiple base probability values, then the encoding sequences of multiple base probability values ​​are ORed by bit to obtain the encoding sequence of the current probability value.

Images