Universal one-dimensional convolutional neural network accelerator

Abstract

A universal one-dimensional convolutional neural network accelerator disclosed by the present invention comprises a global memory and a computing unit array, the computing unit array comprises an access unit and a computing unit, and configuration parameters of a one-dimensional convolutional neural network and used feature base addresses, weight base addresses and label base addresses are input into the computing unit array. The method comprises the following steps: firstly, generating a data base address according to a convolution parameter and an access cycle sequence through an access unit, and obtaining a label reading address, a characteristic value reading address and a weight reading address; the global memory is used for receiving the feature value reading request and the weight reading request, reading a feature value and a weight by utilizing a feature value reading address and a weight reading address, and sending the feature value and the weight to the calculation unit; multiply-accumulate operation of the characteristic value and the weight is carried out through the calculation unit, and a data stream with a preferential input channel and a fixed output is formed, so that the calculation efficiency in a sparse data scene is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of neural network accelerators, and in particular to a universal one-dimensional convolutional neural network accelerator. Background Art

[0002] Wearable devices often need to process complex physiological signals (such as EEG, ECG, etc.) in real time. At the same time, they are limited by battery capacity and heat dissipation conditions, and have extremely stringent requirements on power consumption and energy efficiency. Compared with traditional general-purpose processors (such as CPUs) or graphics processing units (GPUs), neural network accelerators have higher energy efficiency and can complete complex neural network computing tasks with extremely low power consumption. Through hardware and software co-design, hardware architecture optimization, and even customization of underlying circuits, neural network accelerators can significantly improve the computational efficiency of neural network models, reduce power consumption, and meet real-time requirements. This high energy efficiency feature not only extends the battery life of the device, but also supports more complex algorithms and a wider range of application scenarios.

[0003] Research on accelerators for convolutional neural networks (CNNs) has made significant progress, with extensive work dedicated to optimizing the computational efficiency, resource utilization, and energy efficiency of two-dimensional (2D) convolutional neural networks. In contrast, despite the widespread application of one-dimensional convolution in fields such as speech processing, time series analysis, and biosignal processing, relatively little research has been conducted on accelerators optimized for this purpose. Most existing accelerator designs are primarily focused on 2D convolution, failing to fully consider the unique characteristics of 1D convolution in terms of data characteristics, computational model, and memory access behavior. Specifically, the data stream of 1D convolution typically has a higher channel dimension and a longer time series length, and its data locality and reuse patterns differ significantly from the data distribution in the spatial dimension of 2D convolution. In terms of computational model, the convolution kernel of 1D convolution slides only along a single time dimension, making its data parallel characteristics distinct from the spatial sliding window-based computational model of 2D convolution. The computing architecture and bandwidth allocation strategy of existing accelerators are mainly optimized for the spatial data access mode of two-dimensional convolution. They adopt a fixed data partitioning strategy and a fixed computing unit mapping method, resulting in problems such as low data reuse efficiency and insufficient bandwidth utilization when processing one-dimensional convolution tasks.

[0004] Currently, one-dimensional convolutional neural network (1D-CNN) accelerators can be divided into two categories: dedicated 1D-CNN accelerators and general-purpose 1D-CNN accelerators. Dedicated accelerators are optimized for specific tasks and models and typically perform well in specific scenarios. However, these accelerators optimized for specific 1D-CNN models typically adopt relatively fixed computing units and data flow designs. Due to their architectural limitations, these accelerators can only efficiently run a single or slightly modified neural network model, which greatly limits their versatility on other types of neural network models. In contrast, general-purpose 1D-CNN accelerators focus more on flexibility and compatibility. These designs support general-purpose 1D-CNN computations through configurable computation arrays or operator optimization strategies, which often introduces additional area overhead or leads to reduced computational efficiency.

[0005] Most existing neural network accelerators are optimized for only one or two specific types of convolution. This specialized design leads to compatibility challenges and efficiency bottlenecks when processing other types of convolution operations. When deploying other types of convolutional neural networks, such accelerators often experience a cliff-like drop in computational efficiency, and in extreme cases, they may even fail to operate normally. To further complicate matters, convolutional neural networks themselves contain a variety of configurable parameters (including input and output channel dimensions, convolution kernel size, residual connections, convolution step size, etc.). The combination of these parameter spaces greatly increases the difficulty of accelerator compatibility design. Summary of the Invention

[0006] The purpose of this application is to propose a universal one-dimensional convolutional neural network accelerator to address the above-mentioned technical problems.

[0007] In the first aspect, the present invention provides a universal one-dimensional convolutional neural network accelerator, including a global memory and a computing unit array, the computing unit array including a data access unit and 16 computing units arranged in a 4×4 array, the data access unit including an address generator, a data information generation logic module and a weight data cache; the configuration parameters of the one-dimensional convolutional neural network and the feature base address, weight base address and label base address used in the calculation process of the one-dimensional convolutional neural network stored in the global memory are input into the computing unit array and first pass through the data access unit; the configuration parameters include convolution parameters, data access mode and task division method; in the data access unit, the address generator is used to generate a data base address according to the convolution parameters and the data access loop sequence, and the data base address is added to the label base address, feature base address and weight base address respectively to obtain a label reading address, a feature value reading address and a weight reading address; the data information generation logic module selects whether to read the corresponding data according to the label reading address according to the data access mode. The characteristic value label is used to determine whether to send a characteristic value read request to the global memory according to the characteristic value label and the data access mode, generate characteristic value information and weight information according to the data base address, use the weight information to determine whether to use the weight read address to obtain the corresponding weight in the weight data cache or generate a weight read request and send it to the global memory to read the corresponding weight. The global memory is used to read the corresponding characteristic value and weight using the characteristic value read address and the weight read address after receiving the characteristic value read request and the weight read request and send them to the data information generation logic module. The data information generation logic module is used to send the characteristic value and the characteristic value information together to the calculation unit of each column, and send the corresponding weight obtained in the weight data cache together with the weight information to the calculation unit of each row. The calculation unit performs multiplication and accumulation operations on the characteristic value and weight, performs partial and cache and quantized output operations, obtains the multiplication and accumulation results of the quantized output, and forms a data stream with input channel priority and fixed output.

[0008] Preferably, the convolution parameters include the convolution kernel length W, the number of input channels I, the input eigenvalue length P, the output eigenvalue length Q, the expansion factor D, the number of output channels O, and whether batch normalization is used. The convolution types of the one-dimensional convolutional neural network include conventional one-dimensional convolution, depth-wise one-dimensional convolution, point-by-point one-dimensional convolution, and dilated one-dimensional convolution, wherein the depth-wise one-dimensional convolution and point-by-point one-dimensional convolution are achieved by adjusting the convolution kernel length W or the number of input channels I in the acquisition loop; the calculation loop order of the conventional one-dimensional convolution from inside to outside is the convolution kernel length W, the number of input channels I, the input eigenvalue length P , output eigenvalue length Q and output channel number O, the corresponding number acquisition loop sequence contains four nested loops, namely the convolution kernel length loop, the input channel number loop, the input eigenvalue loop and the output channel number loop. The calculation loop sequence of the extended one-dimensional convolution is, from the inside to the outside, the convolution kernel length W, the input channel number I, the input eigenvalue length P, the output eigenvalue length Q, the dilation factor D and the output channel number O, and the corresponding number acquisition loop sequence contains five nested loops, namely the convolution kernel length loop, the input channel number loop, the input eigenvalue loop, the dilation factor loop and the output channel number loop.

[0009] Preferably, in the process of generating the data base address, the convolution type of the one-dimensional convolutional neural network is determined to be a conventional one-dimensional convolution or an extended one-dimensional convolution according to the convolution parameter. If the convolution type of the one-dimensional convolutional neural network is an extended one-dimensional convolution and the dilation factor is D, the address increment step in the input eigenvalue loop is set to D; the data acquisition mode is determined by the sparsity of the eigenvalue used in the calculation process of the one-dimensional convolutional neural network. If the sparsity is less than the threshold, the data acquisition mode adopts the sparse data acquisition mode. The amount of data read in the sparse data acquisition mode is calculated using the following formula:

[0010] Bit read =1×sparsity+9×(1-sparsity);

[0011] Among them, sparsity represents sparsity, which is the number of zero eigenvalues ​​divided by the total number of eigenvalues. read Indicates the amount of data read in sparse access mode.

[0012] Preferably, the data information generation logic module includes a sparse mode control unit, a label buffer, an address buffer, an information generator and a feature value buffer. The address buffer is connected to the information generator and is used to temporarily store the data base address. The information generator is used to generate feature value information and weight information according to the data base address. The feature value buffer is used to temporarily store the feature value; the data acquisition mode includes a sparse data acquisition mode and a dense data acquisition mode. In the dense data acquisition mode, the row calculation unit and the column calculation unit in the calculation unit array maintain synchronous calculation. In the sparse data acquisition mode, except for the feature value of the last input channel in the last input feature map and its corresponding last weight, the data acquisition unit skips reading the feature value and weight corresponding to the feature value label of 0; in the current cycle, the sparse mode control unit is used to generate a label read request according to the label read address and determine whether to send it to the global memory according to the data acquisition mode. If the data acquisition mode is the sparse data acquisition mode, the label read request is sent to the global memory; if the data acquisition mode is the dense data acquisition mode, the label read request is not sent to the global memory; in the next cycle, the global memory returns the corresponding feature value label after receiving the label read request and temporarily stores it in the label buffer.

[0013] Preferably, determining whether to send a feature value read request to a global memory according to the feature value tag and the data acquisition mode specifically includes:

[0014] Generate a characteristic value read request based on the characteristic value read address, and after generating the characteristic value read request, read the corresponding characteristic value tag from the tag buffer;

[0015] When the eigenvalue label is 1, or the current data acquisition mode is the dense data acquisition mode, or the eigenvalue is determined to be the eigenvalue of the last input channel in the last input feature map according to the eigenvalue information, a eigenvalue read request is sent to the global memory;

[0016] When the eigenvalue label is 0, the current data acquisition mode is the sparse data acquisition mode, and it is determined based on the eigenvalue information that the eigenvalue is not the eigenvalue of the last input channel in the last input feature map, the eigenvalue read request is not sent to the global memory.

[0017] Preferably, the weight information is used to determine whether the corresponding weight is obtained in the weight data cache or to generate a weight read request and send it to the global memory to read the corresponding weight, specifically including:

[0018] If the eigenvalue label is 1, or the current data acquisition mode is the dense data acquisition mode, or the weight is determined to be the last weight corresponding to the eigenvalue of the last input channel in the last input feature map according to the weight information, the weight in the weight data cache is retrieved through the weight information. If the weight corresponding to the weight information is retrieved, the corresponding weight is obtained in the weight data cache, and no weight read request is generated; if the weight corresponding to the weight information cannot be retrieved, a weight read request is generated and sent to the global memory, the corresponding weight is read from the global memory and temporarily stored in the weight data cache, and the weight is read in the weight data cache. The weight data cache adopts a first-in-first-out (FIFO) cache replacement strategy.

[0019] Preferably, the task division method includes task division based on inter-channel parallelism and task division based on intra-channel parallelism. During the generation of the data base address, the selection results of the weights corresponding to the left and right boundaries of the input eigenvalues ​​are adjusted through the task division method, and it is determined whether to splice the convolution calculation results obtained by the computing unit processing to obtain the multiplication and accumulation results according to the task division method; if the current task division method is task division based on inter-channel parallelism, it is only necessary to adjust the selection results of the weights corresponding to the left and right boundaries of the input eigenvalues, and the weights only need to obtain the weights within the coverage range of the convolution sliding window; if the current task division method is task division based on intra-channel parallelism and the convolution kernel size is not 1, then the left and right boundaries of the input eigenvalues ​​are judged to be real boundaries or boundaries that need to be spliced. If they are boundaries that need to be spliced, additional eigenvalues ​​are obtained from the sub-segment spliced ​​with the current sub-segment to reach the required number within the coverage range of the convolution sliding window, and the weights only need to obtain the weights within the coverage range of the convolution sliding window.

[0020] Preferably, the convolution sliding window coverage range is a length value obtained by dividing the convolution kernel length by 2 and rounding down.

[0021] Preferably, the computing unit includes a control unit, a multiplication-addition unit, a partial sum register stack, and an output quantization unit, wherein the partial sum register stack includes a plurality of partial sum registers; the control unit selects a corresponding partial sum register in the partial sum register stack based on the eigenvalue information and weight information input synchronously with the eigenvalue and weight, and obtains the partial sum to be accumulated; the multiplication-addition unit is used to multiply the synchronously input eigenvalue and weight, and accumulate the multiplication result with the partial sum to be accumulated to obtain a partial sum result, and store the partial sum result back in the selected partial sum register;

[0022] The multiplication and addition unit includes a 24×8-bit multiplier and a 48-bit accumulator, which completes three 24×8-bit multiplication operations and four 48-bit addition operations through time division multiplexing;

[0023] When the control unit determines that the partial sum accumulation is completed, the accumulated partial sum result is quantized and output through the output quantization unit. The output quantization unit is used to use the quantization scaling factor and bias in the configuration parameters or the quantization scaling factor and bias of the fused batch normalization to quantize the partial sum structure and output it to obtain the quantization result, and then send the quantization result to the global memory.

[0024] Preferably, the working modes of the computing unit include a dense working mode and a sparse working mode corresponding to the dense data acquisition mode and the sparse data acquisition mode respectively. The computing units in different rows in the same column share the same eigenvalues ​​for calculating the convolution tasks of different output channels, and the computing units in different columns in the same row share the same weights for calculating the convolution tasks of different sub-segments. The number of output channels of the eigenvalues ​​used in the calculation process of the one-dimensional convolutional neural network is a multiple of 4.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] (1) The universal one-dimensional convolutional neural network accelerator proposed in this invention proposes a data stream with input channel priority and fixed output. This data stream adopts the computational loop sequence with input channel priority and can flexibly support various types of one-dimensional convolution computations. At the same time, the fixed output feature enables the computational unit to perform the multiplication and accumulation operations of partial sums in situ, completely avoiding the memory access operations of high-precision partial sums during the computation process, thereby significantly reducing the memory bandwidth requirements.

[0027] (2) The universal one-dimensional convolutional neural network accelerator proposed in this invention designs a universal one-dimensional convolutional neural network computing unit array. The computing unit array adopts a flexible data acquisition strategy adopted by the data acquisition unit, so that the computing unit array with a fixed structure can efficiently perform various types of one-dimensional convolution calculations. In addition, combined with a flexible task division method and an efficient eigenvalue and weight reuse strategy, the computing efficiency is further improved, while the data access overhead is significantly reduced.

[0028] (3) The universal one-dimensional convolutional neural network accelerator proposed in this paper supports both dense and sparse computing modes. In sparse computing mode, the computational hope array can skip the access operation of zero-valued eigenvalues ​​based on the label, thus avoiding the invalid calculation and storage access of zero-valued data from the source. At the cost of minimal hardware overhead, it significantly improves the computing efficiency in sparse data scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 A schematic diagram of a general one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a conventional one-dimensional convolution computation cycle of a universal one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0032] Figure 3 A schematic diagram of a computational loop for an extended one-dimensional convolution of a universal one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0033] Figure 4 A schematic diagram of task partitioning based on inter-channel parallelism of a universal one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0034] Figure 5 A schematic diagram of task partitioning based on intra-channel parallelism of a general one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0035] Figure 6 Schematic diagram of the internal structure of a data acquisition unit of a universal one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0036] Figure 7 Schematic diagram of a flow chart of an acquisition loop of a dilated one-dimensional convolution with a dilation factor of 4 in dense acquisition mode of a universal one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0037] Figure 8 Schematic diagram of a data acquisition loop in a sparse data acquisition mode of a universal one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0038] Figure 9 Schematic diagram of the internal structure of a computing unit of a universal one-dimensional convolutional neural network accelerator according to an embodiment of the present application;

[0039] Figure 10 A flowchart illustrating the computational efficiency of a general one-dimensional convolutional neural network accelerator under different convolutional layer configuration parameters according to an embodiment of the present application;

[0040] Figure 11This is a schematic diagram showing how the computational efficiency of a general one-dimensional convolutional neural network accelerator with a convolution kernel of 1 changes under different label bit widths as affected by eigenvalue sparsity in an embodiment of the present application;

[0041] Figure 12 This is a schematic diagram showing how the speedup ratio of a general one-dimensional convolutional neural network accelerator with a convolution kernel of 1 changes under different label bit widths as affected by eigenvalue sparsity.

[0042] Figure 13 This is a schematic diagram showing how the computational efficiency of a general one-dimensional convolutional neural network accelerator with a convolution kernel of 3 changes under different label bit widths as affected by eigenvalue sparsity in an embodiment of the present application;

[0043] Figure 14 This is a schematic diagram showing how the speedup ratio of a general one-dimensional convolutional neural network accelerator with a convolution kernel of 3 changes under different label bit widths as affected by eigenvalue sparsity.

[0044] Figure numerals: 1. global memory; 2. computing unit; 21. control unit; 22. output quantization unit; 3. data acquisition unit; 31. address generator; 32. weight data cache; 33. sparse pattern control unit; 34. label buffer; 35. address buffer; 36. information generator; 37. eigenvalue buffer. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0046] Figure 1A general one-dimensional convolutional neural network accelerator provided by an embodiment of the present application is shown, including a global memory 1 and a computing unit array, the computing unit array including a data access unit 3 and 16 computing units 2 arranged in a 4×4 array, the data access unit 3 including an address generator 31, a data information generation logic module and a weight data cache 32; the configuration parameters of the one-dimensional convolutional neural network and the feature base address, weight base address and label base address used in the calculation process of the one-dimensional convolutional neural network stored in the global memory 1 are input into the computing unit array and first pass through the data access unit 3; the configuration parameters include convolution parameters, data access mode and task division method; in the data access unit 3, the address generator 31 is used to generate a data base address according to the convolution parameters and the data access loop sequence, and the data base address is added to the label base address, feature base address and weight base address respectively to obtain a label reading address, a feature value reading address and a weight reading address; the data information generation logic module selects whether to read according to the label reading address according to the data access mode. Get the corresponding eigenvalue label, determine whether to send a eigenvalue read request to the global memory 1 according to the eigenvalue label and the data access mode, generate eigenvalue information and weight information according to the data base address, use the weight information to determine whether to use the weight read address to obtain the corresponding weight in the weight data cache 32 or generate a weight read request and send it to the global memory 1 to read the corresponding weight, the global memory 1 is used to read the corresponding eigenvalue and weight using the eigenvalue read address and the weight read address after receiving the eigenvalue read request and the weight read request and send them to the data information generation logic module, the data information generation logic module is used to send the eigenvalue and the eigenvalue information together to the calculation unit 2 of each column, and send the corresponding weight obtained in the weight data cache 32 together with the weight information to the calculation unit 2 of each row, perform multiplication and accumulation operations on the eigenvalue and the weight through the calculation unit 2, and perform partial sum caching and quantization output operations to obtain the multiplication and accumulation results of the quantized output, and form a data stream with input channel priority and fixed output.

[0047] In a specific embodiment, the convolution parameters include the convolution kernel length W, the number of input channels I, the input eigenvalue length P, the output eigenvalue length Q, the expansion factor D, the number of output channels O, and whether to use batch normalization. The convolution types in the one-dimensional convolutional neural network include conventional one-dimensional convolution, depth-wise one-dimensional convolution, point-by-point one-dimensional convolution, and dilated one-dimensional convolution, wherein the depth-wise one-dimensional convolution and point-by-point one-dimensional convolution are achieved by adjusting the convolution kernel length W or the number of input channels I in the acquisition loop; the calculation loop order of the conventional one-dimensional convolution from inside to outside is the convolution kernel length W, the number of input channels I, the input eigenvalue length P, and the dilated one-dimensional convolution. The calculation loop sequence of the extended one-dimensional convolution is from the inside to the outside, which is the convolution kernel length W, the input channel number I, the input eigenvalue length P, the output eigenvalue length Q, the expansion factor D and the output channel number O. The corresponding number acquisition loop sequence contains five nested loops, which are the convolution kernel length loop, the input channel number loop, the input eigenvalue loop, the expansion factor loop and the output channel number loop.

[0048] Specifically, the universal one-dimensional convolutional neural network accelerator proposed in the embodiment of the present application adopts an input channel first, output fixed (IFOS) data stream, which is mainly used to perform one-dimensional convolution or matrix multiplication operations. The universal one-dimensional convolutional neural network accelerator proposed in the embodiment of the present application can efficiently support various types of one-dimensional convolution operations, including conventional one-dimensional convolution, depth-wise one-dimensional convolution, point-by-point one-dimensional convolution, and dilated one-dimensional convolution. In addition, it also supports a variety of configurable convolution parameters, such as convolution kernel size, step size, number of input and output channels, dilation factor, and whether to use batch normalization (BatchNorm). At the same time, the proposed accelerator also supports zero jumping of eigenvalues ​​based on labels.

[0049] Figure 1The overall architecture of the general one-dimensional convolutional neural network accelerator proposed in the embodiment of the present application is shown, which is mainly composed of a global memory 1 (Global Buffer, GB) and a computing unit array (PE Array). The global memory 1 is composed of 16 blocks of 4KB static random access memory (SRAM), which is used to store eigenvalues ​​and weights represented in an 8-bit fixed point format. The computing unit array serves as the core computing unit 2 of the accelerator, and its structure includes a dynamically configured data acquisition unit 3 and 16 parallel computing units 2 (PE). During the calculation process, the data acquisition unit 3 reads the eigenvalues ​​and weights from the GB according to the configuration, and distributes them to the 16 PEs for parallel processing. After each PE completes the multiplication and accumulation operation of the eigenvalues ​​and weights, it will perform an output quantization operation and write the calculation result back to the specified address of the global memory 1 as the input feature map of the subsequent convolution layer. Through multiple configurations and startups, the accelerator can fully execute the computing tasks of all network layers in the one-dimensional convolutional neural network.

[0050] The computational unit array in the general one-dimensional convolutional neural network accelerator proposed in the embodiments of the present application consists of one data access unit 3 and 16 computational units 2 (PEs). The data access unit 3 is responsible for providing paired eigenvalues ​​and weights to the computational unit 2; the 16 computational units 2 (PEs) are arranged in a 4×4 array and are responsible for performing multiplication and accumulation operations and quantized outputs. Thanks to the good support of the input channel first, output fixed (IFOS) data flow for one-dimensional convolution, the proposed computational unit array supports different convolution parameters mainly through the data access unit 3. This design simplifies the computational unit 2 (PE). At the same time, the computational unit array does not need to rely on a configurable topology to adapt to different convolution parameters, thereby achieving high energy efficiency in different computing scenarios. The data access unit 3 flexibly adjusts the data reading and allocation strategy according to different convolution parameters (such as convolution kernel size, step size, dilation factor, etc.), thereby decoupling the task of adapting to different computing needs from the computational unit array. This division of labor not only reduces the design complexity of the PE, but also reduces the hardware overhead of the data transmission network, while maintaining the efficiency and flexibility of the computational unit array.

[0051] Specifically, there are several common types of convolution in one-dimensional convolutional neural networks. In addition to standard 1D convolution neural networks, there are also depthwise 1D convolution, pointwise 1D convolution, and dilated 1D convolution. Among them, depthwise separable convolution, which consists of depthwise 1D convolution and pointwise 1D convolution, can significantly reduce the number of model parameters and computational complexity by decoupling the extraction process of spatial features and channel features. Therefore, it is widely used in lightweight networks such as the MobileNet series. Dilated 1D convolution expands the receptive field of the convolution kernel, allowing smaller convolution kernels to achieve feature extraction capabilities comparable to large conventional convolution kernels. These differently designed convolution types each have unique advantages and can be flexibly combined and applied to neural network architectures according to actual task requirements.

[0052] The one-dimensional convolutional neural network accelerator proposed in the embodiment of this application needs to ensure that it can maintain stable and efficient computing performance when dealing with one-dimensional convolutions of different types and parameters through deep optimization of data flow and hardware structure. The neural network accelerator proposed in this article adopts an input channel first, output fixed (Input channel First-Output Stationary, IFOS) data flow. The calculation process of the proposed data flow for different types of one-dimensional convolutions is described as follows.

[0053] The computational loop of a conventional one-dimensional convolution consists of five layers of loops, such as Figure 2As shown. Specifically, the calculation loop of the conventional one-dimensional convolution is, from the inside to the outside, the convolution kernel length W, the number of input channels I, the input eigenvalue length P, the output eigenvalue length Q, and the number of output channels O. In the IFOS data stream, the eigenvalue acquisition process prioritizes the input channel (InputChannel First), that is, the data corresponding to the input channel is read first. Whenever an eigenvalue is read, the weight corresponding to the eigenvalue of the input channel is read at the same time. The number of weights read is determined by the convolution kernel length W, and the weight is read in the form of a sliding convolution window coverage. The size of the convolution window coverage is the result obtained by dividing the convolution kernel length W by 2 and rounding down. The computing unit 2 (PE) multiplies the eigenvalue in each input channel with its corresponding weight in sequence. The controller inside the PE accumulates the current multiplication result to the corresponding part and register inside the PE according to the address and index information of the eigenvalue and weight. In the IFOS data flow, the output eigenvalues ​​N and the number of output channels O are located at the outermost layer of the convolution loop, keeping the output stationary within a single computational element (PE). This avoids the transmission of intermediate results (i.e., partial sums). Because the convolution kernel length W and the number of input channels I are looped within the input eigenvalue length P, once the eigenvalues ​​of all input channels covered by the convolution are input, the computational element (PE) has completed the convolution calculation for the corresponding output feature map. At this point, the stored partial sums can be directly quantized and output, freeing up the buffer space they occupied. Therefore, the maximum number of partial sums required to be stored in a PE is only the maximum length Wmax of the convolution kernel. For most one-dimensional convolutions, the maximum length Wmax of the convolution kernel is typically less than 8. Therefore, only a small number of partial sum registers are required within each computational element (PE) to support the IFOS data flow computation method.

[0054] The support of the IFOS data stream for depth-wise one-dimensional convolution or point-by-point one-dimensional convolution proposed in the embodiments of the present application can be achieved by adjusting the convolution kernel length W or the number of input channels I in the data acquisition loop. The support of the IFOS data stream for the convolution step length Stride can be achieved by adjusting the data acquisition address in the input eigenvalue length P loop. The support of the IFOS data stream for the expanded one-dimensional convolution can be achieved by modifying the order of the data acquisition loop. The expanded one-dimensional convolution is equivalent to first downsampling the original signal by a dilation factor D, and then performing a conventional one-dimensional convolution calculation. Therefore, compared with other types of one-dimensional convolution, the calculation of the expanded one-dimensional convolution has an additional layer of loop for the dilation factor D. In the IFOS data stream, the calculation of the expanded one-dimensional convolution includes a total of six layers of loops. Figure 3The cyclic structure of the dilated one-dimensional convolution used when the dilation factor D is 2 is detailed: first, the data is downsampled by a factor of 2 to obtain the downsampled data of the odd and even sequences in the original signal; then, the data of the odd and even sequences are convolved separately using a compact convolution kernel after removing the dilation holes; finally, the convolution results of the odd and even sequences are rearranged to obtain the calculation results of the dilated one-dimensional convolution. For different dilation factors D, only the number of cycles of the dilation factor D needs to be adjusted. In practice, the downsampling and data rearrangement operations can be achieved by adjusting the addresses of the read and write data, so the dilated one-dimensional convolution operation does not require the introduction of an additional computing unit 2.

[0055] In a specific embodiment, the task division method includes task division based on inter-channel parallelism and task division based on intra-channel parallelism. During the generation process of the data base address, the selection results of the weights corresponding to the left and right boundaries of the input eigenvalues ​​are adjusted through the task division method, and it is determined whether to splice the convolution calculation results obtained by processing the computing unit 2 according to the task division method to obtain the multiplication and accumulation results; if the current task division method is task division based on inter-channel parallelism, it is only necessary to adjust the selection results of the weights corresponding to the left and right boundaries of the input eigenvalues, and the weights only need to obtain the weights within the coverage range of the convolution sliding window; if the current task division method is task division based on intra-channel parallelism and the convolution kernel size is not 1, then it is judged that the left and right boundaries of the input eigenvalues ​​are real boundaries or boundaries that need to be spliced. If they are boundaries that need to be spliced, additional eigenvalues ​​are obtained from the sub-segment spliced ​​with the current sub-segment to reach the required number within the coverage range of the convolution sliding window, and the weights only need to obtain the weights within the coverage range of the convolution sliding window.

[0056] In a specific embodiment, the convolution sliding window coverage range is a length value obtained by dividing the convolution kernel length by 2 and rounding down.

[0057] Specifically, efficiently supporting one-dimensional convolutional neural networks with a variety of different parameters is an important goal of the one-dimensional convolutional neural network accelerator proposed in the embodiments of this application. Since the parameters such as the number of input channels, the number of output channels, the size of the convolution kernel, and the expansion factor of the convolution kernel may change in the supported convolution calculations, it is difficult to extract the commonality of weights in different convolution operations. Therefore, the one-dimensional convolutional neural network accelerator proposed in the embodiments of this application does not use a pulsating array, but chooses to transmit data by broadcasting. At the same time, by reasonably dividing the convolution tasks, data reuse of weights in different PEs is achieved.

[0058] The task division methods in the embodiments of the present application can be mainly divided into two types: task division based on inter-channel parallelism and task division based on intra-channel parallelism, such as Figure 4 and 5As shown in the figure. The first division method targets the fact that one-dimensional signals such as ECG and EEG signals have many channels. One-dimensional convolutional neural networks usually use the same convolution kernel to convolve data collected from different channels separately to reduce the number of weights in the neural network. The input feature values ​​of these convolution tasks come from different channels, and the tasks are independent of each other, but the convolution kernels used are exactly the same. Therefore, a reasonable task division method is to have different computing units 2 (PEs) calculate the convolution of different channels respectively, and share weights between PEs, thereby achieving weight data reuse. The second division method targets the fact that one-dimensional signal sequences are long. Since one-dimensional signals are usually long in the time dimension, they can be divided into multiple sub-segments along the time dimension for processing, and these sub-segments also use the same convolution kernel. Therefore, another reasonable task division method is to divide the signal segment into multiple sub-segments along the time dimension and assign the convolution tasks of different sub-segments to different PEs for calculation, so that the weights are shared between PEs, thereby achieving weight data reuse. It should be noted that this task division method requires special processing of boundary region data: when the convolution kernel spans adjacent sub-segments, the calculation of the eigenvalues ​​within its coverage area must simultaneously obtain data from both sub-segments. In this case, the data obtained by each PE is not completely independent (the larger the convolution kernel size, the wider the boundary region is affected). Only when using a single convolution kernel, the calculation of each sub-segment is completely independent, and no special processing of the boundary region data is required. The final calculation result is completed by splicing the outputs of each sub-segment.

[0059] In order to meet the support requirements of the proposed one-dimensional convolutional neural network accelerator for diverse convolution parameters, two task division methods are flexibly used to optimize the data reuse efficiency of weights in different scenarios: when the data involves a large number of independently calculated channels, the task division method based on inter-channel parallelism is preferred, at this time the weights can be efficiently reused between PEs; when the number of independently calculated channels is small, the task division method based on intra-channel parallelism is switched to, at the expense of a small amount of boundary area processing, to ensure that the weights can still be reused between PEs.

[0060] The computational unit arrays described in the embodiments of this application support two one-dimensional convolution task partitioning methods: task partitioning based on inter-channel parallelism and task partitioning based on intra-channel parallelism. When deploying a neural network, the optimal task partitioning method is selected based on the parameters of the current convolutional layer and the configuration parameters are sent to the computational unit array. After receiving the configuration parameters, the computational unit array will execute the computational tasks of the corresponding convolutional layer. Based on the configuration parameters, the data acquisition unit 3 reads data in the cyclic order of the IFOS data stream and simultaneously transmits the eigenvalues ​​and weights to the computational unit 2 (PE) array via broadcast. For the subtasks performed by each computational unit 2 (PE), the embodiments of this application adopt the following partitioning strategy: PEs in different rows of the same column are responsible for calculating the convolution tasks of different output channels, while PEs in different columns of the same row are responsible for calculating the convolution tasks of different sub-segments. Whether the sub-segments belong to the same feature map and whether their results need to be spliced ​​depends on the one-dimensional convolution task partitioning method and the convolution kernel size. This design enables the computational unit array to achieve data reuse in both the row and column dimensions: PEs in each column share the same eigenvalues, and PEs in each row share the same weights, thereby maximizing the data reuse efficiency of eigenvalues ​​and weights. It's important to note that this task partitioning approach imposes a constraint: the number of output channels for the current convolution calculation must be a multiple of 4. Otherwise, some PE rows will be idle. However, this constraint is generally easy to meet in practice and has minimal impact on overall performance.

[0061] In a specific embodiment, the data information generation logic module includes a sparse mode control unit 33, a label buffer 34, an address buffer 35, an information generator 36 and a eigenvalue buffer 37. The address buffer 35 is connected to the information generator 36 for temporarily storing the data base address. The information generator 36 is used to generate eigenvalue information and weight information according to the data base address. The eigenvalue buffer 37 is used to temporarily store the eigenvalue; the data acquisition mode includes a sparse data acquisition mode and a dense data acquisition mode. In the dense data acquisition mode, the row calculation unit 2 and the column calculation unit 2 in the calculation unit array are kept synchronously calculated. In the sparse data acquisition mode, except for the last input feature map, Except for the eigenvalue of the last input channel and its corresponding last weight, the data acquisition unit 3 skips reading the eigenvalue and weight corresponding to the eigenvalue label of 0; in the current cycle, the sparse mode control unit 33 is used to generate a label read request according to the label read address and determine whether to send it to the global memory 1 according to the data acquisition mode. If the data acquisition mode is the sparse data acquisition mode, the label read request is sent to the global memory 1; if the data acquisition mode is the dense data acquisition mode, the label read request is not sent to the global memory 1; in the next cycle, the global memory 1 returns the corresponding eigenvalue label after receiving the label read request and temporarily stores it in the label buffer 34.

[0062] In a specific embodiment, in the process of generating the data base address, the convolution type of the one-dimensional convolutional neural network is determined to be a conventional one-dimensional convolution or an extended one-dimensional convolution according to the convolution parameter. If the convolution type of the one-dimensional convolutional neural network is an extended one-dimensional convolution and the dilation factor is D, the address increment step in the input eigenvalue loop is set to D; the data acquisition mode is determined by the sparsity of the eigenvalue used in the calculation process of the one-dimensional convolutional neural network. If the sparsity is less than the threshold, the data acquisition mode adopts the sparse data acquisition mode. The amount of data read in the sparse data acquisition mode is calculated using the following formula:

[0063] Bit read =1×sparsity+9×(1-sparsity);

[0064] Among them, sparsity represents sparsity, which is the number of zero eigenvalues ​​divided by the total number of eigenvalues. read Indicates the amount of data read in sparse access mode.

[0065] In a specific embodiment, determining whether to send a feature value read request to the global memory 1 according to the feature value tag and the data acquisition mode specifically includes:

[0066] Generate a characteristic value read request based on the characteristic value read address, and after generating the characteristic value read request, read the corresponding characteristic value tag from the tag buffer 34;

[0067] When the eigenvalue label is 1, or the current data acquisition mode is the dense data acquisition mode, or the eigenvalue is determined to be the eigenvalue of the last input channel in the last input feature map according to the eigenvalue information, a eigenvalue read request is sent to the global memory 1;

[0068] When the eigenvalue label is 0, and the current data acquisition mode is the sparse data acquisition mode, and it is determined based on the eigenvalue information that the eigenvalue is not the eigenvalue of the last input channel in the last input feature map, the eigenvalue read request is not sent to the global memory 1.

[0069] In a specific embodiment, the weight information is used to determine whether the corresponding weight is obtained in the weight data cache 32 or to generate a weight read request and send it to the global memory 1 to read the corresponding weight, specifically including:

[0070] If the eigenvalue label is 1, or the current data acquisition mode is a dense data acquisition mode, or the weight is determined to be the last weight corresponding to the eigenvalue of the last input channel in the last input feature map according to the weight information, the weight in the weight data cache 32 is retrieved through the weight information. If the weight corresponding to the weight information is retrieved, the corresponding weight is obtained in the weight data cache 32, and no weight read request is generated; if the weight corresponding to the weight information cannot be retrieved, a weight read request is generated and sent to the global memory 1, the corresponding weight is read in the global memory 1 and temporarily stored in the weight data cache 32, and the weight is read in the weight data cache 32. The weight data cache 32 adopts a first-in-first-out (FIFO) cache replacement strategy.

[0071] Specifically, the data fetch unit 3, as a key component of the computing unit array, is mainly responsible for obtaining data from the on-chip cache and providing paired feature values, weights, and related address and index information to the computing unit 2 (PE) array. Figure 6 As shown in FIG, the data acquisition unit 3 is mainly composed of three functional modules: an address generator 31, a data information generation logic module and a weight data cache 32. Among them, the address generator 31 determines whether it is a conventional one-dimensional convolution or an expanded one-dimensional convolution according to the configuration parameters, and selects different data acquisition loop sequences. The data acquisition loop sequence of the address generator 31 is roughly consistent with the proposed calculation loop sequence of the input channel priority and output fixed data stream (IFOS): the data acquisition loop of the conventional one-dimensional convolution contains five layers of loops, such as Figure 2 Specifically, the calculation loop of the conventional one-dimensional convolution is the convolution kernel length W loop, the input channel number I loop, the input eigenvalue length P loop, the output eigenvalue length Q loop and the output channel number O loop from the inside to the outside; the data acquisition loop of the expanded one-dimensional convolution contains six layers of loops, as shown in Figure 3As shown. Specifically, the calculation loop of the expanded one-dimensional convolution is, from the inside to the outside, the convolution kernel length W loop, the input channel number I loop, the input eigenvalue length P loop, the output eigenvalue length Q loop, the expansion factor D loop, and the output channel number O loop, wherein the output eigenvalue length Q loop in the calculation loop is only used as an output and is not parameterized into the number acquisition loop. The address generator 31 generates and sends a data read request according to the selected number acquisition loop sequence, and the data read request includes a eigenvalue read request and a weight read request. When the data is returned, the number acquisition unit 3 will package it with the corresponding address and index information, and transmit it to the computing unit array for processing. The weight data cache 32 in the number acquisition unit 3 is 32 bytes in size and adopts a first-in-first-out (FIFO) cache replacement strategy. When the corresponding weight is hit in the weight data cache 32 using the weight read address, the weight is obtained directly from the weight data cache 32, and there is no need to initiate a weight read request to obtain the weight from the global memory 1; when the corresponding weight is not hit in the weight data cache 32 using the weight read address, and the feature value label Flag is 1, or the current dense data access mode is, a weight read request is initiated to obtain the corresponding weight from the global memory 1 using the weight read address. Since each row of PE in the computing unit array performs the same convolution task of the output channel, the weights it uses are exactly the same. Thanks to the characteristics of data locality, using a smaller weight data cache 32 can significantly reduce the weight bandwidth requirements in the sparse data access mode.

[0072] The data acquisition unit 3 in the embodiment of the present application supports two working modes: sparse data acquisition mode and dense data acquisition mode, which correspond to the on and off states of the eigenvalue zero jump function respectively. In the sparse data acquisition mode, the data acquisition unit 3 will first initiate a label reading request. When the eigenvalue label shows that the current eigenvalue is zero, the data acquisition loop of the data acquisition unit 3 will skip the acquisition of the eigenvalue and the corresponding weight; only when the eigenvalue label shows that the current eigenvalue is non-zero, or the current working mode is the dense data acquisition mode, the data acquisition unit 3 will initiate a read request for the eigenvalue and the corresponding weight. In the sparse working mode, an 8-bit non-zero eigenvalue will be read together with its 1-bit eigenvalue label. When the sparsity of the convolutional layer eigenvalue is too low, the amount of data read in the sparse working mode may be more than that in the dense working mode, which violates the original intention of using the sparse working mode to reduce the data reading bandwidth. Therefore, it is necessary to adjust the amount of data read in the sparse working mode, and the amount of data read Bit read Related to sparsity. In one embodiment, when the sparsity is greater than 12.5%, it is more optimal to adopt the sparse working mode.

[0073] refer to Figure 6The address generator 31 generates a data base address (Addr.) based on the configuration parameters and the input channel priority and output fixed (IFOS) data stream. The data base address is added to the tag base address (Flag Addr.) read directly from the global memory 1 to form the tag read address.

[0074] In the current cycle, the sparse mode control unit 33 (Sparsity Mode Control) determines whether to send a tag read request (Request Flag) to read the feature value tag from global memory 1 based on the data access mode: sparse access mode sends a request; dense access mode does not send a request. In the next cycle, global memory 1 returns the feature value tag flag, which is temporarily stored in the tag buffer 34 (Flag Buffer). When the feature value tag flag is needed later, the feature value tag corresponding to the feature value is read from the tag buffer 34.

[0075] After the data base address (Addr.) passes through the address buffer 35 (Addr Buffer), the eigenvalue base address (Act.Addr.) and the weight base address (Wei.Addr.) are added respectively to become the eigenvalue read address and the weight read address. At this time, the eigenvalue tag Flag is synchronously read from the tag buffer 34 (Flag Buffer), and it is determined whether to send the eigenvalue read request (Requset Act.) and the weight read request (Requset Wei.) based on the eigenvalue tag Flag. At the same time, the information generator 36 (Info.Generator) generates eigenvalue information (Act.Info.) and weight information (Wei.Info.) according to the data base address (Addr.). Specifically, the eigenvalue information is the eigenvalue address, and the weight information is the weight index.

[0076] For the eigenvalue read request (Requset Act.): when the eigenvalue tag Flag is 1, or the current mode is dense data acquisition, the eigenvalue read request is sent; when the eigenvalue tag Flag is 0, and the current mode is sparse data acquisition, the eigenvalue read request is not sent.

[0077] For weight read request (Requset Wei.): First, obtain the corresponding weight from the weight data cache 32 (Cache) according to the weight information (Wei.Info). When the corresponding weight is hit in the weight data cache 32 using the weight read address, the weight is obtained directly from the weight data cache 32, and there is no need to initiate a weight read request (Requset Wei.) to obtain the corresponding weight from the global memory 1; only when the corresponding weight is not hit (Miss) in the weight cache using the weight read address, and the eigenvalue tag determines that data needs to be retrieved (eigenvalue tag Flag is 1, or the current mode is dense data retrieval), a weight read request will be initiated to obtain the corresponding weight from the global memory 1 using the weight read address.

[0078] If a feature value read request (Requset Act.) is sent, the corresponding feature value is read from global memory 1 using the feature value read address. After the feature value (Act.) is returned from global memory 1, it is temporarily stored in the feature value buffer 37 (Act.Buffer). It is then packaged with the feature value information (Act.Info.) and sent to each PE column (PE Column).

[0079] If a weight read request (Requset Wei.) is sent, the corresponding weight is read from global memory 1 using the weight read address. When the weight (Wei.) is returned from global memory 1, it is stored in the weight cache (Wei.Cache). As long as a feature value read request (Requset Act.) is sent, the weight will be synchronized with the feature value (some weights can be obtained from the weight data cache 32 without sending a weight read request (Requset Wei.), so the decision of whether to send the weight to the PE is based only on the feature value), and packaged together with the weight information (Wei.Info.) and sent to each row of PE (PE Row).

[0080] By introducing the eigenvalue zero-jumping mechanism, the data acquisition unit 3 can effectively reduce the reading overhead of zero-value eigenvalues ​​and corresponding weights, thereby reducing data bandwidth requirements and improving system energy efficiency. When the eigenvalue is returned, the data acquisition unit 3 will synchronously send it together with the eigenvalue information to the PEs of all rows in each PE column; when the weight is returned, the data acquisition unit 3 will synchronously send it together with the weight information to the PEs of all columns in each PE row. After receiving the eigenvalue, weight, and its corresponding eigenvalue address and weight index, each PE can work independently to complete the corresponding convolution task. This design not only improves the efficiency of data distribution, but also ensures that the parallel computing capabilities of the PE array are fully utilized.

[0081] The generation of feature value labels is also completed by data acquisition unit 3. After the calculation of a convolutional layer is completed, if feature value labels need to be generated, data acquisition unit 3 traverses the feature map of the next layer in order of input channel priority and generates the corresponding feature value labels. Finally, these feature value labels are written back to global memory 1 for subsequent calculations.

[0082] The following example illustrates the specific workflow of data acquisition unit 3 in dense data acquisition mode, using a dilated one-dimensional convolutional layer. This dilated one-dimensional convolutional layer uses the task partitioning method of parallel convolution between channels. Its parameters are as follows: convolution kernel length of 5, number of input channels of 32, input eigenvalue length of 64, and dilation factor of 4. Figure 7 The data acquisition loop of data acquisition unit 3 is shown in detail. This process consists of five nested loops: the convolution kernel length loop (W_LOOP), the input channel loop (I_LOOP), the input eigenvalue loop (P_LOOP), the dilation factor loop (D_LOOP), and the output channel loop (O_LOOP). It should be noted that since the dilation factor of the currently running dilated one-dimensional convolution layer is 4, the address increment step size in the input eigenvalue loop (P_LOOP) is set to 4 to accommodate the special computation mode of the dilated one-dimensional convolution.

[0083] Because this convolutional layer uses parallel convolution across channels, meaning each column of computational elements 2 (PEs) processes convolution computations from different independent channels, the left and right boundaries of the input feature map are both true boundaries. When the input feature map has no padding or the padding value is 0, only the weights within the convolution window's coverage affect the multiplication and addition results. Therefore, for eigenvalues, the data acquisition unit 3 only needs to acquire data from the respective input data segments. For weights, the data acquisition unit 3 only needs to acquire weights within the convolution window's coverage. Furthermore, due to the dilation factor loop (D_LOOP), the number of boundaries in the input eigenvalue loop (P_LOOP) increases by a factor of D, resulting in a corresponding increase in the number of affected eigenvalues. Specifically, for the left boundary, the affected eigenvalues ​​P0, P1, P2, and P3 only need to take weights W0-W2, while P4, P5, P6, and P7 only need to take weights W0-W3. For the right boundary, the affected eigenvalues ​​P60, P61, P62, and P63 only need to take weights W2-W4, while P56, P57, P58, and P59 only need to take weights W1-W4.

[0084] Figure 8This example shows how to access data from a conventional one-dimensional convolutional layer in sparse access mode. This convolutional layer uses the task division method of intra-channel parallel convolution, with the following parameters: convolution kernel length is 7, number of input channels is 32, and input eigenvalue length is 64. In sparse computation mode, access unit 3 skips accessing all eigenvalues ​​with zero values ​​(except for the last data entry, i.e., the eigenvalue of the last input channel in the last input feature map and its corresponding last weight). This is reflected in the possible discontinuity of addresses in the input channel loop (I_LOOP) and input eigenvalue loop (P_LOOP). For the last data entry, regardless of whether the eigenvalue is zero, access unit 3 will initiate an eigenvalue read request. This design ensures that the input eigenvalue loop (P_LOOP) is completely traversed, and the eigenvalue addresses it carries facilitate the computation unit 2 (PE) in determining whether the convolution (O_LOOP) of the current output channel is complete.

[0085] This convolution layer uses the task division method of intra-channel parallel convolution, and the data acquisition of the boundary part of its feature map needs special processing. In the intra-channel parallel convolution, each column of computing unit 2 (PE) is responsible for processing the convolution calculation tasks of different sub-segments from the same channel data, and the results of these sub-segments need to be spliced ​​later. At this time, the left and right boundaries of the input feature map may not be the real boundaries (depending on the position of the current sub-segment in the overall segment, that is, whether its left and right boundaries need to be spliced ​​with other sub-segments). For the boundaries that need to be spliced, the data acquisition of the data acquisition unit 3 will not be limited to the current sub-segment, but will additionally obtain a part of the feature values ​​from the spliced ​​sub-segments to ensure the correctness of the boundary data calculation. The number of feature values ​​that need to be obtained additionally depends on the coverage of the convolution sliding window.

[0086] like Figure 8 As shown, the left boundary of the sub-segment of the convolution task is the real boundary, and the right boundary is the boundary to be spliced. In this case, the data acquisition unit 3 will cross the right boundary of the sub-segment to acquire data, and additionally acquire the eigenvalues ​​P64*, P65*, and P66* that affect the convolution results of the data near the right boundary. For the acquisition of weights, the data acquisition unit 3 will only acquire the weights that affect the sub-segment results of the current convolution task. Specifically: for the left boundary, the eigenvalues ​​P0, P1, and P2 only need to take the weights W0-W3, W0-W4, and W0-W5, respectively. For the right boundary, the eigenvalues ​​P61, P62, P63, P64, P65, and P66 only need to take the weights W1-W6, W2-W6, W3-W6, W4-W6, W5-W6, and W6, respectively.

[0087] In a specific embodiment, the calculation unit 2 includes a control unit 21, a multiplication and addition unit, a partial sum register stack, and an output quantization unit 22. The partial sum register stack includes a plurality of partial sum registers. The control unit 21 selects the corresponding partial sum register in the partial sum register stack based on the eigenvalue information and weight information input synchronously with the eigenvalue and weight, and obtains the partial sum to be accumulated. The multiplication and addition unit is used to multiply the synchronously input eigenvalue and weight, and accumulate the multiplication result with the partial sum to be accumulated to obtain a partial sum result, and store the partial sum result back in the selected partial sum register.

[0088] The multiplication and addition unit includes a 24×8-bit multiplier and a 48-bit accumulator, which completes three 24×8-bit multiplication operations and four 48-bit addition operations through time division multiplexing;

[0089] When the control unit 21 determines that the partial sum accumulation is completed, the accumulated partial sum result is quantized and output through the output quantization unit 22. The output quantization unit 22 is used to use the quantization scaling factor and bias in the configuration parameters or the quantization scaling factor and bias of the fused batch normalization to quantize the partial sum structure and output it to obtain the quantization result, and then send the quantization result to the global memory 1.

[0090] In a specific embodiment, the working modes of the computing unit 2 include a dense working mode and a sparse working mode corresponding to the dense data acquisition mode and the sparse data acquisition mode respectively. The computing units 2 in different rows in the same column share the same eigenvalues, which are used to calculate the convolution tasks of different output channels. The computing units 2 in different columns in the same row share the same weights, which are used to calculate the convolution tasks of different sub-segments; the number of output channels of the eigenvalues ​​used in the calculation process of the one-dimensional convolutional neural network is a multiple of 4.

[0091] Specifically, the computing unit 2 is responsible for performing multiplication and accumulation operations, partial sum accumulation, and quantization output operations. Figure 9 As shown, the internal structure of the computing unit 2 includes a control unit 21, a multiplication and addition unit, a partial sum cache unit, and an output quantization unit 22. The computing unit 2 receives eigenvalues ​​and weights as input, caches the results of the multiplication and addition operations in the partial sum register stack, and finally quantizes the multiplication and accumulation results and outputs them. The eigenvalues ​​and weights are both 8 bits, and the partial sum register is 24 bits to meet the requirement of a maximum number of input channels of 256. The output quantization unit 22 is responsible for quantizing the 24-bit partial sum results to 8 bits as the input of the next convolution layer. Since EEG data is usually a signed number and is not suitable for conversion into a full positive number through a normalization operation, all operands inside the PE are represented by signed numbers. The number of partial sums required to be stored inside the PE is directly related to the size of the convolution kernel. In an embodiment of the present application, the number of partial sum register stacks is set to 8 to support one-dimensional convolution operations from 1 to 8.

[0092] During operation, the address and index information of the eigenvalues ​​and weights are input synchronously with the eigenvalues ​​and weights, and the control unit 21 accumulates the current multiplication result into the corresponding partial sum register based on this information. When the address of the input eigenvalue changes, the control unit 21 determines whether the current input eigenvalue has exceeded the coverage of the convolution sliding window, thereby determining whether it no longer contributes to the partial sum of the cache. In the convolution calculation, the sliding direction of the convolution sliding window is fixed from left to right. Therefore, if the current input eigenvalue does not contribute to a partial sum of a certain cache, in this round of convolution cycle, the subsequent input eigenvalue will no longer be accumulated to the partial sum (that is, the partial sum must have completed all the multiplication and accumulation calculations). At this time, the control unit 21 will send it to the output quantization unit 22 for quantization, and write the quantization result into the address space corresponding to the output cache. This working mode can effectively deal with the address jump of the input data in the channel dimension and the sliding direction of the convolution sliding window, thereby being compatible with the working scenarios of changing convolution step size and sparse data input. In sparse working mode, PE does not need to perform additional processing on zero-value eigenvalues ​​because the access unit 3 will not pass zero-value eigenvalues ​​and corresponding weights (except for the last input channel eigenvalue in the last input feature map and its corresponding last weight). At the same time, as introduced in the design of the access unit 3 above, the access unit 3 will ensure that the last data in each partial sum calculation is passed. This design ensures that the calculation loop of each partial sum will be traversed, simplifying the judgment logic of whether the partial sum has been accumulated in the sparse working mode at a small time cost. When the partial sum quantization is completed, the zero-value data will also be output to overwrite the old value of the corresponding address in the output cache, thereby avoiding the introduction of additional cache refresh steps. This working mechanism achieves efficient support for different convolution step sizes and sparse data inputs through simple address judgment logic.

[0093] refer to Figure 9 The control unit 21 (PE Ctrl.) in the computing unit 2 selects the corresponding partial sum register in the partial sum register stack by using the information (Info., including the address and index information of the eigenvalues ​​and weights) input synchronously with the eigenvalues ​​and weights to obtain the partial sum (Psum) to be accumulated. The partial sum (Psum) to be accumulated is 24 bits and is stored back in the partial sum register selected by Info. The multiplication and addition unit is used to multiply the synchronous input eigenvalues ​​(Act.) and weights (Wei.), and then add the result to the partial sum (Psum) to obtain the partial sum result.

[0094] When the control unit 21 (PE Ctrl.) determines that the partial sum accumulation is completed, the output quantization unit 22 (OutScaler) quantizes the partial sum that has been accumulated and outputs it. The output quantization unit 22 (Out Scaler) uses the configured quantization scaling factor scale q and bias q The 24-bit partial sum is quantized to an 8-bit output (Out) and then passed to the global memory 1.

[0095] The output quantization unit 22 (Out Scaler) inside the calculation unit 2 quantizes the 24-bit partial sum to 8 bits through high-bit-width multiplication and addition calculations. The quantization process formula is:

[0096] y q =round(scale q ×y-bias q );

[0097] Among them, scale q is the quantization scaling factor, bias q is the offset, round is the rounding operation, y is the data to be quantized, y q The output quantization unit 22 can also support batch normalization (BatchNorm). BatchNorm is a technology used in deep neural networks that accelerates the training process and improves the stability of the model by normalizing the input distribution of each layer. Its operation formula is as follows:

[0098]

[0099] Among them, μ(x) and σ(x) are the mean and variance of the data respectively, x is the data to be normalized, y bn is the normalization result, γ and β are learnable scaling factors and translation factors respectively. Since the BatchNorm operation can also be completed through a signed addition and multiplication like the quantization operation, the output quantization unit 22 also supports the BatchNorm operation. In the data processing stage of neural network parameter adaptation, by integrating the mean and variance of BatchNorm into the quantization scaling factor (Scale) and bias (Bias) of the current convolutional layer, the quantization and BatchNorm operations can be completed simultaneously in a single multiplication and addition operation. Therefore, support for BatchNorm operations will not introduce any additional area and power consumption overhead in hardware. The only impact is the slight quantization error introduced by the fusion of two quantizations into one quantization. This error has almost no effect on the inference result under high-bit precision quantization. The fusion process formula is:

[0100] y bn+q =round(scale bn+q ×y-bias bn+q );

[0101]

[0102] Among them, scale bn+q and bias bn+q They are the quantization scaling factor and bias of BatchNorm, round is the rounding operation, and y bn+q Represents the quantization result that incorporates BatchNorm. In order to maintain high quantization accuracy, the output quantization unit 22 in the PE uses 24-bit fixed-point numbers to represent the scaling factor and bias. This requires the output quantization unit 22 to be able to perform multiplication and addition operations of 24-bit partial sums, 24-bit Scale, and 24-bit Bias. However, directly implementing high-bit-width multiplication and addition operations requires high-cost hardware resources (24×24-bit multipliers and 48-bit adders), which will result in a significant increase in the area and power consumption of the PE. In order to solve this problem, an embodiment of the present application proposes an operation architecture reconstruction scheme based on bit width decoupling: the original single-cycle 24×24-bit multiplication and addition operation is decomposed into a multi-stage pipeline structure consisting of 3 24×8-bit multiplications and 4 48-bit additions. In the specific implementation, a single 24×8-bit multiplier uses a time-division multiplexing mechanism to sequentially process the product calculation of the high, medium, and low 8-bit segments and partial sums of the quantization scaling factor Scale in the configuration parameters or the quantization scaling factor integrated with BatchNorm to obtain an intermediate result, and cooperates with a 4-cycle accumulator to complete the accumulation of the bias Bias in the configuration parameters or the bias integrated with BatchNorm and the intermediate result. This architectural improvement significantly reduces the bit width of the multiplier, replacing the original 24×24-bit multiplier with a 24×8-bit multiplier, thereby significantly optimizing the hardware cost. Although the operation cycle is extended from 1 to 4, since the output quantization operation is located at the end of the convolution calculation, and the convolution operation itself (except for the special case where the convolution kernel is 1 and the number of channel inputs is 1) requires multiple cycles to complete, the increased timing overhead of the output quantization stage can be effectively absorbed by the system-level pipeline. Actual measurement results show that in typical working scenarios, the reconstructed output quantization unit 22 will not become the throughput bottleneck of the control unit 21.

[0103] The computing unit array proposed in the embodiment of the present application supports two working modes: dense computing mode and sparse computing mode, which correspond to the dense access mode and sparse access mode of the access unit 3 respectively. In dense computing mode, the row computing unit 2 and column computing unit 2 in the computing unit array are strictly synchronized. For 8-bit quantized eigenvalues ​​and weights, the maximum read bandwidth of the computing unit array is 32 bits / cycle (i.e., 4 rows or 4 columns of computing units 2 access data at the same time: 4×8 bits / cycle). In sparse computing mode, the computing unit array achieves energy efficiency optimization by skipping the data reading and calculation operations related to zero-value eigenvalues, but this will cause the operation of PE units between columns to become asynchronous. In this mode, weights cannot be shared directly through the row computing unit 2, and local data sharing must be achieved by relying on the cache mechanism of the access unit 3. At this time, the maximum read bandwidth of the eigenvalue is still 32 bits / cycle, while the peak read bandwidth of the weight may reach 128 bits / cycle in extreme cases (such as when the weight cache function is turned off or the cache is completely missed) (16 computing units 2 need to be supported to read weights in parallel).

[0104] The following is an explanation of the technical effects of the technical solutions of the embodiments of the present application through specific experiments.

[0105] The general one-dimensional convolutional neural network accelerator proposed in the embodiment of the present application is implemented using Verilog HDL and synthesized by the Cadence Genus tool. The TSMC 65-nm LP CMOS process technology is used in the synthesis process, the operating voltage is set to 1.08V, and the synthesis frequency is set to 100MHz. After the synthesis is completed, Cadence NC-Sim is used to generate a TCF (Toggle Count Format) file, which is obtained by simulating the synthesized circuit and can accurately reflect the flip information of the circuit. Based on the TCF file and the synthesized netlist, Cadence Genus further accurately evaluated the power consumption of the accelerator. For the evaluation of power consumption, the experiment is based on 8-bit quantized PDNet and the public data set CHB-MIT. The experimental results show that the area of ​​the designed accelerator is 894.1μm 2 , the power consumption is 20.7mW in typical working scenarios.

[0106] The general one-dimensional convolutional neural network accelerator proposed in the embodiments of the present application supports one-dimensional convolution operations of various types and parameters. In order to evaluate the impact of convolution type and configuration parameters on the performance of the accelerator, the embodiments of the present application compare the computational efficiency of the accelerator in dense working mode for different convolution types and their configuration parameter combinations. The experiment adopts the configuration parameters of the convolution layer shown in Table 1, covering various convolution types such as conventional one-dimensional convolution, depth-wise separable convolution (including depth-wise one-dimensional convolution and point-wise one-dimensional convolution) and dilated one-dimensional convolution, while considering the influence of key parameters such as convolution kernel size, step size, dilation factor, number of input channels, number of output channels, eigenvalue length and task division strategy. It is particularly noted that through specific configuration parameters, conventional one-dimensional convolution operations can also realize traditional matrix multiplication. For example, the conventional convolution configuration shown in L9 in Table 1 is equivalent to the matrix multiplication operation of 256×4 matrix A and 8×256 matrix B.

[0107] Table 1 Configuration parameters of the convolutional layer used in the experiment

[0108]

[0109]

[0110] Figure 10 The computational efficiency of the accelerator under different convolutional layer configuration parameters is shown. The calculation formula for computational efficiency is defined as follows:

[0111]

[0112] The theoretical computation time is calculated by dividing the total number of multiplication-add operations in the convolutional layer by the total number of computational elements (PEs) in the PE array. The actual computation time is obtained through post-synthesis simulation. When the computational efficiency reaches 100%, it indicates that all PEs in the PE array are fully utilized, achieving the theoretical peak performance. When the computational efficiency is lower than 100%, it indicates that there is varying degrees of idle PE resources.

[0113] Figure 10The results show that under the configuration parameters of the convolution layer given in the experiment, the computational efficiency of the accelerator proposed in the embodiment of the present application is higher than 95%. Only when the convolution loop switches (the initial stage needs to wait for data loading, and the end stage needs to wait for the last data inside the PE to complete the quantization output), a small amount of bubbles will appear in the computing pipeline. The length of the bubble is fixed, and the number is related to the number of convolution cycles. For the convolution layer L1 with a large number of multiplication and addition operations, the bubble duration accounts for a low proportion, so the computational efficiency of this layer is relatively high, reaching 99.8%. The computational efficiency of the convolution layers L4 and L9 with a large number of convolution loop switches and a small total amount of multiplication and addition operations is relatively low. The experimental results show that the accelerator proposed in the embodiment of the present application shows high computational efficiency under the configuration parameters of different convolution layers.

[0114] The accelerator proposed in the embodiments of the present application supports two operating modes: dense computing mode and sparse computing mode. In sparse computing mode, the computing unit array skips the number acquisition and calculation operations corresponding to zero-valued eigenvalues. The specific operating mode adopted depends on the sparsity of the eigenvalues ​​of the current convolutional layer. In order to evaluate the impact of eigenvalue sparsity, the embodiments of the present application compare the computing speed and energy efficiency performance of the accelerator under different eigenvalue sparsities.

[0115] Because the access unit utilizes a zero-skipping technique based on eigenvalue tags, it first reads the eigenvalue tags and then determines whether to read the eigenvalue based on the tag's contents. Under this mechanism, the efficiency of eigenvalue tag reading becomes a key factor limiting computational speed. Specifically, the eigenvalue tag uses a single bit of "1" or "0" to indicate whether the corresponding eigenvalue is non-zero. Each time the access unit reads an N-bit eigenvalue tag, it simultaneously obtains the non-zero state information for N eigenvalues. Since the access unit only needs to read non-zero eigenvalues, if the eigenvalue sparsity is high and the N-bit eigenvalue tags currently being read are all zero, the access unit must pause the access operation during that cycle. This can cause the PE array to idle due to a lack of valid eigenvalues. Theoretically, a larger value of N results in a higher speedup achieved with zero-skipping. However, increasing the value of N is subject to practical limitations: as the value of N increases, the amount of eigenvalue information that needs to be spliced ​​increases, significantly increasing the complexity of eigenvalue tag generation. To explore the impact of data eigenvalue tag reading on computational speed, the first experiment tested point convolution, which places the greatest strain on eigenvalue tag reading. The configuration parameters of other convolutions in the experimental settings are as follows: the convolution step size is 1, the number of input channels is 64, the number of output channels is 256, and the eigenvalue length is 1024. Figure 11 and 12 The computational efficiency of the accelerator and the corresponding speedup ratio (convolution kernel size is 1) are shown under different bit widths of feature value labels (N=2, 4, 8, 16) as the sparsity changes.

[0116] from Figure 11 and 12 It can be seen that with the increase of eigenvalue sparsity, due to the limitation of eigenvalue label reading bandwidth, the computing efficiency of the accelerator shows a downward trend, and its speedup ratio gradually deviates from the theoretical optimal value. This phenomenon is particularly significant when the eigenvalue label bit width N is small. However, this problem can be effectively alleviated by increasing the convolution kernel size. In the data stream with input channel priority and fixed output proposed in the embodiment of the present application, each computing unit (PE) needs to multiply it with the corresponding W weights after obtaining an eigenvalue (W is the convolution kernel size). Therefore, when the W value increases, the bubbles generated by the sparse eigenvalue reading will be masked by the weight reading operation, thereby reducing the impact of the eigenvalue label reading limitation on the accelerator computing efficiency. In order to verify this phenomenon, the second experiment selected a convolution layer with a convolution kernel size of 3 for testing, and the configuration parameters of the remaining experiments remained consistent with the first experiment. Figure 13 and 14 It further demonstrates the computational efficiency of the accelerator and the corresponding speedup ratio (convolution kernel size is 3) as the sparsity changes under different feature value label bit widths (N=2, 4, 8, 16).

[0117] from Figure 13 and 14 It can be seen from the experimental results that when the convolution kernel size is 3, even if the eigenvalue sparsity reaches 90%, if the eigenvalue label bit width N is greater than or equal to 4, the computing unit array can maintain a computing efficiency greater than 90%. This result shows that, except for the special case of point convolution (convolution kernel size is 1), the accelerator proposed in the embodiment of this application can maintain a high computing efficiency in most practical application scenarios. By effectively utilizing the sparse characteristics of the input data, the architecture can achieve an acceleration ratio close to the theoretical value, fully demonstrating its advantages in sparse computing tasks.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A general one-dimensional convolutional neural network accelerator, characterized in that: It includes a global memory and a computing unit array, the computing unit array includes a data acquisition unit and 16 computing units arranged in a 4×4 array, the data acquisition unit includes an address generator, a data information generation logic module and a weight data cache; the configuration parameters of the one-dimensional convolutional neural network and the feature base address, weight base address and label base address used in the calculation process of the one-dimensional convolutional neural network stored in the global memory are input into the computing unit array and first pass through the data acquisition unit; the configuration parameters include convolution parameters, data acquisition mode and task division method; in the data acquisition unit, the address generator is used to generate a data base address according to the convolution parameters and the data acquisition loop sequence, the data base address is added to the label base address, the feature base address and the weight base address respectively to obtain a label reading address, a feature value reading address and a weight reading address; the data information generation logic module selects whether to read the corresponding feature value label according to the label reading address according to the data acquisition mode, and selects whether to read the corresponding feature value label according to the feature The value tag and the data access mode determine whether to send a characteristic value read request to the global memory, generate characteristic value information and weight information according to the data base address, and use the weight information to determine whether to use the weight read address to obtain the corresponding weight in the weight data cache or generate a weight read request and send it to the global memory to read the corresponding weight. The global memory is used to read the corresponding characteristic value and weight using the characteristic value read address and the weight read address after receiving the characteristic value read request and the weight read request and send them to the data information generation logic module. The data information generation logic module is used to send the characteristic value together with the characteristic value information to the calculation unit of each column, and send the corresponding weight obtained in the weight data cache together with the weight information to the calculation unit of each row. The calculation unit performs multiplication and accumulation operations on the characteristic value and weight, and performs partial and cache and quantized output operations to obtain the multiplication and accumulation results of the quantized output, and form a data stream with input channel priority and fixed output.

2. The universal one-dimensional convolutional neural network accelerator according to claim 1, characterized in that The convolution parameters include convolution kernel length W, input channel number I, input eigenvalue length P, output eigenvalue length Q, dilation factor D, output channel number O, and whether batch normalization is used. The convolution types of the one-dimensional convolutional neural network include conventional one-dimensional convolution, depth-wise one-dimensional convolution, point-by-point one-dimensional convolution, and dilated one-dimensional convolution, wherein the depth-wise one-dimensional convolution and point-by-point one-dimensional convolution are achieved by adjusting the convolution kernel length W or the input channel number I in the acquisition loop; the calculation loop order of the conventional one-dimensional convolution from inside to outside is convolution kernel length W, input channel number I, input eigenvalue length P, and dilated one-dimensional convolution. P, output eigenvalue length Q and output channel number O, the corresponding number acquisition loop sequence contains four nested loops, namely the convolution kernel length loop, the input channel number loop, the input eigenvalue loop and the output channel number loop, the calculation loop sequence of the extended one-dimensional convolution is, from inside to outside, the convolution kernel length W, the input channel number I, the input eigenvalue length P, the output eigenvalue length Q, the dilation factor D and the output channel number O, the corresponding number acquisition loop sequence contains five nested loops, namely the convolution kernel length loop, the input channel number loop, the input eigenvalue loop, the dilation factor loop and the output channel number loop.

3. The universal one-dimensional convolutional neural network accelerator according to claim 1, characterized in that In the process of generating the data base address, the convolution type of the one-dimensional convolutional neural network is determined to be conventional one-dimensional convolution or extended one-dimensional convolution according to the convolution parameters. If the convolution type of the one-dimensional convolutional neural network is extended one-dimensional convolution and the dilation factor is D, the address increment step in the input eigenvalue loop is set to D. The data acquisition mode is determined by the sparsity of the eigenvalue used in the calculation process of the one-dimensional convolutional neural network. If the sparsity is less than a threshold, the data acquisition mode adopts a sparse data acquisition mode. The amount of data read in the sparse data acquisition mode is calculated using the following formula: Bit read =1×sparsity+9×(1-sparsity); Among them, sparsity represents sparsity, which is the number of zero eigenvalues ​​divided by the total number of eigenvalues. read Indicates the amount of data read in the sparse access mode.

4. The universal one-dimensional convolutional neural network accelerator according to claim 1, characterized in that The data information generation logic module includes a sparse mode control unit, a label buffer, an address buffer, an information generator and a feature value buffer, wherein the address buffer is connected to the information generator and is used to temporarily store the data base address, the information generator is used to generate feature value information and weight information according to the data base address, and the feature value buffer is used to temporarily store the feature value; The data acquisition mode includes a sparse data acquisition mode and a dense data acquisition mode. In the dense data acquisition mode, the row calculation units and the column calculation units in the calculation unit array maintain synchronous calculation. In the sparse data acquisition mode, except for the eigenvalue of the last input channel in the last input feature map and its corresponding last weight, the data acquisition unit skips reading the eigenvalues ​​and weights corresponding to the eigenvalue label of 0; in the current cycle, the sparse mode control unit is used to generate a label read request according to the label read address and determine whether to send it to the global memory according to the data acquisition mode. If the data acquisition mode is the sparse data acquisition mode, the label read request is sent to the global memory; if the data acquisition mode is the dense data acquisition mode, the label read request is not sent to the global memory; in the next cycle, the global memory returns the corresponding eigenvalue label after receiving the label read request and temporarily stores it in the label buffer.

5. The universal one-dimensional convolutional neural network accelerator according to claim 4, characterized in that Determining whether to send a feature value read request to the global memory according to the feature value tag and the data acquisition mode specifically includes: generating a characteristic value read request based on the characteristic value read address, and after generating the characteristic value read request, reading a corresponding characteristic value tag from the tag buffer; When the eigenvalue label is 1, or the current data acquisition mode is a dense data acquisition mode, or it is determined according to the eigenvalue information that the eigenvalue is the eigenvalue of the last input channel in the last input feature map, a eigenvalue read request is sent to the global memory; When the eigenvalue label is 0, and the current data acquisition mode is a sparse data acquisition mode, and it is determined based on the eigenvalue information that the eigenvalue is not the eigenvalue of the last input channel in the last input feature map, no eigenvalue read request is sent to the global memory.

6. The universal one-dimensional convolutional neural network accelerator according to claim 1, characterized in that Using the weight information to determine whether a corresponding weight is obtained in the weight data cache or generating a weight read request and sending it to the global memory to read the corresponding weight, specifically includes: If the eigenvalue label is 1, or the current data acquisition mode is a dense data acquisition mode, or it is determined according to the weight information that the weight is the last weight corresponding to the eigenvalue of the last input channel in the last input feature map, the weight in the weight data cache is retrieved through the weight information. If the weight corresponding to the weight information is retrieved, the corresponding weight is obtained in the weight data cache, and no weight read request is generated; if the weight corresponding to the weight information cannot be retrieved, a weight read request is generated and sent to the global memory, the corresponding weight is read from the global memory and temporarily stored in the weight data cache, and the weight is read in the weight data cache. The weight data cache adopts a first-in-first-out (FIFO) cache replacement strategy.

7. The universal one-dimensional convolutional neural network accelerator according to claim 1, characterized in that: The task division method includes task division based on inter-channel parallelism and task division based on intra-channel parallelism. During the generation process of the data base address, the selection results of the weights corresponding to the left and right boundaries of the input eigenvalues ​​are adjusted by the task division method, and it is determined whether to splice the convolution calculation results obtained by the computing unit according to the task division method to obtain the multiplication and accumulation results; if the current task division method is task division based on inter-channel parallelism, it is only necessary to adjust the selection results of the weights corresponding to the left and right boundaries of the input eigenvalues, and the weights only need to obtain the weights within the coverage range of the convolution sliding window; if the current task division method is task division based on intra-channel parallelism and the convolution kernel size is not 1, then it is determined that the left and right boundaries of the input eigenvalues ​​are real boundaries or boundaries that need to be spliced. If they are boundaries that need to be spliced, additional eigenvalues ​​are obtained from the sub-segment spliced ​​with the current sub-segment to reach the required number within the coverage range of the convolution sliding window, and the weights only need to obtain the weights within the coverage range of the convolution sliding window.

8. The universal one-dimensional convolutional neural network accelerator according to claim 7, characterized in that: The convolution sliding window coverage range is the length of the convolution kernel divided by 2 and rounded down.

9. The universal one-dimensional convolutional neural network accelerator according to claim 2, characterized in that: The calculation unit includes a control unit, a multiplication and addition unit, a partial sum register stack, and an output quantization unit, wherein the partial sum register stack includes a plurality of partial sum registers; the control unit selects a corresponding partial sum register in the partial sum register stack based on the eigenvalue information and weight information input synchronously with the eigenvalue and weight, and obtains the partial sum to be accumulated; the multiplication and addition unit is used to multiply the synchronously input eigenvalue and weight, and accumulate the multiplication result with the partial sum to be accumulated to obtain a partial sum result, and store the partial sum result back in the selected partial sum register; The multiplication and addition unit includes a 24×8-bit multiplier and a 48-bit accumulator, which completes three 24×8-bit multiplication operations and four 48-bit addition operations through time division multiplexing; When the control unit determines that the partial sum accumulation is completed, the accumulated partial sum result is quantized and output through the output quantization unit. The output quantization unit is used to use the quantization scaling factor and bias in the configuration parameters or the quantization scaling factor and bias of the fused batch normalization to quantize the partial sum structure and output it to obtain a quantization result, and then send the quantization result to the global memory.

10. The universal one-dimensional convolutional neural network accelerator according to claim 1, characterized in that: The working modes of the computing unit include a dense working mode and a sparse working mode corresponding to a dense data acquisition mode and a sparse data acquisition mode respectively. The computing units in different rows in the same column share the same eigenvalues ​​for calculating convolution tasks of different output channels. The computing units in different columns in the same row share the same weights for calculating convolution tasks of different sub-segments. The number of output channels of the eigenvalues ​​used in the calculation process of the one-dimensional convolutional neural network is a multiple of 4.

Images