A method and system for detecting abnormal EEG signals in a long short-term memory network that can be efficiently quantized using FPGA hardware acceleration.

Abstract

This invention relates to an FPGA hardware acceleration method and system for an efficiently quantizable Long Short-Term Memory (LSTM) network and its abnormal EEG signal detection. The method includes: quantizing the efficiently quantizable LSM network and deploying it on an FPGA hardware-accelerated programmable logic (PL); subsequently compiling the quantized LSM network into Verilog code to generate an IP core for acceleration; transmitting input signals, quantization biases, and weights via an AXI bus; and after the quantized LSM network completes its calculations, transmitting the output data back to the FPGA hardware-accelerated ARM processor unit (PS) via the same AXI bus. This invention significantly reduces the memory footprint of the LSM network, lowers its computational power consumption, facilitates the deployment and efficient operation of LSM networks on low-power edge hardware devices, and promotes real-time processing and response.

Description

Technical Field

[0001] This invention relates to an efficient quantifiable FPGA hardware acceleration method for long short-term memory networks and a method and system for detecting abnormal electroencephalogram (EEG) signals, belonging to the fields of neural networks, artificial intelligence, and FPGA technology. Background Technology

[0002] Recurrent Neural Networks (RNNs), especially those based on Long Short-Term Memory (LSTM) units, are end-to-end trainable neural networks specifically designed for processing time series data. They have shown outstanding performance and wide application in time series analysis, natural language processing, and other fields, and are currently becoming a research hotspot in the field of electroencephalogram (EEG) signal analysis.

[0003] Compared to traditional time-series modeling methods, Long Short-Term Memory (LSTM) networks can better integrate and learn discriminative EEG features from raw EEG signals. However, LTM networks have a large number of parameters, and all parameters are typically stored as 32-bit or 16-bit floating-point numbers in computers, consuming excessive memory space and making them difficult to deploy in mobile phones or other low-power edge computing hardware. Currently, the main approach to solving this problem is to quantize the model parameters into low-bit-width integers for easy deployment on FPGA hardware. However, existing low-bit-width quantization methods for model parameters generally require training calibration during quantization or data calibration after quantization to maintain the original accuracy of the model, which is cumbersome and limits the deployment and application of LTM networks in FPGAs. Therefore, this paper proposes a novel LTM network that can be efficiently quantized and its FPGA hardware acceleration method to address the problems of LTM networks, and applies it to the detection of abnormal EEG signals. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes an efficient quantization method for FPGA hardware acceleration of Long Short-Term Memory (LSTM) networks. The invention proposes an LSM unit with normalized cell state output, facilitating LSM network quantization and avoiding data calibration operations in traditional post-quantization methods. It also proposes a quantization scheme for the nonlinear activation function of LSM units based on a Sigmoid piecewise function lookup table, using a piecewise function to approximate the nonlinear activation function, achieving precision-free quantization even with low quantization bit widths. Furthermore, it proposes an efficient quantization method for FPGA hardware acceleration of LSM networks.

[0005] This invention also proposes a method and system for detecting abnormal electroencephalogram (EEG) signals based on a FPGA hardware accelerator with highly efficient quantization of long short-term memory networks.

[0006] Terminology Explanation:

[0007] 1. Quantization: The process of mapping a set of parameters to another value range through mathematical transformation. Linear mapping is typically used to achieve this transformation. The original values ​​of the parameters are usually floating-point numbers, while the quantized parameter values ​​are generally integers.

[0008] 3. Training: Input a set of data into the neural network, compare the output with the corresponding label of the data and calculate the error, obtain the gradient value of each parameter through the backpropagation algorithm and update it.

[0009] 4. Long Short-Term Memory (LSTM): LSTM is a special form of recurrent neural network. Each LSTM consists of three gates: a forget gate (determines which information is discarded), an input gate (determines which new information is updated to the unit state), and an output gate (determines the next hidden state). These gating structures enable long-distance state retention and propagation between LSTM units, effectively learning and remembering long-distance dependencies.

[0010] 5. Long Short-Term Memory (LSTM) Networks: These are neural networks composed of multiple LSM units connected in a specific order, used to process, predict, and classify time-series data. LSM networks maintain and update information in each unit through their unique gating mechanism, allowing the network to retain and transmit information across units over long distances when processing time-series data.

[0011] 6. EEG Signal Feature Extraction Based on Convolutional Neural Networks: This method utilizes convolutional neural networks (CNNs) to extract features from EEG signals. The CNN contains one or more convolutional layers that receive raw EEG signals as input and automatically extract features through the network's convolutional, activation, and pooling layers.

[0012] The technical solution of the present invention is as follows:

[0013] A method for efficiently quantizing long short-term memory (LSTM) networks using FPGA hardware acceleration is disclosed. The method operates within an FPGA hardware accelerator, which includes an ARM processor unit (PS) and a programmable logic unit (PL).

[0014] A highly quantizable long short-term memory network consists of multiple sequentially connected highly quantizable long short-term memory units;

[0015] The trained Long Short-Term Memory (LSTM) network with floating-point parameters is quantized to obtain the quantized LSM network.

[0016] The quantized Long Short-Term Memory network is compiled into Verilog code and used to generate IP cores for acceleration, which are then deployed in programmable logic units (PLs).

[0017] The ARM processor unit PS is responsible for data preparation and preprocessing, as well as softmax mapping operations;

[0018] The configuration parameters of each unit of the quantized Long Short-Term Memory network, including the number of input / output channels, feature vector dimension, number of neurons, and quantization coefficients, are first transmitted to the programmable logic unit (PL) via a simplified version of the Advanced Extension Interface (AXI Lite) bus at the ARM processor unit PS.

[0019] The input signal, quantization bias, and weights are transmitted from the PS of the ARM processor unit to the PL via the Advanced Extension Interface (AXI) bus; after the quantized Long Short Time Memory unit completes the calculation at the PL of the programmable logic unit, the output data is returned to the PS of the ARM processor unit via the same AXI bus.

[0020] Configure the parameters of the next quantized Long Short-Term Memory (LSTM) unit, start a new round of feature vector calculation, and repeat this process until the entire quantized LSM network is calculated.

[0021] Output the calculation results.

[0022] According to a preferred embodiment of the present invention, the segmented activation function of the efficiently quantizable long short-term memory unit...

[0023]

[0024] Among them, T a For an adjustable approximation parameter of the Sigmoid function;

[0025] Further preferred, T a =2.5;

[0026] According to a preferred embodiment of the present invention, the calculation process for the cell state parameters in the efficiently quantifiable long short-term memory unit is as follows:

[0027]

[0028] Where ⊙ is the Hadamard product, c t For the cell state at time step t in a long short-term memory unit that can be efficiently quantified, i t f t g t These represent the input gate gating value, forget gate gating value, and cell candidate unit value at the t-th time step in a quantifiable long short-term memory unit, respectively.

[0029] According to a preferred embodiment of the present invention, the parameter quantization process of a trained, efficiently quantizable Long Short-Term Memory network with floating-point parameters includes:

[0030] (1) Initialize the quantization parameters, including:

[0031] Let the overall weight matrix W = [W f W i W o W g ], then W

[0032] Let the overall cyclic weight matrix R = [R f ;R i ;R o ;R g ],but

[0033] Initialize all weighted bit widths B WR =8;

[0034] Initialize the segmentation function lookup table quantization bit width B LUT =4;

[0035] Initialize fixed-point bit width B Fix =24;

[0036] Initialize weight quantization factor

[0037] Initialize the weight matrix quantization scale Q W =max(|W|) / M WR ;

[0038] Initialize the cyclic weight matrix quantization scale Q R =max(|R|) / M WR ; where max(·) is the function to find the maximum value, and |·| is the function to find the absolute value;

[0039] Initialize input quantization scale Q X =1 / M WR ;

[0040] Initialize hidden unit quantization scale Initialize input weight quantization scale Initialize input loop weight quantization scale Initialize candidate cell quantization scale

[0041] (2) Generate a quantized lookup table; including:

[0042] First, with -T a The initial value is T. aTo determine the termination value, set the increment (i.e., the step size) to... Generate arithmetic sequence I d Then, calculate the piecewise function. Quantization lookup table:

[0043] (3) Quantization weight matrix; including:

[0044] The quantization forget gate weight matrix is ​​shown in equation (3):

[0045]

[0046] The quantization input gate weight matrix is ​​shown in equation (4):

[0047]

[0048] The quantization output gate weight matrix is ​​shown in equation (5):

[0049]

[0050] The quantified cell candidate weight matrix is ​​shown in equation (6):

[0051]

[0052] The quantized forget gate loop weight matrix is ​​shown in equation (7):

[0053]

[0054] The quantized input gate recurrent weight matrix is ​​shown in equation (8):

[0055]

[0056] The quantized output gate loop weight matrix is ​​shown in equation (9):

[0057]

[0058] The quantified cell candidate cycle weight matrix is ​​shown in Equation (10):

[0059]

[0060] The quantization of the forget gate bias is shown in equation (11):

[0061]

[0062] The quantization input gate bias is shown in equation (12):

[0063]

[0064] The quantization output gate bias is shown in equation (13):

[0065]

[0066] The cell candidate bias is quantified as shown in Equation (14):

[0067]

[0068] (4) Calculate the quantized gating and state parameter values; including:

[0069] Calculate the quantification forgetting gate gating value:

[0070]

[0071] Calculate the quantization input gate gating value:

[0072]

[0073] Calculate the gating value of the quantized output gate:

[0074]

[0075] Calculate and quantify candidate cell values:

[0076]

[0077] Calculate and quantify cell state parameters:

[0078]

[0079] Calculate the quantized hidden state parameter values:

[0080]

[0081] A highly efficient quantizable long short-term memory (LSTM) network FPGA hardware accelerator includes an ARM processor unit PS and a programmable logic unit PL connected via an Advanced Extension Interface (AXILite) bus, enabling a highly efficient quantizable LSM network FPGA hardware acceleration method.

[0082] A method for detecting abnormal electroencephalogram (EEG) signals based on an FPGA hardware accelerator with highly efficient quantizable long short-term memory (LSTM) networks, comprising:

[0083] The data acquisition module, consisting of an EEG amplifier and an A / D converter, is used to acquire the EEG signals to be detected and store them in a computer.

[0084] Training a long short-term memory network that can be efficiently quantized;

[0085] The trained, efficiently quantizable Long Short-Term Memory (LSTM) network is deployed on an FPGA hardware accelerator for efficiently quantizable LSM networks.

[0086] Feature extraction is performed on the EEG signals to be detected;

[0087] The features of the EEG signal to be detected are input into a FPGA hardware accelerator for efficient quantization of long short-term memory networks to obtain the output value;

[0088] The output value of the FPGA hardware accelerator with efficient quantization is dequantized to obtain the dequantized floating-point features, and the results (abnormal EEG or normal EEG) are output after being mapped by the softmax function.

[0089] If the test result is abnormal EEG, an alarm will be triggered via the alarm module.

[0090] According to a preferred embodiment of the present invention, the training process of an efficiently quantifiable long short-term memory network includes:

[0091] First, EEG data for training is acquired through a data acquisition module consisting of an EEG amplifier and an A / D converter;

[0092] Secondly, the floating-point parameters and weight matrices of each unit are initialized based on the set number of internal units of the long short-term memory network that can be efficiently quantized, including:

[0093] The cell state parameters at the initial time step t=0 Initialize all values ​​to 0;

[0094] The hidden state parameters at the initial time step t=0 Initialize all values ​​to 0;

[0095] Forget gate weight matrix Input gate weight matrix Output gate weight matrix Cell candidate weight matrix Initialize to random numbers;

[0096] Forget gate loop weight matrix Input gate loop weight matrix Output gate loop weight matrix Cell candidate cycle weight matrix Initialize to random numbers;

[0097] Bias the forget gate Input gate bias Output gate bias Cell candidate bias All parameters are initialized to 0; then, based on the parameters and weight matrix, the values ​​of each gate and state parameter are calculated, including:

[0098] For the extracted EEG signal features, the data at time step t is... t Calculate the floating-point parameter values ​​for each gate according to the following formula:

[0099] Calculate the forgetting gate gating value f t As shown in equation (21):

[0100]

[0101] Among them, h t-1 Let be the hidden state parameter at time step t-1;

[0102] Calculate the input gate gating value i t As shown in equation (22):

[0103]

[0104] Calculate the output gate gating value o t As shown in equation (23):

[0105]

[0106] Calculate the cell candidate unit value g t As shown in equation (24):

[0107]

[0108] Calculate the cell state parameter c t As shown in equation (25):

[0109]

[0110] Calculate the hidden state parameter h t As shown in equation (26):

[0111] h t =o t ⊙c t (26)

[0112] Next, the loss function of the efficiently quantizable long short-term memory network, which consists of multiple sequentially connected efficiently quantizable long short-term memory units, is calculated. Based on the value of the loss function, the floating-point parameters are iteratively updated multiple times using the backpropagation algorithm. After reaching the maximum number of iterations, the efficiently quantizable long short-term memory network stops iteratively updating, and the floating-point parameter values ​​of each parameter and weight matrix are fixed and stored.

[0113] According to a preferred embodiment of the present invention, the floating-point characteristics after dequantization are calculated, as shown in equation (27):

[0114]

[0115] in: Q represents the quantized integer feature of the output of a Long Short-Term Memory (LSTM) network at time step t, which can be efficiently quantized. X Use the input quantization scale.

[0116] According to a preferred embodiment of the present invention, calculating the loss value includes:

[0117] Based on the output hidden layer features h t The current loss value is calculated using the loss function E; the loss function E is defined as follows:

[0118]

[0119] Where θ represents all learnable parameters in a long short-term memory network that can be efficiently quantized; related to θ Let represent the j-th feature value of the hidden layer at time step t for the i-th sample, and m represent the number of samples in the backpropagation optimization process;

[0120] According to a preferred embodiment of the present invention, parameter updating includes:

[0121] The calculated loss value is used to update all learnable parameters in the efficiently quantizable Long Short-Term Memory network according to Equation (29):

[0122]

[0123] Where μ is the learning rate; θ v This represents all learnable parameters in the Long Short-Term Memory network that can be efficiently quantized at the v-th iteration. It is the loss function E(θ) v ) for θ v The gradient value; if v = N max N max =200, N max If the maximum number of iterations is not set, the network stops iterating and the parameter values ​​of the weight matrix are fixed; otherwise, v is incremented by 1 and the calculation of the gating and state parameter values ​​continues, and iterative updates are performed.

[0124] According to a preferred embodiment of the present invention, the electroencephalogram (EEG) signal feature extraction process includes:

[0125] A single-layer convolutional neural network is used for EEG signal feature extraction. This single-layer convolutional neural network includes eight single-channel one-dimensional convolutional kernels with a length of 5. This single-layer convolutional neural network maps the raw EEG signal at each time step into EEG signal features with a dimension of 1024.

[0126] An abnormal EEG signal detection system based on an FPGA hardware accelerator with highly efficient quantizable long short-term memory networks, comprising:

[0127] The data acquisition module is configured to acquire the EEG signal to be detected through an EEG amplifier and an A / D converter;

[0128] The feature extraction module is configured to: extract features from the EEG signal to be detected and map the original EEG signal into a feature vector of a certain dimension;

[0129] The abnormal EEG signal detection module is configured to: input the feature vector into the FPGA hardware accelerator that can be efficiently quantized, dequantize the output value of the FPGA hardware accelerator to obtain the dequantized floating-point feature, and output the detection result (abnormal EEG or normal EEG) after mapping by the softmax function.

[0130] The abnormal EEG alarm module is configured to trigger an alarm based on the category label output by the abnormal EEG signal detection module.

[0131] The beneficial effects of this invention are as follows:

[0132] Quantizing the floating-point data format parameters of an efficiently quantizable Long Short-Term Memory (LSTM) network into low-bit-width signed integer parameters significantly reduces the network's memory footprint and power consumption. This facilitates the deployment and efficient operation of LSM networks on low-power edge hardware devices, promoting real-time processing and response. Furthermore, the efficiently quantizable LSM network proposed in this invention eliminates the need for data calibration in traditional quantization methods, completing quantization without any additional data, thus enhancing the flexibility of LSM network quantization. Attached Figure Description

[0133] Figure 1 A schematic diagram of an FPGA hardware accelerator for a highly quantizable long short-term memory network;

[0134] Figure 2 This is a schematic diagram of the structure of the efficient quantizable long short-time memory unit with floating-point parameters of the present invention;

[0135] Figure 3 This is a schematic diagram of the structure of the efficient quantizable long short-term memory unit after parameter quantization in this invention;

[0136] Figure 4 This is a schematic diagram of a highly quantizable long short-term memory network structure consisting of multiple sequentially connected, highly quantizable long short-term memory units.

[0137] Figure 5 This is a schematic diagram of the training process of the Long Short-Term Memory Network of the present invention;

[0138] Figure 6 A schematic diagram illustrating the accuracy of 10-fold cross-validation under different quantization bit widths;

[0139] Figure 7 This is a schematic diagram of an abnormal EEG signal detection system based on an FPGA hardware accelerator for a long short-term memory network with high efficiency quantization. Detailed Implementation

[0140] The present invention will be further described below with reference to the accompanying drawings and examples, but the present invention is not limited thereto;

[0141] Example 1

[0142] An efficient quantizable long short-term memory (LSTM) network FPGA hardware acceleration method, running within an FPGA hardware accelerator, such as... Figure 1 As shown, the FPGA hardware accelerator uses the Xilinx Zynq Zedboard, which includes the ARM processor unit PS of the Zynq-7000 SoC and the 7-series programmable logic unit PL; including:

[0143] A highly quantizable long short-term memory network consists of multiple sequentially connected highly quantizable long short-term memory units;

[0144] The trained Long Short-Term Memory (LSTM) network with floating-point parameters is quantized to obtain the quantized LSM network.

[0145] The quantized Long Short-Term Memory network is compiled into Verilog code and used to generate IP cores for acceleration, which are then deployed in programmable logic units (PLs).

[0146] The ARM processor unit PS is responsible for data preparation and preprocessing, including receiving input data, loading weights and configuring them; it is also responsible for softmax mapping operations.

[0147] The configuration parameters of each unit of the quantized Long Short-Term Memory Network, including the number of input / output channels, feature vector dimension, number of neurons, and quantization coefficients, are transmitted to the programmable logic unit (PL) via a simplified version of the Advanced Extension Interface (AXI Lite) bus at the ARM processor unit PS.

[0148] The input signal, quantization bias, and weights are transmitted from the PS of the ARM processor unit to the PL via the Advanced Extension Interface (AXI) bus; after the quantized Long Short Time Memory unit completes the calculation at the PL of the programmable logic unit, the output data is returned to the PS of the ARM processor unit via the same AXI bus.

[0149] Configure the parameters of the next quantized Long Short-Term Memory (LSTM) unit, start a new round of feature vector calculation, and repeat this process until the entire quantized LSM network is calculated.

[0150] Output the calculation results.

[0151] The quantized Long Short-Term Memory (LSTM) units share a quantization lookup table, which is stored in the Block Random Access Memory (BRAM) of the programmable logic unit (PL). All weight data is stored in the DDR-3 memory of the ARM processor unit (PS). Table 1 details the resource utilization of the designed efficient quantization-enabled LSM network FPGA hardware acceleration method running on the Xilinx Zynq Zedboard. The on-chip power consumption of the entire hardware acceleration system is 1.778W, of which the ARM processor unit (PS) consumes 1.542W and the programmable logic unit (PL) consumes 0.236W.

[0152] Table 1

[0153]

[0154] Example 2

[0155] The difference between the efficient quantization method for FPGA hardware acceleration of long short-term memory networks described in Example 1 and the method described in Example 1 is as follows:

[0156] Segmented activation function for efficiently quantizable long short-term memory units

[0157]

[0158] Among them, T a For an adjustable approximation parameter of the Sigmoid function;

[0159] T a =2.5;

[0160] The calculation process for cell state parameters in a highly quantifiable long short-term memory unit is as follows:

[0161]

[0162] Where ⊙ is the Hadamard product, c tFor the cell state at time step t in a long short-term memory unit that can be efficiently quantified, i t f t g t These represent the input gate gating value, forget gate gating value, and cell candidate unit value at time step t in an efficiently quantizable long short-term memory unit (LSTM). The structure of an efficiently quantizable LSM with floating-point parameters is as follows: Figure 2 As shown, the structure of an efficient quantizable long short-term memory unit with low-bit-width integer parameters is as follows: Figure 3 As shown, a highly quantizable Long Short-Term Memory (LSTM) network, composed of multiple sequentially connected, efficiently quantizable LSTM units, is as follows: Figure 4 As shown.

[0163] The difference between the aforementioned highly quantifiable long short-term memory network and the traditional long short-term memory network lies in:

[0164] The segmented activation function of the long short-term memory network in formula (1) has a smaller computational cost and is easier to quantize than the sigmoid activation function in the traditional long short-term memory network.

[0165] The efficient quantification process for cell state parameters in long short-term memory networks (LSTM) in formula (2) involves the use of piecewise activation functions compared to the traditional LSM calculation process. The activation process.

[0166] The parameter quantization process for a trained, efficiently quantizable Long Short-Term Memory (LSTM) network includes:

[0167] (1) Initialize the quantization parameters, including:

[0168] Let the overall weight matrix W = [W f W i W o W g ],but

[0169] Let the overall cyclic weight matrix R = [R f ;R i ;R o ;R g ],but

[0170] Initialize all weighted bit widths B WR =8;

[0171] Initialize the segmentation function lookup table quantization bit width B LUT =4;

[0172] Initialize fixed-point bit width B Fix =24;

[0173] Initialize weight quantization factor

[0174] Initialize the weight matrix quantization scale Q W =max(|W|) / M WR ;

[0175] Initialize the cyclic weight matrix quantization scale Q R =max(|R|) / M WR ; where max(·) is the function to find the maximum value, and |·| is the function to find the absolute value;

[0176] Initialize input quantization scale Q X =1 / M WR ;

[0177] Initialize hidden unit quantization scale

[0178] Initialize input weight quantization scale

[0179] Initialize input loop weight quantization scale

[0180] Initialize candidate cell quantization scale

[0181] (2) Generate a quantized lookup table; including:

[0182] First, with -T a The initial value is T. a To determine the termination value, set the increment (i.e., the step size) to... Generate arithmetic sequence I d ;

[0183] Then, calculate the piecewise function. Quantization lookup table: Here, Ld = {0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,64,67,71,75,79,83,87,91,95,99,103,107,111,115,119,123,127}

[0184] (3) Quantization weight matrix; including:

[0185] The quantization forget gate weight matrix is ​​shown in equation (3):

[0186]

[0187] The quantization input gate weight matrix is ​​shown in equation (4):

[0188]

[0189] The quantization output gate weight matrix is ​​shown in equation (5):

[0190]

[0191] The quantified cell candidate weight matrix is ​​shown in equation (6):

[0192]

[0193] The quantized forget gate loop weight matrix is ​​shown in equation (7):

[0194]

[0195] The quantized input gate recurrent weight matrix is ​​shown in equation (8):

[0196]

[0197] The quantized output gate loop weight matrix is ​​shown in equation (9):

[0198]

[0199] The quantified cell candidate cycle weight matrix is ​​shown in Equation (10):

[0200]

[0201] The quantization of the forget gate bias is shown in equation (11):

[0202]

[0203] The quantization input gate bias is shown in equation (12):

[0204]

[0205] The quantization output gate bias is shown in equation (13):

[0206]

[0207] The cell candidate bias is quantified as shown in Equation (14):

[0208]

[0209] (4) Calculate the quantized gating and state parameter values; including:

[0210] Calculate the quantification forgetting gate gating value:

[0211]

[0212] Calculate the quantization input gate gating value:

[0213]

[0214] Calculate the gating value of the quantized output gate:

[0215]

[0216] Calculate and quantify candidate cell values:

[0217]

[0218] Calculate and quantify cell state parameters:

[0219]

[0220] Calculate the quantized hidden state parameter values:

[0221]

[0222] Example 3

[0223] A highly efficient quantizable long short-term memory (LSTM) network FPGA hardware accelerator includes an ARM processor unit PS and a programmable logic unit PL of a Zynq-7000 SoC connected via an Advanced Extension Interface (AXILite) bus, enabling a highly efficient quantizable LSM network FPGA hardware acceleration method.

[0224] Example 4

[0225] A method for detecting abnormal electroencephalogram (EEG) signals based on an FPGA hardware accelerator with highly efficient quantizable long short-term memory (LSTM) networks, comprising:

[0226] The EEG signal to be detected is acquired using a data acquisition module consisting of an EEG amplifier and an A / D converter;

[0227] Training a long short-term memory network that can be efficiently quantized;

[0228] The trained, efficiently quantizable Long Short-Term Memory (LSTM) network is deployed on the FPGA hardware accelerator for the efficiently quantizable LSTM network described in Example 3.

[0229] Feature extraction is performed on the EEG signals to be detected;

[0230] The features of the EEG signal to be detected are input into a FPGA hardware accelerator for efficient quantization of long short-term memory networks to obtain the output value;

[0231] The output value of the FPGA hardware accelerator with efficient quantization is dequantized to obtain the dequantized floating-point features, and the results (abnormal EEG or normal EEG) are output after being mapped by the softmax function.

[0232] If the test result is abnormal EEG, an alarm will be triggered via the alarm module.

[0233] like Figure 5 As shown, the training process for an efficiently quantifiable Long Short-Term Memory (LSTM) network includes:

[0234] First, EEG data for training is acquired through a data acquisition module consisting of an EEG amplifier and an A / D converter;

[0235] Secondly, the floating-point parameters and weight matrices of each unit are initialized based on the set number of internal units of the long short-term memory network that can be efficiently quantized, including:

[0236] The cell state parameters at the initial time step t=0 Initialize all values ​​to 0;

[0237] The hidden state parameters at the initial time step t=0 Initialize all values ​​to 0;

[0238] Forget gate weight matrix Input gate weight matrix Output gate weight matrix Cell candidate weight matrix Initialize to random numbers;

[0239] Forget gate loop weight matrix Input gate loop weight matrix Output gate loop weight matrix Cell candidate cycle weight matrix Initialize to random numbers;

[0240] Bias the forget gate Input gate bias Output gate bias Cell candidate bias All are initialized to 0;

[0241] Then, based on the parameters and weight matrix, the values ​​of each gate and state parameter are calculated, including:

[0242] For the extracted EEG signal features, the data at time step t is... t Calculate the floating-point parameter values ​​for each gate according to the following formula:

[0243] Calculate the forgetting gate gating value f tAs shown in equation (21):

[0244]

[0245] Among them, h t-1 Let be the hidden state parameter at time step t-1;

[0246] Calculate the input gate gating value i t As shown in equation (22):

[0247]

[0248] Calculate the output gate gating value o t As shown in equation (23):

[0249]

[0250] Calculate the cell candidate unit value g t As shown in equation (24):

[0251]

[0252] Calculate the cell state parameter c t As shown in equation (25):

[0253]

[0254] Calculate the hidden state parameter h t As shown in equation (26):

[0255] h t =o t ⊙c t (26)

[0256] Next, the loss function of the efficiently quantizable long short-term memory network, which consists of multiple sequentially connected efficiently quantizable long short-term memory units, is calculated. Based on the value of the loss function, the floating-point parameters are iteratively updated multiple times using the backpropagation algorithm. After reaching the maximum number of iterations, the efficiently quantizable long short-term memory network stops iteratively updating, and the floating-point parameter values ​​of each parameter and weight matrix are fixed and stored.

[0257] Calculate the characteristics of the dequantized floating-point number, as shown in equation (27):

[0258]

[0259] in: Q represents the quantized integer feature of the output of a Long Short-Term Memory (LSTM) network at time step t, which can be efficiently quantized. X Use the input quantization scale.

[0260] Calculate the loss value; including:

[0261] Based on the output hidden layer features h t The current loss value is calculated using the loss function E, and backpropagation optimization is then performed. The loss function E is defined as follows:

[0262]

[0263] Where θ represents all learnable parameters in a long short-term memory network that can be efficiently quantized; related to θ Let represent the j-th feature value of the hidden layer at time step t for the i-th sample, and m represent the number of samples in the backpropagation optimization process;

[0264] Parameter updates include:

[0265] The calculated loss value is used to update all learnable parameters in the efficiently quantizable Long Short-Term Memory network according to Equation (29):

[0266]

[0267] Where μ is the learning rate; θ v This represents all learnable parameters in the Long Short-Term Memory network that can be efficiently quantized at the v-th iteration. It is the loss function E(θ) v ) for θ v The gradient value; if v = N max N max =200, N max If the maximum number of iterations is not set, the network stops iterating and the parameter values ​​of the weight matrix are fixed; otherwise, v is incremented by 1 and the calculation of the gating and state parameter values ​​continues, and iterative updates are performed.

[0268] The process of extracting features from electroencephalogram (EEG) signals includes:

[0269] Feature extraction is performed using a single-layer convolutional neural network, which consists of eight single-channel one-dimensional convolutional kernels with a length of 5. This single-layer convolutional neural network maps the raw EEG signal at each time step into EEG signal features with a dimension of 1024.

[0270] Floating-point characteristics The category label is output after mapping using the softmax function.

[0271] The quantized Long Short-Term Memory (LSTM) network was tested on abnormal EEG classification data using ten-fold cross-validation. The performance of different networks was compared. Figure 6 As shown. Figure 6All accuracies are averages from 10-fold cross-validation. For reference, the original floating-point parameters of a traditional Long Short-Term Memory (LSTM) network yielded an accuracy of 97.67%. It can be seen that with 5-bit quantization, the network performance improves average accuracy by approximately 1% compared to a traditional LSTM network. Furthermore, the quantization lookup table of the efficiently quantizable LSTM network can quantize down to 2 bits without sacrificing the model's inference accuracy.

[0272] Example 5

[0273] An abnormal EEG signal detection system based on an FPGA hardware accelerator with highly efficient quantizable long short-term memory networks, such as... Figure 7 As shown, it includes:

[0274] The data acquisition module is configured to acquire the EEG signal to be detected through an EEG amplifier and an A / D converter;

[0275] The feature extraction module is configured to: extract features from the EEG signal to be detected and map the original EEG signal into a feature vector of a certain dimension;

[0276] The abnormal EEG signal detection module is configured to: input the feature vector into the FPGA hardware accelerator that can be efficiently quantized, dequantize the output value of the FPGA hardware accelerator to obtain the dequantized floating-point feature, and output the detection result (abnormal EEG or normal EEG) after mapping by the softmax function.

[0277] The abnormal EEG alarm module is configured to trigger an alarm based on the category label output by the abnormal EEG signal detection module.

Claims

1. A method for FPGA hardware acceleration of long short-term memory networks with high efficiency quantization, characterized in that, It runs within an FPGA hardware accelerator, which includes an ARM processor unit (PS) and a programmable logic unit (PL); including: A highly quantizable long short-term memory network consists of multiple sequentially connected highly quantizable long short-term memory units; The trained Long Short-Term Memory (LSTM) network with floating-point parameters is quantized to obtain the quantized LSM network. The quantized Long Short-Term Memory network is compiled into Verilog code and used to generate IP cores for acceleration, which are then deployed in programmable logic units (PLs). The ARM processor unit PS is responsible for data preparation and preprocessing, as well as softmax mapping operations; The configuration parameters of each unit of the quantized Long Short-Term Memory Network, including the number of input / output channels, feature vector dimension, number of neurons, and quantization coefficients, are first transmitted to the programmable logic unit (PL) via a simplified version of the Advanced Extension Interface bus at the PS end of the ARM processor unit. The input signal, quantization bias, and weights are transmitted from the PS of the ARM processor unit to the PL of the programmable logic unit via the Advanced Extension Interface bus; after the quantized Long Short-Term Memory unit completes the calculation at the PL of the programmable logic unit, the output data is returned to the PS of the ARM processor unit via the same AXI bus. Configure the parameters of the next quantized Long Short-Term Memory (LSTM) unit, start a new round of feature vector calculation, and repeat this process until the entire quantized LSM network is calculated. Output the calculation results; The segmented activation function of the highly quantizable long short-term memory unit. for: （1） in, For an adjustable approximation parameter of the Sigmoid function; ； The calculation process for the cell state parameters in the highly quantifiable long short-term memory unit is as follows: （2） in For Hadama accumulation, This represents the cell state at time step t in a long short-term memory unit that can be efficiently quantified. , , These represent the input gate gating value, forget gate gating value, and cell candidate unit value at the t-th time step in a quantifiable long short-term memory unit, respectively.

2. The FPGA hardware acceleration method for efficiently quantizable long short-term memory networks according to claim 1, characterized in that, The parameter quantization process for a trained, efficiently quantizable Long Short-Term Memory (LSTM) network includes: (1) Initialize the quantization parameters, including: Let the overall weight matrix be... , This refers to the forget gate weight matrix. This refers to the input gate weight matrix. This refers to the output gate weight matrix. This refers to the cell candidate weight matrix; Let the overall cyclic weight matrix be... , This refers to the forget gate loop weight matrix. This refers to the input gate loop weight matrix. This refers to the output gate loop weight matrix. This refers to the cell candidate cycle weight matrix; Initialize all weighted bit widths ; Initialize the segmentation function lookup table quantization bit width ; Initialize fixed-point bit width ; Initialize weight quantization factor ; Initialize weight matrix quantization scale ; Initialize the cyclic weight matrix quantization scale ;in, The function is a function that finds the maximum value. The function is an absolute value function; Initialize input quantization scale ; Initialize hidden unit quantization scale ; Initialize input weight quantization scale ; Initialize input loop weight quantization scale ; Initialize candidate cell quantization scale ; (2) Generate a quantized lookup table; including: First of all, with The initial value is, The increment is set to the termination value. Generate an arithmetic sequence ; Then, calculate the piecewise function. Quantization lookup table: ; (3) Quantization weight matrix; including: The quantized forget gate weight matrix is ​​shown in equation (3): （3） The quantization input gate weight matrix is ​​shown in equation (4): （4） The quantization output gate weight matrix is ​​shown in equation (5): （5） The quantified cell candidate weight matrix is ​​shown in equation (6): （6） The quantized forget gate loop weight matrix is ​​shown in equation (7): （7） The quantized input gate loop weight matrix is ​​shown in equation (8): （8） The quantized output gate loop weight matrix is ​​shown in equation (9): （9） The quantified cell candidate cycle weight matrix is ​​shown in Equation (10): （10） The quantization of the forget gate bias is shown in equation (11): （11） The quantization input gate bias is shown in equation (12): （12） The quantization output gate bias is shown in equation (13): （13） The cell candidate bias is quantified as shown in Equation (14): （14） (4) Calculate the quantized gating and state parameter values; including: Calculate the quantification of the forgetting gate gating value: （15） Calculate the quantization input gate gating value: （16） Calculate the gating value of the quantized output gate: （17） Calculate and quantify candidate cell values: （18） Calculate and quantify cell state parameters: （19） Calculate the quantized hidden state parameter values: （20）。 3. A highly efficient quantizable Long Short-Term Memory (LSTM) network FPGA hardware accelerator, characterized in that, The method includes an ARM processor unit PS and a programmable logic unit PL connected via an advanced expansion interface bus, to implement the FPGA hardware acceleration method for a long short-term memory network as described in claim 1 or 2.

4. The FPGA hardware accelerator for high-efficiency quantization of long short-term memory networks according to claim 3, characterized in that, include: The EEG signal to be detected is acquired using a data acquisition module consisting of an EEG amplifier and an A / D converter; Training a long short-term memory network that can be efficiently quantized; The trained, efficiently quantizable Long Short-Term Memory (LSTM) network is deployed on an FPGA hardware accelerator for efficiently quantizable LSM networks. Feature extraction is performed on the EEG signals to be detected; The features of the EEG signal to be detected are input into a FPGA hardware accelerator that can efficiently quantize long short-term memory networks to obtain the output value; The output value of the FPGA hardware accelerator with efficient quantization network is dequantized to obtain the dequantized floating-point characteristics, and the output check result is output after being mapped by the softmax function. If the test result is abnormal EEG, an alarm will be triggered via the alarm module.

5. The FPGA hardware accelerator for a high-efficiency quantizable long short-term memory network according to claim 4, characterized in that, The training process for an efficiently quantifiable Long Short-Term Memory (LSTM) network includes: First, EEG data for training is acquired through a data acquisition module consisting of an EEG amplifier and an A / D converter; Secondly, the floating-point parameters and weight matrices of each unit are initialized based on the set number of internal units of the long short-term memory network that can be efficiently quantized, including: The cell state parameters at the initial time step t=0 Initialize all values ​​to 0; The hidden state parameters at the initial time step t=0 Initialize all values ​​to 0; Forget gate weight matrix Input gate weight matrix Output gate weight matrix Cell candidate weight matrix Initialize to random numbers; Forget gate loop weight matrix Input gate loop weight matrix Output gate loop weight matrix Cell candidate cycle weight matrix Initialize to random numbers; Bias the forget gate Input gate bias Output gate bias Cell candidate bias All are initialized to 0; Then, based on the parameters and weight matrix, the values ​​of each gate and state parameter are calculated, including: For the data at time step t in the extracted EEG signal features Calculate the floating-point parameter values ​​for each gate according to the following formula: Calculate the forget gate gating value As shown in equation (21): （21） in, Let be the hidden state parameter at time step t-1; Calculate the input gate gating value As shown in equation (22): （22） Calculate the output gate gating value As shown in equation (23): （23） Calculate the cell candidate unit value As shown in equation (24): （24） Calculate cell state parameters As shown in equation (25): （25） Calculate hidden state parameters As shown in equation (26): （26） Next, the loss function of the efficient quantizable long short-term memory network, which consists of multiple sequentially connected efficient quantizable long short-term memory units, is calculated. Based on the value of the loss function, the floating-point parameters are iteratively updated multiple times using the backpropagation algorithm. After reaching the maximum number of iterations, the efficient quantizable long short-term memory network stops iteratively updating, and the floating-point parameter values ​​of each parameter and weight matrix are fixed and stored. Calculate the characteristics of the dequantized floating-point number, as shown in equation (27): （27） in: This represents the quantized integer feature of the output of a Long Short-Term Memory (LSTM) network at time step t, which can be efficiently quantized. Use the input quantization scale.

6. The FPGA hardware accelerator for a high-efficiency quantizable long short-term memory network according to claim 4, characterized in that, Calculate the loss value; include: Based on the output hidden layer features The current loss value is calculated using the loss function E; the loss function E is defined as follows: （28） in, Represents all learnable parameters in a Long Short-Term Memory (LSTM) network that can be efficiently quantized; and Related Let represent the j-th feature value of the hidden layer at time step t for the i-th sample, and m represent the number of samples in the backpropagation optimization process; Parameter updates include: The calculated loss value is used to update all learnable parameters in the efficiently quantizable Long Short-Term Memory network according to Equation (29): （29） in, The learning rate; This represents all learnable parameters in the Long Short-Term Memory network that can be efficiently quantized at the v-th iteration. It is a loss function right The gradient value; if , , If the maximum number of iterations is not set, the network stops iterating and the parameter values ​​of the weight matrix are fixed; otherwise, v is incremented by 1 and the calculation of the gating and state parameter values ​​continues, and iterative updates are performed.

7. The FPGA hardware accelerator for a high-efficiency quantizable long short-term memory network according to claim 4, characterized in that, The process of extracting features from electroencephalogram (EEG) signals includes: Feature extraction is performed using a single-layer convolutional neural network, which consists of eight single-channel one-dimensional convolutional kernels with a length of 5. This single-layer convolutional neural network maps the raw EEG signal at each time step into EEG signal features with a dimension of 1024.

8. An abnormal EEG signal detection system based on an FPGA hardware accelerator for a highly quantizable long short-term memory network, used to implement the FPGA hardware acceleration method for a highly quantizable long short-term memory network as described in claim 1 or 2, characterized in that, include: The data acquisition module is configured to acquire the EEG signal to be detected through an EEG amplifier and an A / D converter; The feature extraction module is configured to: extract features from the EEG signal to be detected and map the original EEG signal into a feature vector of a certain dimension; The abnormal EEG signal detection module is configured to: input the feature vector into the FPGA hardware accelerator that can be efficiently quantized, dequantize the output value of the FPGA hardware accelerator to obtain the dequantized floating-point feature, and output the detection result after mapping by the softmax function. The abnormal EEG alarm module is configured to trigger an alarm based on the category label output by the abnormal EEG signal detection module.

Images