An epilepsy prediction system based on adjustable inhibitory firing pulse neural networks

Abstract

The application relates to the technical field of epilepsy prediction, and is an epilepsy prediction system based on an adjustable-inhibition-emission pulse neural network, specifically comprising a data preprocessing module, a pulse neural network module, a neural network optimization module, a transmission mode judgment module and an accelerator module; a data set for neural network training and testing is obtained through the data preprocessing module, then the pulse neural network is software-optimized and hardware-optimized through the pulse neural network module, the neural network optimization module and the transmission mode judgment module, and finally the epilepsy prediction system based on the adjustable-inhibition-emission pulse neural network is realized through the accelerator module; the application solves the problems in the prior art that the calculation amount of the pulse neural network is huge, power consumption is large, the calculation speed is slow, and the wearable device demand cannot be met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of epilepsy prediction, and is an epilepsy prediction system based on an adjustable inhibitory firing pulse neural network. BACKGROUND

[0002] Epilepsy affects about 1% of the global population, especially children and adolescents, and is characterized by abnormal electrical activity in the brain, leading to convulsions, loss of consciousness, etc. In addition to physical complications, it often causes psychological problems and social function impairment, so treating epilepsy is particularly important. The current mainstream way to treat epilepsy is drug therapy, but its effect is not ideal. Drug therapy can only control epilepsy, not completely cure it, and many patients will show side effects, which can even lead to death. Therefore, the research of the present application focuses on predicting seizures. If patients can be informed in advance that seizures will occur before they occur, they can take medication in advance, find a safe place for themselves, and also reduce their fear of the unknown psychologically. The field of epilepsy prediction was established around the 1980s. Although no paper has shown that epilepsy can be accurately predicted so far, the rapid development of neural networks has had a huge impact on the field of epilepsy prediction.

[0003] The input signal of epilepsy prediction is generally the electroencephalogram of the patient, and the intracranial electroencephalogram (iEEG) and the scalp electroencephalogram (EEG) are commonly used. The electroencephalogram signal can be divided into pre-seizure, seizure, post-seizure and interictal. The purpose of predicting seizures is to distinguish between interictal and pre-seizure. The purpose of using neural networks is to extract features from the above electroencephalogram signals, and then through learning, interictal and pre-seizure can be distinguished, and finally the classification of new electroencephalogram signals can be realized.

[0004] Epilepsy varies greatly from person to person, so it is very important to develop a wearable device that can meet the needs of different people. For some electroencephalogram implant devices, small size and low power consumption are obviously necessary indicators. The most commonly used method based on deep learning in the field of epilepsy prediction is convolutional neural network (CNN) and recurrent neural network (RNN), but CNN and RNN have a large amount of computation and consume a lot of power, which is not suitable for wearable devices. Therefore, in order to meet the needs of electroencephalogram-based wearable devices and iEEG-based implantable devices, the network model needs to be further optimized. SUMMARY

[0005] The present application solves the technical problem that the computation of the pulse neural network is large, the power consumption is large, the calculation speed is slow, and the wearable device demand cannot be met in the prior art. An epilepsy prediction system based on an adjustable inhibitory firing pulse neural network is provided.

[0006] In order to achieve the above object, the application discloses an epilepsy prediction system based on an adjustable inhibitory emission pulse neural network, which comprises the following modules:

[0007] a data preprocessing module, a pulse neural network module, a neural network optimization module, an emission mode judgment module and an accelerator module;

[0008] The data preprocessing module is used to obtain a data set for training and testing the pulse neural network;

[0009] The pulse neural network module is used to construct a pulse neural network;

[0010] The neural network optimization module is used to optimize the neural network in hardware;

[0011] The emission mode judgment module judges the emission mode according to the time step, thereby performing software optimization;

[0012] The accelerator module is used to implement the adjustable inhibitory emission pulse neural network in hardware;

[0013] Specifically, the data preprocessing module is used to perform the following specific steps:

[0014] S11: Selecting the electroencephalogram data of a plurality of subjects with a pilot seizure under a fixed electrode configuration in a CHB-MIT data set to constitute a training set of a pulse neural network model;

[0015] S12: Extracting the electroencephalogram data of a seizure and performing stage annotation processing on the electroencephalogram data of a seizure according to the data characteristics of the pre-seizure period and the interictal period;

[0016] S13: Extracting features in the time domain of the electroencephalogram data of a seizure after the stage annotation processing is completed, to obtain the time domain features of the electroencephalogram data of a seizure;

[0017] S14: Extracting window samples of the pre-seizure period and the interictal period, mixing and shuffling the window samples, and dividing the window samples into a training set and a test set in a ratio of 4:1, wherein the unit time length of the window samples is T.

[0018] Specifically, the stage annotation processing in S12 comprises:

[0019] Selecting 30 minutes before the seizure period as the pre-seizure period, selecting 2 hours after the seizure as the post-seizure period, and regarding the data from the post-seizure period to the pre-seizure period of the next seizure as the interictal period;

[0020] The data acquisition method of the pre-ictal stage includes: first screening effective seizures, then selecting the acquisition segment of the effective seizures, and finally selecting the data from 5 minutes to 35 minutes before the seizure as the pre-ictal stage.

[0021] The data acquisition method of the inter-ictal stage includes: first screening the acquisition segment outside 2 hours before and after the seizure, second filtering out the segment 2 hours before and after the missing segment in the acquisition segment, and finally filtering out the acquisition segment of the first 2 hours and the last 2 hours of acquisition.

[0022] Specifically, the pulse neural network module includes: an input layer, an output layer, 5 convolutional layers and 2 fully connected layers, wherein the neuron model of the pulse neural network includes: an IF model and an LIF model.

[0023] Specifically, the 5 convolutional layers specifically include:

[0024] The first convolutional layer is one-dimensional convolution, the input channel number is 1, the output channel number is 16, the convolution kernel size is (1, 21), the neuron is an IF neuron, and the connected maximum pooling layer is (1, 10);

[0025] The second convolutional layer is one-dimensional convolution, the input channel number is 16, the output channel number is 32, the convolution kernel size is (1, 15), the neuron is an IF neuron, and the connected maximum pooling layer is (1, 6);

[0026] The third convolutional layer is one-dimensional convolution, the input channel number is 32, the output channel number is 64, the convolution kernel size is (1, 5), the neuron is an IF neuron, and the connected maximum pooling layer is (1, 2);

[0027] The fourth convolutional layer is two-dimensional convolution, the input channel number is 64, the output channel number is 128, the convolution kernel size is (3, 3), the neuron is an IF neuron, and the connected maximum pooling layer is (4, 4);

[0028] The fifth convolutional layer is two-dimensional convolution, the input channel number is 128, the output channel number is 256, the convolution kernel size is (4, 4), and the neuron is an IF neuron, wherein the fifth convolutional layer includes 524288 weights;

[0029] The 2 fully connected layers specifically include:

[0030] The sixth fully connected layer has an input channel number of 256, an output channel number of 64, and an LIF neuron;

[0031] The seventh full connection layer has 64 input channels, 2 output channels, and LIF neurons.

[0032] Specifically, the hardware optimization strategy performed by the neural network optimization module is specifically: modifying the sizes of the convolution kernel and the pooling kernel to powers of 2, and reducing the number of weights by y, where y is a weight control factor.

[0033] Specifically, the transmission mode judgment module is configured to optimize the software.

[0034] Further, the transmission mode judgment module is configured to perform the following specific steps:

[0035] S21: preset a running time threshold of a time step, and perform neuron transmission mode judgment.

[0036] S22: When the time step runs to the time threshold, it is judged that the neuron is in a normal transmission mode, and the pulse is allowed to be normally transmitted according to the rules.

[0037] S23: When the time step runs to the time threshold, it is judged that the neuron is in an inhibition transmission mode, and the subsequent pulse transmission is inhibited.

[0038] Specifically, the accelerator module includes a computing unit, a storage unit, and a control unit.

[0039] The computing unit is configured to perform convolution, full connection, activation, and pooling calculation operations in the network model.

[0040] The storage unit is configured to store input data, intermediate calculation results, and model weight parameters.

[0041] The control unit is configured to coordinate and manage the flow and data flow of each calculation and storage operation.

[0042] The computing unit includes a one-dimensional convolution calculation subunit, a two-dimensional convolution calculation subunit, a full connection calculation subunit, a neuron excitation subunit, and a maximum pooling calculation subunit.

[0043] The storage unit includes an input data storage subunit, an intermediate result storage subunit, and a weight storage subunit.

[0044] The control unit includes a master controller, a one-dimensional convolution memory sub-controller, a two-dimensional convolution memory sub-controller, a maximum pooling memory sub-controller, a full connection memory sub-controller, a DMA sub-controller, and an output sub-controller.

[0045] A storage medium, the storage medium has stored instructions, when a computer reads the instructions, the computer runs the epilepsy prediction system based on the adjustable inhibitory emission pulse neural network.

[0046] An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implements the epilepsy prediction system based on the adjustable inhibitory emission pulse neural network when running the computer program.

[0047] Compared with the prior art, the technical effects of the present application are as follows:

[0048] The present application uses a pulse neural network (SNN), which simulates the working mode of biological neurons and can naturally and efficiently process time series data such as brain waves and other biological signals. Compared with CNN and RNN, SNN saves computing resources through an event-driven mechanism, has the characteristics of low power consumption and high computing efficiency, and is an ideal choice for epilepsy prediction, especially for portable and implantable devices; at the same time, the neurons of the present application adopt two working modes, one is normal emission, and the other is inhibitory emission. When the time step runs to the time threshold, the neuron is in normal emission mode, allowing pulses to be normally emitted according to rules, and when a certain time threshold is exceeded, the inhibitory emission mode is turned on, and if the neuron emits a pulse at this time, the subsequent pulse emission will be inhibited. Using this adjustable inhibitory neuron, the neurons that emit too frequently can be restrained to some extent, while the possibility of emission of neurons with low emission probability is retained, thereby helping to avoid the situation that some neurons emit excessively and some neurons "die". The present application realizes the improvement of the computing speed and the reduction of the power consumption of the pulse neural network, and can meet the needs of wearable devices. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below, and obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor. Among them:

[0050] Figure 1 It is a structure schematic diagram of the epilepsy prediction system based on the adjustable inhibitory emission pulse neural network of the present application;

[0051] Figure 2 It is a seizure process schematic diagram of the present application;

[0052] Figure 3 It is a pre-seizure judgment flowchart of the present application;

[0053] Figure 4 This is a schematic diagram of a process for determining the interictal period of epilepsy according to the present invention;

[0054] Figure 5 This is a schematic diagram of the structure of a spiking neural network model for epilepsy prediction according to the present invention;

[0055] Figure 6 This is a schematic diagram of an adjustable pulse suppression emission workflow according to the present invention;

[0056] Figure 7 This is a schematic diagram of the structure of an accelerator module according to the present invention. Detailed Implementation

[0057] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0058] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0059] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0060] Example 1:

[0061] like Figure 1 As shown in the figure, an epilepsy prediction system based on an adjustable suppression emission spiking neural network according to an embodiment of the present invention includes the following modules:

[0062] Data preprocessing module, spiking neural network module, neural network optimization module, emission mode determination module, and accelerator module;

[0063] The data preprocessing module is used to obtain the dataset for training and testing the spiking neural network;

[0064] The data preprocessing module is used to perform the following specific steps:

[0065] S11: Select the EEG data of several subjects with lead epileptic seizures under a fixed electrode configuration from the CHB-MIT dataset to form the training set of the spiking neural network model.

[0066] The CHB-MIT dataset is an easily accessible public dataset, and the signal acquisition mode of the dataset is scalp EGG, which is conducive to the implementation of a wearable device.

[0067] S12: Extract the epileptic seizure electroencephalogram data, and perform stage labeling processing on the epileptic seizure electroencephalogram data according to the data characteristics of the pre-seizure period and the interictal period;

[0068] As shown in the example, Figure 2 The pre-seizure period refers to the time period before the seizure, and in this stage, the patient may have premonitory symptoms such as mood changes, paresthesia, cognitive changes, etc. The interictal period refers to the time period between two seizures, i.e., the relatively stable period between seizures, at which time the patient's electroencephalogram and clinical symptoms may be in a relatively normal state.

[0069] As shown in the example, Figure 3 The pre-seizure period judgment logic is provided to solve the data division difficulty problem, and specifically includes: selecting the collection segment of the effective seizure, judging whether the seizure time is after 35 minutes of the start of the collection segment, if yes, it means that the data of 5 minutes to 35 minutes before the seizure, i.e., the pre-seizure period data, can be obtained through the collection segment. If not, it is necessary to judge whether there is an adjacent previous collection segment, if the adjacent previous collection segment is missing, only this seizure can be discarded. If there is an adjacent previous collection segment, the collection segment of the effective seizure and the adjacent previous collection segment are combined, so that the data of 5 minutes to 35 minutes before the seizure, i.e., the pre-seizure period data, can be obtained.

[0070] The stage labeling processing in S12 includes:

[0071] 30 minutes before the seizure period is selected as the pre-seizure period, 2 hours after the seizure is selected as the post-seizure period, and the data from after 2 hours to before the next pre-seizure period is calculated as the interictal period;

[0072] The acquisition method of the pre-seizure period data includes: first screening the effective seizure, then selecting the collection segment of the effective seizure, and finally selecting the data of 5 minutes to 35 minutes before the seizure in the collection segment as the pre-seizure period;

[0073] The acquisition method of the interictal period data includes: first screening the collection segments other than 2 hours before and after the seizure, second filtering out the segments of 2 hours before and after the missing segments in these collection segments, and finally filtering out the collection segments of the first 2 hours and the last 2 hours of collection.

[0074] As Figure 4 shown, in this embodiment, a specific data processing method of interictal period is also provided, specifically including: first, screening the collection segments other than 2 hours before and after the seizure, noting that the collection segments of this seizure include all the collection segments of the seizure, not only the effective seizure collection segments. The preictal and postictal periods can be excluded outside 2 hours before and after the seizure. Secondly, filter out the segments of 2 hours before and after the missing segments in these collection segments, because it is unknown whether the missing segments have occurred epilepsy, it is safer to process all according to the occurrence of epilepsy. Finally, filter out the collection segments of the first 2 hours and the last 2 hours of the collection;

[0075] S13: performing deep learning on the ictal electroencephalogram data whose stage labeling processing is completed in the time domain to extract features, and obtaining time domain features of the ictal electroencephalogram data;

[0076] In this embodiment, the feature extraction by deep learning in the time domain is selected, which is beneficial to hardware implementation, in addition, the electrical signal itself is a function in the time dimension, and the feature extraction of the time domain signal is intuitive and easy to understand, without additional conversion operation;

[0077] S14: extracting preictal and interictal window samples, mixing and shuffling the preictal and interictal window samples, and dividing the window samples into a training set and a test set at a ratio of 4:1, wherein the unit time length of the window sample is T.

[0078] Exemplarily, for the sample imbalance problem occurring when extracting window samples, in this embodiment, when selecting preictal samples, the window sliding of this embodiment is sliding with a step length smaller than the window length, while the interictal window sliding is still sliding with a window length as a step length, and further, this embodiment starts the experiment from a 20-second time window, gradually reduces the window size, tries to keep the network structure consistent, and controls the overlapping part to be 1 / 4 or 1 / 5 of the window size.

[0079] The pulse neural network module is used to construct a pulse neural network;

[0080] The pulse neural network module includes an input layer, an output layer, 5 convolutional layers and 2 fully connected layers, wherein the neuron model of the pulse neural network includes an IF model and an LIF model; wherein the IF model is an integral firing model, the IF model is simple and clear and has high emission rate, and for the first few layers of the network, the features can be extracted relatively quickly, which is suitable for describing the firing behavior and basic electrical activity characteristics of neurons;

[0081] The LIF model is a leaky integrate-and-fire model, which is relatively accurate and suitable for subsequent fine operations of classification. Both models are relatively easy to implement and friendly to subsequent hardware implementation.

[0082] The internal signals of spiking neural networks propagate through spikes, and the input data also needs to be input into the network in the form of spikes. For image and voice data, they need to be converted into spikes through a set of encoding methods before being transmitted into the spiking neural network. The two most common encoding methods are frequency encoding and time encoding.

[0083] Frequency encoding is based on the firing frequency of spikes to represent the intensity or size of input information. The greater the numerical value of the input signal, the higher the frequency of neuron firing. Frequency encoding has good robustness and relatively high computational efficiency, and can represent a relatively large numerical range with fewer spikes, thereby reducing the computational load of the system. And frequency encoding can achieve higher information transmission bandwidth by adjusting the firing frequency of the pulse. Compared with other encoding methods, frequency encoding can transmit more information under the condition of maintaining a low peak pulse rate.

[0084] Time encoding usually uses the arrival time and time interval of the pulse to encode the intensity and timing relationship of the input signal. Time encoding contains rich information and is closer to the working mode of neurons in the biological nervous system, which can save energy and resource consumption, but it is more complex to implement.

[0085] This embodiment uses frequency encoding. The input image is first increased in dimension by copying, adding a time step dimension. Specifically, the shape of the input image is (18, 2560), and after adding the time step dimension, it becomes (T, 18, 2560). Using frequency encoding, the data flowing in the network is only pulse signals, only 1 and 0, which makes the multiplication operation in convolution and fully connected operations all be omitted, greatly reducing the hardware complexity and power consumption. Since hardware implementation is required, T is selected as a power of 2, so that the input data or intermediate results are stored in the storage more appropriately, without wasting storage space. Experiments show that when T is 8, the accuracy has reached the highest level, so the time step is selected as 8.

[0086] The window sample size obtained by the embodiment is 18-channel data, each channel window is 10 seconds, and the sampling rate is 256 Hz, so the data on one channel is 2560, and the total input image size is 18*2560. This is quite different from the size of a general image. Most images have similar vertical and horizontal resolutions, and two-dimensional images with similar resolutions have relatively large correlations in the vertical and horizontal directions, so that two-dimensional convolution can effectively extract hidden features. However, the electroencephalogram signal is much less correlated in the channel axis than in the time axis, so one-dimensional and two-dimensional convolution kernels are used for convolution and max-pooling operations. The early layers use one-dimensional kernels to learn the features in the time axis, and when the time axis size is reduced to be similar to the channel axis, two-dimensional kernels are used to learn the features in the space and time domain.

[0087] Exemplarily, the pulse neural network model used in the embodiment is as shown in the figure Figure 5 As shown in the figure, in addition to the input layer and the output layer, the proposed model also consists of 5 convolutional layers and 2 fully connected layers.

[0088] The 5 convolutional layers specifically include:

[0089] The first convolutional layer is one-dimensional convolution, the input channel number is 1, the output channel number is 16, the convolution kernel size is (1, 21), the neuron is an IF neuron, and the connected max-pooling layer is (1, 10);

[0090] The second convolutional layer is one-dimensional convolution, the input channel number is 16, the output channel number is 32, the convolution kernel size is (1, 15), the neuron is an IF neuron, and the connected max-pooling layer is (1, 6);

[0091] The third convolutional layer is one-dimensional convolution, the input channel number is 32, the output channel number is 64, the convolution kernel size is (1, 5), the neuron is an IF neuron, and the connected max-pooling layer is (1, 2);

[0092] The fourth convolutional layer is two-dimensional convolution, the input channel number is 64, the output channel number is 128, the convolution kernel size is (3, 3), the neuron is an IF neuron, and the connected max-pooling layer is (4, 4);

[0093] The fifth convolutional layer is two-dimensional convolution, the input channel number is 128, the output channel number is 256, the convolution kernel size is (4, 4), and the neuron is an IF neuron, wherein the fifth convolutional layer includes 524288 weights;

[0094] The 2 fully connected layers specifically include:

[0095] a sixth fully connected layer, an input channel number of the sixth fully connected layer is 256, an output channel number is 64, and a neuron is a LIF neuron;

[0096] a seventh fully connected layer, an input channel number of the seventh fully connected layer is 64, an output channel number is 2, and a neuron is a LIF neuron.

[0097] The neural network optimization module is configured to perform hardware optimization on the neural network.

[0098] The optimization strategy performed by the neural network optimization module is specifically that the sizes of the convolution kernel and the pooling kernel are all modified into shapes of powers of 2, and the number of weights is reduced by y, where y is a weight regulation factor.

[0099] Exemplarily, the embodiment adopts an SGD optimizer to perform loss minimization, a learning rate is 0.5, a tau constant of the LIF model is 2, a training batch size is 32, and a training epoch is 100. The pulse neural network model in the embodiment is implemented in Python 3.6, and the pulse neural network is built by using SpikingJelly.

[0100] In the embodiment, the pulse neural network structure is hardware-optimized, and the new pulse neural network structure is as follows:

[0101] First, the data on the time axis of the input data is cropped from 2560 to 2559.

[0102] A first convolutional layer, an input channel number is 1, an output channel number is 16, a convolution kernel size is (1, 16), a neuron is an IF neuron, and a maximum pooling layer connected is (1, 8); wherein, the size of an input image of the first convolutional layer is 18*2560, and the size of an output image is 18*318; then, a data 0 is filled at the end of the data on the time axis, and the image size becomes 18*319.

[0103] A second convolutional layer, an input channel number is 16, an output channel number is 32, a convolution kernel size is (1, 8), a neuron is an IF neuron, and a maximum pooling layer connected is (1, 8); wherein, the size of an input image of the second convolutional layer is 18*254, and the size of an output image is 18*39.

[0104] A third convolutional layer, an input channel number is 32, an output channel number is 64, a convolution kernel size is (1, 4), a neuron is an IF neuron, and a maximum pooling layer connected is (1, 2); wherein, the size of an input image of the third convolutional layer is 18*40, and the size of an output image is 18*18.

[0105] The fourth convolutional layer has 64 input channels, 128 output channels, a convolution kernel size of (3, 3), and uses IF neurons, and is connected to a maximum pooling layer with a size of (8, 8), wherein the size of the input image of the fourth convolutional layer is 18*18, and the size of the output image is 2*2;

[0106] The fifth convolutional layer has 128 input channels, 128 output channels, a convolution kernel size of (2, 2), and uses IF neurons, wherein the size of the input image of the fifth convolutional layer is 4*4, and the size of the output image is 1*1;

[0107] The sixth fully connected layer has 128 input channels, 64 output channels, and uses LIF neurons.

[0108] The seventh fully connected layer has 64 input channels, 2 output channels, and uses LIF neurons.

[0109] The transmission mode judgment module judges the transmission mode according to the time step, and the transmission mode judgment module is used for software optimization.

[0110] As shown in Figure 6 The transmission mode judgment module is used to run the following specific steps:

[0111] S21: preset a running time threshold of the time step, and judge the neuron transmission mode;

[0112] S22: when the time step runs to the time threshold, it is judged that the neuron is in a normal transmission mode, and the pulse is allowed to be normally transmitted according to the rules;

[0113] S23: when the time step runs to the time threshold, it is judged that the neuron is in an inhibited transmission mode, and the subsequent pulse transmission is inhibited.

[0114] In this embodiment, using this adjustable inhibitory neuron, the neurons that transmit too frequently can be restrained to a certain extent, and the possibility of transmission of neurons with a relatively low transmission probability is retained, thereby helping to avoid the situation that some neurons transmit excessively and some neurons "die". Using this method can not only reduce the number of transmissions, reduce power consumption, and balance excessive transmission and insufficient transmission, but also reduce time. Since the inhibitory mechanism is added, the inhibited neurons will not continue to trigger subsequently, so an "early stop" mechanism can be added to directly end the current time step and enter the next stage.

[0115] As shown in Figure 7 The accelerator module is used to implement the adjustable inhibitory transmission pulse neural network in hardware.

[0116] The accelerator module comprises a computing unit, a storage unit and a control unit

[0117] The computing unit is configured to perform convolution, full connection, activation and pooling calculation operations in the network model.

[0118] The storage unit is configured to store input data, intermediate calculation results and model weight parameters.

[0119] The control unit is configured to coordinate and manage the flow and data flow of various calculation and storage operations.

[0120] The computing unit comprises a one-dimensional convolution calculation subunit, a two-dimensional convolution calculation subunit, a full connection calculation subunit, a neuron excitation subunit and a maximum pooling calculation subunit.

[0121] In the embodiment, the PE needs to support multiply-accumulate operations with sizes of 16, 64, 128, 512 and 576, and maximum pooling operations with window sizes of 2, 8 and 64.

[0122] A reconfigurable design is adopted, and different calculation modes can reuse part of the same calculation resources to improve the utilization rate of hardware resources. The computing unit comprises four identical computing units (PEs), which differ in the used weights, aiming to improve the parallelism of calculation and shorten the operation time.

[0123] The storage unit comprises an input data storage subunit, an intermediate result storage subunit and a weight storage subunit.

[0124] Exemplarily, in the embodiment, after each layer is calculated, the storage of the result data is used as the storage of the input data, and the storage of the input data is used as the storage of the result. The weight memory needs to be designed in cooperation with the operation process, and needs to take into account the problem of saving memory space, so the design of the weight memory is relatively complex. According to these rules, the most weight memory is divided into three parts, which are respectively referred to as the first layer of the weight memory group, the weight memory group 1 and the weight memory group 2. The first layer of the weight memory group is mainly used to store the data of the first layer of convolution operation. The weight memory group 1 and the weight memory group 2 are used to store the data of the operation of other layers except the first layer.

[0125] Exemplarily, in the embodiment, in addition to the above-mentioned units, a direct memory access unit (DMA) and an AXI bus interface are designed. Finally, the adjustable inhibitory spiking neural network accelerator module in the embodiment can be connected with the DDR through the AXI bus and the DDR, so as to realize the input of data and weights of the DDR, facilitate the update of different input data and weights, and the whole accelerator can be connected with the CPU or other control modules, so as to control the use of the neural network accelerator.

[0126] The control unit includes a master controller, a one-dimensional convolution memory access sub-controller, a two-dimensional convolution memory access sub-controller, a maximum pooling memory access sub-controller, a full connection memory access sub-controller, a DMA sub-controller and an output sub-controller.

[0127] Exemplarily, in the embodiment, the master controller is responsible for scheduling the overall workflow and is composed of states. The one-dimensional convolution memory access controller works in the first, second and third one-dimensional convolution modes. Since the one-dimensional convolution and the maximum pooling matched therewith are completed in the calculation unit at a time, it is not necessary to store twice. The two-dimensional convolution memory access controller is mainly for the two-dimensional convolution operation of the fourth and fifth layers. The maximum pooling memory access controller is mainly designed for the pooling operation after the two-dimensional convolution of the fourth layer. The full connection memory access controller is mainly designed for the full connection operation of the sixth and seventh layers. The DMA controller is mainly used to initialize the original input data and the first batch of weight data to be moved to the SRAM and the subsequent update of the weight data. The output controller receives and temporarily stores the output result, and then sends the output result to the outside and synchronously pulls up the output valid signal.

[0128] Embodiment two:

[0129] The embodiment provides an electronic device, including a processor and a memory, wherein the memory has a computer program stored therein and available for the processor to call;

[0130] The processor runs the above-mentioned epilepsy prediction system based on the adjustable inhibitory spiking neural network by calling the computer program stored in the memory.

[0131] The electronic device can have a large difference due to configuration or performance, and can include one or more processors (Central Processing Units, CPUs) and one or more memories, wherein the memory stores at least one computer program, the computer program is loaded and run by the processor to implement the above-mentioned method embodiment to provide an epilepsy prediction system based on an adjustable inhibition emission pulse neural network. The electronic device can also include other components for implementing device functions, for example, the electronic device can also have a wired or wireless network interface and an input and output interface, and the like, so as to perform data input and output. This embodiment will not be described here.

[0132] Embodiment three:

[0133] The embodiment provides a computer readable storage medium, which stores an erasable computer program.

[0134] When the computer program runs on the computer device, the computer device runs the above-mentioned epilepsy prediction system based on an adjustable inhibition emission pulse neural network.

[0135] For example, the computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a read-only compact disc (Compact Disc Read-Only Memory, CD-ROM), a magnetic tape, a floppy disk and an optical data storage device, etc.

[0136] It should be understood that in various embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0137] It should be understood that according to A, B is determined, which means that B is determined only according to A, but also B can be determined according to A and / or other information.

[0138] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions according to the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired network or / and a wireless network. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0139] Those skilled in the art can clearly understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device, and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0141] In several embodiments provided by the present application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the above-described device embodiments are merely schematic, for example, the division of units is only one, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices, or units, which can be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0143] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0144] In the description of the specification, the description of the terms "one embodiment", "example", "specific example" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0145] In summary, compared with the prior art, the technical effects of the application are as follows:

[0146] The application uses a spiking neural network (SNN) which simulates the working mode of biological neurons and can naturally and efficiently process time series data such as biological signals such as brain waves. Compared with CNN and RNN, SNN saves computing resources through an event-driven mechanism, has the characteristics of low power consumption and high computing efficiency, is an ideal choice for epilepsy prediction, and is especially suitable for portable and implanted devices; at the same time, the neurons of the application adopt two working modes, one is normal emission, and the other is inhibitory emission. When the time step runs to the time threshold, the neuron is in normal emission mode, allowing pulses to be normally emitted according to rules, and when a certain time threshold is exceeded, the inhibitory emission mode is started, and if the neuron emits pulses at this time, the subsequent pulse emission will be inhibited. Using this adjustable inhibitory neuron, the neurons that emit too frequently can be restrained to some extent, while the emission probability of the neurons that emit less is retained, thereby helping to avoid the situation that some neurons emit excessively and some neurons "die". The application realizes the improvement of the computing speed of the spiking neural network, and can meet the needs of wearable devices.

[0147] The above shows and describes the basic principles and main features of the present application and the advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above embodiments, and the above embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. An epilepsy prediction system based on an adjustable inhibitory firing pulse neural network, characterized by, The system comprises the following modules: a data preprocessing module, a spiking neural network module, a neural network optimization module, a firing mode judgment module and an accelerator module; the data preprocessing module is configured to obtain a data set for training and testing the spiking neural network; the spiking neural network module is configured to construct a spiking neural network; the neural network optimization module is configured to optimize the neural network in hardware; the firing mode judgment module is configured to judge the firing mode according to the time step, thereby optimizing the software; the accelerator module is configured to implement the spiking neural network with adjustable inhibitory firing in hardware.

2. The seizure prediction system based on adjustable inhibitory firing of spiking neural networks of claim 1, wherein, The data preprocessing module is configured to perform the following specific steps: S11: selecting electroencephalogram data of several subjects with a pilot seizure under a fixed electrode configuration in a CHB-MIT data set to constitute a training set of a spiking neural network model; S12: extracting the electroencephalogram data and performing stage labeling processing on the electroencephalogram data according to the data characteristics of the preictal period and the interictal period; S13: extracting features in the time domain of the electroencephalogram data after the stage labeling processing is completed, to obtain time domain features of the electroencephalogram data; S14: extracting window samples of the preictal period and the interictal period, mixing and shuffling the window samples, and dividing the window samples into a training set and a test set at a ratio of 4:1, wherein the unit time length of the window samples is T.

3. The seizure prediction system based on adjustable inhibitory firing of spiking neural networks of claim 2, wherein, The stage labeling processing in S12 comprises: selecting 30 minutes before the ictal period as the preictal period, selecting 2 hours after the seizure as the postictal period, and regarding the data from after the 2 hours to before the next preictal period as the interictal period; wherein the data acquisition method of the preictal period comprises: first screening valid seizures, then selecting the collection segment of the valid seizures, and finally selecting the data from 5 minutes to 35 minutes before the seizure in the collection segment as the preictal period; the data acquisition method of the interictal period comprises: first screening the collection segments other than the 2 hours before and after the seizure, then filtering out the segments of 2 hours before and after the missing segments in these collection segments, and finally filtering out the collection segments of the first 2 hours and the last 2 hours of collection.

4. The seizure prediction system based on adjustable inhibitory firing of spiking neural networks of claim 3, wherein, The spiking neural network module comprises: an input layer, an output layer, 5 convolutional layers and 2 fully connected layers, wherein the neuron model of the spiking neural network comprises: an IF model and a LIF model.

5. The seizure prediction system based on adjustable inhibitory firing of spiking neural networks of claim 4, wherein, The 5 convolutional layers specifically comprise: a first convolutional layer, which is a one-dimensional convolution, has an input channel number of 1, an output channel number of 16, a convolution kernel size of (1, 21), and IF neurons, and is connected to a maximum pooling layer of (1, 10); a second convolutional layer, which is a one-dimensional convolution, has an input channel number of 16, an output channel number of 32, a convolution kernel size of (1, 15), and IF neurons, and is connected to a maximum pooling layer of (1, 6); a third convolutional layer, which is a one-dimensional convolution, has an input channel number of 32, an output channel number of 64, a convolution kernel size of (1, 5), neurons are IF neurons, and a maximum pooling layer connected has a size of (1, 2); a fourth convolutional layer, which is a two-dimensional convolution, has an input channel number of 64, an output channel number of 128, a convolution kernel size of (3, 3), neurons are IF neurons, and a maximum pooling layer connected has a size of (4, 4); a fifth convolutional layer, which is a two-dimensional convolution, has an input channel number of 128, an output channel number of 256, a convolution kernel size of (4, 4), and neurons are IF neurons, wherein the fifth convolutional layer includes 524288 weights; the two fully connected layers specifically include: a sixth fully connected layer, which has an input channel number of 256, an output channel number of 64, and neurons are LIF neurons; a seventh fully connected layer, which has an input channel number of 64, an output channel number of 2, and neurons are LIF neurons.

6. The seizure prediction system based on adjustable inhibitory firing of spiking neural networks of claim 5, wherein, The hardware optimization strategy executed by the neural network optimization module specifically is: modifying the sizes of convolution kernels and pooling kernels into shapes of powers of 2, and reducing the number of weights by y, wherein y is a weight regulation factor.

7. The seizure prediction system based on adjustable inhibitory firing of spiking neural networks of claim 6, wherein, The firing mode judgment module is configured to execute the following specific steps: S21: preset a running time threshold of a time step, and perform neuron firing mode judgment; S22: when the time step runs to before the time threshold, it is judged that the neuron is in a normal firing mode, and the pulses are allowed to be normally fired according to rules; S23: when the time step runs to after the time threshold, it is judged that the neuron is in an inhibited firing mode, and the subsequent pulse firing is inhibited.

8. The seizure prediction system based on adjustable inhibitory firing of spiking neural networks of claim 7, wherein, The accelerator module includes a calculation unit, a storage unit, and a control unit, the calculation unit is configured to perform convolution, full connection, activation, and pooling calculation operations in the network model; the storage unit is configured to store input data, intermediate calculation results, and model weight parameters; the control unit is configured to coordinate and manage the flow and data flow of each calculation and storage operation; wherein the calculation unit includes a one-dimensional convolution calculation subunit, a two-dimensional convolution calculation subunit, a full connection calculation subunit, a neuron excitation subunit, and a maximum pooling calculation subunit; the storage unit includes an input data storage subunit, an intermediate result storage subunit, and a weight storage subunit; the control unit includes a mother controller, a one-dimensional convolution memory access sub-controller, a two-dimensional convolution memory access sub-controller, a maximum pooling memory access sub-controller, a full connection memory access sub-controller, a DMA sub-controller, and an output sub-controller.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, runs a seizure prediction system based on an adjustable inhibitory firing pulse neural network as claimed in any one of claims 1-8.

10. An electronic device, comprising: comprises: a memory configured to store instructions; a processor configured to execute the instructions to cause the device to run a module that implements a seizure prediction system based on an adjustable inhibitory firing pulse neural network as claimed in any one of claims 1-8.

Images