Epilepsy prediction system based on adjustable suppression emission pulse neural network

Abstract

The invention relates to the technical field of epilepsy prediction, in particular to an epilepsy prediction system based on an adjustable emission suppression spiking neural network, which specifically comprises a data preprocessing module, a spiking neural network module, a neural network optimization module, an emission mode judgment module and an accelerator module, a data set capable of being provided for neural network training and testing is obtained through the data preprocessing module, and then software optimization and hardware optimization are carried out on a pulse neural network through the pulse neural network module, the neural network optimization module and the emission mode judgment module. And finally, realizing the epilepsy prediction system based on the spiking neural network capable of adjusting suppression emission through an accelerator module. The epilepsy prediction system solves the problems that in the prior art, the calculation speed is low and the requirements of wearable equipment cannot be met due to the fact that the calculation amount of the spiking neural network is huge and the power consumption is large.

Description

Technical Field

[0001] The invention relates to the technical field of epilepsy prediction, and is an epilepsy prediction system based on a pulse neural network with adjustable emission inhibition. Background Art

[0002] Epilepsy affects about 1% of the world's population, especially children and adolescents. It is characterized by abnormal electrical activity in the brain, leading to convulsions, loss of consciousness, etc. In addition to physical complications, it often causes psychological problems and social function impairment, so the treatment of epilepsy is particularly important. The mainstream way to treat epilepsy now is drug therapy, but its effect is not ideal. Drug therapy can only control epilepsy, but cannot completely cure epilepsy, and many patients will show side effects, and severe cases may even lead to death. Therefore, the research of the present invention focuses on predicting epileptic seizures. If the patient knows in advance that epilepsy is about to occur before the epileptic seizure, he can take medicine in advance, find a safe place for himself, and psychologically reduce the fear of the unknown. The field of epilepsy prediction was established around the 1980s. Although no paper has shown that epilepsy can be accurately predicted so far, many technological developments have promoted the progress of this field. Among them, the rapid development of neural networks has had a huge impact on the field of epilepsy prediction.

[0003] The input signal for epilepsy prediction is generally the patient's electroencephalogram (EEG), and the commonly used ones are intracranial electroencephalogram (iEEG) and scalp electroencephalogram (EEG). EEG signals can be divided into preictal, ictal, postictal and interictal periods. The purpose of predicting epileptic seizures is to distinguish between interictal and preictal periods. The purpose of using neural networks is to extract the features of the above EEG signals, and then distinguish between interictal and preictal periods through learning, and finally realize the classification of new EEG signals.

[0004] Epilepsy varies greatly from person to person, so it is very important to realize a wearable device that can meet the needs of different people. For some EEG implants, small size and low power consumption are obviously indicators that must be met. Currently, the most commonly used deep learning-based methods in the field of epilepsy prediction are convolutional neural networks (CNN) and recurrent neural networks (RNN), but CNN and RNN have a large amount of computation and consume a lot of power, which is not suitable for wearable devices. Therefore, in order to meet the needs of EEG-based wearable devices and iEEG-based implantable devices, the network model needs to be further optimized. Summary of the invention

[0005] The technical problem to be solved by the present invention is that in the prior art, the calculation amount of the pulse neural network is huge, the power consumption is also high, resulting in slow calculation speed and cannot meet the needs of wearable devices. An epilepsy prediction system based on a pulse neural network with adjustable emission inhibition is proposed.

[0006] In order to achieve the above object, the present invention provides an epilepsy prediction system based on a pulse neural network with adjustable emission inhibition, which includes the following modules:

[0007] Data preprocessing module, pulse neural network module, neural network optimization module, emission mode judgment module and accelerator module;

[0008] The data preprocessing module is used to obtain a data set for training and testing of a spiking neural network;

[0009] The spiking neural network module is used to construct a spiking neural network;

[0010] The neural network optimization module is used to perform hardware optimization on the neural network;

[0011] The emission mode judgment module judges the emission mode according to the time step, thereby optimizing the software;

[0012] The accelerator module is used for hardware implementation of a pulse neural network with adjustable emission suppression;

[0013] Specifically, the data preprocessing module is used to perform the following specific steps:

[0014] S11: The EEG data of several subjects with epileptic seizures under a fixed electrode configuration were selected from the CHB-MIT dataset to form the training set of the spiking neural network model;

[0015] S12: extracting the EEG data of epileptic seizures, and performing stage labeling processing on the EEG data of epileptic seizures according to the data characteristics of the pre-ictal period and the interictal period;

[0016] S13: performing deep learning to extract features in the time domain on the epileptic seizure EEG data after the stage labeling process, so as to obtain the time domain features of the epileptic seizure EEG data;

[0017] S14: extracting window samples of the pre-ictal period and the inter-ictal period, mixing and shuffling the order of the window samples of the pre-ictal period and the inter-ictal period, and dividing the window samples into a training set and a test set at a ratio of 4:1, wherein the unit time length of the window samples is T.

[0018] Specifically, the stage annotation process in S12 includes:

[0019] The 30 minutes before the onset of the seizure were selected as the preictal period, the 2 hours after the onset of the epilepsy were selected as the postictal period, and the data from two hours to the next preictal period were counted as the interictal period;

[0020] The method for obtaining the pre-ictal data includes: firstly screening out effective seizures, then selecting a collection segment of the effective seizures, and finally selecting data from 5 minutes to 35 minutes before the epileptic seizure in the collection segment as the pre-ictal data;

[0021] The method for acquiring the interictal data includes: first, screening the collected segments outside 2 hours before and after the epileptic seizure, secondly filtering out the segments 2 hours before and after the missing segments among these collected segments, and finally filtering out the collected segments of the first 2 hours and the last 2 hours.

[0022] Specifically, the pulse neural network module includes: an input layer, an output layer, 5 convolutional layers and 2 fully connected layers, wherein the neuron model of the pulse neural network includes: an IF model and a LIF model.

[0023] Specifically, the five convolutional layers include:

[0024] The first convolution layer is a one-dimensional convolution, the number of input channels is 1, the number of output channels is 16, the convolution kernel size is (1,21), the neuron is an IF neuron, and the maximum pooling layer connected is (1,10);

[0025] The second convolutional layer is a one-dimensional convolution, the number of input channels is 16, the number of output channels is 32, the convolution kernel size is (1,15), the neuron is an IF neuron, and the maximum pooling layer connected is (1,6);

[0026] The third convolution layer is a one-dimensional convolution, the number of input channels is 32, the number of output channels is 64, the convolution kernel size is (1,5), the neuron is an IF neuron, and the maximum pooling layer connected is (1,2);

[0027] The fourth convolution layer is a two-dimensional convolution, the number of input channels is 64, the number of output channels is 128, the convolution kernel size is (3,3), the neuron is an IF neuron, and the maximum pooling layer connected is (4,4);

[0028] A fifth convolutional layer, wherein the fifth convolutional layer is a two-dimensional convolution, the number of input channels is 128, the number of output channels is 256, the convolution kernel size is (4,4), and the neurons are IF neurons, wherein the fifth convolutional layer includes 524288 weights;

[0029] The two fully connected layers specifically include:

[0030] A sixth fully connected layer, wherein the number of input channels of the sixth fully connected layer is 256, the number of output channels is 64, and the neurons are LIF neurons;

[0031] The seventh fully connected layer has 64 input channels, 2 output channels, and LIF neurons.

[0032] Specifically, the hardware optimization strategy executed by the neural network optimization module is: modifying the sizes of the convolution kernel and the pooling kernel to a power of 2, and reducing the number of weights by y, where y is a weight control factor.

[0033] Specifically, the transmission mode determination module is used to optimize the software;

[0034] Furthermore, the transmission mode determination module is used to execute the following specific steps:

[0035] S21: preset the running time threshold of the time step and make neuron firing mode judgment;

[0036] S22: When the time step reaches the time threshold, the neuron is judged to be in normal firing mode, and the pulse is allowed to be fired normally according to the rules;

[0037] S23: When the time step reaches the time threshold, the neuron is judged to be in the inhibited firing mode, and the subsequent pulse firing is inhibited.

[0038] Specifically, the accelerator module includes: a computing unit, a storage unit, and a control unit.

[0039] The computing unit is used to perform convolution, full connection, activation and pooling computing operations in the network model;

[0040] The storage unit is used to store input data, intermediate calculation results and model weight parameters;

[0041] The control unit is used to coordinate and manage the processes and data flows of various computing and storage operations.

[0042] The computing unit includes a one-dimensional convolution computing subunit, a two-dimensional convolution computing subunit, a fully connected computing subunit, a neuron excitation subunit, and a maximum pooling computing subunit for implementing the network model;

[0043] The storage unit includes an input data storage subunit, an intermediate result storage subunit and a weight storage subunit;

[0044] The control unit includes a main mother controller, a one-dimensional convolution memory access sub-controller, a two-dimensional convolution memory access sub-controller, a maximum pooling memory access sub-controller, a fully connected memory access sub-controller, a DMA sub-controller and an output sub-controller.

[0045] A storage medium stores instructions, and when a computer reads the instructions, the computer is caused to run the epilepsy prediction system based on a pulse neural network with adjustable emission inhibition.

[0046] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned epilepsy prediction system based on a pulse neural network with adjustable emission inhibition when executing the computer program.

[0047] Compared with the prior art, the technical effects of the present invention are as follows:

[0048] The present invention uses a spiking neural network (SNN), which simulates the working mode of biological neurons and can naturally and efficiently process biological signals such as time series data such as brain waves. Compared with CNN and RNN, SNN saves computing resources through an event-driven mechanism, has the characteristics of low power consumption and high computing efficiency, and is an ideal choice for epilepsy prediction, especially for portable and implantable devices; at the same time, the neurons of the present invention adopt two working modes, one is normal emission and the other is suppression emission. Before the time step runs to the time threshold, the neuron is in the normal emission mode, allowing the pulse to be emitted normally according to the rules. When a certain time threshold is exceeded, the suppression emission mode is turned on. If the neuron emits a pulse again at this time, it will suppress the subsequent pulse emission. Using this adjustable inhibitory neuron, neurons that are emitted too frequently can be constrained to a certain extent, while retaining the possibility of firing neurons with a relatively low probability of emission, which is conducive to avoiding the situation where some neurons are over-emitted and some neurons are "dead". The present invention realizes the improvement of the computing speed and the reduction of power consumption of the spiking neural network, which can meet the needs of wearable devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0050] Figure 1 It is a structural schematic diagram of an epilepsy prediction system based on a pulse neural network with adjustable emission inhibition according to the present invention;

[0051] Figure 2 A schematic diagram of an epileptic seizure process of the present invention;

[0052] Figure 3 A schematic diagram of a pre-epilepsy seizure judgment process of the present invention;

[0053] Figure 4 A schematic diagram of a flow chart of epileptic seizure interval determination process of the present invention;

[0054] Figure 5 It is a structural schematic diagram of an epilepsy prediction pulse neural network model of the present invention;

[0055] Figure 6 A schematic diagram of an adjustable suppression pulse emission working process of the present invention;

[0056] Figure 7 The figure is a schematic structural diagram of an accelerator module of the present invention. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0059] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0060] Embodiment 1:

[0061] like Figure 1 As shown, an epilepsy prediction system based on a pulse neural network with adjustable emission inhibition according to an embodiment of the present invention includes the following modules:

[0062] Data preprocessing module, pulse neural network module, neural network optimization module, emission mode judgment module and accelerator module;

[0063] The data preprocessing module is used to obtain a data set for training and testing of a spiking neural network;

[0064] The data preprocessing module is used to perform the following specific steps:

[0065] S11: The EEG data of several subjects with epileptic seizures under a fixed electrode configuration were selected from the CHB-MIT dataset to form the training set of the spiking neural network model;

[0066] Among them, the CHB-MIT dataset is an easily accessible public dataset, and the signal collection method of this dataset is scalp EGG, which is conducive to the implementation of wearable devices;

[0067] S12: extracting the EEG data of epileptic seizures, and performing stage labeling processing on the EEG data of epileptic seizures according to the data characteristics of the pre-ictal period and the interictal period;

[0068] For example, Figure 2 As shown, the preictal period refers to the period before an epileptic seizure, during which the patient may experience premonitory symptoms such as mood changes, sensory abnormalities, cognitive changes, etc. The interictal period refers to the period between two epileptic seizures, that is, the relatively stable period between epileptic seizures, when the patient's EEG and clinical symptoms may be in a relatively normal state.

[0069] For example, in this embodiment, the CHB-MIT data set is not continuous and complete data, and data may be missing in the data, such as Figure 3 As shown, this embodiment also provides a pre-ictal judgment logic for solving the problem of data division difficulties, specifically including: by selecting the acquisition segment of the effective attack, judging whether the epileptic seizure time is 35 minutes after the start of the acquisition segment, if yes, it means that the data from 5 minutes to 35 minutes before the epileptic seizure can be obtained through this acquisition segment, that is, the pre-ictal data. If not, it is necessary to judge whether there is an adjacent previous acquisition segment. If the adjacent previous acquisition segment is missing, this attack can only be discarded. If there is an adjacent previous acquisition segment, the acquisition segment of the effective attack and the adjacent previous acquisition segment are combined, so that the data from 5 minutes to 35 minutes before the epileptic seizure, that is, the pre-ictal data, can be obtained.

[0070] The stage annotation processing in S12 includes:

[0071] The 30 minutes before the onset of the seizure were selected as the preictal period, the 2 hours after the onset of the epilepsy were selected as the postictal period, and the data from two hours to the next preictal period were counted as the interictal period;

[0072] The method for obtaining the pre-ictal data includes: firstly screening out effective seizures, then selecting a collection segment of the effective seizures, and finally selecting data from 5 minutes to 35 minutes before the epileptic seizure in the collection segment as the pre-ictal data;

[0073] The method for acquiring the interictal data includes: first, screening the collected segments outside 2 hours before and after the epileptic seizure, secondly filtering out the segments 2 hours before and after the missing segments among these collected segments, and finally filtering out the collected segments of the first 2 hours and the last 2 hours.

[0074] like Figure 4 As shown, a specific method for processing data between epileptic seizures is also provided in this embodiment, specifically including: first, screening the collected segments outside 2 hours before and after the epileptic seizure. Note that the collected segments of epileptic seizures include all collected segments of epileptic seizures, not just the collected segments of effective seizures. Pre-seizure and post-seizure periods can be excluded if the segments are outside 2 hours before and after the seizure. Secondly, filter out the segments 2 hours before and after the missing segments from these collected segments, because it is unknown whether epilepsy has occurred in the missing segments. To be on the safe side, all of them are processed as if epilepsy has occurred. Finally, filter out the collected segments of the first 2 hours and the last 2 hours;

[0075] S13: performing deep learning to extract features in the time domain on the epileptic seizure EEG data after the stage labeling process, so as to obtain the time domain features of the epileptic seizure EEG data;

[0076] In this embodiment, feature extraction for deep learning is selected in the time domain, which is conducive to hardware implementation. In addition, the electrical signal itself is a function in the time dimension, and feature extraction of time domain signals is intuitive and easy to understand, without the need for additional conversion operations.

[0077] S14: extracting window samples of the pre-ictal period and the inter-ictal period, mixing and shuffling the order of the window samples of the pre-ictal period and the inter-ictal period, and dividing the window samples into a training set and a test set at a ratio of 4:1, wherein the unit time length of the window samples is T.

[0078] Exemplarily, for the sample imbalance problem that occurs when extracting window samples, in this embodiment, when selecting pre-attack samples, the window of this embodiment slides with a step size smaller than the window length, while the window sliding between attacks still slides with the window length as the step size. Furthermore, this embodiment starts the experiment from a 20-second time window, gradually reduces the window size, tries to keep the network structure consistent, and controls the overlapping part to 1 / 4 or 1 / 5 of the window size.

[0079] The spiking neural network module is used to construct a spiking neural network;

[0080] The pulse neural network module includes: an input layer, an output layer, 5 convolutional layers and 2 fully connected layers, wherein the neuron model of the pulse neural network includes: an IF model and a LIF model; wherein the IF model is an integral ignition model, which is simple and clear and has a high firing rate. For the first few layers of the network, features can be extracted relatively quickly, and it is suitable for describing the firing behavior and basic electrical activity characteristics of neurons;

[0081] The LIF model is a leaky integral ignition model, which is relatively accurate and suitable for the fine operation of subsequent classification. Both models are relatively easy to implement and are also relatively friendly to the subsequent hardware implementation.

[0082] The internal signals of the spiking neural network are transmitted through pulses, and the input data also needs to be input into the network in the form of pulses. For data such as images and voices, they need to be converted into pulses through a set of encoding methods before they can be transmitted to the spiking neural network. The two most common encoding methods are frequency encoding and time encoding.

[0083] Frequency coding represents the intensity or size of input information based on the frequency of pulse emission. The larger the value of the input signal, the higher the frequency of the neuron's pulse emission. Frequency coding has good robustness and relatively high computational efficiency. It can use a smaller number of pulses to represent a relatively large numerical range, thereby reducing the computational load of the system. And frequency coding can achieve a higher information transmission bandwidth by adjusting the frequency of pulse emission. Compared with other coding methods, frequency coding can transmit more information while maintaining a lower peak pulse rate.

[0084] Temporal coding usually uses the arrival time and time interval of pulses to encode the intensity and timing relationship of the input signal. Temporal coding contains rich information, is closer to the working mode of neurons in biological nervous systems, can save energy and resource consumption, but is more complicated to implement.

[0085] This embodiment uses frequency coding. The input image is first copied to increase the dimension, and the dimension of time step is added. Specifically, the shape of the input image is (18, 2560), and after adding the dimension of time step, it becomes (T, 18, 2560). Using frequency coding, the data flowing in the network are only pulse signals, only 1 and 0, which makes the convolution and multiplication operations in the full connection operation all saved, greatly reducing the hardware complexity and power consumption. Due to the need for hardware implementation, T selects data that is a power of 2, so that the storage of input data or intermediate results in the storage will be more appropriate and will not waste storage space. Experiments have shown that when T is 8, the accuracy has reached the highest level, so the time step is selected as 8.

[0086] The window sample size obtained in this embodiment is data from 18 channels, each channel window is 10 seconds, and the sampling rate is 256Hz, so the data on one channel is 2560, so the total input image size is 18*2560. This is quite different from the size of general images. The vertical and horizontal resolutions of most images are similar, and the features of two-dimensional images with similar resolutions have relatively large correlations in the vertical and horizontal directions, so that two-dimensional convolution can effectively extract hidden features. However, the correlation of EEG signals on the channel axis is far less strong than that on the time axis, so one-dimensional and two-dimensional convolution kernels are used for convolution and maximum pooling operations. The early layer uses a one-dimensional kernel to learn the features on the time axis. When the time axis size is reduced to a size close to the channel axis, a two-dimensional kernel is used to learn the features in the space and time domain.

[0087] For example, the pulse neural network model used in this embodiment is as follows: Figure 5 As shown, in addition to the input layer and the output layer, the proposed model also consists of 5 convolutional layers and 2 fully connected layers;

[0088] The five convolutional layers specifically include:

[0089] The first convolution layer is a one-dimensional convolution, the number of input channels is 1, the number of output channels is 16, the convolution kernel size is (1,21), the neuron is an IF neuron, and the maximum pooling layer connected is (1,10);

[0090] The second convolutional layer is a one-dimensional convolution, the number of input channels is 16, the number of output channels is 32, the convolution kernel size is (1,15), the neuron is an IF neuron, and the maximum pooling layer connected is (1,6);

[0091] The third convolution layer is a one-dimensional convolution, the number of input channels is 32, the number of output channels is 64, the convolution kernel size is (1,5), the neuron is an IF neuron, and the maximum pooling layer connected is (1,2);

[0092] The fourth convolution layer is a two-dimensional convolution, the number of input channels is 64, the number of output channels is 128, the convolution kernel size is (3,3), the neuron is an IF neuron, and the maximum pooling layer connected is (4,4);

[0093] A fifth convolutional layer, wherein the fifth convolutional layer is a two-dimensional convolution, the number of input channels is 128, the number of output channels is 256, the convolution kernel size is (4,4), and the neurons are IF neurons, wherein the fifth convolutional layer includes 524288 weights;

[0094] The two fully connected layers specifically include:

[0095] A sixth fully connected layer, wherein the number of input channels of the sixth fully connected layer is 256, the number of output channels is 64, and the neurons are LIF neurons;

[0096] The seventh fully connected layer has 64 input channels, 2 output channels, and LIF neurons.

[0097] The neural network optimization module is used to perform hardware optimization on the neural network;

[0098] The optimization strategy implemented by the neural network optimization module is specifically: modifying the sizes of the convolution kernel and the pooling kernel to a power of 2, and reducing the number of weights by y, where y is a weight control factor.

[0099] Exemplarily, this embodiment uses SGD optimizer to minimize loss, the learning rate is 0.5, the tau constant of the LIF model is 2, the training batch size is 32, and the training epoch is 100. The spiking neural network model in this embodiment is implemented in Python 3.6, and the spiking neural network is built using SpikingJelly;

[0100] In this embodiment, the spiking neural network structure is hardware optimized, and the new spiking neural network structure is as follows:

[0101] First, the time axis data of the input data is cut from 2560 to 2559.

[0102] The first convolution layer has an input channel number of 1, an output channel number of 16, a convolution kernel size of (1,16), an IF neuron, and a maximum pooling layer of (1,8); the input image size of the first convolution layer is 18*2560, and the output image size is 18*318; then a data 0 is padded at the end of the data on the time axis, and the image size becomes 18*319;

[0103] The second convolutional layer has 16 input channels, 32 output channels, a convolution kernel size of (1,8), IF neurons, and a maximum pooling layer of (1,8). The input image size of the second convolutional layer is 18*254, and the output image size is 18*39.

[0104] The third convolutional layer has 32 input channels, 64 output channels, a convolution kernel size of (1,4), IF neurons, and a maximum pooling layer of (1,2). The input image size of the third convolutional layer is 18*40, and the output image size is 18*18.

[0105] The fourth convolutional layer has 64 input channels, 128 output channels, a convolution kernel size of (3,3), IF neurons, and a maximum pooling layer of (8,8). The size of the input image of the fourth convolutional layer is 18*18, and the size of the output image is 2*2.

[0106] The fifth convolutional layer has 128 input channels, 128 output channels, a convolution kernel size of (2,2), and IF neurons. The size of the input image of the fifth convolutional layer is 4*4, and the size of the output image is 1*1.

[0107] The sixth fully connected layer has 128 input channels and 64 output channels, and the neurons used are LIF neurons;

[0108] The seventh fully connected layer has 64 input channels and 2 output channels, and the neurons used are LIF neurons.

[0109] The emission mode determination module performs emission mode determination according to the time step, wherein the emission mode determination module is used for software optimization;

[0110] like Figure 6 As shown, the transmission mode determination module is used to run the following specific steps:

[0111] S21: preset the running time threshold of the time step and make neuron firing mode judgment;

[0112] S22: When the time step reaches the time threshold, the neuron is judged to be in normal firing mode, and the pulse is allowed to be fired normally according to the rules;

[0113] S23: When the time step reaches the time threshold, the neuron is judged to be in the inhibited firing mode, and the subsequent pulse firing is inhibited.

[0114] In this embodiment, the use of such adjustable inhibitory neurons can, to a certain extent, constrain neurons that fire too frequently, while retaining the possibility of firing neurons with relatively low firing probabilities, thereby helping to avoid situations where some neurons fire excessively and some neurons "die". In addition to reducing the number of firings, reducing power consumption, and balancing excessive firing and insufficient firing, this method can also reduce time. Due to the addition of an inhibitory mechanism, the inhibited neurons will no longer continue to trigger in the future, so an "early stop" mechanism can be added to directly end this time step and enter the next stage.

[0115] like Figure 7 As shown, the accelerator module is used for hardware implementation of a pulse neural network with adjustable emission suppression;

[0116] The accelerator module includes: a computing unit, a storage unit, and a control unit.

[0117] The computing unit is used to perform convolution, full connection, activation and pooling computing operations in the network model;

[0118] The storage unit is used to store input data, intermediate calculation results and model weight parameters;

[0119] The control unit is used to coordinate and manage the processes and data flows of various computing and storage operations.

[0120] The computing unit includes a one-dimensional convolution computing subunit, a two-dimensional convolution computing subunit, a fully connected computing subunit, a neuron excitation subunit, and a maximum pooling computing subunit for implementing the network model;

[0121] In this embodiment, PE needs to support multiplication and accumulation calculations with sizes of 16, 64, 128, 512, and 576, and maximum pooling operations with window sizes of 2, 8, and 64, respectively.

[0122] A reconfigurable design is adopted, and different computing modes can reuse some of the same computing resources to improve the utilization of hardware resources. The computing unit includes four identical computing units (PEs), the difference is that the weights used are different, aiming to improve the parallelism of the calculation and shorten the calculation time.

[0123] The storage unit includes an input data storage subunit, an intermediate result storage subunit and a weight storage subunit;

[0124] Exemplarily, in this embodiment, after each layer of calculation is completed, the storage for storing the result data will serve as the storage for providing input data, and the storage for providing input data will serve as the storage for receiving the result. Weight memory access needs to be designed in conjunction with the calculation process, and the issue of saving memory space needs to be taken into account, so the design of weight memory access is relatively complex. According to these rules, the final weight memory access is divided into three parts, respectively recorded as the first layer of weight memory access group, weight memory access group 1, and weight memory access group 2. The first layer of the weight memory access group is mainly used to store data for the first layer of convolution operations. Weight memory access group 1 and weight memory access group 2 are used to store data for operations in other layers other than the first layer.

[0125] For example, in this embodiment, in addition to the above-mentioned units, a direct memory access unit (DMA) and an AXI bus interface are also designed. Finally, the pulse neural network accelerator module with adjustable emission suppression in this embodiment can be connected to DDR via the AXI bus, so that DDR can store input data and weights, which is convenient for updating different input data and weights, and the overall accelerator can be connected to the CPU or other control modules to control the use of the neural network accelerator.

[0126] The control unit includes a main mother controller, a one-dimensional convolution memory access sub-controller, a two-dimensional convolution memory access sub-controller, a maximum pooling memory access sub-controller, a fully connected memory access sub-controller, a DMA sub-controller and an output sub-controller.

[0127] Exemplarily, in this embodiment, the main controller is responsible for scheduling the overall workflow and is composed of a state machine. The one-dimensional convolution memory access controller works in the one-dimensional convolution mode of the first, second and third layers respectively. Since the one-dimensional convolution and its matching maximum pooling will be completed together in the computing unit, there is no need to store them twice. The two-dimensional convolution memory access controller is mainly for the two-dimensional convolution operations of the fourth and fifth layers. The maximum pooling memory access controller is designed mainly for the pooling operation after the fourth layer of two-dimensional convolution. The fully connected memory access controller is designed mainly for the fully connected operations of the sixth and seventh layers. The DMA controller is mainly used to move the original input data and the first batch of weight data to the SRAM during initialization and to update the weight data subsequently. The output controller receives the output result and temporarily stores it, then sends the output result to the outside and synchronously pulls up the output valid signal.

[0128] Embodiment 2:

[0129] This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0130] The processor runs the above-mentioned epilepsy prediction system based on a pulse neural network with adjustable emission inhibition by calling a computer program stored in the memory.

[0131] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) and one or more memories, wherein the memory stores at least one computer program, which is loaded and run by the processor to implement an epilepsy prediction system based on a pulse neural network with adjustable emission suppression provided by the above method embodiment. The electronic device may also include other components for implementing the functions of the device, for example, the electronic device may also have components such as a wired or wireless network interface and an input and output interface, so as to input and output data. This embodiment will not be described in detail here.

[0132] Embodiment three:

[0133] This embodiment provides a computer-readable storage medium having a rewritable computer program stored thereon;

[0134] When the computer program is executed on a computer device, the computer device is caused to execute the above-mentioned epilepsy prediction system based on a pulse neural network with adjustable inhibitory emission.

[0135] For example, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0136] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0137] It should be understood that determining B based on A does not mean determining B only based on A. B can also be determined based on A and / or other information.

[0138] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through a wired network or / and a wireless network. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0139] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0141] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0144] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0145] In summary, compared with the prior art, the technical effects of the present invention are as follows:

[0146] The present invention uses a spiking neural network (SNN), which simulates the working mode of biological neurons and can naturally and efficiently process biological signals such as time series data such as brain waves. Compared with CNN and RNN, SNN saves computing resources through an event-driven mechanism, has the characteristics of low power consumption and high computing efficiency, and is an ideal choice for epilepsy prediction, especially for portable and implantable devices; at the same time, the neurons of the present invention adopt two working modes, one is normal emission and the other is suppression emission. Before the time step runs to the time threshold, the neuron is in the normal emission mode, allowing the pulse to be emitted normally according to the rules. When a certain time threshold is exceeded, the suppression emission mode is turned on. If the neuron emits a pulse again at this time, it will suppress the subsequent pulse emission. Using this adjustable inhibitory neuron, neurons that are emitted too frequently can be constrained to a certain extent, while retaining the possibility of firing neurons with a relatively low probability of firing, which is conducive to avoiding the situation where some neurons are over-emitted and some neurons are "dead". The present invention realizes the improvement of the computing speed of the spiking neural network, which can meet the needs of wearable devices.

[0147] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An epilepsy prediction system based on a spiking neural network with adjustable inhibitory emission, characterized in that: The system includes the following modules: Data preprocessing module, pulse neural network module, neural network optimization module, emission mode judgment module and accelerator module; The data preprocessing module is used to obtain a data set for training and testing of a spiking neural network; The spiking neural network module is used to construct a spiking neural network; The neural network optimization module is used to perform hardware optimization on the neural network; The emission mode judgment module judges the emission mode according to the time step, thereby optimizing the software; The accelerator module is used for hardware implementation of a pulse neural network with adjustable emission suppression.

2. The epilepsy prediction system based on a spiking neural network with adjustable emission inhibition according to claim 1, characterized in that: The data preprocessing module is used to perform the following specific steps: S11: The EEG data of several subjects with epileptic seizures under a fixed electrode configuration were selected from the CHB-MIT dataset to form the training set of the spiking neural network model; S12: extracting the EEG data of epileptic seizures, and performing stage labeling processing on the EEG data of epileptic seizures according to the data characteristics of the pre-ictal period and the interictal period; S13: performing deep learning to extract features in the time domain on the epileptic seizure EEG data after the stage labeling process, so as to obtain the time domain features of the epileptic seizure EEG data; S14: extracting window samples of the pre-ictal period and the inter-ictal period, mixing and shuffling the order of the window samples of the pre-ictal period and the inter-ictal period, and dividing the window samples into a training set and a test set at a ratio of 4:1, wherein the unit time length of the window samples is T.

3. The epilepsy prediction system based on a pulse neural network with adjustable emission inhibition according to claim 2, characterized in that: The stage annotation processing in S12 includes: The 30 minutes before the onset of the seizure were selected as the preictal period, the 2 hours after the onset of the epilepsy were selected as the postictal period, and the data from two hours to the next preictal period were counted as the interictal period; The method for obtaining the pre-ictal data includes: firstly screening out effective seizures, then selecting a collection segment of the effective seizures, and finally selecting data from 5 minutes to 35 minutes before the epileptic seizure in the collection segment as the pre-ictal data; The method for acquiring the interictal data includes: first, screening the collected segments outside 2 hours before and after the epileptic seizure, secondly filtering out the segments 2 hours before and after the missing segments among these collected segments, and finally filtering out the collected segments of the first 2 hours and the last 2 hours.

4. The epilepsy prediction system based on a spiking neural network with adjustable emission inhibition according to claim 3, characterized in that: The pulse neural network module includes: an input layer, an output layer, 5 convolutional layers and 2 fully connected layers, wherein the neuron model of the pulse neural network includes: an IF model and a LIF model.

5. The epilepsy prediction system based on a spiking neural network with adjustable emission inhibition according to claim 4, characterized in that: The five convolutional layers specifically include: The first convolution layer is a one-dimensional convolution, the number of input channels is 1, the number of output channels is 16, the convolution kernel size is (1,21), the neuron is an IF neuron, and the maximum pooling layer connected is (1,10); The second convolutional layer is a one-dimensional convolution, the number of input channels is 16, the number of output channels is 32, the convolution kernel size is (1,15), the neuron is an IF neuron, and the maximum pooling layer connected is (1,6); The third convolution layer is a one-dimensional convolution, the number of input channels is 32, the number of output channels is 64, the convolution kernel size is (1,5), the neuron is an IF neuron, and the maximum pooling layer connected is (1,2); The fourth convolution layer is a two-dimensional convolution, the number of input channels is 64, the number of output channels is 128, the convolution kernel size is (3,3), the neuron is an IF neuron, and the maximum pooling layer connected is (4,4); A fifth convolutional layer, wherein the fifth convolutional layer is a two-dimensional convolution, the number of input channels is 128, the number of output channels is 256, the convolution kernel size is (4,4), and the neurons are IF neurons, wherein the fifth convolutional layer includes 524288 weights; The two fully connected layers specifically include: A sixth fully connected layer, wherein the number of input channels of the sixth fully connected layer is 256, the number of output channels is 64, and the neurons are LIF neurons; The seventh fully connected layer has 64 input channels, 2 output channels, and LIF neurons.

6. The epilepsy prediction system based on a spiking neural network with adjustable emission inhibition according to claim 5, characterized in that: The hardware optimization strategy executed by the neural network optimization module is specifically: modifying the sizes of the convolution kernel and the pooling kernel to a power of 2, and reducing the number of weights by y, where y is a weight control factor.

7. The epilepsy prediction system based on a spiking neural network with adjustable emission inhibition according to claim 6, characterized in that: The transmission mode determination module is used to execute the following specific steps: S21: preset the running time threshold of the time step and make neuron firing mode judgment; S22: When the time step reaches the time threshold, the neuron is judged to be in normal firing mode, and the pulse is allowed to be fired normally according to the rules; S23: When the time step reaches the time threshold, the neuron is judged to be in the inhibited firing mode, and the subsequent pulse firing is inhibited.

8. The epilepsy prediction system based on a spiking neural network with adjustable emission inhibition according to claim 7, characterized in that: The accelerator module includes: a computing unit, a storage unit, and a control unit. The computing unit is used to perform convolution, full connection, activation, and pooling computing operations in the network model; The storage unit is used to store input data, intermediate calculation results and model weight parameters; The control unit is used to coordinate and manage the processes and data flows of various computing and storage operations. The computing unit includes a one-dimensional convolution computing subunit, a two-dimensional convolution computing subunit, a fully connected computing subunit, a neuron excitation subunit, and a maximum pooling computing subunit for implementing the network model; The storage unit includes an input data storage subunit, an intermediate result storage subunit and a weight storage subunit; The control unit includes a main mother controller, a one-dimensional convolution memory access sub-controller, a two-dimensional convolution memory access sub-controller, a maximum pooling memory access sub-controller, a fully connected memory access sub-controller, a DMA sub-controller and an output sub-controller.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the epilepsy prediction system based on a pulse neural network with adjustable inhibitory emission as described in any one of claims 1 to 8 is run.

10. An electronic device, characterized in that: include: A memory for storing instructions; A processor is used to execute the instructions so that the device runs a module that implements an epilepsy prediction system based on a pulse neural network with adjustable emission inhibition as described in any one of claims 1 to 8.

Images