Multi-mode epilepsy monitoring system

Abstract

The invention relates to the technical field of medical monitoring, in particular to a multi-mode epilepsy monitoring system which comprises a brain wave collecting module, an electrocardiosignal collecting module and a skin conductivity collecting module. The signal assembly module is used for processing the brain wave signal, the electrocardiosignal and the real-time skin conductivity to obtain a multi-modal signal feature and constructing a short-time prediction image; and the prediction module predicts and outputs an epilepsy risk score according to the short-term prediction image. In order to solve the problem that in the prior art, an epilepsy monitoring system carries out monitoring only based on a single dimension, and misinformation is likely to happen, in the scheme, three dimensions of electrocardiosignals, brain wave signals and real-time skin conductivity are introduced to detect an epileptic, then prediction is carried out based on multi-modal signals, the risk of epilepsy of the patient is evaluated, and a score is generated; therefore, the false alarm rate is effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of medical monitoring technology, specifically to a multimodal epilepsy monitoring system. Background Technology

[0002] Epilepsy is a brain disorder characterized by recurrent, non-triggering (i.e., not caused by other diseases or conditions) seizures. Seizures result from abnormal, excessive, or synchronized discharges of neurons in the brain, leading to a variety of transient symptoms depending on the affected brain region. Because seizures are sudden and often cause falls and loss of self-control, patients are highly susceptible to injury during an attack. Therefore, monitoring and prediction of epilepsy have become a key focus in the nursing profession.

[0003] Existing technologies include monitoring methods that utilize brainwaves.

[0004] For example, patent document CN202310071071.X discloses an automatic epileptic seizure monitoring system and device, including: an EEG acquisition module, an EEG data preprocessing module, an EEG data feature extraction module, an epileptiform EEG detection module, a video acquisition module, a video feature extraction module, an epileptic seizure detection module, and a patient protection module. The automatic epileptic seizure monitoring system analyzes both EEG and video information simultaneously, enabling more accurate detection of whether a patient is having an epileptic seizure. By first analyzing the EEG signal and then analyzing the video data when epileptiform EEG is detected, the computational load can be greatly reduced without compromising detection accuracy. By simultaneously analyzing EEG and video information, the restraint straps are only used to restrict the patient when the patient has an epileptic seizure and experiences significant limb movement. This minimizes the impact on the patient's normal activities when not having an epileptic seizure and protects the patient from accidents during a seizure, reducing the burden on caregivers.

[0005] For example, patent document CN202111569988.X discloses a home-based interactive intelligent monitoring system and method for epileptic seizures. The system includes a carrier assembly component, an EEG detection component, a data transmission component, a positioning beacon component, an image acquisition component, and a monitoring and care component. The detection electrodes in the EEG detection component are in contact with the EEG signal acquisition sites on the patient's head, corresponding to F3 and F4 in the forehead, T7 and T8 in the midtemporal region, and O1 and O2 in the frontal lobe according to the standard EEG electrode placement. This monitoring system can acquire, preprocess, and extract features of EEG signals locally in real time to determine whether there is an epileptic seizure and assess the seizure level. When an epileptic seizure is detected, the system calls the camera to capture and upload the patient's current video, reminding the caregiver to take timely measures to check the patient's condition and avoid life-threatening situations.

[0006] However, in practice, the inventors discovered that this type of technical solution typically monitors patients based on only a single dimension, and is greatly affected by other physiological factors of the patient. For example, when a patient exercises, signals such as electrocardiogram (ECG) and electroencephalogram (EEG) will change significantly, which can lead to the monitoring system being falsely triggered. Summary of the Invention

[0007] To address the aforementioned problems in the existing technology, a multimodal epilepsy monitoring system is provided.

[0008] The specific technical solution is as follows:

[0009] A multimodal epilepsy monitoring system, comprising:

[0010] The brainwave acquisition module is connected to an external scalp electrode patch to acquire brainwave signals.

[0011] An electrocardiogram (ECG) signal acquisition module, which is connected to an external ECG monitor to acquire ECG signals;

[0012] A skin conductivity acquisition module, which is connected to an external skin electrode patch to acquire real-time skin conductivity.

[0013] A signal assembly module, which is connected to the electroencephalogram (EEG) acquisition module, the electrocardiogram (ECG) signal acquisition module, and the skin conductance acquisition module, respectively;

[0014] The signal assembly module processes the EEG signal, the ECG signal, and the real-time skin conductivity to obtain multimodal signal features and construct a short-term prediction image;

[0015] A prediction module, connected to the signal assembly module, predicts and outputs an epilepsy risk score based on the short-term prediction image.

[0016] On the other hand, the signal assembly module includes:

[0017] The sequence alignment module reads the timestamps of the EEG signal, the ECG signal, and the real-time skin conductivity, and aligns the sequence signals in the time domain according to the timestamps to form a multi-signal sequence.

[0018] A filtering module is connected to the sequence alignment module;

[0019] The filtering module filters the multiple signal sequences to form a filtered sequence;

[0020] A feature extraction module is connected to the filtering module;

[0021] The feature extraction module assembles the filtered sequence to obtain a reference image, and extracts the multimodal signal feature sequence from the reference image.

[0022] On the other hand, the signal assembly module also includes:

[0023] A time configuration module calibrates the acquisition timestamps of the EEG acquisition module, the ECG signal acquisition module, and the skin conductance acquisition module according to a predetermined calibration timer.

[0024] On the other hand, the sequence alignment module includes:

[0025] A timestamp alignment module reads the timestamps of the EEG signal, the ECG signal, and the real-time skin conductivity, and aligns them according to the timestamps to obtain a multi-signal alignment sequence.

[0026] A resampling module, wherein the resampling module is connected to the timestamp alignment module;

[0027] The resampling module resamples the multi-signal aligned sequence to obtain a multi-signal resampled sequence.

[0028] The resampling module marks missing sampling points during the resampling process;

[0029] A filling module is connected to the resampling module;

[0030] The filling module performs mean processing on the adjacent points of the missing sampling points to fill the missing sampling points and obtain the multi-signal sequence.

[0031] On the other hand, the filtering module includes:

[0032] A channel denoising module, wherein the channel denoising module performs high-frequency noise removal on each channel of the multi-signal sequence to obtain a multi-signal denoised sequence;

[0033] A motion detection module, wherein the motion detection module is connected to the channel noise reduction module;

[0034] The motion detection module detects the ECG channels in the multi-signal denoising sequence to obtain ECG signal mutation segments, and uses the ECG signal mutation segments as motion intervals;

[0035] A replacement module, wherein the replacement module is connected to the motion detection module;

[0036] The replacement module searches for a forward interval for the skin conductivity channel in the multi-signal denoising sequence according to the motion interval, and uses the forward interval to replace the data in the motion interval to obtain the filtered sequence.

[0037] On the other hand, the feature extraction module includes:

[0038] A multi-channel overlay module is provided, wherein each channel in the filter sequence is colored separately to form channel signal images with different colors, and then the channel signal images are overlaid to obtain an overlay image;

[0039] A sliding window module, wherein the sliding window module is connected to the multi-channel overlay module;

[0040] The sliding window module uses a sliding window to crop the overlapping image to obtain the short-term prediction image.

[0041] On the other hand, the prediction module includes:

[0042] The image feature extraction module extracts high-dimensional features from the short-time prediction image;

[0043] A classifier module, which is connected to the image feature extraction module;

[0044] The classifier module predicts the high-dimensional features to obtain the epilepsy risk score.

[0045] On the other hand, the image feature extraction module includes:

[0046] A first convolutional network receives the short-term prediction image and extracts the short-term prediction image using a series of convolutional kernels to obtain a first convolutional image;

[0047] A downsampling layer, wherein the downsampling layer is connected to the first convolutional network;

[0048] The downsampling layer downsamples the first convolutional image to obtain a downsampled image;

[0049] A second convolutional network, which is connected to the downsampling layer;

[0050] The second convolutional network uses a series of convolutional kernels to extract the downsampled image to obtain the second convolutional image;

[0051] An upsampling layer, wherein the upsampling layer is connected to the second convolutional network;

[0052] The upsampling layer performs upsampling processing on the second product image to obtain the high-dimensional features.

[0053] On the other hand, the classifier module is built based on the SVM classifier.

[0054] The above technical solution has the following advantages or beneficial effects:

[0055] To address the issue of false alarms in existing epilepsy monitoring systems that rely on a single dimension for monitoring, this solution introduces three dimensions—electrocardiogram (ECG) signals, electroencephalogram (EEG) signals, and real-time skin conductivity—to detect epilepsy patients. Based on these multimodal signals, predictions are made to assess the patient's risk of developing epilepsy and generate a score, thereby effectively reducing the false alarm rate. Attached Figure Description

[0056] Embodiments of the invention will be described more fully with reference to the accompanying drawings. However, the drawings are for illustration and explanation only and do not constitute a limitation on the scope of the invention.

[0057] Figure 1 This is an overall schematic diagram of an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of the signal assembly module in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the time configuration module in an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of the sequence alignment module in an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the filtering module in an embodiment of the present invention;

[0062] Figure 6 This is a schematic diagram of the feature extraction module in an embodiment of the present invention;

[0063] Figure 7 This is a schematic diagram of the prediction module in an embodiment of the present invention;

[0064] Figure 8 This is a schematic diagram of the image feature extraction module in an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0067] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0068] This invention includes:

[0069] A multimodal epilepsy monitoring system, such as Figure 1 As shown, it includes:

[0070] EEG acquisition module 1, which is connected to an external scalp electrode patch to acquire brainwave signals;

[0071] ECG signal acquisition module 2 is connected to an external ECG monitor to acquire ECG signals.

[0072] Skin conductivity acquisition module 3, which is connected to an external skin electrode patch to acquire real-time skin conductivity;

[0073] Signal assembly module 4 is connected to EEG acquisition module 1, ECG signal acquisition module 2 and skin conductance acquisition module 3 respectively.

[0074] Signal assembly module 4 processes EEG signals, ECG signals, and real-time skin conductivity to obtain multimodal signal features and construct short-time prediction images;

[0075] Prediction module 5 is connected to signal assembly module 4. Prediction module 5 predicts and outputs an epilepsy risk score based on a short-term prediction image.

[0076] Specifically, to address the issue that existing epilepsy monitoring systems, which rely on a single dimension for monitoring, are prone to false alarms, this solution introduces three dimensions—electrocardiogram (ECG) signals, electroencephalogram (EEG) signals, and real-time skin conductivity—to detect epilepsy patients. Based on these multimodal signals, predictions are made to assess the patient's risk of developing epilepsy and generate a score, thereby effectively reducing the false alarm rate.

[0077] Specifically, the aforementioned multimodal epilepsy monitoring system is implemented using a combination of software and hardware, typically suitable for scenarios where patients do not move frequently, such as inpatient wards or home monitoring. The physical carrier of this multimodal epilepsy monitoring system is a set of computer devices configured with specific software, connected to external scalp electrode patches, an electrocardiogram (ECG) monitor, and skin electrode patches. Patients provide corresponding signals to the monitoring system by wearing the scalp electrode patches, ECG monitor, and skin electrode patches.

[0078] Among them, scalp electrode patches are patches attached to specific areas of the patient's scalp to sense and collect the patient's electroencephalogram (EEG) signals. Electrocardiogram (ECG) monitors collect the patient's ECG signals using devices such as heart rate monitors. Skin electrode patches are a group of adjacent electrodes attached to the patient's skin surface, measuring the skin's conductivity in the form of microcurrents. Because patients may experience corresponding muscle twitching, sweating, or other signs before a seizure, skin conductivity may change abruptly during this phase; this signal can, to some extent, be used to predict whether a seizure is imminent.

[0079] In response to the three sets of signals mentioned above, this solution constructs an EEG acquisition module 1, an ECG signal acquisition module 2, and a skin conductance acquisition module 3 to interact with each acquisition device, thereby obtaining the corresponding signals for input.

[0080] In this context, real-time skin conductivity is represented by a sequence of sampled values ​​collected in chronological order.

[0081] To enable prediction based on multimodal signals, this embodiment also processes the signals through a signal assembly module 4. The processing mainly includes denoising the acquired signals to remove noise components introduced during high-frequency signal transmission; and aligning the acquired signals on the time axis to facilitate subsequent model prediction.

[0082] Finally, considering the significant differences in numerical range and variation patterns among the sequences of electrocardiogram signals, electroencephalogram signals, and real-time skin conductivity, and the difficulty in normalizing them, this solution adopts a graphical processing approach. That is, the signals from multiple dimensions are rendered and superimposed into an image to construct a short-term prediction image.

[0083] During normalization, the scaling ratio should be controlled appropriately to preserve local waveform signals. A common approach is to use relatively subtle EEG signals as a reference image and then scale the ECG signals and real-time skin conductivity to avoid the loss of subtle waveform features.

[0084] The subsequent prediction module 5 will use a pre-trained artificial intelligence model to extract the waveform features of the signal in the short-term prediction image, classify the waveform features according to the pre-training results, and convert them into the corresponding risk level for prompting.

[0085] In one embodiment, such as Figure 2 As shown, the signal assembly module 4 includes:

[0086] The sequence alignment module 41 reads the timestamps of the EEG signal, ECG signal and real-time skin conductivity, and aligns the sequence signals in the time domain according to the timestamps to form a multi-signal sequence.

[0087] Filtering module 42, which is connected to sequence alignment module 41;

[0088] Filtering module 42 filters multiple signal sequences to form a filtered sequence;

[0089] Feature extraction module 43, which is connected to filtering module 42;

[0090] The feature extraction module 43 processes the filtered sequence to obtain a short-time predicted image.

[0091] Specifically, in order to achieve better image feature construction, in this embodiment, the input EEG signal, ECG signal and real-time skin conductance are first added to the cache queue.

[0092] Subsequently, for each sampling point of the EEG signal, ECG signal, and real-time skin conductivity, matching was performed by reading the timestamps to find sampling points that matched multiple channels. Then, the sequence signals were aligned in the time domain according to the timestamps to form a multi-signal sequence. At this point, the multi-signal sequence has multiple channels, including multiple channels of ECG signal, multiple channels of EEG signal, and a sequence of real-time skin conductivity.

[0093] Then, the filtering module 42 filters the multiple signal sequences to form a filtered sequence. This part mainly removes the spikes and noise introduced during signal transmission.

[0094] It is important to note that during this process, user behavior can also be detected, such as monitoring electrocardiogram (ECG) and electroencephalogram (EEG) signals. When a single indicator shows a sudden change, it may indicate that the user is in a period of activity. Such indicators are caused by the user's own activities, and these signals should be filtered out to avoid false detection by the model.

[0095] Finally, the feature extraction module 43 processes the filtered sequence to obtain a short-time predicted image.

[0096] In one embodiment, such as Figure 3 As shown, the signal assembly module 4 also includes:

[0097] The time configuration module 44 calibrates the acquisition timestamps of the EEG acquisition module 1, ECG signal acquisition module 2, and skin conductance acquisition module 3 according to a predetermined calibration timer.

[0098] Specifically, to facilitate the alignment of sampling times at each sampling point, a time configuration module 44 is provided in this embodiment, which contains a corresponding timer. When the timer is triggered, the time configuration module 44 interacts with the EEG acquisition module 1, ECG signal acquisition module 2, and skin conductance acquisition module 3 according to the pre-constructed message, and provides a new timestamp so that the EEG acquisition module 1, ECG signal acquisition module 2, and skin conductance acquisition module 3 have consistent acquisition timestamps.

[0099] In one embodiment, such as Figure 4 As shown, the sequence alignment module 41 includes:

[0100] The timestamp alignment module 411 reads the timestamps of the EEG signal, ECG signal and real-time skin conductivity, and aligns them according to the timestamps to obtain a multi-signal alignment sequence.

[0101] Resampling module 412, which is connected to timestamp alignment module 411;

[0102] The resampling module 412 resamples the multi-signal aligned sequence to obtain a multi-signal resampled sequence;

[0103] The resampling module 412 marks missing sampling points during the resampling process;

[0104] Filling module 413, which is connected to resampling module 412;

[0105] The filling module 413 performs mean processing on the adjacent points of the missing sampling points to fill the missing sampling points and obtain a multi-signal sequence.

[0106] Specifically, to achieve better sequence alignment, in this embodiment, the timestamp alignment module 411 first reads the timestamps of the EEG signal, ECG signal, and real-time skin conductivity, and then aligns them according to the timestamps to obtain a multi-signal aligned sequence. This multi-signal sequence has multiple channels, including multiple channels of the ECG signal, multiple channels of the EEG signal, and a sequence of real-time skin conductivity.

[0107] Based on this, and considering the differences in sampling intervals between sequences, a resampling module 412 is used to resample the multi-signal aligned sequence to obtain a multi-signal resampled sequence. During the sampling process, the resampling module 412 marks sampling points where accurate data was not obtained during resampling as missing sampling points.

[0108] Finally, the filling module 413 performs mean processing on the adjacent points of the missing sampling points to fill the missing sampling points and obtain a multi-signal sequence.

[0109] In one embodiment, such as Figure 5 As shown, the filtering module 42 includes:

[0110] The channel denoising module 421 performs high-frequency noise removal on each channel of the multi-signal sequence to obtain a multi-signal denoised sequence.

[0111] Motion detection module 422, which is connected to channel noise reduction module 421;

[0112] The motion detection module 422 detects the ECG channel in the multi-signal denoising sequence to obtain the ECG signal mutation segment, and uses the ECG signal mutation segment as the motion interval;

[0113] Replace module 423, and connect module 423 to motion detection module 42;

[0114] The replacement module 423 searches for the forward interval of the skin conductivity channel in the multi-signal denoising sequence according to the motion interval, and uses the forward interval to replace the data in the motion interval to obtain the filtered sequence.

[0115] Specifically, in order to achieve a better filtering effect, in this embodiment, the channel denoising module 421 first performs a fast Fourier transform on each channel of the multi-signal sequence, and then removes noise from obviously irrelevant high-frequency parts, thereby achieving filtering of each channel signal and obtaining a multi-signal denoised sequence.

[0116] Then, considering that patients may experience short-term mutations in specific signals due to physiological activities during the active period, in this embodiment, the motion detection module 422 is used to detect the ECG channels in the multi-signal denoising sequence to obtain the ECG signal mutation segment.

[0117] This detection process can measure heart rate and ST segment in ECG signals. When only the heart rate increases, while the ST segment and P wave show no significant changes, it indicates that the patient is engaged in normal activity. The process then reverses the lookup in the buffer queue to find the starting point of the heart rate change as the beginning of the exercise interval, and the point when the heart rate returns to normal as the end point of the exercise interval. Whether the heart rate is within the normal range can be determined based on a Long Short-Term Memory (LSTM) model.

[0118] Then, the replacement module 423 searches for the forward interval of the skin conductivity channel in the multi-signal denoising sequence according to the motion interval. That is, the interval with a predetermined length before the user begins to move. This part of the interval is used as the forward interval. The data of this part is used to replace the motion interval and the continuous data of the motion interval, so as to avoid the change of skin conductivity data from affecting the model detection.

[0119] Since this type of monitoring scenario is usually applicable to home monitoring and hospital monitoring, patients in this scenario usually do not exhibit continuous, large-amplitude movements. Therefore, the length of the movement range should be appropriately selected to avoid filtering the time process and making it impossible to effectively judge the changes in the patient's skin conductivity.

[0120] In one embodiment, such as Figure 6 As shown, the feature extraction module 43 includes:

[0121] The multi-channel overlay module 431 colors each channel in the filter sequence to form channel signal images with different colors, and then overlays the channel signal images to obtain an overlay image.

[0122] Sliding window module 432, which is connected to multi-channel overlapping module 431;

[0123] The sliding window module 432 uses a sliding window to crop overlapping images to obtain short-term prediction images.

[0124] Specifically, to achieve the conversion of the signal sequence into an image portion, in this embodiment, the multi-channel overlay module 431 first performs normalization processing on each channel in the filtered sequence, scaling it to a similar scale, and then performs coloring and rendering to form a multi-channel display image. Then, the channel signal images of different colors are overlaid on the same time axis.

[0125] Finally, the sliding window module 432 uses a sliding window to crop the overlapping images according to the input scale of the model, generating a short-term prediction image at the current time point.

[0126] In one embodiment, such as Figure 7 As shown, prediction module 5 includes:

[0127] Image feature extraction module 51 extracts high-dimensional features from short-time predicted images;

[0128] Classifier module 52, which is connected to image feature extraction module 51;

[0129] Classifier module 52 predicts high-dimensional features to obtain an epilepsy risk score.

[0130] Specifically, in order to achieve better prediction results, in this embodiment, the image feature extraction module 51 first uses a convolutional network to extract high-dimensional features from the short-term prediction image. Based on this, the classifier module 52 predicts the high-dimensional features to obtain an epilepsy risk score.

[0131] In one embodiment, such as Figure 8As shown, the image feature extraction module 51 includes:

[0132] The first convolutional network 511 receives a short-term prediction image and extracts the short-term prediction image using a series of convolutional kernels to obtain a first convolutional image.

[0133] Downsampling layer 512, which is connected to the first convolutional network 511;

[0134] The downsampling layer 512 downsamples the first convolutional image to obtain the downsampled image;

[0135] The second convolutional network 513 is connected to the downsampling layer 512;

[0136] The second convolutional network 513 uses a series of convolutional kernels to extract the downsampled image to obtain the second convolutional image;

[0137] Upsampling layer 514, which is connected to the second convolutional network 513;

[0138] The upsampling layer 514 performs upsampling processing on the second convolutional image to obtain high-dimensional features.

[0139] In one embodiment, the classifier module is built based on an SVM classifier.

[0140] Specifically, to achieve better classification results, in this embodiment, an SVM classifier is used to construct a classifier module 52. This SVM classifier is pre-trained using images of corresponding aliased signals. The training set covers sample data from multiple age groups and genders. It can predict risks based on the waveforms of overlapping images and assign them to corresponding risk scores to provide early warnings.

[0141] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-modal epilepsy monitoring system, characterized in that, The device comprises: a brain wave acquisition module connected with an external scalp electrode patch to acquire brain wave signals; an electrocardio signal acquisition module connected with an external electrocardio monitor to acquire electrocardio signals; a skin conductance acquisition module connected with an external skin electrode patch to acquire real-time skin conductance; a signal assembly module connected with the brain wave acquisition module, the electrocardio signal acquisition module and the skin conductance acquisition module; the signal assembly module processes the brain wave signals, the electrocardio signals and the real-time skin conductance to obtain multi-modal signal features and construct a short-time prediction image; a prediction module connected with the signal assembly module, which predicts an epilepsy risk score according to the short-time prediction image and outputs the score.

2. The multi-modal seizure monitoring system of claim 1, wherein, The signal assembly module comprises: a sequence alignment module which reads time stamps of the brain wave signals, the electrocardio signals and the real-time skin conductance respectively, and aligns the sequence signals in time domain according to the time stamps to form multi-signal sequences; a filter module connected with the sequence alignment module; the filter module filters the multi-signal sequences to form filtered sequences; a feature extraction module connected with the filter module; the feature extraction module processes the filtered sequences to obtain the short-time prediction image.

3. The multi-modal seizure monitoring system of claim 2, wherein, The signal assembly module further comprises: a time configuration module which calibrates the acquisition time stamps of the brain wave acquisition module, the electrocardio signal acquisition module and the skin conductance acquisition module according to a predetermined calibration timer.

4. The multi-modal seizure monitoring system of claim 2, wherein, The sequence alignment module comprises: a time stamp alignment module which reads time stamps of the brain wave signals, the electrocardio signals and the real-time skin conductance respectively, and aligns the time stamps to obtain multi-signal aligned sequences; a resampling module connected with the time stamp alignment module; the resampling module resamples the multi-signal aligned sequences to obtain multi-signal resampled sequences; the resampling module marks missing sampling points during the resampling process; a filling module connected with the resampling module; the filling module performs mean value processing on adjacent points of the missing sampling points to fill the missing sampling points to obtain the multi-signal sequences.

5. The multi-modal epilepsy monitoring system of claim 2, wherein, The filter module comprises: a channel denoising module which removes high-frequency noise from each channel of the multi-signal sequences to obtain multi-signal denoised sequences; a motion detection module connected with the channel denoising module; the motion detection module detects electrocardio channels in the multi-signal denoised sequences to obtain electrocardio signal mutation segments, and takes the electrocardio signal mutation segments as motion intervals; a replacement module connected with the motion detection module; The replacement module searches for a forward interval according to the motion interval for a skin conductivity channel in the multi-signal denoising sequence, and replaces data in the motion interval with the forward interval to obtain the filtered sequence.

6. The multi-modal epilepsy monitoring system of claim 2, wherein, The feature extraction module includes: A multi-channel overlapping module, which respectively colors each channel in the filtered sequence to form channel signal images with different colors, and then overlaps the channel signal images to obtain an overlapping image; A sliding window module connected to the multi-channel overlapping module; The sliding window module uses a sliding window to intercept the overlapping image to obtain the short-time prediction image.

7. The multi-modal epilepsy monitoring system of claim 1, wherein, The prediction module includes: An image feature extraction module that extracts high-dimensional features from the short-time prediction image; A classifier module connected to the image feature extraction module; The classifier module predicts the high-dimensional features to obtain the epilepsy risk score.

8. The multi-modal seizure monitoring system of claim 7, wherein, The image feature extraction module includes: A first convolutional network that receives the short-time prediction image and extracts the short-time prediction image using a series of convolution kernels to obtain a first convolutional image; A down-sampling layer connected to the first convolutional network; The down-sampling layer down-samples the first convolutional image to obtain a down-sampled image; A second convolutional network connected to the down-sampling layer; The second convolutional network extracts the down-sampled image using a series of convolution kernels to obtain a second convolutional image; An up-sampling layer connected to the second convolutional network; The up-sampling layer up-samples the second convolutional image to obtain the high-dimensional features.

9. The multi-modal seizure monitoring system of claim 7, wherein, The classifier module is based on an SVM classifier.

Images