Deep learning-based neonatal hypoxic-ischemic encephalopathy grading method and device

Abstract

The application provides a new-born baby HIE grading method and device based on deep learning, raw EEG data (including EEG data of multiple differential channels) of a HIE new-born baby is acquired first, the raw EEG data is preprocessed, then the preprocessed EEG data is predicted through a pre-trained neural network model, so that a HIE grading result of the HIE new-born baby is obtained. The application can improve the efficiency and accuracy of HIE grading of new-born babies.

Description

Technical Field

[0001] This invention relates to the field of biomedical engineering technology, and in particular to a method and apparatus for grading hypoxic-ischemic encephalopathy in newborns based on deep learning. Background Technology

[0002] Hypoxic-ischemic encephalopathy (HIE) is a functional disorder or brain lesion caused by hypoxia and ischemia in brain tissue. HIE can occur at any age, but it is most common in newborns. Onset can occur during pregnancy, childbirth, or postpartum. Symptoms of HIE typically depend on the severity of the brain damage and the affected brain regions. Based on severity, HIE is classified into four degrees: mild, moderate, severe, and no brain activity. Some HIE patients have no health problems or only mild or moderate effects, while others experience more serious health problems or even permanent disabilities, such as developmental delays, cerebral palsy (motor disorders), epilepsy, and cognitive impairment.

[0003] Currently, the main methods for diagnosing neonatal hypoxic-ischemic encephalopathy (HIE) are as follows: First, the physician visually assesses the newborn to obtain clinical characteristics (such as consciousness, muscle tone, primitive reflexes, seizures, central respiratory failure, pupillary changes, etc.). Then, based on these clinical characteristics, the neonatal shock score, transport physiological stability index (TRIPS) score, critical case score, emergency physiological score, and Apgar score are calculated, supplemented by MRI, EEG, and computed tomography (CT) images to diagnose whether the newborn has HIE. Because different degrees of HIE require drastically different treatments, the severity classification (grading) of HIE is a crucial step in the diagnostic process. Currently, HIE grading in clinical practice is mainly achieved by physicians observing the patient's EEG and amplitude integrated EEG (aEEG) and making judgments based on experience.

[0004] Electroencephalography (EEG) is a non-invasive neural signal acquisition technique that records electrical potential signals from the human head. It is characterized by low cost, portability, and high temporal resolution, making it highly valuable in basic research in cognitive neuroscience and clinical applications for neurological diseases such as epilepsy and depression. Continuous EEG monitoring plays an indispensable role in the multi-dimensional and precise management of neonatal neurocritical care. However, to date, there is no unified standard for HIE grading based on EEG data from newborns with hypoepileptic hyperencephalopathy (HIE). The interpretation and assessment of EEG activity patterns vary significantly among different doctors, and different hospitals employ different diagnostic methods, leading to a lack of consensus among medical personnel. aEEG monitoring, on the other hand, lacks in-depth analysis of raw EEG data and provides an incomplete assessment of the degree of brain damage. Manual interpretation of EEG signals is extremely complex and noisy. Furthermore, the differences in EEG data between children with mild to moderate HIE are not significant to the naked eye, posing a significant challenge to doctors' diagnosis and increasing the possibility of misdiagnosis. In addition, the treatment window for HIE is very short, and doctors often need a long time to classify HIE patients. This may cause HIE patients to miss the best treatment time, which is especially serious in some areas and hospitals that lack highly qualified psychiatrists. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method and apparatus for grading hypoxic-ischemic encephalopathy in newborns based on deep learning, so as to alleviate the above-mentioned problems existing in related technologies.

[0006] In a first aspect, embodiments of the present invention provide a deep learning-based method for grading hypoxic-ischemic encephalopathy (HIE) in newborns. The method includes: acquiring raw electroencephalogram (EEG) data of HIE newborns; wherein the raw EEG data includes EEG data from multiple differential channels; preprocessing the raw EEG data; wherein the preprocessing includes at least one of the following: screening, sampling, standardization, and slicing; and predicting the preprocessed EEG data using a pre-trained neural network model to obtain the HIE grading result of the newborns.

[0007] Secondly, embodiments of the present invention also provide a deep learning-based grading device for hypoxic-ischemic encephalopathy (HIE) in newborns. The device includes: an acquisition module for acquiring raw EEG data of HIE newborns; wherein the raw EEG data includes EEG data from multiple differential channels; a preprocessing module for preprocessing the raw EEG data; wherein the preprocessing includes at least one of the following: screening, sampling, standardization, and slicing; and a prediction module for predicting the preprocessed EEG data using a pre-trained neural network model to obtain the HIE grading result of the newborns.

[0008] This invention provides a method and apparatus for grading hypoxic-ischemic encephalopathy (HIE) in newborns based on deep learning. First, raw EEG data (including EEG data from multiple differential channels) of the newborns with HIE is acquired. Then, the raw EEG data is preprocessed. Finally, a pre-trained neural network model is used to predict the HIE grading result of the newborn. Using this technique, only the raw EEG data of the newborn with HIE is needed for automatic HIE grading via a neural network model, significantly reducing the time required for diagnosis and ensuring timely treatment for HIE patients. Even in areas and hospitals lacking highly skilled psychiatrists, rapid and accurate HIE grading can be achieved.

[0009] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0010] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0011] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a deep learning-based grading method for hypoxic-ischemic encephalopathy in newborns, as described in an embodiment of the present invention.

[0013] Figure 2 This is a schematic diagram of the electrode placement method of the EEG cap in an embodiment of the present invention;

[0014] Figure 3 This is a schematic diagram of one structure of the neural network model in an embodiment of the present invention;

[0015] Figure 4 This is another structural diagram of the neural network model in an embodiment of the present invention;

[0016] Figure 5 This is a schematic diagram of channel attention calculation in an embodiment of the present invention;

[0017] Figure 6This is an example diagram of a deep learning-based grading method for hypoxic-ischemic encephalopathy in newborns, as described in an embodiment of the present invention.

[0018] Figure 7 This is a structural example diagram of the neural network model in an embodiment of the present invention;

[0019] Figure 8 This is a structural example diagram of the residual convolution block in an embodiment of the present invention;

[0020] Figure 9 This is a structural example diagram of a convolutional layer in an embodiment of the present invention;

[0021] Figure 10 This is a schematic diagram of a deep learning-based grading device for hypoxic-ischemic encephalopathy in newborns, as described in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] To facilitate understanding of this embodiment, a detailed description of a deep learning-based grading method for hypoxic-ischemic encephalopathy in newborns, as disclosed in this embodiment of the invention, will be provided first. (See [link to relevant documentation]). Figure 1 As shown, the method may include the following steps:

[0024] Step S102: Obtain raw EEG data from the newborn with HIE.

[0025] The raw EEG data may include EEG data from multiple differential channels.

[0026] The raw EEG data described above can be acquired by placing an EEG cap on the head of a newborn with HIE. The electrode placement on the EEG cap can directly follow international standard electrode placement methods (such as the 10-20 standard electrode placement method specified by the International Society for Electroencephalography), or it can be customized or adjusted according to actual needs. For example, see... Figure 2 As shown, the 10-20 standard electrode placement method is slightly modified. The current electrodes Fp3 and Fp4 are placed at a position 10% of the length of the sagittal line (the line connecting the anterior and posterior ends of the nasal root to the external occipital protuberance) posteriorly of the original electrodes Fp1 and Fp2. Figure 2In the diagram, hollow dots without text represent the electrode placement positions in the 10-20 standard electrode placement method, while hollow dots with text represent the actual electrode placement positions after calibration by the doctor. The measuring electrodes include Fp3, Fp4, T3, C3, Cz, C4, T4, P3, Pz, P4, O1, and O2, and the reference electrodes include GND and REF.

[0027] When doctors make diagnoses, they use differential channel EEG data instead of single-lead EEG data. This means the raw EEG data is actually multi-channel EEG data that has been differentially processed (the difference between data collected by one measuring electrode and data collected by another). According to... Figure 2 The electrode placement shown indicates that the original EEG data consists of 9 differential channels, namely C3-C4, C3-O1, C4-O2, Fp1-C3, Fp1-T3, Fp2-C4, Fp2-T4, T3-O1, and T4-O2.

[0028] Step S104: Preprocess the raw EEG data.

[0029] Preprocessing may include screening, sampling, standardization, slicing, etc., and there are no restrictions on this.

[0030] Step S106: The pre-trained neural network model is used to predict the pre-processed EEG data to obtain the HIE grading results of the newborn with HIE.

[0031] This invention provides a deep learning-based method for grading hypoxic-ischemic encephalopathy (HIE) in newborns. First, raw EEG data (including EEG data from multiple differential channels) is acquired. Then, the raw EEG data is preprocessed. Finally, a pre-trained neural network model is used to predict the HIE grading result of the newborn. Using this technique, only the raw EEG data of the newborn is needed for automatic HIE grading via a neural network model, significantly reducing the diagnostic time and ensuring timely treatment for HIE patients. Even in areas and hospitals lacking highly skilled psychiatrists, rapid and accurate HIE grading is possible.

[0032] As one possible implementation, step S104 (i.e., preprocessing the raw EEG data) may include:

[0033] (11) Remove the first type of data from the original EEG data, where all differential channels of the EEG data meet the preset conditions, to obtain the first EEG data.

[0034] The conditions for meeting the preset conditions may include: the voltage value being less than a preset first voltage threshold for a duration exceeding a first preset duration, and / or, the voltage amplitude exceeding a preset second voltage threshold within a duration exceeding a second preset duration. The first voltage threshold is less than the second voltage threshold.

[0035] (12) Sample the second EEG data from the first EEG data according to the preset sampling frequency, and perform notch filtering and low-pass filtering on the second EEG data in sequence to obtain the third EEG data.

[0036] (13) The ICA algorithm was used to remove noise signals from the third EEG data to obtain the fourth EEG data.

[0037] Noise signals may include electrocardiogram (ECG) signals, electrooculogram (EOG) signals, etc., and there are no restrictions on this.

[0038] Independent component analysis (ICA) is a statistical method that uses linear transformations to separate data or signals into a linear combination of statistically independent non-Gaussian signal sources. For acquired electroencephalogram (EEG) data, which contains many bioelectrical signals that are considered noise by EEG standards (such as electrocardiogram signals from heartbeats and electrooculogram signals from blinking), the ICA algorithm can be used to remove these noisy bioelectrical signals, thus achieving denoising of the EEG data.

[0039] (14) The EEG data of each differential channel in the fourth EEG data are standardized to obtain the fifth EEG data.

[0040] The specific standardization methods mentioned above can include range standardization, Z-score standardization, linear proportional standardization, log function standardization, arctangent function standardization, etc., without limitation. For example, Z-score standardization can be performed on the EEG data of each differential channel. The formula for Z-score standardization is... Where x is the EEG data of this differential channel, σ(x) is the mean of the EEG data for this differential channel, and σ(x) is the variance of the EEG data for this differential channel. Standardizing the EEG data facilitates subsequent data processing, thereby ensuring the reliability of the model's input and output.

[0041] (15) The EEG data of each differential channel in the fifth EEG data is sliced ​​according to the preset slice length and preset slice step size to obtain the sixth EEG data.

[0042] The sixth EEG data may include multiple sets of EEG data segments, each containing EEG data segments from various differential channels. The slice length and slice step size can be customized as needed and are not limited thereto.

[0043] As one possible implementation method, see Figure 3 As shown, the aforementioned neural network model may include a first convolutional module 31, a second convolutional module 32, and a third convolutional module 33, as well as a concatenation module 34, a fourth convolutional module 35, and a classification module 36 connected in sequence. The input to the concatenation module 34 is the output of the first convolutional module 31, the second convolutional module 32, and the third convolutional module 33, and the output of the concatenation module 34 is the input of the fourth convolutional module 35. Based on this, the aforementioned step S106 (i.e., predicting the preprocessed EEG data using a pre-trained neural network model to obtain the HIE grading result for newborns with HIE) may include:

[0044] (21) For each set of EEG data segments, the first convolution module 31, the second convolution module 32 and the third convolution module 33 are used to perform the first processing on the set of data segments to obtain the first feature vector, the second feature vector and the third feature vector corresponding to the set of data segments.

[0045] For example, see Figure 4As shown, the first convolution module 31 may include a first SE convolutional layer 311, a first convolutional layer 312, and a first residual convolutional layer 313 connected in sequence; the second convolution module 32 may include a second SE convolutional layer 321, a second convolutional layer 322, and a second residual convolutional layer 323 connected in sequence; and the third convolution module 33 may include a third SE convolutional layer 331, a third convolutional layer 332, and a third residual convolutional layer 333 connected in sequence. The input to the splicing module 34 is the first residual convolutional layer 313, the second residual convolutional layer 323, and the third residual convolutional layer 323. The output of layer 333; the kernel sizes of the first SE convolutional layer 311, the second SE convolutional layer 321 and the third SE convolutional layer 331 are different, and the kernel sizes of the first SE convolutional layer 311, the second SE convolutional layer 321 and the third SE convolutional layer 331 are all determined according to the preset sampling frequency in (12) above; the padding of the first convolutional layer 312, the second convolutional layer 322 and the third convolutional layer 332 are different; the structures of the first residual convolutional layer 313, the second residual convolutional layer 323 and the third residual convolutional layer 333 are the same. Based on this, for a set of EEG data segments input into the above neural network model, channel attention is calculated for the data segments through the first SE convolutional layer 311, the second SE convolutional layer 321, and the third SE convolutional layer 331, respectively, to obtain the first attention feature vector, the second attention feature vector, and the third attention feature vector corresponding to the data segments. The first attention feature vector is convolved by the first convolutional layer 312 to calculate the first intermediate feature vector, the second attention feature vector is convolved by the second convolutional layer 322 to calculate the second intermediate feature vector, and the third attention feature vector is convolved by the third convolutional layer 332 to calculate the third intermediate feature vector. The first intermediate feature vector is residually convolved by the first residual convolutional layer 313 to calculate the first feature vector, the second intermediate feature vector is residually convolved by the second residual convolutional layer 323 to calculate the second feature vector, and the third intermediate feature vector is residually convolved by the third residual convolutional layer 333 to calculate the third feature vector.

[0046] (22) The first feature vector, the second feature vector and the third feature vector corresponding to each data segment are spliced ​​together by the splicing module 34 to form the corresponding fourth feature vector.

[0047] The number of channels in the fourth eigenvector is the sum of the number of channels in the first, second, and third eigenvectors.

[0048] (23) The fourth feature vector corresponding to each data segment is processed by the fourth convolution module 35 to obtain the corresponding fifth feature vector.

[0049] Following the previous example, see Figure 4As shown, the fourth convolution module 35 may include a fourth SE convolutional layer 351, a fourth residual convolutional layer 352, a fifth SE convolutional layer 353, and a sixth SE convolutional layer 354 connected in sequence. The output of the concatenation module 34 is the input of the fourth SE convolutional layer 351. Based on this, the fourth SE convolutional layer 351 performs channel attention calculation on the fourth feature vector corresponding to each data segment to obtain the corresponding fourth attention feature vector. The fourth residual convolutional layer 352 performs residual convolution calculation on the fourth attention feature vector corresponding to each data segment to obtain the corresponding fourth vector. The fifth SE convolutional layer 353 and the sixth SE convolutional layer 354 sequentially perform channel attention calculation on the fourth vector corresponding to each data segment to obtain the corresponding fifth feature vector.

[0050] (24) Classify the fifth feature vector corresponding to each data segment through the classification module 36 to obtain the corresponding classification results.

[0051] Continuing from the previous example, the classification module 36 may include at least one fully connected layer and at least one activation layer connected in sequence, with the input of the first fully connected layer being the output of the fourth convolution module 35.

[0052] (25) Determine the HIE grading results based on the classification results corresponding to all group data segments.

[0053] Continuing the previous example, for each set of data segments input into the neural network model, the classification module 36 outputs a corresponding classification result. This classification result can represent one of the following: mild, moderate, severe, or no EEG activity. Furthermore, the classification result can include the magnitude of the classification probability. The final HIE grading result for a newborn with HIE can be obtained by statistically analyzing the classification results corresponding to all sets of data segments. For example, if the majority of the classification results for all sets of data segments for a newborn are classified as mild, then the final HIE grading result for that newborn can be determined as mild.

[0054] As one possible implementation, the channel attention calculation steps in (21) and (23) above may include: mapping the input data into an initial feature vector in a preset feature space through convolution; compressing the initial feature vector into an intermediate vector through global average pooling; wherein the number of channels in the intermediate vector is the same as the number of channels in the initial feature vector; activating the intermediate vector into a weight vector through fully connected and nonlinear activation; wherein the number of channels in the weight vector is the same as the number of channels in the intermediate vector, and the weight vector includes the weight values ​​of each channel; and multiplying the weight vector by the initial feature vector.

[0055] See Figure 5 As shown, the input data X∈R is processed by convolution. H′×W′×C′The initial feature vector U∈R mapped to the feature space H×W×C Center: U=F tr (X), F tr (·) represents a regular convolution operation; global average pooling is used to calculate the average value of the elements in the two-dimensional matrix for each channel: zc is the average value of the elements in the two-dimensional matrix of channel C, u c Let u be a two-dimensional matrix of channel C. c (i, j) represents the value of the element in the i-th row and j-th column of the two-dimensional matrix of channel C, where H and W are the number of rows and columns of the two-dimensional matrix of channel C, respectively, and F... sq (·) represents a squeeze operation. This calculation method assigns the calculated average value to the corresponding channels, thus obtaining a 1×1×C intermediate vector Z∈R. C The 1×1×C intermediate vector is excitationd into a 1×1×C weight vector through fully connected and nonlinear activation: S=F ex (Z, W) = σ(W2δ(W1Z)), S is the weight vector, which includes the weight values ​​of each channel, F ex (·, W) represents the activation operation, δ(·) is the ReLU nonlinear activation function, σ(·) is the Sigmoid nonlinear activation function, W1 is the parameter of one of the connected layers, and W2 is the parameter of the other fully connected layer; multiply the 1×1×C weight vector with the 1×1×C initial feature vector, that is, multiply the weight values ​​of each channel of S with the two-dimensional matrix of the corresponding channel of U respectively: By applying the weights of each channel to each channel of the initial feature vector, attention can be distributed across different feature spaces. Using this attention calculation method, each SE convolutional module can adaptively select and emphasize important features, thereby improving the neural network model's ability to discriminate features.

[0056] As one possible implementation, the residual convolution calculation steps in (21) and (23) above may include: performing a first convolution process, a second convolution process and a third convolution process on the input vector in sequence, and superimposing the intermediate feature vector with the result of the third convolution process.

[0057] As one possible implementation, the training steps of the above neural network model may include: establishing a training set and a test set, and using the Adam (Adaptive Moment Estimation) optimizer to train the neural network model based on the training set and test the performance of the neural network model based on the test set.

[0058] To facilitate understanding, the above-mentioned deep learning-based grading method for hypoxic-ischemic encephalopathy in newborns is described below using a specific application as an example. See [link to documentation]. Figure 6 As shown, the method can be performed according to the following steps:

[0059] Step 1, Data Collection.

[0060] According to Figure 2 The electrodes on the EEG cap are arranged in the manner shown, so that by wearing the EEG cap on the head of a newborn with HIE, EEG data of 9 differential channels can be collected, thereby obtaining EEG data of each differential channel.

[0061] Step 2, data preprocessing.

[0062] Data preprocessing can be accomplished using the MNE package in Python. The main components of data preprocessing include:

[0063] Step 2.1: Determine whether the EEG cap has fallen off (i.e., no voltage activity appears in the EEG of each channel) and / or whether there are parts of vigorous neonatal movement (i.e., large amplitude voltage activity unrelated to EEG appears in the EEG of each channel). The program judges no voltage activity as a voltage of less than 10 for a duration of more than 0.5 seconds. -9 μV, the program judges large amplitude voltage activities unrelated to EEG as voltages lasting more than 2 seconds with a maximum amplitude exceeding 300μV. If such activities are found, the part of the EEG data from all differential channels that shows the EEG cap falling off and / or the newborn's vigorous movement will be deleted.

[0064] Step 2.2: Normalize the sampling rate of the remaining data (i.e., all differential channel EEG data after processing in Step 2.1 above) to 500Hz and sample according to this sampling rate. Then, perform notch filtering at 50Hz, 100Hz and 150Hz respectively to eliminate the power frequency interference of 50Hz AC as much as possible, and then perform low-pass filtering at 0.1-40Hz to eliminate high-frequency noise interference.

[0065] Step 2.3: Based on the EEG data of each differential channel (i.e., the EEG data of each differential channel after processing in Step 2.2 above), draw the power spectrum of the corresponding channel for doctors to refer to.

[0066] Step 2.4: Use the ICA algorithm to remove ECG and electrooculogram signals from the EEG data (i.e., the EEG data of all differential channels after processing in Step 2.2 above).

[0067] Step 2.5: Perform Z-Score standardization on the EEG data of each differential channel (i.e., the EEG data of each differential channel after processing in Step 2.4 above).

[0068] Step 2.6: Slice the EEG data of each differential channel (i.e., the EEG data of each differential channel after processing in Step 2.5 above) into slices with a slice length of 30 seconds and a slice step size of 3 seconds.

[0069] At this point, the data preprocessing is complete, and the multiple sets of 30-second EEG data obtained after slicing (each set contains 30-second EEG data of all differential channels) can be input into the model for training or prediction.

[0070] Step 3, model classification.

[0071] The model can be built using the PyTorch package in Python, and it employs a convolutional neural network with attention assignment and residuals. The model structure is as follows: Figure 7 As shown. Each trainable layer or trainable module (i.e. Figure 7 Each trainable block has its own parameters, which may include kernel size, type, number of output channels, stride, padding, etc.

[0072] See Figure 7 As shown, since smaller filters capture temporal information better, while larger filters capture frequency information better, three different SE convolutional layers are used in the first layer of the model. Figure 7 The blocks in the image are labeled “fs, SEconv, 16, [fs / 8]”, “fs / 2, SEconv, 16, [fs / 8]”, and “fs / 4, SEconv, 16, [fs / 8]”, respectively. fs represents the sampling rate of the remaining data in step 2.2. The text within each block, from left to right, represents the kernel size, type, number of output channels, and stride. Time-invariant features are extracted from the 30-second neonatal EEG data obtained after data preprocessing. The SE convolutional layer is an attention-assigned convolutional layer. Unlike ordinary convolutions or residual convolutions, the SE convolutional layer adaptively calibrates the feature importance of each channel based on the dependencies between channels. The working principle of the SE convolutional layer is as described above. Figure 5 The principle of attention calculation is the same as that shown, so it will not be elaborated upon. The kernel sizes of the three SE convolutional layers in the first layer of the model are related to the sampling rate (i.e., fs) of the EEG data, which are fs, fs / 2, and fs / 4, respectively; correspondingly, the stride of the three SE convolutional layers in the first layer of the model is also related to fs, which is fs / 8. This design can extract information in both the time domain and the frequency domain simultaneously through the three SE convolutional layers in the first layer of the model, rather than simply assuming that a larger kernel means extracting information in the frequency domain and a smaller kernel means extracting information in the time domain.

[0073] See Figure 7As shown, since different convolutional kernels output data with different dimensions after processing the same data, three convolutional layers are used in the second layer of the model. Figure 7 The blocks in the middle are labeled “10, Conv, 32, 1, 3”, “10, Conv, 32, 1, 1”, and “10, Conv, 32, 1, 0”, respectively. The text in each block, from left to right, represents the kernel size, type, number of output channels, stride, and padding. The three output vectors of the first layer are normalized to the same size. The kernel size and stride of these three normalized convolutional layers are the same, but the difference lies in the padding operation used.

[0074] See Figure 7 As shown, three convolutional blocks with residuals (i.e., residual convolutional blocks) are used in the third layer of the model. Figure 7 The diagram shows three squares labeled "5, ResBlock, 32, 1, 2". The text within each square, from left to right, represents the kernel size, kernel type, number of output channels, stride, and padding, respectively. Residual convolutions are then performed on the three output vectors of the second layer. (See also...) Figure 8 As shown, each residual convolutional block consists of three sequentially connected ordinary convolutional layers (i.e., ... Figure 8 The three squares contain text, and the text in each square, from left to right, represents the kernel size, type, number of output channels, stride, and padding. Each residual convolutional block also includes a structure, and padding is used to keep the feature dimensions consistent. Finally, the convolutional blocks are added back to the original features to superimpose the short-circuit paths.

[0075] See Figure 9 As shown, any convolutional layer consists of a series of interconnected ordinary one-dimensional convolutional layers (i.e., ... Figure 9 The block containing the words "K, (SE)Conv, N, S, P" (from left to right, these represent the kernel size, type, number of output channels, stride, and padding), and a batch normalization layer (i.e., ... Figure 9 The block containing the words "1D BatchNorm" and a non-linear activation function layer (i.e. Figure 9 The squares containing the word "RELU" are composed of the non-linear activation function RELU.

[0076] See Figure 7 As shown, after processing a set of 30 seconds of EEG data input into the model through the first three layers, the feature vectors output by the three residual convolutional blocks of the third layer of the model (i.e., Figure 7 The three feature vectors H1, H2, and H3, each with 32 channels, are concatenated at the channel level to obtain a new feature vector with 96 channels. Then, an SE convolutional layer (...) is used... Figure 7The middle block contains the text "5, SEconv, 32, 5, 0". From left to right, the text within this block represents the kernel size, kernel type, number of output channels, stride, and padding, respectively. A convolution is performed on the new feature vector, and a residual convolution block is used. Figure 7 The middle section contains a square labeled "3, ResBlock, 32, 1, 1". The text within this square, from left to right, represents the kernel size, kernel type, number of output channels, stride, and padding, respectively. A residual convolution is then performed on the convolved feature vector, followed by two SE convolutional layers. Figure 7 The blocks in the middle are labeled "4, SEconv, 16, 4, 0" and "2, SEconv, 16, 2, 0". The text within each block, from left to right, represents the kernel size, kernel type, number of output channels, stride, and padding, respectively. These are used to convolve the feature vectors calculated from the residual convolution. Finally, the convolved feature vectors are input into a classification block (i.e.,...). Figure 7 The block labeled "Fully Connect and 4SoftMax" contains one fully connected layer and four non-linear activation function layers. Each non-linear activation function layer uses the SoftMax non-linear activation function for classification, and the classification block outputs the HIE classification to complete the HIE grading of this set of 30-second EEG data. The convolutional feature vectors are then used to extract time-invariant features from the 30-second neonatal EEG data obtained after data preprocessing. In the above operation, the concatenated dimensionality-reduced convolution and residual convolution can reduce the amount of feature information, thereby improving the efficiency and accuracy of classification calculation.

[0077] The model used in step 3 above needs to be trained with expert-labeled data. The training of the model mainly includes the following processes:

[0078] Raw EEG data from real medical cases were obtained from hospitals and categorized and labeled by senior experts in relevant departments. Among the obtained cases: 6 were normal, 35 had mild symptoms, 26 had moderate symptoms, 25 had severe symptoms, and 13 had no EEG activity, totaling 105 cases.

[0079] Since the data used are all from real cases, the data are consistent with real conditions and the labels are highly reliable. Therefore, the trained model is expected to have strong generalization ability. The real cases obtained have five different scales (i.e., classifications): normal, mild, moderate, severe, and no EEG response (i.e., no EEG activity). The raw EEG data for each of the five scales can be directly sliced ​​into 30-second segments. All data segments from each scale are used as training samples to establish the training set. Subsequently, all data segments from two randomly selected individuals within each category in the training set are used as validation samples to establish the validation set for subsequent model performance verification. Model training includes pre-training and fine-tuning. Pre-training data can be obtained by oversampling the training set (i.e., adjusting the raw EEG data of the four scales in the training set to have similar data volumes). For example, if a person's raw EEG data does not meet the required data volume, a portion of that person's raw EEG data can be copied multiple times, and the copied EEG data can be combined with the original EEG data to form new EEG data to meet the required data volume. Fine-tuning can be performed directly using the training set. Table 1 shows the distribution of the number of each category in the test set and the training set.

[0080] Table 1 shows an example of the distribution of the number of classes in the test and training sets.

[0081] normal Mild moderate Severe No EEG response total test set 2012 2574 2346 2219 2432 11583 training set 5205 68797 36486 53151 21039 184678 total 7217 71371 38832 55370 23471 196261

[0082] The training batch size was set to 256. Both pre-training and fine-tuning were performed using the Adam optimizer. During pre-training, the learning rate, β1, and β2 of the Adam optimizer were set to 10. -4 The learning rate, β1, and β2 of the Adam optimizer were set to 10, 0.9, and 0.999 respectively during fine-tuning training. -6 The training epochs for both pre-training and fine-tuning were set to 50, 0.9, and 0.999. For batch normalization of convolutional layers, to ensure numerical stability, the ε of the batch normalization layer in each convolutional layer was set to 10. -5 When calculating the loss function during model training, the cross-entropy function can be used. Table 2 shows the model's accuracy, recall, and F1-score on the training set, and Table 3 shows the model's accuracy, recall, and F1-score on the test set.

[0083] Table 2 shows examples of the model's accuracy, recall, and F1-Score on the training set.

[0084] accuracy Recall rate F1-Score normal 84.115 82.326 83.211 Mild 93.153 91.474 92.306 moderate 90.641 92.851 91.733 Severe 91.613 88.498 90.029 No EEG response 92.188 87.923 90.005 mean 90.342 88.614 89.457

[0085] Table 3 shows examples of the model's accuracy, recall, and F1-Score on the test set.

[0086] accuracy Recall rate F1-Score normal 75.124 79.412 77.209 Mild 81.264 80.883 81.073 moderate 79.412 80.349 79.878 Severe 84.246 75.421 79.590 No EEG response 82.679 78.127 80.339 mean 80.545 78.838 79.618

[0087] Among the deep learning-based grading methods for hypoxic-ischemic encephalopathy (HIE) in newborns described above, the deep learning model provides a HIE grading method with an accuracy no less than that of physician judgment, and a diagnostic speed far exceeding that of visual assessment. This method has a wide range of applications, few limitations, and a fully automated diagnostic process. Physicians do not need to monitor the infant's EEG data throughout the process; they only need to make a diagnosis based on the model's output. This method only requires basic and simple preprocessing of the newborn's EEG data, such as filtering, before it can be fed into the model for prediction, making the data processing process simple.

[0088] In addition, machine learning algorithms can be used to grade HIE in newborns. However, it is necessary to first extract multiple features from the raw EEG data (such as high-amplitude EEG abundance, low-amplitude EEG abundance, burst interval, etc.), and then input the extracted features into the machine learning model for processing.

[0089] The calculation methods for high-amplitude and low-amplitude EEG abundance mainly include: First, taking the absolute value of all differential channel EEG data in the original EEG data to remove negative numbers, then performing a Hilbert transform to obtain the envelope information of the EEG signal. Differentiating the envelope information of each differential channel with respect to time, finding all maxima, and considering these maxima as the highest amplitude points. Then, finding the number of amplitude maxima exceeding 100µV and less than 10µV respectively, and dividing by the total number of amplitude maxima to obtain the corresponding percentages, thereby calculating the high-amplitude and low-amplitude EEG abundance. Alternatively, the high-amplitude and low-amplitude EEG abundance of each differential channel can be calculated separately, and then the average value can be calculated to obtain the high-amplitude and low-amplitude EEG abundance characteristics of the original EEG data.

[0090] The calculation of burst intervals mainly involves: obtaining all amplitude points in the EEG data of all differential channels by taking amplitude values, and defining the time interval between two amplitudes greater than 100uV as the burst interval. The burst interval can be calculated separately for each differential channel, and then averaged to obtain the burst interval characteristics of the raw EEG data.

[0091] After obtaining the high-amplitude EEG abundance, low-amplitude EEG abundance, and burst interval of the nine differential channels, a 27 (3×9) dimensional feature vector can be constructed. This feature vector is then input into the K-Nearest Neighbor (KNN) model and the random forest model for calculation, thereby achieving HIE grading through the machine learning model.

[0092] Based on the aforementioned deep learning-based grading method for hypoxic-ischemic encephalopathy (HIE) in newborns, this invention also provides a deep learning-based grading device for HIE in newborns. (See [link to relevant documentation]). Figure 10 As shown, the device may include the following modules:

[0093] The acquisition module 1002 is used to acquire raw EEG data of HIE newborns; wherein the raw EEG data includes EEG data of multiple differential channels.

[0094] The preprocessing module 1004 is used to preprocess the raw EEG data; wherein the preprocessing includes at least one of the following: screening, sampling, standardization, and slicing.

[0095] The prediction module 1006 is used to predict the preprocessed EEG data using a pre-trained neural network model to obtain the HIE grading result of the newborn with HIE.

[0096] This invention provides a deep learning-based grading device for hypoxic-ischemic encephalopathy (HIE) in newborns. First, it acquires raw EEG data (including EEG data from multiple differential channels) from HIE newborns. Then, it preprocesses the raw EEG data. Finally, a pre-trained neural network model predicts the HIE grading result of the newborn. Using this technology, only the raw EEG data of the HIE newborn is needed for automatic HIE grading via a neural network model, significantly reducing the time required for diagnosis and ensuring timely treatment for HIE patients. Even in areas and hospitals lacking highly skilled psychiatrists, rapid and accurate HIE grading can be achieved.

[0097] The preprocessing module 1004 described above can also be used to: remove a first type of data from the original EEG data where all differential channels of EEG data meet preset conditions, to obtain first EEG data; wherein, meeting the preset conditions includes: the duration for which the voltage value is less than a preset first voltage threshold exceeds a first preset duration, and / or, the voltage amplitude exceeds a preset second voltage threshold within a duration exceeding a second preset duration; the first voltage threshold is less than the second voltage threshold; sampling second EEG data from the first EEG data according to a preset sampling frequency, and sequentially performing notch filtering and... Low-pass filtering is used to obtain third EEG data; the ICA algorithm is used to remove noise signals from the third EEG data to obtain fourth EEG data; wherein, the noise signals include ECG signals and / or ECG signals; the EEG data of each differential channel in the fourth EEG data is standardized to obtain fifth EEG data; the EEG data of each differential channel in the fifth EEG data is sliced ​​according to a preset slice length and a preset slice step size to obtain sixth EEG data; wherein, the sixth EEG data includes multiple sets of EEG data segments, and each set of EEG data segments contains EEG data segments of each differential channel.

[0098] The aforementioned neural network model may include a first convolutional module, a second convolutional module, and a third convolutional module, as well as a concatenation module, a fourth convolutional module, and a classification module connected in sequence. The input of the concatenation module is the output of the first convolutional module, the second convolutional module, and the third convolutional module, and the output of the concatenation module is the input of the fourth convolutional module. Based on this, the aforementioned prediction module 1006 can also be used to: for each group of EEG data segments, perform a first processing on the group of data segments through the first convolutional module, the second convolutional module, and the third convolutional module respectively, to obtain the first feature vector, the second feature vector, and the third feature vector corresponding to the group of data segments. The first, second, and third feature vectors corresponding to each data segment are concatenated into a fourth feature vector by the splicing module. The number of channels in the fourth feature vector is the sum of the number of channels in the first, second, and third feature vectors. The fourth convolution module performs a second processing on the fourth feature vector corresponding to each data segment to obtain a fifth feature vector. The classification module classifies the fifth feature vector corresponding to each data segment to obtain a classification result. The HIE grading result is determined based on the classification results for all data segments.

[0099] The first convolutional module mentioned above may include a first SE convolutional layer, a first convolutional layer, and a first residual convolutional layer connected in sequence; the second convolutional module includes a second SE convolutional layer, a second convolutional layer, and a second residual convolutional layer connected in sequence; the third convolutional module includes a third SE convolutional layer, a third convolutional layer, and a third residual convolutional layer connected in sequence; the input of the splicing module is the output of the first residual convolutional layer, the second residual convolutional layer, and the third residual convolutional layer; the kernel sizes of the first SE convolutional layer, the second SE convolutional layer, and the third SE convolutional layer are different, and the kernel sizes of the first SE convolutional layer, the second SE convolutional layer, and the third SE convolutional layer are all determined according to the preset sampling frequency; the padding of the first convolutional layer, the second convolutional layer, and the third convolutional layer is different; the structures of the first residual convolutional layer, the second residual convolutional layer, and the third residual convolutional layer are the same; Therefore, the prediction module 1006 can also be used to: perform channel attention calculations on the group of data segments through the first SE convolutional layer, the second SE convolutional layer, and the third SE convolutional layer respectively, to obtain the first attention feature vector, the second attention feature vector, and the third attention feature vector corresponding to the group of data segments; calculate the first attention feature vector into a first intermediate feature vector through convolution of the first convolutional layer, calculate the second attention feature vector into a second intermediate feature vector through convolution of the second convolutional layer, calculate the third attention feature vector into a third intermediate feature vector through convolution of the third convolutional layer; calculate the first intermediate feature vector into a first feature vector through residual convolution of the first residual convolutional layer, calculate the second intermediate feature vector into a second feature vector through residual convolution of the second residual convolutional layer, and calculate the third intermediate feature vector into a third feature vector through residual convolution of the third residual convolutional layer.

[0100] The aforementioned fourth convolutional module may include a fourth SE convolutional layer, a fourth residual convolutional layer, a fifth SE convolutional layer, and a sixth SE convolutional layer connected in sequence, with the output of the concatenation module serving as the input of the fourth SE convolutional layer. Based on this, the aforementioned prediction module 1006 may also be used to: perform channel attention calculation on the fourth feature vector corresponding to each data segment through the fourth SE convolutional layer to obtain the corresponding fourth attention feature vector; perform residual convolution calculation on the fourth attention feature vector corresponding to each data segment through the fourth residual convolutional layer to obtain the corresponding fourth vector; and perform channel attention calculation on the fourth vector corresponding to each data segment sequentially through the fifth SE convolutional layer and the sixth SE convolutional layer to obtain the corresponding fifth feature vector.

[0101] The classification module described above may include at least one fully connected layer and at least one activation layer connected in sequence, with the input of the first fully connected layer being the output of the fourth convolutional module.

[0102] See Figure 10 As shown, the device may further include:

[0103] The training module 1008 is used to establish a training set and a test set, and to use the Adam optimizer to train the neural network model based on the training set and to test the performance of the neural network model based on the test set.

[0104] The deep learning-based neonatal hypoxic-ischemic encephalopathy grading device provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned deep learning-based neonatal hypoxic-ischemic encephalopathy grading method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0105] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0106] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0108] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep learning-based EEG data processing method, characterized in that, The method includes: Acquire raw EEG data of the subject; wherein, the raw EEG data includes EEG data from multiple differential channels; First EEG data is obtained by removing all differential channel EEG data from the original EEG data, specifically the first type of data that meets preset conditions; wherein, meeting the preset conditions includes: the duration for which the voltage value is less than a preset first voltage threshold exceeds a first preset duration, and / or, the voltage amplitude exceeds a preset second voltage threshold within a duration exceeding a second preset duration; the first voltage threshold is less than the second voltage threshold. The second EEG data is obtained by sampling from the first EEG data according to a preset sampling frequency, and the second EEG data is then subjected to notch filtering and low-pass filtering in sequence to obtain the third EEG data. The noise signal in the third EEG data is removed using the ICA algorithm to obtain the fourth EEG data; wherein, the noise signal includes electrocardiogram signal and / or electrooculogram signal; The EEG data of each differential channel in the fourth EEG data are standardized to obtain the fifth EEG data; The EEG data of each differential channel in the fifth EEG data is sliced ​​according to the preset slice length and preset slice step size to obtain the sixth EEG data; wherein, the sixth EEG data includes multiple sets of EEG data segments, and each set of EEG data segments contains EEG data segments of each differential channel. The sixth EEG data is predicted by a pre-trained neural network model to obtain the final classification result corresponding to the EEG activity state of the object; The neural network model includes a first convolutional module, a second convolutional module, and a third convolutional module, as well as a concatenation module, a fourth convolutional module, and a classification module connected in sequence. The input of the concatenation module is the output of the first, second, and third convolutional modules, and the output of the concatenation module is the input of the fourth convolutional module. The step of predicting the sixth EEG data using the pre-trained neural network model to obtain the final classification result corresponding to the EEG activity state of the object includes: for each group of EEG data segments, performing a first processing on each group of data segments through the first, second, and third convolutional modules respectively to obtain a first feature vector, a second feature vector, and a third feature vector corresponding to each group of data segments. The splicing module concatenates the first, second, and third feature vectors corresponding to each data segment into a corresponding fourth feature vector; wherein the number of channels of the fourth feature vector is the sum of the number of channels of the first, second, and third feature vectors; the fourth convolution module performs a second processing on the fourth feature vector corresponding to each data segment to obtain a corresponding fifth feature vector; the classification module classifies the fifth feature vector corresponding to each data segment to obtain a corresponding classification result; wherein the classification result represents one of normal, mild, moderate, severe, and no EEG activity; the final classification result is determined by statistically analyzing the classification results corresponding to all data segments of the object. The first convolutional module includes a first SE convolutional layer, a first convolutional layer, and a first residual convolutional layer connected in sequence. The second convolutional module includes a second SE convolutional layer, a second convolutional layer, and a second residual convolutional layer connected in sequence. The third convolutional module includes a third SE convolutional layer, a third convolutional layer, and a third residual convolutional layer connected in sequence. The input of the splicing module is the output of the first residual convolutional layer, the second residual convolutional layer, and the third residual convolutional layer. The convolutional kernels of the first SE convolutional layer, the second SE convolutional layer, and the third SE convolutional layer are each large. The differences are as follows: the kernel size of the first SE convolutional layer is fs, the kernel size of the second SE convolutional layer is fs / 2, and the kernel size of the third SE convolutional layer is fs / 4, where fs is the preset sampling frequency; the padding of the first, second, and third convolutional layers is different to normalize the vectors output by each layer to the same size; the structures of the first, second, and third residual convolutional layers are... The steps are the same; the first processing is performed on each group of data segments through the first convolutional module, the second convolutional module, and the third convolutional module respectively to obtain the first feature vector, the second feature vector, and the third feature vector corresponding to each group of data segments. These steps include: performing channel attention calculation on each group of data segments through the first SE convolutional layer, the second SE convolutional layer, and the third SE convolutional layer respectively to obtain the first attention feature vector, the second attention feature vector, and the third attention feature vector corresponding to each group of data segments; convolving the first attention feature vector into a first intermediate feature vector through the first convolutional layer; convolving the second attention feature vector into a second intermediate feature vector through the second convolutional layer; convolving the third attention feature vector into a third intermediate feature vector through the third convolutional layer; and residually convolving the first intermediate feature vector into a first feature vector through the first residual convolutional layer; residually convolving the second intermediate feature vector into a second feature vector through the second residual convolutional layer; and residually convolving the third intermediate feature vector into a third feature vector through the third residual convolutional layer. The fourth convolutional module includes a fourth SE convolutional layer, a fourth residual convolutional layer, a fifth SE convolutional layer, and a sixth SE convolutional layer connected in sequence. The output of the concatenation module is the input of the fourth SE convolutional layer. The step of performing a second processing on the fourth feature vector corresponding to each data segment through the fourth convolutional module to obtain the corresponding fifth feature vector includes: performing channel attention calculation on the fourth feature vector corresponding to each data segment through the fourth SE convolutional layer to obtain the corresponding fourth attention feature vector; performing residual convolution calculation on the fourth attention feature vector corresponding to each data segment through the fourth residual convolutional layer to obtain the corresponding fourth vector; and performing channel attention calculation on the fourth vector corresponding to each data segment through the fifth SE convolutional layer and the sixth SE convolutional layer in sequence to obtain the corresponding fifth feature vector. The steps for channel attention calculation include: mapping the input data into an initial feature vector in a preset feature space through convolution; compressing the initial feature vector into an intermediate vector through global average pooling; wherein the number of channels in the intermediate vector is the same as the number of channels in the initial feature vector; activating the intermediate vector into a weight vector through fully connected and non-linear activation; wherein the number of channels in the weight vector is the same as the number of channels in the intermediate vector, and the weight vector includes the weight values ​​of each channel; and multiplying the weight vector by the initial feature vector. The steps of residual convolution calculation include: performing first convolution, second convolution and third convolution on the input vector in sequence, and superimposing the intermediate feature vector with the result of the third convolution. The classification module includes a fully connected layer and four non-linear activation function layers connected in sequence. The input of the fully connected layer is the output of the fourth convolutional module.

2. The method according to claim 1, characterized in that, The training steps of the neural network model include: Establish training and testing sets, and use the Adam optimizer to train the neural network model based on the training set and test the performance of the neural network model based on the testing set.

3. A deep learning-based neonatal electroencephalogram (EEG) data processing device, characterized in that, The apparatus is used to perform the method according to claim 1 or 2, the apparatus comprising: An acquisition module is used to acquire the raw EEG data of the object; wherein, the raw EEG data includes EEG data from multiple differential channels; The preprocessing module is used to: remove first-class data from the original EEG data where all differential channels of EEG data meet preset conditions, thereby obtaining first EEG data; wherein, meeting the preset conditions includes: the duration for which the voltage value is less than a preset first voltage threshold exceeds a first preset duration, and / or, the voltage amplitude exceeds a preset second voltage threshold within a duration exceeding a second preset duration; the first voltage threshold is less than the second voltage threshold; sample second EEG data from the first EEG data according to a preset sampling frequency, and sequentially perform notch filtering and low-pass filtering on the second EEG data. The third EEG data is obtained; noise signals in the third EEG data are removed using the ICA algorithm to obtain the fourth EEG data; wherein, the noise signals include ECG signals and / or electrooculogram signals; the EEG data of each differential channel in the fourth EEG data are standardized to obtain the fifth EEG data; the EEG data of each differential channel in the fifth EEG data are sliced ​​according to a preset slice length and a preset slice step size to obtain the sixth EEG data; wherein, the sixth EEG data includes multiple sets of EEG data segments, and each set of EEG data segments contains EEG data segments of each differential channel; The prediction module is used to predict the sixth EEG data using a pre-trained neural network model to obtain the final classification result corresponding to the EEG activity state of the object. The neural network model includes a first convolutional module, a second convolutional module, and a third convolutional module, as well as a concatenation module, a fourth convolutional module, and a classification module connected in sequence. The input of the concatenation module is the output of the first, second, and third convolutional modules, and the output of the concatenation module is the input of the fourth convolutional module. The prediction module is further configured to: for each group of EEG data segments, perform a first processing on each group of data segments through the first, second, and third convolutional modules respectively to obtain a first feature vector, a second feature vector, and a third feature vector corresponding to each group of data segments; and use the concatenation module to combine the first feature vectors corresponding to each group of data segments. The first feature vector, the second feature vector, and the third feature vector are concatenated to form a corresponding fourth feature vector; wherein the number of channels of the fourth feature vector is the sum of the number of channels of the first feature vector, the second feature vector, and the third feature vector; the fourth feature vector corresponding to each data segment is processed by the fourth convolution module to obtain the corresponding fifth feature vector; the fifth feature vector corresponding to each data segment is classified by the classification module to obtain the corresponding classification result; wherein the classification result is a classification representing one of normal, mild, moderate, severe, and no EEG activity; the final classification result is determined by statistically analyzing the classification results corresponding to all data segments of the object. The first convolutional module includes a first SE convolutional layer, a first convolutional layer, and a first residual convolutional layer connected in sequence. The second convolutional module includes a second SE convolutional layer, a second convolutional layer, and a second residual convolutional layer connected in sequence. The third convolutional module includes a third SE convolutional layer, a third convolutional layer, and a third residual convolutional layer connected in sequence. The input of the splicing module is the output of the first residual convolutional layer, the second residual convolutional layer, and the third residual convolutional layer. The first SE convolutional layer, the second SE convolutional layer, and the third SE convolutional layer each... The convolutional kernels have different sizes: the first SE convolutional layer uses a kernel size of fs, the second SE convolutional layer uses a kernel size of fs / 2, and the third SE convolutional layer uses a kernel size of fs / 4, where fs is the preset sampling frequency. The first, second, and third convolutional layers each have different padding to normalize the output vectors of each layer to the same size. The first residual convolutional layer, the second residual convolutional layer, and the third SE convolutional layer each have different padding. The third residual convolutional layer has the same structure; the prediction module is further configured to: perform channel attention calculation on each data segment through the first SE convolutional layer, the second SE convolutional layer, and the third SE convolutional layer respectively, to obtain the first attention feature vector, the second attention feature vector, and the third attention feature vector corresponding to each data segment; convolve the first attention feature vector through the first convolutional layer to calculate the first intermediate feature vector, convolve the second attention feature vector through the second convolutional layer to calculate the second intermediate feature vector, and convolve the third attention feature vector through the third convolutional layer to calculate the third intermediate feature vector; residually convolve the first intermediate feature vector through the first residual convolutional layer to calculate the first feature vector, residually convolve the second intermediate feature vector through the second residual convolutional layer to calculate the second feature vector, and residually convolve the third intermediate feature vector through the third residual convolutional layer to calculate the third feature vector; the classification module includes a fully connected layer and four non-linear activation function layers connected in sequence, and the input of the fully connected layer is the output of the fourth convolutional module.

Images