A method for constructing a convolutional neural network with time domain information enhancement in convolutional output

Abstract

The application belongs to the field of brain-computer interface, and particularly relates to a convolutional neural network construction method for enhancing time domain information in convolutional output, which is used for classification tasks or regression tasks in brain-computer interface, and comprises the following steps: improving and training the structure of a convolutional neural network to be enhanced, wherein the improvement mode is as follows: according to the structure of a convolutional layer to be improved, the size of a feature representation output by the convolutional layer is calculated, an average pooling layer with the same output size and feature representation size is constructed, the average pooling layer is used for accompanying convolution kernel sliding point multiplication process of the convolutional layer to be improved, input data is average-pooled to generate an average pooling representation; a trigonometric function encoder is constructed to perform time sequence coding on each element in the average pooling representation to obtain a new average pooling representation; a summation unit is constructed to sum corresponding elements of the new average pooling representation and the feature representation to obtain a feature representation with enhanced time domain information, and the feature representation is input into a next layer structure. The application can enhance time domain information without expanding the size of model parameters.

Description

Technical Field

[0001] This invention belongs to the field of brain-computer interfaces, and more specifically, relates to a method for constructing a convolutional neural network that enhances temporal information in the convolutional output. Background Technology

[0002] Brain-computer interface (BCI) is a system that provides a direct interaction channel between the brain and external devices. Brain-computer interface systems based on electroencephalography (EEG) involve a person wearing an EEG cap equipped with electrodes to collect EEG signals from the brain. These EEG signals are then analyzed and decoded to determine the brain's state or thoughts, enabling disease detection or, further, control of external devices through EEG signals.

[0003] For scalp EEG-based classification tasks in brain-computer interfaces, common AI automation algorithms can be categorized into traditional machine learning algorithms and deep learning algorithms. Traditional machine learning algorithms, after preprocessing the data, primarily use feature extraction and selection techniques, based on expert experience and prior knowledge, to transform the raw scalp EEG signals into a series of time-domain, frequency-domain, time-frequency-domain, and nonlinear features. Then, through normalization and feature selection of the generated features, the original samples are transformed into the final selected feature vectors, thereby training the classifier to adapt to the classification task. The drawback of this approach is that the process of transforming the raw signal into features is time-consuming, and due to its reliance on expert knowledge, the generated features are determined based on previously studied effective representations, neglecting features and patterns contained in the raw signal that have not yet been fully understood. Deep learning (DL) methods, which mainly rely on neural networks, integrate the feature extraction and classification modules into the neural network structure, independent of expert knowledge, enabling the construction of end-to-end systems that can be trained and responded to in real time. However, since deep learning models are data-driven, their training process becomes increasingly dependent on large amounts of data as the network size increases. Otherwise, a small amount of data can easily lead to overfitting and impair the model's generalization performance. However, acquiring scalp EEG data with category labels is extremely costly and exhibits significant individual variability. Therefore, for brain-computer interface tasks with limited sample sizes, it is necessary to design more sophisticated network structures to enable the model to recognize more patterns more efficiently while controlling the parameter scale, thereby improving model performance and generalization ability, and achieving lightweight neural networks.

[0004] Convolutional Neural Networks (CNNs), as classic network structures, are widely used in brain-computer interface (BCI) tasks. For example, EEGNet is a CNN-based deep learning model specifically designed for scalp EEG signals and is applicable to various BCI paradigms. However, due to the unbiasedness of the pooling layers in CNNs and the typically limited receptive field of the convolutional kernels to control parameters, the information received by lower-layer neurons from upper-layer data is confined to a small spatiotemporal range. This makes CNN models insensitive to the temporal characteristics of time-series data. Therefore, designing convolutional layer structures to enhance their learning of temporal information while controlling parameters is a pressing issue. Summary of the Invention

[0005] To address the shortcomings and improvement needs of existing technologies, this invention provides a method for constructing convolutional neural networks that enhances temporal information in convolutional output. The purpose is to design convolutional layer structures to enhance their learning of temporal information while controlling parameters.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for constructing a convolutional neural network with temporal information enhancement in the convolutional output is provided for classification or regression tasks in brain-computer interfaces, comprising:

[0007] The structure of the convolutional neural network to be enhanced is improved; a training sample set is sampled, and the improved convolutional neural network is trained to obtain a convolutional neural network with enhanced temporal information in the convolution output;

[0008] The improvement method is as follows: based on the structure of the convolutional layer to be improved in the convolutional neural network to be enhanced, the size of its output feature representation is calculated to construct an average pooling layer with the same output size as the feature representation size; wherein, the average pooling layer is used to perform average pooling on the input data along with the sliding dot product process of the convolutional kernel in the convolutional layer to be enhanced, and finally generate an average pooling representation; a trigonometric function encoder is constructed to perform temporal encoding on each element in the average pooling representation to obtain a new average pooling representation; a summation unit is constructed to sum the new average pooling representation with the corresponding elements of the feature representation to obtain a temporally enhanced feature representation, and the temporally enhanced feature representation is input into the next layer structure of the convolutional layer to be enhanced in the convolutional neural network to be enhanced.

[0009] Furthermore, the trigonometric functions in the trigonometric function encoder are represented as follows:

[0010]

[0011] In the formula, SE(t) represents a trigonometric function, t represents the temporal position of the output element of the convolutional layer to be temporally encoded, and ω represents the encoding period of the trigonometric function SE(t).

[0012] The trigonometric encoder performs temporal encoding of each element by multiplying the trigonometric function SE(t) with each element in the average pooling representation, thereby obtaining time position information.

[0013] Furthermore, the encoding period ω is determined as follows:

[0014] Based on the scalp EEG rhythm, select the key frequency f. c ; and based on the sampling frequency f of the scalp EEG data s The encoding period ω is generated as follows:

[0015] Furthermore, the encoding period ω is determined as follows:

[0016] Based on scalp EEG rhythms, high-pass and low-pass frequencies of delta, theta, alpha, beta, gamma, and μ waves were selected as the key frequency set f. c Based on the sampling frequency f of scalp EEG data s Determine the candidate set for the coding period

[0017] Generate a set of convolutional neural network models with temporal information enhancement in the candidate convolutional output according to each candidate ω in the candidate set of the encoding period, and train all candidate models in parallel;

[0018] The model with the minimum loss on the validation set is selected as the convolutional neural network model for temporal information enhancement in the final convolutional output, and the ω used in this model is the final determined encoding period ω.

[0019] Furthermore, during the timing encoding, the trigonometric function SE(t) is multiplied by a time scaling factor b to generate scaled-down time position information, the value of which is obtained through the training.

[0020] Furthermore, the convolutional layer to be improved is the first convolutional layer of the convolutional neural network to be enhanced.

[0021] This invention also provides a method for enhancing temporal information in the convolutional output of a convolutional neural network, used for classification or regression tasks in brain-computer interfaces. The method employs a convolutional neural network constructed using the method described above for enhancing temporal information in the convolutional output, to perform brain-computer interface classification or regression tasks, thereby achieving enhancement of temporal information in the convolutional output of the convolutional neural network.

[0022] The present invention also provides an electronic device, comprising: a processor and a memory storing processor-executable instructions, and a data transceiver for data transmission and / or reception, wherein when the instructions are executed by the processor, a method for constructing a convolutional neural network with temporal information enhancement in the convolutional output as described above and / or a method for enhancing temporal information in the convolutional output of a convolutional neural network as described above are executed.

[0023] The present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed by a processor, it controls the device where the storage medium is located to perform a method for constructing a convolutional neural network with temporal information enhancement in the convolutional output as described above and / or a method for enhancing temporal information in the convolutional output of a convolutional neural network as described above.

[0024] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0025] (1) The present invention provides a method that introduces an average pooling layer to perform average pooling on the input data of the convolutional layer to be improved, obtaining an average pooling representation. Each element in the average pooling representation is temporally encoded using a trigonometric encoder. Then, each temporally encoded element is added to each element in the feature representation output by the convolutional layer, thereby embedding additional temporal information into the convolution calculation result and inputting it into the next layer structure. This method can be embedded into the convolutional layer structure of any convolutional neural network, making the generated feature representation contain more explicit temporal information, thus enabling the feature representation generated by the trained model to contain enhanced temporal information and improve the classification performance of the model. Therefore, the present invention can enable the convolutional layer of the convolutional neural network to embed temporal position information into the convolution result during the convolution process without increasing the model parameter scale, thereby achieving temporal information enhancement.

[0026] (2) The trigonometric function used in this invention can control the size of the time position information to be not significantly larger than the convolution result, thus avoiding the original convolution result being covered after summation. It can also characterize the periodic change state of typical waveforms that repeatedly appear in the EEG signal, and has excellent effect on enhancing temporal and temporal domain information.

[0027] (3) The trigonometric function used in this invention has a coding period that can simulate the period corresponding to the candidate frequency in the brainwave rhythm, thus providing coding efficiency.

[0028] (4) To reduce the dependence of the encoding period ω of the trigonometric function encoder on manually specified parameters, this invention proposes a matching parallel training algorithm that can automatically select the encoding period. Although parallel training will increase the computational cost of training the model, the computational cost of the trained model during testing is the same as that of the unimproved model. Therefore, this algorithm can effectively enhance the temporal information of the output feature representation of the convolutional layer of the convolutional neural network under the condition of controlling the computational cost, improve the classification performance of the model, and be used for real-time brain-computer interface classification tasks. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of a convolutional neural network temporal information enhancement method for brain-computer interface classification tasks provided in an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the parallel training method corresponding to the temporal information enhancement method provided in the embodiments of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0032] Example 1

[0033] A method for constructing convolutional neural networks with temporal information enhancement in convolutional output, for use in classification or regression tasks in brain-computer interfaces, such as... Figure 1 As shown, it includes:

[0034] The structure of the convolutional neural network to be enhanced is improved; a training sample set is sampled, and the improved convolutional neural network is trained to obtain a convolutional neural network with enhanced temporal information in the convolution output;

[0035] The improvement method is as follows: Based on the structure of the convolutional layer to be improved in the convolutional neural network to be enhanced, the size of its output feature representation is calculated to construct an average pooling layer with the same output size. The convolutional kernel in the convolutional layer to be enhanced is used to generate a convolution result for each local sample point matrix participating in the dot product calculation during sliding, ultimately generating a feature representation. The average pooling layer is used to generate a corresponding average pooling representation for each local sample point matrix participating in the dot product calculation, accompanying the sliding dot product process of the convolutional kernel, ultimately obtaining the average pooling representation of the entire input sample. A trigonometric encoder is constructed to perform temporal encoding on each element in the average pooling representation to obtain a new average pooling representation. A summation unit is constructed to sum the new average pooling representation with the corresponding elements of the feature representation output by the convolutional layer to obtain a temporally enhanced feature representation, which is then input into the next layer of the convolutional layer to be enhanced in the convolutional neural network.

[0036] Current convolutional neural network models for brain-computer interface classification tasks are limited by the convolutional kernel structure, resulting in a relatively weak ability to represent temporal information of scalp EEG signals over long time spans, thus affecting model classification performance. This embodiment provides a method that introduces an average pooling layer to perform average pooling on the input data to the convolutional layer to be improved, obtaining an average pooling representation. Each element in the average pooling representation is temporally encoded, and then each temporally encoded element is added to a corresponding element in the feature representation output by the convolutional layer. This embeds additional temporal information into the convolution calculation result. This method can be embedded into the convolutional layer structure of any convolutional neural network, making the generated feature representation contain more explicit temporal information. This allows the feature representation generated by the trained model to include enhanced temporal information, improving the model's classification performance. Therefore, without increasing the model parameter size, the convolutional layers of the convolutional neural network can embed temporal location information into the convolution result during the convolution process, thereby achieving temporal information enhancement.

[0037] Preferably, the trigonometric functions in the above trigonometric function encoder are represented as follows:

[0038]

[0039] In the formula, SE(t) represents the trigonometric function encoder, t represents the temporal position of the output element of the convolutional layer to be temporally encoded, and ω represents the encoding period of the trigonometric function SE(t).

[0040] The trigonometric function encoder SE(t) described above achieves temporal encoding of each element by multiplying the trigonometric function with each element in the average pooling representation, thereby obtaining the time position information.

[0041] In practical applications, the first step is to identify the convolutional layer requiring temporal information augmentation for the convolutional neural network, and denote its kernel as K. For each input data X, the kernel K in the standard convolutional layer slides to perform dot product calculations, resulting in a matrix of local sample points participating in the dot product calculation. Generate convolution result Z u,v Org =P u,v ·K; corresponding to the standard convolutional layer, construct an average pooling layer with the same output size, and during the sliding dot product process of the convolutional kernel, P is the local sample point matrix for each participating dot product calculation. u,v Generate the corresponding average pooling representation. Using a trigonometric function encoder (ω is the encoding period of the trigonometric function), for the average pooling representation R u,v Temporal encoding is performed, and the temporal position embedding is generated by summing the result with the convolution result, denoted as Z. u,v =P u,v ·K+SE(t v )*R u,v , where t v The feature representation corresponding to P in the convolution computation u,v The time position of the element.

[0042] Preferably, for different brain-computer interface tasks, the training data varies from task to task, and the time encoding of a single information amplitude cannot be directly applied to multiple tasks. In order to enhance the model's adaptive learning ability and thus improve the model's generalization performance, a parameter b that can be learned during network training is introduced during time encoding to generate scaled time location information (also known as time location embedding).

[0043] Based on this, the convolutional layer temporal information enhancement method proposed in this embodiment can be written as:

[0044]

[0045] It should be noted that, in order to control item 2 The size of f(t) is not significantly larger than the first term, thus covering the original convolution result represented by the first term, while being able to characterize the periodic changes of typical waveforms that repeatedly appear in EEG signals. This invention designs f(t) v It is hoped that the following three conditions can be met: (1) It has upper and lower bounds; (2) It has periodicity and can describe some recurring waveforms in EEG signals; (3) In each cycle, different positions correspond to different time codes and can describe the temporal order of signals in a single cycle.

[0046] Therefore, this invention employs trigonometric functions. As an encoding function.

[0047] Preferably, the encoding period ω is determined as follows:

[0048] Based on the scalp EEG rhythm, select the key frequency f. c ; and based on the sampling frequency f of the scalp EEG data s The encoding period ω is generated as follows:

[0049] It should be noted that trigonometric function encoding encodes the temporal position of each sampling point. For the trigonometric function sin(x), this embodiment aims to ensure that the encoding period can simulate the period corresponding to the candidate frequency in the EEG rhythm.

[0050] Assuming a signal has a sampling frequency of 128Hz, the time interval between each sampling point (e.g., x0 and x1) is 1 / 128s. To simulate the frequency of brainwave rhythms (e.g., the frequency band of delta waves is 0.5-4Hz), taking 4Hz as an example, the desired coding period is 1 / 4s. Therefore, when performing position coding on a signal at 4Hz, the expected interval between two sampling points with a coding period of one should be 1 / 4s. Since the time interval between each sampling point is 1 / 128s, let's assume that one coding period is completed after ω sampling points: 1 / 128s * ω = 1 / 4s. We can then derive ω = 32, meaning that the next coding period begins after every 32 sampling points. This ensures that there is exactly one coding period between two sampling points with a 32-sampling-point interval. Therefore, in this embodiment, the trigonometric function is rewritten as sin(x / 32). Generalizing the above process, we get: Where f s It is the sampling frequency, f c It is the critical frequency, i.e.: ω = f s / f c .

[0051] Preferably, such as Figure 2 As shown, the encoding period ω is determined as follows:

[0052] Based on scalp EEG rhythms, high-pass and low-pass frequencies of delta, theta, alpha, beta, gamma, and μ waves were selected as the key frequency set f. c Based on the sampling frequency f of scalp EEG data s Determine the candidate set for the coding period A set of convolutional neural network models with temporal information enhancement in the convolutional output is generated according to each candidate ω in the above-mentioned candidate encoding period candidate set, and all candidate models are trained in parallel; the model with the minimum loss on the validation set is selected as the final convolutional neural network model with temporal information enhancement in the convolutional output, and the ω used by this model is the final determined encoding period ω.

[0053] To reduce the dependence of the encoding period ω of the trigonometric function encoder on manually specified parameters, this embodiment proposes a matching parallel training algorithm that can automatically select the encoding period. The parallel training algorithm includes the following steps: selecting the high-pass and low-pass frequencies of δ, θ, α, β, γ, and μ waves as the key frequency set f based on the scalp EEG rhythm. c Based on the sampling frequency f of scalp EEG data s Determine the candidate set of coding cycles A set of candidate temporal information-enhanced convolutional neural network models is generated based on each candidate ω in the candidate set, and all candidate models are trained in parallel. The model with the minimum loss on the validation set is selected as the final classification model to classify the test data.

[0054] It should be noted that due to significant individual differences in EEG data—meaning there are marked differences between EEG data from different subjects—and variations in brain development and activity levels among subjects of different ages, addressing these individual differences and improving model generalization performance are crucial issues to consider in brain-computer interface tasks. This embodiment employs an adaptive encoding cycle selection strategy based on human EEG rhythms to enhance the model's learning ability. Since human EEG signals contain rhythms corresponding to different frequency bands such as δ, θ, α, β, γ, and μ, the high-pass and low-pass frequencies corresponding to these rhythms are extracted as key frequencies to generate a candidate encoding cycle set. During model training, the neural network is trained in parallel using each encoding cycle from the encoding cycle set, and the model with the minimum validation set loss is selected as the final training model. Since the minimum validation set loss indicates the model achieves the best performance on validation data, it means the model can learn the most effective feature representation for the task. This candidate method is essentially a grid search hyperparameter selection strategy, but by incorporating prior knowledge such as EEG signal rhythm, it greatly reduces the optimization range of the encoding cycle hyperparameter and improves the training efficiency of the model.

[0055] Preferably, since different convolutional neural networks have different structures, as convolutional layers are repeatedly stacked, the feature representations learned by the deep convolutional layers can incorporate different temporal and spatial mixed information. In order to enable the deep convolutional layers to receive enhanced temporal information, thereby helping the model to better learn effective feature representations and include them in the subsequent feature representations that integrate temporal and spatial information, the aforementioned convolutional layer to be improved is preferably the first convolutional layer of the aforementioned convolutional neural network to be enhanced.

[0056] Example 2

[0057] A method for enhancing temporal information in the convolutional output of a convolutional neural network is provided for classification or regression tasks in brain-computer interfaces. The method employs a convolutional neural network constructed using the aforementioned method for enhancing temporal information in the convolutional output to perform brain-computer interface classification or regression tasks, thereby achieving enhancement of temporal information in the convolutional output of the convolutional neural network.

[0058] The relevant technical solutions are the same as in Embodiment 1, and will not be repeated here.

[0059] Example 3

[0060] An electronic device includes: a processor and a memory storing processor-executable instructions, and a data transceiver for sending and / or receiving data, wherein when the instructions are executed by the processor, a method for constructing a convolutional neural network to enhance temporal information in the convolutional output as described above and / or a method for enhancing temporal information in the convolutional output of a convolutional neural network as described above.

[0061] The relevant technical solutions are the same as in Embodiment 1, and will not be repeated here.

[0062] Example 4

[0063] A computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed by a processor, it controls the device where the storage medium is located to perform a method for constructing a convolutional neural network with temporal information enhancement in the convolutional output as described above and / or a method for enhancing temporal information in the convolutional output of a convolutional neural network as described above.

[0064] The relevant technical solutions are the same as in Embodiment 1, and will not be repeated here.

[0065] In summary, this invention addresses the problem that current convolutional neural network models for classification tasks are relatively weak in representing temporal information of scalp EEG signals over long time spans due to limitations in the convolutional kernel structure, thus affecting model classification performance. This invention proposes an EEGNet temporal information enhancement algorithm that can be implemented without increasing the model parameter size. This algorithm allows EEGNet's convolutional layers to embed temporal location information into the convolution results during the convolution process, thereby achieving temporal information enhancement.

[0066] It should be noted that the method of this invention can be applied to temporal augmentation problems in epilepsy subtype classification tasks and other brain-computer interface tasks. Deep learning brain-computer interfaces all use EEG signals as input, feeding them to a neural network model to learn features from EEG signal samples according to corresponding task labels and then classifying them. The difference between other brain-computer interface tasks and epilepsy subtype classification tasks lies in the different labels of the samples. Therefore, the EEGNet temporal information augmentation algorithm proposed in this invention can also handle other EEG tasks. Furthermore, EEGNet is a deep learning method based on convolutional neural networks for EEG signals. Since the temporal information augmentation convolution calculation algorithm proposed in this invention is an improvement on the convolutional layers in EEGNet, this temporal information augmentation convolution calculation can also be extended to other deep learning methods based on convolutional neural networks for EEG signals by replacing the original convolutional layers with temporal information augmentation convolutions.

[0067] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a convolutional neural network with time domain information enhancement in convolutional output, used for classification tasks or regression tasks in brain-computer interface, characterized in that, The method comprises the following steps: An improved structure of a to-be-enhanced convolutional neural network is obtained; a training sample set is used to train the improved convolutional neural network, and a convolutional neural network with enhanced time domain information in a convolutional output is obtained. The improved method is as follows: according to the structure of a to-be-improved convolutional layer of the to-be-enhanced convolutional neural network, the size of a feature representation output by the to-be-improved convolutional layer is calculated, an average pooling layer with the same output size as the size of the feature representation is constructed, the average pooling layer is used to perform average pooling on input data during a convolution kernel sliding point multiplication process of the to-be-improved convolutional layer, and finally an average pooling representation is generated, a trigonometric function encoder is constructed, which is used to perform time sequence coding on each element in the average pooling representation to obtain a new average pooling representation, and a summation unit is constructed, which is used to sum corresponding elements of the new average pooling representation and the feature representation to obtain a feature representation with enhanced time domain information, and the feature representation with enhanced time domain information is input into a next layer structure of the to-be-improved convolutional layer in the to-be-enhanced convolutional neural network. The trigonometric function in the trigonometric function encoder is represented as follows: ； wherein denotes a trigonometric function, denotes the time position of the convolutional layer output element to be time-encoded, denotes a trigonometric function the encoding period; The trigonometric function encoder realizes the time sequence encoding of each element in the average pooling representation by multiplying the trigonometric function SE( t ) with the element, to obtain time position information. The encoding period The determination mode is: Based on scalp EEG rhythm, select The high-pass and low-pass frequencies of the wave serve as the key frequency set. Based on the sampling frequency of scalp EEG data Determine the candidate set for the coding period ; each candidate in the set of coding period candidates generate a set of convolutional neural network models with temporal information enhancement in the candidate convolutional output, and train all the candidate models in parallel; The model with the minimum validation set loss is selected as the final convolutional neural network model for enhancing time domain information in the convolutional output, and the model adopts The final determined coding period is .

2. The method of claim 1, wherein, In performing the temporal coding, the trigonometric function is also multiplied by a time scaling factor b to generate scaled time position information, b the values of which are obtained through the training. 3.The method of claim 1 or 2, wherein, The to-be-improved convolutional layer is a first layer convolutional layer of the to-be-enhanced convolutional neural network.

4. A method for enhancing time domain information in convolutional output of a convolutional neural network, used for classification tasks or regression tasks in brain-computer interfaces, characterized in that, A convolutional neural network is constructed by using the method for constructing a convolutional neural network with enhanced time domain information in a convolutional output according to any one of claims 1 to 3, a brain-computer interface classification task or a regression task is performed, and the time domain information in the convolutional output of the convolutional neural network is enhanced.

5. An electronic device, comprising: The method comprises the following steps: A processor and a memory storing instructions executable by the processor, and a data transceiver for data transmission and / or reception, when the instructions are executed by the processor, a method for constructing a convolutional neural network with enhanced time domain information in a convolutional output according to any one of claims 1 to 3 and / or a method for enhancing time domain information in a convolutional output of a convolutional neural network according to claim 4 are performed.

6. A computer readable storage medium characterized by, The computer readable storage medium comprises a stored computer program, wherein when the computer program is run by a processor, the device where the storage medium is located is controlled to perform a method for constructing a convolutional neural network with enhanced time domain information in a convolutional output according to any one of claims 1 to 3 and / or a method for enhancing time domain information in a convolutional output of a convolutional neural network according to claim 4.

Images