A searchable electroencephalogram time domain-space-separable fusion binary network

Abstract

This invention discloses a searchable EEG temporal-spatial-separable fusion binary network, belonging to the technical field of computation, inference, or counting. The network comprises a sequentially connected binary temporal modeling stage network skeleton, a binary spatial modeling stage network skeleton, a binary separable fusion stage network skeleton, and an output mapping layer. The weights and activations in each stage network skeleton are represented using binary representation, and weight scaling factors and activation scaling factors are configured to compensate for amplitude information loss. Under the constraint of a fixed stage sequence skeleton, neural architecture search can be used to assist in optimizing the candidate operator set within each stage to determine the target network configuration. This network reduces model storage and computational overhead while maintaining the expressive power of EEG features, making it suitable for EEG emotion recognition, fatigue state detection, sleep stage classification, and epileptic seizure detection.

Description

Technical Field

[0001] This invention relates to EEG signal processing and low bit-width neural network technology, specifically disclosing a searchable EEG temporal-spatial-separable fused binary network, belonging to the technical field of calculation, estimation, or counting. Background Technology

[0002] Electroencephalograms (EEGs) are physiological signals that reflect the state of neural activity in the brain. They are characterized by non-invasive acquisition, high temporal resolution, and wide applicability, and have been widely used in tasks such as EEG emotion recognition, fatigue detection, sleep stage classification, and epileptic seizure detection. However, due to the typically low signal-to-noise ratio, significant individual variability, complex temporal dynamics, and significant cross-channel correlations in EEG signals, constructing EEG recognition networks that balance feature representation ability, training stability, and deployment efficiency remains a crucial research direction in this field.

[0003] In existing EEG recognition networks, one type of network mainly relies on manually designed feature extraction processes and classification models, such as extracting frequency band features, time-frequency features, or differential entropy features after preprocessing, and then combining them with convolutional neural networks, recurrent neural networks, or their variants for recognition. Another type of network enhances the representation ability of EEG signals through multi-scale convolution and spatiotemporal joint modeling. While both types of networks improve recognition performance to some extent, their network structures mostly still rely on human experience and lack a unified, systematic structural organization method, making it difficult to simultaneously address the expressive capabilities and deployment requirements of different EEG tasks.

[0004] On the other hand, as EEG intelligent processing gradually migrates to wearable devices, portable terminals, and edge platforms, the storage and computational costs of models have become significant factors affecting practical deployment. While traditional full-precision networks possess strong expressive power, their large parameter count and high computational cost hinder efficient implementation in resource-constrained scenarios. Low-bit-width reduction, especially binarization, can significantly compress parameter storage and reduce computational complexity; however, directly performing simple binarization on EEG recognition networks can easily lead to problems such as early-stage information loss, unstable training, and decreased recognition performance.

[0005] Especially for inputs like EEG signals that are dynamic and sensitive to noise, if there is a lack of binary network skeleton constraints adapted to the EEG task, particularly a lack of fixed stage sequence constraints between the temporal modeling stage, spatial modeling stage, and separable fusion stage, as well as constraints on the operator form and parameter configuration within each stage, then binary networks built solely based on experience often struggle to simultaneously handle temporal dynamic information extraction, EEG channel spatial relationship modeling, and lightweight feature fusion in the later stages, thus making it difficult to guarantee the network's expressive power and trainability.

[0006] Furthermore, existing low bit-width networks typically only perform simple symbolic processing on weights and activations, lacking an effective compensation mechanism for the amplitude information loss caused by binarization. Especially in EEG recognition tasks, if scaling factors are not introduced in each stage of the network to compensate for the binarization of weights and activations, the information degradation caused by binarization can be further amplified, affecting the network's convergence stability and the final recognition performance.

[0007] While existing neural architecture search methods can reduce reliance on manual design to some extent, their main role is to assist in optimizing candidate network configurations; they cannot replace the construction of binary network skeletons. Without a unified, searchable EEG temporal-spatial-separable fused binary network skeleton, relying solely on search methods for network selection can easily lead to problems such as an overly broad search space, a lack of EEG task specificity in network structures, and insufficient stability of search results.

[0008] Therefore, there is an urgent need to provide a new searchable EEG temporal-spatial-separable fusion binary network to overcome the shortcomings of existing EEG recognition technologies. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of the aforementioned background technology by providing a searchable EEG temporal-spatial-separable fusion binary network. This network constructs a fixed-stage sequence of binary temporal modeling stage network skeletons, binary spatial modeling stage network skeletons, and binary separable fusion stage network skeletons. It introduces weights and activation binarization representations, as well as scaling factor compensation mechanisms, in each stage to solve the technical problems of existing EEG recognition network structures relying on manual experience setting, insufficient binarization stability, and difficulty in balancing recognition performance and deployment efficiency. This achieves the invention's objective of reducing network storage overhead and computational complexity while maintaining the expressive power of EEG features.

[0010] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0011] A searchable EEG temporal-spatial-separable fusion binary network includes an input layer, a binary temporal modeling stage network skeleton, a binary spatial modeling stage network skeleton, a binary separable fusion stage network skeleton, and an output mapping layer connected in sequence.

[0012] The input layer is used to receive input EEG features obtained from the raw EEG signals through preprocessing and feature construction. The preprocessing and feature construction include one or more of the following processes: baseline removal, filtering and denoising, artifact suppression, temporal segmentation, construction of raw temporal feature tensors, frequency band feature extraction, time-frequency feature extraction, and differential entropy feature extraction.

[0013] The binary temporal modeling stage network backbone is used to extract temporal dynamic features of the input EEG features along the time dimension. Preferably, the binary temporal modeling stage network backbone consists of one or more binary temporal convolutional layers. The binary temporal convolutional units perform convolution operations along the time dimension and include a weight and activation binarization module and a scaling factor compensation module. The stage configuration parameters of the binary temporal modeling stage network backbone include at least one of the following: number of layers, number of output channels, convolution kernel size, convolution stride, and grouping parameters.

[0014] The binary space modeling stage network skeleton is used to extract spatial correlation features between different EEG channels from the temporal dynamic features of the input EEG features. Preferably, the binary space modeling stage network skeleton consists of one or more binary space convolutional units, which are used to model the spatial relationships between different EEG channels and include a weight and activation binarization module and a scaling factor compensation module. The convolutional units in the binary space modeling stage network skeleton include one or more of the following: cross-EEG channel binary space convolutional units, binary grouped space convolutional units, and binary depth space convolutional units; the stage configuration parameters of the binary space modeling stage network skeleton include at least one of the following: number of layers, number of output channels, spatial projection method, grouping method, and channel expansion configuration.

[0015] Binary separable fusion stage network backbone is used for lightweight fusion and enhancement of spatial correlation features between different EEG channels. Preferably, the binary separable fusion stage network backbone includes deep convolutional units and pointwise convolutional units, and optionally includes residual connection units; the weights and activations in the deep convolutional units, pointwise convolutional units, and residual connection units are represented in binary form, and weight scaling factors and activation scaling factors are configured. The stage configuration parameters of the binary separable fusion stage network backbone include deep convolutional configuration and pointwise convolutional configuration, and optionally include residual configuration.

[0016] The output mapping layer is used to map the features output by the network skeleton after the binary separable fusion stage to the final recognition result. Preferably, the output mapping layer includes a pooling layer and a classification head; wherein the pooling layer is an average pooling layer or a global average pooling layer, and the classification head is a linear classification layer or a binary linear classification layer.

[0017] The weights and activations in the network skeletons of the binary temporal modeling stage, the binary spatial modeling stage, and the binary separable fusion stage are represented using binarization, with weight scaling factors and activation scaling factors configured respectively. Convolution operations in each stage's network skeleton are implemented based on the binarized representations of the current real-valued weights and activations to compensate for the loss of amplitude information caused by binarization. The binarization representation of weights and activations includes converting the real-valued weights... Mapped to binary weights And activate the real value. Mapping to binary activation The current real-valued weights are converted into binary weights after sign mapping and weight scaling factor compensation, and the current real-valued activations are converted into binary activations after sign mapping and activation scaling factor compensation; where, the binary weights and binary activation They respectively satisfy the following relations:

[0018]

[0019]

[0020] in, This is the weight scaling factor. To activate the scaling factor, It is a symbolic function.

[0021] The symbolic function satisfies:

[0022]

[0023] As a further optimization scheme for searchable EEG temporal-spatial-separable fusion binary networks, the weight scaling factor and activation scaling factor are updated collaboratively with the real-valued weights and the corresponding network skeleton parameters during network training. This is used to compensate for the loss of amplitude information caused by binarization and improve the stability of network training. During the training phase, the loss is calculated based on the convolutional forward output features. During backpropagation, the sign function is approximated by gradient propagation using a pass-through estimation method. The approximate gradient is then propagated to the current real-valued weights and corresponding network parameters. Combined with the normalized gradient operation, the current real-valued weights, corresponding network parameters, weight scaling factor, and activation scaling factor are jointly updated.

[0024] As a further optimization scheme for the searchable EEG temporal-spatial-separable fusion binary network, a step-by-step feature transfer method is adopted between the binary temporal modeling stage network skeleton, the binary spatial modeling stage network skeleton, and the binary separable fusion stage network skeleton. The feature map output by the previous stage is used as the feature map input by the next stage, so as to realize a continuous feature transformation process from EEG temporal dynamic information extraction, to EEG channel spatial relationship modeling, and then to lightweight feature fusion enhancement.

[0025] As a further optimization scheme for searchable EEG temporal-spatial-separable fused binary networks, under the constraint of a network skeleton with a fixed stage sequence, the stage configuration parameters can be determined by performing neural architecture search-assisted optimization on candidate network structures within each stage.

[0026] As a further optimization scheme for searchable EEG temporal-spatial-separable fusion binary networks, neural architecture search-assisted optimization selects the best candidate network structures at each stage based on comprehensive evaluation results. The comprehensive evaluation includes at least convergence evaluation indicators, recognition performance indicators, and efficiency indicators. Among them, the convergence evaluation indicators are used to characterize the training cost required for the target network composed of candidate network structures at each stage to reach the preset verification performance threshold; the recognition performance indicators are used to characterize the task performance of the target network composed of candidate network structures at each stage on the verification set; and the efficiency indicators are used to characterize the resource consumption of the target network composed of candidate network structures at each stage in terms of parameter bit count, computational complexity, or storage overhead.

[0027] As a further optimization of searchable EEG temporal-spatial-separable fusion binary networks, the convolution calculations in the network skeletons of the binary temporal modeling stage, binary spatial modeling stage, and binary separable fusion stage can be equivalently transformed into bitwise operations based on binary weights and accumulated binary activation correlations. For a length of... The cumulative correlation result of the binary input vector and binary weight vector. satisfy:

[0028]

[0029] in, The first binary activation Bit, The first of the binary weights Bit;

[0030] During the inference phase, the correlation accumulation form of binary weights and binary activations is equivalently transformed into bitwise XOR operations and bit counting forms. The equivalent representation is:

[0031]

[0032] in, This indicates a bitwise XOR operation. This indicates the number of "1"s in the digit-wise count result.

[0033] The present invention, by adopting the above technical solution, has the following beneficial effects:

[0034] (1) The present invention takes the three-stage binary network skeleton of EEG time domain-space-separable fusion as the main body. By constructing the binary time domain modeling stage network skeleton, the binary space modeling stage network skeleton and the binary separable fusion stage network skeleton in a fixed stage order, the temporal dynamic modeling, cross-channel spatial modeling and lightweight feature fusion form a clear division of labor, which can improve the stability and relevance of network organization while maintaining the prior structure of EEG task.

[0035] (2) The present invention introduces weight and activation binarization representation and scaling factor compensation mechanism in the network skeleton at each stage, and makes the weight scaling factor and activation scaling factor updated collaboratively during training, thereby reducing parameter storage overhead and computational complexity, mitigating amplitude information loss caused by binarization, and improving the trainability and stability of the network.

[0036] (3) Under the network skeleton constraint of fixed stage sequence, the present invention performs neural architecture search-assisted optimization on the candidate operator set and structural parameters in each stage, so that the network configuration determination process is based on clear skeleton constraints, thereby reducing the dependence on human experience setting and improving the adaptability of the target network in different EEG tasks.

[0037] (4) This invention transforms binary convolution computation into a bitwise operation form based on binary weights and binary activation correlation accumulation, providing a unified computational basis for subsequent low-bit-width inference and deployment, and enabling the network to balance recognition performance and deployment friendliness. It is applicable to various EEG application scenarios such as EEG emotion recognition, fatigue state detection, sleep stage classification and epileptic seizure detection. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the overall searchable EEG temporal-spatial-separable fused binary network proposed in this invention.

[0039] Figure 2 This is a schematic diagram illustrating the stage unfolding of the searchable EEG temporal-spatial-separable fused binary network skeleton proposed in this invention.

[0040] Figure 3 This is a schematic diagram illustrating the relationship between weights and activation binarization, scaling factor settings, and training and inference proposed in this invention.

[0041] Figure 4 This is a schematic diagram illustrating the candidate structure organization and neural architecture search-assisted optimization proposed in this invention. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to explain the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the spirit and essence of the present invention, those skilled in the art can make adjustments, substitutions or equivalent transformations to the input EEG feature construction method, network skeleton internal parameter configuration method, candidate structure organization method within a stage, neural architecture search-assisted optimization method, and training and inference settings, all of which should fall within the scope of protection of the present invention.

[0043] Example 1: This example uses an EEG emotion recognition task to illustrate a searchable EEG temporal-spatial-separable fusion binary network. The overall structure of the network is as follows: Figure 1 As shown, the network includes an input layer, a binary temporal modeling stage network skeleton, a binary spatial modeling stage network skeleton, a binary separable fusion stage network skeleton, and an output mapping layer. Simultaneously, under the constraint of a candidate network macro-skeleton with a fixed stage order, the candidate operator set within each stage is optimized through neural architecture search, and the target network configuration is obtained by combining comprehensive evaluation.

[0044] In this embodiment, the internationally available Dataset for Emotion Analysis using Physiological Signals (DEAP) was used as the source of the raw EEG signals. The dataset contains EEG signals from 32 subjects watching 40 emotion-evoked videos. Each video is 63 seconds long, with the first 3 seconds representing the resting baseline phase and the following 60 seconds representing the emotion-evoked phase. The raw EEG signals were sampled at 128 Hz, with each trial corresponding to 32 channels of EEG sampling signal. Based on these raw EEG signals, preprocessing and feature construction were first performed to form the input EEG features for the subsequent network.

[0045] Specifically, baseline correction is first performed on the raw EEG signal. Let the raw EEG signal be denoted as... ,in Indicates the number of channels. This represents the total number of sampling points. The first 3 seconds of signal are taken as the baseline segment, and the mean of each channel within the baseline segment is calculated. Then, channel-level mean reduction processing is performed on the entire raw EEG signal to reduce the impact of resting-state bias and individual amplitude differences on subsequent recognition. After baseline correction, the subsequent 60 seconds of EEG signal during the emotion-induced phase are retained. This signal is then segmented into time segments of fixed length, and the 60 seconds of EEG signal is further divided into 4-second sample segments. Each sample segment corresponds to 512 sampling points, thus yielding 15 sample segments per experiment.

[0046] After completing the temporal segmentation, differential entropy features are extracted from each sample segment. Specifically, the EEG signal is divided into 5 frequency bands, namely... Wave, Wave, Wave, wave and The signal is processed by bandpass filtering in each frequency band. The filtered signal... If it approximately follows a Gaussian distribution, then the difference entropy can be expressed as:

[0047]

[0048] In equation (1), Let be the variance of the EEG signal sample segments. To preserve temporal dynamics, a sliding window is used to extract time slice features within each 4-second sample segment. The sliding window length is set to 128 sampling points, and the step size is set to 64 sampling points, resulting in 7 time slices. After the above processing, the input EEG feature tensor corresponding to each sample segment can be represented as: That is, 32 channels, 5 frequency bands and 7 time slices.

[0049] In terms of label construction, taking the emotion binary classification task as an example, the original scores can be discretized according to a threshold. In valence or arousal tasks, a classification threshold of 5 is used; scores greater than 5 are recorded as positive, and scores less than or equal to 5 are recorded as negative. After feature extraction and label generation for all sample segments, the training, validation, and test sets are divided in an 8:1:1 ratio. The training set is used for model parameter updates, the validation set is used for candidate network selection and hyperparameter tuning, and the test set is used for final performance verification. This division method ensures that the training, selection, and testing processes are independent of each other.

[0050] After obtaining the input EEG features, a searchable EEG temporal-spatial-separable fused binary network skeleton is constructed. Figure 1 A general schematic diagram of the network is shown. Figure 2 A schematic diagram of the phased unfolding of the network skeleton is shown.

[0051] First, the network skeleton for the binary temporal modeling stage is constructed. This stage is used to extract the temporal dynamic features of the input EEG feature signals along the time dimension, such as... Figure 2 As shown in the upper part. In this embodiment, the binary temporal modeling stage consists of multiple temporal convolutional layers connected sequentially, which can be respectively referred to as temporal convolutional layer 1, temporal convolutional layer 2 to temporal convolutional layer 3. Each temporal convolutional layer corresponds to a specific number of channels. , to and convolution kernel , to In this stage, weights and activations are binarized, and a scaling factor is introduced to compensate for the loss of amplitude information caused by binarization. Through the layer-by-layer operation of the aforementioned temporal convolutional layers, local and continuous change patterns of the input EEG feature signals at different time scales can be extracted.

[0052] Secondly, the network skeleton for the binary space modeling stage is constructed. This stage is used to extract spatial correlation features between different EEG channels based on the temporal domain modeling results, such as... Figure 2As shown in the middle. In this embodiment, the binary space modeling stage consists of multiple spatial convolutional layers connected sequentially, which can be respectively denoted as spatial convolutional layer 1, spatial convolutional layer 2, and so on. Each spatial convolutional layer corresponds to a specific number of channels. , to and spatial projection parameters , to In this stage, weights and activations are also binarized and a scaling factor is introduced for compensation. Through this stage, spatial coupling relationships across EEG channels can be further established based on intermediate features that already contain temporal discrimination information.

[0053] Next, a binary separable fusion stage network skeleton is constructed. This stage is used to perform lightweight fusion and enhancement of features after temporal and spatial modeling, such as... Figure 2 As shown in the lower part. In this embodiment, this stage includes at least depthwise convolution and pointwise convolution, and optionally residual connections. Depthwise convolution is used to extract local features at a lower parameter cost, pointwise convolution is used to achieve inter-channel blending, and optional residual connections are used to enhance feature propagation stability and deep network training stability. In this stage, weights and activations are also represented using binarization, and a scaling factor is introduced for compensation.

[0054] In this embodiment, the binary temporal modeling stage, the binary spatial modeling stage, and the binary separable fusion stage are connected sequentially in a fixed order. The feature map output from the previous stage serves as the feature map input to the next stage, thus forming a continuous feature transformation process from extracting EEG temporal dynamic information to modeling the spatial relationships of EEG channels, and then to lightweight feature fusion enhancement. These stages together constitute the main framework of a searchable EEG temporal-spatial-separable fusion binary network. The output features extracted from the three-stage network framework are then input to the output mapping layer, which includes a pooling layer and a classification head, used to output the final recognition result.

[0055] In this embodiment, the weights and activations in each stage of the network are adopted. Figure 3 The binary representation relation is shown. Let the real-valued weights be... Real-value activation is After symbolic mapping and scaling factors, binary weights are obtained respectively. and binary activation ,satisfy:

[0056]

[0057]

[0058] In equations (2) and (3), This is the weight scaling factor. , To prevent the weight scaling factor from being a constant of 0, This is the maximum value of the weight scaling factor. , To activate the scaling factor, , To prevent the activation of a constant with a scaling factor of 0, To activate the maximum scaling factor, , The sign function is used. The binarized weights and activations are then combined for binary convolution computation, and their cumulative correlation is denoted as... ,satisfy:

[0059]

[0060] In equation (4), The first binary activation Bit, The first of the binary weights Bit, represents the number of bits for the binary weights and binary activations.

[0061] Furthermore, in the inference phase, binary convolution computation can be equivalently represented as a bitwise XOR operation and bit counting, i.e.:

[0062]

[0063] In equation (5), This indicates a bitwise XOR operation. This indicates that the number of "1"s in the result of a bitwise XOR operation is counted.

[0064] Figure 3 This further illustrates the relationship between the training and inference phases. During the training phase, based on the current input features and the current real-valued weights... and current real value activation Binary weights are obtained after sign mapping and scaling factor compensation. and binary activation The system performs binary convolution forward computation based on these features to obtain the convolution forward output features. After calculating the loss based on the convolution forward output features and task labels, during backpropagation, a pass-through estimation method is used to approximate the gradient propagation of the sign function, allowing the gradient to be backpropagated to the current real-valued weights and related network parameters. Simultaneously, the backpropagated gradients or corresponding intermediate features are normalized to stabilize the gradient magnitude and correct the training instability caused by binarization. Subsequently, based on the gradient obtained from backpropagation, the current real-valued weights, related network parameters, weight scaling factors, and activation scaling factors are jointly updated to obtain the trained network parameters. During the inference phase, using the trained network parameters and the updated weight scaling factors and activation scaling factors, the corresponding binary activation is generated based on the current input features, and the corresponding binary weights are generated based on the trained real-valued weights. Then, a bitwise XOR operation and bit counting are performed to obtain the cumulative correlation result. This leads to the output of the corresponding features.

[0065] In this embodiment, neural architecture search is not the main body of the invention, but rather an auxiliary optimization method under the constraints of a fixed-stage sequential network skeleton. Figure 4 The process of candidate structure organization and neural architecture search-assisted optimization within each stage is illustrated. First, a macroscopic skeleton of the candidate network with a fixed stage order is constructed, namely, the binary temporal modeling stage, the binary spatial modeling stage, and the binary separable fusion stage are connected sequentially. Then, candidate sets for the temporal stage, the spatial stage, and the fusion stage are constructed respectively.

[0066] Among them, the candidate set for the time-domain stage includes candidate structures. to Candidate structures, each candidate structure consists of the number of layers. to Number of channels to and convolution kernel parameters to Representation; the candidate set for spatial stages includes candidate structures To candidate structure Each candidate structure consists of a number of layers. to Number of channels to and spatial projection parameters to Characterization; the candidate set for the fusion stage includes candidate structures To candidate structure Each candidate structure consists of depthwise convolution configuration, pointwise convolution configuration, and residual configuration. to Characterization. Combining the candidate structures from the above stages yields a set of candidate network structures, resulting in candidate network 1, candidate network 2, and so on. .

[0067] After obtaining the candidate network structure set, a comprehensive evaluation is performed on each candidate network to select the best one. The comprehensive evaluation metrics include convergence evaluation metrics, recognition performance metrics, and efficiency metrics. The convergence evaluation metrics characterize the training cost required for a candidate network to reach a preset validation performance threshold; the recognition performance metrics characterize the recognition effect of the candidate network on the validation set; and the efficiency metrics characterize the resource consumption of the candidate network in terms of parameter bits, computational complexity, or storage overhead. After comprehensive evaluation, candidate networks that meet the requirements are determined as the target network structure, while those that do not meet the requirements are discarded and returned to the candidate set for recombination.

[0068] As a specific implementation of the input layer, the input end can be set with a multi-scale time-domain front-end layer to perform initial time-domain processing on the input EEG signal; the subsequent binary time-domain modeling stage, binary space modeling stage, and binary separable fusion stage together constitute a fixed three-stage network skeleton.

[0069] In this embodiment, after neural architecture search-assisted optimization and comprehensive evaluation, a set of specific target network instances is obtained. These instances include the following structures:

[0070] A multi-scale time-domain front-end layer is set at the input end to perform initial time-domain processing on the input EEG signals;

[0071] The first binary temporal convolutional layer uses a convolutional kernel. The grouping parameter is 4;

[0072] The second binary temporal convolutional layer uses a convolutional kernel. The grouping parameter is 4;

[0073] The third binary temporal convolutional layer uses a convolutional kernel. The grouping parameter is 8;

[0074] The binary space modeling stage employs spatial depthwise convolution, with the spatial convolution kernel being... ;

[0075] The first binary separable fusion unit uses a deep convolution kernel. and pointwise convolution kernel ;

[0076] The second binary separable fusion unit adopts a residual separable structure, and its depthwise convolution kernel is... The pointwise convolution kernel is ;

[0077] The output mapping layer uses an average pooling layer and a binary linear classifier, where the average pooling kernel is... .

[0078] The above examples demonstrate that, under the constraints of a fixed temporal-spatial-separable fusion three-stage skeleton, by performing neural architecture search-assisted optimization on candidate structures within each stage, and by combining weight and activation binarization representations with scaling factor settings, a target network structure that balances recognition capability and deployment efficiency can be obtained.

[0079] To improve network training stability, this embodiment employs supervised learning during training, using the cross-entropy loss function as the optimization objective and an adaptive gradient optimization method for parameter updates. Learning rate scheduling includes a warm-up phase and an annealing phase. The warm-up phase linearly increases the learning rate, while the annealing phase gradually decreases the learning rate according to a preset strategy. Lightweight temporal augmentation is also introduced during training to improve network robustness. Temporal augmentation includes one or more of amplitude scaling, Gaussian noise perturbation, and time shifting.

[0080] Through the aforementioned network skeleton construction, binarization representation, scaling factor setting, neural architecture search-assisted optimization, and output mapping layer configuration, this embodiment implements a searchable EEG temporal-spatial-separable fusion binary network. This network reduces parameter storage overhead and computational complexity while maintaining the ability to express EEG features, and is suitable for EEG tasks such as EEG emotion recognition, fatigue state detection, sleep stage classification, and epileptic seizure detection.

[0081] It should be noted that the above embodiments are merely specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Those skilled in the art can adjust, replace, or make equivalent transformations to the input EEG feature construction method, three-stage network skeleton configuration, weight and activation binarization representation method, scaling factor setting method, intra-stage candidate structure organization method, neural architecture search-assisted optimization method, comprehensive evaluation and selection method, and output mapping layer form without departing from the spirit and substance of the present invention. All technical solutions falling within the scope of protection of the claims of the present invention should be included within the scope of protection of the present invention.

Claims

1. A searchable electroencephalogram time domain-space-separable fusion binary network, characterized in that, include: The input layer receives input EEG features obtained from the preprocessing and feature construction of the raw EEG signals. The binary temporal modeling stage network skeleton is used to extract temporal dynamic features of the input EEG features along the time dimension. The binary space modeling stage network skeleton is used to extract spatial correlation features between different EEG channels from the temporal dynamic features of the input EEG features. A binary separable fusion stage network skeleton is used for lightweight fusion and enhancement of spatial correlation features between the different EEG channels, and... The output mapping layer is used to map the features output by the binary separable fusion stage network skeleton to the final recognition result; The convolution operations in the binary temporal modeling stage network skeleton, the binary spatial modeling stage network skeleton, and the binary separable fusion stage network skeleton are implemented based on the binarized representations of the current real-valued weights and the current real-valued activations. The binarized representation of the current real-valued weights is the binary weights obtained after sign mapping and weight scaling factor compensation. The binarized representation of the current real-valued activations is the binary activations obtained after sign mapping and activation scaling factor compensation. The weight scaling factor and activation scaling factor are updated collaboratively with the real-valued weights and corresponding network parameters during the training phase.

2. The searchable EEG temporal-spatial-separable fused binary network according to claim 1, characterized in that, The preprocessing and feature construction include at least one of the following processes: baseline removal, filtering and denoising, artifact suppression, temporal segmentation, construction of original temporal feature tensors, frequency band feature extraction, time-frequency feature extraction, and differential entropy feature extraction.

3. The searchable EEG temporal-spatial-separable fused binary network according to claim 2, characterized in that, The network skeleton of the binary temporal modeling stage consists of at least one binary temporal convolutional layer, which performs convolution operations on binary weights and binary activations along the time dimension.

4. The searchable EEG temporal-spatial-separable fused binary network according to claim 3, characterized in that, The network skeleton of the binary space modeling stage consists of at least one binary space convolutional layer, which is used to model the spatial relationship between different EEG channels; the binary space convolutional layer is a binary space convolutional unit across EEG channels, or a binary grouping space convolutional unit across EEG channels, or a binary depth space convolutional unit across EEG channels.

5. The searchable EEG temporal-spatial-separable fused binary network according to claim 4, characterized in that, The binary separable fusion stage network skeleton includes deep convolutional units and pointwise convolutional units.

6. The searchable EEG temporal-spatial-separable fused binary network according to claim 5, characterized in that, The binarized representation of the current real-value weight and the current real-value activation specifically involves: representing the current real-value weight... Mapped to binary weights And activate the current real value. Mapping to binary activation , , The convolution operations in the network skeletons of the binary temporal modeling stage, the binary spatial modeling stage, and the binary separable fusion stage are represented as the cumulative correlation of binary weights and binary activations: ,in, This is the weight scaling factor. , To prevent the weight scaling factor from being a constant of 0, This is the maximum value of the weight scaling factor. ; To activate the scaling factor, , To prevent the activation of a constant with a scaling factor of 0, To activate the maximum scaling factor, ; For symbolic functions, For the cumulative results of correlation, For binary activation of the first Bit, The first of the binary weights Bit, The number of bits for binary weights and binary activations; during the inference phase, the correlation accumulation form of the binary weights and binary activations is equivalently transformed into a bitwise XOR operation and a bit count form: ,in, This indicates a bitwise XOR operation. This indicates that the number of "1"s in the result of a bitwise XOR operation is counted.

7. The searchable EEG temporal-spatial-separable fused binary network according to claim 6, characterized in that, The weight scaling factor and activation scaling factor are updated collaboratively with the real-valued weights and corresponding network parameters during the training phase. Specifically, the loss is calculated based on the convolutional forward output features, which are obtained from the binarized representations of the current real-valued weights and the current real-valued activations. During backpropagation, the sign function is directly estimated, and the approximate gradient is passed to the current real-valued weights and corresponding network parameters. The passed gradient is then normalized, and subsequently, the current real-valued weights, corresponding network parameters, weight scaling factor, and activation scaling factor are jointly updated based on the normalized gradient.

8. The searchable EEG temporal-spatial-separable fused binary network according to claim 7, characterized in that, The stage configuration parameters of the binary temporal modeling stage network skeleton, the binary spatial modeling stage network skeleton, and the binary separable fusion stage network skeleton are determined by neural architecture search-assisted optimization of the candidate network structures for each stage under the constraint of the network skeleton in a fixed stage order.

9. The searchable EEG temporal-spatial-separable fused binary network according to claim 8, characterized in that, The stage configuration parameters of the network skeleton in the binary temporal modeling stage include at least one of the following: number of layers, number of output channels, kernel size, stride, and grouping parameters. The set of stage configuration parameters for the network skeleton in the binary space modeling stage includes at least one of the following: number of layers, number of output channels, spatial projection method, grouping method, and channel expansion configuration. The stage configuration parameters of the binary separable fusion stage network skeleton include depthwise convolution configuration and pointwise convolution configuration.

10. The searchable EEG temporal-spatial-separable fused binary network according to claim 9, characterized in that, The neural architecture search-assisted optimization selects the best candidate network structures at each stage based on the comprehensive evaluation results. The comprehensive evaluation includes at least convergence evaluation indicators, recognition performance indicators, and efficiency indicators. The convergence evaluation indicators are used to characterize the training cost required for the target network composed of candidate network structures at each stage to reach the preset verification performance threshold. The recognition performance indicators are used to characterize the task performance of the target network composed of candidate network structures at each stage on the verification set. The efficiency indicators are used to characterize the resource consumption of the target network composed of candidate network structures at each stage in terms of parameter bits, computational complexity, or storage overhead.

Images