Electroencephalogram signal processing method and device based on memristor two-dimensional collaborative learning architecture

Abstract

The invention relates to an electroencephalogram signal processing method and device based on a memristor two-dimensional collaborative learning architecture, and the method comprises the steps: importing a multi-channel electroencephalogram signal from an electroencephalogram emotion data set, and carrying out the preprocessing of the electroencephalogram signal, so as to construct a frequency-time two-dimensional feature map; inputting the frequency-time two-dimensional feature map into a local feature learning module to extract multi-level spatial features, and judging an emotion category corresponding to the electroencephalogram signal according to the spatial features; carrying out physical disturbance modeling based on memristor hardware to obtain a population variation mechanism; inputting variation sources of population individuals into a global evolution optimization module to increase memristive hardware disturbance, and constructing an optimization strategy to evaluate emotion categories corresponding to the electroencephalogram signals and select suitable persons, so as to screen out an optimal emotion category recognition result of the electroencephalogram signals and feed back the optimal emotion category recognition result to a local feature learning module; and forming a closed-loop process of local learning and global evolution cooperative iteration. The accuracy of the emotion category recognition result of the electroencephalogram signal is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for processing electroencephalogram (EEG) signals based on a memristor two-dimensional collaborative learning architecture. Background Art

[0002] With the rapid development of artificial intelligence technology, the field of EEG signal processing has also ushered in new opportunities and challenges. As an important physiological signal reflecting brain activity, EEG signals contain rich information and can be used in many applications such as emotion recognition, disease diagnosis, brain-computer interface, etc. However, traditional EEG signal processing methods face many limitations and are difficult to meet the growing application needs.

[0003] Most of the existing EEG signal processing models based on machine learning are trained and optimized on specific data sets, lacking the ability to generalize to different individuals and different emotion-inducing conditions. When faced with new data or scenarios, the performance of the model often drops significantly, limiting the scope of its practical application. In addition, the traditional convolutional neural network training method is sensitive to the non-idealities of the hardware itself (such as weight quantization errors, write-read changes, conductance fluctuations, etc.). These physical layer perturbations continue to accumulate during the training process, resulting in unstable convergence and performance degradation during model training. Although the existing compensation strategies can mitigate the impact of hardware errors to a certain extent, they usually require the introduction of complex error correction circuits or algorithm modules, which significantly increases the system complexity and resource overhead. Therefore, the traditional convolutional neural network training method fails to fully utilize hardware perturbations as a positive learning driver. Most hardware-aware training methods regard hardware perturbations as negative factors and take compensation, avoidance or concealment methods to deal with them. This approach ignores the natural randomness and variability contained in hardware perturbations and fails to tap their potential functional value in the learning process. Therefore, existing methods have limited adaptability and robustness in low-precision and dynamic disturbance environments, and are difficult to support long-term stable learning in edge scenarios.

[0004] In addition, most existing EEG signal processing methods do not fully draw on the brain's own information processing mechanism. The brain is efficient and adaptive when processing information such as emotions, but traditional methods fail to effectively simulate these biological mechanisms and are limited to a single optimization dimension, that is, relying on the gradient descent of a single model parameter for learning. There is a lack of a synergistic mechanism between local learning and global adaptation, and it is difficult to achieve global adaptation in a hardware environment with strong disturbances and limited resources. As a result, the processing process is quite different from the natural processing method of the brain, and it is difficult to achieve the level of intelligent processing at the brain level. Summary of the invention

[0005] The present invention provides a method and device for processing electroencephalogram (EEG) signals based on a memristor two-dimensional collaborative learning architecture, aiming to solve at least one of the technical problems existing in the prior art.

[0006] The technical solution of the present invention is a method for processing electroencephalogram (EEG) signals based on a memristive two-dimensional collaborative learning architecture, which includes:

[0007] Import multi-channel EEG signals from an EEG emotion dataset, and perform short-time Fourier transform, channel-wise splicing, and segment-wise chunking preprocessing on the EEG signals to construct a frequency-time two-dimensional feature map;

[0008] Input the frequency-time two-dimensional feature map into the local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture. The local feature learning module in the horizontal dimension simulates the hierarchical perception mechanism of the mammalian visual cortex based on a convolutional neural network, extracts multi-level spatial features of the frequency-time two-dimensional feature map through the hierarchical perception mechanism, and determines the emotion category corresponding to the EEG signal according to the spatial features;

[0009] Model a population mutation mechanism based on the physical perturbation during the weight writing and reading processes of memristive hardware, and provide a mutation source for population individuals through the population mutation mechanism;

[0010] Input the mutation source of the population individuals into the global evolutionary optimization module in the vertical dimension of the memristive two-dimensional collaborative learning architecture to increase the memristive hardware perturbation. Construct an optimization strategy by using the individual mutation and adaptive selection caused by the memristive hardware perturbation, and evaluate and perform survival of the fittest on the emotion category corresponding to the EEG signal through the optimization strategy to screen out the optimal emotion category recognition result of the EEG signal;

[0011] Feed back the optimal emotion category recognition result of the EEG signal to the local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture to form a closed-loop process of local learning and global evolution collaboration iteration.

[0012] According to some embodiments of the present invention, the hierarchical perception mechanism includes four convolutional layers, a pooling layer, three fully connected layers, and an output layer. The extracting multi-level spatial features of the frequency-time two-dimensional feature map through the hierarchical perception mechanism and determining the emotion category corresponding to the EEG signal according to the spatial features includes:

[0013] Perform convolutional stacking processing on the frequency-time two-dimensional feature map through four convolutional layers to perform feature learning and feature extraction on the frequency-time two-dimensional feature map and obtain convolutional features;

[0014] Input the frequency-time two-dimensional feature map into the pooling layer for max-pooling processing to reduce the spatial dimension;

[0015] Flatten the convolutional features and then input them into three fully connected layers in sequence. Through the connections and weight calculations between the neurons in the fully connected layers, global features are obtained, and based on the global features, the probabilities of the frequency-time two-dimensional feature map belonging to each category are determined;

[0016] Output the positive emotional state and its corresponding probability, the neutral emotional state and its corresponding probability, and the negative emotional state and its corresponding probability through the output layer.

[0017] According to some embodiments of the present invention, the method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture further includes:

[0018] Map each convolutional neural network in the local feature learning module in the horizontal dimension to a logical partition in the memristive crossbar array for performing local feature extraction and feature learning, and update the convolutional kernel weights and fully connected layer weights in the convolutional neural network based on the quantization-aware backpropagation mechanism to achieve local autonomous learning of the local feature learning module in the horizontal dimension;

[0019] Adopt a random initialization training strategy and introduce a quantization-aware training method, and introduce weight quantization constraints during the training process of the memristive two-dimensional collaborative learning architecture, realizing end-to-end training and integrating the characteristics of memristive hardware.

[0020] According to some embodiments of the present invention, the adopting a random initialization training strategy and introducing a quantization-aware training method, and introducing weight quantization constraints during the training process of the memristive two-dimensional collaborative learning architecture, realizing end-to-end training and integrating the characteristics of memristive hardware includes:

[0021] Randomly initialize the weights of the convolutional neural network;

[0022] During the training process of the memristive two-dimensional collaborative learning architecture, dynamically perform quantization processing on the weight parameters of each convolutional neural network individual using a symmetric and uniform quantization strategy;

[0023] Scale according to the maximum absolute value of the weights, round the scaled weights, and translate them to the non-negative conductance range of the memristive hardware.

[0024] According to some embodiments of the present invention, the dynamically performing quantization processing on the weight parameters of each convolutional neural network individual using a symmetric and uniform quantization strategy includes:

[0025] During the training process of the memristive two-dimensional collaborative learning architecture, apply symmetric uniform quantization to the weight parameters of each convolutional neural network individual, and the quantization process is expressed as:

[0026] ,

[0027] Among them, W i quant is the quantized weight matrix of the i-th convolutional neural network individual, and W i is the weight parameter of each convolutional neural network individual, b is the quantization bit number, and S i = max(|W i |) is the maximum absolute value of the weights in the i-th convolutional neural network individual and is used as a scaling factor;

[0028] Each frequency-time two-dimensional feature map is processed by symmetric uniform quantization, and the quantization process is expressed as:

[0029] ,

[0030] Among them, Δ is the quantization step size, Δ = max(|X|) / α, α is the semi-level parameter, and X is the frequency-time two-dimensional feature map;

[0031] Each quantized frequency-time two-dimensional feature map is processed by padding and sliding window expansion to construct an input matrix for convolution calculation. The expression of the input matrix is:

[0032] ,

[0033] Among them, X patch is the input matrix of the frequency-time two-dimensional feature map after convolution expansion, with a shape of k×M. It performs matrix multiplication operations with the convolution kernel weight matrix. is a real number matrix with a dimension of 𝑘×𝑀, indicating that all elements of the input matrix are real numbers. Unfold(·) unfolds the padded image into a two-dimensional matrix in a sliding window manner, with each column being a convolution receptive field. Pad(·) performs edge padding operations on the quantized frequency-time two-dimensional feature map to maintain or control the spatial size of the frequency-time two-dimensional feature map. X quant is the quantized frequency-time two-dimensional feature map, k is the flattened convolution kernel size, and M is the total number of sliding windows determined by the resolution of the frequency-time two-dimensional feature map.

[0034] According to some embodiments of the present invention, the physical perturbations include conductance fluctuations and programming variabilities. The population mutation mechanism is modeled based on the physical perturbations during the weight writing and reading processes of memristive hardware. The sources of variation for population individuals provided by the population mutation mechanism include:

[0035] Modeling based on the conductance fluctuations and programming variabilities that occur during the weight writing and reading processes of memristive hardware to obtain a population mutation mechanism to provide a source of variation for population individuals;

[0036] Inject additive Gaussian noise into the quantized weight matrix of the i-th convolutional neural network individual as the mutation source of the memristive hardware to obtain a mutated weight matrix after adding perturbations. The expression of the mutated weight matrix after adding perturbations is:

[0037] ,

[0038] where, W i mut-G is the mutated weight matrix after adding perturbations, W i quant is the quantized weight matrix of the i-th convolutional neural network individual, ΔW i is the perturbation caused by the memristive hardware, 𝜎 𝑡 is the standard deviation of the perturbation in the current t-th round of training, (0, σ t 2 ) represents a Gaussian distribution with a mean of 0 and a variance of 𝜎 t 2 , and 𝜎 𝑡 is the perturbation intensity;

[0039] where, the perturbation intensity is dynamically adjusted according to the exponential decay law, and the formula for the perturbation intensity is:

[0040] ,

[0041] where, 𝜎0 is the initial standard deviation of the perturbation, 𝜎 𝑇 is the final standard deviation of the perturbation, T is the total number of training rounds, and t is the current round;

[0042] Obtain the physical characteristics of the memristive hardware during the actual programming process according to the mutated weight matrix after adding perturbations.

[0043] According to some embodiments of the present invention, the adding of memristive hardware perturbations by inputting the mutation source of the population individuals into the global evolutionary optimization module in the longitudinal dimension of the memristive two-dimensional collaborative learning architecture includes:

[0044] Input the mutation source of the population individuals into the global evolutionary optimization module in the longitudinal dimension of the memristive two-dimensional collaborative learning architecture. During the weight programming stage, inject random perturbations caused by the memristive hardware into the quantized weights of each convolutional neural network individual to generate the mutated population in the current round. The mutated weight of the i-th convolutional neural network individual in the mutated population is calculated as:

[0045] ,

[0046] where, W i mut is the weight matrix of the i-th convolutional neural network individual after mutation, Wi quant is the quantized weight of the convolutional neural network individual, ⊙ is the element-wise multiplication, and M i is the mutation mask, and ΔW i is the perturbation caused by the memristive hardware;

[0047] In the current training batch (X patch , y), the cross-entropy loss is used to calculate the classification error to evaluate the performance of each mutated individual. The formula for calculating the classification error using the cross-entropy loss is:

[0048] ,

[0049] where is the loss value of the i-th convolutional neural network individual in the t-th round of training, is the cross-entropy loss function, which is used for loss evaluation in multi-classification tasks. f(·) is the forward propagation function, and X patch is the two-dimensional frequency-time feature map matrix after convolutional expansion, and y is the true label corresponding to the current training sample;

[0050] The loss value of the i-th convolutional neural network individual in the t-th round of training is encoded in the form of a one-hot vector as:

[0051] ,

[0052] where is the loss value of the i-th convolutional neural network individual in the t-th round of training, C is the number of classification categories, and y C is the one-hot label of category C, and Y i (t) is the output logits of the i-th convolutional neural network individual, and exp(·) is the exponential function, which is used for softmax calculation;

[0053] Determine the impact of the memristive hardware perturbation according to the loss value of the i-th convolutional neural network individual in the t-th round of training.

[0054] According to some embodiments of the present invention, the construction of the optimization strategy by using the individual mutation and adaptive selection caused by the memristive hardware perturbation includes:

[0055] In the current mutated population, select the convolutional neural network individual with the smallest loss value as the parent, and determine the index of the optimal individual in the current training round according to the parent. The expression of the index of the optimal individual is:

[0056] ,

[0057] where i ∗is the index of the optimal individual in the current training round, is the cross-entropy loss of the i-th convolutional neural network individual, and argmin is the index corresponding to the minimum value;

[0058] Perform local learning on the selected parent convolutional neural network individuals, and perform parameter update operations through the backpropagation mechanism. The parameter update operation expression is:

[0059] ,

[0060] where, is the weight of the parent convolutional neural network individual in the current round, 𝜂 is the learning rate, is the gradient of the corresponding loss function, is the weight update;

[0061] Synchronously copy the updated parent weights to all other convolutional neural network individuals, and initialize the population for the next training round:

[0062] .

[0063] where, is the initial weight of the i-th convolutional neural network individual in the t+1 round, is the updated weight of the parent convolutional neural network individual after the previous round of training, represents synchronous update of all convolutional neural network individuals except the parent;

[0064] Construct an optimization strategy by synchronously updating and continuously evolving all convolutional neural network individuals.

[0065] The technical solution of the present invention also relates to a computer device, including a memory and a processor. When the processor executes the computer program stored in the memory, the above method is implemented.

[0066] The technical solution of the present invention also relates to a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above method is implemented.

[0067] The processing method and device for electroencephalogram (EEG) signals based on the memristive two-dimensional collaborative learning architecture provided by the embodiments of the present invention at least have one of the following advantages or beneficial effects: Import multi-channel EEG signals from the publicly available EEG emotion dataset, and perform short-time Fourier transform on the EEG signals. The short-time Fourier transform can analyze the frequency components and time variations of the signals simultaneously, so as to convert the EEG signals from the time domain to the frequency-time two-dimensional feature representation; splice the EEG signals of different channels together to form a comprehensive feature map, retaining the information of different brain regions; divide the feature map into multiple small blocks for subsequent processing and analysis; through these preprocessing steps, a frequency-time two-dimensional feature map is constructed, providing the basic data for subsequent emotion recognition. Input the preprocessed frequency-time two-dimensional feature map into the local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture. The local feature learning module in the horizontal dimension simulates the hierarchical perception mechanism of the mammalian visual cortex based on the convolutional neural network, and gradually extracts multi-level spatial features in the feature map through the convolutional layer and the pooling layer. The convolutional neural network can automatically learn the local features in the frequency-time two-dimensional feature map, such as the distribution of frequency components, the changes in time series, etc. According to the extracted multi-level spatial features, judge the emotion category corresponding to the EEG signal (such as positive emotion, neutral emotion, negative emotion, etc.). There are physical perturbations during the process of writing and reading weights in the memristive hardware. Based on these intrinsic physical perturbations of the memristor, a population mechanism mutation is modeled. Through this perturbation mechanism, a mutation source is provided for the population individuals. The mutation source can increase the diversity of the population, providing the basis for global optimization. Input the mutation source of the population individuals into the global evolutionary optimization module in the vertical dimension of the memristive two-dimensional collaborative learning architecture, and utilize the perturbation characteristics of the memristive hardware to further increase the mutation degree of the individuals. Based on the individual mutation and adaptive selection caused by the memristive hardware perturbation, an optimization strategy is constructed. Through the optimization strategy, adaptive selection is carried out to evaluate and select the fittest for the emotion category corresponding to the EEG signal, and finally the output of the individual with high fitness is selected as the optimal emotion category recognition result. Finally, feedback the optimal emotion category recognition result to the local feature learning module in the horizontal dimension to form a closed-loop process of local learning and global evolution collaboration iteration. Through continuous iterative optimization, the accuracy and robustness of emotion recognition are improved.

[0068] In addition, some of the additional aspects and advantages of the present invention will be given in the following description, some will become obvious from the following description, or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is the overall flowchart of the processing method for EEG signals based on the memristive two-dimensional collaborative learning architecture provided by the embodiments of the present invention;

[0070] Figure 2It is a detailed flowchart of step S1100 in the method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture provided by an embodiment of the present invention;

[0071] Figure 3 It is a detailed flowchart of step S1160 in the method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture provided by an embodiment of the present invention;

[0072] Figure 4 It is a schematic diagram of the test accuracy of different training strategies provided by an embodiment of the present invention;

[0073] Figure 5 It is a schematic diagram of the training loss curve of different training strategies provided by an embodiment of the present invention;

[0074] Figure 6 It is a schematic diagram of the adaptive selection of the population-level evolutionary dynamics based on the memristive two-dimensional collaborative learning architecture provided by an embodiment of the present invention. Detailed implementation manners

[0075] The following will clearly and completely describe the concept, specific structure and technical effects generated by the present invention in combination with the embodiments and the drawings, so as to fully understand the purpose, solution and effects of the present invention.

[0076] It should be noted that, unless otherwise specified, when a certain feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. The singular forms of "a", "the" and "said" used herein are also intended to include the plural forms unless the context clearly indicates otherwise. In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the description of the present invention herein are only for the purpose of describing specific embodiments, rather than for limiting the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0077] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, without departing from the scope of the present invention, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary languages (such as "for example", "such as", etc.) provided herein is only intended to better illustrate the embodiments of the present invention, and unless otherwise required, will not impose a limitation on the scope of the present invention.

[0078] Most of the existing machine learning-based electroencephalogram (EEG) signal processing models are trained and optimized on specific datasets, lacking generalization ability for different individuals and different emotion induction conditions. When faced with new data or scenarios, the performance of the model often drops significantly, limiting the scope of its practical applications. Moreover, traditional convolutional neural network training methods are sensitive to the non-ideality of the hardware itself (such as weight quantization errors, read / write variations, conductance fluctuations, etc.). These physical layer perturbations accumulate during the training process, resulting in unstable convergence and performance degradation during model training. Although existing compensation strategies can mitigate the impact of hardware errors to a certain extent, they usually require the introduction of complex error correction circuits or algorithm modules, significantly increasing the system complexity and resource overhead. Therefore, traditional convolutional neural network training methods fail to fully utilize hardware perturbations as a positive learning driver. Most hardware-aware training methods regard hardware perturbations as negative factors and deal with them by compensation, avoidance, or masking. This approach ignores the natural randomness and variability contained in hardware perturbations and fails to exploit their potential functional value in the learning process. Therefore, the adaptability and robustness of existing methods in low-precision and dynamic perturbation environments are limited, making it difficult to support long-term stable learning in edge scenarios.

[0079] In addition, most of the existing EEG signal processing methods do not fully draw on the information processing mechanism of the brain itself. The brain is efficient and adaptive in processing information such as emotions, while traditional methods fail to effectively simulate these biological mechanisms, being limited to a single optimization dimension, that is, relying on gradient descent of a single model parameter for learning, lacking a collaborative mechanism for local learning and global adaptation, and being difficult to achieve global adaptation in a hardware environment with strong perturbations and limited resources, resulting in a large difference between the processing process and the natural processing mode of the brain and being difficult to reach the level of brain-level intelligent processing.

[0080] Based on this, the embodiments of the present invention provide a method and device for processing EEG signals based on a memristive two-dimensional collaborative learning architecture. Through a two-dimensional collaborative manner of horizontal local feature learning and vertical global evolutionary optimization, the adaptability and robustness of the convolutional neural network in the memristive hardware environment are effectively improved. Through quantization-aware training and memristive hardware conductance mapping, combined with memristive hardware intrinsic perturbation modeling and perturbation-driven evolution, a population mechanism mutation is modeled based on the memristive hardware intrinsic physical perturbation. Through this perturbation mechanism, a mutation source is provided for population individuals, and the mutation source can increase the diversity of the population, providing a basis for global optimization, and an adaptive learning process that conforms to the hardware physical characteristics and precision constraints is realized. By synchronously performing local backpropagation learning and global population evolutionary screening, the memristive two-dimensional collaborative learning architecture is promoted to achieve stable convergence and performance improvement in a dynamic perturbation environment.

[0081] Refer to Figure 1 as shown in Figure 1It is the overall flowchart of the method for processing electroencephalogram (EEG) signals based on the memristive two-dimensional collaborative learning architecture provided by the embodiments of the present invention. The method for processing EEG signals based on the memristive two-dimensional collaborative learning architecture includes, but is not limited to, steps S1000 to S1400. Specifically,

[0082] S1000: Import multi-channel EEG signals from the EEG emotion dataset, perform short-time Fourier transform, channel-wise stitching, and segment-wise chunking preprocessing on the EEG signals to construct a frequency-time two-dimensional feature map;

[0083] S1100: Input the frequency-time two-dimensional feature map into the local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture. The local feature learning module in the horizontal dimension simulates the hierarchical perception mechanism of the mammalian visual cortex based on a convolutional neural network, extracts multi-level spatial features of the frequency-time two-dimensional feature map through the hierarchical perception mechanism, and determines the emotion category corresponding to the EEG signals according to the spatial features;

[0084] S1200: Model the population mutation mechanism based on the physical perturbation during the weight writing and reading processes of memristive hardware, and provide a mutation source for population individuals through the population mutation mechanism;

[0085] S1300: Input the mutation source of population individuals into the global evolutionary optimization module in the vertical dimension of the memristive two-dimensional collaborative learning architecture to increase the memristive hardware perturbation, construct an optimization strategy by using the individual mutation and adaptive selection caused by the memristive hardware perturbation, evaluate and perform survival of the fittest on the emotion category corresponding to the EEG signals through the optimization strategy, so as to screen out the optimal emotion category recognition result of the EEG signals;

[0086] S1400: Feed back the optimal emotion category recognition result of the EEG signals to the local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture to form a closed-loop process of local learning and global evolutionary collaboration iteration.

[0087] In some embodiments of the present invention, the method for processing EEG signals based on the memristive two-dimensional collaborative learning architecture includes: reading multi-channel EEG signals from a publicly available emotion EEG signal dataset (such as the SEED dataset). The multi-channel signals reflect the neural activities of different regions of the brain of the subject during the emotion induction task, which helps to achieve accurate discrimination of the emotional state. The short-time Fourier transform can simultaneously analyze the frequency components and time variations of the signal. Performing the short-time Fourier transform on the EEG signals can convert the EEG signals from the time domain to a frequency-time two-dimensional feature representation; stitching the EEG signals of different channels together to form a comprehensive feature map, retaining the information of different brain regions; dividing the feature map into multiple small blocks for subsequent processing and analysis; through these preprocessing steps, a frequency-time two-dimensional feature map is constructed, providing the basic data for subsequent emotion recognition.

[0088] Input the preprocessed two-dimensional frequency-time feature map into the local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture. The local feature learning module in the horizontal dimension simulates the hierarchical perception mechanism of the mammalian visual cortex based on a convolutional neural network, and gradually extracts multi-level spatial features in the feature map through convolutional layers and pooling layers. The convolutional neural network can automatically learn local features in the two-dimensional frequency-time feature map, such as the distribution of frequency components, the changes in time series, etc. According to the extracted multi-level spatial features, judge the emotional category corresponding to the EEG signal (such as positive emotion, neutral emotion, negative emotion, etc.).

[0089] There are physical perturbations during the weight writing and reading processes of memristive hardware. Based on these memristive intrinsic physical perturbations, model them as a population mechanism variation. Through this perturbation mechanism, provide a variation source for population individuals. The variation source can increase the diversity of the population and provide a basis for global optimization. Input the variation source of population individuals into the global evolutionary optimization module in the vertical dimension of the memristive two-dimensional collaborative learning architecture. Utilize the perturbation characteristics of memristive hardware to further increase the degree of individual variation. Based on the individual variation and adaptive selection caused by memristive hardware perturbation, construct an optimization strategy. Through an evolutionary algorithm (such as a genetic algorithm) for adaptive selection, evaluate and select the fittest for the emotional category corresponding to the EEG signal. Finally, select the output of individuals with high fitness as the optimal emotional category recognition result. Finally, feedback the optimal emotional category recognition result to the local feature learning module in the horizontal dimension to form a closed-loop process of local learning and global evolution collaborative iteration. Through continuous iterative optimization, improve the accuracy and robustness of emotion recognition.

[0090] In one embodiment of the present invention, multi-channel electroencephalogram (EEG) signals are read from a publicly available EEG signal dataset (such as the SEED dataset). The EEG signals are first passed through a local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture to extract multi-level features. Each convolutional neural network (CNN) individual operates on memristive hardware, and memristive hardware perturbations are introduced during the process of the memristive two-dimensional collaborative learning architecture. Subsequently, the perturbed CNN individuals are subjected to performance evaluation and survival of the fittest selection through the global evolution module in the vertical dimension of the memristive two-dimensional collaborative learning architecture. The global evolution module draws on the Darwinian evolution mechanism to construct an optimization strategy based on evolutionary programming, and uses the individual mutation and adaptive selection process triggered by the memristive hardware perturbations to continuously screen out the CNN individuals with the best performance. The emotion category output by the optimal CNN individual is used as the optimal emotion category recognition result. After that, the selected optimal emotion category recognition result and the optimal CNN individual are fed back to the local learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture, forming a closed-loop process of collaborative iteration between local learning and global evolution. The memristive two-dimensional collaborative learning architecture enhances the adaptability, robustness, and energy efficiency performance of the neural network in the memristive hardware perturbation environment through the dual collaboration of horizontal local feature learning and vertical global adaptive evolution. Multiple CNN individuals in the population are collaboratively optimized during the evolution process, and the adaptive convergence of the globally optimal memristive two-dimensional collaborative learning architecture is achieved through iterative updates.

[0091] It should be noted that the memristive hardware is memristive computing-in-memory (CIM). The memristive CIM hardware has the advantages of high parallelism, low power consumption, and high integration, and is suitable for constructing the next-generation efficient and compact intelligent inference system. The method for processing EEG signals based on the memristive two-dimensional collaborative learning architecture provided in the embodiments of the present invention is a neural network training and deployment method for the memristive CIM hardware platform, belonging to the cross field of brain-like intelligence, neuromorphic computing, intelligent learning of edge computing devices, and application of cross-species bionic intelligent mechanisms. In particular, the method for processing EEG signals based on the memristive two-dimensional collaborative learning architecture proposed in the present invention is applicable to the two-dimensional collaborative adaptive learning architecture based on memristive hardware in a low-precision perturbation environment, and can improve the robustness and energy efficiency of intelligent systems on resource-constrained platforms.

[0092] Refer to Figure 2 as shown in Figure 2 is a detailed flowchart of step S1100 in the method for processing EEG signals based on the memristive two-dimensional collaborative learning architecture provided in the embodiments of the present invention, including but not limited to steps S1110 to S1140. Specifically,

[0093] Step S1110: Perform convolutional stacking processing on the frequency-time two-dimensional feature map through four convolutional layers to conduct feature learning and feature extraction on the frequency-time two-dimensional feature map, and obtain convolutional features;

[0094] Step S1120: Input the frequency-time two-dimensional feature map into a pooling layer for max pooling processing to reduce the spatial dimension;

[0095] Step S1130: Flatten the convolutional features and then input them into three fully connected layers in sequence. Through the connections and weight calculations between the neurons in the fully connected layers, obtain global features, and determine the probabilities of the frequency-time two-dimensional feature map belonging to each category according to the global features;

[0096] Step S1140: Output the positive emotional state and its corresponding probability, the neutral emotional state and its corresponding probability, and the negative emotional state and its corresponding probability through the output layer.

[0097] In some embodiments of the present invention, in the horizontal local feature learning dimension, the local feature learning module simulates the hierarchical perception mechanism of the mammalian visual cortex based on a convolutional neural network, extracts multi-level spatial features of the frequency-time two-dimensional feature map through the hierarchical perception mechanism, and determines the emotion category corresponding to the EEG signal according to the spatial features. Specifically, the hierarchical perception mechanism consists of four convolutional layers. The size of the convolutional kernel of the first layer is 5×5, the input channel is 8, and the output channel is 64. The second convolutional layer and the third convolutional layer use 4×4 convolutional kernels, and the number of output channels are 128 and 256 respectively. The fourth convolutional layer is a bottleneck convolutional layer, using a 1×1 convolutional kernel, and the output channel is 64. All convolutional layers use the ReLU activation function. Through the convolutional stacking process of the four convolutional layers on the frequency-time two-dimensional feature map, the multi-layer convolutional stacking can capture features of different scales and provide rich information for subsequent classification. After the convolutional stacking, the frequency-time two-dimensional feature map is input into the pooling layer, and a 2×2 pooling layer is applied for max-pooling operation to reduce the spatial dimension, thereby reducing the computational complexity while retaining important features. The convolutional features are flattened and then input into three fully connected layers in sequence. The dimensions of the three fully connected layers are: 960→512, 512→128, 128→3. Through the connection and weight calculation between the neurons of the fully connected layers, global features are obtained. Each fully connected layer will perform complex non-linear transformations on the input features to learn higher-level feature representations, and determine the probability that the frequency-time two-dimensional feature map belongs to each category according to the global features. The output of the last fully connected layer is a probability distribution, indicating the probability that the frequency-time two-dimensional feature map belongs to each category. Finally, the output layer outputs the corresponding three emotional states, namely the positive emotional state (positive) and its corresponding probability, the neutral emotional state (neutral) and its corresponding probability, and the negative emotional state (negative) and its corresponding probability. The Softmax activation function is used for classification prediction, and each probability value represents the confidence that the input frequency-time two-dimensional feature map belongs to a certain emotion category.

[0098] Through the combination of the convolutional layer, the pooling layer and the fully connected layer, the feature learning and emotion category recognition of the frequency-time two-dimensional feature map are realized. When processing the EEG signal emotion recognition task, it can effectively extract features and perform accurate classification, with high practicability and accuracy.

[0099] In some embodiments of the present invention, the method for processing EEG signals based on the memristive two-dimensional collaborative learning architecture further includes, but is not limited to, steps S1150 to S1160. Specifically,

[0100] S1150: Map each convolutional neural network of the local feature learning module in the horizontal dimension to a logical partition in the memristive crossbar array for performing local feature extraction and feature learning, and update the convolutional kernel weights and fully connected layer weights in the convolutional neural network based on the quantization-aware backpropagation mechanism to achieve the local autonomous learning of the local feature learning module in the horizontal dimension;

[0101] S1160: Adopt a random initialization training strategy and introduce a quantization-aware training method, and introduce weight quantization constraints during the training process of the memristive two-dimensional collaborative learning architecture to achieve end-to-end training and incorporate the characteristics of memristive hardware.

[0102] In some embodiments of the present invention, mapping each convolutional neural network of the local feature learning module in the horizontal dimension to a logical partition in the memristive crossbar array, it can be understood that

[0103] The memristive crossbar array is a structure in which multiple memristive hardware are arranged in a matrix form, which can efficiently implement matrix operations and is particularly suitable for weight storage and calculation of neural networks. Memristive hardware is a new type of non-volatile storage device with the characteristics of integrated storage and calculation. The memristive crossbar array is divided into multiple logical partitions, and each partition corresponds to a convolutional layer or a convolutional kernel for storing and calculating the weights and activation values of the convolutional layer. Utilize the integrated storage and calculation characteristics of memristive hardware to accelerate convolutional operations and improve the running efficiency of the model. In local feature extraction and learning, each logical partition is responsible for processing local feature extraction tasks to achieve multi-level feature learning of the frequency-time two-dimensional feature map. Introduce a quantization-aware mechanism during the training process to simulate the quantization characteristics of memristive hardware in practice. Quantization-aware training makes the model adapt to the hardware limitations during training by simulating this quantization characteristic during the training stage. During the backpropagation process, the gradient is quantized and the quantized weights are updated at the same time. This mechanism ensures that the actual performance of the model on memristive hardware is consistent with its performance during the training stage. Through quantization-aware training, it can better adapt to the characteristics of memristive hardware and reduce the performance loss during the implementation of memristive hardware; and each logical partition can independently perform feature learning and parameter update to achieve local autonomous learning.

[0104] At the beginning of training, adopt a random initialization training strategy to randomly initialize the weights of the convolutional neural network. Random initialization can break the symmetry of the weights and avoid all neurons learning the same features in the initial stage of training. Reasonable random initialization can accelerate the convergence speed of the convolutional neural network and learn richer feature representations.

[0105] During the training process, the weights of the convolutional neural network are quantized to conform to the storage and computing characteristics of memristive hardware. The weights of memristors can usually only be stored with limited precision, so it is necessary to quantize the weights to simulate the characteristics of this hardware. By introducing weight quantization constraints in the training stage, the characteristics of memristive hardware are considered during the training stage, making full use of the integrated storage and computing characteristics of memristors, while adapting to the quantization limitations of the hardware, achieving end-to-end optimization from training to hardware deployment, and reducing the optimization work during the implementation of memristive hardware. Through the integration of memristive hardware characteristics, the operating efficiency and accuracy of the convolutional neural network on memristive hardware are improved.

[0106] Map the convolutional neural network onto the memristive crossbar array, and update the trainable parameters such as the convolutional kernel weights and fully connected layer weights in the convolutional neural network through the quantization-aware backpropagation mechanism to achieve local autonomous learning, making full use of the advantages of memristive hardware. At the same time, through quantization-aware training and weight quantization constraints, ensure the efficient implementation and performance optimization of the convolutional neural network on memristive hardware.

[0107] Refer to Figure 3 as shown Figure 3 is the detailed flowchart of step S1160 in the method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture provided by the embodiment of the present invention. Step S1160 includes but is not limited to steps S1161 to S1163. Specifically,

[0108] S1161: Randomly initialize and generate the weights of the convolutional neural network;

[0109] S1162: During the training process of the memristive two-dimensional collaborative learning architecture, dynamically quantize the weight parameters of each convolutional neural network individual using a symmetric and uniform quantization strategy;

[0110] S1163: Scale according to the maximum absolute value of the weights, round the scaled weights, and translate them to the non-negative conductance range of the memristive hardware.

[0111] In some embodiments of the present invention, the design of the weight quantization and mapping process of the convolutional neural network includes: First, the weights of the convolutional neural network are generated by random initialization at the beginning of training and do not rely on any pre-trained model. During the training process of the memristive two-dimensional collaborative learning architecture, a symmetric and uniform quantization strategy is adopted to dynamically quantize the weight parameters of each individual convolutional neural network, that is, scale according to the maximum absolute value of the weights, then round the scaled weights, and translate them to the non-negative conductance range of the memristive hardware. This quantization process is continuously applied during training, enabling the convolutional neural network to adapt to quantization errors in real time and enhancing its robustness under memristive hardware deployment. The memristive hardware adopted in the embodiments of the present invention is based on the TiN / HfOx / TaOy / TiN material system and is fabricated through a CMOS-compatible process. This material system is widely used in experimental-level memristive hardware.

[0112] Based on the above weight quantization and mapping of the convolutional neural network, in the horizontal dimension of the two-dimensional collaborative learning architecture proposed in the embodiments of the present invention, each individual convolutional neural network is deployed on a memristive crossbar array partition with a size of 128×576. For larger convolutional neural networks, multiple memristive crossbar arrays can be occupied to accommodate all weight parameters. To adapt to the limited storage precision of the memristor, symmetric and uniform quantization is applied to the weight parameters of each individual convolutional neural network during the training process.

[0113] The process of dynamically quantizing the weight parameters of each individual convolutional neural network by adopting a symmetric and uniform quantization strategy in step S1162 includes but is not limited to the following steps:

[0114] During the training process of the memristive two-dimensional collaborative learning architecture, symmetric and uniform quantization is applied to the weight parameters of each individual convolutional neural network. The quantization process is expressed as:

[0115] ,

[0116] where, W i quant is the quantized weight matrix of the i-th individual convolutional neural network, W i is the weight parameter of each individual convolutional neural network, b is the number of quantization bits, and S i =max(|W i |) is the maximum absolute value of the weights in the i-th individual convolutional neural network, which is used as the scaling factor;

[0117] Each frequency-time two-dimensional feature map is processed by a symmetric and uniform quantization method. The quantization process is expressed as:

[0118] ,

[0119] where Δ is the quantization step size, Δ = max(|X|) / α, α is the half-level parameter, and X is the two-dimensional frequency-time feature map;

[0120] In the embodiments of the present invention, the memristive crossbar array supports 4-bit (15-level) storage precision, achieving a good balance among programming stability, energy efficiency, and requirements for brain-inspired computing. The present invention adopts 15-level conductance precision to reflect the actual capabilities of mainstream memristive crossbar arrays and verify the effectiveness and robustness of the proposed two-dimensional collaborative learning architecture under realistic hardware constraints.

[0121] It should be noted that b represents the number of quantization bits, which is set to b = 4 in the embodiments of the present invention, corresponding to 15-level conductance precision; Δ is the quantization step size, and its value is determined by the half-level parameter α, i.e., Δ = max(|X|) / α; in the embodiments of the present invention, the input two-dimensional frequency-time feature map is uniformly quantized to 4-bit precision, corresponding to α = 4.

[0122] Each quantized two-dimensional frequency-time feature map is processed by padding and sliding window unfolding to construct an input matrix for convolution calculation. The expression of the input matrix is:

[0123] ,

[0124] where X patch is the input matrix of the two-dimensional frequency-time feature map after convolution unfolding, with a shape of k × M, and is used for matrix multiplication operation with the convolution kernel weight matrix. is a real matrix with a dimension of 𝑘 × 𝑀, indicating that all elements of the input matrix are real numbers. Unfold(·) unfolds the padded image into a two-dimensional matrix in a sliding window manner, with each column being a convolution receptive field. Pad(·) performs edge padding on the quantized two-dimensional frequency-time feature map to maintain or control the spatial size of the two-dimensional frequency-time feature map. Xquant is the quantized two-dimensional frequency-time feature map, k is the flattened convolution kernel size, and M is the total number of sliding windows determined by the resolution of the two-dimensional frequency-time feature map. For example, when the size of the input two-dimensional frequency-time feature map image is 28 × 28, zero padding is used for the padding operation to maintain the spatial resolution. The convolution kernel size is set to k = 9 (corresponding to a 3 × 3 convolution kernel), resulting in M = 784 sliding windows. The above parameter configuration can be flexibly adjusted according to different datasets or task characteristics.

[0125] In some embodiments of the present invention, the method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture further includes: deploying multiple layers of convolution calculations on the memristive crossbar array. Specifically,

[0126] The first layer of convolution calculation deployed on the memristive crossbar array is expressed as:

[0127] ,

[0128] Among them, f ReLU (·) is the ReLU activation function, defined as f(x) = max(0, x), which is applied element-wise to the multiplication result to introduce non-linearity. G1 ⊤ represents the transpose of the weight matrix of the first convolutional layer and is used for matrix multiplication in the memristive crossbar array. Similarly, the second convolutional layer is calculated as follows:

[0129] ,

[0130] Among them, G2 ⊤ represents the transpose of the weight matrix of the second convolutional layer and is also used for matrix multiplication in the memristive crossbar array. For the fully connected layer part, after the output of the convolutional layer is flattened, the calculation process is as follows:

[0131] ,

[0132] ,

[0133] Among them, Y fc1 is the output feature vector of the first fully connected layer, Y fc2 is the output feature vector of the second fully connected layer, G fc1 ⊤ is the transpose of the weight matrix of the first fully connected layer, G fc2 ⊤ is the transpose of the weight matrix of the second fully connected layer, f Softmax is the Softmax activation function, which is used to normalize the Logits vector into a probability distribution. Taking the deployment example of the FashionMNIST dataset as an example, the first convolutional layer and the second convolutional layer are respectively configured with 20 and 40 output channels, the convolutional kernel size is 3×3, and the stride is 1. The fully connected layer maps 320 input neurons to 100 hidden neurons, and finally outputs neurons corresponding to 10 categories, matching the 10-classification task requirements of FashionMNIST. The weights of all convolutional layers and fully connected layers are uniformly quantized to 4-bit precision.

[0134] During the training of each convolutional neural network individual, its scaling factor Si is dynamically updated according to the maximum value of the current weights. The number of convolutional layers, the number of neurons in the fully connected layer, and its structural parameters can be flexibly configured according to the task scenario. This hierarchical mapping and quantization strategy ensures the effective deployment of the complete convolutional neural network on the memristive crossbar array, and at the same time embeds hardware characteristic constraints during the training process, improving the final inference performance and hardware friendliness.

[0135] In some embodiments of the present invention, during the hardware deployment of the memristive two-dimensional collaborative learning architecture, physical layer errors such as conductance fluctuations and programming variability will inevitably occur in the memristive hardware during the weight writing and reading phases. These inherent physical perturbations caused by the hardware are usually regarded as performance bottlenecks in hardware deployment. However, the embodiments of the present invention innovatively introduce these inherent physical perturbations into the longitudinal evolutionary learning dimension and model them as injected perturbations of individual parameters. Specifically, the random perturbations generated during the weight programming process of the memristive hardware are regarded as the source driving individual-level mutations, thereby promoting population diversity and adaptive evolution in the entire global learning process.

[0136] In step S1200 of the method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture, a population mutation mechanism is obtained by modeling the physical perturbations during the weight writing and reading processes of the memristive hardware, and a mutation source for population individuals is provided through the population mutation mechanism, including but not limited to the following steps:

[0137] First, a population mutation mechanism is obtained by modeling the conductance fluctuations and programming variability that occur during the weight writing and reading processes of the memristive hardware to provide a mutation source for population individuals;

[0138] Additive Gaussian noise is injected into the quantized weight matrix of the i-th convolutional neural network individual as the mutation source of the memristive hardware to obtain a mutated weight matrix after adding perturbations. The expression of the mutated weight matrix after adding perturbations is:

[0139] ,

[0140] where, W i mut-G is the mutated weight matrix after adding perturbations, W i quant is the quantized weight matrix of the i-th convolutional neural network individual, ΔW i is the perturbation caused by the memristive hardware, 𝜎 𝑡 is the standard deviation of the perturbation in the current t-th round of training, (0, σ t 2 ) represents a Gaussian distribution with a mean of 0 and a variance of 𝜎 t 2 , 𝜎 𝑡 is the perturbation intensity;

[0141] Among them, the perturbation intensity is dynamically adjusted according to the exponential decay law, and the formula for the perturbation intensity is:

[0142] ,

[0143] where, 𝜎0 is the initial standard deviation of the perturbation, 𝜎 𝑇is the final perturbation standard deviation, T is the total number of training rounds, and t is the current round;

[0144] The physical characteristics of the memristive hardware in the actual programming process are obtained according to the mutated weight matrix after adding perturbations.

[0145] This dynamic decay strategy reflects the physical characteristics of the memristive hardware in the actual programming process. That is, in the initial programming stage, due to the instability of the conductive channel formation or the large matching error of the memristive hardware, the write-read variability is relatively strong; as the number of programming times increases, the conductance state of the memristive hardware gradually stabilizes. For example, the initial perturbation intensity is set to 𝜎0 = 0.1, and finally decays to 𝜎 𝑇 = 2×10 −5 , which is determined according to the actual measurement results of the memristive hardware.

[0146] In the memristive two-dimensional collaborative learning architecture proposed in the embodiments of the present invention, the local feature learning process and the global adaptive evolutionary learning process are carried out synchronously. Specifically, in the horizontal dimension (local feature learning), multiple convolutional neural network individuals are respectively deployed on a memristive crossbar array partition with a size of 128×576 for feature extraction and local learning. In the vertical dimension (global evolutionary optimization), all convolutional neural network individuals form a population to execute the evolutionary learning process. For a larger convolutional neural network, multiple memristive crossbar arrays can be jointly deployed to accommodate the complete weight parameters. Therefore, based on the above perturbation modeling, the evolutionary learning process in the vertical dimension of the memristive two-dimensional collaborative learning architecture proposed in the embodiments of the present invention is iteratively executed within each training round.

[0147] In some embodiments of the present invention, in the method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture, in step S1300, inputting the mutation source of the population individuals into the global evolutionary optimization module in the vertical dimension of the memristive two-dimensional collaborative learning architecture to increase the memristive hardware perturbation includes, but is not limited to, the following steps:

[0148] Input the mutation source of the population individuals into the global evolutionary optimization module in the vertical dimension of the memristive two-dimensional collaborative learning architecture. In the weight programming stage, inject random perturbations caused by the quantized weights of each convolutional neural network individual into the memristive hardware to generate the mutated population in the current round. The weight of the i-th convolutional neural network individual after mutation in the mutated population is calculated as:

[0149] ,

[0150] Among them, W i mut is the weight matrix of the i-th convolutional neural network individual after mutation, W i quant is the quantized weight of the convolutional neural network individual, ⊙ is the element-wise multiplication, M iis the mutation mask, ΔW i is the perturbation caused by the memristive hardware;

[0151] In the embodiments of the present invention, M i The mutation mask follows a Bernoulli distribution Mi ∼ Bernoulli(pt), where pt is the mutation ratio of the current training round, and it also gradually decreases according to the same exponential decay law as 𝜎𝑡. For example, in the FashionMNIST deployment, pt decays from the initial value of 1 to 0.001, realizing a smooth transition from high-frequency mutation to low-frequency fine optimization.

[0152] In the current training batch (X patch , y), the cross-entropy loss is used to calculate the classification error to evaluate the performance of each mutated individual. The formula for calculating the classification error using the cross-entropy loss is:

[0153] ,

[0154] where, is the loss value of the i-th convolutional neural network individual in the t-th round of training, is the cross-entropy loss function, used for loss evaluation in multi-classification tasks, f(·) is the forward propagation function, X patch is the two-dimensional frequency-time feature map matrix after convolutional expansion (a set of patches formed by a sliding window), and y is the true label corresponding to the current training sample;

[0155] The loss value of the i-th convolutional neural network individual in the t-th round of training is encoded in the form of a one-hot vector as:

[0156] ,

[0157] where, is the loss value of the i-th convolutional neural network individual in the t-th round of training, C is the number of classification categories (in FashionMNIST, C = 10), y C is the one-hot label of category C (i.e., the C-th category is 1 and the rest are 0), Y i (t) is the output logits of the i-th convolutional neural network individual, and exp(·) is the exponential function, used for softmax calculation;

[0158] Determine the influence of the memristive hardware perturbation according to the loss value of the i-th convolutional neural network individual in the t-th round of training.

[0159] In some embodiments of the present invention, in the method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture, step S1300 further includes but is not limited to the following steps:

[0160] In the current mutant population, select the individual of the convolutional neural network with the smallest loss value as the parent. Determine the index of the optimal individual in the current training round according to the parent. The expression for the index of the optimal individual is:

[0161] ,

[0162] where i ∗ is the index of the optimal individual in the current training round, is the cross-entropy loss of the i-th convolutional neural network individual, and argmin is the index corresponding to the minimum value;

[0163] Perform local learning on the selected parent convolutional neural network individual, and perform parameter update operations through the backpropagation mechanism. The expression for the parameter update operation is:

[0164] ,

[0165] where is the weight of the parent convolutional neural network individual in the current round, 𝜂 is the learning rate, is the gradient of the corresponding loss function, is the weight update;

[0166] In the embodiment of the present invention, the Adam optimizer is adopted, and the initial learning rate is 𝜂 = 10 −3 , and it is gradually decayed to the lowest 10 −5 (implemented through the ReduceLROnPlateau scheduler).

[0167] Synchronously copy the updated parent weights to all other convolutional neural network individuals, and initialize the population for the next training round:

[0168] .

[0169] where is the initial weight of the i-th convolutional neural network individual in the t+1 round, is the updated weight of the parent convolutional neural network individual after the previous round of training, represents the synchronous update of all convolutional neural network individuals except the parent;

[0170] Construct an optimization strategy by synchronously updating and continuously evolving all convolutional neural network individuals.

[0171] Based on the optimization strategy, by using the individual mutation and adaptive selection process triggered by memristive hardware perturbation, the individuals with the optimal performance are continuously selected. Multiple convolutional neural network individuals in the population are collaboratively optimized during the evolution process, and through iterative updates, the self-adaptive convergence of the global optimal model is achieved, thereby enhancing the evolutionary adaptability and resilience of the convolutional neural network in the memristive hardware perturbation environment.

[0172] In some embodiments of the present invention, in the longitudinal global evolution optimization dimension, an evolutionary population containing N = 10 convolutional neural network individuals is maintained, which is deployed to multiple memristive crossbar arrays with a size of 128×576 in the same way. The quantized weights of each convolutional neural network individual are injected with noise from memristive hardware perturbation in each round of training to achieve population-level mutation. To more realistically simulate the application scenario where the conductance state of the memristive crossbar array is unstable under frequent read and write operations, the experiments in the embodiments of the present invention adopt a stronger hardware perturbation configuration to better reflect the device fluctuation characteristics common in compact and low-power neuromorphic systems. Specifically, the perturbation intensity 𝜎 t gradually decays exponentially from the initial 0.1 to 0.001 (a total of 100 rounds of training); the mutation ratio p t also decays from the initial 1 to 0.001 to simulate the process of the memristive crossbar array gradually tending to be stable during frequent read and write processes.

[0173] The experimental design compares three different training strategies, which are specifically defined as follows.

[0174] (1) QAT (Quantization-Aware Training): The standard quantization-aware training method, where the network weights are all 4-bit precision, but no injection perturbation from hardware perturbation is introduced;

[0175] (2) QAT-Dis (Disturbance): Based on QAT, an additional perturbation caused by memristive hardware non-ideality is introduced to simulate the randomness in the memristive hardware programming process and enhance the adaptability to hardware fluctuations;

[0176] (3) QAT-Dis-2D: Based on the memristive two-dimensional collaborative learning architecture proposed in the present invention, the hardware perturbation is modeled as an evolutionary mutation source, and robust optimization is carried out under the collaborative mechanism of local feature learning and global evolutionary selection.

[0177] Refer to Figure 4 and Figure 5As shown, all three training strategies showed a convergence trend during 100 rounds of training, but there were obvious differences in convergence speed and robustness. The QAT-Dis strategy had large fluctuations during the training process due to the lack of adaptability to hardware perturbations, and finally had a low accuracy; the QAT baseline method was relatively stable during training, but got stuck in a performance bottleneck at an early stage; in contrast, the QAT-Dis-2D strategy based on the memristive two-dimensional collaborative learning architecture of the present invention showed good convergence behavior, with a smooth loss decline process and higher test accuracy, indicating that through the synergistic effect of vertical evolutionary selection and horizontal weight fine-tuning, the population can gradually absorb internal perturbations and evolve into a hardware-robust solution. In terms of quantitative indicators, the average accuracy of the QAT-Dis-2D strategy based on the memristive two-dimensional collaborative learning architecture of the present invention on the test set was 80.10%, with a standard deviation of 1.89, which was better than the QAT baseline (79.52% ± 0.95) and the QAT-Dis strategy (76.54% ± 1.68). Although its variance was slightly higher, this reflected its exploration ability under evolutionary drive and its sensitivity to hardware perturbations. Overall, the strategy based on the memristive two-dimensional collaborative learning architecture of the present invention showed good adaptability in a noisy interference and resource-constrained environment.

[0178] In the embodiments of the present invention, a visual analysis was further carried out on the population evolution process of the QAT-Dis-2D strategy based on the memristive two-dimensional collaborative learning architecture, as Figure 6 shown: In the case of memristive hardware perturbations, the early population maintained a high diversity, supporting extensive exploration; as the training rounds progressed, through mutation and screening in the vertical dimension, the best-performing individuals were identified and continuously refined through learning in the horizontal dimension, finally achieving a progressive convergence to the low-loss region. This result indicates that even in complex scenarios such as electroencephalogram signal processing, the memristive two-dimensional collaborative learning architecture of the present invention can still effectively utilize memristive hardware perturbations as an evolutionary driving factor to achieve a stable and adaptive learning process.

[0179] Therefore, the memristive two-dimensional collaborative learning architecture and its hardware deployment strategy proposed by the present invention can still achieve adaptive and robust learning in the continuous presence of hardware perturbations, demonstrating good effectiveness and potential for edge AI deployment applications, and are expected to be used in typical scenarios such as emotion-aware human-computer interaction. In summary, the embodiments of the present invention propose a method and device for processing electroencephalogram signals based on a memristive two-dimensional collaborative learning architecture, which systematically integrates the local feature perception mechanism of the mammalian visual cortex and the Darwinian evolution mechanism of biological populations. Through the two-dimensional collaborative method of horizontal local feature learning and vertical global evolutionary optimization, the adaptability and robustness of the convolutional neural network in the memristive CIM hardware environment are effectively improved. By means of quantization-aware training and memristive hardware conductance mapping, combined with memristive hardware intrinsic perturbation modeling and perturbation-driven evolution, an adaptive learning process that conforms to the physical characteristics and accuracy constraints of the hardware is constructed. Local backpropagation learning and global population evolutionary screening are carried out synchronously to promote the stable convergence and performance improvement of the memristive two-dimensional collaborative learning architecture in a dynamic perturbation environment. The embodiments of the present invention can give full play to the high parallelism and energy efficiency advantages of the memristive CIM hardware array, have good hardware adaptability, edge deployment potential and scalability of bio-inspired intelligent systems, and have good application value and popularization prospects.

[0180] It should be recognized that the method steps in the embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or computer instructions stored in a non-transitory computer-readable memory. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, for this purpose, the program can run on a dedicated integrated circuit programmed for this purpose.

[0181] In addition, the operations of the processes described herein can be performed in any suitable order, unless otherwise indicated herein or otherwise clearly contradicted by the context. The processes described herein (or variations and / or combinations thereof) can be executed under the control of one or more computer systems configured with executable instructions and can be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executed commonly on one or more processors, by hardware, or by a combination thereof. The computer program includes multiple instructions executable by one or more processors.

[0182] Further, the method may be implemented in any type of computing platform operatively connected to a suitable one, including but not limited to personal computers, minicomputers, mainframes, workstations, network or distributed computing environments, separate or integrated computer platforms, or communicating with charged particle tools or other imaging devices, etc. Aspects of the present invention may be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into the computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer and, when read by the storage medium or device, can be used to configure and operate the computer to perform the processes described herein. Additionally, the machine-readable code, or portions thereof, may be transmitted via wired or wireless networks. When such media includes instructions or programs that implement the above-described steps in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention may also include the computer itself.

[0183] A computer program can be applied to input data to perform the functions described herein, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the transformed data is a physical and tangible object, including a specific visual depiction of the physical and tangible object generated on the display.

[0184] As described above, these are only the preferred embodiments of the present invention. The present invention is not limited to the above-described embodiments. As long as the same means are used to achieve the technical effects of the present invention, any modifications, equivalent replacements, improvements, etc., made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention. Within the scope of protection of the present invention, various different modifications and variations may be made to its technical solutions and / or implementation manners.

Claims

1. A method for processing electroencephalogram signals based on a memristive two-dimensional collaborative learning architecture, characterized in that Including: Import multi-channel electroencephalogram (EEG) signals from an EEG emotion dataset, and perform short-time Fourier transform, channel-wise stitching, and segment-wise chunking preprocessing on the EEG signals to construct a frequency-time two-dimensional feature map. Input the frequency-time two-dimensional feature map into the local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture. The local feature learning module in the horizontal dimension simulates the hierarchical perception mechanism of the mammalian visual cortex based on a convolutional neural network, extracts multi-level spatial features of the frequency-time two-dimensional feature map through the hierarchical perception mechanism, and determines the emotion category corresponding to the EEG signal according to the spatial features. Model a population mutation mechanism based on the physical perturbations during the weight writing and reading processes of memristive hardware, and provide a mutation source for population individuals through the population mutation mechanism. Input the mutation source of the population individuals into the global evolutionary optimization module in the vertical dimension of the memristive two-dimensional collaborative learning architecture to increase the memristive hardware perturbation. Construct an optimization strategy using the individual mutation and adaptive selection caused by the memristive hardware perturbation, and evaluate and perform survival of the fittest on the emotion category corresponding to the EEG signal through the optimization strategy to screen out the optimal emotion category recognition result of the EEG signal. Feed back the optimal emotion category recognition result of the EEG signal to the local feature learning module in the horizontal dimension of the memristive two-dimensional collaborative learning architecture to form a closed-loop process of local learning and global evolution collaboration iteration.

2. The method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture according to claim 1, wherein, The hierarchical perception mechanism includes four convolutional layers, a pooling layer, three fully connected layers, and an output layer. The process of extracting multi-level spatial features of the frequency-time two-dimensional feature map through the hierarchical perception mechanism and determining the emotion category corresponding to the EEG signal according to the spatial features includes: Perform convolutional stacking processing on the frequency-time two-dimensional feature map through four convolutional layers to perform feature learning and feature extraction on the frequency-time two-dimensional feature map and obtain convolutional features. Input the frequency-time two-dimensional feature map into the pooling layer for max-pooling processing to reduce the spatial dimension. Flatten the convolutional features and then input them into three fully connected layers in sequence. Through the connections and weight calculations between the neurons in the fully connected layers, obtain global features, and determine the probability of the frequency-time two-dimensional feature map belonging to each category according to the global features. Output the positive emotional state and its corresponding probability, the neutral emotional state and its corresponding probability, and the negative emotional state and its corresponding probability through the output layer.

3. The method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture according to claim 1, wherein Also including: Map each convolutional neural network in the local feature learning module in the horizontal dimension to a logical partition in the memristive crossbar array for performing local feature extraction and feature learning, and update the convolutional kernel weights and fully connected layer weights in the convolutional neural network based on the quantization-aware backpropagation mechanism to achieve the local autonomous learning of the local feature learning module in the horizontal dimension. Adopt a random initialization training strategy and introduce a quantization-aware training method, and introduce weight quantization constraints during the training process of the memristive two-dimensional collaborative learning architecture to achieve end-to-end training and incorporate the characteristics of memristive hardware.

4. The method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture according to claim 3, wherein The random initialization training strategy is adopted, and the quantization-aware training method is introduced. During the training process of the memristive two-dimensional collaborative learning architecture, weight quantization constraints are introduced to achieve end-to-end training and incorporate the characteristics of memristive hardware, including: Randomly initialize the weights of the convolutional neural network; During the training process of the memristive two-dimensional collaborative learning architecture, a symmetric and uniform quantization strategy is used to dynamically quantize the weight parameters of each convolutional neural network individual; Scale according to the maximum absolute value of the weights, round the scaled weights, and translate them to the non-negative conductance range of the memristive hardware.

5. The method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture according to claim 4, wherein The use of a symmetric and uniform quantization strategy to dynamically quantize the weight parameters of each convolutional neural network individual includes: During the training process of the memristive two-dimensional collaborative learning architecture, apply symmetric uniform quantization to the weight parameters of each convolutional neural network individual. The quantization process is expressed as: , Among them, W i quant is the quantized weight matrix of the i-th convolutional neural network individual, and W i are the weight parameters of each convolutional neural network individual, b is the quantization bit number, and S i = max(|W i |) is the maximum absolute value of the weights in the i-th convolutional neural network individual, which is used as the scaling factor; Process each frequency-time two-dimensional feature map using a symmetric uniform quantization method. The quantization process is expressed as: , where Δ is the quantization step size, Δ = max(|X|) / α, α is the half-level parameter, and X is the frequency-time two-dimensional feature map; For each quantized frequency-time two-dimensional feature map, construct an input matrix for convolution calculation through padding and sliding window expansion. The expression of the input matrix is: , Among them, X patch is the input matrix of the frequency-time two-dimensional feature map after convolution expansion, with a shape of k×M, and performs matrix multiplication with the convolution kernel weight matrix as the input matrix. is a real matrix with a dimension of 𝑘×𝑀, indicating that all elements of the input matrix are real numbers. Unfold(·) unfolds the padded image into a two-dimensional matrix in a sliding window manner, with each column being a convolution receptive field. Pad(·) performs edge padding on the frequency-time two-dimensional feature map after quantization processing to maintain or control the spatial size of the frequency-time two-dimensional feature map, and X quant is the frequency-time two-dimensional feature map after quantization processing, k is the flattened convolution kernel size, and M is the total number of sliding windows determined by the resolution of the frequency-time two-dimensional feature map.

6. The method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture according to claim 5, characterized in that, The physical perturbations include conductance fluctuations and programming variability. Based on the physical perturbations during the weight writing and reading processes of the memristive hardware, a population mutation mechanism is modeled. The population mutation mechanism provides a source of variation for population individuals, including: Model the conductance fluctuations and programming variability that occur during the weight writing and reading processes of the memristive hardware to obtain a population mutation mechanism to provide a source of variation for population individuals; Inject additive Gaussian noise into the quantized weight matrix of the i-th convolutional neural network individual as the source of variation of the memristive hardware to obtain a perturbed mutated weight matrix. The expression of the perturbed mutated weight matrix is: , Among them, W i mut-G is the mutated weight matrix after adding perturbations, and W i quant is the quantized weight matrix of the i-th convolutional neural network individual. ΔW i is the perturbation caused by memristive hardware, and 𝜎 𝑡 is the standard deviation of the perturbation in the current t-th round of training. (0, σ t 2 ) represents a Gaussian distribution with a mean of 0 and a variance of 𝜎 t 2 , and 𝜎 𝑡 is the perturbation intensity. where the perturbation intensity is dynamically adjusted according to the exponential decay law. The formula for the perturbation intensity is: , Among them, 𝜎0 is the standard deviation of the initial perturbation, 𝜎 𝑇 is the standard deviation of the final perturbation, T is the total number of training rounds, and t is the current round; Obtain the physical characteristics of the memristive hardware during the actual programming process based on the perturbed mutated weight matrix.

7. The method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture according to claim 1, wherein The input of the source of variation of the population individuals into the global evolutionary optimization module in the longitudinal dimension of the memristive two-dimensional collaborative learning architecture to increase memristive hardware perturbations includes: Input the source of variation of the population individuals into the global evolutionary optimization module in the longitudinal dimension of the memristive two-dimensional collaborative learning architecture. During the weight programming stage, inject random perturbations caused by the memristive hardware into the quantized weights of each convolutional neural network individual to generate a mutated population in the current round. The mutated weight of the i-th convolutional neural network individual in the mutated population is calculated as: , Among them, W i mut is the weight matrix of the i-th individual of the convolutional neural network after mutation, and W i quant is the quantized weight of the convolutional neural network individual, ⊙ is the element-wise multiplication, and M i is the mutation mask, and ΔW i is the perturbation caused by the memristive hardware; On the current training batch (X patch , y), the cross-entropy loss is used to calculate the classification error for performance evaluation of each mutant individual. The formula for calculating the classification error using the cross-entropy loss is as follows: , Among them, is the loss value of the i-th convolutional neural network individual in the t-th round of training, is the cross-entropy loss function, used for loss evaluation in multi-classification tasks, f(·) is the forward propagation function, and X patch is the two-dimensional frequency-time feature map matrix after convolutional expansion, and y is the true label corresponding to the current training sample; Encode the loss value of the i-th convolutional neural network individual in the t-th round of training in the form of a one-hot vector as: , Among them, is the loss value of the i-th convolutional neural network individual in the t-th round of training, C is the number of classification categories, and y C is the one-hot label of category C, and Y i (t) is the output logits of the i-th convolutional neural network individual, and exp(·) is the exponential function used for softmax calculation; Determine the impact of the memristive hardware perturbations based on the loss value of the i-th convolutional neural network individual in the t-th round of training.

8. The method for processing electroencephalogram signals based on the memristive two-dimensional collaborative learning architecture according to claim 7, characterized in that The construction of an optimization strategy using individual mutation and adaptive selection triggered by the memristive hardware perturbations includes: In the current mutated population, select the individual of the convolutional neural network with the smallest loss value as the parent, and determine the index of the optimal individual in the current training round according to the parent. The expression for the index of the optimal individual is: ， where \(i^*\) is the index of the optimal individual in the current training round, is the cross-entropy loss of the \(i\)-th convolutional neural network individual, and argmin is the index corresponding to the minimum value; Perform local learning on the selected parent convolutional neural network individual, and perform parameter update operations through the backpropagation mechanism. The expression for the parameter update operation is: , Among them, is the weight of the parent convolutional neural network individual in the current round, 𝜂 is the learning rate, is the gradient of the corresponding loss function, is the weight update; Synchronously copy the updated parent weights to all other convolutional neural network individuals, and initialize the population for the next training round: . Among them, is the initial weight of the i-th convolutional neural network individual in the (t + 1)-th round, is the updated weight of the parent convolutional neural network individual after the previous round of training, indicates the synchronous update of all convolutional neural network individuals except the parent. Construct an optimization strategy by synchronously updating and continuously evolving all convolutional neural network individuals.

9. A computer device, comprising a memory and a processor, characterized in that, When the processor executes the computer program stored in the memory, it implements the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, it implements the method according to any one of claims 1 to 8.

Images