Brainwave imagined digital recognition and identity authentication method and system based on Transformer autoencoder

Abstract

The present invention relates to a method and system for brain wave imaginary digital recognition and identity authentication based on Transformer autoencoder, and belongs to the field of data information processing technology. The scheme mainly includes three stages: information extraction, basic identity authentication and digital recognition. This scheme authenticates users through complex EEG signals, avoiding the problems of easy leakage and easy copying in traditional password systems or traditional physiological signal authentication systems. At the same time, it combines basic identity authentication and digital decoding to further improve the robustness and security of the entire authentication system. In addition, this scheme solves the discrete EEG decoding problem well by combining Transformer autoencoder with mask modeling, and provides a new paradigm for more EEG decoding tasks. In addition, the pre-trained model is used in this scheme for universal representation of EEG features, forming a universal solution paradigm in the field of EEG decoding, which is helpful to build a universal large model of EEG; at the same time, the registration cost of new users is low, and it has good commercial value.

Description

Technical Field

[0001] The present invention belongs to the technical field of data information processing, and relates to a method and system for brain wave imagined digital recognition and identity authentication based on a Transformer autoencoder. Background Art

[0002] Electroencephalogram (EEG) identity authentication uses individual differences in EEG signals as the only corresponding identity feature for authentication. EEG signals meet the basic requirements of authentication. Unlike biometric features such as fingerprints and faces that are easily affected by external factors, EEG signals are relatively stable. Since EEG signals are generated by neural activity in the brain, they have unique neural pathway patterns and are difficult to forge in physiological theory. Studies have shown that different individuals produce different EEG signals through endogenous spontaneous generation and exogenous stimulation-induced generation, which to a certain extent confirms the uniqueness of human EEG signals. EEG signals in a stimulated state are different from those in a normal state, that is, EEG signals can be used as a biometric to monitor abnormal states of people. When the brain is in a threatened state, it cannot produce normal EEG signals. This phenomenon can be used as an effective indicator to distinguish between voluntary and coercive behavior and to avoid threatening intrusion. In addition, since EEG signals can only be generated by living bodies, once the individual dies, the EEG signals will no longer exist, making them highly resistant to stress. The characteristics of EEG signals provide a theoretical basis for identity authentication systems based on EEG signals.

[0003] There are many shortcomings in the existing EEG identity authentication work: (1) It is difficult to extract steady-state features from EEG data: the original EEG signal is a composite signal with low signal-to-noise ratio and difficult blind source separation. It is difficult to separate task-related features from the composite signal, and the high randomness of the noise can easily cause misjudgment; (2) The task is fixed and the user is prone to fatigue: most EEG authentication work requires the subject to complete a fixed task or receive specific stimulation, which can easily make the user resistant and change the EEG expression trend, reducing the overall security and robustness of the authentication system; (3) The user's physiological privacy is leaked: in the traditional physiological authentication system, once the data is collected and trained, it is difficult for the user to completely revoke it. The user's physiological privacy will be saved by the model, resulting in information leakage. In general, the composition of EEG signals is complex, and traditional neural networks can only express their characteristics on a single task. The emergence of general large models, attention mechanisms and other technologies has made EEG identity authentication have a better generalization trend. Summary of the invention

[0004] In view of this, in order to solve the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide a method and system for EEG imagination digital recognition and identity authentication based on Transformer autoencoder. This method and system can replace the traditional fixed generation paradigm so that the authenticator can independently choose the combination of EEG tasks, thereby improving the independence and security of the authentication system. A deep learning model framework is constructed by combining a large model decoding and encoding architecture to achieve efficient and accurate EEG feature extraction.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A brainwave imagined digital recognition and identity authentication method based on Transformer autoencoder, which is divided into three stages: information extraction, basic identity authentication and digital recognition, and specifically includes the following steps:

[0007] S1. Information extraction: high-quality imaginary digital EEG data is obtained through brain electrode positioning, data collection and data preprocessing. Then, the multi-channel and complex EEG data is compressed through Transformer autoencoder combined with mask modeling to obtain a universal EEG feature representation as a unified authentication feature;

[0008] S2. Basic identity authentication: Use a deep learning algorithm to further process the unified EEG feature representation obtained in step S1, extract user-specific features, and perform preliminary identity differentiation; use an incremental learning method for new users to add new output representations to the model;

[0009] S3, digital recognition: process the unified EEG feature representation obtained in step S1, extract similar feature information of the imagined digital task, embed the unified authentication signal into the aligned feature space, and then implement digital segmentation and complete independent digital recognition through the pre-trained model; digital decoding is combined with basic identity authentication to form a dual authentication system to improve the accuracy of identity authentication.

[0010] Further, in step S1, it specifically includes:

[0011] S11. EEG data collection: Use EEG data collection equipment to collect 14 channels of EEG data, and accurately locate the EEG of each user according to the international 10-20 standard; use the fifth-order Butterworth filter to limit the EEG signal frequency to 0.5-70hz: Filter out unnecessary low-frequency eye blinks and high-frequency electromyographic noise; then use discrete wavelet transform and inverse transform to remove Gaussian noise on different frequency sub-bands, and finally obtain high-quality, task-related EEG data;

[0012] S12. Obtain feature representation: Segment the EEG signal into multiple semantic embeddings (patch: And add category information and position information to each embedding: The spatial features of EEG data are represented by position embedding, and the category information is trained to obtain a unified feature representation of EEG. The masked embedding is randomly selected in equal proportions, and the masked embedding representation is sent to the Transformer encoder-decoder under the multi-head attention mechanism. The masked part is reconstructed through self-supervised training to obtain the intermediate vector representation cls of the EEG data. c ;

[0013] S13. Feature screening: Due to the non-stationary nature of EEG data, it is necessary to screen the intermediate vectors, reconstruct the EEG data through the intermediate vectors, and compare them with the original data using the Pearson product-moment correlation coefficient: Calculate the reconstruction quality, set the reconstruction quality threshold, and filter out the unified authentication information of the EEG signal.

[0014] Further, in step S2, it specifically includes:

[0015] S21, identity recognition: The unified EEG authentication information is classified and processed through a two-layer recurrent neural network BiRNN to further extract time domain features; the forward layer processes data in chronological order to capture information from the beginning to the current time point t 1 ; At the same time, the reverse layer processes the data in reverse time order, capturing information from the end of the sequence to the current time point t 2 ; Finally, the outputs of these two layers are combined at each time point to form a representation t that combines past and future information ～ , and finally calculate the similarity of the last layer of the network to obtain the identity category score max{s cl |c=1,2,…,N};

[0016] S22. Incremental learning: When a new user registers, the network needs to be updated through incremental learning. New users are managed through the iCaRL-like incremental representation learning method; by selecting a small number of representative samples and keeping them in memory The model can be fine-tuned on new users’ data while using knowledge distillation to retain the knowledge of old users. KD =∑ x KL(q(x)||p(x)).

[0017] Further, in step S3, it specifically includes:

[0018] S31. Feature space alignment: In the pre-training process, CLIP is trained by training a large number of digital images-single digital EEG data to achieve spatial alignment between EEG data and digital images. The loss function is defined as follows: The obtained pre-trained model completes the feature space alignment;

[0019] S32, Digital Recognition: Input the EEG unified authentication information corresponding to a single number into the CLIP model to obtain the corresponding digital score: Num i =f clip (cls), and then calculate the similarity through cosine similarity, and finally realize independent digit recognition.

[0020] The present invention also provides a brain wave imagined digital recognition and identity authentication system based on Transformer autoencoder.

[0021] The beneficial effects of the present invention are:

[0022] Due to the generalization and effectiveness of the large model encoding and decoding structure, in order to avoid the difficulty of extracting steady-state features from EEG data, this solution uses the Transformer autoencoder combined with mask modeling to extract the intermediate feature vectors of EEG data, and applies a variety of noise reduction processing methods to screen the original data to ensure the effectiveness of the steady-state features. On the other hand, high-quality steady-state features are screened out by evaluating the reconstruction effect, and then combined with the bidirectional recurrent neural network and digital recognition under the CLIP model fine-tuning to complete the user's identity authentication. Because of the flexibility of the task of imagining numbers, the leakage of user biological privacy is avoided. At the same time, the low-interaction thinking task ensures the user's attention and improves the task characteristics of the data. In addition, this solution uses a pre-trained model for universal representation of EEG features, forming a universal solution paradigm in the field of EEG decoding, which is helpful to build a universal large model of EEG waves; at the same time, the registration cost of new users is low, which has good commercial value.

[0023] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 It is a schematic diagram of the process of the present invention;

[0026] Figure 2This is a comparison chart of signal preprocessing;

[0027] Figure 3 This is the structure diagram of the Transformer autoencoder;

[0028] Figure 4 Reconstruct the effect map for the mask;

[0029] Figure 5 This is a diagram of basic identity authentication results. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0031] The present invention relates to a method and system for brain wave imaginary digital recognition and identity authentication based on Transformer autoencoder. It is mainly divided into three stages: information extraction, basic identity authentication and digital recognition. In the information extraction stage, the brain electrodes are first positioned according to the 10-20 international standard, and the collected brain wave data is preprocessed; the high-quality brain wave data is represented by the masked electroencephalogram modeling (MEM) task of the Transformer autoencoder to obtain the intermediate feature representation of the brain wave; then the feature representation is evaluated by signal reconstruction; finally, the unified authentication information of the brain wave signal is obtained. Based on the obtained authentication information, the basic identification of the user's identity is completed by a deep learning algorithm. In order to ensure the user's concentration and improve the accuracy of authentication, a dual authentication system is formed by combining a digital decoding method. In the digital decoding process, the unified authentication signal is embedded in the alignment feature space, and then the digital segmentation is realized through the pre-training model and independent digital recognition is completed. Figure 1 This is a flowchart diagram of the present invention. As shown in the figure, this solution mainly consists of information extraction and device authentication, where information extraction is further divided into feature extraction and fuzzy extraction. Specifically:

[0032] 1) Feature extraction

[0033] Since EEG signals are non-stationary and easily affected by environmental noise, traditional methods cannot well represent the general features of EEG. However, the Transformer encoder-decoder structure has strong expressive power. It uses the Masked Electroencephalogram Modeling (MEM) process to obtain a unified intermediate feature representation of EEG signals. The specific steps are as follows:

[0034] A. EEG data collection: Use EEG data collection equipment to collect 14 channels of EEG data, and accurately locate each user's EEG according to the international 10-20 standard. The collected 14*128hz (channel is:) time series data is passed through a fifth-order Butterworth filter to limit the EEG signal frequency to 0.5-70hz: Where j is the imaginary unit, ω is the angular frequency of the signal, ω c is the cutoff frequency, and N is the order of the filter, which determines the speed at which the filter attenuates near the cutoff frequency. This formula represents the gain or attenuation of the filter for signals of different frequencies. As the input signal frequency ω increases, the amplitude |H(jω)| will decrease, which means that signals above the cutoff frequency will be attenuated, and the high-pass filter will attenuate ω. c The term and the ω term are swapped. At the same time, a notch filter is used to remove the 60 Hz electrical environment noise. The EEG signal is initially de-noised through filtering operations. Unnecessary low-frequency blinks and high-frequency myoelectric noise are filtered out.

[0035] Then, the Gaussian noise on different frequency sub-bands is removed by discrete wavelet transform and inverse transform. Discrete wavelet transform (DWT) is a mathematical tool for signal and image processing. It analyzes the signal by decomposing it into a series of wavelets of different scales or resolutions. The basic idea of ​​DWT is to decompose the original signal through a series of filters. These filters are usually low-pass and high-pass filters, which are used to extract different frequency components of the signal. The filter then downsamples the output (usually halved) to obtain two sub-bands: the approximation sub-band and the detail sub-band. The approximation sub-band contains the low-frequency components of the signal, while the detail sub-band contains the high-frequency components of the signal. This process can be recursively performed multiple times to further decompose the approximation sub-band, thereby analyzing the signal at different resolution levels. The formula is as follows:

[0036]

[0037]

[0038] where x[n] is the original signal, is the scaling function associated with low-pass filtering, ψ k,n It is a wavelet function related to high-pass filtering, where k and n represent scale and position respectively.

[0039] The data after noise filtering is as follows Figure 2 As shown in the figure, the noise reduction of the AF3 channel is captured. After the signal is filtered and reconstructed by discrete wavelet, the external noise is reduced, and finally high-quality, task-related EEG data is obtained.

[0040] B. Obtain feature representation: Segment the EEG signal into multiple semantic embeddings (patch: And add category information and position information to each embedding: The spatial features of EEG data are represented by position embedding, and the category information can be trained to obtain a unified feature representation of EEG. Randomly select equal proportions of embedding masks, and send the masked embedding representation to the Transformer encoder-decoder under the multi-head attention mechanism. Through self-supervised training, the masked part is reconstructed to obtain the intermediate vector representation of EEG data cls c ;

[0041] The Transformer structure used in the whole process is as follows Figure 3 As shown in the figure, the data of four time acquisition points are divided into a patch and the channel position information and EEG signal are added to the patch for unified embedding representation. Parallel channels are regarded as the semantic representation of the same patch. 16 encoding attention heads and 16 decoding attention heads are used, and the number of recurrent layers of each head is 24. The multi-channel EEG data is uniformly encoded through the Transformer encoding and decoding structure, and the trainable linear projection is used. The patch is projected to a fixed dimension D, where the positional encoding is represented as For each head, transform the input semantics into a query matrix: Key Matrix: Value matrix: (in ), and then get the output of the attention head by scaling

[0042]

[0043] Then, a random masking strategy is used to provide task support for the Transformer autoencoder. The masking rate is adjusted to 0.7. The masking and reconstruction results are shown in the figure below. Figure 4 As shown, it can be seen that the autoencoder can complete the EEG reconstruction task, which means that the intermediate vector representation cls obtained through training can well represent the overall information of the EEG data.

[0044] C. Feature screening: Due to the non-stationary nature of EEG data, the intermediate vector cls needs to be screened, and the reconstructed EEG data is reconstructed through the intermediate vector and compared with the original data using the Pearson product-moment correlation coefficient: Calculate the reconstruction quality, set the reconstruction quality threshold similarity to 0.75, and filter out the unified authentication information CLS of the EEG signal i .

[0045] 2) Basic identity recognition

[0046] Get the unified authentication information CLS of EEG data i Afterwards, bidirectional recurrent neural network and incremental learning are used for identity authentication.

[0047] Identity recognition: BiRNN (two-layer recurrent neural network) is used to classify and process the unified EEG authentication information in order to further extract time domain features. The forward layer processes data in chronological order, capturing information from the beginning to the current time point. 1 At the same time, the reverse layer processes the data in reverse time order, capturing information from the end of the sequence to the current time point t 2 Finally, the outputs of these two layers are combined at each time point to form a representation t that combines past and future information. ～ , and finally calculate the similarity of the last layer of the network to obtain the identity category score max{s cl |c=1,2,…,N};

[0048] Incremental learning: When a new user registers, the network needs to be updated through incremental learning. New users are managed through the iCaRL (incremental representation learning) method. By selecting a small number of representative samples and keeping them in memory The model can be fine-tuned on the data of new users while using knowledge distillation to retain the knowledge of old users L KD =∑ x KL(q(x)||p(x)).

[0049] In this example, 7 subjects were continuously tested and the recognition accuracy was 92.86%. The test results are visualized as follows: Figure 5 shown.

[0050] 3) Digital recognition

[0051] After basic identity recognition, the single imaginary digital data is encoded in the same way to obtain Then input it into the CLIP pre-trained model to complete digital alignment decoding.

[0052] Feature space alignment: In the pre-training process, CLIP is trained by a large number of digital images-single digital EEG data to achieve spatial alignment between EEG data and digital images. The loss function is defined as follows: The obtained pre-trained model completes the feature space alignment;

[0053] Digital recognition: Input the EEG unified authentication information corresponding to a single number into the CLIP model to obtain the corresponding digital score: Num i =fclip (cls), then calculate the similarity through cosine similarity, and finally realize independent digit recognition;

[0054] In this example, the accuracy of digital recognition is 46.43%. The combination of basic identity authentication and digital decoding forms a closed loop system, which further improves the robustness and security of the entire authentication system. It can be seen that the EEG identity recognition system provided by the present invention can realize dual authentication, that is, dual authentication of imagined numbers and EEG identities, and can identify the same user U1 imagining different digital combinations. and Can identify different users U1 and U2 imagine the same number group Differentiate identities at different times; be able to identify the same user U1 imagining the same number group at different times similarity.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for brainwave imagined digital recognition and identity authentication based on Transformer autoencoder, characterized by: The method is divided into three stages: information extraction, basic identity authentication and digital recognition, and specifically includes the following steps: S1. Information extraction: high-quality imaginary digital EEG data is obtained through brain electrode positioning, data collection and data preprocessing. Then, the multi-channel and complex EEG data is compressed through Transformer autoencoder combined with mask modeling to obtain a universal EEG feature representation as a unified authentication feature; S2. Basic identity authentication: Use a deep learning algorithm to further process the unified authentication features obtained in step S1, extract user-specific features, and perform preliminary identity differentiation; use an incremental learning method for new users to add new output representations to the model; S3, digital recognition: the unified authentication features obtained in step S1 are processed to extract similar feature information of the imagined digital task, the unified authentication features are embedded in the aligned feature space, and then digital segmentation and independent digital recognition are achieved through the pre-trained model; digital decoding is combined with basic identity authentication to form a dual authentication system to improve the accuracy of identity authentication; In step S1, it specifically includes: S11. EEG data collection: Use EEG data collection equipment to collect 14 channels of EEG data, and accurately locate the EEG of each user according to the international 10-20 standard; use the fifth-order Butterworth filter to limit the EEG signal frequency to 0.5-70hz: Filter out unnecessary low-frequency eye blinks and high-frequency myoelectric noise, where j is the imaginary unit, ω is the angular frequency of the signal, and ω c is the cutoff frequency, and N is the order of the filter, which determines the speed at which the filter decays near the cutoff frequency. Then, the Gaussian noise on different frequency sub-bands is removed through discrete wavelet transform and inverse transform, and finally high-quality, task-related EEG data is obtained. S12. Obtain feature representation: Segment the EEG signal into multiple semantic embeddings: where c ij represents the jth channel in the i-th data, k is the representation of the time block on each channel, t represents the length of single-channel data in a single experiment, and w is the partition embedding length; then add category information and position information to each embedding: Among them ij Represents the position embedding of each channel; the spatial features of EEG data are represented by position embedding, and the category information is trained to obtain a unified feature representation of EEG; randomly select equal proportions of embedding masks, send the masked embedding representation to the Transformer encoder-decoder under the multi-head attention mechanism, and reconstruct the masked part through self-supervised training to obtain the intermediate vector representation cls of EEG data c ; S13. Feature screening: Due to the non-stationary nature of EEG data, it is necessary to screen the intermediate vectors, reconstruct the EEG data through the intermediate vectors, and compare them with the original data using the Pearson product-moment correlation coefficient: Where X i and Y i represents the traversal observation value, and Represent the sample average, calculate the reconstruction quality, set the reconstruction quality threshold, and filter out the unified authentication information of the EEG signal; In step S2, it specifically includes: S21, Identity Recognition: The unified authentication features are classified and processed through a two-layer recurrent neural network BiRNN to further extract time domain features; the forward layer processes data in chronological order to capture information from the beginning to the current time point t 1 ; At the same time, the reverse layer processes the data in reverse time order, capturing information from the end of the sequence to the current time point t 2 ; Finally, the outputs of these two layers are combined at each time point to form a representation t that combines past and future information ～ , and finally calculate the similarity of the last layer of the network to obtain the identity category score max{s cl |cl=1,2,…,N}, where cl represents the number of identities, s cl represents the calculated score of the corresponding identity; S22. Incremental learning: When a new user registers, the network needs to be updated through incremental learning. New users are managed through the iCaRL-like incremental representation learning method; by selecting a small number of representative samples and keeping them in memory The model can be fine-tuned on new users’ data while using knowledge distillation to retain the knowledge of old users. KD =∑ i KL(q(x i )||p(x i )), where U c' Represents the new class update parameter form, RNN is a recurrent neural network, L KD represents the knowledge distillation loss, KL(q(x i )||p(x i )) represents the Courbecquer-Leibler divergence of the i-th data, q(x i ) represents the teacher model output, p(x i ) represents the student model output; In step S3, it specifically includes: S31. Feature space alignment: In the pre-training process, CLIP is trained by training a large number of digital images-single digital EEG data to achieve spatial alignment between EEG data and digital images. The loss function is defined as follows: Where L clip is the loss function expression of the CLIP model, E I (I) represents the encoded feature vector of image I, h(τ θ (y)) represents the text encoding vector, τ θ It is the text embedding process, maximizing the cosine similarity between the image representation and its corresponding text representation. The final pre-trained model completes the feature space alignment. S32, Digital Recognition: Input the EEG unified authentication information corresponding to a single number into the CLIP model to obtain the corresponding digital score: Num iz =f clip (cls c ), where Num iz represents the alignment score of a single number in each data, f clip It represents the CLIP alignment process, and then calculates the similarity through cosine similarity, and finally realizes independent digit recognition.

2. A brainwave imagined digital recognition and identity authentication system based on Transformer autoencoder, characterized by: The system adopts the method as claimed in claim 1.

Images