Multi-mode intelligent massage chair system based on SSVEP and MI

Abstract

The invention discloses a multi-mode intelligent massage chair system based on SSVEP and MI, and the system comprises an operation interface module which comprises an SSVEP stimulation interface of an SSVEP stimulation normal form and an MI stimulation interface of an MI stimulation normal form; the data acquisition module is used for acquiring original electroencephalogram signals generated when the user uses the operation interface module; the data processing and analysis module is used for processing and analyzing the original electroencephalogram signals acquired by the data acquisition module to obtain SSVEP stimulation and MI stimulation recognition results of the original electroencephalogram signals; and the actuator module is used for receiving the recognition result obtained by the data processing and analyzing module and converting the recognition result into a control instruction which can be recognized by the massage chair, so that different massage functions are realized. According to the invention, non-contact control of a user on the massage chair is realized, the operation problem that two arms are limited during massage is solved, convenient autonomous massage nursing is provided for patients suffering from gradual freezing and high paraplegia, and the applicability and user experience of the massage chair are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of brain-computer interface, and in particular to a multi-modal intelligent massage chair system based on SSVEP and MI. Background Art

[0002] Patients who are bedridden for a long time and have difficulty in moving, especially those with ALS and high-level paralysis, often face extremely limited mobility, and in severe cases, they may even be unable to move their limbs. These patients need regular rehabilitation massage to prevent muscle stiffness and atrophy, but the economic and time costs of hiring professional medical staff are high, making it difficult to fully meet their rehabilitation training needs. Therefore, developing a system that can help these patients achieve autonomous massage care has become an urgent problem to be solved in the field of rehabilitation medicine.

[0003] For general users, most traditional massage chairs on the market are controlled by handles, requiring users to manually operate buttons to start or stop massage. Moreover, during the massage, the user's arms may be restricted, affecting the control of the massage chair.

[0004] As a stable, safe and easy-to-collect bioelectric signal, EEG signals have been widely used in many fields, such as fatigue driving monitoring and brain-computer interface (BCI) technology. In BCI technology, Steady-State Visual Evoked Potential (SSVEP) and Motor Imagery (MI) are the two main control paradigms. SSVEP is the brain's response to periodic visual stimulation. By looking at a light source that flashes at a specific frequency, the brain produces synchronous potential fluctuations, which can be recorded and used to control the BCI system. The advantage of SSVEP is that it can construct a rich instruction set, but due to its synchronous nature, it is not suitable for operations that require instant response.

[0005] On the other hand, MI relies on the EEG signals generated by the user when imagining limb movements, allowing the BCI system to be directly controlled by thought, achieving instant control. However, the instruction set of MI is relatively limited because users need to learn and distinguish different motor imagination tasks, which limits its complexity in practical applications.

[0006] This paper proposes an autonomous rehabilitation training method based on EEG signals for patients who are bedridden for a long time and have difficulty in moving. This method aims to reduce the patient's dependence on professional medical staff and improve their autonomous rehabilitation ability. At the same time, it combines the advantages of the two BCI control paradigms of SSVEP and MI to realize a massage care system that not only enriches the instruction set but also provides instant control. Summary of the invention

[0007] The present invention aims to overcome the defects and limitations of the prior art and proposes a multimodal intelligent massage chair system that combines SSVEP and MI technology. The system uses EEG signals to achieve non-contact control and is particularly suitable for patients who are bedridden for a long time and have limited mobility, so that they can independently undergo rehabilitation treatment. The innovation of this system lies in the combination of SSVEP and MI technology, where SSVEP is used to select the functions of the massage chair, and MI is used to control the start and end of the selected function. Users can easily start or turn off the corresponding functions of the massage chair simply by imagining the movement of stretching their wrists with their left and right hands. This integrated method not only realizes the non-contact operation of the massage chair, but also solves the problem of being unable to operate conventional physical button handles during the massage process due to the restriction of both arms.

[0008] To achieve the above purpose, the technical solution provided by the present invention is: a multi-modal intelligent massage chair system based on SSVEP and MI, comprising:

[0009] The operation interface module includes an SSVEP stimulation interface for the SSVEP stimulation paradigm and an MI stimulation interface for the MI stimulation paradigm; in the SSVEP stimulation paradigm, different stimulation buttons are matched with visual stimulations of specific frequencies to induce EEG activities of corresponding frequencies; for the MI stimulation paradigm, the user is guided to imagine the corresponding limb movements by visually simulating the movements of the left and right arms combined with voice prompts. After hearing the voice prompts, the user needs to imagine performing the corresponding arm movements, and the arm movements displayed on the MI stimulation interface are only used as visual aids to help the user imagine the movements more accurately;

[0010] A data acquisition module, used to collect the original EEG signals generated by the user when using the operation interface module;

[0011] The data processing and analysis module is used to process and analyze the original EEG signals collected by the data acquisition module to obtain the recognition results of SSVEP stimulation and MI stimulation of the original EEG signals;

[0012] The actuator module is used to receive the recognition results obtained by the data processing and analysis module and convert them into control instructions that can be recognized by the massage chair, so as to realize different massage functions.

[0013] Furthermore, the operation interface module includes the following parts:

[0014] Basic parameter settings: includes basic parameter configuration for connection with the EEG signal acquisition headband and operation mode selection, where the operation modes are divided into training mode and use mode; the training mode means that the user needs to be trained before the first use, and the system builds a model based on the training results. When the model meets the requirements, it is automatically saved for direct use in subsequent use modes, where the model means that in the training mode, a learning model will be generated after the SSVEP EEG signal is put into the FBCCA algorithm and the MI EEG signal is put into the MultiScale-CAEEGNet neural network for training. In the use mode, you only need to input the data into the corresponding model to get the recognition result;

[0015] Massage chair function control: It includes five massage functions, namely "shoulder massage", "arm massage", "hand massage", "waist massage" and "leg massage". Each function is realized through five stimulation buttons on the SSVEP stimulation interface. Each stimulation button flashes continuously at a preset frequency. The preset frequencies corresponding to the five stimulation buttons are 7Hz, 7.5Hz, 8Hz, 8.5Hz and 9Hz respectively. When the user looks at the stimulation buttons of different frequencies, the corresponding SSVEP EEG signals will be triggered, thereby selecting different massage functions. Among them, the brightness change formula of the stimulation button is as follows:

[0016]

[0017] Where: s(f,i) represents the brightness of the stimulation button at the i-th moment with a preset frequency of f, and freshrate represents the refresh rate of the SSVEP stimulation interface;

[0018] Massage chair start and stop control: After hearing the voice prompt, the user needs to imagine performing the corresponding body movements, among which, imagining extending the left wrist represents turning on the corresponding massage chair function selected by the SSVEP EEG signal, and imagining extending the right wrist represents turning off the corresponding massage chair function selected by the SSVEP EEG signal.

[0019] Furthermore, the data acquisition module adopts a six-channel EEG signal acquisition head ring, whose sampling rate is set to 250Hz. The EEG signal acquisition head ring is equipped with eight electrodes, including a reference electrode, a ground electrode and six EEG signal channel electrodes. The ground electrode is used to set the zero potential reference point of the EEG signal, and the reference electrode and the ground electrode are both located at the temples on both sides of the head. The six EEG signal channel electrodes correspond to six channels respectively, which are used to capture the user's EEG signals. Their placement positions follow the standards of the "International 10-20 System" and are located at C3, C4, P3, P4, O1 and O2 positions. The collected EEG signals are converted into digital signals by a 24-bit resolution analog-to-digital conversion module inside the EEG signal acquisition head ring.

[0020] Further, the data processing and analysis module includes a data preprocessing submodule and a data analysis submodule;

[0021] The data preprocessing submodule includes an SSVEP data preprocessing module and an MI data preprocessing module, wherein:

[0022] The specific operation process of the SSVEP data preprocessing module is as follows:

[0023] First, baseline drift correction was performed on the EEG signals of the four channels O1, O2, P3, and P4 to eliminate DC offset. Subsequently, a notch filter was used to filter out 50 Hz power frequency interference. After that, the signals were filtered through an 11th-order Butterworth infinite impulse response (IIR) bandpass filter to retain the signal components in the frequency range of 6-30 Hz. Considering the delay of the human visual system, the start extraction time of the SSVEP EEG signal was delayed by m1 milliseconds to avoid interference from the initial signal. In the training mode and use mode of the SSVEP stimulation paradigm, the EEG data segment from m1 milliseconds to 5 seconds after the start of the stimulation was finally selected for model training and recognition analysis.

[0024] The details of the MI data preprocessing module are as follows:

[0025] First, remove the baseline drift, use the 11th-order Butterworth infinite impulse response bandpass filter to filter, retain the 8-30HZ signal component, and then extract the 0-5s signal segment from the beginning of the MI paradigm stimulation and put it into the MultiScale-CAEEGNet neural network to complete the recognition;

[0026] The data analysis submodule includes an SSVEP data analysis module and an MI data analysis module, wherein: the specific details of the SSVEP data analysis module are as follows:

[0027] The FBCCA algorithm is used to identify EEG signals. FBCCA is an algorithm used for SSVEP signal recognition. Its function is to identify the user's intention by extracting EEG signal features of specific frequencies. The principle is to use a filter bank to decompose the original EEG signal into multiple frequency bands, and apply canonical correlation analysis (CCA) to each frequency band to calculate the correlation between the signal and the reference signal, so as to find the frequency with the largest correlation among multiple target frequencies and realize accurate target recognition.

[0028] The details of the MI data analysis module are as follows:

[0029] Construct the user's MI paradigm model, specifically, construct a model of two states: imagining the left wrist extending and imagining the right wrist extending after the SSVEP stimulus flashes, collect the user's EEG signals in the above two states, and put the collected EEG signals into the MultiScale-CAEEGNet neural network for recognition.

[0030] Furthermore, the MultiScale-CAEEGNet neural network is a model proposed based on the EEGNet neural network, that is, an improved EEGNet neural network, which introduces data enhancement and regularization techniques on the basis of the EEGNet neural network, optimizes the depth-separable convolution module in the EEGNet neural network into a multi-scale convolution module, and adds a hole convolution module and a channel attention mechanism. The details are as follows:

[0031] The data enhancement optimizes the input data, simulates more sample distributions, and prevents overfitting, specifically by generating diversified inputs by randomly shifting data in the time dimension;

[0032] The regularization adopts L2 regularization, adds L2 regularization terms to constrain the model complexity, and adjusts the inactivation ratio in each layer of random inactivation to prevent overfitting;

[0033] The multi-scale convolution module uses depth-wise separable convolutional layer operations with different convolution kernel sizes to capture information at different time scales, wherein shorter convolution kernels help identify short-term fluctuations, while longer convolution kernels help capture long-term trends;

[0034] The dilated convolution module expands the receptive field while maintaining the resolution of the feature map by setting an appropriate dilation rate. Since no additional parameters are introduced, the dilated convolution module effectively reduces the computational complexity while extracting a wider range of contextual information. In addition, with its sensitivity to the global context, the dilated convolution module can help the model capture signal features more comprehensively, thereby improving the performance of the task.

[0035] The channel attention mechanism assigns different weights to each convolution channel, helping the model to automatically focus on more important frequency information or specific signal channels. By adaptively weighting different channels, the model can more effectively ignore irrelevant noise signals while amplifying key frequency bands that contribute to the task, thereby improving model performance.

[0036] The third convolutional layer, i.e. convolutional layer 3, in the EEGNet neural network is replaced by three convolutional layers, namely convolutional layer 3-1, convolutional layer 3-2, and convolutional layer 3-3. The convolution kernel sizes of the three convolutional layers are 8, 16, and 32, respectively. The convolution kernel of size 8 is a shorter convolution kernel, which helps to identify short-term fluctuations; the convolution kernel of size 16 is the original convolution kernel; the convolution kernel of size 32 is a longer convolution kernel, which helps to capture long-term trends.

[0037] After the multi-scale convolution module, dilated convolution modules with dilation rates of 1, 2, and 4 are added to realize the extraction of multi-scale features.

[0038] The role of the channel attention mechanism is to automatically learn the importance of each channel according to the characteristics of the input data, and adjust the contribution of each channel by weight. It summarizes each channel through global average pooling, and then uses two fully connected layers to calculate the attention weight of each channel. It then strengthens the features of important channels by weighting each channel, while suppressing unimportant channels. This mechanism enables the model to adaptively adjust the influence of different channels, thereby improving the model's sensitivity to key information.

[0039] Furthermore, the format of the input data X of the MultiScale-CAEEGNet neural network is: X∈R B×C×T , where B is the batch size, C is the number of channels, T is the number of time points, and R is the real number domain; in order to increase the diversity of training data and the robustness of the model, data augmentation with random displacement in the time dimension is used to provide the model with a richer input distribution;

[0040] X'=Roll(X,Δt)

[0041] Where: Δt is the random offset; Roll(X,Δt) means periodically shifting the input data X in the time dimension to obtain richer input data X'; if the signal is shifted in the time dimension, assuming Δt is a positive value, the time points in the signal will be shifted backward by Δt positions, and the parts that exceed the range will reappear from the front.

[0042] Furthermore, the multi-scale convolution module uses a depth-separable convolution layer with large and small convolution kernels of k1=8, k2=16, and k3=32 to extract features respectively, and introduces a dilated convolution module to expand the receptive field of the model and enhance the ability to capture long-distance dependencies. That is, a dilated convolution module with dilated rates of 1, 2, and 4 is added after the multi-scale convolution module to realize the extraction of multi-scale features, and the features of three scales are obtained, namely, H scale1 , H scale2 , H scale3 , where H scale1 , H scale2 , Hscale3 It refers to the output features of the input data X′ after it passes through the first convolution layer of the original EEGNet neural network, namely convolution layer 1, the first one-dimensional batch normalization layer, namely one-dimensional batch normalization layer 1, the second convolution layer, namely convolution layer 2, the second one-dimensional batch normalization layer, namely one-dimensional batch normalization layer 2, the first one-dimensional average pooling layer, namely one-dimensional average pooling layer 1, the first random dropout layer 1, namely random dropout layer 1, and then passes through the multi-scale convolution module and the void convolution module.

[0043] The three scale features H scale1 , H scale2 , H scale3 Splice to feature H multi-scale :

[0044] H multi-scale =Concat(H scale1 ,H scale2 ,H scale3 )

[0045] Due to the introduction of the channel attention mechanism, the multi-scale features are optimized by dynamically adjusting the weights of the feature channels; first, global average pooling is performed to calculate the global statistics of each channel:

[0046]

[0047] Where: S c Represents the global average pooling result of the cth channel; Represents the characteristic H multi-scale The value at the tth sampling point of the cth channel;

[0048] Then, the attention weights of the channels are generated through two fully connected layers:

[0049] w c =σ(W2·ReLU(W1·S+b1)+b2)

[0050] Where: σ is the sigmoid activation function; S is a vector containing the S of all channels c Value; w c is the attention weight of the cth channel; W1 and W2 are weight matrices; b1 and b2 are bias terms; ReLU is an activation function;

[0051] Next, use the attention weight to adjust the feature and get the feature H after the channel attention mechanism attention :

[0052] H attention = w·H multi-scale

[0053] Where: w is the matrix composed of the attention weights of all channels;

[0054] Finally, the feature H attention The dimension is reduced by global average pooling, and finally, the recognition result is output using the fully connected layer and Softmax activation function.

[0055]

[0056] Where: GAP represents global average pooling, Dense represents the output processed by GAP through a fully connected layer; finally, where n classes Represents the number of identified categories;

[0057] Among them, in the process of model weight optimization, L2 regularization term is added to constrain model complexity and prevent overfitting. The regularization term is in the form of:

[0058]

[0059] Where: L is the total loss function, L CE is the cross entropy loss; λ is the regularization strength; Represents the square of the L2 norm of the weight matrix U.

[0060] Furthermore, the MultiScale-CAEEGNet neural network has 4 channels, namely C3, C4, P3, and P4, and the number of time points is 1250. It also includes the temporal convolution module and spatial convolution module of the original EEGNet neural network. The temporal convolution module uses F1 convolution kernels of size (1,64) to extract information in the time dimension, and the spatial convolution module uses D*F1 convolution kernels of size (C,1) to integrate data information from different EEG channels, where D represents the depth multiple of the deep convolution and F1 represents the number of filters in the temporal convolution module.

[0061] Furthermore, the actuator module uses the nRF52832 Bluetooth controller to receive the recognition results from the data processing and analysis module, and converts the recognition results into instructions that can be executed by the massage chair, and forwards them to the massage chair controller through the UART serial port.

[0062] Furthermore, before officially using the system, the user needs to go through the training mode first, because the EEG signals of different users vary greatly, and the training mode is needed to determine the parameters suitable for the user;

[0063] Before entering the training mode, the user needs to sit on the massage chair, wear the EEG signal collection headband, watch the operation interface module, and select the training mode;

[0064] In the training mode, the interface is the SSVEP stimulation interface, and the following process is executed: a stimulation button in the SSVEP stimulation paradigm will flash red for m2 seconds, prompting the user to concentrate on the stimulation button. After the flashing ends, all stimulation buttons in the interface will flash simultaneously for m3 seconds. During this period, the user needs to continue to stare at the previously prompted stimulation button. Each stimulation button is collected once according to the above process. After the EEG signals of the five stimulation buttons are collected, one training is completed; one round of experiment is completed after staring at the five stimulation buttons in sequence according to the flashing prompts. Five rounds of SSVEP experiments need to be completed in the training mode; after the training mode is completed, the interface switches to the MI stimulation interface, and voice prompts are played at the same time. When the voice prompts imagine extending the left or right wrist, the virtual arm in the interface will make corresponding wrist extension movements. The user needs to perform corresponding movement imagination according to the voice prompts or the movements in the interface. Each imagination collects m4 groups of data, and a total of m5 groups of movement imagination data need to be collected;

[0065] After the training mode is completed, the collected SSVEP EEG signals and MI EEG signals are input into the data processing and analysis module to generate a learning model;

[0066] Next, the user selects the usage mode in the operation interface module and enters the SSVEP stimulation interface in the usage mode. The user gazes at a stimulation button in the SSVEP stimulation interface, and the SSVEP EEG signal corresponding to the stimulation button will be input into the learning model corresponding to the SSVEP EEG signal and the recognition result will be output. The massage function corresponding to the recognition result will be selected. After completing the function selection, if the user imagines extending the left wrist, the corresponding MI EEG signal will be input into the learning model corresponding to the MI EEG signal and the recognition result will be output. At the same time, the corresponding function start command will be sent to the actuator module. If the user imagines extending the right wrist, the corresponding MI EEG signal will be input into the learning model corresponding to the MI EEG signal and the recognition result will be output. At the same time, the corresponding function close command will be sent to the actuator module.

[0067] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0068] The present invention has made a breakthrough in applying the two brain electrical stimulation paradigms, SSVEP and MI, to the control of massage chairs for the first time. The system combines the rapid response and high accuracy of SSVEP with the excellent asynchronous control capability of MI. This enables users to easily select massage functions through SSVEP and use MI to instantly start and stop the massage chair. By integrating the advantages of the two brain-computer interface (BCI) control paradigms, SSVEP and MI, the present invention constructs a multimodal massage chair system with rich instructions and fast response. The system is particularly suitable for patients who are bedridden for a long time, have limited mobility, and need autonomous massage care. They can independently select the massage area and decide when to stop the massage, thereby achieving independent massage care.

[0069] The non-contact EEG signal control technology adopted by the present invention solves the problems that may be encountered in the use of traditional massage chairs, such as the inability to operate the button handle due to the restriction of the arms, and the use difficulties of patients with ALS and high paraplegia. With the diverse function options provided by SSVEP, users can easily choose the massage mode that suits them, and the asynchronous control of MI ensures that the start and stop operation of the massage chair is more convenient. This design not only improves the convenience of operation of the massage chair, but also enhances the overall user experience.

[0070] The present invention proposes a neural network named MultiScale-CAEEGNet for identifying EEG signals under MI stimulation. The network adopts data enhancement and regularization techniques, optimizes the deep separable convolution module in the EEGNet neural network, upgrades it to a multi-scale convolution module, and introduces a dilated convolution module and a channel attention mechanism. The data enhancement strategy generates diversified input samples by randomly shifting the data in the time dimension to simulate a wider sample distribution, thereby preventing the model from overfitting. In terms of regularization, we adopt L2 regularization, limit the complexity of the model by adding L2 regularization terms, and apply random inactivation technology at each layer to adjust the inactivation ratio to further prevent overfitting. The multi-scale convolution module performs deep separable convolution by using convolution kernels of different sizes to capture information at different time scales. Short convolution kernels help to identify short-term fluctuations in EEG signals, while long convolution kernels help to capture long-term trends. The atrous convolution module expands the receptive field without adding additional parameters by setting an appropriate atrous rate, while maintaining the resolution of the feature map, which not only extracts a wider range of contextual information but also reduces computational complexity. The atrous convolution module can also capture signal features more comprehensively and improve the performance of the model. The channel attention mechanism assigns different weights to each convolution channel, allowing the model to automatically focus on more critical frequency information or specific signal channels. By adaptively weighting different channels, the model can more effectively ignore irrelevant noise signals while enhancing the impact on mission-critical frequency bands, thereby improving overall model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is the schematic diagram of the multimodal intelligent massage chair system.

[0072] Figure 2 This is the architecture diagram of the multimodal intelligent massage chair system. DETAILED DESCRIPTION

[0073] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0074] like Figure 1 and Figure 2 As shown, this embodiment discloses a multi-modal intelligent massage chair system based on SSVEP and MI, including:

[0075] The operation interface module includes an SSVEP stimulation interface for the SSVEP stimulation paradigm and an MI stimulation interface for the MI stimulation paradigm; in the SSVEP stimulation paradigm, different stimulation buttons are matched with visual stimulations of specific frequencies to induce EEG activities of corresponding frequencies; for the MI stimulation paradigm, the user is guided to imagine the corresponding limb movements by visually simulating the movements of the left and right arms combined with voice prompts. After hearing the voice prompts, the user needs to imagine performing the corresponding arm movements, and the arm movements displayed on the MI stimulation interface are only used as visual aids to help the user imagine the movements more accurately;

[0076] A data acquisition module, used to collect the original EEG signals generated by the user when using the operation interface module;

[0077] The data processing and analysis module is used to process and analyze the original EEG signals collected by the data acquisition module to obtain the recognition results of SSVEP stimulation and MI stimulation of the original EEG signals;

[0078] The actuator module is used to receive the recognition results obtained by the data processing and analysis module and convert them into control instructions that can be recognized by the massage chair, so as to realize different massage functions.

[0079] Specifically, the operation interface module includes the following parts:

[0080] Basic parameter settings: includes basic parameter configuration and operation mode selection for connecting to the EEG signal acquisition headband, where the operation modes are divided into training mode and use mode; training mode means that the user needs to be trained before the first use, and the system builds a model based on the training results. When the model meets the requirements, it is automatically saved for direct use in subsequent use modes, where the model means that the training mode generates a learning model by putting the SSVEP EEG signal into the FBCCA algorithm and the MI EEG signal into the MultiScale-CAEEGNet neural network training. In the use mode, you only need to input the data into the corresponding model to get the recognition result;

[0081] Massage chair function control: It includes five massage functions, namely "shoulder massage", "arm massage", "hand massage", "waist massage" and "leg massage". Each function is realized through five stimulation buttons on the SSVEP stimulation interface. Each stimulation button flashes continuously at a preset frequency. The preset frequencies corresponding to the five stimulation buttons are 7Hz, 7.5Hz, 8Hz, 8.5Hz and 9Hz respectively. When the user looks at the stimulation buttons of different frequencies, the corresponding SSVEP EEG signals will be triggered, thereby selecting different massage functions. Among them, the brightness change formula of the stimulation button is as follows:

[0082]

[0083] Where: s(f,i) represents the brightness of the stimulation button with a preset frequency of f at the i-th moment, and freshrate represents the refresh rate of the SSVEP stimulation interface; Massage chair start and stop control: after hearing the voice prompt, the user needs to imagine performing the corresponding body movements, where imagining extending the left wrist represents turning on the corresponding massage chair function selected by the SSVEP EEG signal, and imagining extending the right wrist represents turning off the corresponding massage chair function selected by the SSVEP EEG signal.

[0084] Specifically, the data acquisition module adopts a six-channel EEG signal acquisition headband, and its sampling rate is set to 250Hz. The EEG signal acquisition headband is equipped with eight electrodes, including a reference electrode, a ground electrode, and six EEG signal channel electrodes. The ground electrode is used to set the zero potential reference point of the EEG signal, and the reference electrode and the ground electrode are both located at the temples on both sides of the head. The six EEG signal channel electrodes correspond to six channels respectively, which are used to capture the user's EEG signals. Their placement follows the standard of the "International 10-20 System" and are located at C3, C4, P3, P4, O1 and O2. The collected EEG signals are converted into digital signals by the 24-bit resolution analog-to-digital conversion module inside the EEG signal acquisition headband, and are transmitted to a PC (including an operation interface module and a data processing and analysis module) by wireless.

[0085] Specifically, the data processing and analysis module includes a data preprocessing submodule and a data analysis submodule;

[0086] The data preprocessing submodule includes an SSVEP data preprocessing module and an MI data preprocessing module, wherein:

[0087] The specific operation process of the SSVEP data preprocessing module is as follows:

[0088] First, baseline drift correction was performed on the EEG signals of the four channels O1, O2, P3, and P4 to eliminate DC offset. Subsequently, a notch filter was used to filter out 50 Hz power frequency interference. After that, the signals were filtered through an 11th-order Butterworth infinite impulse response (IIR) bandpass filter to retain the signal components in the frequency range of 6-30 Hz. Considering the delay of the human visual system, the start extraction time of the SSVEP EEG signal was delayed by 140 milliseconds to avoid interference from the initial signal. In the training mode and use mode of the SSVEP stimulation paradigm, the EEG data segment from 140 milliseconds to 5 seconds after the start of the stimulation was finally selected for model training and recognition analysis.

[0089] The details of the MI data preprocessing module are as follows:

[0090] First, the baseline drift is removed, and the signal is filtered using an 11th-order Butterworth infinite impulse response (IIR) bandpass filter to retain the 8-30 Hz signal component. Then, the 0-5 s signal segment from the beginning of the MI paradigm stimulation is extracted and put into the MultiScale-CAEEGNet neural network to complete the recognition.

[0091] The data analysis submodule includes an SSVEP data analysis module and an MI data analysis module, wherein: the specific details of the SSVEP data analysis module are as follows:

[0092] The FBCCA algorithm is used to identify EEG signals. FBCCA is an algorithm used for SSVEP signal recognition. Its function is to identify the user's intention by extracting EEG signal features of specific frequencies. The principle is to use a filter bank to decompose the original EEG signal into multiple frequency bands, and apply canonical correlation analysis (CCA) to each frequency band to calculate the correlation between the signal and the reference signal, so as to find the frequency with the largest correlation among multiple target frequencies and realize accurate target recognition.

[0093] The details of the MI data analysis module are as follows:

[0094] Construct the user's MI paradigm model, specifically, construct a model of two states: imagining the left wrist extending and imagining the right wrist extending after the SSVEP stimulus flashes, collect the user's EEG signals in the above two states, and put the collected EEG signals into the MultiScale-CAEEGNet neural network for recognition.

[0095] The MultiScale-CAEEGNet neural network is a model proposed based on the EEGNet neural network, that is, an improved EEGNet neural network. The input data format of the original EEGNet neural network is: number of samples × number of signal channels × number of sampling points. In this system, the number of signal channels is set to 4, which are four EEG channels C3, C4, P3, and P4, and the number of sampling points is 1250. The EEGNet neural network is mainly composed of the following three parts: temporal convolution module, spatial convolution module, and depthwise separable convolution module. The temporal convolution module mainly uses F1 convolution kernels of size (1,64) to extract information in the time dimension. The spatial convolution mainly uses D*F1 convolution kernels of size (C,1) to integrate data information from different EEG channels, where D represents the depth multiple of the depth convolution, F1 represents the number of filters in the temporal convolution module, and C is the number of channels. The depthwise separable convolution module operates by splitting the standard convolution operation into two parts: depthwise convolution and pointwise convolution. The depth convolution performs a separate convolution operation on each input channel, which allows the network to extract spatial features on each channel without mixing the information between channels. This method effectively captures the local spatial information of the features. The point-by-point convolution uses 1×1 convolution to combine the outputs of the depth convolution, allowing the features between channels to be mixed.

[0096] The structure of the original EEGNet neural network includes convolution layer 1, one-dimensional batch normalization layer 1, convolution layer 2, one-dimensional batch normalization layer 2, one-dimensional average pooling layer 1, random dropout layer 1, convolution layer 3, one-dimensional batch normalization layer 3, one-dimensional average pooling layer 2, random dropout layer 2, and fully connected layer.

[0097] The MultiScale-CAEEGNet neural network introduces data enhancement and regularization techniques on the basis of the EEGNet neural network. At the same time, the depth-separable convolution module in the EEGNet neural network is optimized into a multi-scale convolution module, and a hole convolution module and a channel attention mechanism are added. The details are as follows:

[0098] The data enhancement optimizes the input data, simulates more sample distributions, and prevents overfitting, specifically by generating diversified inputs by randomly shifting data in the time dimension;

[0099] The regularization adopts L2 regularization, adds L2 regularization terms to constrain the model complexity, and adjusts the inactivation ratio in each layer of random inactivation to prevent overfitting;

[0100] The multi-scale convolution module uses depth-wise separable convolutional layer operations with different convolution kernel sizes to capture information at different time scales, wherein shorter convolution kernels help identify short-term fluctuations, while longer convolution kernels help capture long-term trends;

[0101] The dilated convolution module expands the receptive field while maintaining the resolution of the feature map by setting an appropriate dilation rate. Since no additional parameters are introduced, the dilated convolution module effectively reduces the computational complexity while extracting a wider range of contextual information. In addition, with its sensitivity to the global context, the dilated convolution module can help the model capture signal features more comprehensively, thereby improving the performance of the task.

[0102] The channel attention mechanism assigns different weights to each convolution channel, helping the model to automatically focus on more important frequency information or specific signal channels. By adaptively weighting different channels, the model can more effectively ignore irrelevant noise signals while amplifying key frequency bands that contribute to the task, thereby improving model performance.

[0103] The convolution layer 3 in the EEGNet neural network is replaced by three convolution layers, namely convolution layer 3-1, convolution layer 3-2, and convolution layer 3-3. The convolution kernel sizes of the three convolution layers are 8, 16, and 32, respectively. The convolution kernel of size 8 is a shorter convolution kernel, which helps to identify short-term fluctuations; the convolution kernel of size 16 is the original convolution kernel; the convolution kernel of size 32 is a longer convolution kernel, which helps to capture long-term trends.

[0104] After the multi-scale convolution module, dilated convolution modules with dilation rates of 1, 2, and 4 are added to realize the extraction of multi-scale features.

[0105] The role of the channel attention mechanism is to automatically learn the importance of each channel according to the characteristics of the input data, and adjust the contribution of each channel by weight. It summarizes each channel through global average pooling, and then uses two fully connected layers to calculate the attention weight of each channel. It then strengthens the features of important channels by weighting each channel, while suppressing unimportant channels. This mechanism enables the model to adaptively adjust the influence of different channels, thereby improving the model's sensitivity to key information.

[0106] The format of the input data X of the MultiScale-CAEEGNet neural network is: X∈R B×C×T, where B is the batch size, C is the number of channels, T is the number of time points, and R is the real number domain; In order to increase the diversity of training data and the robustness of the model, data augmentation with random displacement in the time dimension is used to provide the model with a richer input distribution;

[0107] X'=Roll(X,Δt)

[0108] Where: Δt is the random offset; Roll(X,Δt) means periodically shifting the input data X in the time dimension to obtain richer input data X'; if the signal is shifted in the time dimension, assuming Δt is a positive value, the time points in the signal will be shifted backward by Δt positions, and the parts that exceed the range will reappear from the front.

[0109] The multi-scale convolution module uses a depth-separable convolution layer with large and small convolution kernels of k1=8, k2=16, and k3=32 to extract features respectively. At the same time, a dilated convolution module is introduced to expand the receptive field of the model and enhance the ability to capture long-distance dependencies. That is, a dilated convolution module with dilated rates of 1, 2, and 4 is added after the multi-scale convolution module to realize the extraction of multi-scale features, and the features of three scales are obtained, namely, H scale1 , H scale2 , H scale3 , where H scale1 , H scale2 , H scale3 It refers to the output features of the input data X′ after it passes through the first convolution layer of the original EEGNet neural network, namely convolution layer 1, the first one-dimensional batch normalization layer, namely one-dimensional batch normalization layer 1, the second convolution layer, namely convolution layer 2, the second one-dimensional batch normalization layer, namely one-dimensional batch normalization layer 2, the first one-dimensional average pooling layer, namely one-dimensional average pooling layer 1, the first random dropout layer 1, namely random dropout layer 1, and then passes through the multi-scale convolution module and the void convolution module.

[0110] The three scale features H scale1 , H scale2 , H scale3 Splice to feature H multi-scale :

[0111] H multi-scale =Concat(H scale1 , H scale2 , H scale3 )

[0112] Due to the introduction of the channel attention mechanism, the multi-scale features are optimized by dynamically adjusting the weights of the feature channels; first, global average pooling is performed to calculate the global statistics of each channel:

[0113]

[0114] Where: S c Represents the global average pooling result of the cth channel; Represents the characteristic H multi-scale The value at the tth sampling point of the cth channel;

[0115] Then, the attention weights of the channels are generated through two fully connected layers:

[0116] w c =σ(W2·ReLU(W1·S+b1)+b2)

[0117] Where: σ is the sigmoid activation function; S is a vector containing the S of all channels c Value; w c is the attention weight of the cth channel; W1 and W2 are weight matrices; b1 and b2 are bias terms; ReLU is an activation function;

[0118] Next, use the attention weight to adjust the feature and obtain the feature H after the channel attention mechanism attention :

[0119] H attention = w·H multi-scale

[0120] Where: w is the matrix composed of the attention weights of all channels;

[0121] Finally, the feature H attention The dimension is reduced by global average pooling, and finally, the recognition result is output using the fully connected layer and Softmax activation function.

[0122]

[0123] Where: GAP represents global average pooling, Dense represents that the output processed by GAP is processed through a fully connected layer; finally, where n classes Represents the number of identified categories;

[0124] Among them, in the process of model weight optimization, L2 regularization term is added to constrain model complexity and prevent overfitting. The regularization term is in the form of:

[0125]

[0126] Where: L is the total loss function; L CE is the cross entropy loss; λ is the regularization strength; Represents the square of the L2 norm of the weight matrix U.

[0127] Specifically, the actuator module uses the nRF52832 Bluetooth controller to receive the recognition results from the data processing and analysis module, converts the recognition results into instructions that can be executed by the massage chair, and forwards them to the massage chair controller through the UART serial port.

[0128] Specifically, before officially using the system, the user needs to go through the training mode first, because the EEG signals of different users vary greatly, and the training mode is needed to determine the parameters suitable for the user;

[0129] Before entering the training mode, the user needs to sit on the massage chair, wear the EEG signal collection headband, watch the operation interface module, and select the training mode;

[0130] In the training mode, the interface is the SSVEP stimulation interface, and the following process is executed: a stimulation button in the SSVEP stimulation paradigm will flash red for 0.2 seconds, prompting the user to concentrate on the stimulation button. After the flashing ends, all stimulation buttons in the interface will flash at the same time for 5 seconds. During this period, the user needs to continue to stare at the previously prompted stimulation button. Each stimulation button is collected once according to the above process. After the EEG signals of the five stimulation buttons are collected, one training is completed; one round of experiment is completed after staring at the five stimulation buttons in sequence according to the flashing prompts. Five rounds of SSVEP experiments need to be completed in the training mode; after the training mode is completed, the interface switches to the MI stimulation interface, and voice prompts are played at the same time. When the voice prompts imagine extending the left or right wrist, the virtual arm in the interface will make corresponding wrist extension movements. The user needs to perform corresponding movement imagination according to the voice prompts or the movements in the interface. One set of data is collected each time imagination, and a total of 80 sets of movement imagination data need to be collected;

[0131] After the training mode is completed, the collected SSVEP EEG signals and MI EEG signals are input into the data processing and analysis module to generate a learning model;

[0132] Next, the user selects the usage mode in the operation interface module and enters the SSVEP stimulation interface in the usage mode. The user gazes at a stimulation button in the SSVEP stimulation interface, and the SSVEP EEG signal corresponding to the stimulation button will be input into the learning model corresponding to the SSVEP EEG signal and the recognition result will be output. The massage function corresponding to the recognition result will be selected. After completing the function selection, if the user imagines extending the left wrist, the corresponding MI EEG signal will be input into the learning model corresponding to the MI EEG signal and the recognition result will be output. At the same time, the corresponding function start command will be sent to the actuator module. If the user imagines extending the right wrist, the corresponding MI EEG signal will be input into the learning model corresponding to the MI EEG signal and the recognition result will be output. At the same time, the corresponding function close command will be sent to the actuator module.

[0133] The multimodal intelligent massage chair system described above in this embodiment has an operating interface that is developed using C++ language on Qt and runs on a Windows device. The training and classification of the algorithm model are written in other development environments, and the relevant algorithms are called in the Qt background. After the algorithm calculation is completed, the output results are fed back to the background, and the background finally sends the command results to the Bluetooth controller via wireless transmission to realize the control of the massage chair.

[0134] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A multi-modal intelligent massage chair system based on SSVEP and MI, characterized in that: include: The operation interface module includes an SSVEP stimulation interface for the SSVEP stimulation paradigm and an MI stimulation interface for the MI stimulation paradigm; in the SSVEP stimulation paradigm, different stimulation buttons are matched with visual stimulations of specific frequencies to induce EEG activities of corresponding frequencies; for the MI stimulation paradigm, the user is guided to imagine the corresponding limb movements by visually simulating the movements of the left and right arms combined with voice prompts. After hearing the voice prompts, the user needs to imagine performing the corresponding arm movements, and the arm movements displayed on the MI stimulation interface are only used as visual aids to help the user imagine the movements more accurately; A data acquisition module, used to collect the original EEG signals generated by the user when using the operation interface module; The data processing and analysis module is used to process and analyze the original EEG signals collected by the data acquisition module to obtain the recognition results of SSVEP stimulation and MI stimulation of the original EEG signals; The actuator module is used to receive the recognition results obtained by the data processing and analysis module and convert them into control instructions that can be recognized by the massage chair, so as to realize different massage functions.

2. The multimodal intelligent massage chair system based on SSVEP and MI according to claim 1, characterized in that: The operation interface module includes the following parts: Basic parameter settings: includes basic parameter configuration for connection with the EEG signal acquisition headband and operation mode selection, where the operation modes are divided into training mode and use mode; the training mode means that the user needs to be trained before the first use, and the system builds a model based on the training results. When the model meets the requirements, it is automatically saved for direct use in subsequent use modes, where the model means that in the training mode, a learning model will be generated after the SSVEP EEG signal is put into the FBCCA algorithm and the MI EEG signal is put into the MultiScale-CAEEGNet neural network for training. In the use mode, you only need to input the data into the corresponding model to get the recognition result; Massage chair function control: It includes five massage functions, namely "shoulder massage", "arm massage", "hand massage", "waist massage" and "leg massage". Each function is realized through five stimulation buttons on the SSVEP stimulation interface. Each stimulation button flashes continuously at a preset frequency. The preset frequencies corresponding to the five stimulation buttons are 7Hz, 7.5Hz, 8Hz, 8.5Hz and 9Hz respectively. When the user looks at the stimulation buttons of different frequencies, the corresponding SSVEP EEG signals will be triggered, thereby selecting different massage functions. Among them, the brightness change formula of the stimulation button is as follows: Where: s(f,i) represents the brightness of the stimulation button at the i-th moment with a preset frequency of f, and freshrate represents the refresh rate of the SSVEP stimulation interface; Massage chair start and stop control: After hearing the voice prompt, the user needs to imagine performing the corresponding body movements, among which, imagining extending the left wrist represents turning on the corresponding massage chair function selected by the SSVEP EEG signal, and imagining extending the right wrist represents turning off the corresponding massage chair function selected by the SSVEP EEG signal.

3. The multimodal intelligent massage chair system based on SSVEP and MI according to claim 2, characterized in that: The data acquisition module adopts a six-channel EEG signal acquisition head ring with a sampling rate set to 250Hz. The EEG signal acquisition head ring is equipped with eight electrodes, including a reference electrode, a ground electrode and six EEG signal channel electrodes. The ground electrode is used to set the zero potential reference point of the EEG signal, and the reference electrode and the ground electrode are both located at the temples on both sides of the head. The six EEG signal channel electrodes correspond to six channels respectively, which are used to capture the user's EEG signals. Their placement follows the standard of the "International 10-20 System" and are located at C3, C4, P3, P4, O1 and O2 positions. The collected EEG signals are converted into digital signals by a 24-bit resolution analog-to-digital conversion module inside the EEG signal acquisition head ring.

4. The multimodal intelligent massage chair system based on SSVEP and MI according to claim 3, characterized in that: The data processing and analysis module includes a data preprocessing submodule and a data analysis submodule; The data preprocessing submodule includes an SSVEP data preprocessing module and an MI data preprocessing module, wherein: The specific operation process of the SSVEP data preprocessing module is as follows: First, baseline drift correction was performed on the EEG signals of the four channels O1, O2, P3, and P4 to eliminate DC offset. Subsequently, a notch filter was used to filter out 50 Hz power frequency interference. After that, the signals were filtered through an 11th-order Butterworth infinite impulse response bandpass filter to retain the signal components in the frequency range of 6-30 Hz. Considering the delay of the human visual system, the start extraction time of the SSVEP EEG signal was delayed by m1 milliseconds to avoid interference from the initial signal. In the training mode and use mode of the SSVEP stimulation paradigm, the EEG data segment from m1 milliseconds to 5 seconds after the start of the stimulation was finally selected for model training and recognition analysis. The details of the MI data preprocessing module are as follows: First, remove the baseline drift, use the 11th-order Butterworth infinite impulse response bandpass filter to filter, retain the 8-30HZ signal component, and then extract the 0-5s signal segment from the beginning of the MI paradigm stimulation and put it into the MultiScale-CAEEGNet neural network to complete the recognition; The data analysis submodule includes an SSVEP data analysis module and an MI data analysis module, wherein: the specific details of the SSVEP data analysis module are as follows: The FBCCA algorithm is used to identify EEG signals. FBCCA is an algorithm used for SSVEP signal recognition. Its function is to identify the user's intention by extracting EEG signal features of specific frequencies. The principle is to use a filter bank to decompose the original EEG signal into multiple frequency bands, and apply canonical correlation analysis (CCA) to each frequency band to calculate the correlation between the signal and the reference signal, so as to find the frequency with the largest correlation among multiple target frequencies and realize accurate target recognition. The details of the MI data analysis module are as follows: Construct the user's MI paradigm model, specifically, construct a model of two states: imagining the left wrist extending and imagining the right wrist extending after the SSVEP stimulus flashes, collect the user's EEG signals in the above two states, and put the collected EEG signals into the MultiScale-CAEEGNet neural network for recognition.

5. The multimodal intelligent massage chair system based on SSVEP and MI according to claim 4, characterized in that: The MultiScale-CAEEGNet neural network is a model proposed based on the EEGNet neural network, that is, an improved EEGNet neural network, which introduces data enhancement and regularization techniques on the basis of the EEGNet neural network, optimizes the depth-separable convolution module in the EEGNet neural network into a multi-scale convolution module, and adds a hole convolution module and a channel attention mechanism. The details are as follows: The data enhancement optimizes the input data, simulates more sample distributions, and prevents overfitting, specifically by generating diversified inputs by randomly shifting data in the time dimension; The regularization adopts L2 regularization, adds L2 regularization terms to constrain the model complexity, and adjusts the inactivation ratio in each layer of random inactivation to prevent overfitting; The multi-scale convolution module uses depth-wise separable convolutional layer operations with different convolution kernel sizes to capture information at different time scales, wherein shorter convolution kernels help identify short-term fluctuations, while longer convolution kernels help capture long-term trends; The dilated convolution module expands the receptive field while maintaining the resolution of the feature map by setting an appropriate dilation rate. Since no additional parameters are introduced, the dilated convolution module effectively reduces the computational complexity while extracting a wider range of contextual information. In addition, with its sensitivity to the global context, the dilated convolution module can help the model capture signal features more comprehensively, thereby improving the performance of the task. The channel attention mechanism assigns different weights to each convolution channel, helping the model to automatically focus on more important frequency information or specific signal channels. By adaptively weighting different channels, the model can more effectively ignore irrelevant noise signals while amplifying key frequency bands that contribute to the task, thereby improving model performance. The third convolutional layer, i.e. convolutional layer 3, in the EEGNet neural network is replaced by three convolutional layers, namely convolutional layer 3-1, convolutional layer 3-2, and convolutional layer 3-3. The convolution kernel sizes of the three convolutional layers are 8, 16, and 32, respectively. The convolution kernel of size 8 is a shorter convolution kernel, which helps to identify short-term fluctuations; the convolution kernel of size 16 is the original convolution kernel; the convolution kernel of size 32 is a longer convolution kernel, which helps to capture long-term trends. After the multi-scale convolution module, dilated convolution modules with dilation rates of 1, 2, and 4 are added to realize the extraction of multi-scale features. The role of the channel attention mechanism is to automatically learn the importance of each channel according to the characteristics of the input data, and adjust the contribution of each channel by weight. It summarizes each channel through global average pooling, and then uses two fully connected layers to calculate the attention weight of each channel. It then strengthens the features of important channels by weighting each channel, while suppressing unimportant channels. This mechanism enables the model to adaptively adjust the influence of different channels, thereby improving the model's sensitivity to key information.

6. The multimodal intelligent massage chair system based on SSVEP and MI according to claim 5, characterized in that: The format of the input data X of the MultiScale-CAEEGNet neural network is: X∈R B×C×T , where B is the batch size, C is the number of channels, T is the number of time points, and R is the real number domain; in order to increase the diversity of training data and the robustness of the model, data augmentation with random displacement in the time dimension is used to provide the model with a richer input distribution; X'=Roll(X,Δt) Where: Δt is the random offset; Roll(X,Δt) means periodically shifting the input data X in the time dimension to obtain richer input data X'; if the signal is shifted in the time dimension, assuming Δt is a positive value, the time points in the signal will be shifted backward by Δt positions, and the parts that exceed the range will reappear from the front.

7. The multimodal intelligent massage chair system based on SSVEP and MI according to claim 6, characterized in that: The multi-scale convolution module uses a depth-separable convolution layer with large and small convolution kernels of k1=8, k2=16, and k3=32 to extract features respectively. At the same time, a dilated convolution module is introduced to expand the receptive field of the model and enhance the ability to capture long-distance dependencies. That is, a dilated convolution module with dilated rates of 1, 2, and 4 is added after the multi-scale convolution module to realize the extraction of multi-scale features, and the features of three scales are obtained, namely, H scale1 , H scale2 , H scale3 , where H scale1 , H scale2 , H scale3 It refers to the output features of the input data X′ after it passes through the first convolution layer of the original EEGNet neural network, namely convolution layer 1, the first one-dimensional batch normalization layer, namely one-dimensional batch normalization layer 1, the second convolution layer, namely convolution layer 2, the second one-dimensional batch normalization layer, namely one-dimensional batch normalization layer 2, the first one-dimensional average pooling layer, namely one-dimensional average pooling layer 1, the first random dropout layer 1, namely random dropout layer 1, and then passes through the multi-scale convolution module and the void convolution module. The three scale features H scale1 , H scale2 , H scale3 Splice to feature H multi-scale : H multi-scale =Concat(H scale1 ,H scale2 ,H scale3 ) Due to the introduction of the channel attention mechanism, the multi-scale features are optimized by dynamically adjusting the weights of the feature channels; first, global average pooling is performed to calculate the global statistics of each channel: Where: S c Represents the global average pooling result of the cth channel; Represents the characteristic H multi-scale The value at the tth sampling point of the cth channel; Then, the attention weights of the channels are generated through two fully connected layers: w c =σ(W2·ReLU(W1·S+b1)+b2) Where: σ is the sigmoid activation function; S is a vector containing the S of all channels c Value; w c is the attention weight of the cth channel; W1 and W2 are weight matrices; b1 and b2 are bias terms; ReLU is an activation function; Next, use the attention weight to adjust the feature and obtain the feature H after the channel attention mechanism attention : H attention =w·H multi-scale Where: w is the matrix composed of the attention weights of all channels; Finally, the feature H attention The dimension is reduced by global average pooling, and finally, the recognition result is output using the fully connected layer and Softmax activation function. Where: GAP represents global average pooling, Dense represents that the output processed by GAP is processed through a fully connected layer; finally, where n classes Represents the number of identified categories; Among them, in the process of model weight optimization, L2 regularization term is added to constrain model complexity and prevent overfitting. The regularization term is in the form of: Where: L is the total loss function, L CE is the cross entropy loss; λ is the regularization strength; Represents the square of the L2 norm of the weight matrix U.

8. The multi-modal intelligent massage chair system based on SSVEP and MI according to claim 7, characterized in that: The MultiScale-CAEEGNet neural network has 4 channels, namely C3, C4, P3, and P4, and 1250 time points. It also includes the temporal convolution module and spatial convolution module of the original EEGNet neural network. The temporal convolution module uses F1 convolution kernels of size (1,64) to extract information in the time dimension, and the spatial convolution module uses D*F1 convolution kernels of size (C,1) to integrate data information from different EEG channels, where D represents the depth multiple of the deep convolution and F1 represents the number of filters in the temporal convolution module.

9. The multimodal intelligent massage chair system based on SSVEP and MI according to claim 8, characterized in that: The actuator module uses the nRF52832 Bluetooth controller to receive the recognition results from the data processing and analysis module, converts the recognition results into instructions that can be executed by the massage chair, and forwards them to the massage chair controller through the UART serial port.

10. The multi-modal intelligent massage chair system based on SSVEP and MI according to claim 9, characterized in that: Before officially using the system, users are required to go through the training mode first. The reason is that the EEG signals of different users vary greatly, and the training mode is needed to determine the parameters that are suitable for the user; Before entering the training mode, the user needs to sit on the massage chair, wear the EEG signal collection headband, watch the operation interface module, and select the training mode; In the training mode, the interface is the SSVEP stimulation interface, and the following process is executed: a stimulation button in the SSVEP stimulation paradigm will flash red for m2 seconds, prompting the user to concentrate on the stimulation button. After the flashing ends, all stimulation buttons in the interface will flash simultaneously for m3 seconds. During this period, the user needs to continue to stare at the previously prompted stimulation button. Each stimulation button is collected once according to the above process. After the EEG signals of the five stimulation buttons are collected, one training is completed; one round of experiment is completed after staring at the five stimulation buttons in sequence according to the flashing prompts. Five rounds of SSVEP experiments need to be completed in the training mode; after the training mode is completed, the interface switches to the MI stimulation interface, and voice prompts are played at the same time. When the voice prompts imagine extending the left or right wrist, the virtual arm in the interface will make corresponding wrist extension movements. The user needs to perform corresponding movement imagination according to the voice prompts or the movements in the interface. Each imagination collects m4 groups of data, and a total of m5 groups of movement imagination data need to be collected; After the training mode is completed, the collected SSVEP EEG signals and MI EEG signals are input into the data processing and analysis module to generate a learning model; Next, the user selects the usage mode in the operation interface module and enters the SSVEP stimulation interface in the usage mode. The user gazes at a stimulation button in the SSVEP stimulation interface, and the SSVEP EEG signal corresponding to the stimulation button will be input into the learning model corresponding to the SSVEP EEG signal and the recognition result will be output. The massage function corresponding to the recognition result will be selected. After completing the function selection, if the user imagines extending the left wrist, the corresponding MI EEG signal will be input into the learning model corresponding to the MI EEG signal and the recognition result will be output. At the same time, the corresponding function start command will be sent to the actuator module. If the user imagines extending the right wrist, the corresponding MI EEG signal will be input into the learning model corresponding to the MI EEG signal and the recognition result will be output. At the same time, the corresponding function close command will be sent to the actuator module.

Images