An intelligent wheelchair control method and system based on fusion of electroencephalogram and electromyogram

Abstract

The application discloses a kind of intelligent wheelchair control method and system based on electroencephalogram and electromyogram fusion, including synchronous acquisition electroencephalogram and electromyogram signal and preprocessing, current intention probability is output by local hybrid model of wavelet channel attention and parallel transform module containing;Predict future intention probability using lightweight Transform encoder, combine historical prediction accuracy and signal quality dynamic fusion decision;Cache high confidence sample without intervention or safety event, fine-tune local model base after threshold value, upload after realizing privacy protection by Laplace noise;Global model is generated and updated by cloud according to data weight aggregation.This application improves the coherence and robustness of wheelchair control, shortens the calibration period of new users, realizes continuous evolution of the model under privacy protection, and is suitable for independent movement assistance for people with physical disabilities and movement disorders.

Description

Technical Field

[0001] This invention belongs to the field of biosignal processing technology, specifically relating to an intelligent wheelchair control method and system based on the fusion of electroencephalography (EEG) and electromyography (EMG). Background Technology

[0002] With the increasing aging population and growing demand for autonomous mobility among people with physical disabilities, intelligent wheelchairs, as core rehabilitation aids, have become crucial for technological breakthroughs in terms of control precision, adaptability, and safety. Brain-computer interface (BCI) technology enables human-computer interaction by interpreting human biosignals (such as electroencephalography (EEG) and electromyography (EMG). EEG signals directly reflect the user's motor intentions, while EMG signals have strong motor correlation and good anti-interference capabilities. The fusion of these two technologies can complement each other's advantages and improve the accuracy of intention recognition.

[0003] Existing technologies have been used in the research of EEG and EMG fusion control of intelligent wheelchairs, but three major pain points still exist: 1. Insufficient utilization of the temporal continuity of motor intention: Traditional control methods treat the recognition of intention at each moment as an independent event, relying on instantaneous signal features (such as blinking, specific electromyographic pulses) to generate control commands. They fail to capture the temporal correlation of motor intention and do not make full use of the temporal continuity of user motor intention, resulting in instantaneous signal fluctuations causing command interruption or misjudgment.

[0004] 2. Poor generalization ability of cross-individual models: Existing models are mostly trained based on single user data, making it difficult to achieve cross-individual knowledge transfer while protecting privacy. Differences in biological signals between individuals (such as EEG amplitude and EMG response sensitivity) cause new users to undergo a lengthy calibration period.

[0005] 3. Data privacy protection and collaborative learning are difficult to balance: Biosignals (especially EEG) contain user privacy information. Traditional centralized training requires the collection of raw data, which poses a risk of privacy leakage. On the other hand, decentralized training cannot achieve group knowledge sharing, making it difficult to continuously optimize model performance.

[0006] Furthermore, existing methods lack sufficient data privacy protection measures, limiting the construction of multi-user collaborative learning frameworks. The limitations of current technologies, such as simplistic feature extraction methods, fixed fusion decision weights, and a lack of personalized adaptation mechanisms, further restrict the practical application of intelligent wheelchairs. Therefore, there is an urgent need for an intelligent wheelchair control scheme that can fully utilize the temporal continuity of motion intentions, achieve model evolution under cross-individual privacy protection, and balance individual adaptation with group collaboration. Summary of the Invention

[0007] To address the problems of lost temporal continuity of intent, poor generalization ability of cross-individual models, and difficulties in data privacy protection and collaborative group learning in existing technologies, this invention provides an intelligent wheelchair control method and system based on the fusion of electroencephalography (EEG) and electromyography (EMG).

[0008] In a first aspect, the present invention proposes an intelligent wheelchair control method based on the fusion of electroencephalography (EEG) and electromyography (EMG), the method comprising the following steps: S1. Simultaneously acquire the user's EEG and EMG signals, preprocess the raw signals, and then divide them into continuous time-series data frames through a sliding window; S2. Input the current time-series data frame into the local hybrid model. The model extracts the time-frequency features of the signal through the attention mechanism and captures the spatiotemporal features based on the parallel transformation path, and outputs the preliminary probability of the motion intention corresponding to the current frame. S3. Store the preliminary probabilities of motion intentions from multiple consecutive historical windows into a buffer queue, and predict the probability of predicted intentions for future windows based on the historical intention sequences in the queue; wherein, the training labels of the prediction model are delayed labels based on subsequent actual execution instructions. S4. The preliminary probability of the current frame is fused with the predicted intention probability of the future window. Based on the fusion result, the current execution instruction is determined and the wheelchair is driven to execute. At the same time, the data frame corresponding to the execution instruction is stored in the local cache as a high-confidence sample. S5. When the number of samples in the local cache reaches a preset threshold, the high-confidence sample is used as training data to fine-tune the base part of the local hybrid model, and the model update parameters to be uploaded are calculated based on the fine-tuned model. After privacy protection processing, the model is uploaded to the cloud server. S6: The cloud server aggregates model update parameters uploaded by multiple clients to update the globally shared base model, and then distributes the updated global model base parameters to each client to update their respective local hybrid models.

[0009] Preferably, the specific working process of the local hybrid model in step S2 is as follows: The current time-series data frame is input into the wavelet channel attention module. This module performs discrete wavelet transform on the multi-channel signal to obtain multi-scale wavelet coefficients, and performs global average pooling on the coefficients of each scale to obtain the channel descriptor. The channel descriptors are sequentially input into two fully connected layers. The first fully connected layer uses the ReLU activation function for dimensionality reduction, and the second fully connected layer uses the Sigmoid activation function for dimensionality increase, outputting an attention weight vector with the same number of channels as the original. The attention weight vector is multiplied with the wavelet coefficients by channel to obtain the weighted multi-scale features, and then the weighted features are reconstructed into a temporal feature map. The temporal feature map is input into the parallel transformation module, which includes a one-dimensional convolutional path, a non-linear activation path, and a fully connected linear path. The outputs of the three paths are concatenated along the feature dimension, and then added to the input through a residual connection. Finally, after global average pooling and a fully connected layer, the module outputs the preliminary probability vector of the motion intent corresponding to the current frame.

[0010] Preferably, the specific steps for predicting the prediction intent probability of the future window in step S3 are as follows: Construct and maintain an instruction buffer queue that stores the preliminary probability vectors of motion intentions corresponding to multiple consecutive historical windows, as well as the wheelchair motion state data (including movement speed, steering angle, and seat posture) for each window. The historical probability vector sequence and motion state data sequence in the buffer queue are concatenated for features and input into a pre-trained lightweight Transformer encoder, which captures the temporal dependencies within the sequence through a self-attention mechanism. The Transformer encoder outputs a sequence of hidden states, extracts the hidden state of the last time step and inputs it into a fully connected prediction head, which outputs the probability distribution of the prediction intent for a predetermined number of future windows. The predicted intent probability distribution is output together with the initial probability vector of the current window for subsequent fusion decision-making.

[0011] Preferably, the specific steps of fusion and sample caching in step S4 are as follows: Obtain the preliminary probability vector of the current window and the probability distribution of the prediction intent of the future window, and calculate the historical prediction accuracy index and the current signal quality index; The fusion weights are dynamically calculated based on historical prediction accuracy and signal quality indicators. The current preliminary probability vector is then fused with the prediction intent probability distribution to obtain the fused comprehensive probability vector. The category corresponding to the maximum value in the comprehensive probability vector is selected as the current execution command (including forward, backward, left turn, right turn, emergency stop, seat height adjustment, and backrest adjustment), and the command is sent to the wheelchair motor controller to drive the corresponding action. If the wheelchair movement status is monitored after the command is executed, and no user intervention or safety event occurs within a preset time period, and the confidence level corresponding to the maximum value in the comprehensive probability vector is not lower than a preset threshold, then the time-series data frame corresponding to the command and its classification label are stored in the local cache as high-confidence samples.

[0012] Preferably, the specific steps for local fine-tuning and privacy-preserving uploading in step S5 are as follows: When the number of high-confidence samples in the local cache reaches a preset threshold, a batch of samples is randomly selected from the cache. These samples are used as training data to backpropagate the base part of the local hybrid model, and the gradient of the loss function with respect to the base model parameters is calculated. The base model parameters are updated one or more times using a preset small learning rate to obtain the fine-tuned base model parameters, and the difference between the base model parameters before and after fine-tuning is calculated as the model update amount. Add Laplace noise that satisfies differential privacy protection to the model update amount to obtain the noisy model update amount; The updated model with added noise is uploaded to the cloud server via a secure communication protocol, while the used samples in the local cache are cleared.

[0013] Preferably, the specific steps of cloud aggregation and distribution in step S6 are as follows: The cloud server receives updates of the noisy model uploaded from multiple wheelchair clients and records the amount of data attached to each client's upload. The cloud server calculates a weighted average weight based on the amount of data from each client, and then performs a weighted average of all noisy model update amounts to obtain the global model update amount. The updated global model parameters are obtained by adding the global model update amount to the current global base model parameters stored in the cloud. The cloud server broadcasts the updated global base model parameters to all registered wheelchair clients. Each client receives the updated parameters and replaces the base portion in its local hybrid model with them, combining them with its local personalized adaptation layer to form a new local hybrid model.

[0014] Preferably, the specific steps of acquisition and preprocessing in step S1 are as follows: Raw signals were acquired simultaneously using EEG and EMG electrodes, and bandpass filtering was performed on the raw signals to obtain filtered EEG and EMG signals. Independent component analysis was performed on the filtered signal to identify and remove components containing electrooculogram (EOG) and electrocardiogram (ECG) artifacts, resulting in pure electroencephalogram (EEG) and electromyogram (EMG) signals. The clean signal is decomposed into wavelet packets to extract the wavelet coefficients of each sub-band. The wavelet coefficients are then denoised using a soft thresholding function, and the denoised signal is reconstructed. The denoised signal is divided into a sliding window with a preset overlap rate to obtain continuous time-series data frames. Each data frame contains sampling points of all channels within the current time window.

[0015] Secondly, embodiments of the present invention provide an intelligent wheelchair control system based on the fusion of electroencephalography (EEG) and electromyography (EMG), comprising: The acquisition module is used to simultaneously acquire the user's electroencephalogram (EEG) and electromyogram (EMG) signals. After preprocessing the raw signals, the data is divided into continuous time-series data frames through a sliding window. The recognition module is used to input the current time-series data frame into the local hybrid model. The model extracts the time-frequency features of the signal through the attention mechanism and captures the spatiotemporal features based on the parallel transformation path, and outputs the preliminary probability of the motion intention corresponding to the current frame. The prediction module is used to store the preliminary probabilities of motion intentions from multiple consecutive historical windows into a buffer queue, and predict the probability of predicted intentions for future windows based on the historical intention sequence in the queue. The decision module is used to fuse the preliminary probability of the current frame with the predicted intention probability of the future window, determine the current execution instruction based on the fusion result and drive the wheelchair to execute it, and store the data frame corresponding to the execution instruction as a high-confidence sample in the local cache. The update module is used to fine-tune the base part of the local hybrid model using these samples when the number of samples in the local cache reaches a preset threshold, and calculate the model update parameters to be uploaded based on the fine-tuned model, and upload them to the cloud server after privacy protection processing. The aggregation module enables the cloud server to aggregate model update parameters uploaded by multiple clients to update the globally shared base model, and then distribute the updated global model base parameters to each client to update their local hybrid model.

[0016] Thirdly, embodiments of the present invention provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0018] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of embodiments of this application. The computer program product may be a software installation package.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: By constructing an EEG-EMG fusion control framework that combines intention trajectory prediction with federated learning, the effective utilization of the temporal continuity of motor intentions is achieved. Historical intention sequences are used to predict future commands to mitigate misjudgments caused by instantaneous signal fluctuations, improving the coherence and robustness of control commands. Simultaneously, the federated learning framework enables cross-individual model knowledge transfer while protecting the privacy of users' original data, allowing each client-side base model to continuously evolve from group usage. This solves the problem of limited model generalization ability caused by insufficient data from a single user and shortens the calibration cycle for new users. Furthermore, through local cached sample fine-tuning and differential privacy-preserving upload mechanisms, a dual-closed-loop learning architecture of local personalized adaptation and global common knowledge sharing is constructed, achieving a balance between individual difference adaptation and group collaborative optimization. Attached Figure Description

[0020] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Elements in the drawings are not necessarily to scale. The same reference numerals refer to corresponding similar parts.

[0021] Figure 1 This is an exemplary device architecture diagram in which an embodiment of the present invention can be applied; Figure 2 This is a flowchart illustrating an embodiment of the intelligent wheelchair control method based on the fusion of electroencephalography (EEG) and electromyography (EMG) of the present invention. Figure 3 This is a flowchart illustrating an embodiment of the intelligent wheelchair control method based on the fusion of electroencephalography (EEG) and electromyography (EMG) of the present invention. Figure 4 This is a schematic diagram of the modular architecture of an intelligent wheelchair control system based on the fusion of electroencephalography (EEG) and electromyography (EMG) according to an embodiment of the present invention. Figure 5 This is a schematic diagram of the structure of a computer device suitable for implementing electronic devices according to embodiments of the present invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Figure 1 An exemplary system architecture 100 is shown, in which the intelligent wheelchair control method based on EEG and EMG fusion or the intelligent wheelchair control system based on EEG and EMG fusion of embodiments of the present invention can be applied.

[0025] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0026] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0027] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., software programs or software modules used to provide distributed services) or as a single software program or software module. No specific limitations are imposed here.

[0028] Server 105 can be a server that provides various services, such as a background information processing server that processes verification request information sent by terminal devices 101, 102, and 103. The background information processing server can analyze and process the received verification request information and obtain the processing results.

[0029] It should be noted that the intelligent wheelchair control method based on EEG and EMG fusion provided in this embodiment of the invention is generally executed by server 105, and correspondingly, the intelligent wheelchair control system based on EEG and EMG fusion is generally located in server 105. Furthermore, the intelligent wheelchair control method based on EEG and EMG fusion provided in this embodiment of the invention is generally executed by terminal devices 101, 102, and 103, and correspondingly, the intelligent wheelchair control system based on EEG and EMG fusion is generally located in terminal devices 101, 102, and 103.

[0030] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (for example, to provide distributed services), or as a single software program or multiple software modules; no specific limitations are made here.

[0031] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. If the data being processed does not need to be retrieved remotely, the above architecture may not include a network, requiring only servers or terminal devices.

[0032] Figure 2 An embodiment of the present invention discloses an intelligent wheelchair control method based on the fusion of electroencephalography (EEG) and electromyography (EMG), such as... Figure 2 and Figure 3 As shown, the method includes the following steps: S1. Simultaneously acquire the user's EEG and EMG signals, preprocess the raw signals, and then divide them into continuous time-series data frames through a sliding window; S2. Input the current time-series data frame into the local hybrid model. The model extracts the time-frequency features of the signal through the attention mechanism and captures the spatiotemporal features based on the parallel transformation path, and outputs the preliminary probability of the motion intention corresponding to the current frame. S3. Store the preliminary probabilities of motion intentions from multiple consecutive historical windows into a buffer queue, and predict the probability of predicted intentions for future windows based on the historical intention sequences in the queue; wherein, the training labels of the prediction model are delayed labels based on subsequent actual execution instructions. S4. The preliminary probability of the current frame is fused with the predicted intention probability of the future window. Based on the fusion result, the current execution instruction is determined and the wheelchair is driven to execute. At the same time, the data frame corresponding to the execution instruction is stored in the local cache as a high-confidence sample. S5. When the number of samples in the local cache reaches a preset threshold, the high-confidence sample is used as training data to fine-tune the base part of the local hybrid model, and the model update parameters to be uploaded are calculated based on the fine-tuned model. After privacy protection processing, the model is uploaded to the cloud server. S6: The cloud server aggregates model update parameters uploaded by multiple clients to update the globally shared base model, and then distributes the updated global model base parameters to each client to update their respective local hybrid models.

[0033] It should be noted that the raw signals are acquired synchronously through EEG and EMG electrodes, and then bandpass filtering and independent component analysis are used to remove EEG and ECG artifacts. After wavelet packet decomposition and soft thresholding denoising, the data is reconstructed and segmented into continuous temporal data frames according to overlapping sliding windows. The current frame is input into the wavelet channel attention module, and the channel descriptor is obtained through discrete wavelet transform and global average pooling. The attention weights are output through two fully connected layers, and after being weighted with wavelet coefficients, they are reconstructed into a temporal feature map. The map is then concatenated and residually connected through a parallel transformation module (including one-dimensional convolution, nonlinear activation, and fully connected paths). Finally, the preliminary probability vector of motion intention is output through global average pooling and a fully connected layer. Simultaneously, an instruction buffer queue is constructed to store historical probability vectors and motion states. The input is a lightweight converter encoder to capture temporal dependencies. The last hidden state is taken and output through a fully connected prediction head to determine the probability distribution of the prediction intent for the future window. The current preliminary probability and the future prediction probability are obtained. The fusion weight is dynamically calculated by combining the historical prediction accuracy and the current signal quality. The weighted result is a comprehensive probability vector. The maximum value corresponds to the instruction that drives the wheelchair to execute. If there is no intervention or safety event after execution, the current data frame and label are stored as high-confidence samples in the local cache. When the cached samples reach the threshold, a batch is randomly selected and backpropagated to calculate the gradient of the local hybrid model base part. The fine-tuned parameters are obtained by iteratively updating with a small learning rate. The update amount is calculated and Laplacian noise is added. The data is then uploaded to the cloud via a security protocol, and the cache is cleared. The cloud server receives the noise-added update amount and data amount from each client, calculates the weighted average to obtain the global update amount, adds it to the global base parameters to obtain the new global base parameters, and broadcasts it to each client. The client replaces the base part and recombines it with the local personalized adaptation layer to complete the local model update.

[0034] Specifically, the user's electroencephalogram (EEG) and electromyogram (EMG) signals are collected simultaneously. After preprocessing the raw signals, they are segmented into continuous temporal data frames using a sliding window. The specific steps are as follows: Raw signals were acquired simultaneously using EEG and EMG electrodes, and bandpass filtering was performed on the raw signals to obtain filtered EEG and EMG signals. Independent component analysis was performed on the filtered signal to identify and remove components containing electrooculogram (EOG) and electrocardiogram (ECG) artifacts, resulting in pure electroencephalogram (EEG) and electromyogram (EMG) signals. The clean signal is decomposed into wavelet packets to extract the wavelet coefficients of each sub-band. The wavelet coefficients are then denoised using a soft thresholding function, and the denoised signal is reconstructed. The denoised signal is divided into a sliding window with a preset overlap rate to obtain continuous time-series data frames. Each data frame contains sampling points of all channels within the current time window.

[0035] In this embodiment, the specific steps for data acquisition and preprocessing are as follows: Raw signals were simultaneously acquired using EEG and EMG electrode arrays, with EEG signal sampling rates of 250-1000Hz and EMG signal sampling rates of 500-2000Hz. Bandpass filtering was applied to the raw signals, retaining the 1-40Hz frequency band for EEG signals and the 5-250Hz frequency band for EMG signals to remove power frequency interference. Independent component analysis (ICA) was used to identify and remove artifacts from electrooculography (EOG) and electrocardiogram (ECG) signals, resulting in a clean signal. The clean signal underwent 3-5 level wavelet packet decomposition, and the wavelet coefficients were denoised using a soft thresholding function before reconstruction. The signal was then divided into continuous time-series data frames with a 30%-70% overlap rate and a 100-300ms window size, with each data frame containing sampling points from all channels.

[0036] In one specific embodiment, independent component analysis is performed on the filtered EEG signal to identify and remove components containing electrooculogram (EOG) artifacts and electrocardiogram (ECG) artifacts, resulting in a pure EEG signal; bandpass filtering combined with wavelet denoising or adaptive filtering is applied to the filtered electromyography (EMG) signal to obtain a pure EMG signal.

[0037] To address the differences in characteristics between electroencephalogram (EEG) and electromyogram (EMG) signals, a differentiated preprocessing procedure is adopted: EEG signals are susceptible to interference from electrooculogram (EOG) and electrocardiogram (ECG) artifacts, so independent component analysis (ICA) is used to identify and remove these artifacts, which is a conventional method for EEG signal preprocessing. The main interferences to electromyography (EMG) signals are power frequency noise and motion artifacts. ICA is usually not used to remove EEG / ECG artifacts. Instead, denoising is achieved by combining bandpass filtering (5-250Hz) with wavelet denoising or adaptive filtering, which is more in line with the processing characteristics of EMG signals. Those skilled in the art can select appropriate denoising methods based on the actual signal quality. The above-mentioned differentiated processing can improve signal purity and provide a more reliable data foundation for subsequent intent recognition.

[0038] It should be noted that the raw signals are acquired simultaneously using EEG and EMG electrodes. Bandpass filtering is then applied to the raw signals to retain the target frequency band signal components, resulting in filtered EEG and EMG signals. Subsequently, independent component analysis is performed on the filtered signals. By calculating the statistical characteristics of each independent component, the components corresponding to EEG and ECG artifacts are identified. These components are then zeroed out and the signals are reconstructed to obtain clean EEG and EMG signals. Next, wavelet packet decomposition is performed on the clean signals to obtain wavelet coefficients for each sub-band. A soft threshold function is applied to each sub-band coefficient to suppress noise components. The signals are then reconstructed using inverse wavelet transform to obtain the denoised signals. Finally, the denoised signals are segmented according to a sliding window with a preset overlap rate. Each window extracts multi-channel sampling points of a fixed time length to generate continuous time-series data frames.

[0039] Specifically, the current time-series data frame is input into the local hybrid model. This model extracts the time-frequency features of the signal through an attention mechanism and captures spatiotemporal features based on a parallel transformation path, outputting the preliminary probability of the motion intent corresponding to the current frame. The specific steps are as follows: The current time-series data frame is input into the wavelet channel attention module. This module performs discrete wavelet transform on the multi-channel signal to obtain multi-scale wavelet coefficients, and performs global average pooling on the coefficients of each scale to obtain the channel descriptor. The channel descriptors are sequentially input into two fully connected layers. The first fully connected layer uses the ReLU activation function for dimensionality reduction, and the second fully connected layer uses the Sigmoid activation function for dimensionality increase, outputting an attention weight vector with the same number of channels as the original. The attention weight vector is multiplied with the wavelet coefficients by channel to obtain the weighted multi-scale features, and then the weighted features are reconstructed into a temporal feature map. The temporal feature map is input into the parallel transformation module, which includes a one-dimensional convolutional path, a non-linear activation path, and a fully connected linear path. The outputs of the three paths are concatenated along the feature dimension, and then added to the input through a residual connection. Finally, after global average pooling and a fully connected layer, the module outputs the preliminary probability vector of the motion intent corresponding to the current frame.

[0040] In a specific embodiment, the specific process of feature extraction and intent recognition in the local hybrid model is as follows: The current time-series data frame is input into the wavelet channel attention module. This module performs discrete wavelet transform on the multi-channel signal to obtain multi-scale wavelet coefficients, and performs global average pooling on the coefficients at each scale to obtain channel descriptors. The channel descriptors are then input into two fully connected layers. The first fully connected layer uses the ReLU activation function for dimensionality reduction, and the second fully connected layer uses the Sigmoid activation function for dimensionality increase, outputting an attention weight vector with the same number of channels as the original. The attention weight vector is multiplied by the wavelet coefficients by channel to obtain weighted multi-scale features, which are then reconstructed into a time-series feature map. The time-series feature map is input into the parallel transformation module, which contains a one-dimensional convolutional path, a non-linear activation path, and a fully connected linear path. The outputs of the three paths are concatenated along the feature dimension, and then added to the input through a residual connection. Finally, after global average pooling and a fully connected layer, the initial probability vector of motion intent corresponding to the current frame is output.

[0041] Among them, the kernel size of the one-dimensional convolution path is 3-7, the stride is 1-2, and the number of output feature channels is 32-128; the non-linear activation path uses ELU or GELU activation functions, combined with batch normalization; the number of hidden layer neurons in the fully connected linear path is 64-256, and L2 regularization is used to suppress overfitting.

[0042] It should be noted that the current time-series data frame is input into the wavelet channel attention module. This module first performs discrete wavelet transform on the multi-channel signal to obtain multi-scale wavelet coefficients, and then performs global average pooling on the coefficients of each scale to obtain a descriptor for aggregated channel information. Subsequently, the channel descriptor is input into two fully connected layers in sequence. The first fully connected layer uses the ReLU activation function for dimensionality reduction, and the second fully connected layer uses the Sigmoid activation function to restore the original dimension, thereby outputting the attention weight vector corresponding to each channel. Next, the attention weight vector is multiplied element-wise by the aforementioned multi-scale wavelet coefficients for each channel to obtain the weighted multi-scale features, and the weighted features are reconstructed into a time-series feature map. Finally, the time-series feature map is input into the parallel transformation module, which contains parallel one-dimensional convolutional paths, non-linear activation paths, and fully connected linear paths. The outputs of the three paths are concatenated in the feature dimension and added to the module input through residual connections. After processing by global average pooling and fully connected layers, the initial probability vector of motion intent corresponding to the current frame is finally output.

[0043] Specifically, the preliminary probabilities of motion intentions from multiple consecutive historical windows are stored in a buffer queue. Based on the historical intention sequences in this queue, the predicted intention probability for future windows is predicted. The specific steps are as follows: Construct and maintain an instruction buffer queue that stores the preliminary probability vectors of motion intentions corresponding to multiple consecutive historical windows, as well as the wheelchair motion state data corresponding to each window; The historical probability vector sequence and motion state data sequence in the buffer queue are input into a pre-trained lightweight Transformer encoder, which captures the temporal dependencies within the sequence through a self-attention mechanism. The Transformer encoder outputs a sequence of hidden states, and inputs the hidden state of its last time step into a fully connected prediction head, which outputs the probability distribution of the prediction intent for a predetermined number of future windows. The predicted intent probability distribution is output together with the initial probability vector of the current window for subsequent fusion decision-making.

[0044] In one specific embodiment, preliminary probability vectors of motion intentions corresponding to 5-20 consecutive historical windows are stored. The wheelchair motion state data includes movement speed, turning angle, and seat posture. The prediction head outputs the probability distribution of predicted intentions for the next 1-3 windows. The lightweight Transformer encoder has 2-4 layers, 2-8 attention heads, and a hidden layer dimension of 64-256, employing layer normalization and residual connections. The fully connected prediction head contains 1-2 fully connected layers with a dropout rate of 0.1-0.3, and the output dimension is consistent with the number of motion intention categories.

[0045] In this embodiment of the invention, the training of the future intent prediction model employs a delayed labeling mechanism: actual execution commands within a preset time period (e.g., 0.5-3 seconds) after the current command execution, without user intervention or security events, are used as future intent labels for the corresponding historical intent sequences, and are then used to train the prediction model. This design ensures high confidence of the labels, enabling the model to learn the temporal continuity of motion intent. Simultaneously, it forms a closed loop between the "no intervention / security event" judgment and the generation of training labels for the prediction model, verifying the effectiveness of the current command and providing reliable supervisory signals for future prediction.

[0046] It should be noted that a fixed-length instruction buffer queue is initialized. This queue stores the preliminary probability vectors of motion intentions corresponding to multiple consecutive historical windows and the wheelchair motion state data corresponding to each window, following a first-in, first-out (FIFO) principle. When a new window arrives, the latest data is pushed into the queue and the oldest data is removed to ensure that the queue content always reflects the most recent temporal information. Subsequently, the historical probability vector sequence and motion state data sequence stored in the buffer queue are combined and input into a pre-trained lightweight transformer encoder. This encoder calculates the attention weights between positions within the sequence using a self-attention mechanism, capturing the temporal dependencies and outputting the corresponding latent state sequence. Next, the latent state of the last time step in the latent state sequence output by the transformer encoder is extracted and input into a fully connected prediction head. This prediction head maps the latent state to the predicted intention probability distribution corresponding to a predetermined number of future windows through a linear transformation. Finally, the predicted intention probability distribution of the future windows output by the prediction head and the preliminary probability vector of the motion intention of the current window are output together as input data for the subsequent fusion decision module.

[0047] Specifically, the preliminary probability of the current frame is fused with the predicted intent probability of the future window. Based on the fusion result, the current execution instruction is determined and the wheelchair is driven to execute it. At the same time, the data frame corresponding to the execution instruction is stored in the local cache as a high-confidence sample. The specific steps are as follows: Obtain the preliminary probability vector of the current window and the probability distribution of the prediction intent of the future window, and calculate the historical prediction accuracy index and the current signal quality index. The fusion weights are dynamically calculated based on historical prediction accuracy and signal quality indicators. The current preliminary probability vector is then fused with the prediction intent probability distribution to obtain the fused comprehensive probability vector. The category corresponding to the maximum value in the comprehensive probability vector is selected as the current execution instruction, and this instruction is sent to the wheelchair motor controller to drive the wheelchair to perform the corresponding action; If the wheelchair movement status is monitored after the command is executed, and no user intervention or safety event occurs within a preset time period, and the confidence level corresponding to the maximum value in the comprehensive probability vector is not lower than a preset threshold, then the current time-series data frame corresponding to the command and its classification label are stored in the local cache as high-confidence samples.

[0048] In a specific embodiment, the historical prediction accuracy index (prediction-execution matching rate over the last 50-200 windows) and the current signal quality index are calculated; wherein, the signal-to-noise ratio (SNR) is defined as the ratio of target frequency band power to noise frequency band power. In this embodiment, an SNR ≥ 10dB and a signal amplitude variation coefficient ≤ 0.3 are taken as the criteria for judging a high-quality signal.

[0049] The target frequency band is the effective signal frequency band of EEG signals (1-40Hz) and EMG signals (5-250Hz), and the noise frequency band is the interference frequency band that exceeds the above effective frequency bands. The signal-to-noise ratio is obtained by calculating the power of the corresponding frequency band using the power spectral density method and taking the ratio of the two.

[0050] Historical prediction accuracy P∈[0,1], signal quality index Q∈[0,1], current preliminary probability weight W1=0.3+0.5×P×Q, future prediction intention probability weight W2=1-W1. Current execution commands include forward, backward, left turn, right turn, emergency stop, seat height adjustment, and backrest adjustment. Monitoring is conducted within 0.5-3 seconds after command execution.

[0051] In this embodiment of the invention, the determination of high-confidence samples can be further supported by dual screening conditions: 1. If no user intervention or security event occurs within a preset time period (e.g., 0.5-3 seconds) after the command is executed, it indicates that the command is executed safely and is approved by the user. 2. The confidence level corresponding to the maximum value in the fused comprehensive probability vector is not lower than the preset threshold (such as 0.7-0.8), indicating that the model's judgment of the current instruction has high reliability.

[0052] By working together under these two conditions, low-quality samples that are not entered into the cache due to inaccurate model judgments or lack of timely user intervention can be avoided, thereby improving the data quality and stability of subsequent model fine-tuning.

[0053] It should be noted that the initial probability vector of the motion intent of the current window is obtained from the recognition module, and the probability distribution of the predicted intent of the future window is obtained from the prediction module. At the same time, historical prediction matching is statistically analyzed to calculate the historical prediction accuracy index, and the signal-to-noise ratio and electrode impedance are analyzed to calculate the current signal quality index. Subsequently, using the historical prediction accuracy index and the current signal quality index as input, a fusion weight is dynamically generated through a mapping function. The current initial probability vector and the predicted intent probability distribution are weighted and summed according to the weight to obtain the fused comprehensive probability vector. Next, the category corresponding to the maximum value in the comprehensive probability vector is identified as the current execution instruction. This instruction is sent to the wheelchair motor controller through the communication interface to drive the motor to perform the corresponding movement action. Finally, the wheelchair movement status and user feedback signals after the instruction is executed are continuously monitored. If no user intervention is detected and no safety alarm is triggered within a preset time period, the instruction is determined to be safe and effective. The current time series data frame and its instruction category are stored as high-confidence samples in the local cache to provide reliable training data for subsequent model updates.

[0054] Specifically, when the number of samples in the local cache reaches a preset threshold, these samples are used to fine-tune the base part of the local hybrid model. Based on the fine-tuned model, the updated parameters of the model to be uploaded are calculated, and after privacy protection processing, the model is uploaded to the cloud server. The specific steps are as follows: When the number of high-confidence samples in the local cache reaches a preset threshold, a batch of samples is randomly selected from the cache. These samples are used as training data to backpropagate the base part of the local hybrid model, and the gradient of the loss function with respect to the base model parameters is calculated. The base model parameters are updated one or more times using a preset small learning rate to obtain the fine-tuned base model parameters, and the difference between the base model parameters before and after fine-tuning is calculated as the model update amount. Add Laplacian noise that satisfies differential privacy protection to the model update amount to obtain the noisy model update amount; The updated model with added noise is uploaded to the cloud server via a secure communication protocol, while the used samples in the local cache are cleared.

[0055] In a specific embodiment, the specific steps for local fine-tuning and privacy-preserving upload are as follows: When the number of high-confidence samples in the local cache reaches a preset threshold (50-500), 16-64 samples are randomly selected as training batches. The base part of the local hybrid model is backpropagated using the cross-entropy loss function to calculate the parameter gradient. A small learning rate (1e-5-1e-3) is used for 1-5 iterations to update the base model parameters, and the difference between the parameters before and after the fine-tuning is calculated as the model update amount. Laplacian noise is added to the model update amount to achieve differential privacy protection. The noise scale parameter is 0.01-0.1, and the privacy budget ε∈[0.1,1.0]. The noisy model update amount is uploaded to the cloud server through the TLS1.3 secure communication protocol, and the used samples in the local cache are cleared.

[0056] It should be noted that when the number of high-confidence samples in the local cache reaches a preset threshold, the system randomly selects a batch of samples from the cache as training data, performs backpropagation calculation on the base part of the local hybrid model, and obtains the gradient of the loss function with respect to the base model parameters. Subsequently, the base model parameters are iteratively updated using a preset small learning rate to obtain the fine-tuned base model parameters, and the difference between the parameters before and after fine-tuning is calculated as the model update amount. To ensure user data privacy, Laplace noise that satisfies differential privacy protection is added to the model update amount to generate a noisy model update amount. Finally, the noisy model update amount is uploaded to the cloud server through a secure communication protocol, and the used samples in the local cache are cleared simultaneously to free up storage space for the next round of data collection.

[0057] Specifically, the cloud server aggregates model update parameters uploaded by multiple clients to update the globally shared base model, and then distributes the updated global model base parameters to each client to update their local hybrid model. The specific steps are as follows: The cloud server receives updates of the noisy model uploaded from multiple wheelchair clients and records the amount of data attached to each client's upload. The cloud server calculates a weighted average weight based on the amount of data from each client, and then performs a weighted average of all noisy model update amounts to obtain the global model update amount. The updated global model parameters are obtained by adding the global model update amount to the current global base model parameters stored in the cloud. The cloud server broadcasts the updated global base model parameters to all registered wheelchair clients. Each client receives the updated parameters and replaces the base portion in its local hybrid model with them, combining them with its local personalized adaptation layer to form a new local hybrid model.

[0058] It should be noted that the cloud server receives the noisy model updates uploaded from multiple wheelchair clients via a secure communication protocol, and parses the data volume information attached to each client's communication protocol header field. These updates and their corresponding data volumes are stored and associated according to the client identifier. Subsequently, the cloud server calculates a weighted average weight based on the proportion of each client's data volume to the total data volume, and uses this weight to perform a weighted summation and average of all noisy model updates to obtain the aggregated global model update volume. After obtaining the global model update volume, the cloud server adds it element-wise to the currently stored global base model parameters to obtain the updated global base model parameters, and overwrites the old parameters stored in the cloud with the new parameters. Finally, the cloud server broadcasts the updated global base model parameters to all registered wheelchair clients. Each client receives the updated parameters and replaces the base portion in its local hybrid model, then recombines them with the locally stored personalized adaptation layer parameters to form the updated complete local hybrid model.

[0059] Further reference Figure 4 As an implementation of the methods shown in the above figures, this application provides an embodiment of an intelligent wheelchair control system based on the fusion of electroencephalography (EEG) and electromyography (EMG). This system embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0060] Secondly, embodiments of the present invention also disclose an intelligent wheelchair control system based on the fusion of electroencephalography (EEG) and electromyography (EMG), such as... Figure 4 As shown, it includes: a data acquisition module 41, an identification module 42, a prediction module 43, a decision-making module 44, an update module 45, and an aggregation module 46.

[0061] The acquisition module 41 is used to simultaneously acquire the user's electroencephalogram (EEG) and electromyogram (EMG) signals, and after preprocessing the raw signals, it is divided into continuous time-series data frames through a sliding window. The recognition module 42 is used to input the current time-series data frame into the local hybrid model. The model extracts the time-frequency features of the signal through the attention mechanism and captures the spatiotemporal features based on the parallel transformation path, and outputs the preliminary probability of the motion intention corresponding to the current frame. Prediction module 43 is used to store the preliminary probability of motion intention in multiple consecutive historical windows into a buffer queue, and predict the probability of predicted intention in future windows based on the historical intention sequence in the queue. The decision module 44 is used to fuse the preliminary probability of the current frame with the predicted intention probability of the future window, determine the current execution instruction based on the fusion result and drive the wheelchair to execute it, and store the data frame corresponding to the execution instruction as a high-confidence sample in the local cache. The update module 45 is used to fine-tune the base part of the local hybrid model using these samples when the number of samples in the local cache reaches a preset threshold, and calculate the model update parameters to be uploaded based on the fine-tuned model, and upload them to the cloud server after privacy protection processing. The aggregation module 46 is used to enable the cloud server to aggregate model update parameters uploaded by multiple clients to update the globally shared base model, and to distribute the updated global model base parameters to each client to update their local hybrid model.

[0062] The functions and methods of the above modules correspond to each other, and will not be repeated here.

[0063] In summary, this invention improves the coherence and robustness of wheelchair control, shortens the calibration cycle for new users, and enables continuous model evolution under privacy protection, making it suitable for autonomous mobility assistance for people with limb disabilities and movement disorders. The embodiments of this invention utilize an EEG-EMG fusion control framework that combines intention trajectory prediction and federated learning, effectively utilizing the temporal continuity of movement intentions. It uses historical intention sequences to predict future commands to mitigate misjudgments caused by instantaneous signal fluctuations, thus improving the coherence and robustness of control commands. Simultaneously, the federated learning framework enables cross-individual model knowledge transfer while protecting the privacy of users' original data, allowing each client-side base model to continuously evolve from group usage. This solves the problem of limited model generalization ability caused by insufficient data from a single user, shortening the calibration cycle for new users. Furthermore, through local cached sample fine-tuning and a differential privacy-preserving upload mechanism, a dual-closed-loop learning architecture of local personalized adaptation and global common knowledge sharing is constructed, achieving a balance between individual difference adaptation and group collaborative optimization.

[0064] The following is for reference. Figure 5 It illustrates an electronic device suitable for implementing embodiments of the present invention (e.g., Figure 1 A schematic diagram of the structure of a computer device 500 (shown as a server or terminal device). Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0065] like Figure 5As shown, the computer device 500 includes a central processing unit (CPU) 501 and a graphics processing unit (GPU) 502, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 503 or programs loaded from storage section 509 into random access memory (RAM) 504. The RAM 504 also stores various programs and data required for the operation of the device 500. The CPU 501, GPU 502, ROM 503, and RAM 504 are interconnected via a bus 505. An input / output (I / O) interface 506 is also connected to the bus 505.

[0066] The following components are connected to I / O interface 506: an input section 507 including a keyboard, mouse, etc.; an output section 508 including a liquid crystal display (LCD) and speakers, etc.; a storage section 509 including a hard disk, etc.; and a communication section 510 including a network interface card such as a LAN card and a modem, etc. The communication section 510 performs communication processing via a network such as the Internet. A drive 511 may also be connected to I / O interface 506 as needed. A removable medium 512, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 511 as needed so that computer programs read from it can be installed into storage section 509 as needed.

[0067] In particular, according to embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 510, and / or installed from removable medium 512. When the computer program is executed by central processing unit (CPU) 501 and graphics processing unit (GPU) 502, the functions defined in the methods of this invention are performed.

[0068] It should be noted that the computer-readable medium described in this invention can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than a computer-readable medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0069] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using dedicated hardware-based devices that perform the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0071] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be located in a processor.

[0072] In another aspect, the present invention also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: synchronously acquire the user's electroencephalogram (EEG) and electromyogram (EMG) signals; preprocess the raw signals and then segment them into continuous temporal data frames using a sliding window; input the current temporal data frame into a local hybrid model, which extracts the time-frequency features of the signal through an attention mechanism and captures spatiotemporal features based on a parallel transformation path, outputting the preliminary probability of the motion intent corresponding to the current frame; store the preliminary probabilities of the motion intent of multiple consecutive historical windows into a buffer queue, and predict the predicted intent probability of future windows based on the historical intent sequence in the queue; and fuse the preliminary probabilities of the current frame. The probability and the predicted intent probability of the future window are fused together to determine the current execution command and drive the wheelchair to execute. At the same time, the data frame corresponding to the execution command is stored in the local cache as a high-confidence sample. When the number of samples in the local cache reaches a preset threshold, the base part of the local hybrid model is fine-tuned using the high-confidence sample as training data. The model update parameters to be uploaded are calculated based on the fine-tuned model and uploaded to the cloud server after privacy protection processing. The cloud server aggregates the model update parameters uploaded by multiple clients to update the globally shared base model and distributes the updated global model base parameters to each client to update their respective local hybrid models.

[0073] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A method for controlling an intelligent wheelchair based on the fusion of electroencephalography (EEG) and electromyography (EMG), characterized in that, The method includes the following steps: S1. Simultaneously acquire the user's EEG and EMG signals, preprocess the raw signals, and then divide them into continuous time-series data frames through a sliding window; S2. Input the current time-series data frame into the local hybrid model. The model extracts the time-frequency features of the signal through the attention mechanism and captures the spatiotemporal features based on the parallel transformation path, and outputs the preliminary probability of the motion intention corresponding to the current frame. S3. Store the preliminary probabilities of motion intentions from multiple consecutive historical windows into a buffer queue, and predict the probability of predicted intentions for future windows based on the historical intention sequences in the queue; wherein, the training labels of the prediction model are delayed labels based on subsequent actual execution instructions. S4. The preliminary probability of the current frame is fused with the predicted intention probability of the future window. Based on the fusion result, the current execution instruction is determined and the wheelchair is driven to execute. At the same time, the data frame corresponding to the execution instruction is stored in the local cache as a high-confidence sample. S5. When the number of samples in the local cache reaches a preset threshold, the high-confidence sample is used as training data to fine-tune the base part of the local hybrid model, and the model update parameters to be uploaded are calculated based on the fine-tuned model. After privacy protection processing, the model is uploaded to the cloud server. S6: The cloud server aggregates model update parameters uploaded by multiple clients to update the globally shared base model, and then distributes the updated global model base parameters to each client to update their respective local hybrid models.

2. The intelligent wheelchair control method according to claim 1, characterized in that, The specific working process of the local hybrid model in step S2 is as follows: The current time-series data frame is input into the wavelet channel attention module. This module performs discrete wavelet transform on the multi-channel signal to obtain multi-scale wavelet coefficients, and performs global average pooling on the coefficients of each scale to obtain the channel descriptor. The channel descriptors are sequentially input into two fully connected layers. The first fully connected layer uses the ReLU activation function for dimensionality reduction, and the second fully connected layer uses the Sigmoid activation function for dimensionality increase, outputting an attention weight vector with the same number of channels as the original. The attention weight vector is multiplied with the wavelet coefficients by channel to obtain the weighted multi-scale features, and then the weighted features are reconstructed into a temporal feature map. The temporal feature map is input into the parallel transformation module, which includes a one-dimensional convolutional path, a non-linear activation path, and a fully connected linear path. The outputs of the three paths are concatenated along the feature dimension, and then added to the input through a residual connection. Finally, after global average pooling and a fully connected layer, the module outputs the preliminary probability vector of the motion intent corresponding to the current frame.

3. The intelligent wheelchair control method according to claim 1, characterized in that, The specific steps for predicting the prediction intent probability of the future window in step S3 are as follows: Construct and maintain an instruction buffer queue that stores the preliminary probability vectors of motion intentions corresponding to multiple consecutive historical windows, as well as the wheelchair motion state data corresponding to each window; The historical probability vector sequence and motion state data sequence in the buffer queue are concatenated for features and input into a pre-trained lightweight Transformer encoder, which captures the temporal dependencies within the sequence through a self-attention mechanism. The Transformer encoder outputs a sequence of hidden states, extracts the hidden state of the last time step and inputs it into a fully connected prediction head, which outputs the probability distribution of the prediction intent for a predetermined number of future windows. The predicted intent probability distribution is output together with the initial probability vector of the current window for subsequent fusion decision-making.

4. The intelligent wheelchair control method according to claim 1, characterized in that, The specific steps of fusion and sample caching in step S4 are as follows: Obtain the preliminary probability vector of the current window and the probability distribution of the prediction intent of the future window, and calculate the historical prediction accuracy index and the current signal quality index; Based on the historical prediction accuracy index and signal quality index, the fusion weight is dynamically calculated, and the current preliminary probability vector is weighted and fused with the prediction intention probability distribution to obtain the fused comprehensive probability vector. The category corresponding to the maximum value in the comprehensive probability vector is selected as the current execution instruction, and this instruction is sent to the wheelchair motor controller to drive the corresponding action; If the wheelchair movement status is monitored after the command is executed, and no user intervention or safety event occurs within a preset time period, and the confidence level corresponding to the maximum value in the comprehensive probability vector is not lower than a preset threshold, then the time-series data frame corresponding to the command and its classification label are stored in the local cache as high-confidence samples.

5. The intelligent wheelchair control method according to claim 1, characterized in that, The specific steps for local fine-tuning and privacy-preserving upload in step S5 are as follows: When the number of high-confidence samples in the local cache reaches a preset threshold, a batch of samples is randomly selected from the cache. These samples are used as training data to backpropagate the base part of the local hybrid model, and the gradient of the loss function with respect to the base model parameters is calculated. The base model parameters are updated one or more times using a preset small learning rate to obtain the fine-tuned base model parameters, and the difference between the base model parameters before and after fine-tuning is calculated as the model update amount. Add Laplacian noise that satisfies differential privacy protection to the model update amount to obtain the noisy model update amount; The updated model with added noise is uploaded to the cloud server via a secure communication protocol, while the used samples in the local cache are cleared.

6. The intelligent wheelchair control method according to claim 1, characterized in that, The specific steps for cloud aggregation and distribution in step S6 are as follows: The cloud server receives updates of the noisy model uploaded from multiple wheelchair clients and records the amount of data attached to each client's upload. The cloud server calculates a weighted average weight based on the amount of data from each client, and then performs a weighted average of all noisy model update amounts to obtain the global model update amount. The updated global model parameters are obtained by adding the global model update amount to the current global base model parameters stored in the cloud. The cloud server broadcasts the updated global base model parameters to all registered wheelchair clients. Each client receives the updated parameters and replaces the base portion in its local hybrid model with them, combining them with its local personalized adaptation layer to form a new local hybrid model.

7. The intelligent wheelchair control method according to claim 1, characterized in that, The specific steps for data acquisition and preprocessing in step S1 are as follows: Raw signals were acquired simultaneously using EEG and EMG electrodes, and bandpass filtering was performed on the raw signals to obtain filtered EEG and EMG signals. Independent component analysis was performed on the filtered signal to identify and remove components containing electrooculogram (EOG) and electrocardiogram (ECG) artifacts, resulting in pure electroencephalogram (EEG) and electromyogram (EMG) signals. The clean signal is decomposed into wavelet packets to extract the wavelet coefficients of each sub-band. The wavelet coefficients are then denoised using a soft thresholding function, and the denoised signal is reconstructed. The denoised signal is divided into a sliding window with a preset overlap rate to obtain continuous time-series data frames. Each data frame contains sampling points of all channels within the current time window.

8. An intelligent wheelchair control system based on the fusion of electroencephalography (EEG) and electromyography (EMG), characterized in that, include: The acquisition module is used to simultaneously acquire the user's electroencephalogram (EEG) and electromyogram (EMG) signals. After preprocessing the raw signals, the data is divided into continuous time-series data frames through a sliding window. The recognition module is used to input the current time-series data frame into the local hybrid model. The model extracts the time-frequency features of the signal through the attention mechanism and captures the spatiotemporal features based on the parallel transformation path, and outputs the preliminary probability of the motion intention corresponding to the current frame. The prediction module is used to store the preliminary probabilities of motion intentions from multiple consecutive historical windows into a buffer queue, and predict the probability of predicted intentions for future windows based on the historical intention sequence in the queue. The decision module is used to fuse the preliminary probability of the current frame with the predicted intention probability of the future window, determine the current execution instruction based on the fusion result and drive the wheelchair to execute it, and at the same time store the data frame corresponding to the execution instruction as a high-confidence sample in the local cache. The update module is used to fine-tune the base part of the local hybrid model using these samples when the number of samples in the local cache reaches a preset threshold, and calculate the model update parameters to be uploaded based on the fine-tuned model, and upload them to the cloud server after privacy protection processing. The aggregation module enables the cloud server to aggregate model update parameters uploaded by multiple clients to update the globally shared base model, and then distribute the updated global model base parameters to each client to update their local hybrid model.

9. An electronic device, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent wheelchair control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the intelligent wheelchair control method as described in any one of claims 1 to 7.

Images