Dual-mode control method for a robotic dog based on myoelectric gestures and SSVEP

Abstract

The present invention discloses a dual-mode control method for a mechanical dog based on electromyographic gestures and SSVEP. The method first collects environmental data and performs target recognition. When a predetermined target is recognized, the robot enters an alert mode; otherwise, it enters a detection mode. In the detection mode, electromyographic signals are collected for gesture recognition, and the gesture recognition results are converted into control instructions to control the mechanical dog to perform the corresponding operation. In the alert mode, SSVEP signals are collected for recognition, and the recognition results are converted into control instructions to control the mechanical dog to perform the corresponding operation. The present invention provides two control modes that can be adapted to different scenarios. The operator can flexibly select which one to use based on the actual task scenario without the need for complex switching operations, greatly improving operational flexibility.

Description

Technical Field

[0001] The present application belongs to the field of mechanical dog control technology, and specifically relates to a dual-mode control method for a mechanical dog based on electromyographic gestures and SSVEP. Background Art

[0002] Robotic dogs demonstrate enormous potential for application in a variety of environments. However, traditional control methods, such as remote controls and voice commands, often lack sufficient accuracy and responsiveness in complex environments, making them difficult to meet rapidly changing demands. In recent years, bioelectrical signal control has become a research hotspot in the field of human-computer interaction. This technology collects, analyzes, and decodes physiological signals from the human body to ultimately control the device, offering advantages such as fast response and high discreetness. Steady-state visual evoked potentials (SSVEPs) induce specific EEG signals through light stimulation at varying frequencies, providing a control method that requires no complex training. Electromyography (EMG) captures electrical signals generated by muscle activity, enabling rapid recognition of gestures and real-time control of the device. These two technologies have played a significant role in enhancing the operational convenience of robotic dogs.

[0003] However, existing physiological electrical signal control technologies still face numerous challenges in practical application. SSVEP signals are susceptible to environmental noise and electromagnetic interference in complex environments, and prolonged viewing can cause visual fatigue. EMG signals can become unstable due to muscle fatigue. Existing signal processing algorithms and stimulation paradigms still lack real-time performance and anti-interference capabilities. Furthermore, a single control method struggles to meet the demands of complex environments. Summary of the Invention

[0004] The purpose of this application is to provide a dual-mode control method for a mechanical dog based on electromyographic gestures and SSVEP, so as to solve the problems of the existing mechanical dog control system, such as the single control mode and insufficient flexibility, the poor stability of SSVEP signals in complex environments, and the influence of electromyographic signals on user actions and environmental factors.

[0005] In order to achieve the above objectives, the technical solutions of this application are as follows:

[0006] A dual-mode control method for a mechanical dog based on myoelectric gestures and SSVEP, comprising:

[0007] Collect environmental data and perform target recognition. When the predetermined target is recognized, it enters the alert mode, otherwise it enters the detection mode;

[0008] In detection mode, it collects electromyographic signals for gesture recognition and converts the gesture recognition results into control instructions to control the robot dog to perform corresponding operations;

[0009] In the alert mode, the SSVEP signal is collected for identification and the identification result is converted into a control instruction to control the robot dog to perform the corresponding operation.

[0010] Furthermore, the collecting of environmental data and performing target identification includes:

[0011] Collect environmental images and use the YOLOv5 target detection model to identify targets.

[0012] Furthermore, in the detection mode, collecting electromyographic signals for gesture recognition includes:

[0013] Preprocess the collected electromyographic signals;

[0014] The preprocessed electromyographic signals are input into the deep learning neural network model for gesture recognition.

[0015] Furthermore, the deep learning neural network model is a spatiotemporal feature fusion learning network, including a cyclic multi-scale convolution module, a spatiotemporal graph convolution module, a cross attention module and a classification module;

[0016] The cyclic multi-scale convolution module includes a multi-scale convolution sub-module and a convolutional bidirectional long short-term memory neural network sub-module, and performs the following operations:

[0017] The multi-scale convolution submodule uses different convolution kernels to perform multi-scale operations on the pre-processed EMG signals to generate three short-term time features;

[0018] Then the three short-term time features are passed through the convolutional bidirectional long short-term memory neural network submodule to obtain the long-term time features;

[0019] Concatenate short-term time features and long-term time features into long-term and short-term features;

[0020] The spatiotemporal graph convolution module processes the preprocessed EMG signals, including multiple spatiotemporal graph convolution layers connected in sequence. Each layer adds a temporal convolution block and a residual connection on the basis of the traditional graph convolution network (GCN). The spatiotemporal graph convolution layer performs the following operations:

[0021] The input features are used to extract spatial features through the graph convolutional network GCN;

[0022] The spatial features are then temporally convolved through the temporal convolution block and then residually connected with the input features.

[0023] Furthermore, in the alert mode, a stimulus coding method of brightness modulation and radial scaling frequency modulation intermodulation was used to induce SSVEP signals.

[0024] Furthermore, the brightness modulation is encoded based on a sine wave function, and its calculation formula is:

[0025]

[0026] in, is the brightness modulation frequency, is the current frame index, is the screen refresh rate, For the The brightness value of the frame;

[0027] The radial scaling frequency modulation follows the following function:

[0028]

[0029] in, is the motion modulation frequency, It is The zoom radius of the frame;

[0030] When the brightness modulation frequency and motion modulation frequency When working together, the intermodulation frequency (IMF) is generated based on the combination of these two:

[0031]

[0032] in, is the brightness modulation frequency, is the motion modulation frequency, and The integers 1 and 2 respectively.

[0033] Furthermore, the collected SSVEP signals are identified using an attention-enhanced spatiotemporal network, which includes a spatial attention module, a temporal attention module, a temporal processing module, and a classification module.

[0034] Furthermore, the spatial attention module includes a convolution layer, a batch normalization layer, an activation layer, an ECA module and a random inactivation layer; the temporal attention module includes a convolution layer, a batch normalization layer, an activation layer, an ECA module and a random inactivation layer.

[0035] This application proposes a dual-mode control method for a mechanical dog based on electromyographic gestures and SSVEP, which has the following beneficial effects:

[0036] 1. Two control modes are provided to adapt to different scenarios. In detection mode, the operator utilizes electromyographic (ESV) signal-driven control technology to silently command the robot dog, thereby reducing exposure risk and improving the safety of detection missions. Precise control ensures the robot dog's efficient execution of tasks. In alert mode, the SSVEP signal control method is covert, and the operator only needs to focus on the visual stimulus, reducing the risk of detection and improving the safety of alert missions. It can also quickly and accurately control the robot dog's movements, improving mission efficiency. The operator can flexibly choose the mode based on the actual mission scenario, eliminating the need for complex switching operations, greatly improving operational flexibility.

[0037] 2. Signal processing algorithms for both modes are proposed. In detection mode, the electromyographic (EMG) signals are processed using a fusion network model that integrates both long-term and short-term temporal and spatial characteristics of the signals, enabling high-precision gesture recognition and significantly improving control performance. In alert mode, the SSVEP (Sequential Visual Response Emission Program) (SSVEP) signals are processed using a network model based on an attention mechanism. This allows for efficient identification of the operator's visual attention state, ensuring the stability and responsiveness of the robot dog's movements. Furthermore, reliable signal acquisition equipment, such as high-precision EMG bracelets in detection scenarios and highly sensitive EEG devices in alert scenarios, provide a solid foundation for the system's stable operation.

[0038] 3. A stimulus coding method based on brightness and radial scaling frequency intermodulation is adopted to enhance robustness and user comfort in outdoor environments, and promote technological innovation in this field.

[0039] This application combines multiple advantages, demonstrating outstanding operational flexibility, precision, mission safety and efficiency, and innovative bio-signal processing technology. These advantages make it valuable for applications such as detection and surveillance, laying a solid foundation for the future development of human-machine collaborative control of robotic dogs, and is expected to be widely used in more complex environments and mission scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the dual-mode control method of the mechanical dog based on myoelectric gestures and SSVEP in this application.

[0041] Figure 2 Schematic diagram of the spatiotemporal feature fusion learning network structure for electromyographic signal recognition.

[0042] Figure 3 Schematic diagram of the attention-enhanced spatiotemporal network structure for SSVEP signal identification. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0044] This application provides a dual-mode control method and system for a robotic dog based on electromyographic gestures and SSVEP. This system employs a stimulus coding method that synchronizes brightness and radial scaling modulation to enhance the stability and anti-interference capability of the SSVEP signal. Furthermore, a fusion network model incorporating cross-attention is designed to effectively extract signal features and implement gesture recognition. Furthermore, this dual-mode control approach significantly improves the robotic dog's operational efficiency and responsiveness under changing conditions, reduces control costs, enhances system flexibility, and reduces computational effort, ensuring its ability to effectively perform various tasks.

[0045] One embodiment of the present application, such as Figure 1 As shown, a dual-mode control method for a mechanical dog based on myoelectric gestures and SSVEP is provided, including:

[0046] Collect environmental data and perform target recognition. When the predetermined target is recognized, it enters the alert mode, otherwise it enters the detection mode;

[0047] In detection mode, it collects electromyographic signals for gesture recognition and converts the gesture recognition results into control instructions to control the robot dog to perform corresponding operations;

[0048] In the alert mode, the SSVEP signal is collected for identification and the identification result is converted into a control instruction to control the robot dog to perform the corresponding operation.

[0049] In this embodiment, depending on whether the predetermined target is recognized, different modes are adopted to control the robot dog to perform corresponding operations, including forward, backward, stop, turn, etc., to ensure the reliable execution of the task.

[0050] To identify targets in an environment, visual recognition methods are usually used, that is, using a camera to capture video images and then identifying the target in the image to determine whether the intended target has been recognized. Alternatively, auditory recognition methods can be used, that is, collecting sounds from the surrounding environment to identify the target.

[0051] After identifying the target, it is necessary to determine whether it is the intended target. If it is not the intended target, it means that the current situation requires continued detection, that is, the detection mode is executed; if it is the intended target, it means that the current situation requires alert, and the alert mode is executed.

[0052] It should be noted that the predetermined target can be a specific person, a specific object, etc. As long as the predetermined target is identified, it is considered that an emergency is detected and it is necessary to switch to the alert mode.

[0053] This embodiment controls the robot dog in two modes. The first mode is the detection mode, which is to perform control by gesture recognition through electromyographic signals; the second mode is the alert mode, which is to perform control by SSVEP signal recognition.

[0054] A specific embodiment of this application is described using visual recognition as an example, and acoustic recognition is not described in detail here. For visual recognition, an image acquisition device such as a camera is required to capture an image of the environment, and then target recognition is performed based on the image. Target recognition can use the YOLOv5 target detection model to identify the target, or other target detection models can be used, which are not described in detail here.

[0055] The YOLOv5 object detection model takes the input image Divided into The grid is used to predict whether each grid contains an object, as well as the location and category of the object. is the number of grids that the image is divided into in each dimension. Each grid cell It will output a prediction value containing confidence, bounding box coordinates and category probability. It can be expressed as:

[0056] (1)

[0057] in, is the confidence level, which indicates the probability of whether the grid contains the target; is the offset of the bounding box center relative to the grid, are the width and height of the bounding box; Is the probability of each category, indicating the possibility that the target within the grid belongs to different categories. In order to ensure that the bounding box coordinates Adapting to the image size, YOLO v5 normalizes these outputs and uses the activation function Make sure the predicted value is in the range [0,1]. The position and size of the bounding box are calculated using the following formula:

[0058] (2)

[0059] (3)

[0060] (4)

[0061] (5)

[0062] in, and are the width and height of the input image. represents the confidence of the bounding box, Indicates that when a given bounding box contains an object, the object belongs to each conditional probability distribution. The function ensures The output value of is in the range [0,1], and The function normalizes the category probabilities to ensure that the sum of the probabilities of all categories is 1.

[0063] In this way, YOLO v5 can effectively locate and classify objects in images. Identified object information, including the object's bounding box coordinates and category probability, is transmitted to the visualization interface in real time, providing operators with accurate environmental perception information and ensuring that tasks can be completed efficiently even in complex environments.

[0064] In a specific embodiment of the present application, in detection mode, collecting electromyographic signals for gesture recognition includes:

[0065] Step 1.1: Preprocess the collected electromyographic signals.

[0066] In detection mode, this embodiment uses an electromyographic wristband with a built-in 8-channel high-sensitivity electromyographic sensor, a 9-axis motion sensor, and Bluetooth BLE 4.2 modules. It can capture bioelectric changes and arm movement data generated by the user's arm muscle movements. The technology used by myoelectric wristbands to capture the operator's electromyographic signals is a relatively mature technology in this field and will not be elaborated here.

[0067] To preprocess the collected EMG signals, a first-order Butterworth low-pass filter with a cutoff frequency of 1 Hz was used to remove high-frequency noise and extract the signal envelope. The signals were then smoothed using a moving average method to mitigate the effects of random noise, with a smoothing factor of 10. Finally, the EMG signals were segmented based on the model input using a sliding window technique with a length of 200 milliseconds and a step size of 25 milliseconds.

[0068] Step 1.2: Input the preprocessed electromyographic signal into the deep learning neural network model for gesture recognition.

[0069] This embodiment uses a deep learning neural network to perform gesture recognition. Currently, there are several specific network structures of deep learning neural networks for gesture recognition, which will not be described in detail here.

[0070] In a specific embodiment, the deep learning neural network model adopts a spatiotemporal feature fusion learning network, such as Figure 2As shown, it includes a recurrent multi-scale convolution module (RMCM), a spatiotemporal graph convolution module with spatial partitioning strategy (STGCM-SPS), a crisscross attention module (CAFM) and a classification module.

[0071] The cyclic multi-scale convolution module of this embodiment includes a multi-scale convolution sub-module and a convolutional bidirectional long short-term memory neural network sub-module (CNN-BiLSTM).

[0072] In the cyclic multi-scale convolution module, the pre-processed signal is processed by the multi-scale convolution submodule. Perform primary feature extraction, where Indicates the number of sampling samples, is the length of time, The multi-scale convolution submodule uses different convolution kernels to perform multi-scale operations on the pre-processed EMG signal to generate three short-term time features: ,in Represents the convolution output dimension.

[0073] The specific feature extraction process is as follows:

[0074] (6)

[0075] in, represents the convolution operation, Represent the convolution kernel size respectively.

[0076] The innovation of the multi-scale convolution submodule is that it can simultaneously capture the short-term changes and long-term dependency characteristics of the signal through multi-scale convolution operations, which is crucial for the dynamic changes of EMG signals.

[0077] Subsequently, short-term time characteristics It is input into the convolutional bidirectional long short-term memory neural network submodule to capture the long-term temporal characteristics of the signal The convolutional bidirectional long short-term memory neural network (CNN-BiLSTM) is a multi-input, single-output network model. Its core lies in the ingenious combination of CNN's local feature extraction capabilities and BiLSTM's sequence modeling capabilities. We will not go into details here.

[0078] Then, through the splicing operation, the short-term time features and the long-term time features are fused into long-term and short-term time features. :

[0079] (7)

[0080] in, Represents a splicing operation.

[0081] The spatiotemporal graph convolution module in this embodiment processes the preprocessed EMG signals. It includes multiple spatiotemporal graph convolution layers connected in sequence. Each layer adds a temporal convolutional network (TCN) and residual connections to the traditional graph convolutional network (GCN). The spatiotemporal graph convolution layer performs the following operations:

[0082] The input features are used to extract spatial features through the graph convolutional network GCN;

[0083] The spatial features are then temporally convolved through the temporal convolution block and then residually connected with the input features.

[0084] Specifically, an undirected weighted graph based on the functional connectivity between preprocessed EMG signal channels is constructed. ,in is a node set, is the adjacency matrix. The edge weights in an undirected weighted graph are defined by the Pearson correlation coefficient (PCC), which is calculated as:

[0085] (8)

[0086] in, For nodes and The covariance of the nodes is the channels of myoelectric signal acquisition. and Represents nodes respectively and The standard deviation of . The value range of is 0~1, where Indicates that there is no correlation between nodes. It means perfect correlation.

[0087] In addition, according to the set threshold Determine the adjacency matrix Elements in:

[0088] (9)

[0089] Based on the adjacency matrix , calculate the normalized Laplace matrix , the formula is:

[0090] (10)

[0091] in, is the degree matrix, whose diagonal elements are the degrees of the nodes, defined as: .

[0092] Then the Laplace matrix is transformed into and input signal Processing and extraction of spatial features , and its update formula is:

[0093] (11)

[0094] in, is the input feature, is the weight matrix, is the activation function, is the time dimension, is the number of nodes, Represents the output dimension. This not only extracts the relationship between nodes in the spatial dimension, but also ensures that the model can focus on deeper spatial feature relationships.

[0095] Then, temporal convolution and residual connection are performed to obtain the spatial features output by the spatiotemporal graph convolution layer:

[0096] (12)

[0097] The input is, is the ReLU activation function, Represents the output dimension, which ensures the efficiency and accuracy of the model in gesture recognition.

[0098] In one example, the spatiotemporal graph convolution module includes nine layers of spatiotemporal graph convolution layers connected sequentially, which are used to transform the preprocessed EMG signals into The spatial features are obtained in the output of each layer. The input is sent to the next layer for further processing.

[0099] The cross attention module of this embodiment uses a linear projection layer to transform long-term and short-term time features and spatial characteristics Reshaped into and To adapt to the input requirements of the cross attention mechanism. The purpose of this step is to map features of different dimensions to the same feature space for better feature fusion and information interaction. and Generate a query ( ),key( ) and value ( ):

[0100] (13)

[0101] in, are the linear projection weight matrices for query, key, and value, respectively, Represents the feature dimension. They are adjusted according to the dimension of the feature and the dimension of the embedding space to ensure the effectiveness of data transformation.

[0102] Next, the resulting query, key, and value are used to calculate the cross-attention matrix :

[0103] (14)

[0104] in, The function is used to normalize the attention weights to ensure that their sum is 1. is the dimension of the embedding space. Normalizing the attention weights ensures uniform distribution and accuracy. This enables the model to maintain the complexity of the input features while highlighting the most important features for the prediction task.

[0105] After the cross attention matrix is calculated, the number of layers of CAFM is set to , the final cross-attention fusion feature is obtained by the following formula:

[0106] (15)

[0107] (16)

[0108] in, Representation layer normalization, It is a feedforward network implemented by a linear layer with an output dimension of At this stage, the output of each layer is normalized to eliminate the internal covariate shift problem that may occur during training and enhance the training stability of the model.

[0109] The classification module of this embodiment will Input into the classification module, through convolution and pooling operations, the final high-level features are obtained , and finally passed to the Softmax layer to get the predicted label :

[0110] (17)

[0111] in, is the convolution operation, is the average pooling operation.

[0112] This embodiment also converts gesture recognition results into control commands. The gForcePro+ myoelectric wristband captures the operator's electromyographic signals, which are decoded by STCFF-Net and directly output as digital codes corresponding to the gesture category (e.g., "clenched fist" is output as "01," "open palm" is output as "02," and so on). These digital codes are transmitted in real time to the robot dog's control system via a Wi-Fi wireless communication link. Upon receiving the digital codes, the robot dog interprets and executes the corresponding action commands (e.g., "01" for forward movement, "02" for stop, etc.), thus achieving efficient, real-time human-machine collaborative control, driving the robot dog to perform actions such as forward movement, backward movement, stopping, and turning.

[0113] It should be noted that the spatiotemporal feature fusion learning network in this embodiment integrates short-term temporal features (via a recurrent multi-scale convolution module), long-term temporal features (via a CNN-BiLSTM module), and spatial features (via a spatiotemporal graph convolution module). It also introduces a cross-attention mechanism (CAFM) to optimize feature fusion. This multi-scale spatiotemporal feature fusion significantly improves the accuracy of myoelectric gesture recognition, especially in complex environments. The cross-attention mechanism effectively suppresses noise interference, ensuring the stability of gesture control and enhancing anti-interference capabilities. The optimized network structure reduces computational complexity and enables low-latency real-time control.

[0114] In alert mode, the robot dog's movements are controlled using SSVEP signals. Specifically, a display is used as the visual stimulus presentation device. A flashing letter or number is displayed in the center of the display as a stimulus target. The brightness and size of the stimulus target vary at a specific frequency. Brightness modulation causes the brightness of the stimulus target to periodically change between 0 and 1, while radial scaling modulation adjusts the size of the stimulus target to simulate near and far motion, creating a visual "breathing" effect. When the operator focuses on the flashing stimulus target in the center of the display, their visual system is stimulated, inducing SSVEP signals.

[0115] In order to efficiently induce a stable signal, this application proposes a stimulation coding method of intermodulation between brightness modulation and radial scaling frequency modulation, where the brightness modulation part is encoded based on a sine wave function, and its calculation formula is:

[0116] (18)

[0117] in, is the brightness modulation frequency, is the current frame index, is the screen refresh rate. The brightness range is 0-1, For the The brightness value of the frame.

[0118] The radial scaling frequency modulation of the stimulus target ranged from 0.2 Hz to 3.4 Hz with a step size of 0.2 Hz, and the frequency modulation followed the function:

[0119] (19)

[0120] in, is the motion modulation frequency, It is The zoom radius of the frame. By using radial zoom modulation to simulate the target's near and far movement, it creates a visual "breathing" effect, thereby stimulating specific EEG signal frequencies.

[0121] When the brightness modulation frequency and motion modulation frequency When working together, an intermodulation frequency (IMF) is generated based on the combination of these two:

[0122] (20)

[0123] in, is the brightness modulation frequency, is the motion modulation frequency, and are integers 1 and 2 respectively. The generation of intermodulation frequencies enables the visual stimulation signal to be modulated in multiple frequency dimensions, thereby enhancing the strength and stability of the SSVEP signal.

[0124] This embodiment employs an SSVEP stimulation encoding method that intermodulates brightness with radial scaling frequency. The intermodulation frequency excites SSVEP signals across multiple frequency bands, enhancing signal detectability. This combined modulation scheme reduces signal interference from outdoor lighting and motion noise. Breathing visual stimulation reduces fatigue from prolonged gaze.

[0125] The SSVEP signal is then preprocessed, including three key steps: filtering, amplification, and denoising and smoothing, to ensure signal stability and quality and improve the accuracy of subsequent control. First, the SSVEP is band-pass filtered from 8 to 30 Hz to extract the effective frequency band information in the signal:

[0126] (twenty one)

[0127] in, represents the input raw EEG signal, is the impulse response function of the bandpass filter, is the output signal after filtering. The filter retains the important frequency band information in the signal through convolution operation, while reducing the interference of other frequency components.

[0128] Next, the filtered signal is gain-adjusted to further enhance the signal strength. The calculation formula for gain adjustment is:

[0129] (twenty two)

[0130] in, is the gain factor of the amplifier, is the filtered signal, is the amplified signal.

[0131] Finally, the sliding window average method is used to denoise and smooth the signal, reducing local fluctuations and noise interference, and improving signal stability and clarity. The calculation formula for the sliding window average is:

[0132] (twenty three)

[0133] in, is the window length of the sliding window, is the sampling interval, is the denoised signal. Sliding window averaging is used to smooth local fluctuations, reduce noise impact, and improve signal stability.

[0134] To identify SSVEP signals, this embodiment uses the Attention-Enhanced Spatiotemporal Network (AESTN) to extract the spatiotemporal features of SSVEP signals, and combines spatial and temporal attention mechanisms to optimize feature representation, ultimately achieving accurate recognition of visual stimuli of specific frequencies.

[0135] The model architecture of AESTN is as follows Figure 3 As shown in Figure 1, it includes: spatial attention module, temporal attention module, temporal processing module and classification module. The input signal of the network is ,in Represents the number of channels, Represents the time window length.

[0136] In a specific embodiment, the spatial attention module includes a convolutional layer (Conv2d), a batch normalization layer (Batch Normal), an activation layer (PRelu), an ECA module, and a random inactivation layer (Drop out).

[0137] In the spatial attention module, the input signal The spatial feature screening is performed through two-dimensional convolution operation, and the convolution kernel size is , output The calculation is as follows:

[0138] (twenty four)

[0139] in, Represents the spatial convolution kernel weight. The extracted spatial features Then, it is enhanced by batch normalization and PReLU activation function to obtain the enhanced features. , the formula is:

[0140] (25)

[0141] in, represents the activation function, Represents batch normalization.

[0142] To enhance the model's ability to perceive discriminative features in the input signal, a lightweight channel attention (ECA) module is introduced into the spatial attention module. By calculating channel attention weights to highlight important features and suppress invalid information, the feature selection capability of the spatial attention module in the channel dimension is effectively enhanced:

[0143] (26)

[0144] in, represents an element-wise multiplication operation, is the spatial feature weighted by channel attention. The output of the ECA module passes through the random inactivation layer and is output to the temporal attention module.

[0145] The temporal attention module in this embodiment includes a convolutional layer (Conv2d), a batch normalization layer (Batch Normal), an activation layer (PRelu), an ECA module, and a random dropout layer (Drop Out). The temporal attention module uses the temporal attention mechanism to model the dynamic changes of the signal.

[0146] The time characteristic is calculated by the following formula:

[0147] (27)

[0148] in, Represents the time convolution kernel weight. Similarly, applying batch normalization and PReLU activation function yields , and introduce the ECA module to extract more significant features:

[0149] (28)

[0150] in, Represents the output of the temporal attention module.

[0151] The output of the ECA module is sent to the time processing module after passing through the random dropout layer.

[0152] The time processing module of this embodiment adopts BiLSTM network to enhance the time feature Input to the BiLSTM network to capture the global spatiotemporal features in the EEG signal:

[0153] (29)

[0154] in, represents the weight matrix of the LSTM unit, represents the bias vector, and Represent the forward and backward outputs of BiLSTM respectively.

[0155] Finally, the model’s output is classified through the flattening layer and the fully connected layer to accurately judge the operator’s visual attention state:

[0156] (30)

[0157] in, Indicates that the output of the BiLSTM layer is flattened into a one-dimensional vector, and Represents the weights and biases of the fully connected layer, through Output classification probabilities.

[0158] Finally, the SSVEP EEG recognition results are converted into control instructions and transmitted to the robotic dog control system in real time. The instructions trigger the robotic dog to perform operations including forward, backward, stop, and turn, ensuring the reliable execution of the task.

[0159] The system uses a signal processing algorithm on the host computer platform to decode the SSVEP signals and convert the recognition results into control instructions. These instructions are transmitted to the robot dog's control system via wireless communication, controlling the robot dog to execute actions such as forward movement, backward movement, stopping, and steering, thereby achieving efficient task execution. To verify the system's performance in complex environments, outdoor tests were conducted. The test results demonstrated that the SSVEP control system has high stability and accuracy in practical applications.

[0160] This embodiment uses an attention-enhanced spatiotemporal network. It uses a spatial attention module to filter key channel information and a temporal attention module to capture dynamic time series features. This is combined with the ECA module to capture potentially useful signals in the data. The attention mechanism adaptively focuses on valid features and suppresses noise interference. The spatiotemporal attention mechanism, combined with the ECA module to enhance key frequency bands, enables the model to quickly capture the key features of SSVEP signals, improving recognition accuracy.

[0161] This technical solution utilizes a dual-mode dynamic switching mechanism, switching between detection mode (myoelectric control) and alert mode (SSVEP control) based on environmental target detection results (YOLOv5). This allows for flexible adaptation to different mission scenarios (covert detection / rapid alert). Myoelectric mode reduces exposure risk, while SSVEP mode enables contactless, covert control. Mode switching ensures consistent and efficient mission execution.

[0162] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A dual-mode control method for a mechanical dog based on electromyographic gestures and SSVEP, characterized in that: The dual-mode control method of the mechanical dog based on myoelectric gestures and SSVEP includes: Collect environmental data and perform target recognition. When the predetermined target is recognized, it enters the alert mode, otherwise it enters the detection mode; In detection mode, it collects electromyographic signals for gesture recognition and converts the gesture recognition results into control instructions to control the robot dog to perform corresponding operations; In alert mode, the SSVEP signal is collected for recognition and the recognition result is converted into a control instruction to control the robot dog to perform the corresponding operation; The step of collecting electromyographic signals for gesture recognition in the detection mode includes: Preprocessing the collected electromyographic signals; The preprocessed EMG signals are input into the deep learning neural network model for gesture recognition; The deep learning neural network model is a spatiotemporal feature fusion learning network, including a cyclic multi-scale convolution module, a spatiotemporal graph convolution module, a cross attention module and a classification module; The cyclic multi-scale convolution module includes a multi-scale convolution sub-module and a convolutional bidirectional long short-term memory neural network sub-module, and performs the following operations: The multi-scale convolution submodule uses different convolution kernels to perform multi-scale operations on the pre-processed EMG signals to generate three short-term time features; Then the three short-term time features are passed through the convolutional bidirectional long short-term memory neural network submodule to obtain the long-term time features; Concatenate short-term time features and long-term time features into long-term and short-term features; The spatiotemporal graph convolution module processes the preprocessed EMG signals, including multiple spatiotemporal graph convolution layers connected in sequence. Each layer adds a temporal convolution block and a residual connection on the basis of the traditional graph convolution network (GCN). The spatiotemporal graph convolution layer performs the following operations: The input features are used to extract spatial features through the graph convolutional network GCN; The spatial features are then temporally convolved through the temporal convolution block and then residually connected with the input features; Among them, in the alert mode, a stimulus coding method of brightness modulation and radial scaling frequency modulation intermodulation was used to induce SSVEP signals; The brightness modulation is encoded based on a sine wave function, and its calculation formula is: ; in, is the brightness modulation frequency, n is the current frame index, is the screen refresh rate; The radial scaling frequency modulation follows the following function: ; in, is the motion modulation frequency, It is n The zoom radius of the frame; When the brightness modulation frequency and motion modulation frequency When working together, the intermodulation frequency (IMF) is generated based on the combination of these two: ; in, m and n The integers 1 and 2 respectively; The SSVEP signal is collected for recognition and an attention-enhanced spatiotemporal network is used for recognition. The attention-enhanced spatiotemporal network includes a spatial attention module, a temporal attention module, a temporal processing module and a classification module.

2. The dual-mode control method for a mechanical dog based on electromyographic gestures and SSVEP according to claim 1 is characterized in that: The collecting of environmental data and performing target identification includes: Collect environmental images and use the YOLOv5 target detection model to identify targets.

3. The dual-mode control method of a mechanical dog based on electromyographic gestures and SSVEP according to claim 1 is characterized in that: The spatial attention module includes a convolution layer, a batch normalization layer, an activation layer, an ECA module and a random inactivation layer; the temporal attention module includes a convolution layer, a batch normalization layer, an activation layer, an ECA module and a random inactivation layer.

Images