SSVEP-based brain-controlled unmanned aerial vehicle method and system

Abstract

A brain-controlled unmanned aerial vehicle method and system based on SSVEP, the brain-controlled unmanned aerial vehicle method comprising: dividing a visual stimulator into a plurality of independent areas, each independent area corresponding to each action direction of an unmanned aerial vehicle, each independent area flickering and stimulating square blocks at different frequencies; sequentially observing the stimulation square blocks of each independent area to enable the brain occipital area to generate a plurality of different electroencephalogram signals; collecting a plurality of different electroencephalogram signals and converting the electroencephalogram signals into digital signals; the digital signals are extracted and classified, and control instructions of all action directions of the unmanned aerial vehicle are generated; and observing the stimulation blocks corresponding to the independent areas according to the control instruction, and controlling the unmanned aerial vehicle to move according to the control instruction. According to the design, a plurality of different electroencephalogram signals are generated by the cerebral cortex by observing the stimulation square blocks with different frequencies, the electroencephalogram signals are converted into the control instructions of the unmanned aerial vehicle, the unmanned aerial vehicle can be controlled to move in the specified direction only by observing the corresponding stimulation square blocks, and the control precision is high.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a brain-controlled UAV method and system based on SSVEP. Background Technology

[0002] Brain-computer interfaces (BCIs) establish an information communication channel between the brain and the outside world. Electroencephalogram (EEG) signals are first acquired through the BCI, then processed to convert them into control signals. These control signals are then transmitted to a computer or peripheral device via wired or wireless communication, enabling control of the peripheral device. Although BCI research is still in its early stages, numerous studies have demonstrated its effectiveness and it has seen significant development. SSVEPs (Sensory-Motor Rhythms and Steady-State Visual Evoked Potentials) are periodic neural responses to repetitive visual stimuli, appearing in the visual cortex as flicker frequencies and their harmonics. Currently, SSVEP-based BCIs have achieved high information transmission rates.

[0003] The development of brain-computer interface (BCI) technology has prompted researchers to develop real-time robotic systems based on BCIs, including brain-controlled wheelchairs, exoskeletons, and assistive systems for both healthy and disabled individuals. Drones, due to their convenience, flexibility, low cost, safety, and controllability, have been widely used in both military and civilian fields. However, drone control is primarily achieved through joysticks or other control tools. To improve drone efficiency and free up operators' hands, research has been conducted on BCI-controlled drone systems. Operators can control drones to perform tasks via BCIs. While operators can use BCIs to control drones, several limitations remain.

[0004] The limited flight control commands for drones allow only simple flight and landing, resulting in poor control precision and inflexibility, thus hindering accurate control. Summary of the Invention

[0005] The purpose of this invention is to overcome the defects and problems of poor control precision in the existing technology, and to provide a brain-controlled drone method and system based on SSVEP with high control precision.

[0006] To achieve the above objectives, the technical solution of this invention is: a brain-controlled unmanned aerial vehicle method based on SSVEP, comprising:

[0007] The visual stimulator is divided into multiple independent regions, each corresponding to different movement directions of the drone, and each independent region flashes a stimulation block at a different frequency.

[0008] Observe the stimulation blocks in each independent region in turn to generate multiple different EEG signals in the occipital region of the brain;

[0009] Collect multiple different electroencephalogram (EEG) signals and convert them into digital signals;

[0010] Digital signals are extracted and classified to generate control commands for each movement direction of the drone;

[0011] Observe the stimulus blocks in the corresponding independent area according to the control instructions, and control the drone to move according to the control instructions.

[0012] Each independent region flashes a stimulus square at a different frequency, including:

[0013] Define the colors of the stimulus blocks as white and black, with white representing light and black representing dark. Set the sequence length of the stimulus blocks in each independent area. Each sequence length is different. Divide the refresh rate of the visual stimulator by the sequence length of each stimulus block to obtain the actual flashing frequency of each stimulus block.

[0014] Calculate the least common multiple of all sequence lengths, extend each sequence length to the least common multiple length, generate a matrix of uniform length, and ensure that the actual flashing frequency of each stimulus block completes an integer number of cycles within the frame number corresponding to the least common multiple.

[0015] Determine the number of frames for which the stimulus block remains bright or dark. :

[0016] ;

[0017] in, For the round function, This is the actual flicker frequency. The refresh rate of the visual stimulator;

[0018] The number of frames for each stimulus square is counted by a counter, and the count continues until the cumulative number of frames reaches the specified duration. The color is changed periodically, causing the stimulus blocks to continuously switch between light and dark.

[0019] The process of acquiring multiple different electroencephalogram (EEG) signals and converting them into digital signals includes:

[0020] Wear an EEG cap, align the multiple wet electrodes in the cap with the preset scalp positions, use a syringe to draw up conductive paste and inject the conductive paste into the contact point between each wet electrode and the scalp;

[0021] The wet electrode transmits the EEG signal collected from the scalp surface to an amplifier, which amplifies the EEG signal and then transmits it to an analog-to-digital converter (ADC). The ADC converts the amplified EEG signal into a digital signal.

[0022] The converted digital signal is filtered and noise-reduced using a filter.

[0023] By removing artifacts from digital signals using signal analysis methods, the final digital signal is obtained.

[0024] The step of filtering and noise reduction of the converted digital signal using a filter includes:

[0025] A notch filter is used to preprocess the digital signal to remove Hz mains interference;

[0026] A bandpass filter of Hz to Hz is selected to filter the preprocessed digital signal, removing low-frequency noise below Hz and high-frequency noise above Hz.

[0027] The method of removing artifact interference from digital signals through signal analysis includes:

[0028] Calculate the independent component matrix :

[0029] ;

[0030] in, It is a two-dimensional EEG matrix. The columns represent electrode channels. The rows represent the sampled EEG voltage values ​​of the electrode channels at each time step. For the unmixing matrix, The lines represent component elements. The columns represent data points;

[0031] Demixing matrix inverse matrix and independent component matrix Multiply and then perform inverse unmixing to obtain the topology and power spectrum curves of each independent component;

[0032] When the topology or power spectrum curve of an independent component matches a known artifact feature, or when the energy in the topology or power spectrum curve of an independent component is much higher than that of a normal EEG signal, it is identified as an artifact and removed.

[0033] In digital signals Before extraction and classification, the digital signal is calculated first. power spectrum :

[0034] ;

[0035] ;

[0036] in, digital signal conduct Discrete signal after point sampling For discrete signals Fourier transform;

[0037] Power spectrum density was analyzed using different power spectral density methods. Convert the data into a power spectrum simulation plot and calculate the ratio power spectral density for analysis. :

[0038] ;

[0039] ;

[0040] in, It refers to the frequency and number of stimulations. It is the first The frequency of a stimulus, It is the power spectrum The peak frequency that appears in It is the magnitude of the peak frequency close to the target stimulus frequency;

[0041] Comparison of each power spectral density method Size, in The power spectral density method at its minimum is used as a feature extraction method for digital signals.

[0042] The extraction and classification of digital signals includes:

[0043] The average value of SSVEP EEG signals at the same stimulation frequency is obtained by averaging multiple SSVEP EEG signals. The average value of SSVEP EEG signals at each stimulation frequency is then fused with sine and cosine reference signals to form individual template signals at each stimulation frequency.

[0044] Canonical correlation analysis was performed between SSVEP EEG signals at various stimulation frequencies and individual template signals to obtain the optimal individual signal with the highest canonical correlation coefficient.

[0045] Spectral analysis was performed on the SSVEP EEG signal to screen out the sine and cosine reference signal frequency bands that had the highest correlation with the SSVEP EEG signal. Then, the sine and cosine reference signal frequency bands were added to the optimal individual signal to obtain the optimal reference signal.

[0046] The power spectrum of the SSVEP EEG signal and the optimal reference signal were estimated using the power spectral density method to evaluate the characteristic frequencies and intensities of the SSVEP EEG signal. Then, the CCA algorithm was used to analyze the signal and determine the stimulation frequency corresponding to the maximum value of the canonical correlation coefficient as the stimulation frequency of the SSVEP EEG signal.

[0047] The average SSVEP EEG signal at each stimulation frequency The calculation formula is:

[0048] ;

[0049] ;

[0050] in, This indicates SSVEP brainwave signals. Indicates the number of stimulus targets. Indicates the first The SSVEP EEG signal corresponding to each stimulation frequency For the first The stimulus frequency corresponding to each stimulus target. This indicates the EEG signal acquisition channel, and its subscript represents the channel number. Indicates the number of brainwave channels. Indicates the total number of channels. This indicates multiple SSVEP EEG signals at the same stimulus frequency. of The average value of each EEG channel is taken;

[0051] Individual template signals at each stimulation frequency The calculation formula is:

[0052] ;

[0053] ;

[0054] in, Indicates the number of sampling points. Indicates the sampling frequency. This represents the harmonic number of the signal.

[0055] The optimal individual signal obtained when the canonical correlation coefficient is maximized. The calculation formula is:

[0056] ;

[0057] ;

[0058] in, The canonical correlation coefficient is... For the first Canonical correlation coefficients under a single stimulus target Canonical correlation coefficient The frequency corresponding to the maximum value.

[0059] A brain-controlled drone system based on SSVEP includes: an EEG cap, an amplifier, a filter, an analog-to-digital converter, a computer, a visual stimulator, and a drone control unit. The EEG cap is equipped with multiple wet electrodes, which are in contact with the scalp. The wet electrodes are connected to the computer via the amplifier, the analog-to-digital converter, and the filter. The visual stimulator is connected to the computer, and the computer is connected to the drone control unit via a wireless communication module.

[0060] The visual stimulator is used to generate six independent regions, each of which flashes a stimulus square at a different frequency.

[0061] The wet electrode is used to collect EEG signals generated on the scalp surface when observing the stimulation square;

[0062] The amplifier is used to amplify the EEG signal;

[0063] The analog-to-digital converter is used to convert the amplified EEG signal into a digital signal;

[0064] The filter is used to filter and reduce noise in digital signals;

[0065] The computer is used to synchronize the stimulation time of the visual stimulator with the time of EEG signal acquisition by the wet electrode, extract the SSVEP characteristic frequency of the filtered and noise-reduced digital signal and map it into UAV control commands, and transmit the UAV control commands to the UAV control unit through the wireless communication module.

[0066] The drone control unit is used to control the movement of the drone.

[0067] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0068] 1. This invention discloses a brain-controlled drone method and system based on SSVEP. A visual stimulator generates stimulation blocks of different frequencies. When observing these stimulation blocks, the cerebral cortex generates multiple different electroencephalogram (EEG) signals for different frequencies. These EEG signals are converted into control commands for the drone. Each EEG signal corresponds one-to-one with the drone's six movement directions: up, down, left, right, forward, and backward. When the drone needs to move, simply observing the corresponding stimulation block controls the drone to move in the specified direction, resulting in high control precision. Therefore, this invention offers high control precision.

[0069] 2. In the SSVEP-based brain-controlled drone method and system of this invention, black and white are used to achieve the flashing effect. By determining the number of consecutive frames of white and black, the human eye can generate corresponding EEG signals when observing the stimulus square. By extending the length of each sequence to a least common multiple, strict phase alignment of multi-frequency stimuli is achieved, avoiding phase drift from affecting the quality of the SSVEP signal. Therefore, this invention has good stability and high accuracy.

[0070] 3. This invention, a brain-controlled unmanned aerial vehicle (UAV) method and system based on SSVEP, employs a non-invasive approach to acquire electroencephalogram (EEG) signals, eliminating the risks of surgery and preventing damage to the subject's brain. The operation is simpler and safer, and the experimental time is shortened. Simultaneously, the use of wet electrodes effectively reduces electrode impedance, resulting in better quality EEG signals. The system requires high stability and accuracy. By setting filters to remove high-frequency or low-frequency noise interference, and signal analysis methods to remove EEG artifacts, it not only automatically identifies and removes EMG artifacts but also retains a large amount of EEG information, making the data more stable and pure. Therefore, this invention offers high accuracy and good stability.

[0071] 4. In the SSVEP-based brain-controlled unmanned aerial vehicle method and system of this invention, the energy distribution of the signal at different frequencies can be intuitively obtained through topology maps and power spectrum curves, which can clearly identify the characteristic frequencies and intensities of the SSVEP signal, thereby improving the frequency recognition accuracy of subsequent algorithms. By calculating ratio power spectral density analysis and using the power spectra of adjacent frequencies, the relationship between the stimulus frequency and its adjacent frequencies can be estimated, and the adjacent frequency with the lowest relationship value can be found, thereby improving the accuracy of subsequent digital signal classification. Therefore, this invention has good intuitiveness and high accuracy.

[0072] 5. In the SSVEP-based brain-controlled unmanned aerial vehicle method and system of this invention, the CCA algorithm is used to extract EEG signal features and classify them. This method exhibits high robustness and fully utilizes spontaneous EEG signals from non-SSVEP EEG signals, thus completely reflecting the characteristics of the EEG signals. Furthermore, by incorporating standard sine and cosine signals, the classification effect is relatively ideal. Therefore, this invention demonstrates high robustness and good classification performance. Attached Figure Description

[0073] Figure 1 This is a flowchart of a brain-controlled drone method based on SSVEP according to the present invention.

[0074] Figure 2 This is a schematic diagram of the structure of a brain-controlled unmanned aerial vehicle system based on SSVEP according to the present invention.

[0075] Figure 3This is a schematic diagram of the EEG cap and wet electrodes in this invention.

[0076] Figure 4 This is a schematic diagram of the structure of the visual stimulator in this invention.

[0077] Figure 5 This is a connection block diagram of the UAV control unit in this invention.

[0078] The diagram shows: EEG cap 1, electrode holder 11, lead wire 12, wet electrode 2, amplifier 3, analog-to-digital converter 4, filter 5, computer 6, visual stimulator 7, display screen 71, display area 72, flashing module 73, UAV control unit 8, microprocessor module 81, attitude detection module 82, power management module 83, wireless communication module 84, altitude measurement module 85, horizontal positioning module 86, and motor drive module 87. Detailed Implementation

[0079] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0080] Example 1:

[0081] See Figure 1 A brain-controlled drone method based on SSVEP includes:

[0082] The visual stimulator 7 is divided into multiple independent areas, each corresponding to different movement directions of the drone, and each independent area flashes a stimulation block at a different frequency.

[0083] Observe the stimulation blocks in each independent region in turn to generate multiple different EEG signals in the occipital region of the brain;

[0084] Collect multiple different electroencephalogram (EEG) signals and convert them into digital signals;

[0085] Digital signals are extracted and classified to generate control commands for each movement direction of the drone;

[0086] Observe the stimulus blocks in the corresponding independent area according to the control instructions, and control the drone to move according to the control instructions.

[0087] In this embodiment, there are six independent regions, corresponding to the up, down, left, right, forward, and backward movement directions of the drone. Stimulus blocks of different frequencies are generated by a visual stimulator. When observing the stimulation blocks, the cerebral cortex will generate multiple different EEG signals for different frequencies. By converting the EEG signals into control commands for the drone, each EEG signal corresponds one-to-one with the six movement directions of the drone. When the drone needs to move, it can be controlled to move in the specified direction simply by observing the corresponding stimulation block, resulting in high control precision.

[0088] Example 2:

[0089] The basic content is the same as in Example 1, except that:

[0090] Each independent region flashes a stimulus square at a different frequency, including:

[0091] Define the colors of the stimulus blocks as white and black, with white representing light and black representing dark. Set the sequence length of the stimulus blocks in each independent area. Each sequence length is different. Divide the refresh rate of the visual stimulator 7 by the sequence length of each stimulus block to obtain the actual flashing frequency of each stimulus block.

[0092] Calculate the least common multiple of all sequence lengths, extend each sequence length to the least common multiple length, generate a matrix of uniform length, and ensure that the actual flashing frequency of each stimulus block completes an integer number of cycles within the frame number corresponding to the least common multiple.

[0093] Determine the number of frames for which the stimulus block remains bright or dark. :

[0094] ;

[0095] in, For the round function, This is the actual flicker frequency. The refresh rate of the visual stimulator;

[0096] The number of frames for each stimulus square is counted by a counter, and the count continues until the cumulative number of frames reaches the specified duration. The color is changed periodically, causing the stimulus blocks to continuously switch between light and dark.

[0097] In this embodiment, the actual flashing frequencies of each stimulus block are 6.66Hz, 7.5Hz, 8.57Hz, 10Hz, 12Hz, and 15Hz, respectively. The sequence lengths of each stimulus block are 9 frames, 8 frames, 7 frames, 6 frames, 5 frames, and 4 frames, respectively. 0 represents white, and 1 represents black. Therefore, the distribution of bright and dark frames for each stimulus block is as follows:

[0098] freq{9}=[000001111] (5 bright frames, 4 dark frames);

[0099] freq{8}=[00001111] (four bright frames, four dark frames);

[0100] freq{7}=[0001111] (three bright frames, four dark frames);

[0101] freq{6}=[000111](three bright frames, three dark frames);

[0102] freq{5}=

[00111] (Two bright frames, three dark frames);

[0103] freq{4}=

[0011] (Two bright frames, two dark frames).

[0104] Example 3:

[0105] The basic content is the same as in Example 1, except that:

[0106] The process of acquiring electroencephalogram (EEG) signals and converting them into digital signals includes:

[0107] Wear the EEG cap 1, align the multiple wet electrodes 2 in the EEG cap 1 with the preset scalp position, use a syringe to draw out conductive paste and inject the conductive paste into the contact point between each wet electrode 2 and the scalp.

[0108] The wet electrode 2 transmits the collected EEG signal from the scalp surface to the amplifier 3. The amplifier 3 amplifies the EEG signal and transmits it to the analog-to-digital converter 4. The analog-to-digital converter 4 converts the amplified EEG signal into a digital signal.

[0109] The converted digital signal is filtered and noise-reduced using filter 5.

[0110] The final digital signal is obtained by removing artifacts and interference from the digital signal through signal analysis methods.

[0111] The step of filtering and noise reduction of the converted digital signal through filter 5 includes:

[0112] The digital signal is preprocessed using a notch filter 5 to remove 50Hz mains interference.

[0113] A bandpass filter 5 with a frequency range of 5Hz to 80Hz is selected to filter the preprocessed digital signal, removing low-frequency noise below 5Hz and high-frequency noise above 80Hz.

[0114] In this embodiment, artifact interference mainly includes electrocardiogram (ECG) interference, electromyography (EMG) interference, and electrooculography (EOG) interference. ECG interference is an interference signal caused by the subject's heartbeat. ECG interference is relatively rare in routine EEG experiments. EMG interference is an interference signal generated by the movement of the subject's head, limbs, or teeth clenching and swallowing during EEG acquisition. The frequency of this type of interference signal is usually above 30Hz. EOG interference is a relatively common type of interference. It is easy to cause blinking behavior during the experiment. This is vertical EOG. When the eyeball moves laterally, lateral EOG is generated. Low-frequency noise below 5Hz includes artifacts such as slow eye movement and baseline drift. High-frequency noise above 80Hz includes EMG interference and electronic device noise.

[0115] Example 4:

[0116] The basic content is the same as Example 3, except that:

[0117] The method of removing artifact interference from digital signals through signal analysis includes:

[0118] Calculate the independent component matrix :

[0119] ;

[0120] in, It is a two-dimensional EEG matrix. The columns represent electrode channels. The rows represent the sampled EEG voltage values ​​of the electrode channels at each time step. For the unmixing matrix, The lines represent component elements. The columns represent data points;

[0121] Demixing matrix inverse matrix and independent component matrix Multiply and then perform inverse unmixing to obtain the topology and power spectrum curves of each independent component;

[0122] When the topology or power spectrum curve of an independent component matches a known artifact feature, or when the energy in the topology or power spectrum curve of an independent component is much higher than that of a normal EEG signal, it is identified as an artifact and removed.

[0123] In this embodiment, the unmixing matrix It is calculated using the Independent Component Analysis (ICA) algorithm, which generates a demixing matrix by separating the independent components (such as EEG signals and artifacts) in the mixed EEG signals. Demixing matrix The inverse matrix columns contain the relative weights and polarities of the back projections of each wet electrode. After importing the raw EEG data, a series of processes such as electrode coordinate calibration, filtering, and downsampling are performed. Then, ICA identification and analysis are run to obtain the topology map and power spectrum curve of each independent component.

[0124] Example 5:

[0125] The basic content is the same as Example 3, except that:

[0126] In digital signals Before extraction and classification, the digital signal is calculated first. power spectrum :

[0127] ;

[0128] ;

[0129] in, digital signal conduct Discrete signal after point sampling For discrete signals Fourier transform;

[0130] Power spectrum density was analyzed using different power spectral density methods. Convert the data into a power spectrum simulation plot and calculate the ratio power spectral density for analysis. :

[0131] ;

[0132] ;

[0133] in, It refers to the frequency and number of stimulations. It is the first The frequency of a stimulus, It is the power spectrum The frequency of peaks appearing in the middle It is the magnitude of the peak frequency close to the target stimulus frequency;

[0134] Comparison of each power spectral density method Size, in The power spectral density method at its minimum is used as a feature extraction method for digital signals.

[0135] In this embodiment, the power spectral density method is a probabilistic statistical method that can intuitively reflect the distribution and changes of EEG power with frequency. It mainly includes the periodogram method, the segmented average periodogram method, the windowed average periodogram method, the multi-window method, and the multi-signal classification method. In this embodiment, the power spectrum is estimated using an EEG signal with a stimulation frequency of 12.5 Hz. The above methods can improve the classification accuracy of the system. The power spectrum estimation based on the multi-signal classification method has a better effect, so this method is selected as the feature extraction method for the power spectral density of EEG signals.

[0136] Example 6:

[0137] The basic content is the same as Example 5, except that:

[0138] The extraction and classification of digital signals includes:

[0139] The average value of SSVEP EEG signals at the same stimulation frequency is obtained by averaging multiple SSVEP EEG signals. The average value of SSVEP EEG signals at each stimulation frequency is then fused with sine and cosine reference signals to form individual template signals at each stimulation frequency.

[0140] Canonical correlation analysis was performed between SSVEP EEG signals at various stimulation frequencies and individual template signals to obtain the optimal individual signal with the highest canonical correlation coefficient.

[0141] Spectral analysis was performed on the SSVEP EEG signal to screen out the sine and cosine reference signal frequency bands that had the highest correlation with the SSVEP EEG signal. Then, the sine and cosine reference signal frequency bands were added to the optimal individual signal to obtain the optimal reference signal.

[0142] The power spectrum of the SSVEP EEG signal and the optimal reference signal were estimated using the power spectral density method to evaluate the characteristic frequencies and intensities of the SSVEP EEG signal. Then, the CCA algorithm was used to analyze the signal and determine the stimulation frequency corresponding to the maximum value of the canonical correlation coefficient as the stimulation frequency of the SSVEP EEG signal.

[0143] The average SSVEP EEG signal at each stimulation frequency The calculation formula is:

[0144] ;

[0145] ;

[0146] in, This indicates SSVEP brainwave signals. Indicates the number of stimulus targets. Indicates the first The SSVEP EEG signal corresponding to each stimulation frequency For the first The stimulus frequency corresponding to each stimulus target. This indicates the EEG signal acquisition channel, and its subscript represents the channel number. Indicates the number of brainwave channels. Indicates the total number of channels. This indicates multiple SSVEP EEG signals at the same stimulus frequency. of The average value of each EEG channel is taken;

[0147] Individual template signals at each stimulation frequency The calculation formula is:

[0148] ;

[0149] ;

[0150] in, Indicates the number of sampling points. Indicates the sampling frequency. This represents the harmonic number of the signal.

[0151] The optimal individual signal obtained when the canonical correlation coefficient is maximized. The calculation formula is:

[0152] ;

[0153] ;

[0154] in, The canonical correlation coefficient is... For the first Canonical correlation coefficients under a single stimulus target Canonical correlation coefficient The frequency corresponding to the maximum value.

[0155] In this embodiment, SSVEP EEG signals Wavelet denoising needs to be performed beforehand. The dimension is smaller than that of the sine and cosine reference signals. The dimension of the sine and cosine reference signals The definition is as follows:

[0156] ;

[0157] The selection of the sine and cosine reference signal frequency bands with the highest correlation to SSVEP EEG signals involves: performing spectral analysis on the SSVEP EEG signals to determine their dominant frequency bands; selecting the overlapping portions of the sine and cosine reference signals from the dominant frequency bands; and fusing these overlapping portions according to a ratio of 70% for individual EEG signals and 30% for the selected sine and cosine signals to form the optimal reference signal. This ensures that subsequent power spectrum estimation can accurately capture the characteristic frequencies and intensities of individual SSVEPs;

[0158] The specific steps of canonical correlation analysis are as follows:

[0159] Find a set of multidimensional vectors and To make it satisfy a linear combination:

[0160] ;

[0161] ;

[0162] in, SSVEP EEG signals linear combination, Sine and cosine reference signals linear combination, for variance for The variance;

[0163] calculate and Maximum correlation between :

[0164] ;

[0165] ;

[0166] ;

[0167] ;

[0168] ;

[0169] in, , They are respectively and The autocorrelation matrix, for and The cross-correlation matrix, Let be the coefficient vector of the linear combination. This is the canonical correlation coefficient, which allows us to find the frequency corresponding to the maximum canonical correlation coefficient.

[0170] Example 7:

[0171] See Figure 2 and Figure 5 A brain-controlled drone system based on SSVEP includes: an EEG cap 1, an amplifier 3, a filter 5, an analog-to-digital converter 4, a computer 6, a visual stimulator 7, and a drone control unit 8. The EEG cap 1 is equipped with multiple wet electrodes 2, which are in contact with the scalp. The wet electrodes 2 are connected to the computer 6 in sequence through the amplifier 3, the analog-to-digital converter 4, and the filter 5. The visual stimulator 7 is connected to the computer 6. The computer 6 is connected to the drone control unit 8 through a wireless communication module 84.

[0172] The visual stimulator 7 is used to generate six independent regions, each of which flashes a stimulus square at a different frequency.

[0173] The wet electrode 2 is used to collect EEG signals generated on the scalp surface when observing the stimulation square;

[0174] The amplifier 3 is used to amplify the EEG signal;

[0175] The analog-to-digital converter 4 is used to convert the amplified EEG signal into a digital signal;

[0176] The filter 5 is used to filter and reduce noise in digital signals;

[0177] The computer 6 is used to synchronize the stimulation time of the visual stimulator 7 with the time of the EEG signal collected by the wet electrode 2, extract the SSVEP characteristic frequency of the filtered and noise-reduced digital signal and map it into UAV control commands, and transmit the UAV control commands to the UAV control unit 8 through the wireless communication module 84.

[0178] The drone control unit 8 is used to control the movement of the drone.

[0179] In this embodiment, the number and location distribution of the wet electrodes 2 are as follows:

[0180] The head is divided into the prefrontal lobe region, the anterior prefrontal lobe region, the frontal lobe region, the temporal lobe region, the central region, the parietal lobe region, and the occipital lobe region. Fp, AF, F, T, C, P, and O are set to represent the prefrontal lobe region, the anterior prefrontal lobe region, the frontal lobe region, the temporal lobe region, the central region, the parietal lobe region, and the occipital lobe region, respectively. Odd numbers represent the left hemisphere of the head, even numbers represent the right hemisphere of the head, and z represents the midline.

[0181] Two lines are drawn on the scalp. The first line connects the root of the nose to the occipital protuberance, and the second line connects the left and right anterior ear depressions. These two lines intersect at a point where the Cz electrode is placed.

[0182] The following electrodes are placed along the line connecting the root of the nose to the occipital protuberance, from front to back, at 10% intervals between each electrode: Fpz, AFz, Fz, FCz, Cz, CPz, Pz, Poz, Oz. The following electrodes are placed along the line connecting the left and right anterior ear fossae, from left to right, at 10% intervals between each electrode: T7, C5, C3, C1, Cz, C2, C4, C6, T8.

[0183] On the left side, draw an arc along Fpz-T7-Oz, and place the following electrodes at 10% intervals from front to back: Fp1, AF7, F7, FT7, T7, TP7, P7, PO7, and O1. Similarly, on the right side, draw an arc along Fpz-T8-Oz, and place the following electrodes at 10% intervals from front to back: Fp2, AF8, F8, FT8, T8, TP8, P8, PO8, and O2.

[0184] Draw an arc along AF7-AFz-AF8, take the midpoint of the AF7-AFz arc as the AF3 electrode, take the midpoint of the AFz-AF8 arc as the AF4 electrode, and then take the midpoints of the AF7-AF3, AF3-AFz, AFz-AF4, and AF4-AF8 arcs to place the following electrodes: AF5, AF1, AF2, and AF6.

[0185] Similarly, draw arcs and place electrodes on T7-Cz-T8, FT7-FCz-FT8, TP7-CPz-TP8, P7-Pz-P8, and PO7-POz-PO8;

[0186] The final location of 64 wet electrodes 2 was obtained, with a total of 32 channels.

[0187] Example 8:

[0188] The basic content is the same as Example 7, except that:

[0189] See Figure 3 The EEG cap 1 is perforated, and multiple electrode holders 11 are installed on the cap 1. Multiple wet electrodes 12 are installed in the electrode holders 11, and the electrode holders 11 are filled with conductive paste. The EEG cap 1 is made of lightweight, breathable, quick-drying, and highly elastic fabric. Due to its perforated design, EEG point markings can be printed inside the cap to facilitate the positioning of the wet electrodes 2. The wet electrodes 2 are made of powdered silver-silver chloride sintered wet electrodes. By injecting conductive paste, the impedance can be quickly reduced to below 5kΩ, improving signal quality. One end of the wet electrode 2 is connected to the amplifier 3 via a lead 12.

[0190] Example 9:

[0191] The basic content is the same as Example 7, except that:

[0192] See Figure 4The visual stimulator 7 includes a display screen 71 and a control module. The display screen 71 has multiple display areas 72, each containing a flashing module 73. The control module is connected to the flashing modules 73 and controls them to flash at different frequencies. The display screen 71's interface is connected to a synchronization box via a data cable. The synchronization box is connected to an amplifier 3 via a synchronization trigger. There are six flashing modules 73, each corresponding to one of the six movement directions of the drone: up, down, left, right, forward, and backward. When the corresponding flashing module 73 is observed, the drone can be controlled to move in the specified direction. The synchronization box is model S. The YN-200A has a built-in timing processing module. The synchronization box is connected to the display screen 71 via a data cable. It receives the timing signals of visual stimuli emitted by the display screen 71. After the internal timing processing module performs synchronization calibration on the signals, the processed synchronization signals are transmitted to the amplifier 3 through the synchronization trigger. This ensures that the flashing frequency of the visual stimuli corresponds accurately with the EEG signal acquisition in the time dimension, reducing signal analysis errors caused by asynchrony. The structure and principle of the synchronization box and synchronization trigger can be referred to the stimulation synchronizer in the self-awareness disorder auxiliary diagnosis system based on visual EEG signal analysis disclosed in Chinese Patent CN202120683397.4.

[0193] Example 10:

[0194] The basic content is the same as Example 7, except that:

[0195] See Figure 5 The UAV control unit 8 includes a microprocessor module 81, an attitude detection module 82, a power management module 83, a wireless communication module 84, an altitude measurement module 85, a horizontal positioning module 86, and a motor drive module 87. The microprocessor module 81 is connected to the attitude detection module 82, the power management module 83, the wireless communication module 84, the altitude measurement module 85, the horizontal positioning module 86, and the motor drive module 87, respectively.

[0196] The microprocessor module 81 uses the STM32F103C8T6 as the microprocessor chip for the UAV control unit 8. This chip has a built-in high-speed memory of 128k bytes ROM and 20k bytes SRAM. It has peripheral interfaces such as IIC, SPI and UART, which facilitates data transmission with modules such as MPU9250, VL53L0X, NRF24L01 and UP-FLOW.

[0197] The attitude detection module 82 can detect the flight attitude of the quadcopter drone in real time. The attitude detection module 82 selected in this embodiment is MPU9250. This chip integrates a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer sensor, and has a built-in DMP and supports MPL. Compared with the analog output of traditional inertial sensors, MPU9250 has the advantages of high-precision 16-bit ADC digital output and high transmission rate.

[0198] The power management module 83 mainly includes a voltage regulator module and a boost module. Since the lithium battery supply voltage is 5V, while the supply voltage of this circuit is 3.3V, a voltage regulator circuit is needed to convert the 5V voltage to 3.3V. The voltage regulator chip selected is SPX3819M5-3.3, which has the advantages of low voltage drop and low noise. When the quadcopter takes off, the lithium battery voltage will be pulled down, and the output voltage of the voltage regulator chip will also be lower than 3.3V. First, the input voltage of the battery is boosted to 5V, and then the 5V voltage is converted to 3.3V through the voltage regulator chip, so that the voltage can be stabilized at 3.3V. A lithium battery charging circuit is designed on the main control board of the quadcopter, which can make the flight time of the quadcopter longer.

[0199] The wireless communication module 84 is mainly used to transmit control commands issued by the computer 6, and then send the control commands to the microprocessor module 81, thereby realizing the flight control of the UAV.

[0200] The height measurement module 85 is selected as the ATK-VL53L0X laser ranging module, which is suitable for short and medium distance measurement. This module calculates the distance by the time of the received photons, and the measurement distance range is 0-2m. It also features small size, high measurement accuracy, multiple measurement working modes, and support for slave address setting and interrupt.

[0201] The horizontal positioning module 86 uses the UP-FLOW optical flow sensor module, and the motor drive module 87 is connected to the motor of the UAV to control the rotation of the UAV motor.

[0202] The attitude of the UAV is fused using the gradient descent method;

[0203] During attitude calculation, the accelerometer measurements are... With gravitational acceleration In the body coordinate system By subtracting the projections from each other, the objective function is established as follows:

[0204] ;

[0205] ;

[0206] in, For the attitude quaternion of the organism, The measured value is from the accelerometer. Inertial coordinate system The gravitational acceleration vector below, where The acceleration due to gravity is constant. For attitude quaternions The meaning of "from" Tie Rotation matrix of the system;

[0207] Let the accelerometer measurement value for:

[0208] ;

[0209] Taking the partial derivative of the above equation yields the Jacobian matrix:

[0210] ;

[0211] The gradient of the error function is then:

[0212] ;

[0213] The gradient descent method is used to find the minimum value of the error function. Through each iteration, the gravitational acceleration value expressed as a quaternion is made to approximate the accelerometer measurement, thus determining the attitude. The attitude update equation is:

[0214] ;

[0215] in, This is the output value of this attitude calculation. This is the output value from the previous attitude calculation cycle. Let be the step size of the gradient descent method. Let be the norm of the gradient of the error function;

[0216] Let the spatial location information be The position information obtained by the second integral of acceleration is The actual location information is ,but:

[0217] ;

[0218] in, For high-frequency noise of position sensors, This refers to the low-frequency noise of the accelerometer.

[0219] Let the low-pass filter be The high-pass filter is , For the filter coefficients, then we have This yields an estimate of the spatial location. for:

[0220] ;

[0221] Substituting the above filter into the equation, and applying the inverse Laplace transform, we get:

[0222] ;

[0223] In the formula, Position information calculated for the position sensor. for Accelerometer integral position information at all times.

[0224] Although embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A brain-controlled drone method based on SSVEP, characterized in that, include: The visual stimulator (7) is divided into multiple independent regions, each corresponding to the various movement directions of the drone, and each independent region flashes a stimulation block at a different frequency. Observe the stimulation blocks in each independent region in turn to generate multiple different EEG signals in the occipital region of the brain; Collect multiple different electroencephalogram (EEG) signals and convert them into digital signals; Digital signals are extracted and classified to generate control commands for each movement direction of the drone; Observe the stimulus blocks in the corresponding independent area according to the control instructions, and control the drone to move according to the control instructions.

2. The brain-controlled drone method based on SSVEP according to claim 1, characterized in that: Each independent region flashes a stimulus square at a different frequency, including: Define the colors of the stimulus blocks as white and black, with white representing light and black representing dark. Set the sequence length of the stimulus blocks in each independent area. Each sequence length is different. Divide the refresh rate of the visual stimulator (7) by the sequence length of each stimulus block to obtain the actual flashing frequency of each stimulus block. Calculate the least common multiple of all sequence lengths, extend each sequence length to the least common multiple length, generate a matrix of uniform length, and ensure that the actual flashing frequency of each stimulus block completes an integer number of cycles within the frame number corresponding to the least common multiple. Determine the number of frames for which the stimulus block remains bright or dark. : ； in, For the round function, This is the actual flicker frequency. The refresh rate of the visual stimulator; The number of frames for each stimulus square is counted using a counter, and the count continues until the cumulative number of frames reaches the specified duration. The color is changed periodically, causing the stimulus blocks to continuously switch between light and dark.

3. The brain-controlled drone method based on SSVEP according to claim 1, characterized in that: The process of acquiring multiple different electroencephalogram (EEG) signals and converting them into digital signals includes: Wear an EEG cap (1), align multiple wet electrodes (2) in the EEG cap (1) with a preset scalp position, use a syringe to draw out conductive paste and inject the conductive paste into the contact point between each wet electrode (2) and the scalp; The wet electrode (2) transmits the collected EEG signal from the scalp surface to the amplifier (3), the amplifier (3) amplifies the EEG signal and transmits it to the analog-to-digital converter (4), and the analog-to-digital converter (4) converts the amplified EEG signal into a digital signal. The converted digital signal is filtered and noise-reduced using filter (5); By removing artifacts from digital signals using signal analysis methods, the final digital signal is obtained.

4. The brain-controlled drone method based on SSVEP according to claim 3, characterized in that: The filtering and noise reduction process of the converted digital signal through filter (5) includes: A notch filter is used to preprocess the digital signal to remove 50Hz mains interference. A bandpass filter of 5Hz to 80Hz is selected to filter the preprocessed digital signal, removing low-frequency noise below 5Hz and high-frequency noise above 80Hz.

5. The brain-controlled drone method based on SSVEP according to claim 3, characterized in that: The method of removing artifact interference from digital signals through signal analysis includes: Calculate the independent component matrix : ； in, It is a two-dimensional EEG matrix. The columns represent electrode channels. The rows represent the sampled EEG voltage values ​​of the electrode channels at each time step. For the unmixing matrix, The lines represent component elements. The columns represent data points; Demixing matrix inverse matrix and independent component matrix Multiply and then perform inverse unmixing to obtain the topology and power spectrum curves of each independent component; When the topology or power spectrum curve of an independent component matches a known artifact feature, or when the energy in the topology or power spectrum curve of an independent component is much higher than that of a normal EEG signal, it is identified as an artifact and removed.

6. The brain-controlled drone method based on SSVEP according to claim 3, characterized in that: In digital signals Before extraction and classification, the digital signal is calculated first. power spectrum : ； ； in, digital signal conduct Discrete signal after point sampling, For discrete signals Fourier transform; Power spectrum was analyzed using different power spectral density methods. Convert the data into a power spectrum simulation plot and calculate the ratio power spectral density for analysis. : ； ； in, It refers to the frequency and number of stimulations. It is the first The frequency of a stimulus It is the power spectrum The peak frequency that appears in It is the magnitude of the peak frequency close to the target stimulus frequency; Comparison of each power spectral density method Size, in The power spectral density method at its minimum is used as a feature extraction method for digital signals.

7. The brain-controlled drone method based on SSVEP according to claim 6, characterized in that: The extraction and classification of digital signals includes: The average value of SSVEP EEG signals at the same stimulation frequency is obtained by averaging multiple SSVEP EEG signals. The average value of SSVEP EEG signals at each stimulation frequency is then fused with sine and cosine reference signals to form individual template signals at each stimulation frequency. Canonical correlation analysis was performed between SSVEP EEG signals at various stimulation frequencies and individual template signals to obtain the optimal individual signal with the highest canonical correlation coefficient. Spectral analysis was performed on the SSVEP EEG signal to screen out the sine and cosine reference signal frequency bands that had the highest correlation with the SSVEP EEG signal. Then, the sine and cosine reference signal frequency bands were added to the optimal individual signal to obtain the optimal reference signal. The power spectrum of the SSVEP EEG signal and the optimal reference signal were estimated using the power spectral density method to evaluate the characteristic frequencies and intensities of the SSVEP EEG signal. Then, the CCA algorithm was used to analyze the signal and determine the stimulation frequency corresponding to the maximum value of the canonical correlation coefficient as the stimulation frequency of the SSVEP EEG signal.

8. The brain-controlled drone method based on SSVEP according to claim 7, characterized in that: The average SSVEP EEG signal at each stimulation frequency The calculation formula is: ； ； in, This indicates SSVEP brainwave signals. Indicates the number of stimulus targets. Indicates the first The SSVEP EEG signal corresponding to each stimulation frequency For the first The stimulus frequency corresponding to each stimulus target. This indicates the EEG signal acquisition channel, and its subscript represents the channel number. Indicates the number of brainwave channels. Indicates the total number of channels. This indicates multiple SSVEP EEG signals at the same stimulus frequency. of The average value of each EEG channel is taken; Individual template signals at each stimulation frequency The calculation formula is: ； ； in, Indicates the number of sampling points. Indicates the sampling frequency. This represents the harmonic number of the signal.

9. A brain-controlled drone method based on SSVEP according to claim 7, characterized in that: The optimal individual signal obtained when the canonical correlation coefficient is maximized. The calculation formula is: ； ； in, The canonical correlation coefficient is... For the first Canonical correlation coefficients under a single stimulus target Canonical correlation coefficient The frequency corresponding to the maximum value.

10. A brain-controlled unmanned aerial vehicle system based on SSVEP, characterized in that, include: The system includes an EEG cap (1), an amplifier (3), a filter (5), an analog-to-digital converter (4), a computer (6), a visual stimulator (7), and a drone control unit (8). The EEG cap (1) is equipped with multiple wet electrodes (2), which are in contact with the scalp. The wet electrodes (2) are connected to the computer (6) in sequence through the amplifier (3), the analog-to-digital converter (4), and the filter (5). The visual stimulator (7) is connected to the computer (6). The computer (6) is connected to the drone control unit (8) through a wireless communication module (84). The visual stimulator (7) is used to generate six independent regions, each of which flashes a stimulus square at a different frequency; The wet electrode (2) is used to collect EEG signals generated on the scalp surface when observing the stimulation square; The amplifier (3) is used to amplify the EEG signal; The analog-to-digital converter (4) is used to convert the amplified EEG signal into a digital signal; The filter (5) is used to filter and reduce noise in digital signals; The computer (6) is used to control the stimulation time of the visual stimulator (7) to synchronize with the time of the EEG signal collected by the wet electrode (2), extract the SSVEP characteristic frequency of the filtered and noise-reduced digital signal and map it into the UAV control command, and transmit the UAV control command to the UAV control unit (8) through the wireless communication module (84). The UAV control unit (8) is used to control the movement of the UAV.

Images