Method for controlling unmanned aerial vehicle through brain-computer interface based on signal fusion, medium and equipment

Abstract

The invention discloses a method for controlling an unmanned aerial vehicle through a brain-computer interface based on signal fusion, a medium and equipment. According to the method, electroencephalogram signals of a user are collected in real time, power spectrum density analysis is carried out, a dynamic attention score is calculated, and concentration is divided into three levels of a resting state, a preparation state and a concentration state; meanwhile, head inertial attitude parameters and masseter surface electromyogram signals are collected. And generating a take-off instruction when the concentration state continuously reaches a preset duration, generating a direction control command based on weighted fusion of the electroencephalogram signal and the attitude parameter, and generating a landing instruction when an occlusion action is detected and the electroencephalogram power is lower than a resting state reference value. According to the invention, full-process natural control of the unmanned aerial vehicle from takeoff, flight to landing is realized through multi-mode signal fusion, and the control precision and reliability are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of brain-computer interface and unmanned aerial vehicle control, and particularly relates to a method, medium and device for controlling unmanned aerial vehicle based on signal fusion of brain-computer interface. BACKGROUND

[0002] The application of brain-computer interface technology in the field of unmanned aerial vehicle control still faces many technical bottlenecks, mainly in the following aspects:

[0003] Firstly, the existing brain-controlled unmanned aerial vehicle system generally adopts a single attention threshold triggering mechanism, which can only realize simple take-off and landing control, and cannot meet the continuous adjustment requirements of parameters such as direction and speed in the flight process. This discrete control method leads to system response lag and is prone to false triggering due to user attention fluctuations.

[0004] Secondly, the current system relies on single electroencephalogram signal as the control source, and lacks the ability to fuse multiple modal signals. In actual application, electroencephalogram signals are easily affected by environmental interference and individual differences, resulting in insufficient control stability. Especially in mobile scenarios, signal quality decreases significantly, but existing technologies fail to effectively combine inertial sensor data or other physiological signals for compensation and correction.

[0005] Thirdly, the existing scheme has poor adaptability to user physiological characteristics. Since the electroencephalogram signal characteristics of different users differ significantly, the method of using fixed thresholds often needs frequent calibration, increasing the use threshold. At the same time, the system lacks the ability to dynamically evaluate the real-time state of the user, and cannot adaptively adjust according to the changes in cognitive load.

[0006] In addition, the existing brain-controlled unmanned aerial vehicle system generally lacks an interactive closed loop. On the one hand, the system cannot accurately identify the user's control intent level; on the other hand, it lacks an effective state feedback mechanism, making it difficult for the user to perceive the current control state and execution result, affecting the operation experience.

[0007] In view of the above problems, it is urgent to develop a brain-controlled unmanned aerial vehicle system that can realize multi-modal signal fusion, support continuous fine control, and has adaptive ability, in order to break through the limitations of existing technologies. SUMMARY

[0008] In view of the above problems, the present application provides a method, medium and device for controlling unmanned aerial vehicle based on signal fusion of brain-computer interface, which realizes natural control of unmanned aerial vehicle throughout the process by fusing multi-modal signals of attention level, head posture and masseter action, and solves the problems of single traditional brain control instruction and high false triggering rate.

[0009] To solve the above problems, in a first aspect, the application provides a method for controlling a UAV based on signal fusion of a brain-computer interface, which is applicable to a brain-computer interface control system, the brain-computer interface control system comprising an electroencephalogram signal acquisition module, a concentration discrimination module, an inertial attitude perception module, an electromyography control module, an instruction generation module, and a flight control communication module.

[0010] The method comprises the following steps:

[0011] The electroencephalogram signal acquisition module acquires electroencephalogram signals of a user in real time.

[0012] The concentration discrimination module performs power spectral density analysis on the electroencephalogram signals, extracts frequency domain features of multiple wave bands, and calculates a dynamic attention score according to the frequency domain features of the multiple wave bands. Resting-state electroencephalogram signals of a user are acquired as reference data, and the first threshold value and the second threshold value are calibrated by the reference data, so as to divide the concentration into three levels of resting state, preparation state, and concentration state. When the dynamic attention score exceeds the first threshold value, the preparation state instruction is triggered, and when the dynamic attention score exceeds the second threshold value, the concentration state is triggered.

[0013] The inertial attitude perception module acquires inertial attitude parameters, the inertial attitude parameters comprising a rotation direction and an attitude angle of a head of the user, the attitude angle comprising a pitch angle, a yaw angle, and a roll angle.

[0014] The electromyography control module acquires surface electromyography signals of the masseter muscle, calculates short-time energy of the electromyography signals, and generates a clenching action signal when the short-time energy of M consecutive time windows exceeds an intensity threshold value, M being a positive integer greater than 2.

[0015] The instruction generation module performs any one of the following steps:

[0016] When the duration of the concentration state reaches a first preset duration, a UAV takeoff instruction is generated.

[0017] In the concentration state, a direction control command is generated based on a weighted fusion result of the electroencephalogram signals and the inertial attitude parameters.

[0018] If the clenching action signal is triggered and the power of the frequency domain features of the electroencephalogram signals is lower than a preset proportion of the resting-state reference value, a UAV landing instruction is generated.

[0019] The flight control communication module sends the generated takeoff instruction or direction control instruction or landing instruction to the UAV.

[0020] Further, the concentration discrimination module performs power spectral density analysis on the electroencephalogram signals, extracts frequency domain features of multiple wave bands, and calculates a dynamic attention score according to the frequency domain features of the multiple wave bands, which comprises:

[0021] The brain electrical signal acquisition module collects the brain electrical signal of the user in real time as a first brain electrical signal, the attention discrimination module performs filtering and denoising processing on the first brain electrical signal to obtain a second brain electrical signal, and the second brain electrical signal is divided into a plurality of time windows in time sequence, adjacent time windows have an overlapping region, and the power spectral density of each time window is calculated according to the following formula:

[0022]

[0023] Wherein, PSD(f) represents the power spectral density, N represents the total number of the divided time windows, n represents the index of the time window, X(n) represents the second brain electrical signal in the original time domain, f n represents the nth discrete frequency point, and the calculation formula is: f s is the sampling frequency, represents the complex base function of the discrete Fourier transform, and represents the frequency component.

[0024] The frequency domain features of a plurality of wave bands are extracted according to the following formula:

[0025]

[0026] Wherein, PSD(f,t) represents the power spectral density at frequency f in the tth time window, F x represents the frequency set covered by the frequency band x, |F x | represents the number of discrete frequency points contained in the frequency band x, P x (t) is the frequency domain feature, and specifically represents the average power of the frequency band x in the time window t, the value range of the frequency band 0 is [4, 8] Hz, the value range of the frequency band a is [8, 12] Hz, and the value range of the frequency band b is [13, 30] Hz.

[0027] The calculation formula of the dynamic attention score according to the frequency domain features of a plurality of wave bands is as follows:

[0028]

[0029] Wherein, EI(t) is the dynamic attention score.

[0030] Further, the brain electrical signal of the user in the resting state is collected as reference data, and the first threshold and the second threshold are calibrated by the reference data, including the following steps:

[0031] The brain electrical data of the user in the resting state for 30-60 seconds is collected, the average value and the standard deviation of the dynamic attention scores of the frequency bands 0, a and b are calculated respectively, the average value of the dynamic attention scores of each frequency band corresponding to the brain electrical data in the resting state is recorded as the reference score, and the calculation formula of the first threshold and the second threshold is as follows:

[0032] The first threshold = the reference score + 1 x the standard deviation;

[0033] The second threshold = the reference score + 2 x the standard deviation.

[0034] Further, the inertial attitude sensing module comprises a three-axis accelerometer and a three-axis gyroscope.

[0035] The inertial attitude sensing module collects the inertial attitude parameters, comprising:

[0036] The inertial attitude sensing module detects the head linear acceleration through the accelerometer to determine the rotation direction, measures the angular velocity through the gyroscope to calculate the attitude angle, and obtains the inertial attitude parameters according to the rotation direction and the attitude angle.

[0037] Further, the calculation formula of the short-time energy of the electromyographic signal is as follows:

[0038]

[0039] Wherein, E t represents the short-time energy of the electromyographic signal, w(n) represents the window function for smoothing the electromyographic signal, N represents the total number of the divided time windows, n represents the index of the time window, and x(n+t) represents the electromyographic signal in the tth time window.

[0040] When the short-time energy of the continuous M time windows is detected to exceed the intensity threshold, the occlusion action signal is generated, comprising:

[0041] When the short-time energy of the continuous M time windows is detected to exceed the intensity threshold, the middle time point of the first window in the continuous M time windows is taken as the time point of triggering the occlusion action signal.

[0042] Further, the method comprises:

[0043] When it is detected that the cognitive load of the user exceeds the preset load, the instruction generation module generates a hovering instruction, and sends the hovering instruction to the unmanned aerial vehicle through the flight control communication module to control the unmanned aerial vehicle to enter the hovering mode;

[0044] If the time of exceeding the preset load reaches a second preset time length, the instruction generation module generates a prompt instruction, and sends a prompt information through the flight control communication module, wherein the prompt information is used to guide the user to rest;

[0045] The calculation formula of the cognitive load of the user is as follows:

[0046]

[0047] Wherein, CL(t) is the cognitive load, P θ (t) and P α(t), P β (t) respectively represent the average power of the theta, alpha, beta band in the current time window, the value range of the frequency band theta is [4, 8] Hz, the value range of the frequency band alpha is [8, 12] Hz, and the value range of the frequency band beta is [13, 30] Hz.

[0048] Further, generating the direction control command based on the weighted fusion result of the electroencephalogram signal and the inertial attitude parameter includes:

[0049] According to the signal quality, the decision weights of each mode are dynamically adjusted, specifically including: according to the signal-to-noise ratio of the electroencephalogram signal and the acceleration variance of the inertial attitude parameter, the weights are distributed in real time, when the electroencephalogram signal is disturbed, the decision proportion of the inertial attitude parameter is automatically increased;

[0050] According to the adjusted weight, the electroencephalogram signal and the inertial attitude parameter are spliced into a feature vector, and the spliced feature vector is input into a pre-trained classification model to generate a direction control command containing a steering angle and a speed.

[0051] Further, the brain-computer interface control system includes a display module;

[0052] The method further includes:

[0053] The flight control communication module receives real-time flight state data feedback by the unmanned aerial vehicle, and displays the flight state data on the display module in real time, the flight state data including any one or more of flight height, current power of the unmanned aerial vehicle, attitude angle of the unmanned aerial vehicle and execution result of the control instruction.

[0054] In a second aspect, the present application provides a computer readable storage medium, which stores computer program instructions, the computer program instructions being executed by a processor to implement the method of the first aspect of the present application.

[0055] In a third aspect, the present application further provides an electronic device, including a memory and a processor, the memory being used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method of the first aspect.

[0056] Distinguish from the prior art, the technical scheme discloses a method, medium and equipment for controlling a UAV based on signal fusion of a brain-computer interface, which is applicable to a brain-computer interface control system comprising an electroencephalogram signal acquisition module, a concentration discrimination module, an inertial attitude sensing module, an electromyography control module, an instruction generation module and a flight control communication module. The method calculates a dynamic attention score by collecting electroencephalogram signals of a user in real time and performing power spectral density analysis, and divides the concentration into three levels of resting state, preparation state and concentration state; meanwhile, head inertial attitude parameters and surface electromyography signals of the masseter muscle are collected. A take-off instruction is generated when the concentration state lasts for a preset time length, a direction control command is generated based on the weighted fusion of the electroencephalogram signals and the attitude parameters, and a landing instruction is generated when a clenching action is detected and the electroencephalogram power is lower than a resting state reference value. The application realizes natural control of the whole process from take-off, flight to landing of the UAV through multi-modal signal fusion, and improves the control accuracy and reliability.

[0057] The above invention content is only a summary of the technical scheme of the application, in order to enable those skilled in the art to more clearly understand the technical scheme of the application, and then can be implemented according to the content of the description and the drawings, and in order to make the above purpose and other purposes, characteristics and advantages of the application more easily understood, the following is described in combination with the specific embodiments and drawings of the application. BRIEF DESCRIPTION OF DRAWINGS

[0058] The drawings are only used to show the principles, implementation modes, applications, characteristics and effects of the specific embodiments and other related contents of the application, and cannot be considered as a limitation of the application.

[0059] In the drawings of the specification:

[0060] Figure 1 The flow chart of the method for controlling a UAV based on signal fusion of a brain-computer interface according to the first exemplary embodiment of the application;

[0061] Figure 2 The module schematic diagram of the brain-computer interface control system according to an exemplary embodiment of the application;

[0062] Figure 3 The flow chart of the method for controlling a UAV based on signal fusion of a brain-computer interface according to the second exemplary embodiment of the application;

[0063] Figure 4 The interface schematic diagram displayed by the display module according to an exemplary embodiment of the application;

[0064] Figure 5 The interface schematic diagram displayed by the display module according to another exemplary embodiment of the application;

[0065] Figure 6A block diagram of a module of an electronic device according to an example embodiment of the present application.

[0066] Reference signs mentioned in the above-mentioned drawings are explained as follows:

[0067] 10. An electronic device;

[0068] 101. A processor;

[0069] 102. A storage medium;

[0070] 20. A brain-computer interface control system;

[0071] 201. An electroencephalogram signal acquisition module;

[0072] 202. A concentration discrimination module;

[0073] 203. An inertial attitude sensing module;

[0074] 204. An electromyography control module;

[0075] 205. An instruction generation module;

[0076] 206. A flight control communication module;

[0077] 207. A display module. DETAILED DESCRIPTION

[0078] To explain possible application scenarios, technical principles, specific schemes that can be implemented, and purposes and effects that can be achieved of the present application in detail, the following will be described in detail in combination with specific embodiments listed and with the accompanying drawings. The embodiments described in the present document are only used to more clearly illustrate the technical schemes of the present application, and therefore only serve as examples, and cannot limit the protection scope of the present application.

[0079] In the present document, the term "embodiment" means that the specific features, structures or characteristics described in combination with the embodiment can be included in at least one embodiment of the present application. The term "embodiment" appearing at various positions in the specification does not necessarily refer to the same embodiment, and does not particularly limit the independence or association between other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, each technical feature mentioned in each embodiment can be combined in any way to form a corresponding implementable technical scheme.

[0080] Unless otherwise defined, the meanings of the technical terms used in the present document are the same as those commonly understood by those skilled in the art to which the present application belongs; the use of related terms in the present document is only for the purpose of describing specific embodiments, and is not intended to limit the present application.

[0081] In the description of the present application, the phrase "and / or" is a description of a logical relationship between objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases: A exists, B exists, and A and B exist at the same time. In addition, the character " / " herein generally represents that the associated objects before and after are a "or" logical relationship.

[0082] In the present application, phrases such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, primary and secondary, or order relationship between the entities or operations.

[0083] In the present application, without more limitation, the "includes", "contains", "has" or other similar open expressions used in the sentence are intended to cover non-exclusive inclusion, and these expressions do not exclude the presence of other elements in the process, method or product including the described elements, so that the process, method or product including a series of elements can not only include those limited elements, but also include other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0084] As the same understanding in the "Guidelines for Review", in the present application, the expressions such as "greater than", "less than", "exceed" are understood as not including the number; "above", "below", "within" and other expressions are understood as including the number. In addition, in the description of the embodiments of the present application, the meaning of "multiple" is more than two (including two), and similar expressions related to "multiple" are also understood in this way, for example, "multiple groups", "multiple times" and the like, unless otherwise explicitly limited.

[0085] Please refer to Figure 1 and Figure 2 The present application provides a method for controlling unmanned aerial vehicle based on signal fusion brain-computer interface, which is suitable for brain-computer interface control system, the brain-computer interface control system 20 includes electroencephalogram signal acquisition module 201, concentration discrimination module 202, inertial attitude perception module 203, electromyography control module 204, instruction generation module 205 and flight control communication module 206.

[0086] The method comprises the following steps:

[0087] Step S101: The electroencephalogram signal acquisition module acquires the electroencephalogram signal of the user in real time;

[0088] Step S102: The concentration discrimination module performs power spectral density analysis on the electroencephalogram signal, extracts frequency domain features of multiple wave bands, and calculates a dynamic attention score according to the frequency domain features of the multiple wave bands. The electroencephalogram signal of the user in a resting state is collected as reference data, and the first threshold value and the second threshold value are calibrated through the reference data. The concentration is divided into three levels of resting state, preparation state and concentration state. When the dynamic attention score exceeds the first threshold value, the preparation state instruction is triggered, and when the dynamic attention score exceeds the second threshold value, the concentration state is triggered.

[0089] Step S103: The inertial attitude perception module collects inertial attitude parameters, including the rotation direction and attitude angle of the user's head.

[0090] Step S104: The electromyography control module collects the surface electromyography signal of the masseter muscle, calculates the short-time energy of the electromyography signal, and generates a clenching action signal when the short-time energy of M consecutive time windows exceeds the intensity threshold value. M is a positive integer greater than 2.

[0091] Step S105: The instruction generation module performs any of the following steps: when the duration of the concentration state reaches a first preset duration, a UAV takeoff instruction is generated; in the concentration state, a direction control command is generated based on the weighted fusion result of the electroencephalogram signal and the inertial attitude parameters; if the clenching action signal is triggered and the frequency domain feature power of the electroencephalogram signal is lower than a preset proportion of the resting state reference value, a UAV landing instruction is generated.

[0092] Step S106: The flight control communication module sends the generated takeoff instruction or direction control instruction or landing instruction to the UAV.

[0093] In the present embodiment, the electroencephalogram signal acquisition module 201 can be selected from a brain-computer interface device, which has multi-channel acquisition capability (such as AF7, AF8, Tp9, Tp10, etc. commonly used for attention detection channels), uses flexible electrode material, does not require traditional conductive glue, improves wearing comfort, and simultaneously has a built-in high-precision signal conditioning module and amplifier, which can stably acquire electroencephalogram signals in a motion state, and realize low-latency and high-stability data transmission through a Bluetooth BLE communication module.

[0094] The concentration discrimination module 202 analyzes the signal output by the electroencephalogram signal acquisition module through frequency domain analysis (such as power spectral density, α / β wave band ratio), time domain variation trend and electroencephalogram rhythm feature extraction, constructs a dynamic attention index, divides the concentration into three levels of resting state, preparation state and concentration state, and supports individualized threshold calibration based on user resting state data.

[0095] The inertial attitude perception module 203 integrates the IMU (inertial measurement unit) of the brain-computer interface device, and is used for collecting the rotation direction (forward, backward, left, right, and roll) and attitude angle (including pitch angle, yaw angle, and roll angle) of the head of the user in real time, and providing data support for the direction control of the unmanned aerial vehicle

[0096] The electromyography control module 204 collects the surface electromyography signal of the masseter muscle through the frontal lobe and temporal lobe electrodes of the brain-computer interface device. After the electromyography signal is preprocessed, the short-time energy is calculated and the continuous time window is analyzed, the effective biting action is recognized, and the biting action signal is generated as the trigger source of the landing instruction. The preprocessing can include mean removal, 50Hz notch filtering, low-pass filtering, etc.

[0097] The instruction generation module 205 integrates the electroencephalogram, inertial attitude, and electromyography multi-modal signals, and generates the take-off, direction control, and landing instructions of the unmanned aerial vehicle according to a preset logic, and is the “decision center” of the system.

[0098] The flight control communication module 206 communicates with the unmanned aerial vehicle through Bluetooth or WiFi, transmits the control instructions and receives the real-time state (height, power, attitude angle, etc.) of the unmanned aerial vehicle, supports short instruction triggering and periodic heartbeat packet synchronization.

[0099] In the actual application process, the electroencephalogram signal acquisition module collects the electroencephalogram signal in real time through the brain-computer interface device, and the concentration discrimination module processes the electroencephalogram signal, extracts the frequency domain features of the theta (4-8Hz), alpha (8-12Hz), and beta (13-30Hz) bands through power spectrum density analysis, and calculates the dynamic attention score. At the same time, the 30-60 second resting state electroencephalogram signal of the user is collected as the baseline data, the first threshold value (triggering the preparation state, such as lighting the signal light of the unmanned aerial vehicle) and the second threshold value (triggering the concentration state) are calibrated, and the three-level division of concentration is realized.

[0100] The inertial attitude perception module detects the linear motion direction of the head through the three-axis accelerometer, measures the angular velocity through the three-axis gyroscope, and calculates the attitude angle (the pitch angle corresponds to the forward and backward motion, and the yaw angle corresponds to the left and right rotation). The minimum action threshold of ±5° is set (to avoid false triggering caused by slight tremor), and the instruction is automatically centered when the head is returned to the normal position.

[0101] The electromyography control module collects the electromyography signal of the masseter muscle, calculates the short-time energy after filtering and denoising, generates the biting action signal when the short-time energy of the continuous M (M>2) time windows exceeds the intensity threshold value, and the electroencephalogram signal is in the inactive state (the frequency domain feature power is lower than the resting state baseline value by a preset proportion).

[0102] The instruction generation module generates instructions according to the following logic: ① when the focused state lasts for a first preset time length (such as 2 seconds), generate a take-off instruction; ② under the focused state, generate a direction control command (such as forward for bowing head) by fusing the electroencephalogram signal and the inertial attitude parameter; ③ when the biting action signal is triggered, generate a landing instruction. The flight control communication module transmits the instructions to the unmanned aerial vehicle and feeds back the flight state.

[0103] The application breaks through the dimensional restriction of traditional single-mode brain control by fusing multi-modal signals such as electroencephalogram signals, inertial attitude parameters, and electromyogram signals, and realizes natural control of the whole process of "take-off-flight-landing" of the unmanned aerial vehicle. Through the double-threshold concentration discrimination and individualized calibration mechanism, the false triggering caused by instantaneous attention fluctuation is reduced, and the system stability is improved. The natural head posture control replaces the traditional manual control, and the masseter signal triggers the landing, which conforms to the human physiological habit, enhances the naturalness and immersion of the interaction, and is suitable for special groups such as disabled people. The flight control communication module feeds back the flight state in real time, constructs a "intention-execution-feedback" closed loop, and reduces the cognitive burden of the user.

[0104] In some embodiments, the concentration discrimination module performs power spectral density analysis on the electroencephalogram signal, extracts frequency domain features of multiple bands, and calculates a dynamic attention score according to the frequency domain features of the multiple bands, including:

[0105] The electroencephalogram signal acquisition module acquires the electroencephalogram signal of the user in real time as a first electroencephalogram signal, the concentration discrimination module performs filtering and denoising processing on the first electroencephalogram signal to obtain a second electroencephalogram signal, and the second electroencephalogram signal is divided into multiple time windows in time sequence, there is an overlapping area between adjacent time windows, and the power spectral density of each time window is calculated according to the following formula (1):

[0106]

[0107] Wherein, PSD(f) represents the power spectral density, N represents the total number of divided time windows, n represents the index of the time window, X(n) represents the original time domain second electroencephalogram signal, f n represents the nth discrete frequency point, and its calculation formula is: f s is the sampling frequency, represents the complex base function of discrete Fourier transform, and represents the frequency component;

[0108] The frequency domain features of multiple bands are extracted according to the following formula (2):

[0109]

[0110] Wherein, PSD(f, t) represents the power spectral density of frequency f in the tth time window, F xFrepresents the frequency set covered by the frequency band x, |F x P represents the number of discrete frequency points contained in the frequency band x, P x (t) is a frequency domain feature, specifically representing the average power of the frequency band x in the time window t, the value range of the frequency band θ is [4, 8] Hz, the value range of the frequency band α is [8, 12] Hz, and the value range of the frequency band β is [13, 30] Hz;

[0111] The calculation formula of the dynamic attention score according to the frequency domain features of multiple wave bands is shown in the following formula (3):

[0112]

[0113] Wherein, EI(t) is the dynamic attention score.

[0114] In this embodiment, the power spectral density (PSD) is a description of the energy distribution of the electroencephalogram signal on different frequency components, which is calculated by discrete Fourier transform and is the core index for analyzing the frequency domain features.

[0115] The time window is a segment of continuous electroencephalogram signal divided by a fixed time length (such as 1 second), and adjacent windows overlap (such as 50%) to ensure real-time performance, which is used for dynamic updating of the attention score.

[0116] The dynamic attention score (EI(t)) is a quantitative index calculated based on the weighted θ, α and β wave band frequency domain features, reflecting the real-time concentration degree of the user and supporting dynamic adjustment.

[0117] In calculating the dynamic attention score, the attention discrimination module first pre-processes the first electroencephalogram signal collected by the brain-computer interface device, which includes filtering and denoising operations, aiming to remove 50Hz power frequency interference, electromyographic artifacts, etc., to obtain a second electroencephalogram signal, and divide the second electroencephalogram signal into multiple overlapping time windows in time sequence to ensure the continuity of signal analysis.

[0118] Then, the power spectral density is calculated, specifically, the PSD(f) of each time window is calculated according to the formula (1) shown in the foregoing, the frequency components are decomposed by discrete Fourier transform, and the energy distribution of the θ, α and β wave bands is reflected. Then, the frequency domain features are extracted, specifically, the average power (i.e. the frequency domain feature) of the θ (4-8Hz), α (8-12Hz) and β (13-30Hz) wave bands is calculated according to formula (2), wherein the β wave band is positively correlated with attention, and the α wave band is related to the resting state.

[0119] Then, the dynamic attention score is calculated, and the three wave band features are fused by formula (3) to quantify the real-time concentration degree.

[0120] The above scheme improves the robustness of attention discrimination through multi-band feature fusion, avoiding misjudgment caused by single-band fluctuation. Through the overlapping time window and sliding window updating mechanism, the real-time performance of the attention score is guaranteed, meeting the low delay requirement of the unmanned aerial vehicle control. Based on the high-precision signal acquisition and preprocessing of the brain-computer interface device on the electroencephalogram signal, reliable data basis is provided for the calibration of the first threshold and the second threshold.

[0121] In some embodiments, the electroencephalogram signal of the user in the resting state is collected as the reference data, and the calibration of the first threshold and the second threshold based on the reference data comprises the following steps:

[0122] The electroencephalogram data of the user in the resting state for 30-60 seconds is collected, the average value and the standard deviation of the dynamic attention scores of the frequency bands 0, alpha and beta are calculated respectively, the average value of the dynamic attention scores of each frequency band corresponding to the electroencephalogram data in the resting state is recorded as the reference score, and the calculation formula (4) of the first threshold and the second threshold is as follows:

[0123] The first threshold = the reference score + 1 x the standard deviation;

[0124] The second threshold = the reference score + 2 x the standard deviation.

[0125] In this embodiment, the resting state electroencephalogram data refers to the electroencephalogram signal of the user in a relaxed state without specific tasks, reflecting the individual basic attention level, and is the reference for individual threshold calibration. The reference score refers to the average value of the dynamic attention scores in the resting state electroencephalogram data, which is used as a reference for individual concentration. The standard deviation refers to the dispersion degree of the resting state attention score, which quantifies the natural fluctuation range of individual concentration.

[0126] The working principle of threshold calibration is as follows: after the user wears the brain-computer interface device (such as the Xmuse device), the user maintains a relaxed state (such as closing eyes and resting) for 30-60 seconds, the system synchronously collects the electroencephalogram signal, and calculates the dynamic attention scores of the theta, alpha and beta bands. Then, statistical analysis is performed on the resting state data to obtain the average value of the dynamic attention scores (reference score) and the standard deviation (reflecting the fluctuation range). Then, the first threshold and the second threshold are calculated according to formula (4), which correspond to the preparation state and the concentration state respectively.

[0127] The above scheme solves the individual difference problem of electroencephalogram signal through individual threshold calibration, without manual parameter adjustment, realizing the "plug and play" of the brain-computer interface device. At the same time, the threshold design based on the standard deviation is compatible with the natural fluctuation of individual concentration, reducing the false triggering caused by instantaneous interference. The two-level threshold forms a buffer mechanism, improves the user's perception of takeoff preparation, and enhances the interaction safety.

[0128] In some embodiments, the inertial attitude sensing module comprises a three-axis accelerometer and a three-axis gyroscope;

[0129] The inertial attitude sensing module collects inertial attitude parameters, including:

[0130] The inertial attitude sensing module detects head linear acceleration through an accelerometer to determine a rotation direction, measures angular velocity through a gyroscope to calculate an attitude angle, and obtains the inertial attitude parameters according to the rotation direction and the attitude angle.

[0131] In this embodiment, when collecting acceleration and angular velocity, the inertial attitude sensing module detects the acceleration of head linear motion (for example, Y-axis positive acceleration corresponds to forward inclination) through a three-axis accelerometer, and measures the angular velocity of rotation (for example, angular velocity around the Y-axis corresponds to left and right rotation speed) through a three-axis gyroscope. When calculating the attitude angle, the angular velocity output by the gyroscope is integrated to obtain the pitch angle, the yaw angle, and the roll angle, and the accelerometer data is used to correct the drift error.

[0132] Preferably, when judging the effectiveness of the action, a minimum action threshold of ±5° can be set, and only when the attitude angle exceeds the threshold is the action determined to be effective (to avoid false triggering due to slight tremor); when the head naturally returns to the normal position, the attitude angle is zero, and the instruction is automatically terminated.

[0133] The above scheme improves the attitude recognition accuracy through the fusion of inertial sensors, avoids the drift error of a single sensor (for example, the accelerometer is resistant to vibration, and the gyroscope measures dynamic), directly maps the natural head movement and the direction of the unmanned aerial vehicle, replaces the traditional remote controller, and enhances the naturalness and immersion of the interaction. By setting the minimum action threshold and the centering strategy, the misoperation is reduced, and the continuity and predictability of the control are ensured.

[0134] In some embodiments, the calculation formula (5) of the short-time energy of the electromyographic signal is as follows:

[0135]

[0136] wherein E t represents the short-time energy of the electromyographic signal, w(n) represents a window function for smoothing the electromyographic signal, N represents the total number of divided time windows, n represents the index of the time window, and x(n+t) represents the electromyographic signal in the tth time window;

[0137] When the short-time energy of the continuous M time windows exceeds the intensity threshold, the occlusion action signal is generated, including:

[0138] When the short-time energy of the continuous M time windows exceeds the intensity threshold, the middle time point of the first window in the continuous M time windows is taken as the time point of triggering the occlusion action signal.

[0139] In this embodiment, the masseter surface electromyography signal refers to a weak electrical signal generated when the masseter muscle contracts, which is collected by the prefrontal / temporal lobe electrodes of the brain-computer interface device and used as a physiological trigger source of the landing instruction.

[0140] The short-time energy (Et) refers to the total energy of the electromyography signal in a certain time window, which quantifies the contraction strength of the masseter muscle (the higher the energy, the more forceful the contraction).

[0141] The window function (w(n)) can be used to smooth the electromyography signal with a Hanning window, reduce the spectral leakage caused by signal truncation, and improve the accuracy of energy calculation.

[0142] In actual application, the electromyography control module pre-processes the collected masseter signal, which includes but is not limited to: mean removal (eliminate direct current component), 50Hz notch filtering (remove power frequency interference), low-pass filtering (retain 20-500Hz effective frequency band). The short-time energy Et of the time window is calculated according to formula (5). When the Et of the continuous M (M>2) time windows exceeds the intensity threshold, and the electroencephalogram is in an inactive state (excluding false action in the focused state), a clenching action signal is generated; the middle time point of the first window in the continuous window is taken as the trigger time to ensure real-time performance.

[0143] The above scheme significantly reduces the false trigger rate by combining continuous time window detection with electroencephalogram inactive state determination (such as excluding short-term muscle activities such as speaking and chewing). The masseter signal as a physiological trigger source conforms to the natural behavior habits of humans, does not require additional learning cost, and is particularly suitable for special groups. The window function smoothing process improves the accuracy of energy calculation and provides a reliable basis for clenching action recognition.

[0144] In some embodiments, the method comprises:

[0145] When it is detected that the cognitive load of the user exceeds the preset load, the instruction generation module generates a hovering instruction, and sends the hovering instruction to the unmanned aerial vehicle through the flight control communication module to control the unmanned aerial vehicle to enter a hovering mode;

[0146] If the time when the cognitive load exceeds the preset load reaches a second preset duration, the instruction generation module generates a prompt instruction, and sends a prompt information through the flight control communication module, the prompt information being used to guide the user to rest;

[0147] The calculation formula (7) of the cognitive load of the user is as follows:

[0148]

[0149] Wherein, CL(t) is the cognitive load, P θ (t), P α (t), P β(t) respectively represent the average power of the theta, alpha, beta band in the current time window, the value range of the frequency band theta is [4, 8] Hz, the value range of the frequency band alpha is [8, 12] Hz, and the value range of the frequency band beta is [13, 30] Hz. Specifically, the theta band (4-8 Hz) is positively correlated with fatigue, the alpha band (8-12 Hz) is positively correlated with relaxation, and the beta band (13-30 Hz) is positively correlated with concentration.

[0150] In the embodiment, the cognitive load (CL(t)) refers to the brain cognitive pressure of the user when controlling the unmanned aerial vehicle, which is quantified by the power ratio of the theta, alpha and beta bands (θ enhancement and β weakening indicate that the load is increased).

[0151] The hovering instruction refers to an instruction to suspend the flight of the unmanned aerial vehicle when the cognitive load is too high, so as to reduce the operation pressure of the user and avoid the fatigue operation error of the user.

[0152] The preset load refers to a load threshold value (such as 50% higher than the resting state) set based on the resting state data of the user, and the preset load triggers a warning when exceeded.

[0153] If the cognitive load exceeds the standard for a second preset duration (such as 30 seconds), a prompt instruction (such as a beeping sound or an APP pop-up window) is generated to guide the user to rest.

[0154] The above scheme actively prevents operation errors caused by fatigue or overload through real-time cognitive load monitoring, thereby improving flight safety. The quantification method based on the brain wave band characteristics accurately reflects the cognitive state of the user, thereby providing a scientific basis for the warning mechanism. At the same time, the hierarchical response mechanism (hovering + prompt) balances safety and task continuity, thereby improving the user experience.

[0155] In some embodiments, as shown in FIG. 1, Figure 3 The generation of the direction control command based on the weighted fusion result of the electroencephalogram signal and the inertial attitude parameter includes:

[0156] Step S301: dynamically adjusting the decision weights of each mode according to the signal quality, specifically including: real-time distribution of weights according to the signal-to-noise ratio of the electroencephalogram signal and the acceleration variance of the inertial attitude parameter, and automatically increasing the decision proportion of the inertial attitude parameter when the electroencephalogram signal is disturbed;

[0157] Step S302: according to the adjusted weights, performing feature vector splicing on the electroencephalogram signal and the inertial attitude parameter, inputting the spliced feature vector into a pre-trained classification model, and generating a direction control command containing a turning angle and a speed.

[0158] In this embodiment, the signal quality refers to the signal-to-noise ratio (SNR, effective signal / noise) of the electroencephalogram signal and the acceleration variance of the inertial attitude parameter (the smaller the variance, the more stable), which is used to dynamically adjust the weight. The decision weight refers to the contribution ratio of the electroencephalogram signal and the inertial attitude parameter in the instruction generation (for example, when the electroencephalogram is disturbed, the weight is reduced. The pre-trained classification model refers to a model trained based on SVM, random forest and other algorithms, which maps the fusion feature vector to the unmanned aerial vehicle direction instruction (including turning angle, speed).

[0159] In actual application process, first, the electroencephalogram signal-to-noise ratio and the inertial parameter acceleration variance are calculated in real time; when the electroencephalogram is disturbed (SNR<10dB), the electroencephalogram weight is reduced (for example, from 0.6 to 0.3), and the inertial parameter weight is increased (for example, from 0.4 to 0.7). Then, the feature vectors of the electroencephalogram signal (dynamic attention score) and the inertial attitude parameter (attitude angle) are spliced according to the adjusted weight, and the pre-trained model (such as SVM) is input, and the direction control command (such as turning 30, speed 2m / s) is output.

[0160] The above scheme improves the control stability in complex environment (such as electromagnetic interference) through dynamic weight mechanism. Through machine learning model, the instruction accuracy is optimized, and the upgrade from "rough control" to "fine-grained adjustment" is realized. Multi-modal feature fusion enhances the robustness of the system, and adapts to the operation habits of different users and environmental changes.

[0161] In some embodiments, the brain-computer interface control system includes a display module 207;

[0162] The method further includes:

[0163] The flight control communication module receives the real-time flight state data feedback by the unmanned aerial vehicle, and displays the flight state data on the display module in real time, the flight state data including any one or more of flight height, current power of the unmanned aerial vehicle, attitude angle of the unmanned aerial vehicle and execution result of the control instruction.

[0164] In this embodiment, the display module includes unmanned aerial vehicle LED state light, mobile APP graphical interface and the like, which are used to show system state and flight data. The flight state data includes information feedback by the unmanned aerial vehicle in real time, including flight height, power, attitude angle (pitch angle / yaw angle / roll angle), instruction execution result (such as "taking off") and the like.

[0165] Status data feedback refers to the flight control communication module receiving real-time status data (altitude, battery level, attitude angle, etc.) from the drone via Bluetooth / WiFi and transmitting it to the display module. Flight status data can be displayed through multiple channels: LEDs use different colors / blinking patterns to indicate the status (e.g., solid green indicates readiness, flashing blue indicates flight); the APP interface graphically displays EEG fluctuation curves, flight trajectory, and command response results. In some embodiments, the content displayed by the display module is as follows: Figure 4 and Figure 5 As shown.

[0166] The above solution reduces the cognitive burden on users by constructing a closed loop of "intent-execution-feedback". Real-time data display improves operational safety (such as proactively landing when the battery is low). Multi-channel feedback enhances user immersion and makes the interaction more natural.

[0167] In a second aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for controlling a drone based on signal fusion using a brain-computer interface as described in the first aspect of the present invention.

[0168] The computer-readable storage medium may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0169] The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD ROM); the magnetic surface memory may be a disk storage device or a magnetic tape storage device.

[0170] The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synclink dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The computer-readable storage media described in the embodiments of the present invention are intended to include these and any other suitable types of memory.

[0171] like Figure 6 As shown, in a third aspect, the present invention provides an electronic device 10, including a processor 101 and a storage medium 102, wherein a computer program is stored on the storage medium, and the computer program, when executed by the processor, implements the method for controlling a drone based on a brain-computer interface using signal fusion as described in the first aspect of the present invention.

[0172] In some embodiments, the processor can be implemented by software, hardware, firmware or a combination thereof, and can use at least one of circuit, single or multiple Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), controller, microcontroller, microprocessor, so that the processor can execute part or all of the steps of the method of controlling the UAV based on the signal fusion-based brain-computer interface in various embodiments of the present application or any combination of the steps.

[0173] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of the present application, the patent protection scope of the present application should not be limited. Any technical solutions obtained by replacing or modifying the equivalent structure or equivalent process based on the essential concept of the present application, using the content described in the specification and drawings of the present application, and directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are all included in the patent protection scope of the present application.

Claims

1. A method for controlling a drone based on a signal fusion brain-computer interface, characterized in that, The application is suitable for a brain-computer interface control system, which comprises an electroencephalogram signal acquisition module, a concentration discrimination module, an inertial attitude sensing module, an electromyography control module, an instruction generation module and a flight control communication module. The method comprises the following steps: The electroencephalogram signal acquisition module acquires the electroencephalogram signal of a user in real time. The concentration discrimination module performs power spectral density analysis on the electroencephalogram signal, extracts the frequency domain features of multiple wave bands, and calculates a dynamic attention score according to the frequency domain features of the multiple wave bands; the electroencephalogram signal of a user in a resting state is acquired as reference data, the first threshold value and the second threshold value are calibrated through the reference data, the concentration is divided into three levels of resting state, preparation state and concentration state, wherein: the preparation state instruction is triggered when the dynamic attention score exceeds the first threshold value, and the concentration state is triggered when the dynamic attention score exceeds the second threshold value. The inertial attitude sensing module acquires inertial attitude parameters, which include the rotation direction and attitude angle of the head of the user, and the attitude angle includes the pitch angle, the yaw angle and the roll angle. The electromyography control module acquires the surface electromyography signal of the masseter muscle, calculates the short-time energy of the electromyography signal, and generates a clenching action signal when it is detected that the short-time energy of M consecutive time windows exceeds the intensity threshold value, M being a positive integer greater than 2. The instruction generation module performs any one of the following steps: When the duration of the concentration state reaches a first preset duration, a UAV takeoff instruction is generated; In the concentration state, a direction control command is generated based on the weighted fusion result of the electroencephalogram signal and the inertial attitude parameters; If the clenching action signal is triggered and the frequency domain feature power of the electroencephalogram signal is lower than the preset proportion of the resting state reference value, a UAV landing instruction is generated; The flight control communication module sends the generated takeoff instruction or direction control instruction or landing instruction to the UAV.

2. The method of controlling a drone using a signal fusion based brain computer interface as claimed in claim 1, wherein, The concentration discrimination module performs power spectral density analysis on the electroencephalogram signal, extracts the frequency domain features of multiple wave bands, and calculates a dynamic attention score according to the frequency domain features of the multiple wave bands, which comprises: The electroencephalogram signal acquisition module acquires the electroencephalogram signal of a user in real time. wherein PSD(f) represents a power spectral density, N represents a total number of divided time windows, n represents an index of a time window, X(n) represents a second electroencephalogram signal in an original time domain, f n represents an nth discrete frequency point, and a calculation formula thereof is: f s is a sampling frequency, represents a complex base function of a discrete Fourier transform, and f represents a frequency component; The concentration discrimination module performs power spectral density analysis on the electroencephalogram signal, extracts the frequency domain features of multiple wave bands, and calculates a dynamic attention score according to the frequency domain features of the multiple wave bands; the electroencephalogram signal of a user in a resting state is acquired as reference data, the first threshold value and the second threshold value are calibrated through the reference data, the concentration is divided into three levels of resting state, preparation state and concentration state, wherein: the preparation state instruction is triggered when the dynamic attention score exceeds the first threshold value, and the concentration state is triggered when the dynamic attention score exceeds the second threshold value. wherein PSD(f, t) represents the power spectral density at frequency f in the tth time window, F x represents the frequency set covered by the frequency band x, |F x represents the number of discrete frequency points contained in the frequency band x, P x (t) is a frequency domain feature, specifically representing the average power of the frequency band x in the time window t, the value range of the frequency band θ is [4, 8] Hz, the value range of the frequency band α is [8, 12] Hz, and the value range of the frequency band β is [13, 30] Hz. The inertial attitude sensing module acquires inertial attitude parameters, which include the rotation direction and attitude angle of the head of the user, and the attitude angle includes the pitch angle, the yaw angle and the roll angle. The electromyography control module acquires the surface electromyography signal of the masseter muscle, calculates the short-time energy of the electromyography signal, and generates a clenching action signal when it is detected that the short-time energy of M consecutive time windows exceeds the intensity threshold value, M being a positive integer greater than 2.

3. The method of controlling a drone with a signal fusion based brain computer interface according to claim 2, wherein, The instruction generation module performs any one of the following steps: When the duration of the concentration state reaches a first preset duration, a UAV takeoff instruction is generated; In the concentration state, a direction control command is generated based on the weighted fusion result of the electroencephalogram signal and the inertial attitude parameters; If the clenching action signal is triggered and the frequency domain feature power of the electroencephalogram signal is lower than the preset proportion of the resting state reference value, a UAV landing instruction is generated; The flight control communication module sends the generated takeoff instruction or direction control instruction or landing instruction to the UAV. The concentration discrimination module performs power spectral density analysis on the electroencephalogram signal, extracts the frequency domain features of multiple wave bands, and calculates a dynamic attention score according to the frequency domain features of the multiple wave bands, which comprises: The electroencephalogram signal acquisition module acquires the electroencephalogram signal of a user in real time. The concentration discrimination module performs power spectral density analysis on the electroencephalogram signal, extracts the frequency domain features of multiple wave bands, and calculates a dynamic attention score according to the frequency domain features of the multiple wave bands; the electroencephalogram signal of a user in a resting state is acquired as reference data, the first threshold value and the second threshold value are calibrated through the reference data, the concentration is divided into three levels of resting state, preparation state and concentration state, wherein: the preparation state instruction is triggered when the dynamic attention score exceeds the first threshold value, and the concentration state is triggered when the dynamic attention score exceeds the second threshold value. The inertial attitude sensing module acquires inertial attitude parameters, which include the rotation direction and attitude angle of the head of the user, and the attitude angle includes the pitch angle, the yaw angle and the roll angle. The electromyography control module acquires the surface electromyography signal of the masseter muscle, calculates the short-time energy of the electromyography signal, and generates a clenching action signal when it is detected that the short-time energy of M consecutive time windows exceeds the intensity threshold value, M being a positive integer greater than 2. The instruction generation module performs any one of the following steps: When the duration of the concentration state reaches a first preset duration, a UAV takeoff instruction is generated; In the concentration state, a direction control command is generated based on the weighted fusion result of the electroencephalogram signal and the inertial attitude parameters; If the clenching action signal is triggered and the frequency domain feature power of the electroencephalogram signal is lower than the preset proportion of the resting state reference value, a UAV landing instruction is generated; The flight control communication module sends the generated takeoff instruction or direction control instruction or landing instruction to the UAV.

4. The method of controlling a drone with a signal fusion based brain computer interface according to claim 1, wherein, The inertial attitude perception module comprises a three-axis accelerometer and a three-axis gyroscope. The inertial attitude perception module collects inertial attitude parameters, comprising: The inertial attitude perception module detects head linear acceleration through the accelerometer to determine the rotation direction, measures angular velocity through the gyroscope to calculate the attitude angle, and obtains the inertial attitude parameters according to the rotation direction and the attitude angle.

5. The method of controlling a drone with a signal fusion based brain computer interface according to claim 1, wherein, The calculation formula of the short-time energy of the electromyographic signal is as follows: where E t represents the short-time energy of the myoelectric signal, w(n) represents a window function used to smooth the myoelectric signal, N represents the total number of divided time windows, n represents the index of the time window, and x(n + t) represents the myoelectric signal within the tthtime window; When the short-time energy of the continuous M time windows is detected to exceed the intensity threshold, the occlusion action signal is generated, comprising: When the short-time energy of the continuous M time windows is detected to exceed the intensity threshold, the middle time point of the first window in the continuous M time windows is taken as the time point of triggering the occlusion action signal.

6. The method of controlling a drone with a signal fusion based brain computer interface according to claim 1, wherein, The method comprises: When the user cognitive load is detected to exceed the preset load, the instruction generation module generates a hovering instruction, and sends the hovering instruction to the unmanned aerial vehicle through the flight control communication module to control the unmanned aerial vehicle to enter the hovering mode; If the time of exceeding the preset load reaches a second preset time length, the instruction generation module generates a prompt instruction, and sends a prompt information through the flight control communication module, wherein the prompt information is used to guide the user to rest; The calculation formula of the user cognitive load is as follows: Wherein, CL(t) is cognitive load, P θ (t), P α (t), P β (t) respectively represent the average power of the theta, alpha, beta band in the current time window, the value range of the frequency band theta is [4, 8] Hz, the value range of the frequency band alpha is [8, 12] Hz, and the value range of the frequency band beta is [13, 30] Hz.

7. The method of controlling a drone with a signal fusion based brain computer interface according to claim 1, wherein, The direction control command is generated based on the weighted fusion result of the electroencephalogram signal and the inertial attitude parameters, comprising: According to the signal quality, the decision weights of each mode are dynamically adjusted, specifically comprising: according to the signal-to-noise ratio of the electroencephalogram signal and the acceleration variance of the inertial attitude parameters, the weights are real-time allocated, when the electroencephalogram signal is disturbed, the decision proportion of the inertial attitude parameters is automatically improved; According to the adjusted weights, the electroencephalogram signal and the inertial attitude parameters are feature vector spliced, and the spliced feature vector is input into the pre-trained classification model to generate the direction control command containing the turning angle and the speed.

8. The method of controlling a drone with a signal fusion based brain computer interface according to claim 1, wherein, The brain-computer interface control system comprises a display module; The method further comprises: The flight control communication module receives the real-time flight state data feedback by the unmanned aerial vehicle, and displays the flight state data on the display module in real time, wherein the flight state data comprises any one or more of the flight height, the current power of the unmanned aerial vehicle, the attitude angle of the unmanned aerial vehicle and the execution result of the control instruction.

9. A computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1-8.

10. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method of any one of claims 1-8.

Images