A remote drone control method and apparatus

Abstract

The application discloses a remote unmanned plane control method and device, and the method comprises the following steps: acquiring a local control scene, wherein the local control scene is obtained by processing three-dimensional environment information of the remote unmanned plane by using a compressed sensing control paradigm; acquiring an electroencephalogram signal generated when an operator watches the local control scene; analyzing the electroencephalogram signal by using an intention analysis algorithm to obtain original control instructions of the remote unmanned plane by the operator; optimizing the environment information and the original control instructions by using an environment self-adaptive control algorithm to obtain continuous unmanned plane control instructions; and sending the continuous unmanned plane control instructions to the remote unmanned plane and controlling the remote unmanned plane based on a kinematics model. The application effectively solves the problem of poor space perception and control ability of the existing brain-controlled remote unmanned plane technology.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to a remote unmanned aerial vehicle (UAV) control method and device. Background Technology

[0002] With the continuous advancement of technology, drone technology has been widely applied in military, civilian, and commercial fields. While traditional remote control methods are convenient, their limitations become increasingly apparent in certain special situations, such as complex environments, high-risk areas, or missions requiring high accuracy. To overcome these limitations, brain-computer interface technology is gradually emerging, with brain-controlled remote drones becoming a highly anticipated control method.

[0003] Brain-controlled remote-controlled drone technology is an innovative control method that uses brain signals to decode the operator's intentions and translate them into control commands for the drone, enabling precise control over long distances. Compared to traditional remote control methods, brain-controlled remote-controlled drone technology has many advantages. First, it eliminates the need for hand operation required in traditional remote control, allowing the operator to focus more on the task at hand. Second, brain-controlled remote-controlled drone technology possesses potentially high precision, enabling the completion of delicate control tasks in complex environments. Furthermore, brain-controlled technology provides a more autonomous means of control for people with mobility impairments or disabilities, expanding its application scope.

[0004] Currently, one of the most commonly used EEG signal decoding technologies in brain-controlled remote drone technology is SSVEP, or blinking visual evoked potentials. This technology triggers specific frequency EEG signals by flashing stimuli at different frequencies on a screen, allowing the operator to convey control intentions by focusing on these stimuli. While SSVEP technology has brought unprecedented possibilities to brain-controlled remote drone technology, several challenges and problems remain, particularly in spatial perception and control. Operators tend to perceive two-dimensional space better, but their cognitive abilities significantly decline in three-dimensional space. This makes it difficult for operators to precisely control the drone in three-dimensional space using their thoughts. Therefore, for obstacle information in three-dimensional space, operators need to make rapid and precise adjustments to the drone under remote control. Furthermore, the control commands output by the brain-computer interface decoding module suffer from discretization and latency issues, while real-world scenarios often require continuous and smooth control, limiting the application areas of brain-controlled remote drone technology. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a remote unmanned aerial vehicle (UAV) control method and device, which solves the problems of poor spatial perception and spatial control capabilities in existing brain-controlled remote UAV technologies.

[0006] To achieve the above objectives, in one aspect, the present invention provides a remote unmanned aerial vehicle (UAV) control method, comprising:

[0007] The local control scene is obtained by processing the three-dimensional environmental information of the remote UAV using a compressed sensing control paradigm.

[0008] Acquire the electroencephalogram (EEG) signals generated when the operator views the local control scene;

[0009] The EEG signal is analyzed using an intent analysis algorithm to obtain the operator's original control commands to the remote drone;

[0010] The environmental information and the original control command are optimized by an environmental adaptive control algorithm to obtain continuous UAV control commands, wherein the environmental information is obstacle position information and radar position information, and the original control command is a vector velocity control command;

[0011] The continuous UAV control commands are sent to the remote UAV, and the remote UAV is controlled based on the kinematic model.

[0012] Optionally, local control scenarios include:

[0013] The obstacle location information is obtained by collecting the three-dimensional environmental information using radar.

[0014] And collect the radar position information and target point position information;

[0015] Based on the obstacle location information, the radar location information, the target point location information, and the timestamp, the original data packet is obtained;

[0016] The original data packet is compressed and transmitted to the local machine. The compressed data is then decoded and reconstructed to obtain the local control scene.

[0017] Optionally, after acquiring the local control scene, the method further includes: setting up a plurality of dynamic flashing stimulus blocks around the remote drone in the local control scene, wherein the dynamic flashing stimulus blocks are used to control the direction, and each of the dynamic flashing stimulus blocks has a unique ID and a fixed frequency of flashing stimulus.

[0018] Optionally, the EEG signal is analyzed using an intent analysis algorithm to obtain the operator's original control commands to the remote drone, including:

[0019] The EEG signals are preprocessed and windowed to obtain a data matrix;

[0020] Based on the classification results of the EEG signals and the corresponding flashing stimulation of the dynamic flashing stimulation blocks, a template matrix is ​​obtained;

[0021] Input the data matrix and the template matrix into the correlation network to obtain the correlation coefficient matrix;

[0022] The maximum value in the correlation coefficient matrix is ​​obtained, and the maximum value is compared with a preset threshold to obtain the final EEG classification result;

[0023] Based on the final EEG classification result, the original control command is obtained.

[0024] Optionally, the EEG signals are preprocessed and windowed to obtain a data matrix, including:

[0025] The EEG signal is preprocessed by removing power frequency interference through notch filtering and extracting the frequency band containing characteristic potentials through bandpass filtering.

[0026] The preprocessed EEG signal is extracted and matrixed based on a preset time window to obtain the data matrix.

[0027] Optionally, the correlation network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer;

[0028] The input layer is used to input the data matrix and the template matrix into the convolutional layer;

[0029] The convolutional layer is used to obtain local features of the data matrix and the template matrix, and extract feature maps at different scales;

[0030] The pooling layer is used to downsample the feature map to obtain a pooled feature map;

[0031] The fully connected layer is used to flatten the pooled feature map into a vector, and to map the features to the space of the correlation coefficient matrix to obtain the normalized correlation coefficient matrix.

[0032] The output layer is used to output a normalized correlation coefficient matrix.

[0033] Optionally, the environmental information and the original control commands are optimized using an environmental adaptive control algorithm to obtain continuous UAV control commands, including:

[0034] S1. The kinematic model of the remote UAV is used as the system model, and the system model is discretized;

[0035] S2. Establish an objective function based on the mission characteristics of the remote UAV;

[0036] S3. Obtain the environmental information and the original control command, and in the current discrete time step, establish a predictive optimization problem based on the objective function, the environmental information, and the original control command;

[0037] S4. Solve the prediction optimization problem using an optimization algorithm to obtain the optimal control input for the remote UAV at the current time step;

[0038] S5. Proceed to the next discrete-time step, repeating steps S3-S4 to obtain continuous UAV control commands.

[0039] Optionally, the objective function is:

[0040]

[0041] Where J is the objective function, w speed The weights of the velocity error term are M and M is the prediction time domain length. desired It is the expected drone speed, v k w is the actual speed of the drone at time k. control Control the weights of the input variation terms, u k u k+1 u k+2 These are the UAV control inputs at times k, k+1, and k+2, respectively. obstacle It is the weight of the obstacle avoidance term, w i d is the weight of the i-th obstacle. i w is the distance from the drone to the i-th obstacle. smooth The weights for the smoothness term.

[0042] On the other hand, the present invention also provides a remote unmanned aerial vehicle (UAV) control device, comprising:

[0043] Includes: compressed sensing control unit, EEG signal acquisition unit, intent analysis unit, environment adaptive control unit, remote communication unit, and unmanned aerial vehicle (UAV) unit;

[0044] The compressed sensing control unit is used to process the three-dimensional environmental information of the remote UAV using the compressed sensing control paradigm to obtain the local control scene.

[0045] The EEG signal acquisition unit is used to acquire the EEG signals generated when the operator views the local control scene;

[0046] The intent analysis unit is used to analyze the electroencephalogram (EEG) signal using an intent analysis algorithm to obtain the operator's original control commands to the remote drone.

[0047] The environment adaptive control unit is used to optimize environmental information and the original control commands through an environment adaptive control model to obtain continuous UAV control commands.

[0048] The remote communication unit is used to send the continuous UAV control commands to the remote UAV;

[0049] The UAV unit is used to receive the continuous UAV control commands and control the remote UAV based on a kinematic model.

[0050] Optionally, the compressed sensing control unit includes an environmental information acquisition subunit, a compressed data transmission subunit, a two-dimensional scene reconstruction subunit, and a dynamic stimulus generation subunit;

[0051] The environmental information acquisition subunit is used to acquire the three-dimensional environmental information through radar, obtain obstacle location information, acquire radar location information and target point location information, and acquire the original data packet based on the obstacle location information, the radar location information, the target point location information and the timestamp;

[0052] The compressed data transmission subunit is used to compress the original data packet and transmit it locally;

[0053] The two-dimensional scene reconstruction subunit is used to decode and reconstruct the compressed data to obtain the local control scene;

[0054] The dynamic stimulus generation subunit is used to deploy flashing stimulus blocks around the drone in the local control scenario, assigning a unique ID and a fixed frequency of flashing stimulus to each direction of control.

[0055] Technical effects of the invention:

[0056] This invention employs compressed sensing control technology to transform the real-world scenario of a remotely controlled drone into a control scenario viewed locally by the operator. This eliminates the need for the operator to be on-site, enhancing the convenience of remote control. Through compressed sensing technology, this invention effectively encodes and transmits scenario information, ensuring the operator can accurately perceive the control scenario and further enhancing control precision.

[0057] This invention employs brain-computer interface technology to overcome the limitations of traditional remote control methods in special environments. By acquiring the brain signals generated by the operator while viewing the control scene, these signals are converted into control commands for the drone. This innovative control method not only eliminates the need for hand operation, allowing the operator to focus more on the task itself, but also provides more autonomous control methods for people with mobility impairments or disabilities, expanding the scope of drone applications.

[0058] This invention employs an environment-adaptive control algorithm to address the issues of discretization and latency in control commands within brain-computer interface (BCI) technology. By optimizing the operator's initial control commands and environmental information, continuous UAV control commands are generated. This optimization strategy not only improves the fluency of control commands but also enables more precise and stable UAV operation in complex environments. Attached Figure Description

[0059] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0060] Figure 1 A flowchart illustrating a remote unmanned aerial vehicle (UAV) control method provided in this application embodiment;

[0061] Figure 2 A functional module diagram of the compressed sensing control paradigm provided in the embodiments of this application.

[0062] Figure 3 This is a schematic diagram of the operation interface of the compressed sensing control paradigm provided in the embodiments of this application, wherein 301 is forward, 302 is backward, 303 is right, 304 is left, 305 is up, 306 is down, 307 is the height display of the UAV, and 308 is the obstacle.

[0063] Figure 4 A flowchart of the intent analysis algorithm provided in the embodiments of this application;

[0064] Figure 5 An architecture diagram of the correlation network provided in the embodiments of this application;

[0065] Figure 6 This is a schematic diagram of the input and output of the environment adaptive control algorithm provided in the embodiments of this application;

[0066] Figure 7 A synthetic schematic diagram of the desired drone speed provided in an embodiment of this application;

[0067] Figure 8 This is a schematic diagram of the structure of a remote unmanned aerial vehicle (UAV) control device according to an embodiment of this application;

[0068] Figure 9 This is a schematic diagram of the interface of the traditional SSVEP control paradigm provided in the embodiments of this application. In this diagram, 901 is the forward direction of the drone, 902 is the backward direction of the drone, 903 is the left direction of the drone, 904 is the right direction of the drone, 905 is the upward direction of the drone, 906 is the downward direction of the drone, and 907 is an image of the surrounding environment of the drone. Detailed Implementation

[0069] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0070] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0071] Example 1

[0072] like Figure 1 As shown, this embodiment provides a remote unmanned aerial vehicle (UAV) control method, including:

[0073] S101 uses a compressed sensing control paradigm to convert the real-world scene of a remote drone into a control scene viewed locally by the operator.

[0074] S102, the operator perceives the environmental information of the drone by viewing the control scene, generates the intention to control the remote drone, induces brain signals associated with the control scene, and uses brain signal acquisition equipment to acquire brain signals.

[0075] S103 uses an intent analysis algorithm to analyze EEG signals and obtain the operator's original control commands for the drone;

[0076] S104, The control commands are optimized using an environment adaptive control algorithm to obtain continuous UAV control commands;

[0077] S105 sends continuous UAV control commands to the remote UAV, enabling the remote UAV to adjust its position and attitude in three-dimensional space based on the kinematic model.

[0078] The compressed sensing control paradigm involves four main functional modules, such as... Figure 2The diagram shows the components: an environmental information acquisition module 201, a compressed data transmission module 202, a 2D scene reconstruction module 203, and a dynamic stimulus generation module 204. The implementation process of the compressed perception control paradigm is as follows: First, the 3D environment is acquired using radar deployed on a remote UAV. The radar data can detect the location information of obstacles. The environmental information acquired by the radar is 2D data, which needs to be transmitted back to the local machine. At the same time, the remote UAV's own position information and target point position information also need to be transmitted locally. The environmental information acquisition module packages the radar data, its own position information, target point position information, and timestamp generated by the remote UAV into a raw data packet, completing the preparation work for data transmission. During data transmission, the bandwidth and time required for transmission may become limiting factors because the raw data packet may be too large. The goal of the compressed data transmission module is to compress the acquired raw data to reduce the data volume and thus reduce transmission costs. After the compressed data is transmitted locally, the 2D scene reconstruction module is responsible for converting the compressed data back into the original scene information. By decoding and reconstructing the compressed data, the original 2D scene representation is rebuilt. Finally, the dynamic stimulus generation module deploys flashing stimuli around the UAV in the 2D scene. These flashing stimuli are not fixed-position stimuli, but rather adjust in real time as the drone's spatial position changes. The dynamic stimulus generation module assigns a unique ID and a fixed-frequency flashing stimulus to each direction of control. A schematic diagram of the compressed sensing control paradigm's user interface is shown below. Figure 3 As shown. The drone control requires control in six directions: forward (301), backward (302), right (303), left (304), upward (305), and downward (306). Their IDs are set to id1 to id2 respectively. N Where N=6. During control, the drone needs to avoid potential obstacles 308. The operator can view the drone's altitude display through the compressed sensing control paradigm 307.

[0079] The frequency of the flashing stimulus in the i-th direction is set to

[0080]

[0081] Among them, f min ,f max These correspond to the minimum and maximum feature frequencies set in the compressed sensing control paradigm, respectively. To avoid interference from harmonic factors in brain feature classification, f... min ,f max Set them to 7Hz and 10Hz respectively.

[0082] Intent analysis algorithms consist of four steps, such as... Figure 4The figures show the preprocessing and windowing of EEG signals, the construction of template EEG signals, the generation of correlation coefficients based on correlation networks, and the generation of EEG classification results based on discrimination thresholds.

[0083] S401, Preprocessing and Windowing of EEG Signals:

[0084] The acquired EEG signals were subjected to a 50Hz notch filter to remove power frequency interference, and a 1-30Hz bandpass filter was used to extract the frequency band containing characteristic potentials. The preprocessed EEG signals from each channel were truncated and matrixed based on a specified time window length λ. The resulting matrix is ​​a data matrix X, which can be represented as...

[0085]

[0086] Where N is the number of channels, f s λ is the sampling rate, T is the time length corresponding to the time window length, and λ is the time window length.

[0087] S402, Constructing template EEG signals

[0088] Based on the EEG classification results of each brain-controlled drone system and the flashing frequency f of their corresponding flashing stimulation blocks i Each of these requires generating a template matrix Y, which is defined as follows:

[0089]

[0090] Among them, f i The frequency of the scintillation block corresponding to the i-th EEG classification result is represented by f, where f represents the scintillation frequency for each scintillation stimulus block. i The frequency of the generated template matrix Y, where Q represents the length of the time window.

[0091] S403, Generating correlation coefficients based on correlation networks

[0092] The matrix-transformed data matrix X and template matrix Y are simultaneously input into the correlation network, and the network output is the correlation coefficient matrix R between the data matrix X and the template matrix Y.

[0093] S404, Generating EEG classification results based on discrimination threshold.

[0094] Obtain the maximum value in the correlation coefficient matrix R, and determine its relationship with the preset threshold r. If it is higher than the preset threshold r, output the i-th (index of the maximum value) EEG classification result; otherwise, do not output any EEG classification result.

[0095] Based on the EEG classification results, the original control commands for the remote drone were obtained.

[0096] The relevance network consists of an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer, such as... Figure 5 As shown:

[0097] S501, the input layer, is used to simultaneously input the data matrix X and the template matrix Y into the network. Each matrix is ​​treated as an image, and each element in the matrix is ​​used as the pixel value of the image. The matrixed data is then input into a standard convolutional neural network.

[0098] S502, a convolutional layer, uses multiple convolutional kernels to perform convolution operations on the input data. The convolution operation is used to capture local features of the input data. Each convolutional kernel slides through the input data, extracting features at different scales through learned weights.

[0099] S503, the pooling layer, downsamples the feature map, reducing its dimensionality and extracting the main features. Available pooling methods include max pooling and average pooling.

[0100] S504, a fully connected layer, flattens the pooled feature map into a vector, and then maps the features to the space of the correlation coefficient matrix R through the fully connected layer. The output of the fully connected layer is normalized to obtain the final correlation matrix.

[0101] S505, the output layer, the last layer outputs a normalized correlation coefficient matrix R. This matrix contains correlation information between the data matrix X and the template matrix Y.

[0102] Environmental adaptive control algorithms, such as Figure 6 As shown, the input to the environment adaptive control algorithm is environmental information and the original control commands, and the output is continuous UAV control commands. Environmental information includes radar data collected by the radar deployed on the UAV and the UAV's own position information. The original control commands are vector velocity control commands, and the continuous UAV control commands are also vector velocity control commands. The environment adaptive control algorithm is designed using a model optimization problem approach, with the objective function J being...

[0103]

[0104] Among them, v desired This refers to the desired drone speed, representing the final drone speed achieved through control. desired It is through the original control command v at time k. brain The actual speed v of the drone at time k-1 k Obtained by vector synthesis, such as Figure 7 As shown.

[0105] v k It is the actual speed of the drone at time k.

[0106] u k u k+1 u k+2 These are the UAV control inputs at times k, k+1, and k+2, respectively, and represent the UAV's vector velocity.

[0107] d i It is the distance from the drone to the i-th obstacle, which can be obtained from radar data and the drone's own position information.

[0108] M is the prediction time domain length, representing the number of time points considered in the optimization problem.

[0109] w speed It is the weight of the speed error term, which controls the difference between the actual speed and the expected speed of the drone.

[0110] w control Controlling the weights of input variations and encouraging smooth control input across consecutive time points helps avoid drastic control adjustments.

[0111] w obstacle The weight of the obstacle avoidance term is used to avoid collisions with obstacles. Depending on the distance between the drone and the obstacle, this weight can be adjusted to maintain a safe distance.

[0112] w i The weight of the i-th obstacle is used to assign different weight coefficients to obstacles at different distances, and this weight can be used to maintain a safe distance.

[0113] w smooth The weights of the smoothness term encourage the control input to remain smooth. By controlling the smoothness of the input, the jitter and instability of the drone can be reduced.

[0114] Kinematic models are divided into position kinematic equations and posture kinematic equations.

[0115] The kinematic equations of position show that the change of position of a UAV in the Earth coordinate system can be described by its velocity:

[0116]

[0117] Among them, v x v y v z The x, y, and z axes represent the velocities of the UAV along the Earth coordinate system, where x, y, and z represent the UAV's coordinate positions in the Earth coordinate system. This represents the velocity components of the drone along the x, y, and z axes.

[0118] Attitude kinematics equations: The attitude change of a UAV is described by its angular velocity.

[0119]

[0120] In this embodiment, p, q, and r represent the angular velocities about the three axes of the body coordinate system, respectively. These represent the roll angle, pitch angle, and yaw angle, respectively.

[0121] Under conditions of small perturbations, i.e., when the changes in various angles are relatively small, the rate of change of attitude angles is approximately equal to the rotational angular velocity of the aircraft. Therefore, the attitude kinematic equations can be simplified to:

[0122]

[0123] The calculation process of the environmental adaptive control algorithm is as follows:

[0124] The kinematic model of the UAV is used as the system model, and it is discretized and described using difference equations;

[0125] Design the objective function J based on the characteristics of UAV missions;

[0126] Acquire environmental information and raw control commands;

[0127] At discrete time step k, a predictive optimization problem is established based on the objective function J, environmental information, and original control commands to predict the behavior of the UAV within a future time step (time steps are denoted as k, k+1, ..., k+W, where W is the prediction step size);

[0128] The above-established predictive optimization problem is solved using an optimization algorithm (such as nonlinear programming or quadratic programming) to find the optimal control input sequence u(k), u(k+1),..., u(k+W) for the UAV.

[0129] Extract the optimal control input u(k) of the remote UAV at the current time step k from the solution of the optimization problem; apply u(k) to the actual system and advance the system to the next time step k+1 to obtain the optimal control input sequence u(k+1),...,u(k+W) of the UAV; and extract the optimal control input u(k+1) of the remote UAV at the current time step k+1 from the solution of the optimization problem.

[0130] Repeat the above process, continuously updating the UAV's control input to accommodate the system's actual behavior and measurement errors, ultimately obtaining the input sequence of the remote UAV, i.e., the continuous UAV control commands u(k), u(k+1),..., u(k+W). At each time step k, the optimization problem is recalculated, but only the first control input u(k) is applied.

[0131] Example 2

[0132] like Figure 8 As shown, this embodiment of the invention also provides a remote unmanned aerial vehicle (UAV) control device, including:

[0133] The compressed sensing control unit 801 is responsible for converting the real-world scene from the remote drone into a control scene viewed locally by the operator. Through compressed sensing technology, scene information is effectively encoded and transmitted, ensuring that the operator can obtain an accurate control scene locally.

[0134] The EEG signal acquisition unit 802 is used to acquire the EEG signals generated by the operator while viewing the control scene. These signals are associated with the operator's control intentions towards the remote drone.

[0135] The intent analysis algorithm unit 803 uses an intent analysis algorithm to process and analyze the brain signals acquired from the brain signal acquisition device. By identifying characteristic patterns in the brain signals, the operator's original control commands for the drone are obtained.

[0136] The environment adaptive control unit 804 optimizes and adjusts the control commands based on the operator's original control commands and environmental information, generating continuous UAV control commands. This ensures stable control of the UAV in various environments.

[0137] The remote communication unit 805 is used to send continuous drone control commands to a remote drone. This ensures that control commands generated locally by the operator can be transmitted to the drone in real time.

[0138] The UAV unit 806 allows the UAV to adjust its position and attitude in three-dimensional space based on the received continuous control commands.

[0139] The compressed sensing control unit includes an environmental information acquisition subunit, a compressed data transmission subunit, a two-dimensional scene reconstruction subunit, and a dynamic stimulus generation subunit.

[0140] The environmental information acquisition subunit is used to acquire three-dimensional environmental information through radar, obtain obstacle location information, and acquire radar location information and target point location information. Based on obstacle location information, radar location information, target point location information and timestamp, it acquires raw data packets.

[0141] The compressed data transmission subunit is used to compress the original data packets and transmit them locally;

[0142] The 2D scene reconstruction subunit is used to decode and reconstruct the compressed data to obtain the local control scene;

[0143] The dynamic stimulus generation subunit is used to deploy flashing stimulus blocks around the drone in the local control scenario, assigning a unique ID and a fixed frequency of flashing stimulus to each direction of control.

[0144] Example 3

[0145] To verify the effectiveness and implementation of the remote unmanned aerial vehicle (UAV) control method and device proposed in this application, the following scientific experiments were conducted.

[0146] Subject recruitment:

[0147] To evaluate the methods and equipment proposed in this application, eight healthy undergraduate or graduate students were selected as experimental volunteers. These volunteers were between 23 and 30 years old to ensure a relatively balanced age distribution of the subjects. A series of precautions were taken before the experiment officially began to ensure the health of the volunteers and the accuracy of the experimental data.

[0148] During the volunteer selection process, a detailed questionnaire survey was conducted first to confirm that each volunteer was in good physical condition before the experiment. The questionnaire covered multiple aspects, including but not limited to: (1) whether there was a history of head injury; (2) whether there was any mental illness; (3) whether there was any drug dependence in the past six months; (4) whether there was any intellectual disability (IQ less than 70); (5) whether there was any difficulty in understanding Chinese; and (6) whether there was any electroconvulsive therapy in the past six months. All volunteers completed the questionnaire in detail, and the results showed that there were no relevant conditions in the above aspects, ensuring the suitability of the volunteers. In addition, the volunteers' vision was within the normal range or had been corrected by corrective means, which would not affect the collection of experimental data. There were no significant differences among the volunteers in terms of age, gender, and education level, ensuring the comparability of the experiment.

[0149] Before the experiment, each volunteer was ensured to have sufficient rest to be in optimal condition during the experiment. All volunteers received detailed explanations and instructions before participating, including the experimental procedures, data usage, and possible consequences. Having fully understood the experimental content, they voluntarily agreed to participate and signed the relevant informed consent forms, ensuring the transparency and compliance of the experiment.

[0150] Data collection:

[0151] To ensure the accuracy and reliability of the experimental results, data was collected in a typical everyday environment without using a shielded room or electromagnetic interference shielding measures, and ambient noise interference was not eliminated to maintain the authenticity of the experiment. The experimental operators clearly explained the objectives and content of the experiment to the participants to ensure they understood the process and purpose.

[0152] In this experiment, an OpenBCI amplifier device (https: / / openbci.com / ) was used to collect EEG signals. During EEG acquisition, the reference electrode was located at the top of the head, and the ground electrode was located at the top of the forehead. In addition to the reference and ground electrodes, EEG data from three channels (O1, O2, and OZ) were simultaneously acquired, and the electrode impedance was strictly kept below 10KΩ to ensure signal stability.

[0153] This experiment uses the OpenBCI EEG device, an open-source brain-computer interface hardware designed to make it easier for researchers, developers, and makers to acquire and analyze EEG signals, enabling direct interaction between the brain and a computer. OpenBCI provides open hardware design and software tools, allowing users to build their own EEG signal acquisition systems, while also providing easy-to-use interfaces and libraries for data analysis and processing.

[0154] The visual stimuli in the experiment were presented to the participants via an LCD screen (DELL 27, 1920x1080, 60fps, 44.0x31.9cm). During EEG data acquisition, participants were ensured to maintain a consistent visual distance by keeping their gaze at a distance of 80±10cm from the stimulus display.

[0155] Experimental procedure:

[0156] This experiment aims to compare and analyze the performance differences between the remote UAV control method and device proposed in this application and a UAV control method and device based on the traditional SSVEP control paradigm. Therefore, a UAV control method and device based on the traditional SSVEP control paradigm are also implemented. The interface of the traditional SSVEP control paradigm is as follows: Figure 9 As shown, six flashing squares are fixed on the screen, representing the drone's forward direction (901), backward direction (902), left direction (903), right direction (904), upward direction (905), and downward direction (906). While observing the flashing squares, the operator simultaneously acquires real-time images (907) of the drone's surrounding environment via an external camera, which are then displayed on the screen. This allows the operator to adjust the drone's operational status in real time based on environmental information.

[0157] Experimental results:

[0158] The remote unmanned aerial vehicle (UAV) control method and device proposed in this application have significantly higher response time and information transmission rate than UAV control methods and devices based on the traditional SSVEP control paradigm.

[0159] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A remote drone control method, characterized by, include: Acquire a local control scene, wherein the local control scene is acquired by processing the three-dimensional environmental information of the remote UAV using a compressed sensing control paradigm; Acquire the electroencephalogram (EEG) signals generated when the operator views the local control scene; The EEG signals are analyzed using an intent analysis algorithm to obtain the operator's original control commands to the remote drone, including: The EEG signals are preprocessed and windowed to obtain a data matrix; Based on the classification results of the EEG signals and the corresponding flashing stimulation of the dynamic flashing stimulation blocks, a template matrix is ​​obtained; Input the data matrix and the template matrix into the correlation network to obtain the correlation coefficient matrix; Obtain the maximum value in the correlation coefficient matrix and compare the maximum value with a preset threshold to obtain the final EEG classification result; Based on the final EEG classification result, the original control command is obtained; An environmental adaptive control algorithm is used to optimize the environmental information and the original control command to obtain continuous UAV control commands. The environmental information includes obstacle position information and radar position information, and the original control command is a vector velocity control command. Obtaining continuous UAV control commands includes: S1. The kinematic model of the remote UAV is used as the system model, and the system model is discretized; S2. Establish an objective function based on the mission characteristics of the remote UAV; S3. Obtain the environmental information and the original control command, and in the current discrete time step, establish a predictive optimization problem based on the objective function, the environmental information, and the original control command; S4. Solve the prediction optimization problem using an optimization algorithm to obtain the optimal control input for the remote UAV at the current time step; S5. Proceed to the next discrete-time step, repeat steps S3-S4, and obtain continuous UAV control commands. The objective function is: ； in, Let be the objective function. It is the weight of the speed error term. M It predicts the length of the time domain. The desired drone speed, It is the first k The actual speed of the drone at any given moment. Control the weights of input variation terms. , , They are the first k、k +1 、k The drone control input at time +2 It is the weight of the obstacle avoidance term. It is the first i The weight of each obstacle Is it a drone to the first i The distance to each obstacle The weights for the smoothness term; The continuous UAV control commands are sent to the remote UAV, and the remote UAV is controlled based on the kinematic model.

2. The remote drone control method of claim 1, wherein, Acquiring local control scenarios includes: The obstacle location information is obtained by collecting the three-dimensional environmental information using radar. And collect the radar position information and target point position information; Based on the obstacle location information, the radar location information, the target point location information, and the timestamp, the original data packet is obtained; The original data packet is compressed and transmitted to the local machine. The compressed data is then decoded and reconstructed to obtain the local control scene.

3. The remote drone control method of claim 2, wherein, After acquiring the local control scene, the method further includes: setting up several dynamic flashing stimulus blocks around the remote drone in the local control scene, wherein the dynamic flashing stimulus blocks are used to control the direction, and each of the dynamic flashing stimulus blocks has a unique ID and a fixed frequency of flashing stimulus. 4.The remote drone control method of claim 1, wherein, The EEG signals are preprocessed and windowed to obtain a data matrix, including: The EEG signal is preprocessed by removing power frequency interference through notch filtering and extracting the frequency band containing characteristic potentials through bandpass filtering. The preprocessed EEG signal is extracted and matrixed based on a preset time window to obtain the data matrix.

5. The remote drone control method of claim 1, wherein, The correlation network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; The input layer is used to input the data matrix and the template matrix into the convolutional layer; The convolutional layer is used to obtain local features of the data matrix and the template matrix, and extract feature maps at different scales; The pooling layer is used to downsample the feature map to obtain a pooled feature map; The fully connected layer is used to flatten the pooled feature map into a vector, and to map the features to the space of the correlation coefficient matrix to obtain the normalized correlation coefficient matrix. The output layer is used to output a normalized correlation coefficient matrix.

6. A remote unmanned aerial vehicle (UAV) control device, used to implement the method according to any one of claims 1-5, characterized in that, include: Compressed perception control unit, EEG signal acquisition unit, intent analysis unit, environment adaptive control unit, remote communication unit, and unmanned aerial vehicle (UAV) unit; The compressed sensing control unit is used to process the three-dimensional environmental information of the remote UAV using the compressed sensing control paradigm to obtain the local control scene. The EEG signal acquisition unit is used to acquire the EEG signals generated when the operator views the local control scene; The intent analysis unit is used to analyze the electroencephalogram (EEG) signal using an intent analysis algorithm to obtain the operator's original control commands to the remote drone. The environment adaptive control unit is used to optimize environmental information and the original control commands through an environment adaptive control model to obtain continuous UAV control commands. The remote communication unit is used to send the continuous UAV control commands to the remote UAV; The UAV unit is used to receive the continuous UAV control commands and control the remote UAV based on a kinematic model.

7. The remote unmanned aerial vehicle control device as described in claim 6, characterized in that... The compressed sensing control unit includes an environmental information acquisition subunit, a compressed data transmission subunit, a two-dimensional scene reconstruction subunit, and a dynamic stimulus generation subunit. The environmental information acquisition subunit is used to acquire the three-dimensional environmental information through radar, obtain obstacle location information, acquire radar location information and target point location information, and acquire the original data packet based on the obstacle location information, the radar location information, the target point location information and the timestamp; The compressed data transmission subunit is used to compress the original data packet and transmit it locally; The two-dimensional scene reconstruction subunit is used to decode and reconstruct the compressed data to obtain the local control scene; The dynamic stimulus generation subunit is used to deploy flashing stimulus blocks around the drone in the local control scenario, assigning a unique ID and a fixed frequency of flashing stimulus to each direction of control.

Images