Skin multi-modal human-drone interface system and method of use thereof

Abstract

The invention provides a skin multi-modal human-drone interface system. The skin multi-modal human-drone interface system enables a drone to operate in a dynamic and complex environment. The interface system includes a base station configured to: receive hand orientation data of a user and obstacle data within a warning distance of the drone; based on the correlation between the hand orientation data and the obstacle data, generating corresponding control commands for providing tactile feedback to fingers of a user, stimulating a plurality of muscle groups of arms of the user and controlling the unmanned aerial vehicle; the drone controls a haptic feedback (DCTF) module configured to: collect hand orientation data of the user, and deliver the haptic feedback to a finger of the user; and a neuromuscular electrical stimulation force feedback (NMESF) module configured to collect obstacle data within an alert distance of the drone and to deliver stimulation current to a plurality of muscle groups of an arm of the user.

Description

Technical Field

[0001] This invention generally relates to unmanned aerial vehicle (UAV) control technology. More specifically, this invention relates to a skin-based multimodal human-UAV interfacing system and its method of use. Background Technology

[0002] Unmanned aerial vehicles (UAVs), commonly known as "drones," have attracted widespread attention and shown a significant market expansion trend in recent years. More and more drones are being used in various aspects of people's daily lives, covering transportation, infrastructure, and many other fields. Despite significant progress in artificial intelligence and drone control algorithms in recent years, fully autonomous drone systems are still considered unreliable for navigation in complex and rapidly changing environments. In contrast, human operators demonstrate greater robustness in emergency or dynamic environments, and their involvement still offers significant advantages. This has spurred increasing attention to the research and development of drone control methods.

[0003] Currently, the mainstream control methods for commercial drones still rely primarily on joystick-based controllers or touch interfaces such as smartphones. However, these conventional methods often face challenges such as high control complexity, long training cycles, and heavy cognitive burden on operators. To address these issues, researchers have proposed various novel and intuitive drone operation methods, including brain-computer interface (BCI) systems based on electroencephalography (EEG), voice control, gesture control based on electromyography (EMG), computer vision gesture recognition, and gesture recognition using mechanical sensors or inertial measurement units (IMUs). Some studies have even combined multiple control methods to improve overall performance. These intuitive control methods, due to their ease of learning, have been shown to shorten operator training time, reduce cognitive burden, and allow users to focus more on the flight mission. Furthermore, they offer a stronger sense of control and higher pilot satisfaction. However, unfortunately, most of these systems still rely on bulky or rigid devices, limiting wearability.

[0004] While significant progress has been made in drone control methods in recent years, timely feedback from drones to users remains crucial for efficient operation. Throughout the development of human-machine interaction design, continuous feedback from machines to operators has always played a key role in handling complex situations, as system errors are difficult to completely avoid. In addition to basic visual feedback, research on traditional manned aircraft has shown that introducing additional sensory dimensions such as tactile feedback and constructing multimodal feedback systems can help improve control performance and support more accurate decision-making. Currently, most commercial drones only provide first-person view (FPV) visual feedback. However, a limited field of view may not provide sufficient environmental perception information, easily leading to delayed responses to changes in the flight environment. Furthermore, some key information, such as attitude and motion perception provided by the human vestibular system, is missing in drone operation due to its "unmanned" nature. Some research has attempted to integrate tactile or force feedback mechanisms into the drone control process to guide users and improve operational efficiency. However, constraints such as bulky equipment, insufficient feedback resolution, and reliance on fixed installations limit the widespread application of these feedback systems in drone control.

[0005] While visual feedback has been proven to aid in stable drone flight, limitations remain in achieving omnidirectional perception. Even when drones can capture images from multiple angles, the operator's limited field of vision still presents a challenge. This limitation is more pronounced in drones compared to traditional fixed-wing aircraft, as drones can fly in multiple directions, including left, right, and backward. For example, with intuitive gesture control, when a user raises their wrist to make the drone fly backward, the operator may not be able to detect obstacles behind them in time without rearward visual information, leading to continued backward movement and a risk of collision.

[0006] One potential solution is to use stretchable electronics to replace traditional rigid circuit boards. Stretchable electronics combine high flexibility with skin-like properties, making them ideal for applications requiring human-body integration. They have already found widespread use in biosignal monitoring, medical treatment, and human-computer interaction. Some research has also explored the application of stretchable electronics in drone operation, but most studies focus on the control mechanism itself and have not yet met the need to integrate stretchable electronic feedback systems with drones to achieve rapid and accurate responses to user gestures and provide an intuitive control experience. Summary of the Invention

[0007] To address the aforementioned needs, this invention provides an epidermal multimodal closed-loop system for human-drone interfacing, which enables intuitive drone control, thereby simulating the human vestibular system and meeting the requirement for rapid and accurate response to user gestures.

[0008] According to one aspect of the present invention, a multimodal human-unmanned aerial vehicle (UAV) interfacing system is provided, the system comprising: a base station configured to receive hand orientation data of a user and obstacle data within a warning distance of a UAV, and based on the correlation between the hand orientation data and the obstacle data, generating corresponding control commands for providing tactile feedback to the user, stimulating multiple muscle groups of the user, and controlling the UAV; a UAV control tactile feedback (DCTF) module, the DCTF module being connected to the user and the base station, and configured to collect the hand orientation data of the user and deliver the tactile feedback to the user; and a neuromuscular electrical stimulation force feedback (NMESF) module, the NMESF module being connected to the base station, the user, and the UAV, and configured to collect obstacle data within a warning distance of the UAV and deliver stimulating currents to stimulate multiple muscle groups of the user.

[0009] Preferably, the base station is further configured to: receive video signals from a camera integrated in the drone; and transmit the real-time video signals to a virtual reality headset worn by the user, so as to enable the user to set hand orientation for drone control.

[0010] Preferably, the DCTF module includes a DCTF control unit and a haptic actuator module; the DCTF control unit is attached to the back of the user's hand and configured to: collect the user's hand orientation data, transmit the hand orientation data to the base station, receive actuator control commands from the base station, and transmit the actuator control commands to the haptic actuator module; and the haptic actuator module includes a two-dimensional array of haptic actuators connected via a connector array, the haptic actuator array being attached to the user's fingers and configured to receive the actuator control commands from the DCTF control unit and deliver the haptic feedback to the user's fingers.

[0011] Preferably, the DCTF control unit includes: an inertial measurement unit configured to collect the user's hand orientation data; a first communication module configured to transmit the collected hand orientation data to the base station and receive actuator control commands from the base station; and a first microcontroller configured to generate actuator drive signals to drive the haptic actuator array to deliver the haptic feedback to the user's fingers.

[0012] Preferably, each haptic actuator includes: a magnet; and a polyethylene terephthalate (PET) film having a cantilever structure to facilitate unrestricted movement of the magnet.

[0013] Preferably, the haptic actuator array and the connector array are manufactured on a flexible and stretchable circuit board.

[0014] Preferably, the flexible and stretchable circuit board includes a stretchable substrate sandwiched between a pair of serpentine patterned conductive layers.

[0015] Preferably, the NMESF module includes an obstacle detection unit and an NMESF control unit; the obstacle detection unit is located on the top of the drone and is configured to collect obstacle data within the drone's detection range and transmit the obstacle data to the base station; the NMESF control unit is attached to the user's arm and is configured to receive stimulation commands from the base station and generate stimulation currents to stimulate multiple muscle groups in the user's arm.

[0016] Preferably, the obstacle detection unit includes: at least three laser detectors configured to detect obstacle information in the left, right, and rear regions of the UAV, respectively; a second microcontroller configured to process the detected obstacle information into obstacle data; and a second communication module configured to transmit the obstacle data to the base station for generating the stimulus command.

[0017] Preferably, the NMESF control unit includes: a third communication module configured to receive the stimulation command from the base station; a third microcontroller configured to generate a control signal based on the stimulation command; a current driver configured to provide the stimulation current; a switch configured to receive the control signal to turn the current driver on / off; and a plurality of stimulation electrodes attached to the user's arm and electrically coupled to the current driver to deliver the stimulation current to a plurality of muscle groups in the user's arm.

[0018] Preferably, the plurality of stimulation electrodes are fabricated on a flexible and stretchable circuit board.

[0019] Preferably, the flexible and stretchable circuit board includes a stretchable substrate, a serpentine patterned conductive layer, and a stretchable and replaceable conductive hydrogel.

[0020] According to a second aspect of the invention, a method is provided for enabling a user to control a drone in an intuitive manner using the epidermal multimodal human-drone interfacing system according to the first aspect.

[0021] The method comprises constructing a control-haptic feedback loop by: capturing the user's hand orientation data via the DCTF module; transmitting the captured hand orientation data to the base station via the DCTF module; converting the hand orientation data into drone control commands via the base station; transmitting the drone control commands to the drone via the base station; detecting the drone's flight angle and speed via the drone and transmitting the detected flight angle and speed to the base station; estimating the drone's flight attitude and aerodynamic conditions via the base station; generating a two-dimensional haptic feedback map via the base station; generating actuator control commands based on the two-dimensional haptic feedback map via the base station; and transmitting the actuator control commands to the DCTF module to induce haptic feedback for the user.

[0022] The method further includes constructing a force-control feedback loop by: collecting obstacle data within the drone's detection range via the NMESF module; transmitting the obstacle data to the base station via the NMESF module; converting the obstacle data into muscle stimulation commands via the base station and transmitting the muscle stimulation commands to an NMESF control unit mounted on the user's forearm; and generating force feedback stimuli via the NMESF control unit to influence the user's hand movements, thereby controlling the drone via the DCTF module.

[0023] The method further includes constructing a visual feedback loop by receiving a real-time video signal from a camera integrated in the drone and transmitting the video signal to the user's virtual reality headset to enable the user to set hand orientation for drone control.

[0024] Employing a flexible and stretchable electronic design, the system boasts high ductility and excellent wearability. Compared to previous attempts in human-drone interaction development, this design not only eliminates reliance on bulky devices but also integrates more functions while maintaining a lightweight design and improving overall performance.

[0025] This invention also provides a user-friendly drone control method that enables efficient operation while shortening the training cycle and reducing cognitive burden. By acquiring information about the drone's surrounding flight environment and attitude, flight stability can be significantly improved; and with the help of a haptic feedback loop, the risk of accidental collisions caused by unseen obstacles is effectively reduced, enabling the drone to operate safely in dynamic and complex environments such as cities and narrow spaces. Furthermore, the system's slim, skin-friendly design maximizes wearability and enhances comfort during human-drone interaction. Attached Figure Description

[0026] Embodiments of the invention are described in more detail below with reference to the accompanying drawings, in which:

[0027] Figure 1 A block diagram of a skin-based multimodal human-unmanned aerial vehicle (UAV) interface system according to an embodiment of the present invention is shown.

[0028] Figure 2A A block diagram of a DCTF module according to an embodiment of the present invention is shown; Figure 2B The three-dimensional structure of the DCTF module is shown. Figure 2C and 2D The front and rear views of the DCTF module installed on the back of the user's hand and fingers are shown respectively.

[0029] Figure 3A and 3B The three-dimensional structure and best-view images of a DCTF control unit according to one embodiment are shown respectively.

[0030] Figure 4A A close-up representation of the connector's three-dimensional structure is shown; Figure 4B A close-up representation of the three-dimensional structure of the connector assembled with the actuator and switch is shown; Figure 4C A best-looking image shows two haptic actuators connected via a connector; and Figure 4D A best-looking image of a single connector is shown.

[0031] Figure 5 A cross-sectional view of a double-layer stretchable circuit board used for a haptic actuator module is shown.

[0032] Figure 6 The manufacturing process of the stretchable circuit board used in the haptic actuator module is demonstrated.

[0033] Figure 7 The configuration of a single haptic actuator according to an embodiment of the present invention is shown.

[0034] Figure 8 The pulse width modulation signal used to control the vibration of a haptic actuator with different duty cycles is shown.

[0035] Figure 9 The operating principle of haptic feedback is shown.

[0036] Figure 10 This demonstrates the mapping process of drone angle data onto a virtual haptic feedback map.

[0037] Figure 11 It demonstrates how the center point and effective area move accordingly when the drone undergoes an angle change.

[0038] Figure 12AThe original parabolic decay function with focus 2.5 is shown; and Figure 12B It shows in Figure 12A Under the original decay function, the center point, effective area, and duty cycle of each actuator.

[0039] Figure 13A The decay function after the UAV rotates is shown; and Figure 13B It shows in Figure 13A Under the decay function, the center point, effective area, and duty cycle of each actuator.

[0040] Figure 14A The decay function with a larger effective region is shown, namely the parabolic decay function with a focus of 4.5; and Figure 14B It shows in Figure 14A Under the decay function, the center point, effective area, and duty cycle of each actuator.

[0041] Figure 15A It shows that the decay function becomes linear decay; and Figure 15B It shows in Figure 15A Under the decay function, the center point, effective area, and duty cycle of each actuator.

[0042] Figure 16A A block diagram of an NMESF control unit 310 according to an embodiment of the present invention is shown; and Figure 16B This demonstrates how to install the NMESF control unit on a user's forearm.

[0043] Figure 17A and 17B The three-dimensional structure and best-view images of the NMESF control unit main circuit board according to one embodiment are shown.

[0044] Figure 18A and 18B The three-dimensional structure and best images of the NMESF control unit stimulation patch according to one embodiment are shown.

[0045] Figure 19A A block diagram of an obstacle detection unit according to an embodiment of the present invention is shown; and Figure 19B This demonstrates how to position the obstacle detection unit 320 on the top of the drone.

[0046] Figure 20 The placement of the stimulation electrodes for force feedback in NMES is shown.

[0047] Figure 21 Three rotational directions of the operator's hand induced by NMES are shown.

[0048] Figure 22AThe variable stimulation current output by the NMESF module is shown; Figure 22B The NMESF module outputs stimulation current pulses with different duty cycles, as shown. Figure 22C The relationship between the control voltage of the NMESF control unit DAC and the stimulation output current is shown. Figure 22D The variable frequency output of the NMESF module is shown.

[0049] Figures 23A to 23C The relationship between the stimulation current and the torque in all three bending directions is shown.

[0050] Figure 24 The equivalent circuit of the NMES stimulation pathway is shown.

[0051] Figure 25 The average frequency response of the load, including the two electrodes and the user's muscle, under electrical stimulation is shown.

[0052] Figure 26 This demonstrates the relationship between stimulation frequency and the torque generated by wrist flexion.

[0053] Figure 27 The impedance response of the stimulation electrode under different voltages is shown.

[0054] Figure 28 This paper presents a conceptual description of the operational mechanism of the present invention as an interface between the user and the drone.

[0055] Figure 29A and 29B The diagram and flowchart illustrate the operational logic flow of this system as an interface between the user and the drone.

[0056] Figure 30 The IMU performance evaluation of the UAV is shown, covering three rotational planes: pitch, roll, and yaw.

[0057] Figure 31 The accuracy evaluation results of the UAV IMU across all three axes are shown.

[0058] Figure 32 The results of the long-term stability assessment of the UAV IMU are shown.

[0059] Figure 33 The three axes of rotation of the user's hand are shown.

[0060] Figure 34 The accuracy evaluation results of DCTF's IMU across all three axes are shown.

[0061] Figure 35 The results of the long-term stability assessment of the DCTF's IMU are shown.

[0062] Figure 36A The vibration amplitude of the haptic actuator at different frequencies is shown; and Figure 36B The vibration amplitude of the haptic actuator at different duty cycles is shown.

[0063] Figure 37 This demonstrates the response of the obstacle detection system when an obstacle is detected.

[0064] Figure 38 The accuracy test results of the laser detector are shown.

[0065] Figure 39 The system demonstrated its ability to detect obstacles up to 2.5 meters away.

[0066] Figure 40A This demonstrates how haptic feedback systems can enhance the control stability of drones; and Figure 40B This demonstrates the ability of the obstacle detection unit to trigger muscle contraction via the NMESF module.

[0067] Figure 41 The connectivity protocol used in the epidermal multimodal human-unmanned aerial vehicle (UAV) interfacing system is shown. Detailed Implementation

[0068] In the following description, details of the invention are set forth as preferred embodiments. It will be apparent to those skilled in the art that modifications, including additions and / or substitutions, can be made without departing from the scope and spirit of the invention. Specific details may be omitted to avoid obscuring the invention; however, this disclosure is written to enable those skilled in the art to practice the teachings herein without undue experimentation.

[0069] Figure 1 A block diagram of an epidermal multimodal human-drone interfacing system 10 according to an embodiment of the present invention is shown. As shown, the system includes: a base station 100 configured to receive hand orientation data of a user and obstacle data within a warning distance of a drone, and based on the correlation between the hand orientation data and the obstacle data, generate corresponding control commands for providing tactile feedback to the user's fingers, stimulating multiple muscle groups of the user's arm, and controlling the drone; a drone control tactile feedback (DCTF) module 200 configured to collect hand orientation data of the user's hand and deliver the tactile feedback to the user's fingers; and a neuromuscular electrical stimulation force feedback (NMESF) module 300 configured to collect obstacle data within a warning distance of the drone and deliver stimulating currents to multiple muscle groups of the user's arm.

[0070] Figure 2A A block diagram of a DCTF module 200 according to an embodiment of the present invention is shown. Figure 2B The three-dimensional structure of the DCTF module was shown; and Figure 2C and 2D It shows how to install the DCTF module on the back of the user's hand and fingers.

[0071] DCTF module 200 includes DCTF control unit 210 attached to the back of the user's hand and haptic actuator module 220 attached to the user's fingers.

[0072] Figure 3A The three-dimensional structure of the DCTF control unit is shown, and Figure 3B A preferred image of a DCTF control unit according to one embodiment is shown.

[0073] refer to Figure 2A and 3A The DCTF control unit 210 includes an IMU 211, a microcontroller (MCU) 212, a wireless communication module 213 (e.g., a Bluetooth Low Energy (BLE) chip), and a battery 214. The IMU 211 is configured to monitor changes in the user's hand angle to collect orientation data. This orientation data is acquired by the MCU 212 and wirelessly transmitted via the wireless communication module 213. Additionally, the MCU 212 converts tactile feedback commands (or actuator control commands) received from the communication module 213 into digital output signals (or stimulation current signals) and then transmits them to the tactile actuation module. In some embodiments, the DCTF control unit may be constructed from a flexible printed circuit board (FPCB) 215 and encapsulated between a top silicon encapsulation film 216 and a bottom silicon encapsulation film 217.

[0074] The haptic actuator module 220 includes a two-dimensional array (e.g., 3x3) of vibratory haptic actuators 221 connected via a connector array 222. The haptic actuator module 220 is further equipped with a plurality of switches 223 (e.g., metal-oxide-semiconductor field-effect transistors (MOSFETs)) coupled to the vibratory haptic actuators 221 via the connectors 222, enabling the generation of two-dimensional haptic feedback on the user's fingers, which exhibit the highest sensitivity to haptic stimulation compared to all other body parts.

[0075] Figure 4A A close-up representation of the three-dimensional structure of connector 222 is shown; Figure 4B Another close-up representation of the three-dimensional structure of the connector 222 assembled with the actuator 221 and the switch 223 is shown; Figure 4C A best-looking image shows two haptic actuators connected via a connector; and Figure 4DA best-looking image of a single connector is shown.

[0076] Connector 222 utilizes a thin (500 μm) and stretchable electronic design, characterized by a serpentine FPCB layer 225 encapsulated between a top silicon encapsulation film 226 and a bottom silicon encapsulation film 227, allowing the device shape to stretch up to 50%. This design adapts to the flexion of human fingers and positions the device close to the user's skin, thereby enhancing comfort and user experience. The serpentine FPCB layer 225 may include an upper copper layer 2251 and a bottom copper layer 2252, and a dielectric layer 2253 sandwiched between the upper copper layer 2251 and the bottom copper layer 2252. The serpentine FPCB layer 225 may further include one or more copper vias 2254 that penetrate the dielectric layer 2253 and are configured to be electrically connected to the upper copper layer 2251 and the bottom copper layer 2252.

[0077] Stretchable electronic designs employ a multi-layered structure, designed to reduce circuit size and simplify circuit wiring complexity. Figure 5 A cross-sectional view of a two-layer stretchable circuit board for a haptic actuator module is shown. As illustrated, the stretchable circuit board design comprises two layers of copper circuitry attached and separated by a polydimethylsiloxane (PDMS) substrate and a dielectric layer, respectively. Connections between the two copper layers are established via vias and copper leads. Furthermore, the upper copper layer is encapsulated by PDMS, exposing only the pads. This arrangement facilitates the soldering of the integrated circuit (IC) while ensuring the attachment of the copper layers during device deformation. Although the haptic actuator module contains only a single copper layer, it also incorporates a PDMS substrate and encapsulation to ensure its mechanical properties.

[0078] The fabrication process of the stretchable circuit board begins with the preparation of a quartz glass sheet (75mm x 75mm), which is thoroughly cleaned using acetone, alcohol, and deionized water (DI water) to serve as a support layer. Then, an aqueous solution of sodium stearate is spin-coated onto the glass and dried at 100°C for 5 minutes to form a thin sacrificial layer that facilitates subsequent material peeling. Subsequently, 3.5 ml of a 1:310 (by weight) mixture of 20:1 PDMS and white silicone is spin-coated onto the glass substrate at 500 rpm for 30 seconds. The coated substrate is then baked at 110°C for 6 minutes, and the spin-coating and baking process is repeated with the same parameters to form a 400 μm PDMS film for use as a stretchable substrate. Copper and polyimide (PI) films are then attached to the PDMS film to form a multilayer board. An additional baking step at 110°C for 6 minutes ensures strong adhesion between the copper and PI circuit layers and the PDMS substrate. The multilayer board is then laser-cut. Figure 6Initially, a laser cutter shapes the circuitry, then removes excess copper and PI layers. Subsequently, the multilayer board is spin-coated with a 3.5m 120:1 PDMS substrate at 700 rpm for 30 seconds and baked at 110°C for 12 minutes to form the package layer. The multilayer board is then laser-cut again to outline the device contours and form pads on the surface. For two-layer circuits, vias are also cut using a laser cutter. The two layers are aligned, and an ultra-thin PDMS layer is assembled between them, creating a strong adhesion bond between the layers. The vias are interconnected and soldered using 0.4mm diameter copper leads. MOSFETs are soldered to the pads using low-temperature solder paste.

[0079] Figure 7 This illustrates the configuration of a single haptic actuator used in this invention. The haptic actuator may include a flexible film, a magnet, a plastic ring, and a copper coil. The flexible film may have a cantilever structure to facilitate unrestricted movement of the magnet; and when a square wave signal with a different duty cycle (e.g., ...) is applied to the copper coil... Figure 8 When (as shown), the magnet is induced to move periodically upward and downward, thus generating vibration. The flexible film can be made of any suitable flexible material, such as, but not limited to, polyethylene terephthalate (PET) and polyimide.

[0080] Figure 9 The operating principle of haptic feedback is illustrated. As described in previous chapters, the drone's angle during flight is related to the surrounding aerodynamic conditions. This angular change is replicated on a 3x3 array of haptic actuators attached to the user's fingers. When the drone tilts in a specific direction, the haptic feedback point also shifts from the center of the array in the corresponding direction. This generates a dynamic, real-time haptic sensation similar to a 'rolling ball' on the user's hand, allowing the user to perceive the aerodynamic conditions surrounding the drone.

[0081] refer to Figure 10 To replicate the drone's angular changes to the user via haptic feedback, the angular changes of the drone on the roll and pitch axes are extracted and plotted on a virtual 2D graph, generating a point called the "center point." The drone's angular changes on the roll and pitch axes will correspondingly change the X and Y coordinates of the center point on the graph. Since the drone's angular changes should be continuous and simulated, monitoring multidimensional angular changes in the system can lead to a large number of potential combinations of haptic stimulus locations, posing a challenge if an exact actuator is assigned to each coordinate. For example, monitoring angular changes on both the roll and pitch axes within ±10 degrees with an accuracy of one degree may generate 22x22 possible coordinate combinations that need to be replicated by an array of haptic actuators. If each actuator represents only one coordinate on the graph, this places high demands on the density and resolution of the actuator array.

[0082] To regenerate informative spatial information on a lower-resolution actuator array, a spatial downsampling method is introduced. The distance (Δd) between each actuator and the center point is calculated and input into a decay function. The decay function can be selected from, but is not limited to, parabolic, linear, or natural exponential decay functions. Then, the duty cycle (or duty ratio) of the drive signal to each actuator is determined based on Δd. For example, for a parabolic decay function, the duty cycle can be obtained using the following formula: Where p is the distance between the focus and the vertex in the parabolic decay function. If the result (i.e., the duty cycle) is positive, the corresponding actuator will be activated with the calculated duty cycle; while a complex result will leave the actuator idle.

[0083] Using a spatial downsampling method, a circular region is defined around the center point in the virtual feedback map. When the actuator enters this region, it is activated, and the stimulus intensity gradually increases until it reaches its maximum intensity at the center point. This region is called the "effective region." As the drone experiences changes in angle, the center point and the effective region will move accordingly. Figure 11 This changes the duty cycle of each actuator. Therefore, continuous angular changes are downsampled and reproduced through variations in the stimulus intensity of each actuator.

[0084] Several parameters in downsampling methods can affect the haptic feedback experience and information transmission efficiency. For example, starting from the original parabolic effective region ( Figure 12A and 12B It can adjust the positioning of the area based on changes in the drone's angle. Figure 13A and 13B Furthermore, the size of the effective region can be increased or decreased by adjusting the parameters in the attenuation function. Figure 14A and 14B Additionally, the decay function can be modified to generate different types of dynamic haptic feedback, such as transforming a parabolic function into a linear decay function. Figure 15A and 15B ).

[0085] In one example implementation, due to its excellent performance in user studies, a natural exponential decay function in the decay function and an effective region generated by a focus of 1.5 were adopted, and user studies confirmed that the 3x3 haptic feedback array combined with the spatial downsampling method can effectively transmit two-dimensional spatial information to users for monitoring the flight attitude of the UAV.

[0086] Figure 16A A block diagram of an NMESF control unit 310 according to an embodiment of the present invention is shown.

[0087] The NMESF control unit 310 is configured to receive stimulation commands from a base station via a wireless communication module 311. The NMESF control unit may include an MCU 312, an analog switch 313, and a current driver 314, which cooperate to deliver a specific amount of current to the target muscle. The NMESF control unit 310 may further include a two-stage boost circuit 315 configured to provide sufficient voltage to overcome the high impedance of the human body and deliver the desired current. The current is then applied to the muscle through one or more stimulation electrode arrays 316. Figure 16B This demonstrates how to mount the NMESF control unit on a user's forearm. The wireless communication module 311, MCU 312, analog switch 313, current driver 314, boost circuit 315, and battery 317 can be assembled in a main circuit board 350 that is detachably mounted on the user's forearm. Each stimulation electrode array can be packaged as a stimulation patch 360 that is detachably mounted on the user's forearm.

[0088] refer to Figure 17A and 17B Similar to the DCTF control unit, the main circuit board 350 can also be constructed from a flexible printed circuit board (FPCB) 3501 and encapsulated between a top silicon encapsulation film 3502 and a bottom silicon encapsulation film 3503. (Reference) Figure 18A and 18B The stimulating patch 360 also employs a stretchable silicon substrate 3601 and a serpentine copper layer 3602, allowing it to be bent, twisted, or stretched in various dimensions. Figure 18A and 18B In addition, each stimulation electrode is attached with a stretchable and replaceable conductive hydrogel layer 3603 to reduce the impedance between the electrode and the skin.

[0089] Figure 19A A block diagram of an obstacle detection unit 320 according to an embodiment of the present invention is shown; and Figure 19B This demonstrates how to position the obstacle detection unit 320 on the top of the drone.

[0090] The obstacle detection unit may include three laser detectors (or sensors) 321a-321c, configured to detect obstacles in the left, right, and rear regions of the drone that are not covered by the drone's camera. Obstacle information is processed into obstacle data by the MCU 322, and the obstacle data is wirelessly transmitted to a base station via a communication module 323 (e.g., a radio frequency (RF) module).

[0091] To reduce collision risk, when the obstacle detection unit detects an obstacle within the system's warning distance, it transmits a stimulation command to the NMESF control unit. Utilizing the NMES principle, force feedback stimulation is delivered to the user's muscles (flexor carpi ulnaris and flexor carpi radialis) and induces muscle contraction. As the drone approaches the obstacle, the stimulation intensity gradually increases, resulting in a stronger muscle contraction and wrist flexion in contrast to an upward wrist movement. This involuntary wrist flexion causes the drone to move forward, thus avoiding an impending collision. When the drone moves beyond the warning distance, the NMESF control unit ceases operation, and the stimulation stops. Therefore, force feedback not only enables the user to perceive invisible obstacles but also assists in route correction to avoid potential collisions. Importantly, unlike traditional force feedback devices that rely on exoskeletons or bulky devices, the force generated in the NMESF control unit originates from the user's muscles. Therefore, only thin electrodes and peripheral circuitry are required, enabling the miniaturization of the entire device to a size that allows it to be worn under everyday clothing.

[0092] Three muscle groups (the flexor carpi ulnaris and flexor carpi radialis for flexion, the pronator teres for pronation, and the supinator for supination) will be stimulated via five electrodes, such as... Figure 20 The described action induces contractions in which the wrist rotates in three directions: pronation, flexion, and supination. Figure 21 These rotations occur along the roll and pitch axes of the user's hand, which are used for intuitive drone control and can counteract left, right, and upward rotations of the user's wrist. The NMESF control unit generates force feedback stimuli in the form of current pulses, and the NMESF control unit can deliver different current levels via current control circuitry by adjusting the digital-to-analog converter (DAC) output of the MCU. Figure 22A ), duty cycle ( Figure 22B ) and frequency ( Figure 22D The pulse of ) is used. Since there is a linear relationship between the DAC output and the stimulation current output, a simple control mechanism is established. Figure 22C ).

[0093] To evaluate the NMES's ability to generate force in the user's muscles, a torque test was conducted, and the results are shown in... Figures 23A to 23C The results clearly show that, although different users may exhibit different stimulation thresholds, the average torque generated during supination, pronation, and flexion all demonstrates a proportional or even linear relationship with the stimulation current. This indicates that by adjusting the stimulation intensity, the torque and force exerted on the wrist by the user's own muscles can be precisely manipulated. Notably, even the weakest wrist pronation can generate approximately 0.4 Nm of torque when a high current is applied to the user's muscles, and this significant torque can induce involuntary flexion of the wrist in the corresponding direction, thus aiding in path correction during flight.

[0094] Figure 24 The equivalent circuit of the NMES stimulation path is shown, where the load refers to the impedance of the user's muscle and the impedance of the electrode. MOSFET resistance (R) ds(on) The current flowing through the electrodes and body is controlled by the operational amplifier (OP-AMP) and the DAC output of the MCU to ensure that the current is matched to the target current. According to the equivalent circuit, it is obvious that if the load impedance is high, based on Ohm's law, a boost circuit will be needed to generate a higher input voltage (V). in Therefore, the electrical characteristics of the electrodes and the user's body become a key factor in reducing the requirements for the boost circuit. One possible solution is to increase the stimulation frequency.

[0095] Figure 25 The average frequency response of the load, including the two electrodes and the user's muscle, under electrical stimulation is shown. A significant decrease in average impedance was observed from 2450 Ω to 1360 Ω as the stimulation frequency increased from 40 Hz to 70 Hz. This can be attributed to the same decreasing trend in the frequency response of the electrodes as the frequency increases, and the fact that human body impedance also exhibits a decreasing response to increasing frequency. However, the tests indicate that the torque generated by the user also decreases with increasing frequency. Figure 26 ).

[0096] To achieve a balance between the stimulation path impedance and the generated torque, the inventors chose 60 Hz as the final stimulation frequency. Although the system demonstrated the ability to generate current pulses with various duty cycles, users reported discomfort as the duty cycle increased. Therefore, to mitigate any user discomfort, the duty cycle was maintained at 1% throughout the stimulation.

[0097] Although the impedance of the stimulation path was optimized by adjusting the stimulation frequency, a high voltage was still required at 60 Hz to overcome the average load impedance of 1.6 kΩ. Furthermore, the relationship between the excitation voltage and impedance of the electrodes (…) Figure 27 This indicates that higher stimulation voltages can further reduce electrode impedance. This finding led to the design of a two-stage boost circuit in the NMESF module. The boost circuit was designed to increase the battery's 7.4V input to a 100V output for electrical stimulation.

[0098] Figure 28This invention illustrates a conceptual description of its operational mechanism as an interface between the user and the drone. The invention integrates four main functions: 1) receiving FPV visual feedback from the drone; 2) intuitive drone control via the user's hand gestures; 3) simulating tactile feedback from the human vestibular system to provide real-time flight angle or posture information, allowing dynamic monitoring of flight and aerodynamic conditions around the drone; and 4) delivering electrical stimulation to the user's muscles, thereby triggering muscle contraction and generating tension and force feedback upon detection of an obstacle in the corresponding direction, thus counteracting or reversing hand movements to reduce the risk of potential collisions and accidents.

[0099] The operational logic flow of System 10 as the interface between User 50 and Drone 40 is shown in Figure 29A and 29B The operation begins with IMU 211 in DCTF module 200 capturing hand orientation data. The hand orientation data D01 acquired by the DCTF module is wirelessly transmitted to base station 100 (e.g., a computer) of system 10. Base station 100 then converts the hand orientation (or angle) data D01 into a drone control command D02 and transmits the drone control command D02 to drone 40 via wireless local area network (WLAN), thereby guiding the drone to move according to the hand gesture. IMU 401 in drone 40 detects the drone's flight angle and speed D03 and transmits them back to base station 100 for estimating flight attitude and aerodynamic conditions. The estimated flight attitude and aerodynamic conditions are used to generate a two-dimensional haptic feedback map, and actuator control command D04 is transmitted back to DCTF module 200 to induce haptic feedback on the user's finger via actuator 221. The haptic feedback, together with the intuitive control facilitated by the hand gesture, constitutes a control-haptic feedback loop.

[0100] In addition to haptic feedback, a laser sensor 321 in the obstacle detection unit 320 mounted on the drone detects the presence of obstacles and their distance from the drone, converting the detected obstacle presence and distance into obstacle data D05. The obstacle data D05 is transmitted to the base station 100 via an RF module, and is then converted by the base station 100 into muscle stimulation commands D06. These stimulation commands D06 are transmitted via BLE to the NMESF control unit 310, which generates electrical stimulation through stimulation electrodes 315, thereby providing force feedback that influences the user's hand movements. Since the drone 40 is controlled by hand movements, the NMES force feedback and intuitive control form a force-control feedback loop.

[0101] Furthermore, the integrated camera 402 on the drone 40 transmits a real-time video signal D07 back to the base station 100 via WLAN. The base station then wirelessly transmits the signal D07 to the virtual reality (VR) headset 60, thus forming a visual feedback loop. Combining intuitive control, haptic feedback, and force feedback, these loops constitute a multimodal closed-loop control system.

[0102] Figure 41 The communication protocol is illustrated. IMU data from the DCTF module is transmitted to the base station via Bluetooth mesh, with 13 bytes of information including roll, pitch, and yaw angles. The base station converts the IMU data into 3D control commands and delivers them to the UAV using WLAN. The UAV's roll, pitch, and yaw angle data is sent back to the base station, converted into actuator commands based on an attenuation function, packaged into 16 bytes of data containing the duty cycle of each actuator, and sent back to the DCTF module via Bluetooth mesh. The obstacle detection unit sends a 5-byte message to the base station using radio frequency, containing the distance between obstacles and the sensor identification number. The base station converts the message into electrical stimulation commands and delivers the commands to the NMESF module via Bluetooth mesh in a 12-byte message containing the current, frequency, and target channel.

[0103] The DCTF and NMESF control units are based on an FPCB and both utilize microcontrollers, Bluetooth modules, low-dropout regulators, resistors, and capacitors. The DCTF control unit integrates an inertial measurement unit. The NMESF control unit includes analog switches, operational amplifiers, MOSFETs, and a boost circuit controller. All components are soldered to the FPCB using low-temperature solder paste. The obstacle detection unit mounted on the drone is also PCB-based and includes a microcontroller, a low-dropout regulator, three laser detectors, an RF communication module, resistors, and capacitors. All components are also soldered using low-temperature solder paste.

[0104] The performance of the UAV's IMU is evaluated, covering three rotational planes: pitch, roll, and yaw, and depicted in... Figure 30 During flight, real-time rotational information of the UAV across these three planes is recorded. The accuracy of the UAV's integrated IMU in the three-dimensional rotational planes of pitch, roll, and yaw is assessed as follows: Figure 31 As shown. During the test, the drone rotated from 0 degrees to 90 degrees on the robotic arm in 5-degree increments. The angle readings of the drone in the pitch, roll, and yaw dimensions remained consistent with the test settings, indicating that the drone's IMU has high accuracy and is capable of capturing minute changes in the drone's angles during flight.

[0105] In addition, a long-term test was conducted, in which the robotic arm attached to the drone was continuously rotated 45 degrees for 300 seconds. The test duration was limited to 300 seconds because the drone system would automatically shut down after this period if no control command was issued. During the test period, results were obtained across all three dimensions ( Figure 32 The system exhibited high repeatability, with a sustained 45-degree variation observed. Only the yaw direction showed a slight deviation of 2 to 3 degrees, attributed to the magnetic compass utilized in the 9-axis IMU, which is inherently more susceptible to deviations or disturbances from surrounding magnetic anomalies compared to the pitch and roll axes. The high repeatability and stability, coupled with high accuracy, confirm that the integrated IMU of the UAV can provide reliable angular data during flight.

[0106] After evaluating the performance of the UAV's IMU, the relationship between the UAV's angle during flight and the surrounding wind speed and direction was examined. Similar to other aircraft, aerodynamic conditions such as wind speed and direction can significantly affect UAV flight. Operators encountering strong winds may need to immediately adjust to the correct flight attitude and direction of travel, especially considering that airflow is imperceptible in visual feedback. While some sensors can directly monitor these aerodynamic factors, incorporating additional sensors, particularly when the UAV is exposed to multi-directional winds or turbulence, would increase size and weight, thus placing a significant burden on the UAV. Analysis of the correlation between unexpected changes in UAV flight angle and sudden changes in external aerodynamic conditions revealed that as wind speed increases, the changes in the UAV's angle on the pitch and roll axes intensify, with the most significant angle changes occurring at a wind speed of 8.76 m / s. This direct proportionality highlights that changes in the UAV's angle during flight can reflect drastic changes in aerodynamic conditions. To address this challenge, this invention transmits the UAV's angle data back to the base station for aerodynamic condition estimation.

[0107] Given that the system relies on the user's hand orientation as the intuitive control method for the drone, the performance of the IMU within the DCTF module is crucial to ensuring control quality. The three-axis rotation of the user's hand is depicted in... Figure 33 In this context, pitch, roll, and yaw correspond to the three axes of a drone. Rotation of the user's hand will induce a similar movement in the drone. For example, after recording the initial orientation of the user's hand during calibration, if the user rotates their hand downwards along the pitch axis, the drone will also rotate on the pitch axis, causing it to fly forward. If the DCTF module detects rotation of the user's hand on multiple axes (e.g., both pitch and yaw), the drone will respond by flying in multiple headings (e.g., forward and to the left). Furthermore, a larger rotation detected by the DCTF module will prompt the drone to fly at a higher speed in the corresponding direction. Therefore, the DCTF module must be able to accurately and reliably capture changes in the angle of the user's hand. Figure 34The results of the accuracy test of the DCTF module are shown. The test was similar to that conducted on a drone. The results show that significant increments of 5 degrees and a total rotation of 90 degrees were observed on all three rotation axes.

[0108] 25-minute long-term test results of the DCTF module ( Figure 35 The DCTF system exhibits a continuous 45-degree change, highlighting its high repeatability in angle monitoring. Similar to the IMU system of drones, the DCTF module also demonstrates high accuracy and repeatability, making it suitable for precise drone control.

[0109] The resonant frequency of the actuator was investigated. Figure 36A The vibration amplitude was compared across a frequency range of 45Hz to 185Hz at 35Hz intervals. The results showed that the actuator reached its maximum vibration amplitude at 115Hz, which was adopted in this invention and considered the actuator's resonant frequency. Furthermore, the effect of duty cycle on vibration amplitude was investigated. Applying duty cycles ranging from 20% to 50% to the actuator revealed a strong direct proportionality between duty cycle and vibration amplitude. Figure 36B This indicates that a higher duty cycle leads to stronger vibration intensity. However, the maximum duty cycle used in this invention is limited to 50%. This is because, based on the principles of actuators, a longer duty cycle (such as 70%) produces similar performance to a lower duty cycle (such as 30%), but also has a longer 'on' time and increased power consumption.

[0110] In addition, the performance of the obstacle detection unit was evaluated. Figure 37 The unit was demonstrated to detect obstacles with a short response time of 87ms, which is feasible for scenarios involving drones or obstacles moving at high speeds and requiring rapid response. Figure 38 The accuracy of the laser detector in the unit in detecting obstacles at different distances was demonstrated, while Figure 39 The system demonstrated its ability to detect obstacles up to 2.5 meters away. Furthermore, the medium detection range allows the system to effectively detect obstacles without being affected by obstacles from other directions.

[0111] exist Figure 40A and 40B The paper demonstrates the performance of the multimodal closed-loop human-unmanned aerial vehicle (UAV) interface system in a real-world flight scenario. Figure 40AThis demonstrates how haptic feedback systems can enhance the control stability of drones. It shows the change in drone angle with and without haptic feedback under windy conditions. The X and Y axes represent the drone's roll and pitch rotation, respectively, while color depth indicates the density of data points. In calm conditions, the drone remains stable with an angle change of 1 to 2 degrees. However, when subjected to a wind speed of 5.7 m / s, the angle change increases rapidly. After activating the haptic feedback system, the user can perceive abnormal changes in the drone's attitude and counteract the wind, resulting in reduced angle change and more stable flight. This indicates that haptic feedback systems can enhance the flight stability of drones under complex aerodynamic conditions.

[0112] Figure 40B The ability of the obstacle detection unit to trigger muscle contraction via the NMESF module was verified. The system was configured to activate stimulation when an obstacle was detected within 1 meter. The stimulation intensity gradually increased as the obstacle approached, reaching its maximum at 20 cm. Data showed a consistent correlation between obstacle distance, stimulation current, and muscle-generated torque, confirming the NMESF control unit's ability to induce muscle contraction upon obstacle detection.

[0113] The functional units and modules of the epidermal multimodal human-to-unmanned aerial vehicle (UAV) interfacing system according to the embodiments disclosed herein can be implemented using computing devices, computer processors, or electronic circuit systems, including but not limited to application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, and other programmable logic devices configured or programmed according to the teachings of this disclosure. Based on the teachings of this disclosure, those skilled in the art of software or electronics can readily prepare computer instructions or software code to run on computing devices, computer processors, or programmable logic devices.

[0114] All or part of the methods according to the embodiments can be performed in one or more computing devices, including server computers, personal computers, laptop computers, mobile computing devices such as smartphones and tablet computers.

[0115] Embodiments may include computer storage media, transient and non-transient memory devices storing computer instructions or software code, which can be used to program or configure computing devices, computer processors, or electronic circuit systems to perform any of the processes of the present invention. Storage media, transient and non-transient memory devices may include, but are not limited to, floppy disks, optical disks, Blu-ray discs, DVDs, CD-ROMs and magneto-optical disks, ROMs, RAMs, flash memory devices, or any type of medium or device suitable for storing instructions, code, and / or data.

[0116] Each of the functional units and modules according to the various embodiments can also be implemented in a distributed computing environment and / or cloud computing environment, wherein all or part of the machine instructions are executed in a distributed manner by one or more processing devices interconnected by a network such as an intranet, a wide area network (WAN), a local area network (LAN), the Internet, and other forms of data transmission media.

[0117] While this disclosure has been described and illustrated with reference to specific embodiments thereof, such descriptions and illustrations are not limiting. Illustrations may not necessarily be drawn to scale. There may be differences between artistic representations in this disclosure and actual equipment due to manufacturing processes and tolerances. Other embodiments may exist in this disclosure that are not specifically shown. Modifications may be made to suit particular circumstances, materials, compositions of matter, methods, or processes to the objectives and scope of this disclosure. All such modifications are intended to fall within the scope of the appended claims. Although the methods disclosed herein have been described with reference to specific operations performed in a particular order, it will be understood that these operations may be combined, subdivided, or reordered to form equivalent methods without departing from the teachings of this disclosure. Therefore, the order and grouping of operations are not limiting unless expressly indicated herein.

Claims

1. An epidermal multi-modal human-drone interfacing system comprising: a base station configured to: receive hand orientation data of a user and obstacle data within a vigilance distance of a drone; and generate respective control commands for providing haptic feedback to the user, stimulating a plurality of muscle groups of the user, and controlling the drone based on a correlation of the hand orientation data with the obstacle data; a drone control haptic feedback (DCTF) module in communication with the user and the base station and configured to collect the hand orientation data of the user and deliver the haptic feedback to the user; and a neuromuscular electrical stimulation force feedback (NMESF) module in communication with the base station, the user, and the drone and configured to collect the obstacle data within the vigilance distance of the drone and deliver stimulation current to stimulate the plurality of muscle groups of the user.

2. The epidermal multi-modal human-drone interfacing system of claim 1, wherein the base station is further configured to: receive a real-time video signal from a camera integrated in the drone; and communicate the real-time video signal to a virtual reality headset worn by the user to facilitate the user setting a hand orientation for drone control.

3. The epidermal multi-modal human-drone interfacing system of claim 1, wherein: the DCTF module includes a DCTF control unit and a haptic actuation module; the DCTF control unit is attached to the back of the user's hand and configured to collect the hand orientation data of the user, transmit the hand orientation data to the base station, receive actuator control commands from the base station, and communicate the actuator control commands to the haptic actuation module; and 4. The skin multi-modal human-drone interfacing system of claim 3, wherein the DCTF control unit comprises: the haptic actuation module includes a two-dimensional array of haptic actuators connected by an array of connectors, the haptic actuator array is attached to the user's fingers and configured to receive the actuator control commands from the DCTF control unit and deliver the haptic feedback to the user's fingers. an inertial measurement unit configured to collect the hand orientation data of the user; a first communication module configured to transmit the collected hand orientation data to the base station and receive the actuator control commands from the base station; and 5. The epidermal multi-modal human-drone interfacing system of claim 3, wherein each of the haptic actuators comprises: a first microcontroller configured to generate actuator drive signals to drive the haptic actuator array to deliver the haptic feedback to the user's fingers. a magnet; and a polyethylene terephthalate (PET) film having a cantilever structure to facilitate unrestricted movement of the magnet.

6. The epidermal multi-modal human-drone interfacing system of claim 5, wherein the haptic actuator array and the array of connectors are fabricated on a flexible and stretchable circuit board.

7. The skin multi-modal human-drone interfacing system of claim 6, wherein the flexible and stretchable circuit board comprises a stretchable substrate sandwiched between a pair of serpentine patterned conductive layers.

8. The skin multi-modal human-drone interfacing system of claim 1, wherein: the NMESF module comprises an obstacle detection unit and a NMESF control unit; the obstacle detection unit is located on top of the drone and is configured to collect obstacle data within a vigilance distance of the drone and transmit the obstacle data to the base station; and the NMESF control unit is attached to the user’s arm and is configured to receive stimulation commands from the base station and generate stimulation current to stimulate a plurality of muscle groups of the user’s arm.

9. The skin multi-modal human-drone interfacing system of claim 8, wherein the obstacle detection unit comprises: at least three laser detectors configured to detect obstacle information in left, right and rear regions of the drone, respectively; a second microcontroller configured to process the detected obstacle information into obstacle data; and a second communication module configured to transmit the obstacle data to the base station for generating the stimulation commands.

10. The skin multi-modal human-drone interfacing system of claim 8, wherein the NMESF control unit comprises: a third communication module configured to receive the stimulation commands from the base station; a third microcontroller configured to generate control signals based on the stimulation commands; a current driver configured to provide the stimulation current; a switch configured to receive the control signals to turn on / off the current driver; and a plurality of stimulation electrodes attached to the user’s arm and electrically coupled to the current driver to deliver the stimulation current to a plurality of muscle groups of the user’s arm.

11. The skin multi-modal human-drone interfacing system of claim 10, wherein the plurality of stimulation electrodes are fabricated on a flexible and stretchable circuit board.

12. The skin multi-modal human-drone interfacing system of claim 11, wherein the flexible and stretchable circuit board comprises a stretchable substrate, a serpentine patterned conductive layer and a stretchable and replaceable conductive hydrogel.

13. A method of facilitating a user to control a drone in an intuitive manner using the skin multi-modal human-drone interfacing system of claim 1, the method comprising constructing a control-haptic feedback loop by: capturing hand orientation data of the user by the DCTF module; transmitting the captured hand orientation data to the base station by the DCTF module; converting the hand orientation data into drone control commands by the base station; transmitting the drone control commands to the drone by the base station; detecting, by the drone, a flight angle and velocity of the drone, and transmitting the detected flight angle and velocity to the base station; estimating, by the base station, a flight attitude and aerodynamic conditions of the drone; generating, by the base station, a two-dimensional haptic feedback map; generating, by the base station, actuator control commands based on the two-dimensional haptic feedback map; and transmitting the actuator control commands to the DCTF module to induce haptic feedback for the user.

14. The method of claim 13, further comprising constructing a force-control feedback loop by: collecting, by the NMESF module, obstacle data within a vigilance distance of the drone; transmitting, by the NMESF module, the obstacle data to the base station; translating, by the base station, the obstacle data into muscle stimulation commands, and transmitting the muscle stimulation commands to a NMESF control unit mounted on a user's forearm; generating, by the NMESF control unit, force feedback stimulation to affect the user's hand movement, which in turn controls the drone through the DCTF module.

15. The method of claim 13, further comprising constructing a visual feedback loop by: receiving real-time video signals from a camera integrated in the drone; and transmitting the real-time video signals to a virtual reality headset of the user to facilitate the user setting a hand orientation for drone control.

Images