Unmanned aerial vehicle remote control system based on virtual reality technology

Abstract

The invention discloses an unmanned aerial vehicle remote control system based on a virtual reality technology, relates to the technical field of unmanned aerial vehicle control, and aims to solve the problem that a control instruction is difficult to accurately and quickly send to an unmanned aerial vehicle in complex terrains such as mountainous areas and canyons in the prior art. The system comprises an unmanned aerial vehicle end and a ground operation end. The unmanned aerial vehicle end comprises a signal transmission module, a virtual reality image acquisition module and a flight control module; the ground control end comprises a virtual reality display device, a control instruction generation module and a signal processing and transmission module; the virtual reality image acquisition module is used for acquiring environment images and data around the unmanned aerial vehicle and transmitting the environment images and data to the signal transmission module, and the signal transmission module is used for transmitting the received environment images and data to the signal processing and transmission module through multiple frequency bands. The method has the advantage of remotely, accurately and quickly controlling the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of drone control technology, and more specifically, to a drone remote control system based on virtual reality technology. Background Technology

[0002] With the rapid development of drone technology based on virtual reality, its applications in fields such as resource exploration, emergency rescue, power line inspection, and geological surveying are becoming increasingly widespread. Existing drones mostly use single-band or fixed communication modes for data transmission and command interaction. However, in complex terrains such as mountains and canyons, communication between the drone and the ground station is highly susceptible to environmental interference. For example, electromagnetic interference can disrupt normal signal transmission, and obstacles can cause signal attenuation or reflection. In such situations, single-band transmission often faces the risks of signal instability, increased transmission delay, or even interruption. These problems severely affect the real-time performance and reliability of drone control, making it difficult to accurately and quickly send control commands to the drone. Therefore, we propose a drone remote control system based on virtual reality technology. Summary of the Invention

[0003] The purpose of this invention is to provide a remote control system for unmanned aerial vehicles (UAVs) based on virtual reality technology, which aims to solve the problem that existing technologies are unable to accurately and quickly send control commands to UAVs in complex terrains such as mountains and canyons.

[0004] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a remote control system for unmanned aerial vehicles (UAVs) based on virtual reality technology, the system comprising a UAV terminal and a ground operation terminal; The UAV terminal includes a signal transmission module, a virtual reality image acquisition module, and a flight control module; The ground control terminal includes a virtual reality display device, a control command generation module, and a signal processing and transmission module; The virtual reality image acquisition module is used to acquire environmental images and data around the drone and transmit the environmental images and data to the signal transmission module. The signal transmission module is used to transmit the received environmental images and data to the signal processing and transmission module through multiple frequency bands. The signal processing and transmission module is used to transmit the received environmental images and data to the virtual reality display device. The virtual reality display device is used to construct a virtual control scene based on the received environmental images and data. The control command generation module is used to generate control commands based on user instructions and transmit them to the signal processing and transmission module. The signal processing and transmission module is used to transmit the received control commands to the signal transmission module via multiple frequency bands. The signal transmission module is used to transmit the received control commands to the flight control module. The flight control module is used to control the flight of the UAV based on the control commands.

[0005] Preferably, when the signal transmission module and the signal processing and receiving module transmit environmental images and data, the optimal frequency band is determined and selected using a pre-judgment method among the multiple frequency bands used. Furthermore, when the signal processing and receiving module transmits control commands to the signal transmission module, it uses the same frequency band as when transmitting environmental images and data.

[0006] Preferably, the pre-determination method used by the signal transmission module includes the following steps; Step 1: Calculate the predicted comprehensive score for each frequency band within a preset time period using the pre-judgment algorithm formula. The formula is: ; In the formula, Indicates frequency band In the future The overall score for time prediction. Indicates frequency band At the present moment The overall score is determined by the formula. Calculations show that This is the trend correction coefficient. Number of historical data sampling points The sampling time interval, and They represent frequency bands respectively. Overall ratings at different points in the past. Indicates frequency band The signal strength value, Indicates the signal strength stability coefficient. Indicates frequency band The degree of interference received Indicates the disturbance stability coefficient. Indicates frequency band The actual transmission rate Represents the transmission rate stability coefficient. This represents the historical performance coefficient of the frequency band. , , , These are the weighting coefficients for signal strength and stability, anti-interference capability and stability, transmission rate and stability, and historical performance, respectively. Step 2: Determine the optimal frequency band based on the comprehensive prediction score of each frequency band. The optimal frequency band is the one with the highest comprehensive prediction score. Step 3: Select the optimal frequency band for signal transmission and preset a scoring threshold. When it is predicted that the predicted comprehensive score of the currently used frequency band will be lower than the preset scoring threshold or surpassed by the predicted comprehensive score of other frequency bands, start the handover preparation process in advance.

[0007] Preferably, the handover preparation process includes frequency band pre-connection and data caching operations to reduce signal interruption time during the handover process.

[0008] Preferably, it also includes a satellite communication module, which is activated when the predicted comprehensive score of all frequency bands selectable by the signal transmission module and the signal processing and receiving module is lower than a preset score threshold, and realizes signal transmission between the signal transmission module and the signal processing and receiving module through a satellite link to ensure signal continuity.

[0009] Preferably, the virtual reality display device includes either head-mounted virtual reality glasses or a virtual reality helmet.

[0010] Preferably, the virtual reality image acquisition module includes a high-definition binocular camera group, a panoramic camera, an infrared sensor, and an inertial measurement sensor.

[0011] Preferably, the control commands given by the flight control module to control the flight of the UAV include controlling the UAV's flight attitude, speed, altitude, and flight path.

[0012] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention uses a multi-band signal transmission and pre-determination of the optimal frequency band to dynamically select the optimal frequency band for transmitting control commands in complex terrains such as mountainous areas and canyons that are prone to signal interference. Furthermore, the control commands and environmental images and data use the same frequency band to avoid synchronization delays. Combined with the start-up and switching preparation process to reduce signal interruption time, this invention solves the problems of easy interruption and high latency in command transmission under complex terrains in the prior art, ensuring that control commands are accurately and quickly sent to the UAV.

[0013] 2. In this invention, by setting a preset frequency band scoring threshold, when all frequency band scores are below the threshold, the satellite communication module automatically starts as an emergency link, which solves the problem that existing technologies cannot control drones in remote areas or in scenarios where signals are completely lost, and ensures that the system can still operate stably in extreme environments.

[0014] 3. This invention constructs an immersive virtual control scene through a virtual reality display device, allowing operators to perform intuitive operations through near first-person perspective environmental perception, reducing reliance on experience in remote drone control. At the same time, the flight control module supports direct adjustment of core parameters such as attitude and speed, simplifying complex operation procedures. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system architecture of the present invention. Detailed Implementation

[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0017] Example 1 like Figure 1 As shown, a remote control system for unmanned aerial vehicles (UAVs) based on virtual reality technology includes a UAV terminal and a ground control terminal. The drone terminal includes a signal transmission module, a virtual reality image acquisition module, and a flight control module; The ground control terminal includes a virtual reality display device, a control command generation module, and a signal processing and transmission module; The virtual reality image acquisition module is used to acquire environmental images and data around the drone and transmit the environmental images and data to the signal transmission module. The signal transmission module is used to transmit the received environmental images and data to the signal processing and transmission module through multiple frequency bands. The signal processing and transmission module is used to transmit the received environmental images and data to the virtual reality display device. The virtual reality display device is used to construct a virtual control scene based on the received environmental images and data, so as to realize the transmission of environmental images and data around the drone acquired by the virtual reality image acquisition module through multiple frequency bands, ensuring that ground personnel can obtain an immersive view of the environment around the drone in real time, reducing spatial perception deviation in remote operation, and providing environmental basis for precise control. The control command generation module generates control commands based on user instructions and transmits them to the signal processing and transmission module. The signal processing and transmission module transmits the received control commands to the signal transmission module via multiple frequency bands. The signal transmission module then transmits the received control commands to the flight control module. The flight control module controls the drone's flight based on the control commands, thereby enabling the transmission of control commands to the drone via multiple frequency bands, improving transmission stability, enhancing the flexibility of remote control, and meeting the control needs in complex scenarios.

[0018] Furthermore, when the signal transmission module and the signal processing and receiving module transmit environmental images and data, a pre-judgment method is used to determine and select the optimal frequency band among the multiple frequency bands used. When the signal processing and receiving module transmits control commands to the signal transmission module, the same frequency band is used as when transmitting environmental images and data. This achieves improved anti-interference capability through multi-frequency band transmission. The pre-judgment method ensures the selection of the frequency band with the best performance. The transmission of commands and data on the same frequency band avoids synchronization delays caused by frequency band switching, ensuring consistency between control and environmental perception.

[0019] Furthermore, the pre-determination method used by the signal transmission module includes the following steps; Step 1: Calculate the predicted comprehensive score for each frequency band within a preset time period using the pre-judgment algorithm formula. The formula is: ; In the formula, Indicates frequency band In the future The overall score for time prediction. Indicates frequency band At the present moment The overall score is determined by the formula. Calculations show that This is the trend correction coefficient. Number of historical data sampling points The sampling time interval, and They represent frequency bands respectively. Overall ratings at different points in the past. Indicates frequency band The signal strength value, Indicates the signal strength stability coefficient. Indicates frequency band The degree of interference received Indicates the disturbance stability coefficient. Indicates frequency band The actual transmission rate Represents the transmission rate stability coefficient. This represents the historical performance coefficient of the frequency band. , , , These are weighting coefficients for signal strength and stability, anti-interference capability and stability, transmission rate and stability, and historical performance, respectively. This formula integrates multi-dimensional indicators and trend changes to scientifically predict frequency band performance, avoid misjudgment based on a single indicator, and provide a quantitative basis for optimal frequency band selection. Step 2: Determine the optimal frequency band based on the predicted comprehensive score of each frequency band. The optimal frequency band is the one with the highest predicted comprehensive score. This ensures that the selected frequency band has the best performance in the future, reduces frequent switching, and improves transmission stability by using the highest score as the standard. Step 3: Select the optimal frequency band for signal transmission and preset a scoring threshold. When it is predicted that the predicted comprehensive score of the currently used frequency band will be lower than the preset scoring threshold or surpassed by the predicted comprehensive score of other frequency bands, start the handover preparation process in advance. This allows for proactive preparation for handover by anticipating frequency band performance degradation, avoiding sudden interruptions, and ensuring the continuity of signal transmission.

[0020] Furthermore, the handover preparation process includes frequency band pre-connection and data caching operations to reduce signal interruption time during the handover process. This is achieved by reducing the establishment time of frequency band handover through pre-connection and avoiding data loss during handover through data caching, thereby minimizing signal interruption time and ensuring the smoothness of virtual scenes and control commands.

[0021] Furthermore, it also includes a satellite communication module. The satellite communication module is activated when the predicted comprehensive score of all frequency bands selectable by the signal transmission module and the signal processing and receiving module is lower than the preset score threshold. It realizes signal transmission between the signal transmission module and the signal processing and receiving module through the satellite link. The satellite communication module serves as an emergency backup to ensure uninterrupted signal when the ground frequency band fails. It is suitable for remote areas or complex environments and improves control reliability.

[0022] Furthermore, virtual reality display devices include one of head-mounted virtual reality glasses or virtual reality helmets, to provide an immersive visual experience through head-mounted devices, enabling operators to obtain near first-person perspective environmental perception and improve spatial judgment and operational accuracy of remote control.

[0023] Furthermore, the virtual reality image acquisition module includes a high-definition binocular camera group, a panoramic camera, an infrared sensor, and an inertial measurement sensor, so as to realize the use of binocular cameras to provide depth information, panoramic cameras to cover the 360° environment, infrared sensors to adapt to low light scenes, and inertial measurement sensors to assist in attitude perception, so as to fully restore the environment and improve the realism of the virtual scene.

[0024] Furthermore, the flight control module provides control commands for the UAV's flight, including controlling the UAV's flight attitude, speed, altitude, and flight path. This covers the core parameters of UAV flight, ensuring that operators can fully control the UAV's status, perform complex actions and path planning, and meet diverse remote control needs.

[0025] The embodiments disclosed in this invention are preferred embodiments, but are not limited thereto. Those skilled in the art can easily understand the spirit of this invention based on the above embodiments and make different extensions and variations, but as long as they do not depart from the spirit of this invention, they are all within the protection scope of this invention.

Claims

1. A remote control system for unmanned aerial vehicles based on virtual reality technology, characterized in that, The system includes a drone terminal and a ground operation terminal; The UAV terminal includes a signal transmission module, a virtual reality image acquisition module, and a flight control module; The ground control terminal includes a virtual reality display device, a control command generation module, and a signal processing and transmission module; The virtual reality image acquisition module is used to acquire environmental images and data around the drone and transmit the environmental images and data to the signal transmission module. The signal transmission module is used to transmit the received environmental images and data to the signal processing and transmission module through multiple frequency bands. The signal processing and transmission module is used to transmit the received environmental images and data to the virtual reality display device. The virtual reality display device is used to construct a virtual control scene based on the received environmental images and data. The control command generation module is used to generate control commands based on user instructions and transmit them to the signal processing and transmission module. The signal processing and transmission module is used to transmit the received control commands to the signal transmission module via multiple frequency bands. The signal transmission module is used to transmit the received control commands to the flight control module. The flight control module is used to control the flight of the UAV based on the control commands.

2. The remote control system for unmanned aerial vehicles based on virtual reality technology according to claim 1, characterized in that, When the signal transmission module and the signal processing and receiving module transmit environmental images and data, the optimal frequency band is determined and selected using a pre-judgment method among the multiple frequency bands used. When the signal processing and receiving module transmits control commands to the signal transmission module, the same frequency band is used as when transmitting environmental images and data.

3. The remote control system for unmanned aerial vehicles based on virtual reality technology according to claim 2, characterized in that, The pre-determination method used by the signal transmission module includes the following steps; Step 1: Calculate the predicted comprehensive score for each frequency band within a preset time period using the pre-judgment algorithm formula. The formula is: ； In the formula, Indicates frequency band In the future A comprehensive score for time-based predictions. Indicates frequency band At the present moment The overall score is determined by the formula. Calculations show that This is the trend correction coefficient. Number of historical data sampling points The sampling time interval, and They represent frequency bands respectively. Overall ratings at different points in the past. Indicates frequency band The signal strength value, Indicates the signal strength stability coefficient. Indicates frequency band The degree of interference received Indicates the disturbance stability coefficient. Indicates frequency band The actual transmission rate Represents the transmission rate stability coefficient. This represents the historical performance coefficient of the frequency band. , , , These are the weighting coefficients for signal strength and stability, anti-interference capability and stability, transmission rate and stability, and historical performance, respectively. Step 2: Determine the optimal frequency band based on the comprehensive prediction score of each frequency band. The optimal frequency band is the one with the highest comprehensive prediction score. Step 3: Select the optimal frequency band for signal transmission and preset a scoring threshold. When it is predicted that the predicted comprehensive score of the currently used frequency band will be lower than the preset scoring threshold or surpassed by the predicted comprehensive score of other frequency bands, start the handover preparation process in advance.

4. The remote control system for unmanned aerial vehicles based on virtual reality technology according to claim 3, characterized in that, The handover preparation process includes frequency band pre-connection and data caching operations to reduce signal interruption time during the handover process.

5. A remote control system for unmanned aerial vehicles based on virtual reality technology according to claim 3, characterized in that, It also includes a satellite communication module, which is activated when the combined prediction score of all frequency bands selectable by the signal transmission module and the signal processing and receiving module is lower than a preset score threshold. The satellite communication module enables signal transmission between the signal transmission module and the signal processing and receiving module through a satellite link to ensure signal continuity.

6. The remote control system for unmanned aerial vehicles based on virtual reality technology according to claim 1, characterized in that, The virtual reality display device includes either head-mounted virtual reality glasses or a virtual reality helmet.

7. A remote control system for unmanned aerial vehicles based on virtual reality technology according to claim 1, characterized in that, The virtual reality image acquisition module includes a high-definition binocular camera group, a panoramic camera, an infrared sensor, and an inertial measurement sensor.

8. A remote control system for unmanned aerial vehicles based on virtual reality technology according to claim 1, characterized in that, The control commands from the flight control module for controlling the UAV's flight include controlling the UAV's flight attitude, speed, altitude, and flight path.

Images