Simulated unmanned aerial vehicle operation training server device, system and method

Abstract

The application provides a simulation unmanned aerial vehicle operation training server device, system and method, wherein the server device comprises: a physical engine module, which is used for generating a plurality of sensor data; updating the plurality of sensor data according to unmanned aerial vehicle position and attitude change data; a flight control module, which is used for performing attitude control of unmanned aerial vehicle simulation flight; a virtual engine module, which is used for providing a human-computer interaction interface; selecting unmanned aerial vehicle model information and flight scene information, sensor simulation parameters; collecting video data of unmanned aerial vehicle simulation flight and displaying the video data on the human-computer interaction interface; and a central processor, which is used for performing data forwarding processing between the physical engine module, the flight control module and the virtual engine module. Through the application, a virtual unmanned aerial vehicle training environment can be provided for users to perform unmanned aerial vehicle operation training, the efficiency and effect of unmanned aerial vehicle inspection training are improved, the cost and safety risk of unmanned aerial vehicle inspection training are reduced, and the skill level of unmanned aerial vehicle inspection operators is improved.

Description

Technical Field

[0001] This invention relates to the field of drone simulation technology, and in particular to a simulation drone operation training server device, system and method. Background Technology

[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] The safe operation of power lines is a crucial guarantee for the stability of the power system. With the continuous expansion of the power grid, drones, due to their flexibility, wide coverage, and high efficiency, are gradually becoming an important tool in power line inspection. Drone inspection technology can effectively reduce the risks of manual inspection and significantly improve inspection efficiency, playing a vital role, especially in large-scale power networks.

[0004] While drone technology has brought improvements in efficiency and safety, the wide variety of drones and the significant differences in their flight control systems and operating logic make traditional inspection and training methods inadequate to address this complexity. Operators need to switch flexibly between operating different drones, which places higher demands on the training process. Traditional training methods, including instructional videos and hands-on practice, are not only inefficient but also suffer from poor learning outcomes, limited application scenarios, and potential safety hazards. Summary of the Invention

[0005] This invention provides a simulated drone operation training server device to effectively solve the problems of low efficiency, poor learning outcomes, limited scenarios, and potential safety hazards associated with traditional training methods. The simulated drone operation training server device includes:

[0006] The physics engine module generates various sensor data for the simulated flight of the UAV based on the sensor simulation parameters provided by the virtual engine module, and provides this data to the flight control module; it determines the forces and torques of the simulated flight of the UAV based on the motor speeds provided by the flight control module; it determines the position and attitude change data of the simulated flight of the UAV based on the forces and torques; it updates the various sensor data based on the position and attitude change data; and it provides the updated sensor data to the flight control module.

[0007] The flight control module is used to perform attitude control of the UAV simulated flight based on the aircraft type information and flight scenario information provided by the virtual engine module and the various sensor data provided and updated by the physics engine module. It also provides the motor speed during the attitude control process to the physics engine module.

[0008] The virtual engine module provides a human-machine interface; based on the control commands input by the operator through the human-machine interface: select the drone model information and flight scenario information, and provide the model information and flight scenario information to the flight control module; select sensor simulation parameters, and provide the sensor simulation parameters to the physics engine module; collect video data of the drone's simulated flight and display it on the human-machine interface;

[0009] The central processor is used for data forwarding and processing between the physical engine module, flight control module, and virtual engine module.

[0010] This invention also provides a simulated drone operation training system to effectively solve the problems of low efficiency, poor learning outcomes, limited scenarios, and potential safety hazards associated with traditional training methods; the system includes:

[0011] The aforementioned simulated drone operation training server equipment;

[0012] The operating terminal is used to input control commands to the simulated drone operation training server equipment.

[0013] This invention also provides a simulated drone operation training method to effectively solve the problems of low efficiency, poor learning effect, limited scenarios, and potential safety hazards of traditional training methods; the method is applied to the aforementioned simulated drone operation training server equipment, including:

[0014] The virtual engine module provides a human-computer interaction interface; based on the control commands input by the operator through the human-computer interaction interface: select the drone model information and flight scenario information, and provide the model information and flight scenario information to the flight control module; select sensor simulation parameters, and provide the sensor simulation parameters to the physics engine module; collect video data of the drone's simulated flight and display it on the human-computer interaction interface;

[0015] The physics engine module generates various sensor data for the drone's simulated flight based on the sensor simulation parameters provided by the virtual engine module, and provides this data to the flight control module. Based on the motor speeds provided by the flight control module, it determines the forces and torques acting on the drone during simulated flight. Based on these forces and torques, it determines the drone's position and attitude change data. Based on the position and attitude change data, it updates the various sensor data and provides the updated sensor data to the flight control module.

[0016] The flight control module performs attitude control of the UAV simulated flight based on the aircraft type and flight scenario information provided by the virtual engine module and the various sensor data provided and updated by the physics engine module, and provides the motor speed during the attitude control process to the physics engine module.

[0017] The central processor handles data forwarding between the physical engine module, flight control module, and virtual engine module.

[0018] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described simulated drone operation training method.

[0019] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described simulated drone operation training method.

[0020] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described simulated drone operation training method.

[0021] This invention provides a realistic and efficient drone training system through the collaborative work of the physics engine module, flight control module, virtual engine module, and central processor in the server-side device. Compared with existing technologies, this invention can not only cope with the differences in flight control logic of different drones, but also realize simulated drone operation and training through the operator terminal. It provides users with a virtual drone training environment for drone operation training, improves the efficiency and effectiveness of drone inspection training, reduces the cost and safety risks of drone inspection training, and enhances the skill level of drone inspection operators. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0023] Figure 1 This is a structural example diagram of the simulated drone operation training server device in an embodiment of the present invention;

[0024] Figure 2 This is a structural example diagram of the physics engine module in an embodiment of the present invention;

[0025] Figure 3 This is a structural example diagram of the virtual engine module in an embodiment of the present invention;

[0026] Figure 4 This is a specific example diagram illustrating the structure of the virtual engine module in an embodiment of the present invention;

[0027] Figure 5 This is a structural example diagram of the flight control module in an embodiment of the present invention;

[0028] Figure 6 This is a structural example diagram of the central processor in an embodiment of the present invention;

[0029] Figure 7 This is a specific example diagram of the structure of the central processor in an embodiment of the present invention;

[0030] Figure 8 This is a specific example diagram of a simulated unmanned aerial vehicle (UAV) operation training system in an embodiment of the present invention;

[0031] Figure 9 This is an example diagram of a simulated drone operation training method in an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] The inventors discovered that Virtual Reality (VR) technology offers a new solution for drone inspection training. Through VR, trainees can interact and simulate operations in a safe virtual environment, resulting in a more intuitive and realistic learning experience while significantly reducing safety risks during training. The interactivity and immersive nature of VR make it an ideal tool for operator training, simulating real-world work scenarios in various environments. However, when faced with diverse drone models, traditional virtualization systems struggle to meet the flight control requirements of all models, limiting training effectiveness.

[0034] Therefore, the objective of the embodiments of the present invention is to:

[0035] (1) Addressing the issue of drone model differences in drone inspection training: With the rapid development of drone technology, various drone models have emerged on the market, each with its own flight control system and operating logic. Traditional training methods struggle to handle this complexity, making it difficult for operators to switch flexibly between different models. This invention aims to solve this problem by combining virtual reality technology with the flight control modules of different drones to create a training system capable of handling the differences in flight control logic between different models.

[0036] (2) Improving the efficiency and effectiveness of drone inspection training: Traditional drone inspection training methods, such as instructional videos and on-site practical operations, suffer from problems such as low efficiency, poor learning outcomes, limited scenarios, and potential safety hazards. This invention aims to provide trainees with a realistic virtual training environment by introducing virtual reality technology, enabling them to conduct efficient and safe training without external interference, thereby improving training efficiency and effectiveness.

[0037] (3) Reducing the cost and safety risks of drone inspection training: Drone inspection training often needs to be conducted in a real power line environment, which is not only costly but also poses certain safety risks. This embodiment of the invention aims to reduce the dependence on the real power line environment by creating a virtual training environment, thereby reducing training costs and safety risks.

[0038] (4) Improving the skill level of UAV inspection operators: UAV inspection operators need to possess certain flight skills and knowledge of power line inspection to ensure the smooth progress of inspection tasks. This embodiment of the invention aims to provide a realistic virtual training environment, allowing trainees to continuously practice and correct their operating skills in a simulated environment, thereby improving their skill level and ensuring that they can confidently cope with various complex situations in actual operation.

[0039] In summary, the purpose of this invention is to create a realistic, efficient, safe, and scalable drone inspection training system by combining virtual reality technology and flight control modules of different drones. This system aims to solve the problems existing in traditional training methods, improve training efficiency, effectiveness, and safety, and provide strong support for the stable operation of the power system.

[0040] Figure 1 This is a structural example diagram of the simulated drone operation training server device in an embodiment of the present invention, as shown below. Figure 1 As shown, the server-side device includes:

[0041] The physics engine module 101 is used to generate various sensor data for the simulated flight of the UAV based on the sensor simulation parameters provided by the virtual engine module, and to provide the various sensor data to the flight control module; to determine the forces and torques of the simulated flight of the UAV based on the motor speeds provided by the flight control module; to determine the position and attitude change data of the simulated flight of the UAV based on the forces and torques of the simulated flight of the UAV; to update the various sensor data based on the position and attitude change data; and to provide the updated various sensor data to the flight control module.

[0042] The flight control module 102 is used to perform attitude control of the UAV simulated flight based on the aircraft type information and flight scenario information provided by the virtual engine module and the various sensor data provided and updated by the physical engine module, and to provide the motor speed during the attitude control process to the physical engine module.

[0043] The virtual engine module 103 is used to provide a human-machine interface; based on the control commands input by the operator through the human-machine interface: select the drone model information and flight scenario information, and provide the model information and flight scenario information to the flight control module; select sensor simulation parameters, and provide the sensor simulation parameters to the physics engine module; collect video data of the drone's simulated flight, and display it on the human-machine interface;

[0044] The central processor 104 is used for data forwarding processing between the physical engine module, flight control module, and virtual engine module.

[0045] In this embodiment, the physics engine module is responsible for generating and simulating the physical behavior of the drone in the virtual environment, ensuring the authenticity and accuracy of the simulation training.

[0046] Figure 2 This is a structural example diagram of the physics engine module in an embodiment of the present invention, such as... Figure 2 As shown, the physics engine module includes:

[0047] The sensor data simulation unit 201 is used to generate various sensor data for the UAV flight simulation based on the sensor simulation parameters provided by the virtual engine module, and to provide the various sensor data to the flight control module; among them, the various sensors include: inertial measurement unit, global positioning system, magnetometer and airspeed sensor.

[0048] For example, the sensor data simulation unit can generate and provide data from various key sensors in real time, including the Inertial Measurement Unit (IMU), Global Positioning System (GPS), magnetometer, and airspeed sensor. The IMU provides data from the accelerometer and gyroscope to calculate the UAV's acceleration, angular velocity, and attitude angle; GPS provides geographic location, altitude, and speed information, simulating real satellite signal reception and processing; the magnetometer measures the strength and direction of the Earth's magnetic field, assisting the IMU in heading calculations and attitude corrections; and the airspeed sensor provides relative airflow speed data, crucial for flight dynamics control and aerodynamic analysis.

[0049] The force and torque conversion unit 202 is used to determine the forces and torques of the UAV simulated flight based on the motor speed provided by the flight control module; wherein, based on the motor speed, the thrust and torque of the motor are determined; based on the thrust and torque of the motor, combined with the aerodynamic forces and torques on the UAV, and environmental parameters, the forces and torques of the UAV simulated flight are determined.

[0050] For example, force and torque conversion: The physics engine module receives motor speed commands from the flight control module and converts them into forces and torques acting on the drone model. The specific process includes:

[0051] Motor model: Based on the motor characteristic curve, pulse width modulation signals or speed commands are converted into thrust and torque.

[0052] Aerodynamic calculations: Based on the UAV's geometric characteristics and current flight status (such as speed, angular velocity, angle of attack, etc.), calculate the aerodynamic forces and torques acting on the UAV.

[0053] Environmental impact: The simulation considers the effects of environmental parameters such as wind speed, wind direction, and airflow disturbance on drones, providing a more realistic simulation experience.

[0054] The UAV motion control unit 203 is used to determine the position and attitude change data of the UAV during simulated flight based on the forces and torques acting on the UAV during simulated flight, combined with the mass attributes and inertial matrix of the UAV.

[0055] For example, the physics engine module calculates the drone's position and attitude changes in space based on physical parameters such as mass and inertia matrix, combined with forces and torques. This mainly includes:

[0056] Mass attributes: The impact of the total mass and its distribution of the UAV on its motion characteristics.

[0057] Inertia matrix: describes the rotational inertia of the UAV about each axis, affecting the dynamic response to attitude changes.

[0058] Equations of motion: Using the Newton-Euler equations, the acceleration, velocity and position changes of the UAV under the action of force and torque are calculated, and the state of the UAV is updated in real time.

[0059] The feedback update unit 204 is used to update various sensor data based on position and attitude change data, and to provide the updated various sensor data to the flight control module.

[0060] For example, real-time feedback and updates: the physics engine module updates the drone's status at a high frequency in real time and feeds back the latest sensor data to the flight control module. This high-precision real-time simulation ensures that the flight control module can be accurately tested and validated in a virtual environment.

[0061] The physics engine module provides a solid foundation for flight training, flight control algorithm development, and UAV performance verification through its precise simulation capabilities, significantly improving the practicality and training effectiveness of the simulation system.

[0062] In this embodiment, the virtual engine module provides a human-computer interaction interface specifically designed for drone simulation operations. The virtual engine module boasts rich functionality, significantly improving user experience and operational efficiency.

[0063] Figure 3 This is a structural example diagram of the virtual engine module in an embodiment of the present invention, such as... Figure 3 As shown, the virtual engine module includes:

[0064] The parameter setting unit 301 is used to receive control commands input by the operator terminal through the human-machine interface;

[0065] The model selection unit 302 is used to select the model information and flight scenario information of the UAV according to the control command input by the operator through the human-machine interface, and provide the model information and flight scenario information to the flight control module.

[0066] Scene selection unit 303 is used to select sensor simulation parameters according to the control commands input by the operator through the human-machine interface, and provide the sensor simulation parameters to the physics engine module.

[0067] For example, aircraft selection: a variety of drone models are available for users to choose from, covering fixed-wing, rotary-wing and multi-rotor types, to support simulation training of different drone characteristics and meet diverse training needs.

[0068] Scene selection: Users can choose different flight environments and mission scenarios, including complex terrains such as cities, mountains, and oceans. Realistic scene design and rich details help simulate real operating conditions, enhancing the practicality of training.

[0069] Settings: Provides detailed system settings options, allowing users to customize various parameters as needed, including flight parameters, environmental parameters, and sensor simulation parameters, ensuring the flexibility and accuracy of training.

[0070] The video display unit 304 is used to collect video data of the drone's simulated flight and display it on the human-computer interaction interface;

[0071] For example, real-time video data: It integrates pod real-time video transmission capabilities, providing high-quality real-time video data. It supports the connection of various pod devices to meet the needs of different missions.

[0072] The collaborative processing unit 305 is used to synchronize instruction data and perform collaborative task execution when the parameter setting unit receives control instructions input from multiple operation terminals.

[0073] For example, multi-device networking: It supports collaborative operation of multiple devices, allowing multiple people to conduct simulation training simultaneously. Through real-time data synchronization and collaborative task execution, it significantly improves the efficiency of team collaboration.

[0074] Figure 4 This is a specific example diagram illustrating the structure of the virtual engine module in an embodiment of the present invention. For example... Figure 4 As shown in one embodiment, Figure 3 The structure of the virtual engine module in the embodiment of the present invention shown may further include: a graphics rendering unit 401, used for graphics rendering of video data displayed on the human-computer interaction interface.

[0075] For example, leveraging the powerful graphics rendering capabilities of Unreal Engine 5, it can provide highly realistic visual effects and interactive experiences, greatly enhancing the immersion of simulation training. The system architecture is flexible, supporting functional expansion and customized development, adapting to ever-changing training needs and ensuring the system's long-term applicability. The intuitive interface design and convenient operation process reduce the learning curve and difficulty for users, making operation more efficient and convenient.

[0076] In this embodiment, the flight control module is a key component of the system, which supports the integration of multiple flight control modules. The flight control module is primarily responsible for attitude calculation and control based on sensor data, ensuring stable flight of the UAV in complex environments.

[0077] Figure 5 This is a structural example diagram of the flight control module in an embodiment of the present invention, such as... Figure 5 As shown, the flight control module includes:

[0078] The attitude calculation unit 501 is used to perform attitude calculation for the simulated flight of the UAV based on the aircraft type information and flight scenario information provided by the virtual engine module and the various sensor data provided and updated by the physics engine module.

[0079] For example, attitude calculation: The flight control module uses data from the inertial measurement unit (IMU), global positioning system (GPS), magnetometer, and airspeed sensor to calculate the UAV's attitude in real time, including pitch, roll, and yaw angles. Through high-precision attitude estimation, the flight control module can effectively ensure the stability of the UAV in various complex environments.

[0080] The attitude control unit 502 is used to perform attitude control of the UAV simulated flight based on the attitude calculation results, and to provide the motor speed during the attitude control process to the physics engine module.

[0081] For example, attitude control: Based on the attitude calculation results, the flight control module generates corresponding control commands to adjust the motor speed and control surface angle to maintain or change the UAV's attitude. This module realizes functions such as automatic stabilization, path tracking, and mission execution, optimizing its flight performance and mission completion by dynamically adjusting the UAV's attitude.

[0082] In this embodiment, the central processor is the intelligent core of the server, responsible for data forwarding, video processing, recognition, and augmented reality (AR) overlay.

[0083] Figure 6 This is a structural example diagram of the central processor in an embodiment of the present invention, such as... Figure 6 As shown, the central processor includes:

[0084] The data forwarding unit 601 is used to perform data forwarding processing between the physical engine module, the flight control module, and the virtual engine module.

[0085] For example, data forwarding: The central processor is responsible for efficient data communication with the simulation system, the operator terminal, and various pod modules. Its main task is to ensure the real-time synchronization of flight status data, sensor data, and command information, thereby guaranteeing the overall coordination and response speed of the system.

[0086] The video processing unit 602 is used to perform image stabilization, noise suppression, and enhancement processing on video data of drone simulated flight.

[0087] For example, in video processing: the central processor receives and processes real-time video streams from the pods, performing operations such as image stabilization, noise suppression, and enhancement. This process not only improves the quality of the video data but also provides clear and reliable visual support for the execution and monitoring of flight missions.

[0088] Figure 7 This is a specific example diagram of the structure of the central processor in an embodiment of the present invention. For example... Figure 7 As shown in one embodiment, Figure 6 The structure of the central processor in the embodiment of the present invention shown may further include:

[0089] The intelligent recognition unit 701 is used to identify target objects and / or target areas in video data of drone simulated flight using deep learning algorithms, and to mark the recognition results.

[0090] For example, AI (Artificial Intelligence) recognition: Utilizing deep learning algorithms, the central processor can automatically identify and classify targets, accurately detect and label objects or areas of interest. This function covers a variety of recognition tasks, such as people search, vehicle recognition, and building detection, significantly improving task efficiency and accuracy.

[0091] Augmented reality overlay unit 702 is used to overlay the recognition results with video data of drone simulated flight.

[0092] For example, AR overlay: The central processor overlays AI recognition results with real-time video streams to provide augmented reality functionality. Overlaying virtual information and markers, such as navigation paths, target locations, and status information, onto the video image effectively enhances the operator's perception and decision-making capabilities.

[0093] This invention also provides a simulated unmanned aerial vehicle (UAV) operation training system. Figure 8 This diagram illustrates a specific example of a simulated UAV operation training system according to an embodiment of the present invention. In this embodiment, the system can adopt a B / S architecture design. The instructor's end interacts with the server through a ground station operation terminal, while the trainee's end interacts with the server through a handheld PAD. The server processes core data and simulation calculations, ensuring efficient system operation and real-time feedback. The instructor's end is responsible for setting up, monitoring, and guiding the training process. The trainee's end provides an operation interface for the handheld PAD, supporting simulated training.

[0094] like Figure 8 As shown: The simulated drone operation training system includes:

[0095] The aforementioned simulated drone operation training server equipment;

[0096] The operating terminal is used to input control commands to the simulated drone operation training server equipment.

[0097] The structure and functions of the simulated drone operation training server equipment are as described above and will not be repeated here.

[0098] In practice, the operating terminal may include:

[0099] The instructor-side device is used for instructors to input control commands, including drone simulation aircraft mission release commands, which carry drone model information, flight scenario information, and sensor simulation parameters for the released mission.

[0100] For example, (1) Operation control: Real-time viewing of the drone's status information, including key parameters such as position, speed, altitude, attitude and sensor data; monitoring of trainee operations, and having the function of terminating or taking over control to ensure the safety of trainees in abnormal or dangerous situations.

[0101] (2) Flight route planning: The instructor has the authority to formulate and adjust the flight missions and routes for trainees. During the planning process, the instructor can provide real-time guidance based on the trainee's operation and adjust flight parameters or set missions.

[0102] (3) Parameter Configuration: Provides higher-level parameter configuration permissions. Instructors can not only configure flight control parameters and communication settings, but also lock or restrict certain functions to guide trainees to complete specific training objectives. Scenarios and tasks can be preset according to course requirements to ensure the relevance and effectiveness of training.

[0103] (4) Annotation and Feedback: It has real-time annotation and feedback functions, which can add annotations, reminders or warnings during the trainees' operation. At the same time, instructors can also evaluate the trainees' performance and provide detailed feedback and summary after training.

[0104] The student terminal device is used for students to input control commands, including drone simulation aircraft mission execution commands. The drone simulation aircraft mission execution commands carry the drone model information, flight scenario information, and sensor simulation parameters of the drone performing the mission.

[0105] For example, (1) Operation control: The status information of the drone can be viewed in real time, and tasks can be executed according to operational requirements. However, the operator's operating authority will be restricted, especially in the adjustment of some key parameters, to prevent misoperation from affecting task execution.

[0106] (2) Flight route planning: Trainees can perform basic flight route planning and flight path settings according to mission requirements, but adjustments are required under the guidance of instructors. Trainees' operational authority focuses more on learning how to perform correct flight route planning and mission execution, rather than on free configuration.

[0107] (3) Parameter configuration: Trainees are allowed to adjust basic parameters, such as flight altitude, speed and waypoint missions, but instructors provide guidance and restrictions on the configuration of key parameters to ensure the safety and effectiveness of trainee operations.

[0108] (4) Annotation and feedback: Trainees can record key points or annotations in their operations during the flight for later review.

[0109] Overall, the instructor's side has more management, supervision, and guidance functions, and higher permissions, allowing them to intervene and evaluate students' actions. In contrast, the student's side is mainly used to perform basic operations and learning tasks, has more limited permissions, and must follow the instructor's instructions.

[0110] This invention also provides a method for training operators of simulated unmanned aerial vehicles (UAVs), as described in the following embodiments. Since the principle underlying this method is similar to that of the simulated UAV operation training server equipment, its implementation can be found in the implementation of the simulated UAV operation training server equipment; repeated details will not be elaborated further.

[0111] Figure 9This is an example diagram of a simulated drone operation training method in an embodiment of the present invention, such as... Figure 9 As shown, the method includes:

[0112] Step 901: The virtual engine module provides a human-computer interaction interface; based on the control commands input by the operator through the human-computer interaction interface: select the drone model information and flight scenario information, and provide the model information and flight scenario information to the flight control module; select sensor simulation parameters, and provide the sensor simulation parameters to the physics engine module; collect video data of the drone's simulated flight and display it on the human-computer interaction interface;

[0113] Step 902: The physics engine module generates various sensor data for the UAV simulated flight based on the sensor simulation parameters provided by the virtual engine module, and provides this data to the flight control module; it determines the forces and torques of the UAV simulated flight based on the motor speeds provided by the flight control module; it determines the position and attitude change data of the UAV simulated flight based on the forces and torques; it updates the various sensor data based on the position and attitude change data; and it provides the updated sensor data to the flight control module.

[0114] Step 903: The flight control module performs attitude control of the UAV simulated flight based on the aircraft type information and flight scenario information provided by the virtual engine module and the various sensor data provided and updated by the physics engine module, and provides the motor speed during the attitude control process to the physics engine module.

[0115] Step 904: The central processor performs data forwarding processing between the physical engine module, flight control module, and virtual engine module.

[0116] It should be noted that, Figure 9 The execution order of the steps in the process shown is not limited to sequential execution. Instead, they can be executed sequentially, in parallel, or cyclically, depending on the actual execution situation. No restrictions are imposed here.

[0117] In summary, the simulated drone operation training server equipment, system, and method provided by the embodiments of the present invention can achieve the following:

[0118] (1) Improving Training Efficiency and Effectiveness: By combining virtual reality technology and the flight control module of UAVs, the training system in this embodiment of the invention can provide trainees with a realistic and efficient training environment. This simulation method allows trainees to practice operation in a near-realistic work scenario, thereby mastering UAV inspection skills more quickly and improving training efficiency. At the same time, the realistic simulation environment also helps to enhance the trainees' learning experience and improve training effectiveness.

[0119] (2) Reduced training costs and safety risks: Traditional training methods often require real power line environments, which are not only costly but also pose certain safety risks. The training system in this embodiment of the invention, however, can be conducted in a safe virtual environment, significantly reducing training costs and safety risks. Trainees can focus on improving their skills without external interference, reducing the risk of equipment damage or personal injury due to operational errors.

[0120] (3) Enhancing the flexibility and scalability of training: Due to the wide variety of UAVs and the significant differences in their flight control systems and operating logic, traditional training methods often struggle to address this complexity. However, the training system in this embodiment can flexibly adapt to the differences in flight control logic among different UAVs. By adjusting the flight control module, training support for different models can be achieved. This not only enhances the flexibility of training but also improves the scalability of the system, enabling it to be continuously updated and upgraded as UAV technology develops.

[0121] (4) Improve operators' skill level: Through the immersive and interactive nature of virtual reality technology, trainees can continuously practice and correct their operating skills in a simulated environment, thereby improving their skill level more quickly. This training method not only enables trainees to master basic operating skills, but also allows them to deal with various complex situations with more confidence in actual operation.

[0122] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described simulated drone operation training method.

[0123] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described simulated drone operation training method.

[0124] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described simulated drone operation training method.

[0125] This invention provides a realistic and efficient drone training system through the collaborative work of the physics engine module, flight control module, virtual engine module, and central processor in the server-side device. Compared with existing technologies, this invention can not only cope with the differences in flight control logic of different drones, but also realize simulated drone operation and training through the operator terminal. It provides users with a virtual drone training environment for drone operation training, improves the efficiency and effectiveness of drone inspection training, reduces the cost and safety risks of drone inspection training, and enhances the skill level of drone inspection operators.

[0126] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A simulated unmanned aerial vehicle (UAV) operation training server device, characterized in that, It includes a central processor, and a physical engine module, a flight control module, and a virtual engine module, all connected to the central processor; wherein: The physics engine module generates various sensor data for the simulated flight of the UAV based on the sensor simulation parameters provided by the virtual engine module, and provides this data to the flight control module; it determines the forces and torques of the simulated flight of the UAV based on the motor speeds provided by the flight control module; it determines the position and attitude change data of the simulated flight of the UAV based on the forces and torques; it updates the various sensor data based on the position and attitude change data; and it provides the updated sensor data to the flight control module. The flight control module is used to perform attitude control of the UAV simulated flight based on the aircraft type information and flight scenario information provided by the virtual engine module and the various sensor data provided and updated by the physics engine module. It also provides the motor speed during the attitude control process to the physics engine module. The virtual engine module provides a human-machine interface; based on the control commands input by the operator through the human-machine interface: select the drone model information and flight scenario information, and provide the model information and flight scenario information to the flight control module; select sensor simulation parameters, and provide the sensor simulation parameters to the physics engine module; collect video data of the drone's simulated flight and display it on the human-machine interface; The central processor is used for data forwarding and processing between the physical engine module, flight control module, and virtual engine module.

2. The simulated drone operation training server equipment as described in claim 1, characterized in that, The physics engine module includes: The sensor data simulation unit is used to generate various sensor data for the UAV flight simulation based on the sensor simulation parameters provided by the virtual engine module, and to provide the various sensor data to the flight control module; among them, the various sensors include: inertial measurement unit, global positioning system, magnetometer and airspeed sensor; The force and torque conversion unit is used to determine the forces and torques acting on the drone during simulated flight based on the motor speed provided by the flight control module. Specifically, it determines the thrust and torque of the motor based on the motor speed. Based on the thrust and torque of the motor, combined with the aerodynamic forces and torques acting on the drone, and environmental parameters, it determines the forces and torques acting on the drone during simulated flight. The UAV motion control unit is used to determine the position and attitude change data of the UAV during simulated flight based on the forces and torques acting on the UAV during simulated flight, combined with the UAV's mass attributes and inertial matrix. The feedback update unit is used to update various sensor data based on position and attitude change data; and provides the updated various sensor data to the flight control module.

3. The simulated drone operation training server equipment as described in claim 1, characterized in that, The virtual engine module includes: The parameter setting unit is used to receive control commands input by the operator through the human-machine interface. The model selection unit is used to select the UAV model information and flight scenario information according to the control commands input by the operator through the human-machine interface, and provide the model information and flight scenario information to the flight control module. The scene selection unit is used to select sensor simulation parameters based on the control commands input by the operator through the human-computer interaction interface, and provide the sensor simulation parameters to the physics engine module. The video display unit is used to collect video data from the simulated flight of the drone and display it on the human-computer interaction interface. The collaborative processing unit is used to synchronize instruction data and perform collaborative task execution when the parameter setting unit receives control instructions input from multiple operation terminals.

4. The simulated drone operation training server equipment as described in claim 3, characterized in that, The virtual engine module also includes: The graphics rendering unit is used to render the video data displayed on the human-computer interaction interface.

5. The simulated drone operation training server equipment as described in claim 1, characterized in that, The flight control module includes: The attitude calculation unit is used to calculate the attitude of the drone in simulated flight based on the aircraft type information and flight scenario information provided by the virtual engine module and the various sensor data provided and updated by the physics engine module. The attitude control unit is used to control the attitude of the UAV during simulated flight based on the attitude calculation results, and provides the motor speed during the attitude control process to the physics engine module.

6. The simulated drone operation training server equipment as described in claim 1, characterized in that, The central processor includes: The data forwarding unit is used to perform data forwarding processing between the physical engine module, flight control module, and virtual engine module. The video processing unit is used to perform image stabilization, noise suppression, and enhancement processing on video data from simulated drone flights.

7. The simulated drone operation training server equipment as described in claim 1, characterized in that, The central processor also includes: The intelligent recognition unit is used to identify target objects and / or target areas in video data of drone simulated flight using deep learning algorithms, and to mark the recognition results. Augmented reality overlay unit, used to overlay recognition results with video data of drone simulated flight.

8. A simulated unmanned aerial vehicle (UAV) operation training system, characterized in that, include: The simulated unmanned aerial vehicle (UAV) operation training server equipment according to any one of claims 1 to 7; The operating terminal is used to input control commands to the simulated drone operation training server equipment.

9. The system as described in claim 8, characterized in that, The operating terminal includes: The instructor-side device is used for instructors to input control commands, including drone simulation aircraft mission release commands, which carry drone model information, flight scenario information, and sensor simulation parameters for the released mission. The student terminal device is used for students to input control commands, including drone simulation aircraft mission execution commands. The drone simulation aircraft mission execution commands carry the drone model information, flight scenario information, and sensor simulation parameters of the drone performing the mission.

10. A training method for operating a simulated unmanned aerial vehicle (UAV), characterized in that, This method is applied to the simulated drone operation training server equipment according to any one of claims 1 to 7, comprising: The virtual engine module provides a human-computer interaction interface; based on the control commands input by the operator through the human-computer interaction interface: select the drone model information and flight scenario information, and provide the model information and flight scenario information to the flight control module; select sensor simulation parameters, and provide the sensor simulation parameters to the physics engine module; collect video data of the drone's simulated flight and display it on the human-computer interaction interface; The physics engine module generates various sensor data for the drone's simulated flight based on the sensor simulation parameters provided by the virtual engine module, and provides this data to the flight control module. Based on the motor speeds provided by the flight control module, it determines the forces and torques acting on the drone during simulated flight. Based on these forces and torques, it determines the drone's position and attitude change data. Based on the position and attitude change data, it updates the various sensor data and provides the updated sensor data to the flight control module. The flight control module performs attitude control of the UAV simulated flight based on the aircraft type and flight scenario information provided by the virtual engine module and the various sensor data provided and updated by the physics engine module, and provides the motor speed during the attitude control process to the physics engine module. The central processor handles data forwarding between the physical engine module, flight control module, and virtual engine module.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of claim 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of claim 10.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of claim 10.

Images