Unmanned aerial vehicle hardware-in-the-loop simulation test system and method

Abstract

The invention provides an unmanned aerial vehicle hardware-in-the-loop simulation test system and method. The system comprises a simulation environment module, a working condition simulation module, a flight control simulation module, a sensor simulation module, a data interaction and protocol conversion module and an edge calculation module. The flight control simulation module is used for generating flight control state data based on the unmanned aerial vehicle state data including the wind field environment effect; the sensor simulation module is integrated in the simulation environment module and is used for generating sensor data with fault characteristics according to the fault control signal sent by the fault injection unit; the data interaction and protocol conversion module is used for performing protocol and hardware interface format conversion on the flight control state data and the sensor data; the edge calculation module is used for processing the converted data and generating a control instruction; and the flight control simulation module is used for outputting a flight control signal to the simulation environment module according to the converted control instruction. And comprehensive closed-loop verification of industry-level unmanned aerial vehicle edge computing software under the conditions of multi-source data input, complex working condition simulation and flexible hardware docking is realized.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) simulation testing technology, and in particular to a hardware-in-the-loop simulation testing system and method for UAVs. Background Technology

[0002] In industries such as inspection, logistics, and surveying, edge computing modules of drones undertake core tasks such as real-time data processing and intelligent decision-making from multiple sensors including LiDAR, cameras, and gimbals. The reliability of their software functions directly affects the safety and efficiency of drone operations. Hardware-in-the-Loop (HIL) simulation testing, which verifies the execution logic of embedded software on real hardware by simulating hardware input in a simulation environment, is a key means to ensure the reliability of edge computing modules.

[0003] Currently, most simulation testing systems for UAVs focus on verifying flight control algorithms. However, existing technologies have significant shortcomings in black-box testing of edge computing modules: Regarding test scenario coverage, existing systems lack dedicated test architectures for the black-box nature of edge computing modules, which are verified "only through input and output." This limits test scenarios to aspects like flight control communication, failing to simulate the complete verification process of edge computing modules through real hardware inputs such as LiDAR point clouds and flight control command streams, resulting in incomplete scenario coverage. In terms of simulating complex operating conditions, real UAV operating environments involve complex conditions such as sensor failures, communication interference, and wind disturbances. Existing simulation systems typically only support single-type fault injection, and these operating condition simulation functions are not deeply integrated with in-loop testing of edge computing hardware. This results in a substantial absence of complex operating condition simulation in testing, making it impossible to verify the robustness of edge computing software under extreme conditions. At the hardware interaction level, there is a compatibility gap between the hardware interface of the edge computing module and the simulation data protocol. Existing technologies mostly use fixed interface mapping, which makes it difficult to flexibly adapt to edge computing hardware from different manufacturers. Furthermore, they generally do not simulate network characteristics such as latency and packet loss in real data transmission, reflecting a rigid hardware interaction method and resulting in a deviation between the test environment and the real scenario.

[0004] In summary, existing technologies for black-box testing of UAV edge computing modules suffer from incomplete scenario coverage, lack of complex operating conditions, and rigid hardware interaction. There is an urgent need to build a hardware-in-the-loop simulation testing system that supports multi-source data input, complex operating condition simulation, and flexible hardware integration to meet the comprehensive verification needs of industry-level UAV edge computing software. Summary of the Invention

[0005] This invention provides a hardware-in-the-loop simulation testing system and method for unmanned aerial vehicles (UAVs) to address issues such as incomplete scenario coverage, lack of complex operating condition simulation, and rigid hardware interaction in existing black-box testing of UAV edge computing modules. It achieves comprehensive closed-loop verification of industry-grade UAV edge computing software under conditions of multi-source data input, complex operating condition simulation, and flexible hardware integration. The technical solution proposed by this invention is as follows: In a first aspect, the present invention provides a hardware-in-the-loop simulation testing system for unmanned aerial vehicles (UAVs), comprising: The simulation environment module is used to simulate drones and sensors and run the physics engine; The operating condition simulation module includes a fault injection unit and a wind disturbance unit. The fault injection unit is communicatively connected to the sensor simulation module and is used to send fault control signals to the sensor simulation module. The wind disturbance unit is communicatively connected to the physics engine of the simulation environment module and is used to send wind field configuration parameters to the physics engine. The physics engine of the simulation environment module simulates the wind field environment according to the wind field configuration parameters, and generates and outputs UAV status data containing the effects of the wind field environment. The flight control simulation module is communicatively connected to the simulation environment module and is used to receive the UAV status data and generate flight control status data based on the UAV status data. The sensor simulation module, integrated into the simulation environment module, is used to generate sensor data with fault characteristics based on the fault control signal sent by the fault injection unit. The data interaction and protocol conversion module is communicatively connected to the flight control simulation module, the sensor simulation module and the edge computing module, respectively. It is used to receive the flight control status data and the sensor data with fault characteristics, and to convert the protocol and hardware interface format, and send the converted data to the edge computing module. The edge computing module is used to receive and process the converted data and generate control commands; The data interaction and protocol conversion module is further configured to receive control commands generated by the edge computing module, convert the control commands into protocols and interface formats, and send the converted control commands to the flight control simulation module. The flight control simulation module is configured to output flight control signals to the simulation environment module according to the converted control commands, and the flight control signals are used to drive the simulated UAV model in the simulation environment module.

[0006] Optionally, the data interaction and protocol conversion module includes a flight control data interaction unit, a lidar data interaction unit, a camera data interaction unit, and a gimbal data interaction unit; The flight control data interaction unit is used to receive the flight control status data through the first communication interface, send it to the edge computing module through the first hardware interface, and receive the control commands through the first hardware interface and send them to the flight control simulation module through the second communication interface. The lidar data interaction unit is used to receive lidar point cloud data through the third communication interface and send it to the edge computing module through the second hardware interface. The camera data interaction unit is used to receive camera video stream data through the fourth communication interface and send it to the edge computing module through the third hardware interface. The gimbal data interaction unit is used to receive gimbal control commands through the fourth hardware interface and convert them into control signals executable by the simulated gimbal, and to receive feedback attitude data from the simulated gimbal and send it to the edge computing module through the fourth hardware interface.

[0007] Optionally, the first communication interface and the second communication interface are UDP ports, and the flight control status data and the control commands are encapsulated using the MAVLink protocol; the first hardware interface is a USB to serial port.

[0008] Optionally, the fault injection unit is configured to generate adjustable data packet loss instructions, and the sensor simulation module randomly discards data packets at a specified ratio when generating lidar point cloud data according to the data packet loss instructions.

[0009] Optionally, the wind disturbance unit is configured to generate wind field configuration parameters including wind speed, wind direction, and turbulence model parameters; the physics engine of the simulation environment module simulates the force exerted by the wind field on the simulated UAV according to the wind field configuration parameters, and calculates the motion state of the simulated UAV based on the force, and outputs UAV state data including the wind field disturbance effect.

[0010] Optionally, the lidar simulation unit, camera simulation unit, and gimbal simulation unit in the sensor simulation module communicate through ROS nodes and achieve time-series synchronization of data generation by subscribing to and publishing ROS topics under the same time base.

[0011] Optionally, the system further includes a visualization monitoring module, which is communicatively connected to the flight control simulation module to receive and display UAV status data, and communicatively connected to the edge computing module to receive and display data processing results.

[0012] In a second aspect, the present invention also provides a hardware-in-the-loop simulation test method for unmanned aerial vehicles (UAVs), using the UAV hardware-in-the-loop simulation test system as described in the first aspect, the method comprising: The fault injection unit in the working condition simulation module sends a fault control signal to the sensor simulation module and sends wind field configuration parameters to the physics engine of the simulation environment module through the wind disturbance unit; the physics engine of the simulation environment module simulates the wind field environment according to the wind field configuration parameters, and generates and outputs UAV status data containing the effects of the wind field environment. The flight control simulation module receives the UAV status data and generates flight control status data based on the UAV status data; the sensor simulation module generates sensor data with fault characteristics based on the fault control signal. The data interaction and protocol conversion module receives the flight control status data and the sensor data with fault characteristics, performs protocol and hardware interface format conversion, and sends the converted data to the edge computing module. The edge computing module receives and processes the converted data and generates control commands; The data interaction and protocol conversion module receives control commands generated by the edge computing module, converts the control commands into protocols and interface formats, and sends the converted control commands to the flight control simulation module. The flight control simulation module outputs flight control signals to the simulation environment module according to the converted control commands. The flight control signals are used to drive the simulated UAV model in the simulation environment module.

[0013] Optionally, the physics engine of the simulation environment module simulates the wind field environment according to the wind field configuration parameters, including: The physics engine of the simulation environment module simulates the force exerted by the wind field on the simulated UAV based on wind field configuration parameters including wind speed, wind direction and turbulence model parameters, and calculates the motion state of the simulated UAV based on the force.

[0014] Optionally, the fault control signal and the wind farm configuration parameters are saved as a test case template; when executing the test, the test case template is called to configure the operating condition simulation module.

[0015] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows: The present invention provides a hardware-in-the-loop simulation testing system and method for unmanned aerial vehicles (UAVs). The simulation environment module simulates UAV dynamics and environmental interactions through a physics engine, generating UAV state data including wind field influences. Simultaneously, the sensor simulation module generates sensor data with fault characteristics based on instructions from the fault injection unit. The flight control simulation module receives the aforementioned UAV state data and generates flight control state data, thus forming a complete input chain of wind field disturbance, fault injection, and control command generation, solving the problem of incomplete scene coverage. The sensor simulation module integrates multi-source sensor concurrent data generation and ensures timing consistency through timestamp synchronization. The data interaction and protocol conversion module encapsulates sensor data and flight control state data, and sends them to the edge computing module after protocol conversion and hardware interface adaptation, reproducing multi-sensor collaborative operation scenarios to address insufficient collaboration. The operating condition simulation module dynamically injects sensor faults and communication interference through the fault injection unit, and simulates complex wind field environments through the wind disturbance unit. Combined with a closed-loop test chain of fault signal-sensor data anomaly-edge computing processing-control feedback, it verifies the robustness of the edge computing module under extreme conditions, solving the problem of missing simulations for complex operating conditions. The data interaction and protocol conversion module supports dynamic protocol mapping and network characteristic simulation, eliminating interface adaptation barriers. It also sends control commands generated by the edge computing module to the flight control simulation module via bidirectional data flow after protocol conversion, driving the simulated UAV model in the simulation environment and forming a hardware interaction closed loop. This reduces the deviation between the test environment and the real-world scenario, solving the problem of rigid hardware interaction. Overall, the system provides a high-coverage, high-fidelity hardware-in-the-loop testing solution for industry-grade UAV edge computing software through multi-source data input, complex operating condition closed-loop simulation, and a flexible hardware interaction mechanism.

[0016] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1This is a schematic diagram of the structure of the UAV hardware-in-the-loop simulation test system provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] Existing technologies for black-box testing of edge computing modules have significant shortcomings: Regarding test scenario coverage, existing systems lack a dedicated test architecture designed for the black-box nature of edge computing modules, which are "verified only through input and output." This limits test scenarios to aspects like flight control communication, failing to simulate the complete verification process of edge computing modules through real hardware inputs such as LiDAR point clouds and flight control command streams, resulting in incomplete scenario coverage. In terms of complex operating condition simulation, real-world UAV operating environments involve complex conditions such as sensor failures, communication interference, and wind disturbances. Existing simulation systems typically only support single-type fault injection, and these operating condition simulation functions are not deeply integrated with in-loop testing of edge computing hardware. This results in a substantial absence of complex operating condition simulation in testing, making it impossible to verify the robustness of edge computing software under extreme conditions. Regarding multi-sensor collaboration, industrial UAVs require the fusion of multi-source sensor data. While existing simulation platforms can simulate multiple sensors, they struggle to achieve efficient and high-fidelity interface between sensor simulation data and edge computing hardware. This fails to reproduce real-world operating scenarios involving concurrent and time-series synchronized multi-sensor data, demonstrating insufficient multi-sensor collaborative simulation. At the hardware interaction level, there is a compatibility gap between the hardware interfaces of the edge computing module (such as Ethernet ports and serial ports) and the simulation data protocols (such as MAVLink). Existing technologies mostly use fixed interface mappings, making it difficult to flexibly adapt to edge computing hardware from different manufacturers. Furthermore, they generally do not simulate network characteristics such as latency and packet loss in real data transmission, reflecting a rigid hardware interaction method and leading to deviations between the test environment and real-world scenarios. Based on this, this invention provides a hardware-in-the-loop simulation testing system and method for unmanned aerial vehicles (UAVs).

[0022] The hardware-in-the-loop simulation testing system for unmanned aerial vehicles (UAVs) provided by this invention includes, at the hardware level, one or more high-performance computers running the Linux operating system, and an edge computing module to be tested. At the software level, the system includes: a simulation environment module, a working condition simulation module, a flight control simulation module, a sensor simulation module, a data interaction and protocol conversion module, and an edge computing module.

[0023] The simulation environment module is used to simulate drones and sensors and run the physics engine. It uses the Gazebo simulator to build a 3D simulation scene. A simulated drone model with physical properties is loaded into this scene, and simulated LiDAR, cameras, and gimbals are configured for it. Gazebo's built-in physics engine is used to calculate the motion state of the drone and environmental objects in real time.

[0024] The operational simulation module operates as a software service and includes two functional sub-units: a fault injection unit and a wind disturbance unit. The fault injection unit communicates with the sensor simulation module, sending fault control signals to it. The wind disturbance unit communicates with the physics engine of the simulation environment module, sending wind field configuration parameters. The physics engine simulates the wind field environment based on these parameters, generating and outputting UAV status data incorporating the effects of the wind environment. Specifically, the fault injection unit simulates sensor anomalies. It receives user configurations, generates corresponding fault control signals, and sends them to the sensor simulation module. For example, it might instruct the lidar to randomly drop data packets at a set ratio within a specific time period, or cause the camera to output a specified number of abnormal image frames. The wind disturbance unit simulates atmospheric disturbances. It allows users to configure parameters such as wind speed, wind direction, and turbulence intensity, sending these parameters to the Gazebo simulator's physics engine. The physics engine calculates the continuously changing wind force based on these parameters and applies it to the simulated UAV model, thus affecting its flight attitude and trajectory.

[0025] The flight control simulation module, communicatively connected to the simulation environment module, receives the UAV status data and generates flight control status data based on it. Specifically, the flight control simulation module can be implemented using a PX4 Software-in-the-Loop (PX4 SITL) instance. This instance establishes a connection with the simulated UAV model in the Gazebo emulator via a User Datagram Protocol (UDP) port, receives UAV status data including wind field environmental factors, runs its internal control algorithm based on this data, and ultimately generates flight control status data reflecting the current system state of the UAV.

[0026] A sensor simulation module, integrated into the simulation environment module, is used to generate sensor data with fault characteristics based on the fault control signals sent by the fault injection unit. Specifically, the sensor simulation module is driven by the sensor plugin of the Gazebo emulator and is used to generate simulated sensor data. Sensor plugins may include LiDAR plugins and camera plugins, etc. The LiDAR plugin generates 3D point cloud data, and the camera plugin generates image and video stream data. During the generation process, these data are affected by the fault control signals from the fault injection unit, thereby producing sensor data with specified fault characteristics.

[0027] The data interaction and protocol conversion module is the core of the simulation environment's interface with real hardware, responsible for bidirectional conversion between data protocols and hardware interface formats. This module is communicatively connected to the flight control simulation module, sensor simulation module, and edge computing module. It receives flight control status data and sensor data with fault characteristics, performs protocol and hardware interface format conversion to a format compatible with the edge computing module's physical interface, and then sends the converted data to the edge computing module. The edge computing module receives and processes the converted data and generates control commands. Specifically, the edge computing module serves as the actual physical hardware of the object under test, such as an embedded computing platform based on ARM or x86 architecture. This module receives the converted data from the data interaction and protocol conversion module through its own physical interfaces such as serial ports and network ports, runs its internal application program to process the data, and generates corresponding control commands.

[0028] The data interaction and protocol conversion module is further configured to receive control commands generated by the edge computing module, convert the control commands into a protocol and interface format that can be recognized by the flight control simulation module, and send the converted control commands to the flight control simulation module. The flight control simulation module is configured to output flight control signals to the simulation environment module according to the converted control commands, and the flight control signals are used to drive the simulated UAV model in the simulation environment module.

[0029] The hardware-in-the-loop simulation test system for unmanned aerial vehicles provided by this invention initializes and establishes connections for each module in sequence after the system starts up, forming the following working closed loop.

[0030] S110. Simulation Data Generation and Disturbance Injection: The Gazebo simulation environment runs, and the simulated UAV model and sensors begin to function. The wind disturbance unit applies the configured wind field parameters to the physics engine, generating and outputting UAV state data that includes the effects of the wind field environment. Simultaneously, the fault injection unit sends fault control signals to the sensor simulation module, infusing the generated sensor data with fault characteristics.

[0031] S120. Data Acquisition and Protocol Conversion: The flight control simulation module receives the UAV status data and generates flight control status data based on this data. The sensor simulation module continuously outputs the sensor data with fault characteristics. The data interaction and protocol conversion module synchronously receives the flight control status data and the sensor data with fault characteristics.

[0032] S130, Data Injection Hardware: The data interaction and protocol conversion module converts the flight control status data and the sensor data with fault characteristics into protocol and hardware interface formats, and sends the converted data to the edge computing module through a physical cable.

[0033] S140. Hardware processing and instruction generation: The edge computing module receives and processes the converted data and generates control instructions, such as new flight waypoints, speed commands, or gimbal rotation angles.

[0034] S150, Command Feedback and Simulation Execution: The edge computing module sends the control commands through the physical interface. The data interaction and protocol conversion module receives the control commands, converts the control commands into protocols and interface formats, and sends the converted control commands to the flight control simulation module. The flight control simulation module outputs flight control signals to the simulation environment module according to the converted control commands. The flight control signals are used to drive the simulated UAV model in the simulation environment module.

[0035] Through the above process, the data interaction and protocol conversion module, edge computing module, flight control simulation module, and simulation environment module together constitute a closed-loop test system containing real hardware. The simulation environment module and sensor simulation module generate test data containing disturbances, which is then converted by the data interaction and protocol conversion module and injected into the edge computing module. The edge computing module processes the data, makes decisions, and generates control commands. These control commands are then converted again by the data interaction and protocol conversion module and fed back to the flight control simulation module, ultimately driving the simulated UAV model in the simulation environment module. This allows for thorough verification of the functional correctness, real-time performance, and robustness of the edge computing module under complex input conditions in a highly controllable and repeatable simulation environment.

[0036] Taking the scenario of unmanned aerial vehicle (UAV) power line inspection as an example, this paper illustrates the application of this system.

[0037] A 3D model including power transmission towers, conductors, and surrounding terrain and vegetation is established in the simulation environment module. A continuous low-altitude crosswind is set using the wind disturbance unit in the operating condition simulation module to simulate a valley wind field. Intermittent data packet loss of the lidar is set using the fault injection unit in the operating condition simulation module to simulate occasional sensor failures. The system initiates closed-loop testing. The edge computing module continuously receives point cloud data with packet loss converted by the data interaction and protocol conversion module, UAV status data affected by wind disturbance, and visible light video streams, and runs its autonomous inspection and obstacle avoidance algorithm. Testers observe through ground station software whether the UAV can stably track the preset flight path under wind disturbance and whether it can correctly identify and avoid conductor obstacles, thereby comprehensively evaluating the performance and reliability of the algorithm in the edge computing module. Through the above specific implementation, this invention constructs a high-fidelity, flexibly configurable UAV hardware-in-the-loop simulation test environment, effectively solving the problem of systematic, real-world-like black-box testing of edge computing hardware modules.

[0038] The UAV hardware-in-the-loop simulation testing system provided by this invention includes a simulation environment module that simulates UAV dynamics and environmental interactions through a physics engine, generating UAV state data including wind field influences. Simultaneously, a sensor simulation module generates sensor data with fault characteristics based on instructions from a fault injection unit. The flight control simulation module receives the aforementioned UAV state data and generates flight control state data, thus forming a complete input chain of wind field disturbance, fault injection, and control command generation, solving the problem of incomplete scene coverage. The sensor simulation module integrates multi-source sensor concurrent data generation and ensures timing consistency through timestamp synchronization. The data interaction and protocol conversion module encapsulates sensor data and flight control state data, then sends them to the edge computing module after protocol conversion and hardware interface adaptation, reproducing multi-sensor collaborative operation scenarios to address insufficient collaboration. The operating condition simulation module dynamically injects sensor faults and communication interference through a fault injection unit, and simulates complex wind field environments through a wind disturbance unit. Combined with a closed-loop test chain of fault signal-sensor data anomaly-edge computing processing-control feedback, it verifies the robustness of the edge computing module under extreme conditions, solving the problem of missing simulations for complex operating conditions. The data interaction and protocol conversion module supports dynamic protocol mapping and network characteristic simulation, eliminating interface adaptation barriers. It also sends control commands generated by the edge computing module to the flight control simulation module via bidirectional data flow after protocol conversion, driving the simulated UAV model in the simulation environment and forming a hardware interaction closed loop. This reduces the deviation between the test environment and the real-world scenario, solving the problem of rigid hardware interaction. Overall, the system provides a high-coverage, high-fidelity hardware-in-the-loop testing solution for industry-grade UAV edge computing software through multi-source data input, complex operating condition closed-loop simulation, and a flexible hardware interaction mechanism.

[0039] In one embodiment, reference is made to Figure 1As shown, the data interaction and protocol conversion module consists of four functional sub-units: flight control data interaction unit, lidar data interaction unit, camera data interaction unit, and gimbal data interaction unit. Each unit works in concert to achieve seamless integration between simulation data and real hardware.

[0040] The flight control data interaction unit is used to receive the flight control status data through the first communication interface, send it to the edge computing module through the first hardware interface, and receive the control commands through the first hardware interface and send them to the flight control simulation module through the second communication interface. The lidar data interaction unit is used to receive lidar point cloud data through the third communication interface and send it to the edge computing module through the second hardware interface. The camera data interaction unit is used to receive camera video stream data through the fourth communication interface and send it to the edge computing module through the third hardware interface. The gimbal data interaction unit is used to receive gimbal control commands through the fourth hardware interface and convert them into control signals executable by the simulated gimbal, and to receive feedback attitude data from the simulated gimbal and send it to the edge computing module through the fourth hardware interface.

[0041] In the specific implementation of this system, the first and second communication interfaces are UDP ports, and the flight control status data and control commands are encapsulated using the MAVLink protocol. The flight control simulation module continuously broadcasts flight control status data packets encapsulated using the MAVLink protocol to the outside world through UDP port 14540. These data packets contain key status information such as the UAV's real-time attitude angle, GPS position, airspeed, and battery voltage. The flight control data interaction unit listens for and receives these MAVLink data packets on this port. At the same time, the flight control data interaction unit sends back control commands encapsulated using the MAVLink protocol to the flight control simulation module through UDP port 14580. These commands are generated by the edge computing module and may include target waypoints, speed commands, or mode switching commands.

[0042] To achieve physical connection with the edge computing hardware, the first hardware interface is a USB-to-serial adapter. Specifically, a USB-to-serial chip, such as the FTDIFT232RL or Silicon Labs CP2102, is connected to the computer running the flight control data interaction unit. This adapter appears as a virtual serial port in the operating system. The flight control data interaction unit's program uses this virtual serial port to send the parsed flight control status data in a specific frame format according to the serial communication protocol defined by the edge computing module. Correspondingly, the edge computing module receives these data frames through its own UART serial port and sends back the control commands it generates through the same physical serial port link. The flight control data interaction unit continuously listens to this serial port, repackages the received command data into MAVLink protocol format, and sends it to the flight control simulation module through UDP port 14580, thus completing the bidirectional communication loop.

[0043] This invention ensures standardized and efficient data transmission between the flight control simulation module and the data interaction module by uniformly defining the first and second communication interfaces as UDP ports and encapsulating them using MAVLink, a de facto standard communication protocol in the UAV field. The connectionless nature of the UDP protocol reduces communication overhead, while the lightweight binary message format of the MAVLink protocol guarantees efficient data parsing. This allows flight control status and control commands to be exchanged at a high frequency and with low latency, laying the foundation for real-time hardware-in-the-loop testing.

[0044] By employing a USB-to-serial adapter as the primary hardware interface, this approach creatively solves the common interface mismatch problem between high-performance simulation computers and embedded edge computing hardware. While USB interfaces are ubiquitous on general-purpose computers, serial ports remain the primary debugging and control interface for many embedded hardware devices. The USB-to-serial adapter acts as a bridge, eliminating the system's reliance on the specific and potentially scarce interfaces of edge computing hardware, thus enhancing the physical connection compatibility of the test system with different hardware platforms. Users only need to replace the adapter or configure the serial port parameters to quickly connect to edge computing modules from different manufacturers and models, significantly improving the flexibility and ease of deployment of the test system.

[0045] The flight control simulation side uses the standard UDP / MAVLink protocol stack, ensuring compatibility with mainstream simulation toolchains; the hardware side uses general-purpose serial communication, reducing specific requirements for hardware interfaces. The core function of the flight control data interaction unit is to perform precise conversion and forwarding between these two protocol stacks. This design not only makes the system architecture clear and easy to maintain, but more importantly, it allows for adaptation to different communication protocol requirements by modifying the conversion logic without altering the flight control simulation environment and the edge computing hardware itself, providing possibilities for the functional expansion and customization of the test system.

[0046] Specifically, refer to Figure 1 As shown, the flight control data interaction unit runs a Micro Air Vehicle Link (MAVLink) protocol conversion program. This program listens for and receives data from the flight control simulation module (corresponding to the first communication interface, i.e., User Datagram Protocol (UDP) port 14540) via the first communication interface. Figure 1 The program uses the MAVLink protocol data packets from the PX4 SITL (presumably a specific device or interface) to parse the UAV's attitude, position, velocity, and other flight control status data. Then, through the first hardware interface—a USB to Serial (usb0 to serial) adapter—the program encapsulates the parsed flight control status data according to the serial communication protocol format required by the edge computing module and sends it. When the edge computing module returns control commands (such as waypoint commands) through this serial port, the flight control data interaction unit receives the command, re-encapsulates it into a MAVLink message, and sends it back to the flight control simulation module through the second communication interface, UDP port 14580.

[0047] The LiDAR data interaction unit operates a point cloud data processing node, which receives LiDAR point cloud data from the sensor simulation module via a third communication interface. The data is in the Robot Operating System (ROS) standard Sensor Message Point Cloud 2 (sensor_msgsPointCloud2, PC2) format. The node converts this point cloud data into the native binary data stream format of the specific LiDAR hardware (e.g., Livox Mid-360), resulting in... Figure 1 The mid360 point cloud data is transmitted to the edge computing module's network port via the second hardware interface, i.e., the network port, using the UDP protocol through UDP port 56300.

[0048] The camera data interaction unit runs a video streaming server that receives ROS video streams from the sensor simulation module via a fourth communication interface. The server uses a video encoding library (such as GStreamer or FFmpeg) to encode the video stream into a Real-Time Messaging Protocol (RTMP) video stream in real time, and pushes the RTMP video stream to the edge computing module through a third hardware interface, namely a USB to Ethernet adapter.

[0049] The gimbal data interaction unit connects to the edge computing module via a fourth hardware interface (USB-1 to serial port). This unit contains two programs that work together: a simulated gimbal driver and a simulated gimbal serial interaction program. The simulated gimbal driver runs as a ROS node, responsible for listening to the ROS control topic ` / gimbal / control` to receive gimbal control commands and publishing the gimbal attitude topic ` / gimbal / state` to provide feedback on the current attitude. This driver drives the simulated gimbal model to rotate by calling function interfaces provided by the Gazebo simulator (such as speed control or angle control interfaces). The simulated gimbal serial interaction program runs as a serial-to-ROS bidirectional gateway. This program listens to serial port data, parses and converts the gimbal control commands (such as pitch and yaw commands) sent by the edge computing module via the serial port into ROS standard gimbal control commands, publishes them to the ` / gimbal / control` topic, and then delivers them to the simulated gimbal driver for execution. Simultaneously, the program subscribes to the gimbal / state topic published by the simulation gimbal driver, encodes attitude data (such as the current pitch angle of 30°) into a specific serial protocol format, and sends it back to the edge computing module through the same serial port (USB1 to serial). Through the collaboration of these two programs, gimbal control commands are issued from the edge computing module to the simulation environment, and gimbal attitude status is fed back from the simulation environment to the edge computing module, thus forming a closed loop for gimbal control. The edge computing module can calculate and issue new adjustment commands based on the feedback attitude data and business requirements.

[0050] In the system architecture, a Micro AirVehicle Link-Robot Operating System (MAVROS) communication package can also be optionally configured. This package is used to conveniently convert MAVLink messages from the flight control simulation module into robot operating system topics in software-in-the-loop test mode, allowing other ROS nodes to consume them or use them for debugging. Figure 1 As shown. In the core hardware-in-the-loop test closed loop of this invention, flight control status data and control commands are forwarded through a dedicated data interaction and protocol conversion module, without relying on the MAVROS communication packet.

[0051] This invention utilizes four dedicated interactive units to achieve precise one-to-one conversion from simulation formats to hardware interface formats, addressing the differences in data types and protocols such as flight control status, LiDAR point clouds, camera video streams, and gimbal control. This enables the system to simultaneously and accurately inject multiple sensor data into the edge computing module, solving the realism problem of multi-sensor data collaborative simulation and providing the edge computing module with an input environment consistent with actual operations. Each unit employs lightweight data processing logic, such as direct protocol encapsulation and format transcoding, avoiding the introduction of complex middleware processing. Flight control status data is transmitted via direct UDP connection and serial port pass-through, while point clouds and video streams are sent directly through high-speed network interfaces, significantly reducing data transmission latency. The measured end-to-end latency can be controlled within 50ms. This ensures the real-time performance of the test loop, enabling the edge computing module to respond promptly to changes in the simulation environment, and the test results better reflect its performance in the actual system. Furthermore, the module adopts an interface and protocol separation design: communication interfaces such as UDP ports and ROS interfaces are responsible for interfacing with the simulation system, while hardware interfaces such as serial ports and Ethernet ports are responsible for interfacing with the actual hardware. Users can quickly adapt to edge computing hardware from different manufacturers and with different interface definitions by replacing adapters (such as different models of USB-to-serial chips) or adjusting protocol configurations (such as modifying MAVLink message mapping and point cloud data format). This solves the problem of rigid hardware interface adaptation and significantly improves the versatility and deployment efficiency of the testing system.

[0052] In one embodiment, the fault injection unit is configured to generate adjustable data packet loss instructions, and the sensor simulation module randomly discards data packets at a specified ratio when generating lidar point cloud data according to the data packet loss instructions.

[0053] In practice, the fault injection unit runs an independent configuration service that provides a graphical user interface or configuration file interface, allowing testers to set packet loss ratio parameters, which are continuously adjustable within the range of 0% to 100%. For example, testers can set a command for "data packet loss rate of 30%".

[0054] Once the test is initiated, the fault injection unit sends an instruction containing the packet loss ratio parameter to the LiDAR simulation unit within the sensor simulation module. In the Gazebo simulation environment, the LiDAR simulation unit periodically generates raw point cloud data frames at a preset frequency. Before each frame of point cloud data is published to the ROS topic, the LiDAR simulation unit invokes a packet loss simulation algorithm based on the received packet loss instruction. This algorithm generates a random number between 0 and 1. If the random number is less than the set packet loss ratio (e.g., 0.3), the frame of point cloud data is marked as discarded and not published; if the random number is greater than or equal to the ratio, the frame is published normally. In this way, the effect of randomly discarding data packets at a specified ratio is achieved over time, simulating intermittent data loss caused by hardware failure, obstruction, or strong interference in the LiDAR system.

[0055] This invention provides graphical or configurable packet loss ratio parameter settings, allowing testers to precisely control the severity of injected faults, such as simulating different levels of sensor anomalies like minor (5% packet loss), moderate (30% packet loss), or complete failure (100% packet loss). This adjustable parameter design transforms fault simulation from a binary state of presence or absence into a continuously adjustable, quantifiable test condition, solving the problem of existing technologies' single-mode simulation of complex operating conditions and their inability to be quantitatively configured. The sensor simulation module randomly drops packets proportionally according to instructions, and its output point cloud data stream exhibits intermittent and randomly missing characteristics, highly consistent with the data anomaly characteristics of LiDAR caused by mechanical vibration, contamination, or electromagnetic interference in the real world. This highly faithful fault data stream is injected into the edge computing module through the data interaction and protocol conversion module, forcing the perception, localization, or obstacle avoidance algorithms within the edge computing module to process incomplete and unreliable input data. This effectively tests the algorithm's fault tolerance under abnormal input, the stability of state estimation, and the security of decision logic. Based on an adjustable packet loss ratio, testers can design tiered test cases. For example, they can gradually increase the packet loss rate and observe the attenuation of the edge computing module's output results (such as target detection rate and positioning accuracy), thereby quantitatively assessing its performance boundaries and failure thresholds. This method transforms the assessment of the robustness of the edge computing module from a qualitative judgment to a quantifiable and comparable objective evaluation, providing crucial data support for module reliability certification and determining algorithm optimization directions. Through the above implementation, the fault injection unit and the sensor simulation module work together to transform the abstract operating condition of sensor failure into a precisely controllable, highly realistic, and quantifiable test input. This not only fills the gap in the existing hardware-in-the-loop testing capabilities for simulating complex sensor failures but also significantly improves the depth and effectiveness of testing by constructing near-realistic abnormal scenarios, ensuring that the edge computing module verified by this system possesses higher reliability to cope with complex real-world challenges.

[0056] In one embodiment, the wind disturbance unit is configured to generate wind field configuration parameters including wind speed, wind direction, and turbulence model parameters to simulate atmospheric environmental disturbances. The physics engine of the simulation environment module simulates the force exerted by the wind field on the simulated UAV based on the wind field configuration parameters, calculates the motion state of the simulated UAV based on the force, and outputs UAV state data including the wind field disturbance effect.

[0057] In practice, the wind disturbance unit operates as a standalone configuration service, providing a graphical interface or application programming interface (API) that allows testers to set a complete set of wind field configuration parameters. Wind speed is continuously adjustable from 0 m / s to 20 m / s, for example, set to 8 m / s. Wind direction is set with true north as 0 degrees and can be set from 0 degrees to 360 degrees, for example, set to 180 degrees. Turbulence model parameters can be selected and configured using standard atmospheric turbulence models (such as the Dryden model), including turbulence scale and intensity.

[0058] Once configured, the wind disturbance unit sends a wind field configuration command containing the aforementioned wind field configuration parameters to the physics engine interface integrated in the simulation environment module via a communication interface (such as ROS service or UDP message). Upon receiving the command, the Gazebo simulator's physics engine calculates and generates a dynamic wind field vector field that conforms to the set parameters in real time. This wind field acts as a continuous external force applied to the aerodynamic center of the simulated UAV model.

[0059] Within each simulation step, the physics engine comprehensively calculates the forces exerted by the wind field on the UAV, the UAV's own gravity, aerodynamic forces, and motor thrust, and solves for the six-degree-of-freedom motion state of the simulated UAV using the Newton-Euler equations of motion. The results are directly reflected in the real-time changes of the simulated UAV model's position, attitude, linear velocity, and angular velocity. These dynamically changing state data are output in real time, including UAV state data incorporating the effects of wind field disturbances, and are subsequently acquired by the flight control simulation module as the basis for generating its flight control state data.

[0060] This invention allows for independent and continuous parameterization of wind speed, direction, and turbulence models. The wind disturbance unit enables testers to accurately construct various atmospheric environments, ranging from calm breezes to strong crosswinds, gusts, and complex turbulence. This addresses the shortcomings of existing technologies, such as the lack of environmental disturbance simulation or the use of single models, providing highly controllable and reproducible complex environmental inputs for testing. The physics engine calculates and applies wind field forces in real time based on the configured parameters, causing dynamic responses such as attitude oscillations and trajectory deviations in the simulated UAV model. This response is fed back in real time to the flight control simulation module via UAV state data incorporating the disturbance effect, and then transmitted to the edge computing module through the data interaction module. This simulates the dynamic environmental challenges that the flight control system and decision-making module must jointly address in real flight, allowing testing not only on the stability of the flight control system but also on the perception, planning, and decision-making performance of the edge computing module on a dynamic platform. The deep integration of the wind disturbance unit with the simulation environment physics engine transforms wind field disturbances from an abstract environmental description into a precisely injectable test variable with strict physical causality that dynamically affects the entire system state. This not only fills the gap in dynamic environment simulation in hardware-in-the-loop testing, but also improves the reliability and extrapolation value of test results by constructing a perturbation environment consistent with the physical laws of the real world, ensuring that the UAV edge computing module verified by this system has the ability to operate safely and reliably under complex weather conditions.

[0061] In one embodiment, the lidar simulation unit, camera simulation unit, and gimbal simulation unit in the sensor simulation module communicate through ROS nodes and achieve time-series synchronization of data generation by subscribing to and publishing ROS topics under the same time base.

[0062] In practice, the lidar simulation unit, camera simulation unit, and gimbal simulation unit are each encapsulated as independent ROS nodes or node sets. These nodes start up together during system initialization and connect to the same ROS master node, forming a unified communication network. All simulation unit nodes use a unified simulation time published by the Gazebo emulator or the system master clock as the time reference for data generation and publication. This time signal is typically broadcast through ROS clock topics or a high-precision system clock service to ensure that each node has a consistent understanding of the current simulation time.

[0063] After each simulation unit node completes the simulation calculation for a frame of data (e.g., a LiDAR completes a scan, a camera acquires an image, or a gimbal updates its attitude), it does not immediately publish the data asynchronously. Instead, it waits for or marks a publication time aligned with a unified time reference. For example, all sensors can be configured to publish data at a fixed frequency, and their publication actions can be aligned in time through ROS's time scheduling mechanism.

[0064] To achieve higher-precision application-level synchronization, a synchronization manager node can be designed. This node subscribes to all sensor data topics and uses ROS message filters or an approximate time synchronization strategy to trigger a synchronization packet ready event only when messages from all relevant topics are received within a small time tolerance (e.g., ±10 milliseconds), or to package and publish the synchronized messages to a new fusion topic. This ensures that downstream edge computing modules can receive multi-source sensor data that is strictly time-corresponding.

[0065] Furthermore, in the software-in-the-loop testing mode of the system, one or more in-computer control programs can optionally run as ROS nodes. These programs can subscribe to synchronized sensor data topics or other status topics, generate control commands based on their internal algorithms, and then publish them via ROS topics or the MAVLink protocol for early-stage algorithm development and logic verification, without needing to connect to actual hardware. This provides a smooth transition path for the system from pure software simulation to hardware-in-the-loop testing.

[0066] In the Gazebo simulation environment, the simulation step size and sensor update frequency can be configured. By setting the simulation step size of the physics engine to an integer multiple of the sensor update cycle, and directly triggering the generation and publication of sensor data from the physics engine's callback function, it is possible to ensure from the source that the generation time of each sensor data is strictly aligned with the physical state time of the simulated world.

[0067] This invention fundamentally solves the problem of timing discrepancies caused by different generation and transmission delays in multi-sensor data by forcing all simulation unit nodes to publish data based on the same time base and supplementing it with a message synchronization strategy. This ensures that the lidar point cloud, camera image, and gimbal attitude data correspond precisely in timestamps, simulating the effect achieved in real sensor systems through hardware triggering or precise clock synchronization, and solving the inherent problem of timing asynchrony in existing multi-sensor collaborative simulations. Time-synchronized multi-source data is a prerequisite for accurate perception fusion, such as image-point cloud fusion and vision-inertial odometry. The strictly synchronized data stream provided by this embodiment allows the fusion algorithm within the edge computing module to perform calculations based on environmental information captured at the same time, avoiding fusion errors and state estimation divergence introduced by data timing deviations. This improves the simulation fidelity of the simulation test to the working environment of the real multi-sensor fusion system, making the algorithm performance verified in the simulation more reliably transferable to real hardware. By deeply integrating the ROS communication mechanism with the time management of the simulation environment, this embodiment not only realizes concurrent simulation of multi-sensor data but also achieves time-synchronized simulation. This ensures the authenticity and rigor of hardware-in-the-loop test inputs at the information source level, making it possible to test and verify UAV applications that rely on multi-sensor fusion (such as autonomous navigation, dynamic obstacle avoidance, and precision inspection), and significantly improving the reliability of this test system in verifying complex and highly reliable edge computing software.

[0068] In one embodiment, the system further includes a visualization monitoring module, which can be implemented using ground station software, such as the QGroundControl (QGC) ground station. The visualization monitoring module is communicatively connected to the flight control simulation module to receive and display UAV status data, and also communicatively connected to the edge computing module to receive and display data processing results.

[0069] QGC establishes a MAVLink protocol communication connection with the flight control simulation module (PX4SITL) via a User Datagram Protocol port (e.g., UDP port 18570) to receive UAV status data in real time. Simultaneously, the visualization and monitoring module connects to the edge computing module via the robot operating system network. Specifically, a data bridging and display node runs on the same computer running QGC or on another network-accessible computer. This node subscribes to data processing results published by the edge computing module via ROS topics, such as: perceptionfusedmap (representing a fused perception map) and planningtrajectory (representing a planned trajectory).

[0070] After receiving MAVLink messages from the flight control simulation module, the QGC ground station software displays all key flight parameters of the UAV in real time on its graphical interface, including its 3D position, flight trajectory, attitude angles, airspeed, altitude, battery level, and GPS status. Simultaneously, it provides interactive functions such as virtual joystick, waypoint planning, and command issuance, allowing test personnel to intervene in control or modify the mission.

[0071] The data bridging and display nodes (such as Rviz) receive ROS topic messages from the edge computing module, transforming abstract data processing results into intuitive visualization elements. For example, point cloud fusion results can be rendered as a 3D colored point cloud overlaid on a synchronized view of the Gazebo simulation scene; planned trajectories can be displayed as a continuous path line; and object detection boxes can be superimposed on the camera image stream. This allows testers to directly see the internal decision-making processes and output effects of the edge computing module.

[0072] This invention integrates UAV status data from the flight control simulation module and decision results from the edge computing module into a unified visualization interface. Testers can obtain a complete system operation profile from low-level flight control to high-level intelligent decision-making in a single view. This fundamentally changes the limitation of traditional black-box testing, which only observes the final output behavior, improving the observability and controllability of the test. Because the visualization monitoring module simultaneously receives and synchronizes data streams from both the flight control and edge computing modules, testers can directly correlate qualitative visual observations with quantitative performance indicators. Performance indicators can include command response latency, positioning error, algorithm frame rate, etc. For example, when jitter in the planned trajectory is observed, the corresponding sensor data quality or flight control status can be checked simultaneously. The introduction of the visualization monitoring module upgrades the UAV hardware-in-the-loop testing system of this invention from a purely data input / output verification tool into a comprehensive testing and verification platform integrating real-time monitoring, interactive control, qualitative evaluation, and in-depth diagnostics. By providing intuitive and relevant visual feedback, it significantly improves the depth and efficiency of testing, enabling each test to generate richer insights, thereby more comprehensively and reliably verifying the overall performance and robustness of the UAV edge computing system.

[0073] The hardware-in-the-loop simulation test method for unmanned aerial vehicles (UAVs) provided by this invention is described below. The hardware-in-the-loop simulation test method described below and the hardware-in-the-loop simulation test system described above can be referred to and correspond to each other.

[0074] The UAV hardware-in-the-loop simulation test method provided by this invention uses the UAV hardware-in-the-loop simulation test system described above, and the method includes: S210, the fault injection unit in the working condition simulation module sends a fault control signal to the sensor simulation module, and sends wind field configuration parameters to the physical engine of the simulation environment module through the wind disturbance unit; the physical engine of the simulation environment module simulates the wind field environment according to the wind field configuration parameters, and generates and outputs UAV status data containing the effects of the wind field environment.

[0075] In the configuration interface of the operating condition simulation module, testers set specific test conditions. For example, through the interface of the fault injection unit, the packet loss rate of the lidar was set to 30%, generating a corresponding fault control signal and sending it to the lidar simulation unit in the sensor simulation module. Simultaneously, through the interface of the wind disturbance unit, the wind speed was set to 8 m / s, the wind direction to 180 degrees (due south), and the Dryden turbulence model was selected. Wind field configuration parameters containing these parameters were generated and sent to the physics engine interface of the Gazebo simulator in the simulation environment module. The Gazebo simulator's physics engine receives these parameters in real time, calculates and generates a dynamic southerly wind field in the simulation world. This wind field continuously acts on the simulated UAV model, thereby generating and outputting UAV state data that includes the effects of wind field disturbances, such as the position of the drift and the fluctuating attitude angle.

[0076] S220: The flight control simulation module receives the UAV status data and generates flight control status data based on the UAV status data; the sensor simulation module generates sensor data with fault characteristics based on the fault control signal.

[0077] The flight control simulation module, PX4 SITL, continuously acquires UAV state data, including wind field disturbance effects, from the simulation environment module via a UDP port. Based on this real-time state, PX4 SITL runs its internal control laws and state estimation algorithms to generate standard flight control state data and encapsulates it into MAVLink messages. Simultaneously, the lidar simulation unit in the sensor simulation module, when generating each frame of point cloud data, randomly discards that frame with a 30% probability based on the received fault control signal, thereby generating and outputting lidar point cloud data with packet loss fault characteristics.

[0078] S230, the data interaction and protocol conversion module receives the flight control status data and the sensor data with fault characteristics, performs protocol and hardware interface format conversion, and sends the converted data to the edge computing module.

[0079] The various units in the data interaction and protocol conversion module work synchronously. The flight control data interaction unit receives flight control status data in MAVLink format from PX4 via UDP port 14540, parses it, repackages it according to the serial communication protocol of the edge computing module via USB-to-serial port usb0, and then sends it. The lidar data interaction unit receives lidar point cloud data with packet loss characteristics via ROS topic, converts it from ROS PointCloud2 format to the Livox Mid-360 native binary format required by the edge computing module, and sends it via Ethernet port 56300 using UDP protocol. Other units, such as the camera data interaction unit, also perform similar receiving and conversion operations. Finally, all data that has undergone protocol and hardware interface format conversion is sent in parallel to the corresponding hardware interface of the edge computing module via the corresponding physical cables (serial and Ethernet cables).

[0080] S240, The edge computing module receives and processes the converted data and generates control commands.

[0081] As a real physical hardware edge computing module, it receives converted data from the data interaction and protocol conversion module via its serial port and Ethernet port. Its internally running autonomous algorithms, such as perception fusion and path planning algorithms, perform real-time processing and fusion calculations on received data including flight control status with wind disturbances, point clouds with packet loss, and normal video streams. Based on the processing results, the algorithm makes decisions and ultimately generates control commands. For example, in an obstacle avoidance scenario, it might generate a waypoint command to yaw 10 degrees to the left to avoid obstacles and output it via the serial port.

[0082] S250 The data interaction and protocol conversion module receives the control commands generated by the edge computing module, converts the control commands into protocol and interface formats, and sends the converted control commands to the flight control simulation module.

[0083] The flight controller data interaction unit in the data interaction and protocol conversion module receives control commands generated by the edge computing module via a USB-to-serial port (usb0). This unit parses the meaning of the received serial port command data and repackages it into MAVLink protocol messages recognizable by the PX4 flight controller, such as the MISSION_ITEM message.

[0084] S260. The flight control simulation module outputs flight control signals to the simulation environment module according to the converted control commands. The flight control signals are used to drive the simulated UAV model in the simulation environment module.

[0085] The flight control data interaction unit sends the encapsulated MAVLink control commands to the flight control simulation module, PX4 SITL, via UDP port 14580. PX4 SITL receives and parses these commands, converting them into low-level control commands for the simulated UAV model in the Gazebo simulation environment, such as adjusting motor speeds, thus outputting flight control signals to the simulation environment module. Based on these control signals, the Gazebo simulator's physics engine drives the simulated UAV model to perform corresponding flight maneuvers, such as initiating a left turn, thereby completing a full test loop from environmental disturbance, simulation perception, hardware processing to simulation execution.

[0086] Specifically, after receiving and processing the converted data sent by the data interaction and protocol conversion module, the edge computing module executes a series of predefined algorithmic logics within its internal software. These logics collectively constitute the core functionality of the module. This internal logic specifically includes: parsing the input flight control status data to understand the current flight status; simultaneously, running integrated anomaly detection and fault-tolerance algorithms on parallel-received LiDAR point cloud data containing packet loss anomalies, camera video stream data affected by wind disturbances, and flight control status data fluctuating due to environmental disturbances. For example, it performs point cloud data completion to repair missing information or performs attitude estimation compensation to filter environmental interference. Based on the results of the above multi-source information fusion and anomaly processing, it makes real-time intelligent decisions and generates corresponding control commands. For example, when an obstacle is detected ahead and the attitude is unstable, it decides and outputs a "UAV emergency climb" command. Finally, the control command is output through a physical interface and transmitted back to the flight control simulation module via the data interaction and protocol conversion module. The entire process of data parsing, multi-sensor information fusion, anomaly handling, intelligent decision-making, and instruction generation described above all pertains to the internal software logic and implementation details of the edge computing module. This invention achieves rigorous black-box testing by constructing an external hardware-in-the-loop testing system and a closed-loop method, which only observes and verifies the inputs and outputs without involving the specific internal implementation.

[0087] It should be noted that the core of the system and method described in this invention lies in constructing a general hardware-in-the-loop simulation testing framework and environment. This framework simulates and provides multi-source, perturbed, standardized input data to the edge computing module, and performs closed-loop verification of its output control commands, thereby achieving black-box testing of the overall function and performance of the module. How the edge computing module specifically implements data parsing, information fusion, anomaly handling, and decision generation processes are internal software algorithm details and are not the focus of this invention.

[0088] Using the method of this invention, testers only need to configure the operating parameters in step S210; subsequent data generation, conversion, injection, processing, and closed-loop verification are all automatically completed by the system. This solves the problems of chaotic and inefficient traditional manual or script-based testing processes, improves the execution speed and batch operation capability of test cases, and ensures high repeatability of test results across different times and batches. This invention deeply couples wind farm environmental disturbances and sensor fault simulation to the source of simulation data generation and accurately injects them into real hardware through conversion. This coherent process from environmental modeling to hardware injection forces a deep integration of complex operating conditions and hardware testing at the methodological level, rather than isolated simulation. It ensures that the edge computing module receives a comprehensive input signal that simultaneously includes the effects of dynamic environmental disturbances and sensor anomalies during testing, thereby enabling realistic and integrated verification of its overall performance in actual complex operating environments.

[0089] This invention systematically defines how to inject simulation data into a hardware black box, how to collect hardware output commands, and how to feed these commands back to the simulation environment to verify their correctness through four steps: S230 data conversion and hardware injection, S240 hardware processing and decision-making, S250 command feedback and conversion, and S260 command execution and closed-loop verification. This establishes a rigorous and universal black-box functional verification methodology. Testers do not need to understand the internal code of the edge computing module; they can determine the functional correctness of its internal algorithm simply by observing whether the drone model in the simulation environment ultimately performs the expected actions, such as successful obstacle avoidance and stable hovering. This truly realizes black-box testing of embedded intelligent hardware systems.

[0090] In one embodiment, the physics engine of the simulation environment module simulates the wind field environment according to the wind field configuration parameters, including: The physics engine of the simulation environment module simulates the force exerted by the wind field on the simulated UAV based on wind field configuration parameters including wind speed, wind direction and turbulence model parameters, and calculates the motion state of the simulated UAV based on the force.

[0091] Specifically, physics engines such as the ODE or Bullet engine built into the Gazebo simulator perform the following process after receiving the wind field configuration parameters: Based on the set wind speed and horizontal direction, and combined with the parameters of the selected turbulence model (such as the Dryden model), the physics engine calculates a three-dimensional dynamic wind vector in real time at each simulation step. This vector not only includes a stable average wind component but also superimposed random pulsating components generated by the turbulence model. The calculated forces are applied to the aerodynamic center defined by the simulated UAV model as a continuous external disturbance. Within this simulation step, the physics engine comprehensively calculates all forces and moments, including wind force, gravity, the UAV's own thrust, and aerodynamic forces. By solving the Newton-Euler dynamics equations, the six-degree-of-freedom motion state of the simulated UAV model is directly updated, including its position, attitude, linear velocity, and angular velocity. These updated state data constitute the UAV state data incorporating the effects of the wind field environment.

[0092] This invention directly converts user-configured wind speed, direction, and turbulence model parameters into forces conforming to aerodynamic principles. The physics engine then calculates the motion response in real-time based on rigid body dynamics, achieving a closed-loop causal chain from parameter configuration to dynamic behavior. This ensures that the simulated wind field disturbance is not merely a numerical disturbance, but rather induces dynamic responses in the UAV model, such as position drift and attitude swaying, consistent with real-world physical laws. This solves the problem of overly simplified or distorted environmental disturbance simulations in existing models.

[0093] The UAV's motion state, including wind disturbances, calculated by the physics engine, is output to the flight control simulation module in real time. This provides the flight control algorithm with a realistic feedback system that can be measured during flight in a real wind field. Therefore, the control responses (such as aileron deflection and motor speed adjustment) generated by the flight control algorithm within the simulation module to counteract wind disturbances are more realistic, ensuring the authenticity and dynamism of the input signals in the flight control loop throughout the simulation test loop. This lays the foundation for subsequent performance testing of the edge computing module on a dynamic platform. This implementation transforms the wind field environment from a static background description into a core variable that can dynamically, quantitatively, and physically accurately affect the entire test system. It not only enhances the credibility of the simulation environment itself, but more importantly, by providing a physically consistent dynamic disturbance source, it enables hardware-in-the-loop testing to effectively expose and verify the performance defects of the edge computing module in unsteady, disturbed real-world operating environments, significantly enhancing the engineering practical value and the persuasiveness of reliability verification.

[0094] In one embodiment, the fault control signal and the wind farm configuration parameters are saved as a test case template; when the test is executed, the test case template is called to configure the operating condition simulation module.

[0095] The fault control signals (such as a 30% packet loss rate for lidar) and the wind field configuration parameters (such as wind speed of 8 m / s, wind direction of 180°, and the Dryden turbulence model) can be jointly saved as a test case template. In practice, the operating condition simulation module provides a template management interface, allowing testers to name and save the currently configured operating condition parameters (fault type and parameters, wind field parameters) to a configuration file or database, for example, saving it as a template named "Lidar Anomaly Inspection under Crosswind Interference". When it is necessary to repeat the test under the same operating condition, testers or automated test scripts only need to call the template name, and the operating condition simulation module will automatically load the corresponding configuration file and generate and send the corresponding fault control signals and wind field configuration parameters, thus completing the reproduction and configuration of complex operating conditions with one click.

[0096] This invention achieves standardized, rapid reproducibility, and batch management of test scenarios by saving quantifiable, complex operating condition parameters as reusable test case templates. This solves the problems of inefficiency and error-proneness associated with repetitive manual configuration, significantly improving the efficiency and consistency of regression testing, comparative testing, and batch verification under extreme conditions. It makes it possible to systematically and scalably evaluate the performance of edge computing modules under different typical or extreme scenarios, enhancing the automation level and engineering practical value of the testing process.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hardware-in-the-loop simulation testing system for unmanned aerial vehicles (UAVs), characterized in that, include: The simulation environment module is used to simulate drones and sensors and run the physics engine; The operating condition simulation module includes a fault injection unit and a wind disturbance unit. The fault injection unit is communicatively connected to the sensor simulation module and is used to send fault control signals to the sensor simulation module. The wind disturbance unit is communicatively connected to the physics engine of the simulation environment module and is used to send wind field configuration parameters to the physics engine. The physics engine of the simulation environment module simulates the wind field environment according to the wind field configuration parameters, and generates and outputs UAV status data containing the effects of the wind field environment. The flight control simulation module is communicatively connected to the simulation environment module and is used to receive the UAV status data and generate flight control status data based on the UAV status data. The sensor simulation module, integrated into the simulation environment module, is used to generate sensor data with fault characteristics based on the fault control signal sent by the fault injection unit. The data interaction and protocol conversion module is communicatively connected to the flight control simulation module, the sensor simulation module and the edge computing module, respectively. It is used to receive the flight control status data and the sensor data with fault characteristics, and to convert the protocol and hardware interface format, and send the converted data to the edge computing module. The edge computing module is used to receive and process the converted data and generate control commands; The data interaction and protocol conversion module is further configured to receive control commands generated by the edge computing module, convert the control commands into protocols and interface formats, and send the converted control commands to the flight control simulation module. The flight control simulation module is configured to output flight control signals to the simulation environment module according to the converted control commands, and the flight control signals are used to drive the simulated UAV model in the simulation environment module.

2. The UAV hardware-in-the-loop simulation test system according to claim 1, characterized in that, The data interaction and protocol conversion module includes a flight control data interaction unit, a lidar data interaction unit, a camera data interaction unit, and a gimbal data interaction unit; The flight control data interaction unit is used to receive the flight control status data through the first communication interface, send it to the edge computing module through the first hardware interface, and receive the control commands through the first hardware interface and send them to the flight control simulation module through the second communication interface. The lidar data interaction unit is used to receive lidar point cloud data through the third communication interface and send it to the edge computing module through the second hardware interface. The camera data interaction unit is used to receive camera video stream data through the fourth communication interface and send it to the edge computing module through the third hardware interface. The gimbal data interaction unit is used to receive gimbal control commands through the fourth hardware interface and convert them into control signals executable by the simulated gimbal, and to receive feedback attitude data from the simulated gimbal and send it to the edge computing module through the fourth hardware interface.

3. The UAV hardware-in-the-loop simulation test system according to claim 2, characterized in that, The first and second communication interfaces are UDP ports, and the flight control status data and control commands are encapsulated using the MAVLink protocol; the first hardware interface is a USB to serial port.

4. The UAV hardware-in-the-loop simulation test system according to claim 1, characterized in that, The fault injection unit is configured to generate adjustable data packet loss instructions, and the sensor simulation module randomly discards data packets at a specified ratio when generating lidar point cloud data according to the data packet loss instructions.

5. The UAV hardware-in-the-loop simulation test system according to claim 1, characterized in that, The wind disturbance unit is configured to generate wind field configuration parameters that include wind speed, wind direction, and turbulence model parameters; the physics engine of the simulation environment module simulates the force exerted by the wind field on the simulated UAV based on the wind field configuration parameters, and calculates the motion state of the simulated UAV based on the force, and outputs UAV state data that includes the wind field disturbance effect.

6. The UAV hardware-in-the-loop simulation test system according to claim 1, characterized in that, The lidar simulation unit, camera simulation unit, and gimbal simulation unit in the sensor simulation module communicate through ROS nodes and achieve time-series synchronization of data generation by subscribing to and publishing ROS topics under the same time base.

7. The UAV hardware-in-the-loop simulation test system according to claim 1, characterized in that, The system also includes a visualization monitoring module, which is communicatively connected to the flight control simulation module to receive and display UAV status data, and communicatively connected to the edge computing module to receive and display data processing results.

8. A hardware-in-the-loop simulation test method for unmanned aerial vehicles (UAVs), characterized in that, Using the UAV hardware-in-the-loop simulation test system as described in any one of claims 1 to 7, the method comprises: The fault injection unit in the working condition simulation module sends a fault control signal to the sensor simulation module and sends wind field configuration parameters to the physics engine of the simulation environment module through the wind disturbance unit; the physics engine of the simulation environment module simulates the wind field environment according to the wind field configuration parameters, and generates and outputs UAV status data containing the effects of the wind field environment. The flight control simulation module receives the UAV status data and generates flight control status data based on the UAV status data; the sensor simulation module generates sensor data with fault characteristics based on the fault control signal. The data interaction and protocol conversion module receives the flight control status data and the sensor data with fault characteristics, performs protocol and hardware interface format conversion, and sends the converted data to the edge computing module. The edge computing module receives and processes the converted data and generates control commands; The data interaction and protocol conversion module receives control commands generated by the edge computing module, converts the control commands into protocols and interface formats, and sends the converted control commands to the flight control simulation module. The flight control simulation module outputs flight control signals to the simulation environment module according to the converted control commands. The flight control signals are used to drive the simulated UAV model in the simulation environment module.

9. The UAV hardware-in-the-loop simulation test method according to claim 8, characterized in that, The physics engine of the simulation environment module simulates the wind field environment based on the wind field configuration parameters, including: The physics engine of the simulation environment module simulates the force exerted by the wind field on the simulated UAV based on wind field configuration parameters including wind speed, wind direction and turbulence model parameters, and calculates the motion state of the simulated UAV based on the force.

10. The UAV hardware-in-the-loop simulation test method according to claim 8, characterized in that, The fault control signal and the wind farm configuration parameters are saved as a test case template; when the test is executed, the test case template is called to configure the operating condition simulation module.

Images