Simulation method and system for fusing unmanned aerial vehicle flight control and peripherals based on SITL

Abstract

The application provides a simulation method and system for unmanned aerial vehicle flight control and peripheral integration based on SITL, and the system comprises: an SITL simulation module deployed on a terminal, which is used for establishing an unmanned aerial vehicle model through mathematical modeling and serving as a virtual flight control of the unmanned aerial vehicle, and generating environment data by simulating a virtual environment and simulating the cooperative dynamic behavior of the unmanned aerial vehicle model and the peripheral; a signal conversion and interface module, which is used for establishing a physical interface of the SITL simulation module on the terminal and realizing the bidirectional interaction between the virtual simulation signal of the SITL simulation module and the physical signal of the peripheral; a debugging control module deployed on the terminal, which is used for configuring environment data and configuring virtual flight control parameters and route parameters in the virtual flight control to generate flight data; and a peripheral, which is used for receiving the virtual simulation signal and executing an action instruction and feeding back a virtual feedback signal of the executed action instruction. The application realizes the deep combination of the simulated simulation environment and the hardware interface, and guarantees the simulation debugging reliability of the unmanned aerial vehicle flight control and the peripheral.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, specifically to a simulation method and system for the integration of UAV flight control and peripherals based on SITL. Background Technology

[0002] The debugging of traditional UAV flight control systems and peripherals (gimbals, cameras, etc.) faces challenges such as high cost, high risk, limited application scenarios, and low efficiency. This has inspired researchers to use simulation methods to create a simulated environment for UAVs, and then conduct tests within that simulated environment.

[0003] Current simulation testing solutions typically include:

[0004] 1) SITL (Software-in-the-Loop Simulation): A technology that simulates the UAV flight control system and its operating environment using pure software, allowing verification of flight control algorithms, sensor models, and mission logic without the need for real hardware. SITL simulates the flight control hardware environment by running a compiled version of the flight control software on a computer, and provides a virtual UAV dynamics model and sensor data (IMU, GPS, camera, etc.). The flight control software generates control commands based on this data, forming a closed-loop test.

[0005] 2) HIL (Hardware-in-the-Loop Simulation): A testing and verification method for embedded systems. In UAV systems, HIL simulation can verify the interaction between hardware components and software systems, ensuring they work together effectively in actual flight. Through HIL simulation, developers can test UAV systems in a safer environment, ensuring their performance and safety, and accelerating product iteration and optimization.

[0006] In scenarios such as UAV aerial surveying and inspection, the collaborative performance of the flight control system and peripherals directly determines the operational accuracy and equipment reliability. However, SITL simulation, which is entirely software-based, cannot verify the compatibility of the flight control software with real hardware, the correctness of the drives, timing issues, and electrical characteristics, and cannot simulate hardware-level faults. HIL simulation, on the other hand, typically only connects to the core flight control hardware and cannot include all real airborne systems. It tests subsystems rather than the complete system, which cannot guarantee test reliability, and the maintenance costs are much higher than SITL. Summary of the Invention

[0007] To address the aforementioned technical challenges, this application proposes a simulation method and system for the integration of UAV flight control and peripherals based on SITL. By deeply integrating the simulation environment with hardware interfaces, it enables full-scenario, low-risk, and high-efficiency debugging of flight control and peripherals, providing technical support for optimizing the collaborative performance of UAVs in scenarios such as aerial surveying and inspection.

[0008] According to the first aspect of this application, a simulation system for the fusion of UAV flight control and peripherals based on SITL is proposed, comprising:

[0009] The SITL simulation module, deployed on the terminal, is used to establish a UAV model through mathematical modeling and use it as its virtual flight controller. It also generates environmental data by simulating a virtual environment, simulates the flight state of the UAV model under different working conditions based on the environmental data, generates flight data, and sends virtual simulation signals to the peripheral device based on the flight data to simulate the collaborative dynamic behavior of the UAV model and the peripheral device.

[0010] The signal conversion and interface module is used to establish a physical interface of the SITL simulation module on the terminal, convert the virtual simulation signal of the SITL simulation module into a physical signal that can be executed by the peripheral device, and convert the feedback signal of the peripheral device into a virtual feedback signal that can be executed by the SITL simulation module.

[0011] A debugging control module is deployed on the terminal and communicates with the SITL simulation module. It is used to configure the environmental data, configure virtual flight control parameters and flight path parameters in the virtual flight control, and generate the flight data. The flight path parameters include flight path data and the action commands of the peripheral device in the flight path.

[0012] The peripheral device is used to receive the virtual simulation signal and execute the action command, as well as to provide a virtual feedback signal for the execution of the action command.

[0013] Preferably, the peripherals include a gimbal and a camera, and the SITL simulation module includes:

[0014] The flight simulation unit is used to build the UAV model and serve as its virtual flight control, and to build a dynamic model using the Newton-Euler equations, and to calculate the attitude angles using the quaternion method, thereby providing the peripheral device with attitude angle data and the action commands.

[0015] The flight control parameter unit supports modification of the flight control interface protocol and communication parameters. The flight control interface protocol includes gimbal protocol and camera protocol, and the communication parameters include flight control interface baud rate and data transmission frequency.

[0016] The environmental parameter unit supports custom simulation of the environmental data, which includes wind speed, wind direction, and amplitude. The wind speed model is a superposition model of constant wind and gusts, and the gusts are modulated using periodic sine waves.

[0017] Preferably, the signal conversion and interface module includes:

[0018] The signal conversion unit is used to convert the virtual simulation signal output by the SITL simulation module into a physical signal and transmit it to the peripheral device via a USB to serial port module;

[0019] The peripheral signal acquisition unit is used to receive the feedback signal of the peripheral through the serial port, and transmit it back to the SITL simulation module through digital-to-analog conversion and protocol conversion;

[0020] The protocol adaptation unit is used to map and convert between the MAVLink protocol and the SBUS / RS232 / RS485 protocol through the protocol parsing library.

[0021] Preferably, the debugging control module includes:

[0022] The flight control parameter configuration unit is used to configure the flight control interface protocol and communication parameters of the virtual flight control.

[0023] An environmental parameter configuration unit is used to configure the environmental data;

[0024] The flight path parameter configuration unit is used to configure the flight path data and the action commands. The flight path data includes UAV flight actions, latitude, longitude and altitude of each location point in the custom flight path, and the expected attitude angle to reach each waypoint. The action commands include camera interval shooting commands and / or camera fixed-point shooting commands.

[0025] Preferably, the debugging control module further includes:

[0026] The fault injection unit is used to inject fault data into the UAV model to simulate abnormal operating conditions and to perform emergency verification of abnormal operating conditions, including:

[0027] When the virtual flight controller enters the RTK fixed state, the RTCM data transmission on the RTK module of the UAV model is cut off, and the UAV enters the RTK unlock state.

[0028] Verify whether the virtual flight controller performs RTK positioning hold function during RTK lockout, and verify the camera's image positioning accuracy during RTK positioning hold.

[0029] Preferably, the debugging control module further includes:

[0030] The parameter optimization unit is used to construct fitting functions based on the number of images captured by the camera under different communication parameters, and to obtain the optimal communication parameters through fitting calculations.

[0031] Preferably, the system further includes:

[0032] The data logging module is used to record end-to-end data in binary and text formats, including flight control logs, peripheral logs, and serial communication logs.

[0033] The flight control log includes the environmental data, the flight path data, and the number of shooting commands sent by the virtual flight control. The peripheral log includes the number of shooting commands received by the peripheral, the number of actual captured images, the time of executing the shooting commands, latitude and longitude, altitude, and attitude angle data. The serial communication log includes the data frames sent and received between the virtual flight control and the peripheral.

[0034] According to a second aspect of this application, a simulation method for the fusion of UAV flight control and peripherals based on SITL is proposed, which is executed in a simulation system for the fusion of UAV flight control and peripherals based on SITL as provided in any embodiment of the first aspect above. The method includes:

[0035] The simulation environment is set up by deploying the SITL simulation module on the terminal, establishing a UAV model as its virtual flight controller, connecting the terminal to peripheral hardware through a signal conversion and interface module, and configuring virtual flight control parameters in the virtual flight controller through a debugging and control module. The virtual flight control parameters include flight control interface protocol and communication parameters.

[0036] Scene and parameter configuration: The debugging control module configures environmental data and flight path parameters in the virtual flight controller. The environmental data includes wind speed, wind direction and amplitude, and the flight path parameters include flight path data and action commands.

[0037] During simulation, the SITL simulation module is started. The flight simulation unit simulates the virtual flight controller and sends attitude angle data and the action commands. The signal conversion and interface module converts these into physical signals to drive the peripheral device to execute the action commands. The feedback signal from the peripheral device is then converted into a virtual feedback signal by the signal conversion and interface module and sent back to the SITL simulation module, forming a closed-loop interaction.

[0038] Preferably, the method further includes:

[0039] Data recording and analysis: The data recording module records end-to-end data, and by analyzing data from flight control logs, peripheral logs, and serial communication logs, it identifies coordination issues between the virtual flight controller and the peripherals, including:

[0040] By comparing the number of shooting commands sent by the virtual flight controller in the flight control log with the number of shooting commands recorded in the serial communication log, it is determined whether the serial communication is normal. If not, the data sending frequency is increased.

[0041] By comparing the number of shooting commands recorded in the serial communication log with the number of actual images captured by the camera recorded in the peripheral log, it is determined whether the camera is executing normally. If not, the communication parameters are optimized.

[0042] Preferably, the method further includes:

[0043] Emergency verification under abnormal operating conditions involves injecting fault data into the UAV model through a fault injection unit to simulate abnormal operating conditions and performing emergency verification under abnormal operating conditions, including:

[0044] When the virtual flight controller enters the RTK fixed state, the RTCM data transmission on the RTK module of the UAV model is cut off, and the UAV enters the RTK unlock state.

[0045] Verify whether the virtual flight controller performs RTK positioning hold function during RTK lockout, and verify the shooting positioning accuracy of the camera during RTK positioning hold.

[0046] Preferably, the method further includes:

[0047] Communication parameter optimization involves constructing fitting functions based on the actual number of images captured by the camera under different communication parameters, and then calculating the optimal communication parameters through fitting.

[0048] This application proposes a simulation method and system for the integration of UAV flight control and peripherals based on SITL (System-In-Time) simulation. An SITL simulation module and a debugging control module are deployed at the terminal. The SITL simulation module acts as a virtual flight controller, while the debugging control module is used for virtual flight controller parameter configuration, virtual environment parameter configuration, fault data injection, and parameter optimization. A data recording module is used for end-to-end data recording and analysis. Then, a signal conversion and interface module connects the SITL simulation module to the peripherals, enabling bidirectional interaction between virtual simulation signals and physical signals. The debugging control module then continuously performs simulation debugging. This application achieves full-scenario, low-risk, and high-efficiency debugging of the flight controller and gimbal camera through deep integration of the simulation environment and hardware interface, providing technical support for the collaborative performance optimization of UAVs in scenarios such as aerial surveying and inspection. Attached Figure Description

[0049] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Elements in the drawings are not necessarily to scale. The same reference numerals refer to corresponding similar parts.

[0050] Figure 1This is a schematic diagram of a SITL-based simulation system for the integration of UAV flight control and peripherals according to a specific embodiment of this application;

[0051] Figure 2 This is an architecture diagram of a simulation system for the fusion of flight control and peripheral devices of a UAV based on SITL, according to a specific embodiment of this application;

[0052] Figure 3 This is a flowchart of a simulation method for the fusion of UAV flight control and peripherals based on SITL according to a specific embodiment of this application. Detailed Implementation

[0053] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the invention.

[0054] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0055] This application proposes a simulation system for the integration of UAV flight control and peripherals based on SITL. Figure 1 A schematic diagram of a simulation system for the fusion of UAV flight control and peripherals based on SITL according to a specific embodiment of this application is shown, as follows: Figure 1 As shown, the system includes:

[0056] SITL simulation module 20, deployed on terminal 10, is used to establish a UAV model through mathematical modeling and use it as its virtual flight control, as well as to generate environmental data by simulating a virtual environment, simulate the flight state of the UAV model under different working conditions based on the environmental data, generate flight data, and send virtual simulation signals to peripheral device 50 based on the flight data to simulate the collaborative dynamic behavior of the UAV model and peripheral device 50.

[0057] The signal conversion and interface module 30 is used to establish a physical interface of the SITL simulation module 20 on the terminal 10, convert the virtual simulation signal of the SITL simulation module 20 into a physical signal that can be executed by the peripheral device 50, and convert the feedback signal of the peripheral device 50 into a virtual feedback signal that can be executed by the SITL simulation module 20.

[0058] The debugging control module 40 is deployed on the terminal 10 and communicates with the SITL simulation module 20. It is used to configure environmental data, virtual flight control parameters and flight path parameters in the virtual flight control, and generate flight data. The flight path parameters include flight path data and action commands of the peripheral device 50 in the flight path.

[0059] Peripheral 50 is used to receive virtual simulation signals and execute action commands, as well as to provide virtual feedback signals for the execution of action commands.

[0060] The data logging module 60 is used to record end-to-end data in binary and text formats.

[0061] Figure 2 This paper illustrates the architecture of a simulation system for the fusion of flight control and peripherals of a UAV based on SITL, according to a specific embodiment of this application. Figure 2 The system shown is in Figure 1 Based on the system shown, the architecture of the SITL simulation module 20, signal conversion and interface module 30, debugging control module 40 and data recording module 60 are described in detail.

[0062] The following is about Figure 2 The simulation system for the integration of UAV flight control and peripherals based on SITL is described in detail.

[0063] First, it should be noted that in this embodiment, the peripheral device 50 includes a gimbal 51 and a camera 52, and the terminal 10 is a PC host. In other embodiments, the peripheral device can be selected according to the actual simulation requirements, such as a drone remote controller.

[0064] SITL Simulation Module 20, or Software-in-the-Loop Simulation Module, serves as the "virtual flight control" for the UAV. Through mathematical modeling and environmental simulation, it reproduces the dynamic behavior of the UAV, including:

[0065] The flight simulation unit 21 is used to build a model of the UAV and serve as its virtual flight controller. It uses the Newton-Euler equations to build a dynamic model and calculates the attitude angles using the quaternion method, thereby providing attitude angle data and action commands to the peripheral device 50.

[0066] The flight control parameter unit 22 supports modification of the flight control interface protocol and communication parameters. The flight control interface protocol includes gimbal protocol, camera protocol and proprietary protocols for specific products; the communication parameters include the flight control interface baud rate (such as 57600, 115200, etc.) and data transmission frequency (such as 1-50Hz), adapting to the communication needs of different peripherals.

[0067] Environmental parameter unit 23 supports custom simulation environmental data, including wind speed (e.g., 5 m / s), wind direction (0-360°), and amplitude (e.g., 2 m / s). The wind speed model is a superposition model of constant wind and gusts, and the gusts are modulated by a periodic sine wave (the amplitude is the corresponding value).

[0068] In this embodiment, the SITL simulation module 20 specifically adopts the Ardupilot SITL framework.

[0069] The signal conversion and interface module 30, acting as a "virtual-physical connection bridge," enables bidirectional interaction between virtual simulation signals at the software level and physical signals at the hardware level. It includes:

[0070] The signal conversion unit 31 is used to convert the virtual simulation signal output by the SITL simulation module 20 into a physical signal and transmit it to the peripheral device 50 via the USB to serial port module.

[0071] In this embodiment, the USB to serial port module uses the CH340 chip, which supports 3.5V / 5V level switching.

[0072] In one specific embodiment, the USB interface of terminal 10 and the CH340 chip of signal conversion and interface module 30 are connected by a USB cable, and then the interface on signal conversion and interface module 30 is connected to the peripheral device through a shielded cable, thereby enabling communication between SITL emulation module 20 on terminal 10 and peripheral device 50.

[0073] The peripheral signal acquisition unit 32 is used to receive the feedback signal of the peripheral through the serial port and transmit it back to the SITL simulation module through digital-to-analog conversion and protocol conversion.

[0074] In a specific embodiment, feedback signals from peripheral 50 are received via serial port (e.g., camera 52 shooting status: 0x01 indicates shooting completed, 0x00 indicates shooting failed; gimbal 51 angle: 16-bit binary two's complement, range ±90°), converted into digital signals via AD conversion (12-bit precision), and then converted into virtual feedback signals (compliant with MAVLink feedback protocol format) by the CH340 chip of signal conversion and interface module 30 before being sent back to SITL simulation module 20, forming a closed-loop debugging link of "instruction sending-execution-feedback".

[0075] Protocol adaptation unit 33 is used to implement the mapping and conversion between MAVLink protocol and SBUS / RS232 / RS485 protocol through protocol parsing library.

[0076] In one specific embodiment, the SITL simulation module 20 serves as a virtual flight controller, employing the MAVLink protocol, while the peripheral 50 can adopt protocols such as SBUS / RS232 / RS485.

[0077] In one specific embodiment, a protocol parsing library (such as mavlink-library) is used to implement the mapping between MAVLink and SBUS protocols (SBUS frame format: 11 data bits, 1 stop bit, no parity check; during parsing, the PWM values ​​of channels 1-16 are extracted and mapped to MAVLink instructions). At the same time, it also supports the protocol mapping of RS232 / RS485 custom protocols (the frame header, parity bit, and data field length can be defined through the configuration file), breaking down the compatibility barriers between flight control simulation signals and different peripheral interfaces, and improving the system's versatility.

[0078] The debugging control module 40, acting as the "debugging hub and data base," enables parameter configuration and virtual environment control. It is a software module running on terminal 10 (supporting Windows systems) and communicates with the SITL simulation module 20 via an internal UDP / TCP interface. It includes:

[0079] The flight control parameter configuration unit 41 is used to configure the flight control interface protocol and communication parameters of the virtual flight control.

[0080] In one specific embodiment, the communication protocol and communication parameters are configured in the flight control parameter configuration unit 41, for example:

[0081] Serial port baud rate SERIAL2_BAUD=115200;

[0082] Data transmission frequency SRx_POSITION=10Hz, step size 1Hz;

[0083] Serial port protocol SERIAL2_PROTOCOL=Mavlink;

[0084] PTZ protocol MNT1_TYPE=STRoM32 MAVLINK

[0085] Camera protocol CAM1_TYPE=MAVLinkCamV2.

[0086] The environment parameter configuration unit 42 is used to configure environment data.

[0087] In one specific embodiment, environmental data includes wind speed (e.g., parameter command SIM_WIND_SPD=5, 5m / s), wind direction (e.g., parameter command SIM_WIND_DIR=180, 180°), and amplitude (e.g., constant wind 2m / s or constant wind 2m / s superimposed with gusts). The gusts are modulated using a periodic sine wave, which can be set according to actual needs.

[0088] The route parameter configuration unit 43 is used to configure route data and action commands.

[0089] In one specific embodiment, the flight path data includes UAV flight actions (such as constant speed cruise at 15m / s), latitude, longitude and altitude of each location point in the custom flight path, and the expected attitude angles (pitch angle, roll angle, heading angle) to reach each waypoint. The action commands include camera interval shooting commands (such as camera 52 taking a picture every 5m) and / or camera fixed-point shooting commands (such as camera 52 triggering a picture at the third waypoint in the flight path).

[0090] The fault injection unit 44 is used to inject fault data into the UAV model to simulate abnormal working conditions and to perform emergency verification of abnormal working conditions, thereby verifying the system's fault tolerance capability and emergency response mechanism under abnormal working conditions.

[0091] In one specific embodiment, emergency verification for abnormal operating conditions includes:

[0092] 1) When the virtual flight controller enters the RTK fixed state (i.e., fixed solution, highest accuracy state), the RTCM data transmission on the RTK module of the UAV model is cut off (i.e., GPS signal is lost), and the RTK unlock state is entered.

[0093] 2) Verify whether the virtual flight controller performs RTK positioning maintenance during RTK lockout, and verify the shooting positioning accuracy of camera 52 during RTK positioning maintenance.

[0094] The parameter optimization unit 45 is used to construct fitting functions based on the number of images captured by the camera 52 under different communication parameters, and to obtain the optimal communication parameters through fitting calculation.

[0095] In one specific embodiment, based on multiple sets of test data (such as the number of images taken by camera 52 at different photo intervals under different serial port baud rates and different data transmission frequencies), the optimal parameter curves are obtained by fitting using the least squares method, and the best communication parameters are determined.

[0096] The data logging module 60 records end-to-end data in real time. Data analysis can quantify the collaborative performance of the flight controller and peripherals 50. This includes:

[0097] Flight control log 61 is used to record environmental data, flight path data, and the number of shooting commands sent by the virtual flight controller.

[0098] Peripheral log 62 is used to record the number of shooting commands received by peripheral 50, the number of actual images captured, the time of executing the shooting command, latitude and longitude, altitude and attitude angle data.

[0099] The serial communication log 63, acting as a black box, is used to record the data frames transmitted and received between the virtual flight controller and peripheral device 50.

[0100] In this embodiment, the data recording module 60 is a cache module on the terminal 10. In other embodiments, the data recording module 60 may also be an independent cache module.

[0101] In one specific embodiment, key information can be extracted from the logs of the data recording module 60. Taking the issue of missed shots by camera 52 as an example:

[0102] If the number of shooting commands in the flight control log 61 is greater than the number of shooting commands recorded in the serial communication log 63, adjustments can be made in the serial communication in a timely manner. The data transmission frequency can be increased through the flight control parameter configuration unit 41 (e.g., from 10Hz to 20Hz, modify SRx_POSITION=20), and the effect can be repeatedly tested and verified.

[0103] If the number of shooting commands recorded in the serial communication log 63 is greater than the number of pictures actually captured by the camera 52, then the camera 52 should be checked first for any omissions in the command conversion process.

[0104] In summary, the SITL-based simulation system for the integration of UAV flight control and peripherals provided in this application achieves the following beneficial effects:

[0105] A SITL simulation module 20 and a debugging control module 40 are deployed on terminal 10. The SITL simulation module 20 serves as a virtual flight controller, while the debugging control module 40 is used for configuring virtual flight controller parameters, flight path parameters, environmental data configuration for the virtual environment, fault data injection, and parameter optimization. A data recording module 60 is used for end-to-end data recording. Then, the SITL simulation module 20 is connected to the peripheral device 50 via a signal conversion and interface module 30 to achieve bidirectional interaction between virtual simulation signals and physical signals, simulating the collaborative dynamic behavior of the UAV model and the peripheral device 50. The debugging control module 40 continuously performs simulation debugging, and the data recording module 60 performs testing and analysis. This application achieves full-scenario, low-risk, and high-efficiency debugging of the flight controller and gimbal camera through deep integration of the simulation environment and hardware interface, providing technical support for the collaborative performance optimization of UAV aerial surveying, inspection, and other scenarios.

[0106] Based on the aforementioned SITL-based simulation system for the integration of UAV flight control and peripherals, and based on the same inventive concept, this application also proposes a SITL-based simulation method for the integration of UAV flight control and peripherals. Figure 3 A flowchart illustrating a simulation method for the fusion of UAV flight control and peripherals based on SITL according to an embodiment of this application is shown, as follows: Figure 3 As shown, the method includes:

[0107] Step S101: Simulation environment setup. Deploy the SITL simulation module on the terminal, establish a UAV model and use it as its virtual flight controller. Connect the terminal to the peripheral hardware through the signal conversion and interface module, and configure the virtual flight control parameters in the virtual flight controller through the debugging control module. The virtual flight control parameters include the flight control interface protocol and communication parameters.

[0108] In one specific embodiment, virtual flight control parameters are set through the flight control parameter configuration unit, such as:

[0109] Serial port baud rate SERIAL2_BAUD=115200;

[0110] Data transmission frequency SRx_POSITION=10Hz, step size 1Hz;

[0111] Serial port protocol SERIAL2_PROTOCOL=Mavlink;

[0112] PTZ protocol MNT1_TYPE=STRoM32 MAVLINK

[0113] Camera protocol CAM1_TYPE=MAVLinkCamV2.

[0114] Step S102: Scene and parameter configuration. Configure environmental data and flight path parameters in the virtual flight controller through the debugging control module. Environmental data includes wind speed, wind direction and amplitude. Flight path parameters include flight path data and action commands.

[0115] In a specific embodiment, simulated environmental parameters (such as wind speed of 5 m / s, wind direction of 180°, and gust amplitude of 2 m / s) are set through the environmental parameter configuration unit, and flight path parameters (such as constant speed cruise of 15 m / s, latitude and longitude (such as 4°N, 4°E), altitude (such as 100 m), and the flight path including 3 90° turns) and shooting instructions (such as triggering the camera to take a picture once every 5 m) are set through the flight path parameter configuration unit.

[0116] Step S103: Simulation run. Start the SITL simulation module. The flight simulation unit simulates as a virtual flight controller and sends attitude angle data and action commands. The data is converted into physical signals by the signal conversion and interface module to drive the peripheral device to execute the action commands. The feedback signal of the peripheral device is then converted into a virtual feedback signal by the signal conversion and interface module and sent back to the SITL simulation module, forming a closed-loop interaction.

[0117] In one specific embodiment, the flight dynamics simulation unit outputs real-time attitude angle data and action commands during operation. These are converted into TTL level signals by the CH340 chip in the signal conversion and interface module and transmitted to the gimbal mainboard. The gimbal mainboard parses the data and sends it to the camera, and executes the action commands (such as shooting commands). The camera's feedback signal is transmitted back to the signal conversion and interface module via the gimbal mainboard. It is then converted into a virtual feedback signal by the CH340 chip and input to the SITL simulation module, completing the closed-loop interaction.

[0118] Step S104: Data recording and analysis. Record the entire link data through the data recording module, and locate the coordination problem between the virtual flight controller and peripherals by analyzing the data in the flight control log, peripheral log and serial communication log.

[0119] Specifically, step S104 includes:

[0120] By comparing the number of shooting commands sent by the virtual flight controller in the flight control log with the number of shooting commands recorded in the serial communication log, it can be determined whether the serial communication is normal. If not, the data sending frequency is increased. By comparing the number of shooting commands recorded in the serial communication log with the number of actual images captured by the camera recorded in the peripheral log, it can be determined whether the camera is performing normally. If not, the communication parameters are optimized.

[0121] In a specific embodiment, if the number of shooting commands recorded in the flight control log is greater than the number of shooting commands recorded in the serial communication log, it is considered that the data transmission frequency of the virtual flight control is insufficient. At this time, adjustments can be made in the serial communication in a timely manner, and the data transmission frequency can be increased through the flight control parameter configuration unit (e.g., from 10Hz to 20Hz, by modifying SRx_POSITION=20).

[0122] In one specific embodiment, if the number of shooting commands recorded in the serial communication log is greater than the number of images actually captured by the camera, it is considered that the camera may have missed some images. The first priority should be to check whether there are any omissions in the camera's command conversion process, optimize the camera's internal algorithm code, and reduce the number of missed images.

[0123] Step S105: Emergency verification of abnormal working conditions. Fault data is injected into the UAV model through the fault injection unit to simulate abnormal working conditions and to conduct emergency verification of abnormal working conditions.

[0124] In one specific embodiment, step S105 specifically includes:

[0125] When the virtual flight controller enters the RTK fixed state, the RTCM data transmission on the RTK module of the UAV model is cut off (i.e., the GPS signal is simulated to be lost), and the RTK unlock state is entered; verify whether the virtual flight controller performs the RTK positioning maintenance function during the RTK unlock period, and verify the shooting and positioning accuracy of the camera during the RTK positioning maintenance period.

[0126] Step S106: Optimize communication parameters. Based on the actual number of images captured by the camera under different communication parameters, construct fitting functions respectively, and obtain the optimal communication parameters through fitting calculation.

[0127] In one specific embodiment, step S106 specifically includes:

[0128] Based on multiple sets of test data (such as the number of images taken by the camera at different serial port baud rates and different data transmission frequencies at different shooting intervals), the optimal parameter curves are obtained by fitting using the least squares method, and the best communication parameters are determined.

[0129] In the embodiments of this application, it should be understood that the disclosed technical content can be implemented in other ways. The device / system / method embodiments described above are merely illustrative. For example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0133] It is evident that those skilled in the art can make various modifications and alterations to the embodiments of the present invention without departing from the spirit and scope of the invention. In this way, the invention is also intended to cover such modifications and alterations if they fall within the scope of the claims and their equivalents. The word "comprising" does not exclude the presence of other elements or steps not listed in the claims. The simple fact that certain measures are described in mutually different dependent claims does not indicate that a combination of these measures cannot be used for profit. Any reference numerals in the claims should not be considered as limiting the scope.

Claims

1. A simulation system for the integration of UAV flight control and peripherals based on SITL, characterized in that, include: The SITL simulation module, deployed on the terminal, is used to establish a UAV model through mathematical modeling and use it as its virtual flight controller. It also generates environmental data by simulating a virtual environment, simulates the flight state of the UAV model under different working conditions based on the environmental data, generates flight data, and sends virtual simulation signals to the peripheral device based on the flight data to simulate the collaborative dynamic behavior of the UAV model and the peripheral device. The signal conversion and interface module is used to establish a physical interface of the SITL simulation module on the terminal, convert the virtual simulation signal of the SITL simulation module into a physical signal that can be executed by the peripheral device, and convert the feedback signal of the peripheral device into a virtual feedback signal that can be executed by the SITL simulation module. A debugging control module is deployed on the terminal and communicates with the SITL simulation module. It is used to configure the environmental data, configure virtual flight control parameters and flight path parameters in the virtual flight control, and generate the flight data. The flight path parameters include flight path data and the action commands of the peripheral device in the flight path. The peripheral device is used to receive the virtual simulation signal and execute the action command, as well as to provide a virtual feedback signal for executing the action command. The peripherals include a gimbal and a camera, and the SITL simulation module includes: The flight simulation unit is used to build the UAV model and serve as its virtual flight control, and to build a dynamic model using the Newton-Euler equations, and to calculate the attitude angles using the quaternion method, thereby providing the peripheral device with attitude angle data and the action commands. The flight control parameter unit supports modification of the flight control interface protocol and communication parameters. The flight control interface protocol includes gimbal protocol and camera protocol, and the communication parameters include flight control interface baud rate and data transmission frequency. The environmental parameter unit supports custom simulation of the environmental data, which includes wind speed, wind direction, and amplitude. The wind speed model is a superposition model of constant wind and gusts, and the gusts are modulated using periodic sine waves.

2. The simulation system for the fusion of UAV flight control and peripherals based on SITL as described in claim 1, characterized in that, The signal conversion and interface module includes: The signal conversion unit is used to convert the virtual simulation signal output by the SITL simulation module into a physical signal and transmit it to the peripheral device via a USB to serial port module; The peripheral signal acquisition unit is used to receive the feedback signal of the peripheral through the serial port, and transmit it back to the SITL simulation module through digital-to-analog conversion and protocol conversion; The protocol adaptation unit is used to map and convert between the MAVLink protocol and the SBUS / RS232 / RS485 protocol through the protocol parsing library.

3. The simulation system for the fusion of UAV flight control and peripherals based on SITL as described in claim 1, characterized in that, The debugging control module includes: The flight control parameter configuration unit is used to configure the flight control interface protocol and communication parameters of the virtual flight control. An environmental parameter configuration unit is used to configure the environmental data; The flight path parameter configuration unit is used to configure the flight path data and the action commands. The flight path data includes UAV flight actions, latitude, longitude and altitude of each location point in the custom flight path, and the expected attitude angle to reach each waypoint. The action commands include camera interval shooting commands and / or camera fixed-point shooting commands.

4. The simulation system for the fusion of UAV flight control and peripherals based on SITL as described in claim 1, characterized in that, The debugging control module also includes: The fault injection unit is used to inject fault data into the UAV model to simulate abnormal operating conditions and to perform emergency verification of abnormal operating conditions, including: When the virtual flight controller enters the RTK fixed state, the RTCM data transmission on the RTK module of the UAV model is cut off, and the UAV enters the RTK unlock state. Verify whether the virtual flight controller performs RTK positioning hold function during RTK lockout, and verify the camera's image positioning accuracy during RTK positioning hold.

5. The simulation system for the fusion of UAV flight control and peripherals based on SITL as described in claim 1, characterized in that, The debugging control module also includes: The parameter optimization unit is used to construct fitting functions based on the number of images captured by the camera under different communication parameters, and to obtain the optimal communication parameters through fitting calculations.

6. The simulation system for the fusion of UAV flight control and peripherals based on SITL according to claim 3, characterized in that, The system also includes: The data logging module is used to record end-to-end data in binary and text formats, including flight control logs, peripheral logs, and serial communication logs. The flight control log includes the environmental data, the flight path data, and the number of shooting commands sent by the virtual flight control. The peripheral log includes the number of shooting commands received by the peripheral, the number of actual captured images, the time of executing the shooting commands, latitude and longitude, altitude, and attitude angle data. The serial communication log includes the data frames sent and received between the virtual flight control and the peripheral.

7. A simulation method for the fusion of UAV flight control and peripherals based on SITL, characterized in that, The method, executed in the SITL-based UAV flight control and peripheral fusion simulation system as described in any one of claims 1-6, comprises: The simulation environment is set up by deploying the SITL simulation module on the terminal, establishing a UAV model as its virtual flight controller, connecting the terminal to peripheral hardware through a signal conversion and interface module, and configuring virtual flight control parameters in the virtual flight controller through a debugging and control module. The virtual flight control parameters include flight control interface protocol and communication parameters. Scene and parameter configuration: The debugging control module configures environmental data and flight path parameters in the virtual flight controller. The environmental data includes wind speed, wind direction and amplitude, and the flight path parameters include flight path data and action commands. During simulation, the SITL simulation module is started. The flight simulation unit simulates the virtual flight controller and sends attitude angle data and the action commands. The signal conversion and interface module converts these into physical signals to drive the peripheral device to execute the action commands. The feedback signal from the peripheral device is then converted into a virtual feedback signal by the signal conversion and interface module and sent back to the SITL simulation module, forming a closed-loop interaction.

8. The simulation method for the fusion of UAV flight control and peripherals based on SITL according to claim 7, characterized in that, The method further includes: Data recording and analysis: The data recording module records end-to-end data, and by analyzing data from flight control logs, peripheral logs, and serial communication logs, it identifies coordination issues between the virtual flight controller and the peripherals, including: By comparing the number of shooting commands sent by the virtual flight controller in the flight control log with the number of shooting commands recorded in the serial communication log, it is determined whether the serial communication is normal. If not, the data sending frequency is increased. By comparing the number of shooting commands recorded in the serial communication log with the number of actual images captured by the camera recorded in the peripheral log, it is determined whether the camera is executing normally. If not, the communication parameters are optimized.

9. The simulation method for the fusion of UAV flight control and peripherals based on SITL according to claim 8, characterized in that, The method further includes: Emergency verification under abnormal operating conditions involves injecting fault data into the UAV model through a fault injection unit to simulate abnormal operating conditions and performing emergency verification under abnormal operating conditions, including: When the virtual flight controller enters the RTK fixed state, the RTCM data transmission on the RTK module of the UAV model is cut off, and the UAV enters the RTK unlock state. Verify whether the virtual flight controller performs RTK positioning hold function during RTK lockout, and verify the shooting positioning accuracy of the camera during RTK positioning hold.

10. The simulation method for the fusion of UAV flight control and peripherals based on SITL according to claim 7, characterized in that, The method further includes: Communication parameter optimization involves constructing fitting functions based on the actual number of images captured by the camera under different communication parameters, and then calculating the optimal communication parameters through fitting.

Images