A peripheral simulation system and method for simulating an unmanned aerial vehicle

Abstract

The present invention relates to the field of UAV simulation technology, and discloses a peripheral simulation system and method for simulating a UAV. In this system, hardware sensors and actuators are replaced by an interpreter to achieve data exchange between the simulator and the firmware. The present invention solves the problems existing in the prior art, such as the difficulty in conveniently realizing the interaction between UAV firmware and a flight simulator, etc.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) simulation, and particularly to a peripheral simulation system and method for simulating a UAV. Background Art

[0002] In order to simulate the flight state of a UAV, most of the existing methods rely on modifying the source code of the flight control program. Specifically, the interaction interfaces of sensors and actuators are replaced with the corresponding interfaces of a flight simulator. The modified code is further recompiled into an adaptive control program. Figure 3 Shows this process. The original firmware calls sensors and actuators to manage a real UAV ( Figure 3 a), while the adaptive firmware calls application programming interfaces to manage a virtual UAV in a flight simulator ( Figure 3 b). Although these simulation methods have been successfully applied to simulate UAV flight, such as Ardupilot and PX4, their limitations are also simple: in order to adapt to the designed simulator, researchers need to manually analyze the source code of the control program and modify the interaction interface. Unfortunately, only a limited number of manufacturers provide the control program code.

[0003] The prior art has the following disadvantages:

[0004] By modifying the source code of the flight control program and compiling it into an adaptive firmware, the adaptive firmware calls application programming interfaces to manage a virtual UAV in a flight simulator. Although these simulation methods have been successfully applied to simulate UAV flight, such as Ardupilot and PX4, their limitations are also simple: in order to adapt to the designed simulator, researchers need to manually analyze the source code of the control program and modify the interaction interface. Unfortunately, only a limited number of manufacturers provide the control program code.

[0005] The simulated flight state is inaccurate. Due to the interface modification, the original code for processing the raw sensor data is disrupted, and the data processed by the modified adaptive firmware is actually different from the actual situation, without considering physical effects during simulation.

[0006] The simulator only specifies UAVs of a specific brand. Due to the inconsistent development of UAVs, manufacturers generally design and implement different flight control schemes according to different principles. Therefore, the data interaction interfaces of sensors and actuators may be different.

[0007] According to the requirements of existing simulators, researchers need to analyze each brand of UAV, determine what flight control program to use, and further make interface changes to match the interface of the simulator. This manual inspection is not scalable and time-consuming. Summary of the Invention

[0008] To overcome the deficiencies of the prior art, the present invention provides a peripheral simulation system and method for simulating an unmanned aerial vehicle (UAV), which solves the problems in the prior art such as the difficulty in conveniently realizing the interaction between UAV firmware and a flight simulator.

[0009] The technical solution adopted by the present invention to solve the above problems is as follows:

[0010] A peripheral simulation system for simulating a UAV replaces hardware sensors and actuators with an interpreter to achieve data exchange between a simulator and firmware.

[0011] As a preferred technical solution, a simulator is used to execute firmware and hijack the firmware to access on-chip peripherals. The interpreter processes the firmware's access to on-chip peripherals to simulate sensors or actuators, and the firmware accesses, interprets, and converts simulation-related data. Among them, a single interpreter is semi-automatically constructed according to the data sheet released by the supplier and is responsible for processing the firmware that accesses a single model of sensor or actuator.

[0012] As a preferred technical solution, it includes a sensor identifier, a simulator generator, and a UAV simulator that are electrically connected in sequence, and also includes an actuator identifier electrically connected to the simulator generator;

[0013] Among them, the sensor identifier is used to: identify the sensors on the UAV microcontroller unit; the simulator generator is used to: semi-automatically generate and allocate simulators for additional sensors and actuators; the UAV simulator is used to: execute UAV firmware and simulate UAV flight; the actuator identifier is located on the UAV microcontroller unit and is used to: identify actuators.

[0014] A peripheral simulation method for simulating a UAV, based on the above-mentioned peripheral simulation system for simulating a UAV, replaces hardware sensors and actuators with an interpreter to achieve data exchange between a simulator and firmware.

[0015] As a preferred technical solution, first, identify the models of all sensors and actuators connected to the UAV; then allocate simulators for these sensors and actuators, and the simulator provides the function of converting data; finally, use the allocated simulators to apply flight simulation.

[0016] A peripheral simulation method for simulating a UAV adopts the following peripheral simulation system for simulating a UAV: a sensor identifier, a simulator generator, and a UAV simulator that are electrically connected in sequence, and also includes an actuator identifier electrically connected to the simulator generator;

[0017] Among them, a sensor identifier is used to: identify sensors on the drone microcontroller unit; an emulator generator is used to: semi-automatically generate and allocate emulators for additional sensors and actuators; a drone simulator is used to: execute drone firmware and simulate drone flight; an actuator identifier is located on the drone microcontroller unit and is used to: identify actuators.

[0018] As a preferred technical solution, when performing peripheral simulation, the following steps are included:

[0019] A1, The sensor identifier identifies the model of the sensor, constructs the database record function of the sensor according to the sensor data sheet, and extracts the features of the sensor through dynamic analysis of the firmware; then compares the features with the database to obtain the model of the attached sensor;

[0020] A2, The actuator identifier obtains information about the attached actuator by capturing and evaluating the waveform of the control signal;

[0021] A3, The emulator generator semi-automatically generates or allocates emulators for sensors and actuators by using the information obtained by the sensor identifier and the actuator identifier;

[0022] A4, The drone simulator executes the firmware and simulates drone flight, and uses the sensor emulator and / or actuator emulator allocated by the emulator generator to transfer data between the firmware emulator and the flight emulator, realizing flight simulation on the drone firmware.

[0023] As a preferred technical solution, in step A2, the actuator identifier uses the firmware emulator to execute the firmware and captures the waveform by hijacking the interaction between the firmware and the PWM; and, the actuator identifier distinguishes the protocol by analyzing the waveform in the time domain characteristics: for a digital ESC, the high-level occupancy time of each pulse jump between two values is fixed; for an analog ESC, the time occupancy of the high level of each pulse is not fixed.

[0024] As a preferred technical solution, in step A3, after identifying the sensors and actuators, the emulator generator obtains the information of the attached sensors and actuators, and then allocates emulator modules; among them, the emulator modules allocated by the emulator generator are reusable.

[0025] As a preferred technical solution, in step A4, the drone simulator realizes a simulation loop for transmitting signals sequentially along the firmware emulator, actuator emulator, flight emulator, sensor emulator, and firmware emulator.

[0026] The present invention has the following beneficial effects compared with the prior art:

[0027] (1) The DVATAR of the present invention can successfully identify the sensors and actuators on the unmanned aerial vehicle. DVATAR has the ability to simulate the sensors and actuators of the unmanned aerial vehicle, and convert simulation-related data between flight simulation and firmware; the present invention solves the problem of incompatibility between the firmware and the flight simulator interface;

[0028] (2) The original design of the firmware of the present invention is to operate the peripherals on the chip, rather than the flight simulator, resulting in interface incompatibility. Since the firmware cannot be modified in the near-source environment, it is difficult to meet the consistency requirements of the interface; in order to transmit data, an interpreter is needed to coordinate the inconsistency; on the one hand, the interpreter needs to simulate the logic of real sensors and actuators to provide a communication environment for the correct execution of the firmware; on the other hand, the interpreter acts as an intermediary to handle the data transmission between the firmware simulator and the flight simulator;

[0029] (3) The present invention solves the problems of insufficient common information of the unmanned aerial vehicle sensors and actuators, difficult generation of parsers, and large manpower consumption due to various types of parsers. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the software and hardware stack of the unmanned aerial vehicle;

[0031] Figure 2 It is a schematic diagram of the flight simulation process;

[0032] Figure 3 It is a conceptual diagram of the flight simulation application;

[0033] Figure 4 It is a schematic diagram for the conceptual description of the method of the present invention;

[0034] Figure 5 It is a schematic diagram of the structure of Dvatar of the present invention;

[0035] Figure 6 It is a schematic diagram of the execution path affected by the data register (DR), identity register (IR) and status register (SR);

[0036] Figure 7 It is a conceptual diagram of the sensor emulator and actuator emulator;

[0037] Figure 8 It is a schematic diagram of the reliability evaluation experiment;

[0038] Figure 9 It is a schematic diagram of the overall process of Dvatar. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0040] Embodiment 1

[0041] As Figures 1 to 9 shown, the present invention aims to solve the problem of interaction between the drone firmware and the flight simulator, enabling the simulated drone to fly in a virtual three-dimensional space, and observing the anomalies of the drone in the physical space when vulnerabilities are triggered in the virtual space, so as to capture the vulnerabilities in the physical space during the execution of the firmware. To solve similar problems, the present invention proposes a technology called off-chip peripheral emulation technology, adding a layer of proxy between the drone firmware and the flight simulator for data processing, that is, implementing a parser to simulate the sensors and actuators on the drone firmware.

[0042] The overall problem is to solve the interaction between the drone firmware and the flight simulator. In the process of solving this major problem, some other technical problems need to be solved, such as the incompatibility problem between the drone firmware and the flight simulator interface; the lack of common information of the drone sensors and actuators, the difficulty in generating parsers and the large variety of parser types, resulting in high manpower consumption.

[0043] (1) Achieve the interaction between the drone firmware and the flight simulator

[0044] The main idea is to simulate the sensors and actuators on the drone, replacing the hardware sensors and actuators with interpreters, namely sensor simulators and actuator simulators, to help with data exchange between the firmware and the flight simulator.

[0045] (2) Address the incompatibility problem between the drone firmware and the flight simulator interface

[0046] The initial design of the firmware is to operate the peripherals on the chip rather than the flight simulator, resulting in interface incompatibility. Since the firmware cannot be modified in the near-source environment, it is difficult to meet the consistency requirements of the interface. To transmit data, an interpreter is needed to coordinate the inconsistency. On the one hand, the interpreter needs to simulate the logic of real sensors and actuators to provide a communication environment for the correct execution of the firmware; on the other hand, the interpreter acts as an intermediary to handle the data transmission between the firmware simulator and the flight simulator.

[0047] (3) Address the problem of insufficient common information of the drone sensors and actuators, the difficulty in generating parsers and the large variety of parser types resulting in high manpower consumption

[0048] Applying an interpreter requires information about the real hardware because the interpreter simulates the logic of sensors and actuators. However, drone suppliers rarely disclose the exact models of the sensors and actuators of their products. The lack of this information hinders analysts from obtaining the model information of the sensors and actuators, resulting in their inability to apply the interpreter. Since drones may use sensors or actuators from a wide range of suppliers, it is labor-intensive to build parsers for different models of sensors and actuators.

[0049] The present invention proposes a tool to automatically analyze firmware and then extract information about sensors and actuators. Since different drones may use the same model of sensors or drivers, a single interpreter can be reused to assist different firmwares in exchanging data with a flight simulator. To help new firmwares interact with the flight simulator, analysts only need to assemble different interpreters. This reusability and composability of interpreters address the challenge of diversity, and the present invention proposes a framework for semi-automatically constructing interpreters, thereby reducing manpower.

[0050] 1. Objectives achieved by the present invention:

[0051] The present invention addresses the need for interaction between drone firmware and a flight simulator in a scalable, non-disruptive, and source-code-independent manner. It is a novel and extensible simulation-based technology that converts data between drone firmware and a flight simulator by constructing interpreters, thereby making flight simulation applicable to binary drone firmware and assisting flight simulation.

[0052] The construction of interpreters solves the problem of incompatible interfaces. The present invention proposes a tool to automatically analyze firmware and then extract information about sensors and actuators, addressing the problem of insufficient common information. The reusability and composability of interpreters address the challenge of diversity, and the present invention proposes a framework for semi-automatically constructing interpreters, thereby reducing manpower. By implementing and integrating a simulation-based technology, a prototype system called DVATAR is created.

[0053] 2. Introduction to relevant background knowledge:

[0054] It should be noted that, to more clearly introduce the present invention, in combination with some background knowledge in the art, the present invention is described as follows:

[0055] Introduce the technical background and describe the existing implementation solutions that are closest to the present invention.

[0056] 2.1 Background knowledge:

[0057] 2.1.1 Software and hardware

[0058] The software and hardware stack of a drone is as Figure 1 shown. The hardware includes a microcontroller unit (MCU) and off-chip peripherals. The MCU is a system-on-chip (Soc), usually based on an ARM processor (e.g., the STM32 series).

[0059] In addition to the processor, there are internal on-chip peripherals (e.g., I2C, SPI, PWM), which assist the processor in communicating with off-chip peripherals such as sensors and actuators. The processor invokes sensors and actuators through the intermediate assistance of on-chip peripherals. The software is a control program, compiled into binary firmware, and stored inside the MCU for the processor to execute. Using the hardware, the control program executes logic to manage the flight of the drone.

[0060] This invention outlines how the control program invokes sensors and actuators to manage the flight of the drone: First, the firmware manipulates SPI or I2C to invoke sensors (e.g., accelerometer, gyroscope) to measure data on environmental factors. Then, the firmware fuses this data to estimate the state of the drone. Using the estimate, the firmware determines the response to the current state. Finally, the firmware invokes the actuator to achieve the response (i.e., control the speed of the motor) by manipulating PWM. The firmware loops through these three steps and continuously controls the drone to move in the air.

[0061] 2.1.2. Flight Simulator

[0062] Flight simulators such as Airsim or Gazebo are tools for simulating the flight of a drone in a virtual three-dimensional (3D) space. Similar to a computer game that constructs a 3D space where players control a virtual character to run or jump in the space, a flight simulator provides an interface for the control program to interact with the virtual drone in the 3D space.

[0063] There are two important interfaces to support the control program for controlling the virtual drone: (1) one for controlling the actuators of the virtual drone, and (2) the other for obtaining the state data of the virtual drone (e.g., acceleration or angular velocity). An illustration of the flight simulation is as Figure 2 shown, which includes the control program, the flight simulator, and their data interaction. Specifically, the control program and the flight simulator execute in parallel loops, performing the following processes: (1) The control program obtains the state of the virtual drone from the flight simulator. (2) The control program calculates the state of the drone, determines the response, and then generates a control strategy to achieve the response. (3) The control program uploads the control strategy to the flight simulator, and then the flight simulator simulates the movement of the drone according to the control strategy.

[0064] 2.1.3. Firmware Simulator

[0065] The firmware emulator provides an execution environment and a monitor for dynamically testing firmware. Since the firmware operates on on-chip peripherals, the firmware emulator provides an emulation of the on-chip peripherals to meet this requirement. Recent firmware emulation techniques, such as P2IM[1] and Halucinator[2], emulate the on-chip peripherals and provide an interface for the firmware to read / write data. Generally speaking, the latest firmware emulators can execute the firmware in a virtualized environment and provide an interface to hijack the firmware to read / write data to off-chip peripherals. 2.2, Technical Background

[0067] Nowadays, drones are widely used in various scenarios (e.g., surveillance, cargo transportation); therefore, their reliability and security have become the main concerns of users and operators. To evaluate the security of drone embedded systems, dynamic testing using software vulnerabilities (such as memory corruption, stack overflow) has been proposed. However, when verifying the correctness of the analysis results, it is inappropriate or even infeasible to conduct exploitation attacks on real drone devices because some vulnerabilities may cause serious deviations and even lead to drone crashes.

[0068] To extend the scope of dynamic testing to embedded systems (e.g., STM32-based systems, including drones), researchers have proposed a simulation-based method called rehosting, also known as firmware emulation ([1], [2]). By simulating the execution environment, the firmware of the target system is executed in a simulated environment (e.g., QEMU). The simulation environment provides the ability to work in collaboration with existing dynamic testing tools (e.g., AFL).

[0069] Recent research has proposed methods to explore physical space vulnerabilities ([3]), which utilize a technique called flight simulation. By simulating the execution of the firmware and simulating the flight of the drone in a virtual 3D physical space, analysts can capture anomalies of the drone in the virtual 3D physical space and then report the vulnerabilities. However, this technique is only applicable to a few open-source drones because flight simulation requires modifying the drone firmware to adapt to the interface of the flight simulator. For most off-the-shelf drones or open-source drones, flight simulation is not applicable, which hinders analysts from exploring physical space vulnerabilities.

[0070] Generally speaking, flight simulation utilizes tools called flight simulators (as Figure 2 shown). The firmware executes in cooperation with the flight simulator and exchanges basic data. By continuously exchanging these data, the flight simulator simulates the movement of the drone in a virtual three-dimensional physical space. This process requires the firmware to adapt to the interface provided by the flight simulator. Incompatible interfaces prevent the firmware from directly exchanging data with the flight simulator.

[0071] Previous research on firmware rehosting [1] proposed a tool firmware emulator (see 2.1.3) to execute and hijack the firmware that accesses on-chip peripherals. Based on this feature, analysts can create virtual on-chip peripherals such as I2C, SPI, or PWM to interact with the firmware. The sensors and actuators of the drone are off-chip peripherals that communicate with the firmware using off-chip peripherals (see Figure 1 ). All data exchanged between the firmware and the sensors or actuators crosses the off-chip peripherals. This data contains status and control data, but the format and meaning of the data are unknown without knowledge of the sensors or actuators. If the present invention can understand this data and interpret it to adapt to the flight simulator, data exchange will be feasible. The present invention designs a novel and scalable method to enable data exchange between the firmware and the flight simulator.

[0072] 3. Hereinafter, the technical solution of the present invention will be elaborated in detail with reference to the accompanying drawings.

[0073] I. Design Method

[0074] The method of the present invention solves the need for interaction between the drone firmware and the flight simulator in a scalable, non-intrusive, and source-code-independent manner, called DVATAR.

[0075] Different from existing code-modification-based methods that achieve interaction by modifying and recompiling the source code. DVATAR constructs an interpreter to help convert data between the firmware and the flight simulator. Figure 4 shows a conceptual illustration. DVATAR uses the firmware emulator proposed by firmware rehosting technologies such as P2IM [1] or Halucinator [2] to execute the firmware and hijack the firmware's access to on-chip peripherals. The interpreter processes the firmware's access to on-chip peripherals to simulate sensors or actuators, and the firmware accesses, interprets, and converts simulation-related data (control and status data). A separate interpreter is semi-automatically constructed based on the data sheets released by the vendor and is responsible for processing the firmware that accesses a single model of sensor or actuator. Since different drones may use the same model of sensor or actuator, a single interpreter can be reused to assist different firmware in exchanging data with the flight simulator. To help new firmware interact with the flight simulator, analysts only need to assemble different interpreters.

[0076] The present invention refers to the interpreter as a sensor simulator or an actuator simulator, depending on whether the simulator interprets sensors or actuators. The proposal of the interpreter solves the challenge of incompatible interfaces. To address the problem of insufficient common information, the present invention proposes a tool to automatically analyze the firmware and then extract information about sensors and actuators. The reusability and composability of the interpreter solve the challenge of diversity, and the present invention proposes a framework for semi-automatically constructing interpreters, thereby reducing the manpower.

[0077] II. Design Scheme

[0078] To achieve the flight simulation goal on the unmodified binary drone firmware, the system of the present invention first identifies the additional sensors and actuators of the drone (Sections C and D). Then, it generates emulators for the sensors and actuators (Section E) to guide the data. Finally, the present invention uses the sensor emulator and actuator emulator to connect the firmware emulator and the flight emulator.

[0079] DVATAR( Figure 5 ) consists of four components. The Sensor Identifier is a module used to identify the sensors on the drone microcontroller unit (MCU). The Actuator Identifier is a module used to identify the actuators on the drone MCU. The Simulator Generator is a module used to semi-automatically generate and allocate emulators for the additional sensors and actuators. The Drone Simulation is a module that executes the drone firmware and simulates the drone flight.

[0080] The entire process of DVATAR is as follows: The Sensor Identifier identifies the model of the sensor, which requires constructing the database record function of the sensor according to the Sensor Datasheets (① generating the Sensors Feature Database, ② generating the Identifier). By dynamically analyzing the Firmware, it extracts the features of the sensor (③ Sensors Feature Extraction, ④ Sensors Feature), and then compares the features with the database (⑤⑥) to obtain the model of the Attached Sensors. The Actuator Identifier obtains the information of the attached actuators by capturing and evaluating the waveform of the control signal (⑦ Actuators Feature Extraction, ⑧ obtaining the Actuators Feature, ⑨ obtaining the model of the Attached Actuators). After that, the Simulator Generator semi-automatically generates or allocates emulators for the sensors and actuators using the information obtained by the Sensor Identifier and Actuator Identifier (⑩ generating the emulator and allocating the simulation, Sensors Emulator Allocation, Generate Actuators Emulator Allocation. Finally, the UAV simulator executes the firmware and simulates the UAV flight, using the simulator generator ( Generate Sensors Emulator Module, Generate the intermediate helper of the sensors emulator and actuators emulator allocated by Actuators Emulator Module to transfer data between the Firmware Simulator and the Flight Simulator, and perform the simulation loop to achieve the goal of applying flight simulation to the binary UAV firmware. Between, and perform a simulation loop to achieve the goal of applying flight simulation to the binary UAV firmware.

[0081] B. Prerequisites

[0082] The system of the present invention requires a firmware simulator and a flight simulator. This section describes why they are needed:

[0083] The sensor identifier requires a firmware simulator. The sensor identifier based on dynamic analysis observes the firmware reading and writing SPI and I2C by executing the firmware to obtain accurate information about the connected sensors. The firmware simulator is used to assist firmware execution and dynamically hijack SPI and I2C access.

[0084] The actuator identifier requires a firmware simulator. The actuator identifier based on dynamic analysis observes the firmware writing PWM by executing the firmware to obtain accurate information about additional actuators. The firmware simulator is used to assist firmware execution and dynamically hijack PWM addition.

[0085] The UAV simulator requires a firmware simulator and a flight simulator. The UAV simulator executes the firmware and collaboratively simulates the UAV flight. The firmware executor assists in executing the firmware, and the flight simulator assists in simulating the UAV flight.

[0086] C. Sensor Identifier

[0087] The system of the present invention needs to obtain sensor information on the MCU. More precisely, through the analysis of the firmware, the system of the present invention can identify the manufacturer and product model of the sensor (for example, the Pixhawk4 uses the BMI055 sensor). In order to obtain the model of additional sensors from the firmware, first extract the features of the sensors through dynamic analysis, and then obtain the sensor information by matching with the database established from the common sensor data sheets. This subsection first introduces the features of the sensors, then introduces the method of dynamically inferring sensor information of the present invention, and finally introduces the construction and matching methods of the database.

[0088] 1) Features of the sensor: The firmware is connected to the sensor of the MCU through I2C or SPI. Here, the present invention summarizes the interaction between the process firmware and the sensor. Generally, the sensor provides an interface in the format of register addresses from 0x00 to 0xFF for the firmware to read or write. By reading / writing these registers, the firmware calls the functions of the sensor. For example, the MPU6050 sensor measures acceleration and angular velocity. The register in address 0x19 is used for acceleration sampling rate configuration, and the firmware writes a specific value to tell the sensor to measure acceleration at a specific sampling rate; the registers in addresses 0x3B - 0x40 provide the raw data of acceleration measurement, and the firmware reads values from these registers and then processes these data to obtain the acceleration value. The register definitions between sensors are significantly different, which enables the register definition to represent the characteristics of the sensor. For example, compared with the MPU6050, the ICM - 42670P obtains acceleration from the registers in addresses 0x0B - 0x16, which is different from 0x3B - 0x40 of the MPU6050, although they implement the same function.

[0089] Although the sensor contains registers dedicated to different functions, it can be classified into four types according to its functions: data register (DR), control register (CR), status register (SR), and identity register (IR).

[0090] Data register (DR): The DR is a register that provides the measurement data of the sensor.

[0091] Control register (CR): The CR is a register used to control or configure the sensor. The firmware controls or adjusts the sensor by writing a specific value to the CR (for example, measurement scale or sampling rate).

[0092] Status register (SR): The SR is a set of flags indicating the status of the sensor. When the sensor status changes to notify the firmware of the status, the updated value of the SR is generated. Before performing certain operations, the firmware reads the corresponding SR to ensure that the sensor is ready.

[0093] Identity register (IR): The IR is a register that contains constant values set by the sensor vendor. This register is read - only. Such a design is for convenient hardware integrity verification. The firmware checks whether the IR value is equal to the vendor to ensure the presence of the sensor, which usually occurs when the firmware starts up. If the value checked by the firmware is not equal to the value set by the vendor, the drone take - off is rejected and an abnormal situation is reported.

[0094] As a register, the internal sensor has its unique address and can be labeled as DR, CR, SR, or IR according to its function. The present invention can describe the register of the sensor as an address-function mapping pair, which the present invention simplifies to AFMP. For example, the CR of MPU6050 is located at address 0x19, the DR is located at addresses 0x3B - 0x40, the IR is located at address 0x75, the vendor-set value is 0x68, and the SR is located at address 0x36. According to the rule, the AFMP can be described as {0x19:CR, 0x36:SR, 0x3B:DR, 0x3C:DR, 0x3D:DR, 0x3E:DR, 0x3F:DR, 0x40:DR, 0x75:0x68}. Note that for the IR, the present invention also records the vendor-set value, which is useful for further identification.

[0095] 2) Feature extractor: The feature extractor is a tool used to extract the AFMP of the sensor from the firmware, which helps the sensor identifier distinguish different sensor models. The sensor identifier uses the AFMP to distinguish sensors. To determine the model of an additional sensor, first extract the AFMP of the sensor from the firmware for further matching. This section will show the working principle of the extraction.

[0096] Generally, the firmware reads DR, IR, and SR and writes to CR. The read operation brings the value of the register, which may affect the firmware execution path because reading values from IR or SR usually follows conditional judgment. The firmware decides the path to execute according to the value. The present invention summarizes how the values of DR, IR, and SR affect the execution mode, as Figure 6 shown. The hexadecimal numbers in the boxes represent each possible value of a single register, while the letters in the loops represent the possible execution paths after the firmware reads the value from the register. This figure shows the possible execution paths after the firmware reads values from different registers. For DR (see Figure 6 a), the execution path is not affected by the value; for IR (see Figure 6 b), the execution path is affected by the value because only when the IR value is equal to the vendor-set value can the firmware confirm the existence of the sensor and continue to the next step; for SR (see Figure 6 c), the execution path affected by the value, as the firmware of the sensor checks the status, by judging specific bits of the SR value, the values that meet the specific bits cause the firmware execution path B, as shown in the figure, on the other hand, execution path A.

[0097] The above properties help the feature extractor determine the type of the accessed register.

[0098] The goal of the feature extractor is to infer the AFMP of each sensor as close to the truth as possible from the firmware. The feature extractor uses a firmware emulator to execute the firmware and hijack the SPI or I2C access. The firmware emulator can hijack the firmware read and write of an unknown sensor communicating with the firmware and filter out other sensors.

[0099] For example, if both the MPU6050 and BMI055 sensors communicate with the firmware, the firmware emulator can only hijack the communication with the BMI055 and ignore the communication with the MPU6050. Based on the above features, the feature extractor analyzes each additional sensor separately to explore the type of register. Next, the present invention describes how the feature extractor explores the register type of a single sensor in a section.

[0100] The feature extractor executes the firmware and hijacks the firmware access to the sensor to be evaluated. When a read or write access occurs to an unknown register, the firmware execution pauses to determine the type of register: for a write access, the target register being accessed must be the CR because write access only occurs on the CR.

[0101] For a read access, the target register being accessed may be the DR, IR, or SR. Since the firmware execution path is affected by the SR and IR, in order to infer the extraction type of the accessed register, different values need to be tested and then the execution paths caused by these values are observed. To handle this situation, the feature extractor takes a snapshot of the execution and tries to input different values from 0x00 to 0xFF to the snapshot. The feature extractor infers the type of register by observing the execution paths generated by different values.

[0102] The feature extractor continuously executes the firmware and explores the registers until no unknown registers appear. Finally, the feature extractor combines the types of each register to construct the AFMP of the sensor.

[0103] 3) Database: The sensor recognizer collects the sensor data sheets released by the suppliers to establish a database to identify the sensor models (see ① and ② in Figure 5 . The storage data sheet is indexed by the AFMP of the corresponding sensor. According to the description of the register functions in the data sheet, its AFMP is manually constructed. Some suppliers even summarize the functions of the registers into a table, which helps to simplify the refinement work.

[0104] 4) Matcher: The matcher is used to search in the database for the AFMP of the corresponding sensor inferred by the feature extractor.

[0105] The Matcher searches the database by comparing the IR, CR, SR, and DR. In this way, the Matcher gradually narrows down the search scope and finally hits the target. The reasons for the matching order are as follows: The original purpose of the IR is for identification. When a supplier produces a new sensor model, they usually try to set the IR address and value different from those of existing sensors. Such a practical factor makes the IR the best register for differentiating sensors. The CR is the only register written by firmware among the four types of registers, which differentiates it from the SR and DR and is difficult to be misinterpreted. The SR and DR are sometimes confused with each other. For example, the firmware may read a value from the SR to check the available registers but does not follow further judgment. In this case, the SR may be mistaken for the DR.

[0106] D. Actuator Identifier

[0107] The system of the present invention needs to obtain information about the actuators attached to the MCU. The actuators of a drone (such as a quadcopter or hexacopter) are devices that receive control signals from the firmware and control the motor speed. The key is to determine the protocol for controlling the actuator speed.

[0108] The protocol for controlling the motor speed is called the Electronic Speed Control (ESC) protocol. The ESC protocol transmits the rotational speed to the motor by modulating a square wave. The ESC protocol is divided into analog ESC and digital ESC: The analog ESC represents an analog value by adjusting the high-level occupancy time. The longer the high-level occupancy time, the faster the motor speed. The digital ESC represents a bit value by modulating the high-level occupancy time. Generally, a high-level occupancy time less than 25% in a pulse represents 0, and a high-level occupancy time greater than 75% represents 1. The digital series ESC protocol transmits the speed value by encoding the analog value into binary.

[0109] Since the on-chip peripheral PWM is used to generate an adjustable square wave, the firmware manipulates the PWM to generate signals to control the motor. The actuator identifier executes the firmware using a firmware simulator and captures the waveform by hijacking the interaction between the firmware and the PWM. The actuator identifier differentiates the protocol by analyzing the waveform in the time-domain characteristics: For the digital ESC, the high-level occupancy time of each pulse jump between two values is fixed; for the analog ESC, the time occupancy of the high level of each pulse is not fixed.

[0110] E. Simulator Generator

[0111] After identifying the sensors and actuators, the simulator generator obtains information about the attached sensors and actuators (see Figure 5 ⑩ in Step), and then allocate the emulator module. The emulator module allocated by the emulator generator is manually constructed but reusable. If the configured sensor or actuator has been constructed, it is directly allocated by the emulator generator; otherwise, manual operation is required. In the next subsection, the present invention will introduce how to construct the emulator module.

[0112] 1) Sensor emulator: Figure 7 Figure a shows a conceptual diagram of a sensor that is configured and controlled by receiving control parameters (e.g., measurement ratio), then measures physical factors (e.g., acceleration), and finally generates raw data.

[0113] The sensor emulator simulates such a process. The structure of a single sensor is simulated by the sensor emulator as shown in Figure 7 Figure c. The sensor emulator is divided into two parts: the register part stores a table to record the values of the registers and handles the reading and writing of the firmware; the logic part is responsible for generating the SR and DR values. The logic part converts the UAV state data provided by the flight simulator into SR and DR values according to the values provided by the CR. The register part is common to each sensor, and the only effort in constructing a new sensor emulator is the implementation of the logic part.

[0114] 2) Actuator emulator: Figure 7 Figure b shows a conceptual diagram of an actuator that processes control signals and then adjusts the motor speed. The design of the actuator emulator is straightforward. It first demodulates the signal generated by the firmware using a demodulator to recover the output value, and then adjusts the value using a scaler (see Figure 7 Figure d). The demodulated value should be adjusted by the scaler. For example, the value range of the analog ESC signal is 0 to 1. However, the flight simulator requires the rotational speed of the motor, and the scaler adjusts it to a value consistent with the flight simulator. Finally, the actuator emulator outputs the motor rotational speed value.

[0115] 4. The key points of the present invention are as follows:

[0116] 4.1. A novel and scalable simulation-based technology DVATAR for converting data between UAV firmware and flight simulator, thus making flight simulation applicable to binary UAV firmware.

[0117] 4.2. A sensor identifier, which is a module for identifying sensors on the UAV MCU.

[0118] The feature extractor infers the register address mapping pairs of each sensor as close to the real ones as possible from the firmware through dynamic analysis;

[0119] Collect the sensor data sheets released by the suppliers to establish a database and identify the sensor models;

[0120] Search for the register address mapping pairs of the corresponding sensors inferred by the feature extractor in the database to determine the sensor type.

[0121] 4.3. The actuator recognizer is a module on the UAV MCU used to recognize actuators. The key is to determine the protocol for controlling the actuator speed.

[0122] The actuator recognizer executes the firmware using a firmware simulator, captures waveforms by hijacking the interaction between the firmware and PWM, and distinguishes the protocol by analyzing the waveforms in the time-domain characteristics.

[0123] 4.4. The simulator generator is a module for semi-automatically generating and allocating simulators for additional sensors and actuators.

[0124] The process of generating a sensor simulator;

[0125] The process of generating an actuator simulator.

[0126] 5. Compared with the best prior art in the same category, the present invention has the following advantages:

[0127] The present invention proposes a tool (sensor recognizer and actuator recognizer) to automatically analyze the firmware and then extract information about sensors and actuators. Since different UAVs may use the same type of sensors or drivers, a single interpreter can be reused to assist different firmware in exchanging data with the flight simulator. To help new firmware interact with the flight simulator, analysts only need to assemble different interpreters. This reusability and composability of the interpreters solve the challenge of diversity. The present invention proposes a framework for semi-automatically constructing interpreters (simulator generator), thereby reducing manpower.

[0128] 5.4. Can recognize the models of unknown sensors.

[0129] Matching. After the feature extractor infers the register type of a single unknown sensor, a register address function mapping pair representing the sensor characteristics is constructed. Then, the sensor recognizer matches with the unknown sensors in the database.

[0130] Embodiment 2

[0131] As Figures 1 to 9 shown, as a further optimization of Embodiment 1, on the basis of Embodiment 1, this embodiment further includes the following technical features:

[0132] A. Environment setting

[0133] The system of the present invention requires a firmware simulator and a flight simulator. The present invention uses Halucinator[2] as the firmware simulator and Airsim as the flight simulator. To establish a sensor recognition database, the present invention has collected 80 sensor data sheets. These data sheets are about sensors widely used in open-source drone projects such as PX4, Ardupilot, and Betaflight.

[0134] Example setup. The present invention uses the code provided by the Ardupilot project for experiments. Ardupilot is a widely used flight control program that has been ported to many free or commercial platforms, including development boards or complete drones. It is customizable and provides many driver modules to adapt to different sensors. These drivers handle the logic and IO operations for the firmware to interact with their hardware sensors. The present invention selects 29 drivers from Ardupilot for evaluation. These drivers operate the corresponding sensors and output measurement data. The data sheets of the 29 sensors are included in the 80 data sheets collected by the present invention.

[0135] B. Identifying additional sensors

[0136] To evaluate whether DVATAR can correctly identify sensors, the present invention first evaluates the accuracy of the sensor identifier when inferring the type of unknown sensor registers, and then verifies whether it can correctly match the sensors in the database.

[0137] Type inference. For the 29 drivers corresponding to 29 different sensors, the present invention independently executes each driver using Halucinator. Halucinator hijacks the driver's access to the sensor and applies the method proposed in the sensor identifier to infer the types of the individual registers accessed by the driver. The results are then verified according to the sensor data sheets. Table 1 lists the sensors evaluated by the present invention. Name represents the vendor naming of the sensor, and Usage represents the usage of the sensor, where Acc represents accelerometer, Gyro represents gyroscope, Acc&Gyro represents a multi-functional sensor capable of measuring acceleration and angular velocity. Note that there are duplicate names here, such as BMI055, but the BMI055 used for the usage Acc is a different sensor from the one used for the usage gyro. Baro represents barometer, and Mag represents magnetometer. Num represents the total number of registers accessed by the firmware during execution. "Correct" and "Incorrect" represent the number of registers correctly identified or misidentified. The slash indicates that such a register does not exist.

[0138] Overall, among the 29 sensors, the sensor identifier correctly identified 96.4% of IR, 100% of CR, 75% of SR, and 92.1% of DR. The table also confirmed that the accuracy of inferring SR and DR types is relatively lower than that of IR and CR. When searching for sensor models in the database, IR and CR should be given priority.

[0139] Matching. After the feature extractor infers the register type of a single unknown sensor, an AFMP representing the sensor features is constructed. Then, the sensor identifier is matched with the unknown sensor in the database. The present invention evaluates the accuracy of the sensor identifier for identifying unknown sensors. First, a database is constructed using 80 data sheets for the matcher to search.

[0140] Then, the 29 AFMPs extracted by the feature extractor are matched in the database, and the accuracy rate is calculated. The results are shown in Table 3. The present invention checks the correctly matched sensors in the "matched" column. Generally speaking, the present invention correctly matched 27 out of 29 sensors. The matching results show that by analyzing the firmware, the sensor identifier can identify additional sensors even if the types of some registers are incorrect.

[0141] Table 1 Performance Analysis Table of DVATAR for Identifying Sensors

[0142]

[0143] C. State Data Conversion

[0144] As DVATAR uses a sensor simulator in a flight simulator to convert the state data of a virtual UAV, the present invention established this experiment to evaluate the effectiveness of data conversion, showing that the sensor simulator can correctly convert the state data of the flight simulator into firmware.

[0145] The process of evaluating a single sensor simulator is as Figure 8 shown. The present invention controls a virtual UAV performing a flight mission in Airsim and records the state data of the mission ( Figure 8 ①). Then, the state data is replayed to the sensor actuator using the sensor simulator (②, ③), and the actuator output is collected (④). At the same time, the present invention selects the data related to the evaluated sensor simulator as the expected output (⑤). For example, if the present invention evaluates an accelerometer, the present invention selects the state data regarding acceleration.

[0146] Finally, the present invention compares the similarity of the driver output and the expected output to evaluate the effectiveness of the sensor-converted data. If the driver output performance is similar to the except output, does it prove that the sensor simulator effectively represents the real sensor and correctly converts the data?

[0147] Figure 9 In this system, the additional sensors and (servo) actuators of the drone are first identified. Then, simulators for the sensors and actuators are generated for guiding data. Finally, the present invention uses the sensor simulator and actuator simulator to connect the firmware simulator and the flight simulator.

[0148] To quantify the similarity between the input and output data, the present invention applied the Pearson correlation method to evaluate five sensor simulators, and the results are shown in Table 2. x, y, and z represent the comparison results on these axes. Note that BMP085 is a barometer that only outputs one-dimensional data. For the Pearson correlation method, r > 0.7 is considered a strong correlation, and p < 0.05 is considered that the difference is statistically significant. The results show that the constructed simulator can convert the state data into firmware.

[0149] Table 2 Performance of DVATAR in Identifying Sensors

[0150]

[0151] D. Manual Work

[0152] The structure of the sensor simulator is semi-automatic. Since it involves a manual, in this section, the present invention will evaluate the manual work of constructing a single sensor simulator. Constructing the simulator of the sensor is a manually coded logic part. Essentially, processing the firmware to access the sensor is to process the firmware to read or write registers. However, quantifying the manual work by time or code complexity, these indexes do not reflect the amount of sensor knowledge that the developer must understand. When the logic part manipulates the value of the register, the developer must understand the function of the imported register. The present invention classifies the manipulation of the register values used in the experiment into three categories:

[0153] Simple handlers that return a constant value or simply record the value, which do not implement any logic. The IR is a register that only returns a constant value. The CR part that does not involve data conversion is processed by simply recording the value (for example, the CR for configuring the sensor power).

[0154] Conversion handlers usually convert the CR value into parameters for data conversion, and the data conversion does not implement any logic. Logic processors are usually used to generate the values of SR and DR, which requires understanding the internal logic of the sensor.

[0155] The present invention evaluates the registers of eight sensors in the Ardupilot project, calculates the registers to be processed, and classifies the processor types. The results are shown in Table III. From the table, the present invention concludes that for accelerometers or gyroscopes, most processors are Trivial or Translation, with a logic lower than 30%. However, for barometers or magnetometers, the logic is higher than 80% and 50% because more calibration parameters need to be read from the DR for barometers and magnetometers.

[0156] Table 3 Table of the manual work results for constructing the emulator

[0157]

[0158] E. Case Study

[0159] In this section, the present invention will show how to simulate the flight of drone firmware using the technology of the present invention. The present invention studies a real drone firmware [4], an STM32 target drone firmware that can be executed by the firmware simulator Halucinator [2]. The present invention uses the method of the present invention to connect the firmware flight simulator Airsim for simulation.

[0160] Identification and allocation. The present invention first uses the sensor identifier to scan the sensors used by the drone. The results are shown in Table IV, where the AFMP detected by the sensor identifier and the corresponding sensors are presented. According to the information given by the sensor identifier, the corresponding sensor emulator is allocated for data transmission.

[0161] Table 4 AFMP table for additional sensors

[0162]

[0163] Synchronization. The execution of the firmware should be synchronized with the flight simulator. Generally, the flight simulator can run in synchronization with the real-world time. However, the firmware cannot be executed in synchronization with the real-world time. A QEMU-based firmware simulator is given. QEMU, which depends on the firmware simulator, is limited in terms of timing. It only provides the statistical clock ticks of the firmware execution. The clock tick interval is not synchronized with the real time (sometimes fast or sometimes slow). To synchronize the firmware and the flight simulator, the present invention patches QEMU to convert the clock tick interval into a time interval. Finally, the present invention modifies the flight simulator to synchronize it with the converted time interval of QEMU.

[0164] In addition, the related work of the present invention is introduced as follows:

[0165] A. Firmware Simulation

[0166] Executing firmware in a virtual environment is a widely discussed issue in dynamic testing. Early research, Avatar, proposed a solution using hybrid simulation and concolic execution. The proxy improved the performance of Avatar and supported near-real-time simulation. Recent research has mainly focused on the dynamic testing and fuzzing of firmware for hardware-free embedded systems. They follow the method of simulating peripheral devices on the chip or high-level simulation behavior. P2IM[1] is a framework for automatically modeling the I / O behavior of peripherals while treating the peripherals themselves as black boxes. Emu is a framework for automatically finding appropriate responses to access unknown peripherals. Halucinator[2] is a framework for using high-level simulation to simulate HAL layer firmware. It resides on commercial hardware and models the behavior of general MCUs, supporting hal-enabled hardware logic. In contrast, DVATAR connects drone firmware to a flight simulator and is dedicated to exploring physical space vulnerabilities.

[0167] B. Detection of Physical Space Vulnerabilities in Drones

[0168] Physical space vulnerabilities threaten the safety of drones. Recent research has paid attention to this vulnerability. RVFuzzer[3] proposed a control program testing system that reveals the illegal but acceptable value ranges of dynamically adjustable control parameters. In contrast, DVATAR provides a tool for simulating the state of drones for these methods, thus extending its method to a wider range of drone firmware.

[0169] The system first identifies the additional sensors and (servo) actuators of the drone. Then, it generates simulators for the sensors and actuators to guide data. Finally, the present invention uses the sensor simulator and actuator simulator to connect the firmware simulator and the flight simulator.

[0170] The application prospects of the present invention are as follows:

[0171] Physical space vulnerabilities threaten the safety of drones. Recent research has paid attention to this vulnerability. RVFuzzer[3] proposed a control program testing system that reveals the illegal but acceptable value ranges of dynamically adjustable control parameters, that is, when the system receives a set of tampered commands with illegal parameter values, the drone will execute such commands, resulting in the impact of its mission. This anomaly caused by software that affects the performance of the drone in the physical space is called a physical space vulnerability. Different from traditional software vulnerabilities that disrupt program execution (such as memory corruption) and can be captured by a runtime monitor (such as QEMU), physical space vulnerabilities can only be captured by flying in the physical space and observing the behavior of the drone.

[0172] The detection of physical space vulnerabilities in drones requires the assistance of flight simulation, but most drone firmware already has related technologies. The present invention proposes a new off-chip peripheral emulation technology for simulating drones, which assists flight simulation applications by emulating the sensors and actuators of drones. It converts simulation-related data between flight simulation and firmware, simulates the flight of drones in a virtual three-dimensional space, and can observe the anomalies of drones in the physical space when virtual space vulnerabilities are triggered, thereby capturing physical space vulnerabilities during firmware execution. This brings a new and effective way to analyze and explore physical space vulnerabilities.

[0173] This technology can assist flight simulation on drones with different sensors. Although the method of the present invention requires the manual construction of emulators, these emulators can be reused to simulate different drones. The present invention believes that the manual work of maintaining emulators in cooperation with analysts will be very important.

[0174] The present invention cites the following documents:

[0175] [1] B.Feng, A.Mera, and L.Lu, “{P2IM}: Scalable and hardware-independent firmware testing via automatic peripheral interface modeling,” in 29th USENIX Security Symposium (USENIX Security 20), 2020, pp.1237–1254.

[0176] [2] A.A.Clements, E.Gustafson, T.Scharnowski, P.Grosen, D.Fritz, C.Kruegel, G.Vigna, S.Bagchi, and M.Payer, “{HALucinator}: Firmware re-hosting through abstraction layer emulation,” in 29th USENIX Security Symposium (USENIX Security 20), 2020, pp.1201–1218.

[0177] [3]T. Kim, C. H. Kim, J. Rhee, F. Fei, Z. Tu, G. Walkup, X. Zhang, X. Deng, and D. Xu, “{RVFuzzer}: Finding input validation bugs in robotic vehicles through {Control-Guided} testing,” in 28th USENIX Security Symposium (USENIX Security19), 2019, pp. 425–442.

[0178] [4]“eysip-2017 control and algorithms development for quadcopter,” 2017. [Online]. Available: https: / / github.com / eYSIP-2017 / eYSIP2017_Control_and_Algorithms_development_for_Quadcopter

[0179] As described above, the present invention can be preferably implemented.

[0180] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, can be combined and / or extended and / or replaced in any way, except for mutually exclusive features and / or steps.

[0181] As mentioned above, it is only a preferred embodiment of the present invention, and it does not impose any formal limitations on the present invention. Based on the technical essence of the present invention, any simple modifications, equivalent replacements, and improvements made to the above embodiments within the spirit and principles of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A peripheral simulation method for an unmanned aerial vehicle (UAV) simulator, characterized in that, it is applied to a peripheral simulation system for an unmanned aerial vehicle simulator. The peripheral simulation system for an unmanned aerial vehicle simulator includes: a sensor identifier, a simulator generator, and a UAV simulator that are electrically connected in sequence, and further includes an actuator identifier electrically connected to the simulator generator. Among them, the sensor identifier is used to identify the sensors on the UAV microcontroller unit; the simulator generator is used to semi-automatically generate and allocate simulators with attached sensors and actuators; the UAV simulator is used to execute the UAV firmware and simulate the flight of the UAV; the actuator identifier is located on the UAV microcontroller unit and is used to identify the actuators. When performing peripheral simulation, it includes the following steps: A1, the sensor identifier identifies the model of the sensor, constructs the database record function of the sensor according to the sensor data sheet, and extracts the features of the sensor through dynamic analysis of the firmware; then compares the features with the database to obtain the model of the attached sensor; A2, the actuator identifier obtains the information of the attached actuator by capturing and evaluating the waveform of the control signal; A3, the simulator generator semi-automatically generates or allocates simulators with attached sensors and actuators by using the information provided by the sensor identifier and the actuator identifier; A4, the UAV simulator executes the firmware and simulates the flight of the UAV, and uses the sensor simulator and / or actuator simulator allocated by the simulator generator to transfer data between the firmware simulator and the flight simulator to achieve flight simulation on the UAV firmware.

2. The peripheral simulation method for an unmanned aerial vehicle simulator according to claim 1, characterized in that, in step A2, the actuator identifier uses the firmware simulator to execute the firmware and captures the waveform by hijacking the interaction between the firmware and the PWM; and, the actuator identifier distinguishes the protocol by analyzing the waveform in the time domain characteristics: for a digital ESC, the high-level occupancy time of each pulse jump between two values is fixed; for an analog ESC, the time occupancy of the high level of each pulse is not fixed.

3. The peripheral simulation method for an unmanned aerial vehicle simulator according to claim 1, characterized in that, in step A3, after the simulator generator identifies the sensors and actuators, the simulator generator obtains the information of the attached sensors and actuators, and then allocates the simulator module; among them, the simulator module allocated by the simulator generator is reusable.

4. The peripheral simulation method for an unmanned aerial vehicle simulator according to claim 1, characterized in that, in step A4, the UAV simulator realizes a simulation loop for transmitting signals in sequence along the firmware simulator, actuator simulator, flight simulator, sensor simulator, and firmware simulator.

5. A peripheral simulation system for an unmanned aerial vehicle simulator using the peripheral simulation method for an unmanned aerial vehicle simulator according to any one of claims 1-4, characterized in that, hardware sensors and actuators are replaced with interpreters to achieve data exchange between the simulator and the firmware.

6. A peripheral simulation system for an unmanned aerial vehicle simulator using the peripheral simulation method for an unmanned aerial vehicle simulator according to any one of claims 1-4, characterized in that, Use a simulator to execute the firmware and hijack the firmware to access on-chip peripherals. The interpreter processes the firmware's access to on-chip peripherals to simulate sensors or actuators, and the firmware accesses, interprets, and converts simulation-related data. Among them, a separate interpreter is semi-automatically built based on the data sheet released by the vendor and is responsible for processing the firmware that accesses a single model of sensor or actuator.

Images