Unmanned aerial vehicle multi-sensor unified display and remote configuration method and device

Abstract

The invention relates to the technical field of ground station software, and discloses an unmanned aerial vehicle multi-sensor unified display and remote configuration method and device. The method is applied to a ground station and comprises the following steps: acquiring sensor data of an aircraft through a multi-sensor aggregation service unit; the multi-sensor aggregation service unit is deployed on a ground station and operates independently; generating a visual configuration window corresponding to the sensor data through the multi-sensor plug-in module according to the sensor configuration description file in the preset format; the multi-sensor plug-in module is integrated on a ground station based on a Qt plug-in mechanism; and the multi-sensor plug-in module responds to the operation of a user for the visual configuration window, generates a remote parameter setting instruction, packages a remote parameter setting message, and transmits the remote parameter setting message back to the aircraft through the multi-sensor aggregation service unit, so that the aircraft adjusts the sensor parameters according to the remote parameter setting message. According to the scheme, the sensor configuration efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of ground station software technology, specifically to a method and apparatus for unified display and remote configuration of multiple sensors on unmanned aerial vehicles (UAVs). Background Technology

[0002] QGroundControl (QGC) is an open-source UAV ground control station software that provides flight control and mission planning functions for UAVs supporting the MAVLink protocol. However, QGC can only parse standard MAVLink messages and cannot display data from custom sensors (such as LiDAR, gas detectors, and drop servos). Furthermore, adding a corresponding visual configuration window for a new custom sensor requires recompiling the entire project, making upgrades and maintenance difficult. In addition, in multi-UAV cluster scenarios, different UAVs need to communicate with QGC through different interfaces, resulting in cumbersome operations and low efficiency.

[0003] Therefore, there is an urgent need for a convenient, flexible, and efficient sensor configuration method. Summary of the Invention

[0004] This invention provides a method and apparatus for unified display and remote configuration of multiple sensors in unmanned aerial vehicles (UAVs), in order to solve the problems of complex operation and low efficiency in sensor configuration methods in related technologies.

[0005] In a first aspect, the present invention provides a method for unified display and remote configuration of multiple sensors of an unmanned aerial vehicle (UAV), applied to a ground station, the method comprising: Sensor data from the aircraft is acquired through a multi-sensor aggregation service unit; the multi-sensor aggregation service unit is deployed at a ground station and operates independently. The multi-sensor plug-in module generates a visual configuration window corresponding to the sensor data based on a sensor configuration description file in a preset format; the multi-sensor plug-in module is integrated into the ground station based on the Qt plug-in mechanism. The multi-sensor plug-in module responds to user actions on the visual configuration window and generates remote parameter setting instructions. The multi-sensor plug-in module encapsulates the remote parameter setting message according to the remote parameter setting instruction, and the multi-sensor aggregation service unit transmits the remote parameter setting message back to the aircraft so that the aircraft can adjust the sensor parameters according to the remote parameter setting message.

[0006] In one optional implementation, the multi-sensor aggregation service unit is used to listen to custom sensor messages of the aircraft through the User Datagram Protocol (UDP) network, process the custom sensor messages to obtain sensor data, and forward the sensor data to the local listening interface of the ground station. The acquisition of aircraft sensor data through the multi-sensor aggregation service unit includes: The system receives sensor data sent by the multi-sensor aggregation service unit through a local listening interface.

[0007] In one optional implementation, the step of generating a visual configuration window corresponding to the sensor data through a multi-sensor plug-in module based on a sensor configuration description file of a preset format includes: The multi-sensor plug-in module parses the sensor data according to the sensor configuration description file to generate a sensor configuration file. Register the sensor Qt meta-object according to the sensor configuration file; A visualization configuration window corresponding to the sensor data is generated based on the sensor's Qt meta-object.

[0008] In one alternative implementation, the sensor configuration description file includes at least one of the following: Sensor identification name; The custom sensor message identifier corresponding to the sensor identifier name; Sensor parameter field names; The physical units corresponding to the sensor parameter field names; The display threshold corresponding to the sensor parameter field name; The color code corresponding to the display threshold.

[0009] In one optional implementation, the multi-sensor aggregation service unit is further configured to maintain an aircraft identifier-aircraft address-User Datagram Protocol (UDP) port relationship table, and associate the monitored custom sensor messages with the aircraft identifier, aircraft address, and UDP port according to the aircraft identifier-aircraft address-UDP port relationship table.

[0010] In one optional implementation, the step of encapsulating the remote parameter setting message according to the remote parameter setting instruction includes: The remote parameter setting command is parsed to obtain the aircraft address, sensor parameter field name, and adjustment value corresponding to the sensor parameter field name; The aircraft address, sensor parameter field name, and the adjustment value corresponding to the sensor parameter field name are encapsulated into a corresponding remote parameter setting message.

[0011] Secondly, the present invention provides a unified display and remote configuration device for multiple sensors of an unmanned aerial vehicle (UAV), applied to a ground station, the device comprising: The acquisition module is used to acquire sensor data of the aircraft through a multi-sensor aggregation service unit; the multi-sensor aggregation service unit is deployed at a ground station and operates independently. The visualization module is used to generate a visualization configuration window corresponding to the sensor data based on a sensor configuration description file in a preset format through the multi-sensor plug-in module; the multi-sensor plug-in module is integrated into the ground station based on the Qt plug-in mechanism; The parameter setting module is used to generate remote parameter setting instructions in response to user operations on the visual configuration window via the multi-sensor plug-in module; The sending module is used to encapsulate a remote parameter setting message according to the remote parameter setting instruction through the multi-sensor plug-in module, and to send the remote parameter setting message back to the aircraft through the multi-sensor aggregation service unit, so that the aircraft can adjust the sensor parameters according to the remote parameter setting message.

[0012] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the unmanned aerial vehicle multi-sensor unified display and remote configuration method of the first aspect or any corresponding embodiment described above.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the unmanned aerial vehicle (UAV) multi-sensor unified display and remote configuration method described in the first aspect or any corresponding embodiment thereof.

[0014] Fifthly, the present invention provides a computer program product, including computer instructions, which are used to cause a computer to execute the unmanned aerial vehicle multi-sensor unified display and remote configuration method of the first aspect or any corresponding embodiment described above.

[0015] The technical solution provided by this invention may include the following beneficial effects: The UAV multi-sensor unified display and remote configuration method provided by this invention integrates multi-sensor plug-in modules into the ground station through the Qt plug-in mechanism. This allows for dynamic loading and operation without modifying the original ground station code, resulting in excellent system scalability. Adding a new sensor does not require refactoring or recompiling the entire ground station software, significantly reducing development and maintenance costs. Since the multi-sensor plug-in modules automatically generate corresponding visual configuration windows based on preset sensor configuration description files, the sensor data structure and interface layout are decoupled, achieving a rapid deployment mode of "configuration takes effect immediately." When a new sensor is connected, only the configuration file needs to be updated to generate the corresponding operation interface, eliminating the need to write new UI code and greatly shortening the development cycle. Furthermore, the multi-sensor aggregation service unit is independently deployed on the ground station and is responsible for acquiring sensor data from the aircraft, serving as a unified data aggregation point to process data from multiple aircraft or various sensors simultaneously. The device's data stream enables centralized access and forwarding of data from multiple devices and channels, simplifying the operational complexity for users in multi-machine scenarios and improving monitoring efficiency. Since user operations in the visual configuration window are converted into remote parameter setting commands, which are then encapsulated into standard messages by the multi-sensor plug-in module and transmitted back to the aircraft via the same multi-sensor aggregation service unit, a complete closed loop of "user operation - command generation - message distribution - parameter adjustment" is formed. This supports remote real-time parameter adjustment, reducing reliance on physical contact or on-site serial port debugging, making it suitable for emergency, cluster, or long-distance operation scenarios, and improving adjustment flexibility and efficiency. Both uplink (sensor data upload) and downlink (remote parameter setting message transmission) are completed through the same multi-sensor aggregation service unit, achieving bidirectional multiplexing of the communication link, reducing the number of network connections and port management burden, improving resource utilization, and ensuring the consistency and synchronization of uplink and downlink data. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the first step in the method for unified display and remote configuration of multiple sensors of a drone according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the second process of the method for unified display and remote configuration of multiple sensors of a drone according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a multi-channel structure of a multi-aircraft cluster according to an embodiment of the present invention; Figure 4 This is a flowchart of the multi-sensor plug-in module according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating the remote parameter setting process according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a unified display and remote configuration system for multiple sensors of a drone according to an embodiment of the present invention; Figure 7 This is a structural block diagram of a drone multi-sensor unified display and remote configuration device according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0020] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] QGroundControl (QGC) is an open-source UAV ground control station software that provides flight control and mission planning functions for UAVs supporting the MAVLink protocol. However, QGC can only parse standard MAVLink messages and cannot display data from custom sensors (such as LiDAR, gas detectors, and drop servos). Furthermore, adding a corresponding visual configuration window for a new custom sensor requires recompiling the entire project, making upgrades and maintenance difficult. In addition, in multi-UAV cluster scenarios, different UAVs need to communicate with QGC through different interfaces, resulting in cumbersome operations and low efficiency.

[0022] According to an embodiment of the present invention, a method for unified display and remote configuration of multiple sensors of a UAV is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] This embodiment provides a method for unified display and remote configuration of multiple sensors on a UAV, applied to a ground station that can be configured on a laptop, desktop computer, etc. Figure 1 This is a flowchart of a method for unified display and remote configuration of multiple sensors for unmanned aerial vehicles according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain sensor data from the aircraft through the multi-sensor aggregation service unit.

[0024] The ground station runs a main ground station software program, which executes the main functions of the ground station software. During runtime, it calls multi-sensor plug-in modules and communicates with the multi-sensor aggregation service unit to execute the unified display and remote configuration method for the UAV's multi-sensors. This multi-sensor aggregation service unit is deployed on the ground station and runs independently of the main ground station software program. It can independently acquire sensor data from aircraft (UAVs) within its monitoring range and forward it to the main ground station software program, which in turn receives the sensor data acquired by the multi-sensor aggregation service unit. It should be noted that this multi-sensor aggregation service unit can simultaneously monitor sensor data from multiple aircraft.

[0025] Step S102: Using the multi-sensor plug-in module, a visual configuration window corresponding to the sensor data is generated based on the sensor configuration description file in a preset format.

[0026] This multi-sensor plug-in module, based on the Qt plug-in mechanism, is integrated into the main program of the ground station software. Upon startup, it reads a pre-formatted sensor configuration description file and, upon receiving sensor data, generates a corresponding visualization configuration window based on the pre-formatted sensor configuration description file to visualize the sensor data. The Qt plug-in mechanism is a framework for runtime extensibility and modularity. The pre-formatted sensor configuration description file establishes the mapping relationship between sensor data and visualization attributes, such as field names, physical units, and display colors. This pre-formatted sensor configuration description file can be manually filled in by technical personnel according to the peripheral communication protocol document, or automatically generated by a configuration tool with relevant functions. Furthermore, the visualization configuration window provides controls for modifying sensor parameters, allowing users to modify the sensor parameters corresponding to the sensor data according to their actual needs.

[0027] Step S103: The multi-sensor plug-in module responds to the user's operation on the visual configuration window and generates a remote parameter setting command.

[0028] When users need to modify the sensor parameters of the aircraft, they can perform the corresponding operations through this visual configuration window. The multi-sensor plug-in module responds to the user's operation and generates corresponding remote parameter setting instructions.

[0029] In step S104, the multi-sensor plug-in module encapsulates the remote parameter setting message according to the remote parameter setting instruction, and the multi-sensor aggregation service unit sends the remote parameter setting message back to the aircraft so that the aircraft can adjust the sensor parameters according to the remote parameter setting message.

[0030] Next, the multi-sensor plug-in module encapsulates a remote parameter setting message according to the remote parameter setting instruction and sends the message to the multi-sensor aggregation service unit. This remote parameter setting message indicates sensor parameter adjustment information. Upon receiving the message, the multi-sensor aggregation service unit forwards it to the corresponding aircraft, allowing the aircraft to adjust its sensor parameters based on the message, thus achieving remote sensor parameter configuration.

[0031] The UAV multi-sensor unified display and remote configuration method provided in this embodiment integrates multi-sensor plug-in modules into the ground station through the Qt plug-in mechanism. This allows for dynamic loading and operation without modifying the ground station's original code, resulting in excellent system scalability. Adding a new sensor does not require refactoring or recompiling the entire ground station software, significantly reducing development and maintenance costs. Since the multi-sensor plug-in modules automatically generate corresponding visual configuration windows based on preset sensor configuration description files, the sensor data structure and interface layout are decoupled, enabling a rapid deployment mode of "configuration takes effect immediately." When a new sensor is connected, only the configuration file needs to be updated to generate the corresponding operation interface, eliminating the need to write new UI code and greatly shortening the development cycle. Because the multi-sensor aggregation service unit is independently deployed on the ground station and responsible for acquiring sensor data from the aircraft, it serves as a unified data aggregation point, simultaneously processing data from multiple aircraft or various sensors. The device's data stream enables centralized access and forwarding of data from multiple devices and channels, simplifying the operational complexity for users in multi-machine scenarios and improving monitoring efficiency. Since user operations in the visual configuration window are converted into remote parameter setting commands, which are then encapsulated into standard messages by the multi-sensor plug-in module and transmitted back to the aircraft via the same multi-sensor aggregation service unit, a complete closed loop of "user operation - command generation - message distribution - parameter adjustment" is formed. This supports remote real-time parameter adjustment, reducing reliance on physical contact or on-site serial port debugging, making it suitable for emergency, cluster, or long-distance operation scenarios, and improving adjustment flexibility and efficiency. Both uplink (sensor data upload) and downlink (remote parameter setting message transmission) are completed through the same multi-sensor aggregation service unit, achieving bidirectional multiplexing of the communication link, reducing the number of network connections and port management burden, improving resource utilization, and ensuring the consistency and synchronization of uplink and downlink data.

[0032] This embodiment provides a method for unified display and remote configuration of multiple sensors on a UAV, applied to a ground station that can be configured on a laptop, desktop computer, etc. Figure 2 This is a flowchart of a method for unified display and remote configuration of multiple sensors for unmanned aerial vehicles according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Prepare the environment and deploy the architecture.

[0033] For the airborne end, various peripherals (sensors) are connected to the aircraft's cooperative controller. For example, a lidar can be connected via a CAN-FD interface, a gas sensor via a UART interface, and a launch servo can be controlled via PWM signals. The aircraft's cooperative controller (XRCE-DDS Client) needs to support receiving raw data from peripherals, encapsulating this raw data into custom sensor messages, and sending these custom sensor messages via a User Datagram Protocol (UDP) network. These custom sensor messages can be custom MAVLink extension messages.

[0034] For the ground end, it is necessary to run the ground station software main program, multi-sensor aggregation service unit (SensorHub), multi-sensor plug-in module (libMultiSensorPlugin.so) through terminal devices, such as laptops and desktop computers.

[0035] Step S202: Design the format of the custom sensor message.

[0036] For example, the custom sensor message is a custom message type MAVLINK_MSG_ID_SENSOR_PACK, which uses a reserved range of custom sensor message identifiers (message IDs) to carry peripheral data (sensor data). For instance, compressed sensor packet data can be transmitted using a message identifier range (MSG_ID=230~239), with 230 assigned to lidar data, 231 to gas sensor data, and 232 to the status of the throwing servo, etc. The data frame format of the sensor data source can also be set, such as the range of bytes per frame (e.g., ≤200 bytes per frame), whether field serialization is supported, and the compression algorithm selected (e.g., simple bit packing).

[0037] Step S203: Compile a sensor configuration description file in a preset format.

[0038] For example, the preset format sensor configuration description file is a JSON file (sensor_profile.json), which is used to define the visualization and parsing rules of the peripheral device.

[0039] Optionally, the sensor configuration description file includes at least one of the following: Sensor identifier name, used to indicate sensor ID; The custom sensor message identifier corresponding to the sensor identifier name is used to indicate the custom sensor message ID; Sensor parameter field name, used to indicate the field corresponding to the sensor parameter that needs to be defined, for example, the color field is used to indicate the visual display color corresponding to the sensor parameter; The physical unit corresponding to the sensor parameter field name, for example, the physical unit corresponding to the wavelength of lidar is nanometer (nm). The display threshold corresponding to the sensor parameter field name, for example, displaying the first color when the value is greater than the display threshold, and displaying the second color when the value is less than the display threshold; The color code corresponding to this display threshold, for example, the hexadecimal color code for red is #FF0000.

[0040] Step S204: Start the multi-sensor aggregation service unit.

[0041] This multi-sensor aggregation service unit is used to listen for custom sensor messages from the aircraft via the User Datagram Protocol (UDP) network, process the custom sensor messages to obtain sensor data, and forward the sensor data to the local listening interface of the ground station.

[0042] The multi-sensor aggregation service unit is also used to maintain the aircraft identifier-aircraft address-User Datagram Protocol (UDP) port relationship table, and associate the monitored custom sensor messages with the aircraft identifier, aircraft address and UDP port according to the aircraft identifier-aircraft address-UDP port relationship table.

[0043] For example, the multi-sensor aggregation service unit automatically listens to multiple UDP ports (such as 14550, 14551, ..., 14557) and parses the extended MAVLink frames sent by each aircraft, establishing a hash table mapping (maintaining the relationship table of aircraft identifier - aircraft address - user datagram protocol port).

[0044] Optionally, the multi-sensor aggregation service unit filters sensor data within the message identifier range (MSG_ID=230~239), unpacks the sensor data, adds the corresponding aircraft identifier to the unpacked sensor data, and then forwards it uniformly to the local listening port of the ground station. Through the multi-sensor aggregation service unit, data from multiple aircraft can be viewed with just one virtual aggregation connection, avoiding the need for different aircraft to communicate with the QGC through different interfaces, making it convenient, fast, and improving communication efficiency.

[0045] For example, this multi-sensor aggregation service unit is a lightweight Qt network service independent of the ground station software main program. It can listen to all aircraft UDP data streams and forward them uniformly to the ground station's local listening port, such as port 5777 (localhost:5777). This multi-sensor aggregation service unit can perform bidirectional data conversion with the XRCE-DDS (eXtremely Resource Constrained Environment - Data Distribution Service) protocol used by the airborne cooperative controller, enabling sensor parameter adjustments without modifying the flight control firmware.

[0046] For example, Figure 3 This is a schematic diagram of a multi-channel structure of a multi-aircraft cluster according to an embodiment of the present invention. The multi-sensor aggregation service unit is also used to maintain the relationship table of aircraft identifier-aircraft address-user datagram protocol port. The ground station software main program (QGC Plugin) only displays one aggregation connection, without the need to manually switch between different aircraft.

[0047] Optionally, the multi-sensor aggregation service unit is also equipped with a heartbeat detection mechanism, which can determine the connection status of the aircraft through the heartbeat detection mechanism. If no data is received from the aircraft within a predetermined period of time, the aircraft will be automatically removed from the list of available devices.

[0048] Step S205: Obtain sensor data from the aircraft through the multi-sensor aggregation service unit.

[0049] This multi-sensor aggregation service unit is deployed at a ground station and operates independently. The main program of the ground station software can receive sensor data sent by the multi-sensor aggregation service unit through a local listening interface.

[0050] Step S206: Using the multi-sensor plug-in module, a visual configuration window corresponding to the sensor data is generated based on the sensor configuration description file in a preset format.

[0051] This multi-sensor plug-in module is integrated into the ground station software based on the Qt plug-in mechanism. It does not require modification of the QGC core code, enabling non-intrusive secondary development, which is flexible, convenient, and improves development efficiency.

[0052] For example, the ground station software main program scans the plugin directory when it starts up and loads the multi-sensor plugin module (libMultiSensorPlugin.so) through Qt's QLibrary::load(), without having to recompile the QGC project.

[0053] Optionally, when generating the visualization configuration window corresponding to the sensor data based on the sensor configuration description file in a preset format, the multi-sensor plug-in module first parses the sensor data according to the sensor configuration description file to generate a sensor configuration file. Then, the multi-sensor plug-in module registers the sensor Qt meta-object according to the sensor configuration file. Finally, the multi-sensor plug-in module generates the visualization configuration window corresponding to the sensor data based on the sensor Qt meta-object.

[0054] For example, Figure 4 This is a flowchart of the multi-sensor plug-in module according to an embodiment of the present invention. First, the ground station software main program QGC is started, then the multi-sensor plug-in module libMultiSensorPlugin.so is loaded. Then, the multi-sensor plug-in module loads the multi-sensor plug-in module sensor_profile.json, reads field names, units, thresholds, colors, etc., and verifies the validity of the fields. Then, it automatically generates QML (Qt Meta / Modeling Language) attributes, registers Qt meta-objects (QtMetaObject) for each sensor according to the sensor data, and then dynamically generates a visualization configuration window (QML page). It instantiates tab controls, such as numerical display boxes, progress bars (progress bars with color changes that dynamically change color according to the threshold comparison results), curves (time series curves that display historical data trends), control sliders (draggable slider controls), control buttons, alarm prompt modules (triggering audible and visual alarms when sensor values ​​exceed thresholds), etc., and automatically binds the sensor data with the corresponding visualization elements (binding DDS Topic and QML attributes, where DDS is the Data Distribution Service protocol and Topic refers to the DDS data channel name), realizing plug-and-play display. The multi-sensor plug-in module also refreshes the visualization configuration window in real time at preset time intervals when the window is not refreshed. Here, RadarTab refers to the tag control for the radar, and GasTab refers to the tag control for the gas sensor.

[0055] In related technologies, ground station software uses static compilation, requiring the entire project to be recompiled for each new window. This embodiment, however, moves the business logic to the multi-sensor plugin module libMultiSensorPlugin.so, which loads the plugin at runtime using QLibrary::load() via the Qt plugin mechanism. The main ground station software program does not need to be recompiled, achieving hot-swapping with a single compilation. By setting sensor configuration description files and the multi-sensor plugin module automatically generating sensor Qt meta-objects corresponding to the sensor data based on these files and automatically binding them to visualization elements, adding a new sensor only requires updating the sensor configuration description file, generating a completely new visualization configuration window with zero code modifications, significantly improving sensor configuration efficiency.

[0056] Step S207: The multi-sensor plug-in module responds to the user's operation on the visual configuration window and generates a remote parameter setting command.

[0057] For example, users can input the sensor parameters that need to be adjusted through a visual configuration window, such as by dragging the control slider or clicking the control button. The multi-sensor plug-in module can generate remote parameter setting instructions based on the user's input.

[0058] In step S208, the multi-sensor plug-in module encapsulates the remote parameter setting message according to the remote parameter setting instruction, and the multi-sensor aggregation service unit sends the remote parameter setting message back to the aircraft so that the aircraft can adjust the sensor parameters according to the remote parameter setting message.

[0059] Optionally, when the multi-sensor plug-in module encapsulates the remote parameter setting message according to the remote parameter setting instruction, it first parses the remote parameter setting instruction to obtain the aircraft address, sensor parameter field name, and adjustment value corresponding to the sensor parameter field name. Then, it encapsulates the aircraft address, sensor parameter field name, and adjustment value corresponding to the sensor parameter field name into the corresponding remote parameter setting message.

[0060] For example, the multi-sensor plug-in module encapsulates remote parameter setting instructions into a remote parameter setting message MAVLINK_MSG_ID_REMOTE_PARAM_SET (message ID #360).

[0061] Optionally, when the aircraft adjusts the sensor parameters according to the remote parameter setting message, it parses the remote parameter setting message through the airborne cooperative controller and writes it into the local register to adjust the corresponding sensor parameters, such as filter coefficient adjustment, servo angle setting, or laser gain calibration.

[0062] Optionally, an ACK (Acknowledgement) feedback mechanism is also set up. When the aircraft's airborne controller successfully writes the sensor parameters, it returns an acknowledgment message MAVLINK_MSG_ID_REMOTE_PARAM_ACK, which is forwarded back to the multi-sensor plug-in module by the multi-sensor aggregation service unit, triggering the status indicator light on the visual configuration window to update, thus forming a complete parameter tuning loop.

[0063] For example, Figure 5 This is a flowchart illustrating the remote parameter setting process according to an embodiment of the present invention. The operator adjusts the slider, modifying the gain to 2.5. The QGC Plugin encapsulates the modified slider value into a message MAVLINK_MSG_ID_REMOTE_PARAM_SET (#360), which is then transmitted back to the corresponding aircraft via SensorHub. The aircraft's onboard controller receives this message and writes it into its register. Feedback is then executed through the ACK feedback mechanism, achieving a bidirectional loop of parameter tunneling from "slider value → MAVLink msg → XRCE-DDS → onboard register → ACK msg → SensorHub → QML status light," thus meeting the one-click calibration requirement. Here, msg (message) refers to a message.

[0064] This embodiment provides a unified display and remote configuration system for multiple sensors of a drone. Figure 6 This is a schematic diagram of the structure of a unified display and remote configuration system for multiple sensors of a drone according to an embodiment of the present invention, as shown below. Figure 6 As shown, the system includes: Ground station software main program (QGC ground station); The multi-sensor plugin module (libMultiSensorPlugin.so) generates a visual configuration window corresponding to the sensor data based on a sensor configuration description file in a preset format; the multi-sensor plugin module is integrated into the main program of the ground station software based on the Qt plugin mechanism; The multi-sensor aggregation service unit (SensorHub aggregation service) is used to acquire sensor data from the aircraft; the multi-sensor aggregation service unit is deployed outside the main program of the ground station software and runs independently. The multi-sensor plug-in module is also used to generate remote parameter setting instructions in response to user operations on the visual configuration window, and to encapsulate remote parameter setting messages according to the remote parameter setting instructions; The multi-sensor aggregation service unit is also used to transmit the remote parameter setting message back to the aircraft, so that the aircraft can adjust the sensor parameters according to the remote parameter setting message.

[0065] As one or more specific application embodiments of the present invention, the optimal implementation scheme or the scheme that the inventors most want to embody is described in combination with the specific application scenario.

[0066] In this embodiment, a plug-and-play three-in-one peripheral for laser-gas-drop is implemented. Specifically, the environment is first set up. For the airborne end, the lidar (CAN-FD), MEMS gas sensor (UART), and drop servo (PWM) are connected to the collaborative controller. For the ground end, QGC software 4.3.0 and the plug-in RPM package (such as the multi-sensor plug-in module libMultiSensorPlugin.so) are installed on a laptop.

[0067] Next, edit the JSON file and add the following content to the file sensor_profile.json: { "laser": { "msg_id": 230, "fields": ["dist", "intens"], "unit": ["m", "%"], "threshold": [100, 50], "color": ["#00FF00", "#FF0000"] }, "gas": { "msg_id": 231, "fields": ["ppm", "temp"], "unit": ["ppm", "°C"], "threshold": [500, 60], "color": ["#FFA500", "#FFFFFF"] }, "servo": { "msg_id": 232, "fields": ["angle", "state"], "unit": ["°", "bool"], "threshold": [90, 1], "color": ["#0080FF", "#808080"] } } After saving, restart the QGC software. The multi-sensor plugin module will automatically parse and generate 3 new Tab controls in the top bar.

[0068] Next is the online remote sensor parameter adjustment process. Powering on at time T0, the SensorHub detects three extended MAVLink frames; at T0+200ms, the multi-sensor plug-in module completes QtMetaObject registration, and the tab title, unit, and scale are automatically generated; at T0+1s, the laser ranging threshold of 78m < 100m is displayed in green, the gas level of 420ppm < 500ppm is displayed in green, and the servo motor at 0° is displayed in gray (not unlocked); at T0+5s, the operator drags the slider to 90° in the "Throw" tab control and clicks the "Set" button; the plug-in encapsulates msg#360, and after a 180ms loopback, the onboard register is updated, the servo motor rotates to 90°, and the QML status light changes from gray to green.

[0069] This embodiment achieves interface generation time of <250ms when adding a new sensor, without recompiling QGC throughout the process, and supports simultaneous refresh of 50 peripherals at a frame rate of 32Hz (tested with i7-1165G7).

[0070] In this embodiment, a cluster of 8 aircraft is used for centralized display of peripherals. First, the cluster topology is set up, including 8 real aircraft, each carrying 3 types of peripherals (laser, gas, and RTK mass), totaling 24 data streams; the SensorHub listens on 8 UDP ports (14550-14557). Here, RTK refers to Real-time Dynamic Differential Positioning, and the RTK system uses an Ethernet interface for data transmission.

[0071] Next, the operating steps are as follows: a) One-click startup of SensorHub, hash table automatically created {aircraft-ID:IP:port}; b) QGC only connects to aggregate port 5777, and a drop-down menu "Aircraft Selection" appears at the top of the interface; c) When the operator switches to “UAV-05”, the Tab displays the curves of the three types of peripherals of this machine in real time; d) When the UAV-05 gas concentration is greater than 500 ppm, the progress bar on the Tab tab changes from green to red, and an audio alarm pops up. e) The operator moves the laser gain from 1.0 to 1.5 using the “Parameter Tunnel” slider. The loopback is completed in 220ms, and the onboard register confirms the write of the ACK.

[0072] This embodiment can simultaneously refresh data from 8 machines and 24 channels at an average frame rate of 30Hz; automatically remove offline aircraft with a heartbeat timeout of 1 second; and update the QGC drop-down list in real time; reducing field maintenance time from 20 minutes of “one-machine serial port parameter tuning” to 1 minute.

[0073] In summary, this embodiment, through pluggable dynamic libraries, JSON metadata-driven architecture, and extended MAVLink aggregation, achieves for the first time in QGC zero-compilation hot-plugging of arbitrary sensor peripherals, automatic instrument page generation in less than 250ms, aggregation of data from N aircraft peripherals on a single port with 30Hz synchronous refresh, and a 300ms-level remote parameter tunnel, enabling one-click calibration without modifying the flight control firmware. Field verification in three missions—tunnel rescue, power line inspection, and fire-fighting deployment—shows that the system supports ≥50 peripherals and ≥8 aircraft clusters operating simultaneously, improving maintenance efficiency by 20 times and significantly reducing model compatibility and after-sales costs. It can be widely used in military applications, emergency firefighting, low-altitude logistics, and other UAV swarm application scenarios.

[0074] This embodiment also provides a unified display and remote configuration device for multiple sensors of a UAV, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as described above. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0075] This embodiment provides a unified display and remote configuration device for multiple sensors of an unmanned aerial vehicle (UAV), applied to a ground station, such as... Figure 7 As shown, it includes: Acquisition module 701 is used to acquire sensor data of the aircraft through a multi-sensor aggregation service unit; the multi-sensor aggregation service unit is deployed at a ground station and operates independently. The visualization module 702 is used to generate a visualization configuration window corresponding to the sensor data based on a sensor configuration description file in a preset format through the multi-sensor plug-in module; the multi-sensor plug-in module is integrated into the ground station based on the Qt plug-in mechanism; The parameter setting module 703 is used to generate remote parameter setting instructions in response to user operations on the visual configuration window via the multi-sensor plug-in module. The sending module 704 is used to encapsulate the remote parameter setting message according to the remote parameter setting instruction through the multi-sensor plug-in module, and send the remote parameter setting message back to the aircraft through the multi-sensor aggregation service unit, so that the aircraft can adjust the sensor parameters according to the remote parameter setting message.

[0076] In one optional implementation, the multi-sensor aggregation service unit is used to listen to custom sensor messages of the aircraft through the User Datagram Protocol network, process the custom sensor messages to obtain sensor data, and forward the sensor data to the local listening interface of the ground station. This acquisition module is also used for: The system receives sensor data sent by the multi-sensor aggregation service unit through a local listening interface.

[0077] In one alternative implementation, the visualization module is further used for: This multi-sensor plug-in module parses the sensor data based on the sensor configuration description file and generates a sensor configuration file. Register the sensor Qt meta-object according to the sensor configuration file; A visualization configuration window corresponding to the sensor data is generated based on the Qt meta-object of the sensor.

[0078] In one alternative implementation, the sensor configuration description file includes at least one of the following: Sensor identification name; The custom sensor message identifier corresponding to the sensor identifier name; Sensor parameter field names; The physical units corresponding to the sensor parameter field names; The display threshold corresponding to the sensor parameter field name; The color code corresponding to this display threshold.

[0079] In one optional implementation, the multi-sensor aggregation service unit is further configured to maintain an aircraft identifier-aircraft address-User Datagram Protocol (UDP) port relationship table, and associate the monitored custom sensor messages with the aircraft identifier, aircraft address, and UDP port according to the aircraft identifier-aircraft address-UDP port relationship table.

[0080] In an optional implementation, the sending module is further configured to: Parse the remote parameter setting command to obtain the aircraft address, sensor parameter field name, and the adjustment value corresponding to the sensor parameter field name; The aircraft address, sensor parameter field name, and the corresponding adjustment value are encapsulated into a remote parameter setting message.

[0081] The UAV multi-sensor unified display and remote configuration device provided in this embodiment of the invention can execute the UAV multi-sensor unified display and remote configuration method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0082] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0083] The following is a detailed reference. Figure 8 The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from memory 808 into a random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device. The processor 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0084] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0085] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a memory 808, or installed from a ROM 802. When the computer program is executed by the processor 801, it performs the functions defined in the UAV multi-sensor unified display and remote configuration method of the embodiments of the present invention.

[0086] Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0087] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the UAV multi-sensor unified display and remote configuration method shown in the above embodiments is implemented.

[0088] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0089] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the invention.

Claims

1. A method for unified display and remote configuration of multiple sensors in a UAV, characterized in that, Applied to ground stations, the method includes: Sensor data from the aircraft is acquired through a multi-sensor aggregation service unit; the multi-sensor aggregation service unit is deployed at a ground station and operates independently. The multi-sensor plug-in module generates a visual configuration window corresponding to the sensor data based on a sensor configuration description file in a preset format; the multi-sensor plug-in module is integrated into the ground station based on the Qt plug-in mechanism. The multi-sensor plug-in module responds to user actions on the visual configuration window and generates remote parameter setting instructions. The multi-sensor plug-in module encapsulates the remote parameter setting message according to the remote parameter setting instruction, and the multi-sensor aggregation service unit transmits the remote parameter setting message back to the aircraft so that the aircraft can adjust the sensor parameters according to the remote parameter setting message.

2. The method according to claim 1, characterized in that, The multi-sensor aggregation service unit is used to listen to the aircraft's custom sensor messages through the User Datagram Protocol (UDP) network, process the custom sensor messages to obtain sensor data, and forward the sensor data to the local listening interface of the ground station. The acquisition of aircraft sensor data through the multi-sensor aggregation service unit includes: The system receives sensor data sent by the multi-sensor aggregation service unit through a local listening interface.

3. The method according to claim 1 or 2, characterized in that, The step of generating a visual configuration window corresponding to the sensor data through a multi-sensor plug-in module based on a preset format sensor configuration description file includes: The multi-sensor plug-in module parses the sensor data according to the sensor configuration description file to generate a sensor configuration file. Register the sensor Qt meta-object according to the sensor configuration file; A visualization configuration window corresponding to the sensor data is generated based on the sensor's Qt meta-object.

4. The method according to claim 3, characterized in that, The sensor configuration description file includes at least one of the following: Sensor identification name; The custom sensor message identifier corresponding to the sensor identifier name; Sensor parameter field names; The physical units corresponding to the sensor parameter field names; The display threshold corresponding to the sensor parameter field name; The color code corresponding to the display threshold.

5. The method according to claim 1, characterized in that, The multi-sensor aggregation service unit is also used to maintain a relationship table between aircraft identifier, aircraft address, and user data packet protocol port, and associate the monitored custom sensor messages with the aircraft identifier, aircraft address, and user data packet protocol port according to the relationship table.

6. The method according to claim 5, characterized in that, The step of encapsulating the remote parameter setting message according to the remote parameter setting instruction includes: The remote parameter setting command is parsed to obtain the aircraft address, sensor parameter field name, and adjustment value corresponding to the sensor parameter field name; The aircraft address, sensor parameter field name, and the adjustment value corresponding to the sensor parameter field name are encapsulated into a corresponding remote parameter setting message.

7. A unified display and remote configuration device for multiple sensors of an unmanned aerial vehicle (UAV), characterized in that, Applied to ground stations, the device includes: The acquisition module is used to acquire sensor data of the aircraft through a multi-sensor aggregation service unit; the multi-sensor aggregation service unit is deployed at a ground station and operates independently. The visualization module is used to generate a visualization configuration window corresponding to the sensor data based on a sensor configuration description file in a preset format through the multi-sensor plug-in module; the multi-sensor plug-in module is integrated into the ground station based on the Qt plug-in mechanism; The parameter setting module is used to generate remote parameter setting instructions in response to user operations on the visual configuration window via the multi-sensor plug-in module; The sending module is used to encapsulate a remote parameter setting message according to the remote parameter setting instruction through the multi-sensor plug-in module, and to send the remote parameter setting message back to the aircraft through the multi-sensor aggregation service unit, so that the aircraft can adjust the sensor parameters according to the remote parameter setting message.

8. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the UAV multi-sensor unified display and remote configuration method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the unmanned aerial vehicle (UAV) multi-sensor unified display and remote configuration method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions, which are used to cause a computer to execute the unmanned aerial vehicle (UAV) multi-sensor unified display and remote configuration method as described in any one of claims 1 to 6.

Images