An on-board computer for a drone and a drone control system

Abstract

The application discloses an on-board computer for a UAV and a UAV control system, the on-board computer comprising an on-board host computer, a controllable power supply expansion module, a switch module and an HDMI acquisition module; the controllable power supply expansion module is connected with the on-board host computer, the HDMI acquisition module and the switch module respectively; the HDMI acquisition module is connected with the switch module through an RJ45 network cable, and the switch module is connected with the on-board host computer through an RJ45 network cable; an HDMI interface of the HDMI acquisition module is connected with an on-board camera through an HDMI cable; the on-board computer is light and compact, has rich interfaces, can be matched with the expansion module to realize interface expansion, supports video transmission by using a UDP protocol, and reduces transmission delay from above 10s to below 5s; the on-board computer supports transmission free switching of multiple links; the on-board computer supports detection of the health conditions of each network through a heartbeat packet by establishing a virtual network, and automatically uses one or more available networks to transparently and imperceptibly realize application on the side; video pictures can be fused and sent to an on-board ground station.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to an airborne computer and a control system for unmanned aerial vehicles (UAVs). Background Technology

[0002] While the functional modules of personal computers and handheld mobile devices are mostly fixed, the onboard computers on drones are different, requiring functional customization based on specific mission requirements. Due to the different application scenarios of drone onboard computers, the focus of main control chip selection also differs. Research has found that computers used in the aerospace field typically employ main control chips with very low clock speeds (compared to personal computers). This is partly due to limitations in power consumption and chip radiation resistance, and partly because drone onboard computers do not actually require high computing power, but rather a large number of communication interfaces and I / O ports. Furthermore, a crucial requirement for computers in the aerospace field is high reliability. Theoretically, under the same external conditions (environmental conditions, manufacturing processes, etc.), the more transistors integrated into a chip, the lower its reliability. Therefore, a design principle for drone onboard computers is to just meet the requirements (just enough), and the overall hardware system should be as simple as possible to reduce the possibility of system failure.

[0003] Therefore, there is an urgent need to develop an airborne computer and drone control system for drones to meet application requirements. Summary of the Invention

[0004] One objective of this invention is to provide a new technical solution for a gateway device that supports the aggregation of multi-source heterogeneous sensing data and access to multiple networks.

[0005] According to a first aspect of the present invention, an airborne computer for an unmanned aerial vehicle (UAV) is provided, comprising: an airborne host, a controllable power supply expansion module, a switch module, and an HDMI acquisition module; the controllable power supply expansion module is connected to the airborne host, the HDMI acquisition module, and the switch module respectively to supply power to the airborne host, the HDMI acquisition module, and the switch module; the HDMI acquisition module is connected to the switch module via an RJ45 network cable, and the switch module is connected to the airborne host via an RJ45 network cable to achieve data interaction; the HDMI interface of the HDMI acquisition module is connected to an airborne camera via an HDMI cable.

[0006] Optionally, the airborne host includes: a processor, a CPU unit, a GPU unit, a memory unit, an eMMC storage unit, a network interface, a USB interface, an I / O interface, an HDMI and debugging interface, and a power interface connected to the processor. The airborne host is connected to the controllable power expansion module through the power interface, and the airborne host is connected to the switch module through the network interface.

[0007] Optionally, the onboard host uses an NVIDIA Jetson Xavier NX processor, and the onboard host uses a 6-core NVIDIA Carmel CPU. The onboard host uses a 64-bit CPU and a 384-core NVIDIA Volta GPU. TM GPU@1100MHz with 48Tensor Cores;

[0008] Alternatively, the onboard host may use an NVIDIA Jetson Nano processor, a 4-core ARM Cortex-A57 CPU, and a 128-core NVIDIA Maxwell GPU. TM GPU @ 921MHz.

[0009] Optionally, the controllable power expansion module is a 2S power expansion module or a 12S power expansion module. The power input voltage range of the 2S power expansion module is 7V-26V, and the power input voltage range of the 12S power expansion module is 15V-60V.

[0010] According to a second aspect of the present invention, a drone control system is provided, comprising: an onboard computer for a drone as described in the first aspect of the present invention, a drone flight control device, an onboard camera, and an onboard ground station, wherein the onboard computer for the drone is connected to the onboard camera and the drone flight control device respectively to acquire real-time video stream data and drone flight control data, and the onboard computer for the drone sends the processed fused video data to the onboard ground station.

[0011] Optionally, the system further includes an image transmission communication link consisting of an image transmission sky terminal and an image transmission ground terminal. The onboard computer for the UAV sends the processed fused video data to the image transmission ground terminal through the image transmission sky terminal, and then transmits it to the onboard ground station through the image transmission ground terminal.

[0012] Optionally, the image transmission sky terminal is connected to the network port of the switch module of the onboard computer for the UAV via a video interface, and the power interface of the image transmission sky terminal is connected to the power output interface of the controllable power expansion module of the onboard computer for the UAV.

[0013] Optionally, the network interface of the image transmission ground terminal is connected to the airborne ground station via a GH port to Ethernet cable, and the power input interface of the image transmission ground terminal is directly connected to a DC power supply.

[0014] Optionally, the image transmission ground terminal is a portable mobile terminal, and the image transmission ground terminal is also equipped with a remote control signal input interface, a data transmission output interface, and a power output interface.

[0015] Optionally, the system also includes multiple video application terminals, and the airborne ground station forwards the received fused video data to the multiple video application terminals for application processing.

[0016] According to an embodiment disclosed in this invention, the following beneficial effects are achieved:

[0017] The airborne computer for UAVs of this invention is a high-performance embedded airborne computer. It is compact in size, has rich interfaces, and can be expanded with expansion modules. It supports video transmission using the UDP protocol, reducing transmission latency from more than 10 seconds to less than 5 seconds, providing valuable time for emergency response. It supports free switching between multiple transmission links. It supports the establishment of virtual networks and the detection of the health status of each network through heartbeat packets, automatically using one or more available networks, making it transparent and imperceptible to the application side. Video and images can be merged and sent to the airborne ground station.

[0018] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0020] Figure 1 This is a structural block diagram of an airborne computer for a drone according to an embodiment;

[0021] Figure 2 This is a hardware wiring diagram of an onboard computer for a drone according to an embodiment. Figure 1 ;

[0022] Figure 3 This is a hardware wiring diagram of an onboard computer for a drone according to an embodiment. Figure 2 ;

[0023] Figure 4 This is a structural block diagram of an unmanned aerial vehicle (UAV) control system according to an embodiment.

[0024] Figure 5 This is a schematic diagram of the structure of a ground terminal for image transmission in an unmanned aerial vehicle (UAV) control system according to an embodiment. Detailed Implementation

[0025] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0026] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0027] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0028] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0029] Example 1:

[0030] See Figure 1-3 As shown, this embodiment provides an airborne computer for a drone, including: an airborne host, a controllable power supply expansion module, a switch module, and an HDMI acquisition module; the controllable power supply expansion module is connected to the airborne host, the HDMI acquisition module, and the switch module respectively to supply power to the airborne host, the HDMI acquisition module, and the switch module; the HDMI acquisition module is connected to the switch module via an RJ45 network cable, and the switch module is connected to the airborne host via an RJ45 network cable to realize data interaction; the HDMI interface of the HDMI acquisition module is connected to the airborne camera via an HDMI cable.

[0031] It should be noted that the airborne host in this embodiment has high-load parallel computing and complex graphics processing capabilities, and can be widely used in edge AI applications. As a video transmission edge computing platform, the airborne host is responsible for receiving UAV flight control data and real-time video streams, performing video fusion, and forwarding it to the airborne ground station. Simultaneously, the airborne host provides multiple interfaces to support the connection of different expansion modules, meeting various UAV video transmission application scenarios.

[0032] In this embodiment, the controllable power extension module supplies power to the HDMI acquisition module, the switch module, and the airborne host respectively; the HDMI acquisition module is connected to the switch module via RJ45 network cables, and the airborne host network port is also connected to the switch via RJ45 network cables to achieve information interaction; the two HDMI ports on the back of the HDMI acquisition module are connected to the airborne camera via HDMI cables.

[0033] In this embodiment, the switch module is miniature. It learns the port location and source address by reading complete data frames. The source address port table is thus generated and maintained, thereby restricting network communication to the ports related to the current transmission. Since synchronous transmission is automatically initialized on these ports, the table values ​​are also automatically refreshed according to changes in field wiring. If a frame with an unknown destination is received on one port, all other ports will also receive this frame.

[0034] In this embodiment, the HDMI acquisition module is connected to the airborne camera. It is a fully digital image and sound transmission interface that can transmit uncompressed audio and video signals.

[0035] See Figure 2-3 As shown, in this embodiment, the airborne host, HDMI acquisition module, and switch module are respectively connected to the expandable power module via corresponding XT30 power extension cables. The appropriate 2S or 12S power extension module is selected based on whether the drone's power output is 2S or 12S. The airborne camera is connected to the HDMI acquisition module via a video acquisition HDMI cable.

[0036] Additionally, it should be noted that the onboard computer used in this embodiment for the drone is designed with lightweight materials, which allows it to be compact in size and have a novel appearance.

[0037] Optionally, the onboard computer for the UAV in this embodiment includes an onboard host: a processor, a CPU unit, a GPU unit, a memory unit, an eMMC storage unit, a network interface, a USB interface, an I / O interface, an HDMI and debugging interface, and a power interface connected to the processor. The onboard host is connected to a controllable power expansion module through the power interface and to a switch module through the network interface.

[0038] Optionally, in this embodiment, the onboard computer for the UAV uses an NVIDIA Jetson Xavier NX processor and a 6-core NVIDIA Carmel CPU. The system features a 64-bit CPU and a 384-core NVIDIA Volta™ GPU at 1100MHz with 48 Tensor Cores.

[0039] Alternatively, the onboard host uses an NVIDIA Jetson Nano processor, a 4-core ARM Cortex-A57 CPU, and a 128-core NVIDIA Maxwell™ GPU at 921MHz.

[0040] Optionally, in this embodiment, the controllable power expansion module in the onboard computer for the UAV is a 2S power expansion module or a 12S power expansion module. The power input voltage range of the 2S power expansion module is 7V-26V, and the power input voltage range of the 12S power expansion module is 15V-60V.

[0041] Specifically, the onboard computer for the UAV in this embodiment comes in two versions: RACe7000 and RACe5000. The RACe7000 is equipped with an NVIDIA Jetson Xavier NX processor, possessing high-load parallel computing and complex graphics processing capabilities, and is widely used in edge AI applications. The RACe5000 is equipped with an NVIDIA Jetson Nano processor, serving as an entry-level edge AI product with excellent edge technology and graphics processing capabilities. See the table below for the specific technical parameters of the two versions:

[0042] Table 1:

[0043]

[0044]

[0045] The common problems with current UAV systems equipped with onboard computers are: 1) High latency in video transmission: Currently, in UAV scenario networks (4G, satellite communication), the latency of transmitting video collected by the UAV to the ground station is around 600ms, accompanied by packet loss; when the bandwidth is <2Mb, using the existing video transmission scheme (RTMP based on TCP), there is a video delay of about 10s in actual tests in the scenario network; 2) Single transmission method: Each network may experience circuit breakers during flight. The onboard computer for UAVs in this embodiment supports the use of the UDP protocol (User Datagram Protocol), which does not require a handshake. The application layer completes data retransmission (and may drop packets if necessary), ensuring that data transmission can be restored as quickly as possible after the network is restored, reducing latency. This transmission scheme reduces video latency to about 3-5s. The onboard computer for UAVs in this embodiment supports the establishment of virtual links on the device through Linux virtual device drivers. On the device, a custom application protocol is used to detect the link by sending packets (heartbeat packets) on different links (integrated 4G, satellite communication, and MESH communication methods) via UDP packets. The application layer is used to collect application data through virtual device communication and selects one or more appropriate paths to complete the data transmission based on the current network status.

[0046] In summary, the airborne computer for UAVs in this embodiment of the invention is a high-performance embedded airborne computer. It is compact in size, has rich interfaces, and can be expanded with expansion modules. It supports video transmission using the UDP protocol, reducing transmission latency from over 10 seconds to below 5 seconds, providing valuable time for emergency response. It supports free switching between multiple transmission links. It supports the establishment of virtual networks and detection of the health status of each network via heartbeat packets, automatically using one or more available networks, making it transparent and imperceptible to the application side. Video and images can be merged and sent to the airborne ground station.

[0047] Example 2:

[0048] See Figure 4 As shown, this embodiment provides a drone control system, including: an onboard computer for the drone as described in Embodiment 1, a drone flight control device, an onboard camera, and an onboard ground station. The onboard computer for the drone is connected to the onboard camera and the drone flight control device to obtain real-time video stream data and drone flight control data. The onboard computer for the drone sends the processed fused video data to the onboard ground station.

[0049] Optionally, the UAV control system in this embodiment also includes an image transmission communication link consisting of an image transmission sky terminal and an image transmission ground terminal, for the UAV's onboard computer to send the processed fused video data to the image transmission ground terminal through the image transmission sky terminal, and then transmit it to the onboard ground station through the image transmission ground terminal.

[0050] It should be noted that in this embodiment, a video transmission communication link (i.e., a multi-link adaptive virtual network) is established on the basis of the video transmission sky terminal and the video transmission ground terminal to realize the multi-path transmission of video data.

[0051] Optionally, see Figure 2 As shown, in the UAV control system of this embodiment, the image transmission sky terminal is connected to the network port of the switch module of the UAV's onboard computer via a video interface, and the power interface of the image transmission sky terminal is connected to the power output interface of the controllable power expansion module of the UAV's onboard computer.

[0052] Optionally, see Figure 5 As shown, in this embodiment of the UAV control system, the network interface (LAN) of the image transmission ground terminal is connected to the airborne ground station via a GH port to Ethernet cable, and the power input interface (PWR) of the image transmission ground terminal is directly connected to a DC power supply. It should be noted that in this embodiment, the power input interface of the image transmission ground terminal is directly connected to a DC power supply via an XT30 power extension cable.

[0053] Optionally, see Figure 5 As shown, in this embodiment of the UAV control system, the image transmission ground terminal is a portable mobile terminal. The image transmission ground terminal is also equipped with a remote control signal input interface (RC), a data transmission output interface (UART), and a power output interface (VCC).

[0054] Optionally, see Figure 4 As shown, the UAV control system in this embodiment also includes multiple video application terminals. The airborne ground station forwards the received fused video data to the multiple video application terminals for application processing.

[0055] In this embodiment, the UAV control system includes an airborne computer located in the air and an airborne ground station on the ground. The airborne host acts as a video transmission edge computing platform, responsible for receiving UAV flight control data from the UAV flight control device and real-time video streams from the airborne camera. After completing video fusion, it forwards the data to the airborne ground station. The airborne host also provides multiple interfaces to support the connection of different expansion modules, meeting various UAV video transmission application scenarios. The airborne ground station receives the video stream pushed by the airborne computer and forwards it to video application terminals such as command aircraft. By providing API interfaces and a visual web management backend, it facilitates users to monitor the system operation in real time, configure transmission parameters, and assist in troubleshooting anomalies. The UAV control system in this embodiment supports the use of the UDP (User Datagram Protocol), which eliminates the need for handshakes. The application layer handles data retransmission (and may drop packets if necessary), ensuring that data transmission can be restored as quickly as possible in scenarios where the network recovers, thus reducing latency. This transmission scheme reduces video latency to approximately 3-5 seconds. It supports establishing virtual links on the device via Linux virtual device drivers. A custom application protocol is used to encapsulate UDP packets, and link detection is performed on different links (integrated 4G, satellite communication, and MESH communication methods) by sending packets (heartbeat packets). The application layer collects application data through virtual device communication and selects one or more appropriate paths for data transmission based on the current network status.

[0056] In summary, the UAV control system of this invention adopts a high-performance embedded airborne computer with rich interfaces, and can be expanded with expansion modules; it supports video transmission using the UDP protocol, reducing transmission latency from over 10 seconds to below 5 seconds, providing valuable time for emergency response; it supports free switching between multiple transmission links; it supports detecting the health status of each network through heartbeat packets by establishing a virtual network, automatically using one or more available networks, making it transparent and imperceptible to the application side; and it can merge video and images and send them to the airborne ground station.

[0057] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. An onboard computer for a drone, the onboard computer comprising: The application relates to a machine vision system, which comprises the following: an onboard host computer, a controllable power supply extension module, a switch module and an HDMI acquisition module; the controllable power supply extension module is connected with the onboard host computer, the HDMI acquisition module and the switch module respectively to supply power to the onboard host computer, the HDMI acquisition module and the switch module; the HDMI acquisition module is connected with the switch module through an RJ45 network cable, and the switch module is connected with the onboard host computer through an RJ45 network cable to realize data interaction; an HDMI interface of the HDMI acquisition module is connected with an onboard camera through an HDMI cable; The onboard computer supports the establishment of a virtual link on the device through a virtual device driver of Linux, the detection of the link through packet sending on different links through a self-defined application protocol and a UDP packet. The application layer collects application data through virtual device communication, selects one or more appropriate paths according to the current state of the network and completes the transmission of the data. The application layer completes data retransmission, discards the data if necessary, ensures that the data transmission of the recovery scene can be recovered as soon as possible to reduce the delay and makes the video delay reduce to 3-5s. The onboard computer supports the establishment of a virtual link on the device through a virtual device driver of Linux, the detection of the link through packet sending on different links through a self-defined application protocol and a UDP packet. The application layer collects application data through virtual device communication, selects one or more appropriate paths according to the current state of the network and completes the transmission of the data. The application layer completes data retransmission, discards the data if necessary, ensures that the data transmission of the recovery scene can be recovered as soon as possible to reduce the delay and makes the video delay reduce to 3-5s. The onboard host computer comprises a processor, a CPU unit, a GPU unit, a memory unit, an eMMC storage unit, a network interface, a USB interface, an I / O interface, an HDMI and debugging interface and a power supply interface connected with the processor, wherein the onboard host computer is connected with the controllable power supply extension module through the power supply interface, and the onboard host computer is connected with the switch module through the network interface.

2. The onboard computer for a drone of claim 1, wherein, The processor of the onboard host computer is an NVIDIA Jetson Xavier NX, the CPU unit of the onboard host computer is a 6-core NVIDIA Carmel ARMv8.2 64-bit CPU, and the GPU unit of the onboard host computer is a 384-core NVIDIA VoltaTM GPU@1100MHz with 48 Tensor Cores.

3. The onboard computer for a drone of claim 2, wherein, Alternatively, the processor of the onboard host computer is an NVIDIA Jetson Nano, the CPU unit of the onboard host computer is a 4-core ARM Cortex-A57, and the GPU unit of the onboard host computer is a 128-core NVIDIA MaxwellTM GPU@921MHz. ​ 4. The onboard computer for a drone of claim 3, wherein, The controllable power supply expansion module is a 2S power supply expansion module or a 12S power supply expansion module, the power supply input voltage range of the 2S power supply expansion module is 7V-26V, and the power supply input voltage range of the 12S power supply expansion module is 15V-60V.

5. A drone control system, characterized in that, Comprise: The onboard computer for the unmanned aerial vehicle, the unmanned aerial vehicle flight control device, the onboard camera and the onboard ground station, the onboard computer for the unmanned aerial vehicle is connected with the onboard camera and the unmanned aerial vehicle flight control device respectively to obtain real-time video stream data and unmanned aerial vehicle flight control data, and the onboard computer for the unmanned aerial vehicle sends the processed fusion video data to the onboard ground station.

6. The UAV control system of claim 5, wherein, The system further comprises a video transmission communication link composed of a video transmission sky terminal and a video transmission ground terminal, the onboard computer for the unmanned aerial vehicle sends the processed fusion video data to the video transmission ground terminal through the video transmission sky terminal, and then the video transmission ground terminal transmits the fusion video data to the onboard ground station.

7. The drone control system of claim 6, wherein, The video transmission sky terminal is connected with the network port of the switch module of the onboard computer for the unmanned aerial vehicle through a video interface, and the power supply interface of the video transmission sky terminal is connected with the power supply output interface of the controllable power supply expansion module of the onboard computer for the unmanned aerial vehicle.

8. The drone control system of claim 7, wherein, The network interface of the video transmission ground terminal is connected with the onboard ground station through a GH port to network port line, and the power supply input interface of the video transmission ground terminal is directly connected with a direct current power supply.

9. The drone control system of claim 8, wherein, The video transmission ground terminal is a portable mobile terminal, and the video transmission ground terminal is further provided with a remote control signal input interface, a data transmission data output interface and a power supply output interface.

10. The drone control system of claim 5, wherein, The system further comprises a plurality of video application terminals, and the onboard ground station pushes the received fusion video data to the plurality of video application terminals to realize application processing.

Images