A drone networking communication system

Abstract

The application relates to a UAV networking communication system and belongs to the technical field of unmanned aerial vehicles, which solves the problems of lacking multi-link fusion communication and inflexible streaming media analysis in the prior art. The UAV networking communication system comprises a UAV node and a ground node, wherein the ground node comprises a handheld terminal, a cooperative data link terminal and an antenna; the data link management and control unit is arranged in the handheld terminal and is used for bidirectional interactive communication with the cooperative data link terminal; the data link management and control unit comprises a link management module and a streaming media management module; the link management module is used for managing the UAV node of the narrowband and broadband links, constructing the UAV network in the narrowband and broadband links according to a routing algorithm, synchronizing the node time in the UAV network and sending a parameter configuration command according to a parameter configuration mode; the streaming media management module is used for pushing the real-time audio and video collected in the handheld terminal to a target through the cooperative data link terminal and playing the pulled audio and video in the handheld terminal. The UAV networking communication system is efficient and convenient.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more particularly to a UAV networking communication system. Background Technology

[0002] With the rapid development of drones, the national demand for them is also gradually increasing. Drones are indispensable in various fields such as industry, agriculture, military, and logistics. The efficient collaboration of multiple drones is gradually limiting traditional drone communication methods. To acquire communication information from aerial targets, drone networking communication has become urgent and crucial. Quickly, accurately, and efficiently acquiring the location and status information of aerial targets plays a vital role in understanding battlefield tactical decisions and deployments.

[0003] In existing technologies, single-link communication is common for drones, network communication is not flexible enough, and it is highly dependent. It is not possible to effectively control drones on different links in the communication link, and there is a lack of application of multi-link converged communication technology. Human-machine collaborative data link networking is generally centralized, and the system has low scalability.

[0004] Traditional multi-hop transmission technologies primarily employ routing algorithms in scenarios with fixed infrastructure to achieve network topology communication. However, in ad hoc networks, the network topology changes dynamically, and nodes frequently join and leave the network, causing the availability and effectiveness of routing paths to fluctuate. Traditional routing algorithms cannot adapt to these frequent topology changes.

[0005] Moreover, existing drones use a single, inflexible streaming media communication protocol, which makes it impossible to quickly and accurately parse the data to complete push or playback. Summary of the Invention

[0006] Based on the above analysis, the embodiments of the present invention aim to provide a drone networking communication system to solve the problems of existing drone networking lacking multi-link converged communication and inflexible streaming media parsing.

[0007] This invention provides a UAV networking communication system, including: UAV nodes and ground nodes, wherein the ground nodes include: a handheld terminal, a collaborative data link terminal, and an antenna; the handheld terminal is equipped with a data link management and control unit, which performs bidirectional interactive communication with the collaborative data link terminal;

[0008] The data link control unit includes a link management module and a streaming media management module. The link management module is used to manage drone nodes in narrowband and broadband links, construct drone networks in narrowband and broadband links according to routing algorithms, synchronize the time of nodes in the drone network according to time base level, and send parameter configuration commands according to parameter configuration mode.

[0009] The streaming media management module is used to push the audio and video captured in real time on the handheld terminal to the target via the collaborative data link terminal, and to play the retrieved audio and video on the handheld terminal.

[0010] Based on further improvements to the above system, the link management module constructs UAV networks in narrowband and broadband links according to routing algorithms, including: constructing UAV networks in narrowband links according to flooding algorithms and DSDV routing algorithms; and constructing UAV networks in broadband links according to flooding algorithms and link-state routing algorithms.

[0011] Based on further improvements to the above system, each drone node in the drone network corresponds to a routing table. When the target sequence number in the routing table is even, it means that the target node is reachable; when the target sequence number is odd, it means that the target node is unreachable, and the hop count is set to infinity.

[0012] Based on further improvements to the above system, the collaborative data link terminal can manage one or more drone networks and track data packets that have been flooded in the drone network.

[0013] Based on further improvements to the above system, the link management module synchronizes the node time in the UAV network according to the time base level, including:

[0014] Calculate the hop count between each node in the UAV network and the master node, and use it as the initial local time base level, with the master node having the highest time base level, which is level 0;

[0015] The master node sends the first time reference information upon power-on, including node ID, time reference level, time frame, time element, and time slot. Non-master nodes in the UAV network update their local node time based on the received first time reference information.

[0016] In a drone network, non-controlling nodes send second time reference information. After receiving multiple second time reference information, each node updates its local time based on the second time reference information that is at a higher level than its local time reference.

[0017] Based on further improvements to the above system, the collaborative data link terminal preloads multiple time slot tables and calculates the task rate of the UAV nodes according to the data traffic received from the data link management and control unit. Each UAV node summarizes its task rate to the master control node, which calculates the task rate ratio of each UAV node, subtracts it from the task rate ratio of the corresponding UAV node in each time slot table, selects the time slot table with the smallest difference, and sends it to the non-master control nodes.

[0018] Based on further improvements to the above system, the parameter configuration modes include temporary and permanent effects. Temporary effects consist of two stages: receiving and executing. The configured parameters are only valid in the current power-on state. Permanent effects consist of three stages: receiving, storing, and executing. The configured parameters are stored in memory. In each stage, the receiver immediately provides feedback after receiving the command, and the sender sends the command for the next stage after receiving the feedback.

[0019] Based on further improvements to the above system, the streaming media management module pushes the audio and video collected in real time from the handheld terminal to the target through the collaborative data link terminal. This includes: obtaining the target's streaming address; preprocessing the collected video and audio; creating a streaming thread; encoding and encapsulating the audio and video according to the video transmission protocol to form audio and video stream message, and pushing it to the target's streaming address in a loop until there is no audio or video stream, at which point the streaming thread ends.

[0020] Based on further improvements to the above system, the streaming media management module plays the retrieved audio and video on the handheld terminal, including: selecting to play audio and video; when the playback address is valid, obtaining the message of the audio and video stream; creating a playback thread; decoding the message according to the video transmission protocol; cyclically reading the decoded video and audio; performing video and audio synchronization; and transmitting them to the playback device of the handheld terminal respectively, until there is no more video and audio, and ending the playback thread.

[0021] Based on further improvements to the above system, a message synchronization code field is added to the message to store the video transmission protocol.

[0022] Compared with existing technologies, the present invention can achieve at least one of the following beneficial effects: It enables converged communication of narrowband and broadband links through time synchronization and time slot resource allocation; it employs different dynamic routing algorithms for narrowband and broadband to support interconnection between any nodes and quickly update routes in cases such as node entry and exit from the network; it improves the reliability of parameter configuration through a multi-stage, multiple-handshake protocol; it supports adaptive processing of multiple protocols to improve audio and video parsing efficiency; and it manages the data sent and received by each node in the UAV collaborative data link through a handheld terminal, realizing an efficient and convenient UAV networking communication system.

[0023] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0024] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0025] Figure 1 This is a schematic diagram of a UAV networking communication system in an embodiment of the present invention. Detailed Implementation

[0026] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0027] A specific embodiment of the present invention discloses a network communication system for unmanned aerial vehicles (UAVs), such as... Figure 1 As shown, it includes: UAV nodes and ground nodes, wherein the ground nodes include: handheld terminals, collaborative data link terminals and antennas; the handheld terminals are equipped with data link management and control units, which communicate bidirectionally with the collaborative data link terminals;

[0028] The data link control unit includes a link management module and a streaming media management module. The link management module is used to manage drone nodes in narrowband and broadband links, construct drone networks in narrowband and broadband links according to routing algorithms, synchronize the time of nodes in the drone network according to time base level, and send parameter configuration commands according to parameter configuration mode.

[0029] The streaming media management module is used to push the audio and video captured in real time on the handheld terminal to the target via the collaborative data link terminal, and to play the retrieved audio and video on the handheld terminal.

[0030] During implementation, a handheld terminal is used to efficiently and conveniently manage the data sent and received by each node in the UAV collaborative data chain, effectively ensuring the basic needs of each node for narrowband control data and broadband image data transmission, and enabling UAVs to network and complete inter-device communication.

[0031] It should be noted that the data link management unit deployed in the handheld terminal communicates bidirectionally with the baseband processing module of the collaborative data link terminal and the link access and management software residing within its FPGA chip via Ethernet or serial port, or wireless connection. The link access and management software is responsible for loading and configuring the underlying drivers of the collaborative data link terminal's processing module.

[0032] The handheld terminal can also connect to other control devices via network, such as network cameras and external GPS devices.

[0033] Compared to existing technologies, the system in this embodiment independently maintains both narrowband and broadband networks. A link management module manages the drone nodes on both narrowband and broadband links, allowing for adjustments to the network of a drone node based on actual needs, thus achieving multi-link converged communication. For example, if ground video needs to be transmitted to drone node A, but it is on a narrowband network with limited data transmission resources, drone A can be moved to a broadband network to receive the video, and then switched back to the narrowband network.

[0034] When drone nodes join or leave the network, or when a node fails or malfunctions, the multi-hop transmission paths of each node in the network are updated to maintain overall network communication. Specifically, a drone network in narrowband links is constructed based on the flooding algorithm and the DSDV (destination-sequenced distance-vector) routing algorithm; a drone network in broadband links is constructed based on the flooding algorithm and the link-state routing algorithm. A collaborative data link terminal manages one or more drone networks.

[0035] In the constructed drone network, each drone node corresponds to a routing table, including the sequence numbers of all available target nodes, the sequence number of the next-hop node to reach the target node, and the hop count to reach the target node. When a new drone node joins the network, it broadcasts its information to its neighbors. Upon receiving this information, neighboring nodes update their routing tables and recalculate the paths and hop counts to reachable nodes from their drone nodes. An even-numbered target node sequence number in the routing table indicates that the target node is reachable; an odd-numbered target node indicates that the target node is unreachable, and the hop count is set to infinity. Any update to the routing table by a drone node will be broadcast to its neighbors to update the routing status of the entire drone network. The routing table is sent and forwarded by the underlying links and then resolved by the network layer.

[0036] To avoid generating a large number of duplicate data packets in the drone network, the flooding process was appropriately suppressed. The collaborative data link endpoints tracked data packets that had already been flooded within the managed drone network to prevent duplicate transmission.

[0037] Furthermore, due to the entry and exit of drone nodes from the network, the time synchronization problem across multiple links needs to be addressed. In the link management module, the time of nodes in the drone network is synchronized according to the time base level, including:

[0038] The number of hops between each node in the UAV network and the master node is calculated as the initial local time base level, with the master node having the highest time base level, which is level 0; that is, the further the UAV node is from the master node, the lower its time base level.

[0039] After the master node powers on, it sends the first time reference information, including node ID, time reference level, time frame, time element and time slot. Non-master nodes in the UAV network update their local node time based on the received first time reference information.

[0040] In the drone network, non-controlling nodes send second time reference information. After receiving multiple second time reference information, each node selects the second time reference information that is higher than the local time reference level and has the highest level, and updates the local node time.

[0041] Furthermore, multiple time slot allocation schemes and time slot tables are set up for narrowband and broadband links respectively, including: average allocation among nodes, dynamic allocation via resource pool, and manual allocation. For example, the narrowband link is mainly used for control information exchange and task coordination between ground nodes and the UAV network. In narrowband mode, the ground node and seven aerial UAV nodes share 120kbps data bandwidth. Under the average allocation scheme, each node occupies 15Kbps of transmission bandwidth. Under the manual allocation scheme, the time slot table is selected based on the data type: when transmitting image and video data, a time slot table with higher bandwidth is selected; when transmitting control parameter data, a time slot table with lower bandwidth is selected.

[0042] After the collaborative data link terminal starts up, it preloads multiple time slot tables. When dynamic allocation of resource pool is set, the collaborative data link terminal calculates the task rate of the drone nodes based on the data traffic received from the data link management unit. Each drone node summarizes its task rate to the master control node, which calculates the task rate ratio of each drone node, subtracts it from the task rate ratio of the corresponding drone node in each time slot table, selects the time slot table with the smallest difference, and sends it to the non-master control nodes.

[0043] It should be noted that the above time slot allocation scheme is configured through the parameter configuration module in the data link management unit. The parameters configured by this module include: network parameters, radio frequency parameters, and interface parameters. The network parameters include the device operating mode, operating frequency, node number, time slot table number, and occupied time slot number of the local or remote device; the radio frequency parameters include the device transmit power and other radio frequency parameters of the local or remote device; and the interface parameters include the network port and serial port parameters of the local or remote device.

[0044] It should be noted that remote devices interact remotely via wireless links, including neighbor nodes and multi-hop neighbor nodes. Since variations in the wireless channel itself may lead to occasional packet loss, and the parameters of the UAV platform equipment are not easily maintained via wired connections, this embodiment prioritizes the use of narrowband data transmission channels for parameter configuration command transmission in narrowband mode and in a hybrid mode where both narrowband and broadband exist, providing higher reliability than broadband links.

[0045] Furthermore, the parameter configuration modes include temporary and permanent effects. Temporary effect comprises two phases: receiving and execution, with configured parameters only valid in the current power-on state. Permanent effect comprises three phases: receiving, storing, and executing, with configured parameters stored in memory. It is important to note that in each phase, the receiver immediately responds upon receiving the command, and the sender sends the command for the next phase only after receiving the response. That is, each phase is based on a two-way handshake protocol; temporary effect involves four handshakes, while permanent effect involves six.

[0046] Preferably, the node's state is restored by setting a parameter reset.

[0047] It should be noted that the streaming media management module in the system is mainly responsible for audio and video processing. It adopts FFmpeg audio and video technology and covers video transmission based on H264, RTSP, RTMP and RTP protocols, realizing functions such as audio and video encoding, decoding, transcoding and playback.

[0048] Specifically, the streaming media management module pushes the audio and video collected in real time from the handheld terminal to the target through the collaborative data link terminal. This includes: obtaining the target's push address, preprocessing the collected video and audio; creating a push thread, encoding and encapsulating the audio and video according to the video transmission protocol to form audio and video stream message, and pushing it to the target's push address in a loop until there is no more audio and video stream, at which point the push thread ends.

[0049] The streaming media management module plays the retrieved audio and video on the handheld terminal, including: selecting audio and video to play; when the playback address is valid, obtaining the message of the audio and video stream; creating a playback thread; decoding the message according to the video transmission protocol; cyclically reading the decoded video and audio; performing video and audio synchronization; and transmitting them to the playback device of the handheld terminal respectively, until there is no more video and audio, at which point the playback thread ends.

[0050] During the transmission of video push or playback, a message synchronization code field is added to the message to store the video transmission protocol, which facilitates quick and accurate parsing to complete the push or playback.

[0051] Based on the streaming media management module, ground-to-air broadband video transmission communication and air-to-air broadband video transmission communication are implemented. Specifically, ground-to-air broadband video transmission communication includes: pushing video from ground nodes to airborne UAV nodes, and then transmitting it to the rear command and control station via satellite communication; airborne UAV nodes can relay video between two ground nodes, or relay video from other airborne UAV nodes to ground nodes for monitoring, achieving long-distance video monitoring through multi-hop relay. For example, during disaster relief operations, on-site video collected by ground nodes is pushed to UAV nodes, and then transmitted back to the rear command and control station by the UAV nodes.

[0052] Air-to-air broadband video transmission communication includes: sharing video among multiple aerial drone nodes, and using nodes equipped with satellite communication equipment to transmit designated video back to the rear command and control station or manned aircraft for monitoring.

[0053] It should be noted that the link access and management software in the collaborative data link terminal collects the operating status information of the hardware devices in the communication link and reports it to the data link management and control unit. The data link management and control unit also includes a status monitoring module, which is responsible for monitoring the status of the collaborative task data link, including: device working status, situation display, network topology, and self-test results.

[0054] Specifically, the device operation status monitoring hardware monitors information such as the operating mode, operating status (voltage, current, temperature), and broadband (narrowband) link rate of the devices; the situation display combines a map to show the node location, node trajectory, node label, path planning, and other information of ground nodes and UAV nodes; the network topology provides an overview of the device nodes in the network; and the self-test results display the voltage, temperature, AD / DA status of the baseband module, memory status, overcurrent status of the RF module, and VSWR status of local and remote devices.

[0055] Compared with existing technologies, this embodiment achieves converged communication of narrowband and broadband links through time synchronization and time slot resource allocation; it adopts different dynamic routing algorithms for narrowband and broadband to support interconnection between any nodes and quickly update routes in cases such as node entry and exit from the network; it improves the reliability of parameter configuration through a multi-stage, multiple-handshake protocol; it supports adaptive processing of multiple protocols to improve audio and video parsing efficiency; and it manages the data sent and received by each node in the UAV collaborative data link through a handheld terminal, realizing an efficient and convenient UAV networking communication system.

[0056] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0057] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A network communication system for unmanned aerial vehicles (UAVs), characterized in that, include: The drone node and the ground node, wherein the ground node includes: a handheld terminal, a collaborative data link terminal and an antenna; the handheld terminal is equipped with a data link management and control unit, which performs bidirectional interactive communication with the collaborative data link terminal. The data link management unit includes a link management module and a streaming media management module. The link management module manages UAV nodes on narrowband and broadband links, adjusts the networks where UAV nodes are located, and constructs UAV networks on narrowband and broadband links according to different dynamic routing algorithms. It also synchronizes the time of nodes in the UAV network according to a time base level and sends parameter configuration commands according to a parameter configuration mode. Constructing UAV networks on narrowband and broadband links according to different dynamic routing algorithms includes: constructing UAV networks on narrowband links according to flooding and DSDV routing algorithms; constructing UAV networks on broadband links according to flooding and link-state routing algorithms; and synchronizing the time of nodes in the UAV network according to a time base level, including: calculating the hop count of each node in the UAV network from the master node as the initial local time base level, where the master node's time... The highest reference level is 0. The master node sends first time reference information upon power-on, including node ID, time reference level, time frame, time element, and time slot. Non-master nodes in the UAV network update their local node time based on the received first time reference information. Non-master nodes in the UAV network send second time reference information. After receiving multiple second time reference information messages, each node updates its local node time based on the second time reference information message with a higher level than its local time reference level. The parameter configuration modes include temporary and permanent effects. Temporary effects consist of two stages: receiving and execution, and the configured parameters are only valid in the current power-on state. Permanent effects consist of three stages: receiving, storing, and executing, and the configured parameters are stored in memory. In each stage, the receiver immediately provides feedback upon receiving the command, and the sender sends the next stage command after receiving the feedback. The streaming media management module is used to push real-time audio and video collected in the handheld terminal to the target through a collaborative data link terminal, and to play the retrieved audio and video on the handheld terminal; it is also used to support ground-to-air broadband video transmission communication and air-to-air broadband video transmission communication; wherein, ground-to-air broadband video transmission communication includes: pushing ground node video to airborne UAV nodes, and then transmitting it to the rear command and control station using satellite communication; air-to-air broadband video transmission communication includes: sharing video among multiple airborne UAV nodes, and using nodes equipped with satellite communication equipment to transmit designated video back to the rear command and control station or manned aircraft for monitoring.

2. The UAV networking communication system according to claim 1, characterized in that, In the drone network, each drone node corresponds to a routing table. When the target sequence number in the routing table is even, it means that the target node is reachable; when the target sequence number is odd, it means that the target node is unreachable, and the hop count is set to infinity.

3. The UAV networking communication system according to claim 1, characterized in that, The collaborative data link terminal manages one or more drone networks and tracks data packets that have been flooded within the drone networks.

4. The UAV networking communication system according to claim 1, characterized in that, The The collaborative data link terminal preloads multiple time slot tables and calculates the task rate of the drone nodes based on the data traffic received from the data link management unit. Each drone node summarizes its task rate to the master control node, which calculates the task rate ratio of each drone node and subtracts it from the task rate ratio of the corresponding drone node in each time slot table. The time slot table with the smallest difference is selected and sent to the non-master control nodes.

5. The UAV networking communication system according to claim 1, characterized in that, The streaming media management module pushes the audio and video collected in real time from the handheld terminal to the target through the collaborative data link terminal, including: obtaining the target's push address, preprocessing the collected video and audio; creating a push thread, encoding and encapsulating the audio and video according to the video transmission protocol to form audio and video stream message, and pushing it to the target's push address in a loop until there is no audio or video stream, at which point the push thread ends.

6. The UAV networking communication system according to claim 1, characterized in that, The streaming media management module plays the retrieved audio and video on the handheld terminal, including: selecting to play audio and video; when the playback address is valid, obtaining the message of the audio and video stream; creating a playback thread; decoding the message according to the video transmission protocol; cyclically reading the decoded video and audio; performing video and audio synchronization; and transmitting them to the playback device of the handheld terminal respectively, until there is no more video and audio, and ending the playback thread.

7. The UAV networking communication system according to claim 5 or 6, characterized in that, The message message includes a message synchronization code field, which is used to store the video transmission protocol.

Images