Cluster unmanned aerial vehicle communication system, device and method based on neural network

Abstract

The invention discloses a cluster unmanned aerial vehicle communication system, device and method based on a neural network, and relates to the technical field of unmanned aerial vehicle communication systems, and the system comprises a micro-control processor which is configured on an unmanned aerial vehicle and is used for executing the flight control logic of a task of the unmanned aerial vehicle and obtaining the real-time pose state and environment data of the unmanned aerial vehicle; the neural network communication device is configured to be in communication connection with the micro-control processor; the system comprises a neural network processor, and the processor is internally provided with an inference decision neural network model. The communication decision-making module is used for receiving task information and real-time pose state and environment data of the unmanned aerial vehicle, and outputting communication decision-making instructions including decisions of whether to communicate with the unmanned aerial vehicle or not, who communicates with the unmanned aerial vehicle, communication content and communication opportunity after model calculation; and the micro-control processor executes or suppresses the sending of the communication data according to the instruction. According to the invention, the most valuable information is sent to the target of the most needed communication node at the most appropriate time and place by adopting the optimal communication mode, and the communication efficiency of the cluster unmanned aerial vehicle is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle communication systems, and in particular to a cluster unmanned aerial vehicle communication system, device and method based on a neural network. BACKGROUND

[0002] A cluster unmanned aerial vehicle communication system is a key to realizing information sharing, task coordination and cooperative control among unmanned aerial vehicles, and needs to take into account cost, power consumption and cooperative efficiency, and mainly includes master-slave architecture and distributed architecture modes. Small and medium-sized cluster unmanned aerial vehicles usually adopt a distributed architecture mode, and the cluster communication protocol is mainly represented by the DDS protocol or the Micro XRCE-DDS, DDS-RTPS and other communication protocols derived from the DDS protocol architecture. However, for the core hardware of small and medium-sized unmanned aerial vehicles, the processing capability of the micro control chip (MCU) is mainly used, and the deployment and configuration of the Micro XRCE-DDS, DDS-RTPS and other protocols are relatively complex, and the occupation and consumption of running resources are large, which can easily lead to an increase in transmission delay and affect communication quality, especially in low-bandwidth or weak network environments.

[0003] With the enrichment of the functional performance and application scenarios of cluster unmanned aerial vehicles, the data transmission volume between single machines in the cluster unmanned aerial vehicles and between single machines and telemetry stations has increased dramatically, and under the development trend of diversification and miniaturization of cluster unmanned aerial vehicles, the cluster communication is easily affected by electromagnetic interference, channel conditions and changes in geography and topography, as well as the communication link interruption, topology changes and Doppler effect caused by frequent and rapid movement and formation control dominated by diversified task requirements. Many systems mainly use preset value communication strategies and other fixed logic methods for communication, which lack flexibility and have low communication efficiency. In order to improve communication efficiency, the Micro XRCE-DDS protocol supports a "subscription-published" on-demand communication mechanism, but its communication trigger still depends on the data update of the preset topic, and the trigger mechanism of the publisher when publishing data is that the subscription topic has new data updates, which will actively send the subscription topic data to the subscriber. It can be seen that it is still a passive and mechanical push method, without considering whether the subscriber currently needs the data, and without considering whether the cluster unmanned aerial vehicle of the publisher itself should send the data under the current situation.

[0004] For example, during the formation flight of the swarm unmanned aerial vehicle, the position information of the swarm unmanned aerial vehicle is mutually published and subscribed, so as to prevent interference and collision between the swarm unmanned aerial vehicles. In an ideal state, the subscription and publication frequency of the data topic should be based on the distance between the swarm unmanned aerial vehicles, the satellite positioning accuracy, the weather conditions and the running speed of the swarm unmanned aerial vehicle, and real-time adjustment can make it more effective use of communication resources. However, the processing and decision-making based on multi-dimensional situation information is a complex anthropomorphic thinking and decision-making process, which cannot be realized only by preset threshold and other automatic control means. However, the current swarm unmanned aerial vehicle communication system does not have this ability. SUMMARY

[0005] The application provides a swarm unmanned aerial vehicle communication system, device and method based on a neural network, which solves the problem of how to improve the communication efficiency of a small and medium-sized unmanned aerial vehicle based on a micro control chip (MCU).

[0006] To achieve the above-mentioned purpose, the application adopts the following technical solutions: In a first aspect, a swarm unmanned aerial vehicle communication system based on a neural network is provided, comprising: A micro control processor is configured on an unmanned aerial vehicle, used to execute the flight control logic of the task of the unmanned aerial vehicle, and obtain the real-time pose state and environmental data of the unmanned aerial vehicle itself; A neural network communication device is configured in communication connection with the micro control processor; the neural network communication device comprises a neural network processor, the neural network processor is built-in with an inference decision neural network model; the neural network communication device is used to receive the task information, real-time pose state and environmental data of the unmanned aerial vehicle, and output a communication decision instruction after model calculation; The communication decision instruction at least includes the decision of whether to communicate, with whom to communicate, communication content and communication opportunity; The micro control processor receives the communication decision instruction, and executes or inhibits the sending of communication data according to the instruction.

[0007] In a second aspect, a swarm unmanned aerial vehicle communication device based on a neural network is provided, comprising: A neural network communication device is configured in communication connection with a micro control processor; the neural network communication device comprises a neural network processor, the neural network processor is built-in with an inference decision neural network model; the neural network communication device is used to receive the task information, real-time pose state and environmental data of the unmanned aerial vehicle, and output a communication decision instruction after model calculation; The communication decision instruction at least includes the decision of whether to communicate, with whom to communicate, communication content and communication opportunity; The micro control processor receives the communication decision instruction, and executes or inhibits the sending of communication data according to the instruction. The micro control processor is configured on the unmanned aerial vehicle, and is used for executing flight control logic of a task of the unmanned aerial vehicle, and acquiring real-time pose state and environment data of the unmanned aerial vehicle itself.

[0008] In a third aspect, a neural network-based cluster unmanned aerial vehicle communication method is provided, which uses the neural network-based cluster unmanned aerial vehicle communication system according to the first aspect or the neural network-based cluster unmanned aerial vehicle communication device according to the second aspect to communicate between cluster unmanned aerial vehicles, and includes the following steps. Task information of the unmanned aerial vehicle is acquired, and real-time pose state and environment data collected by the unmanned aerial vehicle itself are acquired; The acquired task information and the real-time pose state and environment data are input into the neural network communication device, and optimal communication decision instructions are generated based on a pre-trained inference decision neural network model; The unmanned aerial vehicle is configured to execute a communication behavior according to the communication decision instructions after the communication data is generated and before the communication data is sent.

[0009] The neural network-based cluster unmanned aerial vehicle communication system has the following beneficial effects: The intelligent computing unit capable of running the neural network algorithm is added to the signal source backend of the traditional cluster unmanned aerial vehicle communication system, the intelligent autonomous perception and decision-making capability of the cluster unmanned aerial vehicle is improved, the cluster unmanned aerial vehicle is caused to make decisions on whether to communicate, with whom to communicate, what to communicate, and when to communicate, starting from the situation of the cluster unmanned aerial vehicle and taking into account the communication needs of other cluster unmanned aerial vehicles, the real-time situation is fully considered, and the cluster communication content is throttled in a personified manner, so that the most valuable information is sent to the most needed communication node at the most appropriate time and place by using the best communication mode, and the communication efficiency of the cluster unmanned aerial vehicle is improved.

[0010] The device and the method corresponding to the neural network-based cluster unmanned aerial vehicle communication system can achieve the same technical effects, and thus details are not repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 A structure block diagram of a neural network-based cluster unmanned aerial vehicle communication system is provided for an embodiment of the application; Figure 2 A first partial schematic view of connection logic of a real-time pose state network is provided for an embodiment of the application; Figure 3 A second partial schematic view of connection logic of a real-time pose state network is provided for an embodiment of the application; Figure 4 A third partial schematic view of connection logic of a real-time pose state network is provided for an embodiment of the application; Figure 5A first partial schematic diagram of connection logic of a task information network provided for an embodiment of the present application; Figure 6 A second partial schematic diagram of connection logic of a task information network provided for an embodiment of the present application; Figure 7 A schematic diagram of connection logic of an environment network provided for an embodiment of the present application; Figure 8 A structural diagram of a cluster unmanned aerial vehicle communication device based on a neural network provided for an embodiment of the present application; Figure 9 A structural diagram of another cluster unmanned aerial vehicle communication device based on a neural network provided for an embodiment of the present application; Figure 10 A schematic flowchart of a cluster unmanned aerial vehicle communication method based on a neural network provided for an embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined purposes, the technical solutions in the embodiments of the present application are described in detail. Obviously, the described embodiments are some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art belong to the scope of protection of the present application.

[0013] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a category, not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification means at least one of the connected objects, and the character " / ", generally represents that the front and rear associated objects are in an "or" relationship.

[0014] The description of the method flow in the specification and the steps of the flowchart in the drawings of the present application do not necessarily strictly execute according to the step number, and the method steps can change the execution order. Moreover, some steps can be omitted, a plurality of steps can be combined into one step for execution, and / or one step can be divided into a plurality of steps for execution.

[0015] The communication system of a small and medium-sized cluster unmanned aerial vehicle usually adopts a "lightweight hardware + distributed protocol" architecture. From the hierarchical architecture, it can be divided into a physical layer, a data link layer, a network layer, a transport layer and an application layer.

[0016] Physical Layer: Responsible for signal transmission and reception, including radio frequency modules, antennas, and other devices in the cluster UAV communication system, whose performance directly affects the distance and quality of communication.

[0017] Data Link Layer: Responsible for reliable data transmission, using multiple access technologies such as Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), or Code Division Multiple Access (CDMA) to improve the capacity and efficiency of the cluster UAV communication system.

[0018] Network Layer: Responsible for network topology management and routing, usually using dynamic routing protocols such as Ad hoc On-Demand Distance Vector Routing (AODV) / Optimized Link State Routing (OLSR) to adapt to the dynamic topology changes between cluster UAVs.

[0019] Transport Layer: Responsible for end-to-end data transmission, such as using Transmission Control Protocol (TCP) or User Datagram Protocol (UDP).

[0020] Application Layer: Provides services for various applications of cluster UAVs, such as task allocation, situation awareness, and cooperative control, and related protocols are usually customized and developed according to the specific application requirements of cluster UAVs. Due to the weak onboard computing power of small and medium-sized cluster UAVs, controlling the resource occupation of the onboard communication system leads to the current cluster UAV communication strategy being simple and limited, with some even broadcasting data in a fixed frequency, resulting in low efficiency and effectiveness.

[0021] Mico XRCE-DDS is designed for IoT, UAV, and industrial control scenarios, and is suitable for lightweight DDS (Data Distribution Service) application layer protocols for resource-constrained embedded systems.

[0022] The protocol is based on the DDS standard and optimized for devices with limited resources such as microcontrollers and sensors, effectively reducing memory and bandwidth usage. It can adapt to heterogeneous network environments and support data interaction between embedded devices and edge / cloud systems. The protocol follows the DDS subscription-publishing mode, actively pushing data when it changes, to some extent achieving the purpose of on-demand communication between cluster UAVs, but its essence is still a fixed-value communication strategy, which is a passive response mode triggered by threshold, with poor flexibility and initiative.

[0023] With the increasing maturity of edge computing hardware and artificial intelligence technology, it has shown high-density information and timely processing capabilities and dynamic resource allocation advantages in the field of communication, making it an ideal choice to improve the communication problems of cluster UAVs.

[0024] The application aims to empower the existing cluster unmanned aerial vehicle "lightweight hardware + distributed protocol" architecture through artificial intelligence technology based on a neural network algorithm; through the perception decision of the dynamic changes of the communication demand of the cluster unmanned aerial vehicle caused by the on-the-spot changes of the task scene and the device performance of the cluster unmanned aerial vehicle, the intelligent communication of the cluster unmanned aerial vehicle is innovatively integrated with the software and hardware of the flight control of the cluster unmanned aerial vehicle, the perception ability of the sensor in the flight control system of the cluster unmanned aerial vehicle is ingeniously empowered to the intelligent communication calculation unit to make intelligent decisions based on a neural network algorithm, the most valuable information is sent to the target communication node in the most suitable time and place by the best communication mode, and the target communication node needs the communication, the key technology that the cluster unmanned aerial vehicle can actively and autonomously perform cluster communication on demand is realized, the communication efficiency of the cluster unmanned aerial vehicle is improved, and the large-scale application level of the cluster unmanned aerial vehicle is improved.

[0025] In the present specification, a cluster unmanned aerial vehicle communication system based on a neural network is provided, and simultaneously relates to a cluster unmanned aerial vehicle communication device based on a neural network, a cluster unmanned aerial vehicle communication method based on a neural network, which are described one by one in combination with the drawings and preferred embodiments.

[0026] Please refer to Figure 1 The embodiment of the present application provides a cluster unmanned aerial vehicle communication system based on a neural network, as shown in Figure 1 , which comprises: A micro control processor (MCU) is configured on the unmanned aerial vehicle and is used for executing the flight control logic of the task of the unmanned aerial vehicle and acquiring the real-time pose state and environmental data of the unmanned aerial vehicle itself; A neural network communication device is configured in communication connection with the micro control processor (MCU); the neural network communication device comprises a neural network processor (NPU), the neural network processor (NPU) is built-in with an inference decision neural network model; the neural network communication device is used for receiving the task information and real-time pose state and environmental data of the unmanned aerial vehicle, and outputting a communication decision instruction after model calculation; The communication decision instruction at least comprises the decision of whether to communicate, with whom to communicate, communication content and communication opportunity; The micro control processor (MCU) receives the communication decision instruction and executes or inhibits the sending of communication data according to the instruction.

[0027] Further, the inference decision neural network model comprises a real-time pose state network, a task information network and an environmental network.

[0028] Further, referring to Figures 2-4The real-time pose state network is connected with operation nodes based on the real-time pose state data as minimum input of the model and sequentially connected with network; the real-time pose state data includes: unmanned aerial vehicle type, running state, take-off mode, actuator state, satellite positioning state, communication category, communication state, message category, communication capability parameter, cluster formation mode, cluster coordination category, power source state, control mode, system fault state, sensor state, recovery mode and self-situation.

[0029] That is, the real-time information of the cluster unmanned aerial vehicle pose state is taken as a "neuron" to construct a reasoning and decision-making neural network model, and reasoning and decision-making operations are performed through a network connection logic preset by the model, and the network connection logic is preset based on node association rules of cluster coordination requirements. Unmanned aerial vehicle type: Different functional cluster unmanned aerial vehicles play different roles in combat scenarios, resulting in different focuses in cluster communication requirements. The unmanned aerial vehicle type includes: transportation delivery, detection and attack integration, reconnaissance and surveillance, suicide attack, bomb-hanging attack, communication relay and illumination coverage, etc.; for example, cluster transportation delivery unmanned aerial vehicles and suicide cluster unmanned aerial vehicles can preset flight routes and target positions in the task setting stage, and completely offline during task execution, without any communication interaction with external nodes. While the detection and attack integration and bomb-hanging attack cluster unmanned aerial vehicles, even if they have intelligent identification capabilities for target reconnaissance on board, must identify and confirm the attack target and obtain the operation authorization of the attack instruction in different real-time situations, and have certain communication interaction requirements for external communication nodes.

[0030] Running state: The running state of the cluster unmanned aerial vehicle refers to each stage in the full life cycle from storage to use. The running state includes: storage debugging, flight preparation, cruising, operation deployment and return recovery, etc.; the communication requirements are different in different stages of the running state, and the situation changes in each state will cause relatively random changes in cluster communication requirements. For example, the cluster communication in the cruising stage can pre-plan the cruising route and target of all cluster unmanned aerial vehicles in the flight preparation stage to achieve radio silence approach and cluster flight inter-vehicle distance maintenance. However, when a cluster unmanned aerial vehicle enters the operation deployment or other state due to early target discovery or other temporary conditions, the pre-planned route is changed, and the unmanned aerial vehicle needs to actively publish its real-time coordinates or modified route information to update the information to other cluster unmanned aerial vehicles that may be affected by the route interference.

[0031] Take-off mode: The take-off mode of the cluster UAVs reflects the specific application scenarios to some extent, and some cluster UAVs have multiple take-off modes, and the communication requirements are different under different take-off modes. The take-off modes include: catapult take-off, taxi take-off, vertical take-off, and air release, etc.; for example, some cluster UAVs can adopt catapult take-off and air release take-off. The catapult take-off process has a high degree of automation. After triggering the take-off, the linkage and coordination between the catapult device and the catapulted cluster UAVs and the cluster UAVs catapulted in sequence follow a fixed timing logic. In this process, unless a fault occurs and the program needs to be stopped through communication, the launch can basically be completed in a communication silent state. In the air release take-off mode, the linkage between the released cluster UAV and the release carrier is carried out during the flight process, and strong interaction is needed to achieve cooperative communication.

[0032] Actuator state: The actuator state of the cluster UAV mainly refers to the locking and unlocking state of the rudder and motor / engine actuators, which determines the change of communication requirements of the UAV in the disabled or non-disabled state. The actuator state includes: locked state and unlocked state, etc.; for example, if the cluster UAV in the cruising and rushing state changes from unlocked to locked, it is obviously disabled due to internal or external reasons, and its communication requirements are naturally different from those of the cluster UAV without disability.

[0033] Satellite positioning state: The satellite positioning state is a key basis for determining the distance between the cluster UAVs, and is one of the indicators for judging whether the UAV is subject to radio denial. The satellite positioning state includes: no fixed solution, 2D fixed solution, 3D fixed solution, RTK floating point solution, RTK fixed solution, and estimated solution, etc.; for example, when the satellite positioning state is normal, the cluster UAVs can completely adopt the radio silent approach to cruise and rush. But once the satellite positioning state precision decreases, the cluster UAVs need to timely externalize this situation, and other UAVs can avoid the denial obstacles after receiving the information. It is particularly important in the high-density formation UAV cluster combat scenario.

[0034] Communication category: including point-to-point communication, broadcast communication and service communication, etc.; used to distinguish the communication frequency and communication range of cluster communication. Among them, the service communication is special, which means that the cluster UAVs assist other cluster UAVs to complete higher-level combat target communication through their own information service, such as target locking and attack guidance, which needs to send target information to the served end with high frequency and high real-time. Similar but different from it is the broadcast communication, for example, when the cluster UAVs enter the satellite positioning denial environment, they need to broadcast the coordinate information before entering to other UAVs in the cluster. Although both of them need to expose their own situation for external communication, the communication frequency and content are different. Communication state: refers to whether the current cluster UAV needs absolute silence or actively communicates externally. The communication state includes absolute silence, active communication, passive communication, and free communication, etc.; this state is not immutable, and will be adjusted in real time according to the situation in different scenarios. For example, if the cluster UAV in the passive communication state suddenly enters an unknown satellite positioning denial environment, it should at least send an alarm information to the outside. In the absolute silence scenario, it may need to be silent at all costs, even if it perceives the satellite positioning denial environment, it should not send data to expose the cluster for the overall situation. Message category: mainly includes three types of messages, control type, state type and sensor type, and their communication needs are different in different scenarios and their own situation. For example, the communication needs of state type information in the two stages of storage debugging and launch preparation are different. In the storage debugging stage, the state information of the cluster UAV needs to be monitored and maintained, so full-featured communication support is needed. While in the launch preparation stage, the state class communication only needs to send the final result of the BIT (in-machine test) system to the control station. If a large amount of state class information is sent out like in the storage debugging stage, not only will it occupy communication resources, but also it is easy to expose the launch site.

[0035] Communication capability parameters: including communication frequency band, communication power, antenna gain, signal strength and communication coverage capability, these five types of communication related neurons reflect the communication capability of the cluster UAV to some extent, which is an important basis for neural network reasoning and decision-making. For example, in some scenarios, if the data receiver is judged to be difficult to receive information according to the self-communication capability and the scene situation, but still endlessly broadcasts data, not only can't achieve the communication purpose, but also wastes communication resources and exposes the communication characteristics of the entire cluster, which is not worth the loss. Cluster formation mode and cluster coordination category: these two can reflect the situation of the surrounding friendly cluster UAVs to some extent, and the urgency of external communication can be judged according to this neuron. The cluster formation mode includes free formation, linear formation, grouping formation and network formation; the cluster coordination category includes no coordination, coordinated reconnaissance, coordinated attack and coordinated reconnaissance and attack; for example, in the case of linear formation, if the previous cluster UAV can only rely on inertial navigation or other visual sensors to continue to execute the task due to satellite denial environment, it should send denial warning information to the subsequent arriving cluster UAV in time, or even send the alarm in the state of hovering, hovering or returning to the original route to ensure information transmission and reduce the threat of subsequent UAV failure. While in the network formation case, since each cluster UAV is deployed in a three-dimensional array, whether to send an alarm and to whom should be decided after considering the pre-installed task situation and whether to be silent and other situations.

[0036] Power source status: mainly refers to the power battery or fuel condition of the cluster UAV, reflecting its endurance capability, so as to maximize its contribution in cluster communication as much as possible. The power source status includes: normal voltage / fuel, low voltage fuel and disabled voltage / fuel state; for example, in the operation deployment stage, the cluster UAV that completes the scheduled task can automatically find the communication blind area to provide communication relay service for the entire cluster UAV in the operation deployment stage.

[0037] Control mode: the change of the control mode of the cluster UAV directly reflects its current performance and function state, which can effectively distinguish whether the UAV is in a self-holding state or an external holding state. The control mode includes: intelligent control, task route, fixed-point hovering, attitude mode, manual mode, return mode, take-off mode, landing mode and self-destruction mode; for example, in the intelligent control mode (off-board control mode), the cluster UAV control scheduling of position coordination and task coordination between UAVs is mainly controlled by the intelligent control unit on the UAV, which needs to support the pre-installed task information and the information changed during operation, thereby generating communication demand. The control mode such as task route and fixed-point hovering can rely on self-holding information support, that is, it is automatically executed in a completely offline state, so the cluster communication demand is not as urgent as in the intelligent control mode.

[0038] System failure and sensor state: this is one of the effective bases for inferring the working state of the cluster UAV itself. Whether the normality of some key on-board sensors can directly infer the subsequent survival state and task accessibility state of the UAV, thereby predicting and satisfying the communication demand to be generated in advance. The system failure state includes: no error, gyroscope error, accelerometer error, magnetic direction error, barometric height error, satellite positioning error, optical flow meter error, visual inertia error, distance perception error, air speed meter error, power supply error, power source error, actuator error, obstacle avoidance function error, control signal error and telemetry signal error, etc. The sensor state includes: working normally, device disabled, data card lag, invalid data, data error, low data rate, data delay and intermittent anomaly, etc. For example, in the task deployment stage, if the multi-channel redundant gyroscopes of the cluster UAV report errors in succession, the system error tolerance gradually accumulates to approach the critical value, and the task of this UAV in the cluster is extremely important and has not been completed, at this time, its communication demand is to report its unfinished task and state to the adjacent cluster UAV and telemetry station.

[0039] Recovery mode: It can be inferred that the cluster UAV can continue to provide communication services for the cluster in what form after completing its own task. The recovery mode includes: parachute recovery, stall landing, sliding landing, vertical landing, net recovery, skyhook recovery and self-destruction, etc.; for example, the cluster UAV using self-destruction recovery mode can perform communication blind filling for the current cluster operation blind area until the power / fuel is almost exhausted before losing ability, and then triggers self-destruction to improve the combat effectiveness of the cluster UAV.

[0040] Self-situation: The position situation of the cluster UAV in the combat scene is one of the effective bases for inferring its communication demand. The self-situation includes: home point distance, task area distance, adjacent platform distance, flight height, flight speed and communication node distance, etc.; if the distance is too far to communicate, it is difficult to contact the target even if there is an urgent communication demand. It is more scientific and cost-effective to wait for the right opportunity to communicate than to send information blindly.

[0041] Similarly, see Figures 5-6 , the task information network is connected with the operation nodes based on the task information as the minimum input and model, and then connected in sequence; the task information includes: relative distance, task flight state, task type, task coordination relationship, task completion degree, target form, target threat to the global, target threat mode, target threat to itself and target threat to the cluster.

[0042] Further: Relative distance: It refers to the distance between the cluster UAV and the preset or intended related party during the task process. It can reflect the situation of the cluster UAV in the task execution to some extent, and some new communication demands can be inferred. It includes: home landing point, emergency landing point, near task point, near platform, near communication node, take-off release point and telemetry station, etc.; for example: in the combat scene, a cluster UAV has completed the preset task and has no new task, and is near the emergency landing point. The cluster UAV can actively request tasks from the disabled cluster UAV near the emergency landing point according to the situation. If there is a disabled cluster UAV that has not completed its own task in the emergency landing point, it can activate task sharing through passive communication to let the non-disabled cluster UAV continue to execute the task.

[0043] Task flight state: It is an important indicator of the state of the cluster UAV during task execution, one of the indicators of the degree of task execution capability, and can also reflect the communication capability and communication demand of the cluster UAV. Task flight state includes: altitude, speed, endurance time, attitude stability, and positioning accuracy, etc. For example: the task flight altitude determines the line-of-sight distance between the cluster UAVs, thereby determining the communication coverage of the cluster UAV. That is, the stability of communication from a cluster UAV with high flight altitude to a cluster UAV with low flight altitude is much higher than the communication capability between two cluster UAVs with low flight altitude.

[0044] Task type and task coordination: The task type and task coordination of the cluster UAV are mainly based on detection, attack, guidance, and evaluation. The communication demand of task coordination under different task types is also different. Task type includes: reconnaissance, attack, target guidance, and effectiveness evaluation, etc. Task coordination includes coordination and non-coordination. For example: under the reconnaissance task type, only in the case of automatic target discovery, is there a need to externally transmit target coordination information.

[0045] Task completion degree: The task completion degree of the cluster UAV can reflect whether the mission of the cluster UAV has been completed, and can also indicate whether the cluster UAV can actively bear the temporary communication demand such as communication blind filling in the cluster according to its own value in the cluster.

[0046] Target form: including living forces, equipment targets, and facility targets, etc. The target action capability in the cluster task is the embodiment of the value of the cluster UAV. In the combat scene, in addition to the pre-set task target, the cluster UAV will also discover new targets that are higher or lower than the pre-set task target. At this time, for the newly discovered high-value target, or the superimposed value of multiple low-value targets, whether to trigger the cluster UAV to change the communication strategy through neural network reasoning decision is one of the purposes of the target form neuron setting.

[0047] Threat degree of the target to itself, to the cluster, and to the global: The threat degree of the task target to itself, to the cluster, and to the global determines whether there is a need to communicate externally in a timely manner.

[0048] Threat mode of the target: In the cluster task, the threat mode of the target will determine the disposal mode and disposal time tolerance of the cluster UAV to the target, thereby bringing different communication demands. The threat mode of the target includes: soft kill, hard kill, and being detected, etc. For example: in the case of hard kill threat mode, the cluster UAV can continue to communicate according to the original communication strategy without sensing the damage to its own system. Once the cluster UAV discovers that it is damaged when facing a hard kill threat, it should timely externally transmit the damage to itself to prevent the difficulty of externally transmitting data after the instantaneous failure of the cluster UAV.

[0049] Similarly, referring to Figure 7 , the environmental network is connected in sequence based on the environmental data as the minimum input of the model and the operation node; the environmental data includes weather classification, visibility, wind power, rainfall, snowfall, special weather, temperature and relative humidity.

[0050] The weather state mainly affects the signal transmission quality, stability and reliability of the swarm unmanned aerial vehicle, and the influence mechanism and degree of different weather conditions are different, so the communication demand changes are also different.

[0051] Precipitation, snowfall, hail, fog, haze and sand can absorb and scatter radio waves, especially high-frequency signals such as millimeter waves, and the signal attenuation is more obvious, resulting in a decrease in signal strength and a shortening of communication distance.

[0052] The decrease of visibility will affect the positioning ability assisted by visual sensors, resulting in an increase in the positioning error of the swarm unmanned aerial vehicle, and then affect the change of the communication demand of the swarm unmanned aerial vehicle.

[0053] Thunder and lightning weather can produce strong electromagnetic pulse interference to the control and communication electronic equipment of the swarm unmanned aerial vehicle, resulting in signal interruption, data loss and even equipment damage, which will also affect the change of the communication demand of the swarm unmanned aerial vehicle.

[0054] Strong wind will cause the attitude of the swarm unmanned aerial vehicle to be unstable, and the position information of each swarm unmanned aerial vehicle in the swarm needs to be frequently updated to prevent collision.

[0055] Temperature and humidity may reduce the working efficiency of the communication radio frequency module, and then affect the communication ability of the swarm unmanned aerial vehicle, resulting in a change in the communication demand.

[0056] Through the connection between the networks in the above three dimensions, the situation information is interactively fused and inferred, the real-time demand of real-time intelligent communication is quickly generated, and the current swarm unmanned aerial vehicle can be actively and autonomously decided whether to communicate, with whom to communicate, what to communicate, and when to communicate, from the swarm unmanned aerial vehicle ontology, the goal of humanized intelligent on-demand communication is realized.

[0057] Referring to Figures 8-9 Corresponding to the above embodiment of the swarm unmanned aerial vehicle communication system based on neural network, the embodiment of the present application provides a swarm unmanned aerial vehicle communication device based on neural network, comprising: The neural network communication device is configured to be in communication connection with the micro-control processor; the neural network communication device comprises a neural network processor, and the neural network processor is built-in with an inference decision neural network model; the neural network communication device is used for receiving the task information, real-time pose state and environmental data of the unmanned aerial vehicle, and outputting a communication decision instruction after model calculation; The communication decision instruction at least includes decisions of whether to communicate, with whom to communicate, communication content, and communication timing; The micro-control processor receives the communication decision instruction and executes or inhibits the sending of communication data according to the instruction; The micro-control processor is arranged on the unmanned aerial vehicle and is configured to execute flight control logic of a task of the unmanned aerial vehicle and acquire real-time pose state and environment data of the unmanned aerial vehicle.

[0058] Further, the inference decision neural network model includes a real-time pose state network, a task information network, and an environment network. The real-time pose state network is based on real-time pose state data as minimum input and operation nodes of the model and sequentially performs network connection; the real-time pose state data includes unmanned aerial vehicle type, running state, take-off mode, actuator state, satellite positioning state, communication category, communication state, message category, communication capability parameter, cluster formation mode, cluster coordination category, power source state, control mode, system fault state, sensor state, recovery mode, and self-situation.

[0059] The task information network is based on task information as minimum input and operation nodes of the model and sequentially performs network connection; the task information includes relative distance, task flight state, task type, task coordination relationship, task completion degree, target form, target threat degree to the whole world, target threat mode, target threat degree to the self, and target threat degree to the cluster.

[0060] The environment network is based on environment data as minimum input and operation nodes of the model and sequentially performs network connection; the environment data includes weather classification, visibility, wind power, rainfall, snowfall, special weather, temperature, and relative humidity.

[0061] Exemplarily, the above neural network communication device is as shown in Figure 9 As shown, the device for improving communication efficiency of the cluster unmanned aerial vehicle based on the neural network algorithm is an airborne intelligent control and communication system integrating "intelligent communication" and "pose control", adopts a dual-U architecture of a domestic neural network processor (NPU) and a high-performance micro-control processor (MCU), and realizes integrated design of intelligent communication and control of the cluster unmanned aerial vehicle. The system integrates an RTK differential satellite direction-finding positioning module and three sets of heterogeneous redundant inertial navigation modules, and has the hardware and software unified standard and unified type installation capability of core control and intelligent communication systems of "sea, land, and air" unmanned equipment platforms such as multi-rotor, fixed-wing, helicopter, vertical take-off fixed-wing unmanned aerial vehicles, wheeled and tracked unmanned vehicles, and propeller and pump unmanned ships below 200 kg. From top to bottom, there are a cooling fan, an upper shell, a neural network calculation module, an inertial navigation cover plate, a combined inertial navigation module, an inertial navigation circuit board, a main circuit board, a satellite navigation module, and a lower shell.

[0062] Performance indicators are as follows: Appearance size: 169mm * 81.5mm * 31mm Whole machine weight: 435 g Power supply voltage: 19v-76v Rated power: 16w Smart communication computing power: 22 TOPS INT8 Pose control computing power: 480Mhz Sensor: 3-axis gyroscope, 3-axis accelerometer, 3-axis magnetometer, 5-axis barometer Satellite positioning: dual-antenna RTK differential direction-finding positioning module Communication interface (J30-144): 1 ETH gigabit network port, 3 RS232, 7 RS422, 3 USB3.0, 2 CAN, 14 PWM, 6 ADC.

[0063] Both open source and throttling are effective means to solve resource shortage. The embodiment of the application adds an intelligent computing unit capable of running a neural network algorithm at the signal source backend of the traditional cluster UAV communication system, improves the intelligent autonomous perception and decision-making capability of the cluster UAV, takes into account the real-time situation, and actively decides whether to communicate, with whom to communicate, what to communicate, and when to communicate, and humanizes the throttling of cluster communication content, so as to realize the goal of sending the most valuable information to the most needed communication node at the most appropriate time and place in the best communication mode, and improve the communication efficiency of the cluster UAV.

[0064] Reference Figure 10 Corresponding to the above-mentioned neural network-based cluster UAV communication system embodiment, the embodiment of the application provides a neural network-based cluster UAV communication method, which uses the above-mentioned neural network-based cluster UAV communication system to communicate between cluster UAVs, comprising: Step S1, obtaining task information of the UAV, and real-time collecting the pose state and environmental data of the UAV itself; Step S2, inputting the obtained task information and real-time pose state and environmental data into the neural network communication device; based on the pre-trained inference decision neural network model, generating an optimal communication decision instruction; Step S3, configuring the UAV to execute a communication behavior according to the communication decision instruction after the communication data is generated and before it is sent; the communication behavior includes: suppressing the sending of unnecessary data, or selecting the best opportunity, the best object and the best content for communication.

[0065] When the current cluster unmanned aerial vehicle is applied in the actual environment, it must rely on a high-performance cluster communication network to support the normal work of the cluster unmanned aerial vehicle. At present, the positioning of the cluster unmanned aerial vehicle is basically between the cruise missile and the intelligent ammunition, especially the suicide cluster unmanned aerial vehicle, and the cost of installing high-performance cluster communication equipment is particularly sensitive. In addition, when the cluster unmanned aerial vehicle is designed and manufactured, the size, weight, power consumption and many other conditions of the cluster communication equipment are limited. Therefore, the cluster communication has become the biggest congenital weakness of the cluster unmanned aerial vehicle in combat application. Various detection countermeasures against cluster communication emerge in an endless stream, which directly and indirectly restricts the equipment process of the cluster unmanned aerial vehicle. The cluster unmanned aerial vehicle communication mainly needs to ensure that the cluster unmanned aerial vehicles in the cluster and the cluster unmanned aerial vehicle and the telemetry station can efficiently, stably and in real time interact information, share running state, task progress, environmental perception and other information in real time, so as to realize the cooperation of functions such as maintaining the cluster formation, distributing the combat task and planning the action path. It needs to have efficient data transmission and processing capability to support the cooperative operation of the cluster unmanned aerial vehicles in the cluster and the smooth completion of the combat task. The application weakens the high dependence of the cluster unmanned aerial vehicle on the cluster communication network in the actual combat scene through the cost-effective emerging communication technology, and effectively improves the battlefield survival ability and task completion ability of the cluster unmanned aerial vehicle.

[0066] It should be noted that in this document, the terms "comprise", "comprising", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of another identical element in the process, method, article or device that includes the element. In addition, it should be pointed out that the scope of the methods and devices in the present application is not limited to the order of performing the functions shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in reverse order, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0067] It can be understood that the embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are only illustrative but not restrictive, and various changes or equivalent replacements can be made to the features and embodiments without departing from the spirit and scope of the present application, which are known to those skilled in the art. In addition, under the inspiration or teaching of the present application, those skilled in the art can modify the features and embodiments to adapt to specific conditions and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application are within the scope of protection of the present application.

Claims

1. A neural network based swarm drone communication system, characterized in that, Comprise: A micro-control processor configured on the UAV for executing flight control logic of the UAV's task and obtaining real-time pose state and environment data of the UAV itself; A neural network communication device configured in communication connection with the micro-control processor; the neural network communication device comprises a neural network processor, the neural network processor is built-in with an inference decision neural network model; the neural network communication device is used for receiving task information and real-time pose state and environment data of the UAV, and outputting communication decision instructions after model calculation; The communication decision instructions at least include decisions of whether to communicate, with whom to communicate, communication content, and communication timing; The micro-control processor receives the communication decision instructions and executes or inhibits sending of communication data according to the instructions.

2. The neural network-based cluster UAV communication system according to claim 1, wherein The inference decision neural network model comprises a real-time pose state network, a task information network, and an environment network; The real-time pose state network is based on real-time pose state data as the minimum input and operation nodes of the model and sequentially performs network connection; The real-time pose state data comprises UAV type, running state, take-off mode, actuator state, satellite positioning state, communication category, communication state, message category, communication capability parameter, cluster formation mode, cluster coordination category, power source state, control mode, system fault state, sensor state, recovery mode, and self-situation.

3. The neural network-based cluster UAV communication system according to claim 2, wherein The task information network is based on task information as the minimum input and operation nodes of the model and sequentially performs network connection; the task information comprises relative distance, task flight state, task type, task coordination relationship, task completion degree, target form, target threat degree to the whole, target threat mode, target threat degree to the self, and target threat degree to the cluster.

4. The neural network-based cluster UAV communication system according to claim 2, wherein The environment network is based on environment data as the minimum input and operation nodes of the model and sequentially performs network connection; the environment data comprises weather classification, visibility, wind power, rainfall, snowfall, special weather, temperature, and relative humidity.

5. A neural network based swarm drone communication device, characterized in that, Comprise: A neural network communication device configured in communication connection with the micro-control processor; the neural network communication device comprises a neural network processor, the neural network processor is built-in with an inference decision neural network model; the neural network communication device is used for receiving task information and real-time pose state and environment data of the UAV, and outputting communication decision instructions after model calculation; The communication decision instructions at least include decisions of whether to communicate, with whom to communicate, communication content, and communication timing; The micro-control processor receives the communication decision instructions and executes or inhibits sending of communication data according to the instructions; The micro-control processor is configured on the UAV for executing flight control logic of the UAV's task and obtaining real-time pose state and environment data of the UAV itself. 6.The neural network-based swarm UAV communication device according to claim 5, wherein, the inference decision neural network model comprises a real-time pose state network, a task information network, and an environment network; the real-time pose state network is based on real-time pose state data as the minimum input of the model and connected with operation nodes in sequence; the real-time pose state data comprises UAV type, running state, take-off mode, actuator state, satellite positioning state, communication category, communication state, message category, communication capability parameter, swarm formation mode, swarm coordination category, power source state, control mode, system fault state, sensor state, recovery mode, and self-situation. 7.The neural network-based swarm UAV communication device according to claim 6, wherein, the task information network is based on task information as the minimum input of the model and connected with operation nodes in sequence; the task information comprises relative distance, task flight state, task type, task coordination relationship, task completion degree, target form, target threat degree to the whole, target threat mode, target threat degree to the self, and target threat degree to the swarm. 8.The neural network-based swarm UAV communication device according to claim 6, wherein, the environment network is based on environment data as the minimum input of the model and connected with operation nodes in sequence; the environment data comprises weather classification, visibility, wind power, rainfall, snowfall, special weather, temperature, and relative humidity.

9. A neural network-based swarm drone communication method, characterized by, The neural network-based swarm UAV communication system according to claim 1 or the neural network-based swarm UAV communication device according to claim 5 is used in communication between swarm UAVs, comprising: acquiring task information of the UAV, and real-time pose state and environment data collected by the UAV itself; inputting the acquired task information, real-time pose state, and environment data into the neural network communication device; generating optimal communication decision instructions based on the pre-trained inference decision neural network model; configuring the UAV to perform communication behavior according to the communication decision instructions after the communication data is generated and before the communication data is sent. 10.The neural network-based swarm UAV communication method according to claim 9, wherein, the communication behavior comprises inhibiting sending of unnecessary data, or selecting optimal timing, optimal object, and optimal content for communication.

Images