Method, device and electronic equipment for cooperative operation of unmanned aerial vehicles

Abstract

The application discloses a method and device for cooperative work of unmanned aerial vehicles and electronic equipment. The method comprises the following steps: obtaining a preset flight route of an unmanned aerial vehicle, wherein the preset flight route is determined by a plurality of discrete waypoints; determining a preset handover point and a preset landing airport of the unmanned aerial vehicle according to the preset flight route; predicting the remaining endurance time of the unmanned aerial vehicle by using a neural network model to obtain a predicted remaining endurance time, and adjusting the preset handover point and the preset landing airport according to the predicted remaining endurance time to obtain a target handover point and a target landing airport; and performing cooperative work of a plurality of unmanned aerial vehicles according to the target handover point and the target landing airport, wherein the last unmanned aerial vehicle lands at the target landing airport and continues to perform work at the target handover point by the next unmanned aerial vehicle. The application solves the technical problems of low resource utilization, poor task continuity and difficulty in coping with large-scale task requirements in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned aerial vehicles, in particular, to a method and device for cooperative operation of unmanned aerial vehicles and an electronic device. BACKGROUND

[0002] In the inspection scene of low-altitude economy widely applied by power lines, pipelines, etc., an integrated automatic platform of unmanned aerial vehicles and airports is usually adopted to realize unattended operation in all time periods. In order to ensure the safe return of unmanned aerial vehicles, the actual operation radius of a single machine is often limited to within half of the maximum endurance distance, which greatly reduces the inspection range of the unmanned aerial vehicle and affects the operation efficiency. At the same time, the industry generally adopts an inefficient mode of "single machine single task", that is, a single unmanned aerial vehicle only completes the task within its operation radius, which has low resource utilization and poor task continuity, and is difficult to meet the demand for large-scale tasks.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] Embodiments of the present application provide a method and device for cooperative operation of unmanned aerial vehicles and an electronic device, to at least solve the technical problem that the inspection mode of unmanned aerial vehicles has low resource utilization, poor task continuity, and is difficult to meet the demand for large-scale tasks in the related art.

[0005] According to an aspect of an embodiment of the present application, a method for cooperative operation of unmanned aerial vehicles is provided, comprising: obtaining a preset flight route of an unmanned aerial vehicle, wherein the preset flight route is determined by a plurality of discrete waypoints; determining a preset handover point and a preset landing airport of the unmanned aerial vehicle according to the preset flight route; predicting the remaining endurance time of the unmanned aerial vehicle by using a neural network model to obtain a predicted remaining endurance time, and adjusting the preset handover point and the preset landing airport according to the predicted remaining endurance time to obtain a target handover point and a target landing airport; and performing cooperative operation of a plurality of unmanned aerial vehicles according to the target handover point and the target landing airport, wherein the last unmanned aerial vehicle lands at the target landing airport and continues to perform the operation at the target handover point by the next unmanned aerial vehicle.

[0006] Optionally, determining the preset handover point and the preset landing airport of the unmanned aerial vehicle according to the preset flight route comprises: obtaining a route starting point in the preset flight route; determining a first airport closest to the route starting point from a set of available airports, wherein each airport in the set of available airports contains a first unmanned aerial vehicle satisfying a preset flight condition; determining a first unmanned aerial vehicle flight power of the first unmanned aerial vehicle from the route starting point to other waypoints in the preset flight route, and determining a remaining battery capacity required by the first unmanned aerial vehicle at each waypoint in the preset flight route to a landable airport; determining, according to the first unmanned aerial vehicle flight power, the farthest waypoint reached by the first unmanned aerial vehicle within the remaining battery capacity as the preset handover point; and determining the available airport closest to the preset handover point as the preset landing airport.

[0007] Optionally, the first unmanned aerial vehicle flight power from the starting point of the route to other waypoints in the preset flight route is determined by: obtaining a power set corresponding to the first unmanned aerial vehicle, wherein the power set comprises induced power, profile power, drag power, and power of the first unmanned aerial vehicle communication and the carried load; and determining the first unmanned aerial vehicle flight power according to the induced power, the profile power, the drag power, the power of the first unmanned aerial vehicle communication and the carried load, and the energy efficiency coefficient.

[0008] Optionally, the power set is determined by: obtaining the induced power of the first unmanned aerial vehicle in hovering, the ground speed of the first unmanned aerial vehicle, and the average rotor induced speed of the first unmanned aerial vehicle in hovering; determining the induced power of the first unmanned aerial vehicle according to the induced power of the first unmanned aerial vehicle in hovering, the ground speed of the first unmanned aerial vehicle, and the average rotor induced speed of the first unmanned aerial vehicle in hovering; obtaining the profile power of the first unmanned aerial vehicle in hovering and the rotor tip speed of the first unmanned aerial vehicle; determining the profile power of the first unmanned aerial vehicle according to the profile power of the first unmanned aerial vehicle in hovering, the ground speed of the first unmanned aerial vehicle, and the rotor tip speed of the first unmanned aerial vehicle; obtaining the fuselage drag ratio of the first unmanned aerial vehicle, the air density, the rotor solidity of the first unmanned aerial vehicle, and the rotor disc area of the first unmanned aerial vehicle; obtaining the fuselage drag ratio of the first unmanned aerial vehicle, the air density, the rotor solidity of the first unmanned aerial vehicle, and the rotor disc area of the first unmanned aerial vehicle; and determining the power set corresponding to the first unmanned aerial vehicle according to the induced power of the first unmanned aerial vehicle, the profile power of the first unmanned aerial vehicle, the drag power of the first unmanned aerial vehicle, and the power of the first unmanned aerial vehicle communication and the carried load.

[0009] Optionally, the ground speed of the first unmanned aerial vehicle is determined by: obtaining the air speed and the flight direction of the first unmanned aerial vehicle, and obtaining the wind speed and the wind direction; and determining the ground speed of the first unmanned aerial vehicle according to the air speed, the flight direction, the wind speed, and the wind direction.

[0010] Optionally, the induced power of the first unmanned aerial vehicle in hovering is determined by: obtaining the aircraft weight of the first unmanned aerial vehicle and the increment correction coefficient; and determining the induced power of the first unmanned aerial vehicle in hovering according to the aircraft weight, the increment correction coefficient, the air density, and the rotor disc area.

[0011] Optionally, the profile power of the first unmanned aerial vehicle in hovering is determined by: obtaining the drag coefficient, the rotor angular speed of the first unmanned aerial vehicle, and the rotor radius; and determining the profile power of the first unmanned aerial vehicle in hovering according to the drag coefficient, the air density, the rotor solidity, the rotor disc area, the rotor angular speed, and the rotor radius.

[0012] Optionally, the determining the remaining battery capacity required for the first unmanned aerial vehicle to reach a landable airport from each waypoint in the preset flight route comprises: obtaining a first remaining battery capacity of the first unmanned aerial vehicle at a previous waypoint, a full charge voltage, a nominal battery capacity, and a slope of a linear curve of voltage drop, wherein the slope is determined by the full charge voltage, the nominal battery capacity, a standard voltage, and a nominal battery discharge coefficient; determining a voltage of the first unmanned aerial vehicle at a next waypoint according to the full charge voltage, the slope, the nominal battery capacity, and the first remaining battery capacity; determining a current of the first unmanned aerial vehicle at the next waypoint according to a flight power of the first unmanned aerial vehicle and the voltage of the first unmanned aerial vehicle at the next waypoint; and determining a second remaining battery capacity of the first unmanned aerial vehicle at the next waypoint according to the current of the first unmanned aerial vehicle at the next waypoint, a rated discharge time, the nominal battery capacity, and a time step, wherein the time step is a number of waypoints flown through by the first unmanned aerial vehicle, and iteration is stopped when the second remaining battery capacity meets a preset battery capacity condition, and the preset battery capacity condition is determined by the nominal battery capacity and the nominal battery discharge coefficient.

[0013] Optionally, the neural network model is obtained by: obtaining sample data, wherein the sample data comprises multi-dimensional feature data of the unmanned aerial vehicle, and the multi-dimensional feature data comprises battery state data, flight state data, environment data, and task load data; processing the sample data by using an initial neural network model to obtain a sample predicted remaining endurance time, wherein the initial neural network model comprises a first model for processing time series data and a second model for processing spatial data; determining a fusion loss function according to the sample predicted remaining endurance time and a real remaining endurance time, and adjusting parameters of the initial neural network model according to a loss value of the fusion loss function until an adjusted loss value meets a preset condition to stop iteration, thereby obtaining the neural network model.

[0014] Optionally, the fusion loss function is determined by: determining a mean square error function of the sample predicted remaining endurance time and the real remaining endurance time, and determining the mean square error function as a main loss; determining an asymmetric loss according to a first penalty coefficient, a second penalty coefficient, the sample predicted remaining endurance time, and the real remaining endurance time; determining a multi-task learning loss according to weight coefficients of multiple auxiliary tasks and corresponding mean square error loss functions, wherein the multiple auxiliary tasks comprise voltage auxiliary prediction and flight state prediction; and determining the fusion loss function according to the main loss, the asymmetric loss, and the multi-task learning loss.

[0015] Optionally, the preset handover point and the preset landing airport are adjusted according to the predicted remaining endurance time to obtain a target handover point and a target landing airport, including: obtaining a reference time required for the UAV to reach the preset handover point; determining that the preset handover point needs to be adjusted when the predicted remaining endurance time is less than the reference time; determining a waypoint closest to the UAV in the set of available waypoints as the target handover point; and determining a target landing airport closest to the target handover point from the set of available airports.

[0016] According to another aspect of the embodiments of the present application, a UAV cooperative operation device is further provided, including: an obtaining module configured to obtain a preset flight route of a UAV, wherein the preset flight route is determined by a plurality of discrete waypoints; a determining module configured to determine a preset handover point and a preset landing airport of the UAV according to the preset flight route; a predicting module configured to predict a remaining endurance time of the UAV by using a neural network model to obtain a predicted remaining endurance time, and adjust the preset handover point and the preset landing airport according to the predicted remaining endurance time to obtain a target handover point and a target landing airport; and an executing module configured to perform a plurality of UAV cooperative operations according to the target handover point and the target landing airport, wherein a previous UAV lands at the target landing airport and a next UAV continues to perform the operation at the target handover point.

[0017] According to still another aspect of the embodiments of the present application, an electronic device is further provided, including: a memory configured to store program instructions; and a processor connected with the memory and configured to execute the program instructions to realize the following functions: obtaining a preset flight route of a UAV, wherein the preset flight route is determined by a plurality of discrete waypoints; determining a preset handover point and a preset landing airport of the UAV according to the preset flight route; predicting a remaining endurance time of the UAV by using a neural network model to obtain a predicted remaining endurance time, and adjusting the preset handover point and the preset landing airport according to the predicted remaining endurance time to obtain a target handover point and a target landing airport; and performing a plurality of UAV cooperative operations according to the target handover point and the target landing airport, wherein a previous UAV lands at the target landing airport and a next UAV continues to perform the operation at the target handover point.

[0018] According to still another aspect of the embodiments of the present application, a non-volatile storage medium is further provided, including a stored computer program, wherein a device in which the non-volatile storage medium is located performs the above-mentioned UAV cooperative operation method by running the computer program.

[0019] According to still another aspect of the embodiments of the present application, a computer program product is further provided, including computer instructions, which, when executed by a processor, realize the above-mentioned UAV cooperative operation method.

[0020] In the embodiment of the present application, the preset flight route of the unmanned aerial vehicle is acquired, wherein the preset flight route is determined by a plurality of discrete waypoints; the preset handover point and the preset landing airport of the unmanned aerial vehicle are determined according to the preset flight route; the neural network model is used to predict the remaining endurance time of the unmanned aerial vehicle, to obtain the predicted remaining endurance time, and to adjust the preset handover point and the preset landing airport according to the predicted remaining endurance time, to obtain the target handover point and the target landing airport; and the coordinated operation of a plurality of unmanned aerial vehicles is performed according to the target handover point and the target landing airport, wherein the previous unmanned aerial vehicle lands at the target landing airport and continues to perform the operation at the target handover point by the next unmanned aerial vehicle, so as to expand the effective operation range of the unmanned aerial vehicle and optimize the resource allocation, thereby achieving the technical effect of improving the resource utilization rate, and further solving the technical problems of low resource utilization rate, poor task continuity and difficulty in coping with large-scale task demand in the unmanned aerial vehicle inspection mode in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate the illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0022] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a coordinated operation method of an unmanned aerial vehicle according to an embodiment of the present application;

[0023] Figure 2 is a flowchart of a coordinated operation method of an unmanned aerial vehicle according to an embodiment of the present application;

[0024] Figure 3 is a simulation diagram of a static pre-planning stage according to an embodiment of the present application;

[0025] Figure 4 is a loss function curve diagram according to an embodiment of the present application;

[0026] Figure 5 is a structure diagram of a coordinated operation device of an unmanned aerial vehicle according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] The information collected by the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards in the relevant region, necessary security measures are taken, public order and good customs are not violated, and appropriate operation portals are provided for users to choose to authorize or refuse automated decision results; if the user chooses to refuse, the expert decision process is entered.

[0030] First, some of the nouns or terms that appear in the process of explaining the embodiments of the present application are applicable to the following explanations:

[0031] Unmanned aerial vehicle (UAV): It is a kind of unmanned equipment that can fly in the air without carrying a human pilot. The unmanned aerial vehicle can be remotely operated by the ground control station or autonomously flown through the pre-set program to complete various tasks.

[0032] Induced power (IP): It is the energy consumed to generate lift during the flight of an aircraft (especially a helicopter or a rotor unmanned aerial vehicle). When the aircraft hovers or flies forward, the rotor or propeller pushes the air downward, which generates an upward reaction force, i.e. lift, so that the aircraft can resist gravity and stay in the air. Induced power is the energy required to maintain this lift process.

[0033] Blade Profile Power (BPP): Blade Profile Power refers to the power consumed by the rotor blades of a rotorcraft or helicopter during flight due to air friction. The shape of the rotor blades and the flight speed are related to the power consumed. When the rotor blades cut through the air, they generate resistance. In order to overcome this resistance and maintain flight speed, the engine needs to output additional energy, i.e. blade profile power. In rotorcraft, blade profile power is usually closely related to the flight speed of the unmanned aerial vehicle and the characteristics of the rotor (such as blade shape, solidity). As the speed increases, the blade profile power also increases.

[0034] Parasite Power (PP): Parasite Power refers to the additional energy consumption of an aircraft during flight due to non-lift-related factors, such as the frictional resistance of the fuselage, rotor, landing gear, and other parts, as well as the energy consumption caused by the operation of electronic devices, communication systems, etc.

[0035] Long Short-Term Memory (LSTM): LSTM is a special type of recurrent neural network (RNN) designed to process and predict time series data. It introduces a "gate" mechanism to selectively remember and forget information, effectively overcoming the gradient vanishing or gradient explosion problem of traditional RNNs when processing long sequence data. LSTM maintains a "cell state" in the network, which controls the flow of information in and out of the network through input gates, forget gates, and output gates, allowing the network to remember longer-term dependencies.

[0036] Convolutional Neural Networks (CNN): CNN is a deep learning network structure that is particularly good at processing input data with grid structure, such as images and videos. It can automatically detect multiple levels of features in the input data through convolutional layers, pooling layers, and fully connected layers.

[0037] Mean-Square Error (MSE): MSE is a commonly used loss function to evaluate the difference between predicted values and true values, which is the average of the squared prediction error of all observations.

[0038] In the related art, in order to break through the single machine endurance limit, there are two automatic solutions in the industry. One is a single unmanned aerial vehicle and multiple airport solution. A single unmanned aerial vehicle uses a jump cabin function to perform a flight task in multiple airports in sections. However, the airport facility cost is high, the single unmanned aerial vehicle has low airport utilization rate, and the charging waiting time accumulated by single machine serial operation significantly prolongs the task cycle. The second is a multiple unmanned aerial vehicle and multiple airport solution. The flight task is divided into multiple unmanned aerial vehicles, and all unmanned aerial vehicles take off at the same time to perform segmented tasks and land at the next airport. However, the single machine operation radius under this mode is still less than half of the maximum endurance distance, and when an abnormal situation occurs, the unmanned aerial vehicle cannot reach the next airport, resulting in task interruption.

[0039] The commonly used unmanned aerial vehicle networking solution in the industry cannot solve the problem of small single machine operation radius, and does not integrate high precision, real-time prediction of remaining endurance depth into multi-machine collaborative decision making, lacks an effective mechanism for adjusting tasks and routes based on dynamic endurance state, resulting in poor multi-machine collaborative endurance, low reliability, and poor efficiency.

[0040] In order to solve the problems in the related art, the embodiments of the present application provide a method for unmanned aerial vehicle collaborative operation, which can be run in Figure 1 The computer terminal is explained and described as follows.

[0041] The unmanned aerial vehicle collaborative operation method provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal, or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal for implementing the unmanned aerial vehicle collaborative operation method is shown. As Figure 1 shown, the computer terminal 10 can include one or more (shown as 102a, 102b, …, 102n in the figure) processors (the processor can include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication function connected through a wired and / or wireless network. In addition, it can also include a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, and it does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or fewer components than those shown in Figure 1 , or have a different configuration than Figure 1 .

[0042] It should be noted that the one or more processors and / or other data processing circuitry described above can be referred to herein generally as "data processing circuitry." The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. Furthermore, the data processing circuitry can be a single standalone processing module, or it can be incorporated in whole or in part within any one of the other elements of the computer terminal 10. As referred to in the embodiments of the present application, the data processing circuitry functions as a processor to control, for example, the selection of the variable resistance terminal path in connection with the interface.

[0043] The memory 104 can be used to store software programs of application software and modules, such as the program instructions / data storage means corresponding to the unmanned aerial vehicle cooperative operation method in the embodiments of the present application. The processor executes the software programs and modules stored in the memory 104 to perform various functional applications and data processing, i.e., to implement the unmanned aerial vehicle cooperative operation method described above. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely disposed relative to the processor, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0044] The transmission module 106 is configured to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication service provider of the computer terminal 10. In one example, the transmission module 106 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0045] The display can be, for example, a touch screen type liquid crystal display (LCD) that enables a user to interact with the user interface of the computer terminal 10.

[0046] It should be noted that in some alternative embodiments, the above Figure 1 The computer terminal shown can include hardware elements (including circuitry), software elements (including computer code stored on a computer readable medium), or a combination of both hardware and software elements. It should be noted that in some embodiments, the functions of the computer terminal described above can be implemented as a plurality of separate modules, or as a plurality of separate hardware components. Figure 1 is merely one example of a particular implementation, and is intended to illustrate the types of components that can be present in the computer terminal described above.

[0047] Under the above operating environment, the embodiment of the present application provides a method for cooperative operation of unmanned aerial vehicles. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Figure 2 is a flowchart of a method for cooperative operation of unmanned aerial vehicles according to an embodiment of the present application, as shown in Figure 2 the method comprises the following steps:

[0049] In step S202, a preset flight route of the unmanned aerial vehicle is obtained, wherein the preset flight route is determined by a plurality of discrete waypoints.

[0050] In the above step S202, in the task planning of the unmanned aerial vehicle, the preset flight route refers to one or more flight paths set in advance before the start of the task according to the task requirements and the flight area conditions, and these paths are composed of a series of discrete waypoints, each of which represents a specific position that needs to be passed on the flight path of the unmanned aerial vehicle. The unmanned aerial vehicle will fly according to the order of these waypoints to complete the intended target.

[0051] In step S204, the preset handover point and the preset landing airport of the unmanned aerial vehicle are determined according to the preset flight route.

[0052] In the above step S204, in the mode of relay flight of multiple unmanned aerial vehicles, the preset handover point refers to a specific position for task handover between the previous unmanned aerial vehicle and the subsequent unmanned aerial vehicle. After reaching the preset handover point, the previous unmanned aerial vehicle will hand over the unfinished task to the subsequent unmanned aerial vehicle, and then go to the nearest landing airport (i.e. the preset landing airport) for charging or maintenance. Before the start of the task, the theoretical energy consumption curve of the unmanned aerial vehicle is calculated through a physical energy consumption model to estimate the endurance of the unmanned aerial vehicle, and the preset handover point and the preset landing airport are determined accordingly.

[0053] In step S206, a neural network model is used to predict the remaining endurance time of the unmanned aerial vehicle, obtain the predicted remaining endurance time, and adjust the preset handover point and the preset landing airport according to the predicted remaining endurance time to obtain the target handover point and the target landing airport.

[0054] In the above step S206, in the actual flight process, the remaining endurance time of the unmanned aerial vehicle is predicted in real time through the neural network model, and if the predicted remaining endurance time cannot reach the preset handover point or the preset landing airport, the target handover point and the target landing airport will be determined to ensure the continuity of the task and the safety of the unmanned aerial vehicle.

[0055] Step S208, performing multiple unmanned aerial vehicle cooperative operations according to the target handover point and the target landing airport, wherein the previous unmanned aerial vehicle lands at the target landing airport and continues to perform the operation at the target handover point by the next unmanned aerial vehicle.

[0056] Through the above steps S202 to S208, the purpose of expanding the effective operation range of the unmanned aerial vehicle and optimizing the allocation of resources is achieved, thereby realizing the technical effect of improving the resource utilization rate, and further solving the technical problems of low resource utilization rate, poor task continuity and difficulty in coping with large-scale task demand in the related art. The following is described.

[0057] In step S204 of the above unmanned aerial vehicle cooperative operation method, the preset handover point and the preset landing airport of the unmanned aerial vehicle are determined according to the preset flight route, comprising: obtaining a route starting point in the preset flight route; determining a first airport closest to the route starting point from a set of available airports, wherein each airport in the set of available airports contains a first unmanned aerial vehicle that meets the preset flight condition; determining the first unmanned aerial vehicle flight power of the first unmanned aerial vehicle from the route starting point to other waypoints in the preset flight route, and determining the remaining battery capacity required for the first unmanned aerial vehicle at each waypoint in the preset flight route to a landable airport; determining the farthest waypoint reached by the first unmanned aerial vehicle under the condition of allowable remaining battery capacity as the preset handover point according to the first unmanned aerial vehicle flight power; and determining the available airport closest to the preset handover point as the preset landing airport.

[0058] In some embodiments of the present application, the route starting point is the position where the unmanned aerial vehicle task starts, i.e. the first waypoint in the preset flight route. From all airports, filter out the airports that meet the preset flight condition to obtain the set of available airports, the preset flight condition such as the airport being in working condition and having an idle and sufficient power unmanned aerial vehicle. The airport closest to the route starting point in the set of available airports is determined as the first airport (containing the first unmanned aerial vehicle), and the first unmanned aerial vehicle flight power at each waypoint is calculated based on the physical characteristics of the first unmanned aerial vehicle and external environmental factors using a physical energy consumption model. Flight power reflects the energy consumption of the unmanned aerial vehicle when flying at this point, which affects the endurance of the unmanned aerial vehicle. It is also necessary to calculate the remaining battery capacity required for each waypoint to a landable airport, and according to the first unmanned aerial vehicle flight power, the farthest waypoint reached by the first unmanned aerial vehicle under the condition of allowable remaining battery capacity is determined as the preset handover point. The selection of the preset landing airport is based on the principle of being closest to the preset handover point, but needs to meet the condition that the remaining battery capacity allows the first unmanned aerial vehicle to safely arrive at the preset landing airport.

[0059] The following is described in conjunction with specific examples:

[0060] Traditional drones, when taking off and landing at the same airport, cannot avoid the limitation that their operating radius is only half of their maximum range. This application's embodiment employs a multi-drone, multi-airport collaborative approach, utilizing multi-drone handover to complete the overall mission. Drones do not need to return to the takeoff container; they land at the nearest container, significantly increasing the operating radius of a single drone. Before takeoff, the drones obtain a preset flight path from the flight control platform and use a physical energy consumption model to predict static range and handover points.

[0061] Let the starting point of the route be First, find the distance. Nearest available drone airports (i.e., the first airport mentioned above). Here, "available" is defined as: 1. The airport itself is in a usable state (e.g., capable of takeoffs and landings, no maintenance, open airspace, etc.); 2. There is idle space within the airport and sufficient power. 95% of drones (i.e., the first drone mentioned above), ensuring its ability to perform subsequent tasks. From all available airports that meet the above conditions... Calculate the distance from each airport to actual flight distance Select the airport with the shortest overall distance using the following formula:

[0062]

[0063] In selected back, Take off to At a constant speed along the preset flight path Flying and performing missions. The core of this process lies in predicting remaining range. Based on a physical energy consumption model, this can be calculated. The cumulative energy consumption during flight along the flight path (determined by the flight power of the first UAV) and the remaining battery capacity (which can be simply referred to as battery power) at any position along the flight path. ,according to Determine the next one An airport that can be safely reached and landed at. This application's embodiment predicts a landable airport near the flight path from the current location. (Required battery capacity to meet airport availability requirements) The safe power threshold was taken into account. The prediction objective is to find the set of waypoints that satisfy the following electrical conditions. :

[0064] +

[0065] in, From waypoint Departure, select the preset landing airport with the shortest actual flight distance under available conditions (selection logic is the same). ).from Select the waypoint closest to the preset landing airport. As a pre-set handover point Arrival at the destination The mission ends and the plane flies to Landing. Arrival at the destination forward, drones It has taken off and is flying to Point, from Points take over along the preset flight path To perform subsequent tasks, due to Earlier arrive Therefore, the two drones will not meet.

[0066] right And the process of replacing the drone is repeated: based on the pre-set handover point. The initial energy level of the (i.e., the new starting point) is used to predict the energy consumption of its flight along the remaining route and the new preset handover point it can reach, using a physical energy consumption model. and the corresponding new pre-designated landing airport The range prediction and handover decisions for different drones are logically independent.

[0067] In the above steps, determining the flight power of the first UAV from the starting point of the flight route to other waypoints in the preset flight route includes: obtaining the power set corresponding to the first UAV, wherein the power set includes induced power, drag power, exhaust drag power and the power of the first UAV's communication and payload; and determining the flight power of the first UAV based on the induced power, drag power, exhaust drag power, the power of the first UAV's communication and payload, and the energy efficiency coefficient.

[0068] In some embodiments of this application, the first UAV flight power The calculation formula is as follows:

[0069]

[0070] in, Indicates induced power, Indicates power resistance. Indicates waste resistance power. This indicates the power consumed by the first UAV's communication and payload, namely the power consumed by the UAV's communication system and its mission equipment (such as cameras, sensors, etc.), while also taking into account the battery's health, i.e., the number of charge-discharge cycles. represents the coefficient of performance. To better predict the endurance, the characteristics of the effective capacity of the battery need to be studied. The influence of the battery discharge rate on the effective capacity of the battery can be obtained from the Peukert law, and the actual available capacity of the battery is as follows:

[0071]

[0072] wherein, is the nominal battery capacity, is the rated discharge time, and I is the discharge current of the battery, is the Peukert coefficient, represents the voltage of the battery at point b. When the discharge power is constant, the voltage of the lithium battery linearly decreases and the remaining capacity has the following functional relationship:

[0073]

[0074] wherein, is the full charge voltage, and m is the slope of the linear curve of the voltage drop. When the voltage drops from to the standard voltage , m is given by the following formula, is the nominal battery discharge coefficient:

[0075]

[0076] In the above steps, the power set is determined by the following manner: obtaining the induced power of the first unmanned aerial vehicle when hovering, the ground speed of the first unmanned aerial vehicle, and the average rotor induced speed of the first unmanned aerial vehicle when hovering; determining the induced power of the first unmanned aerial vehicle according to the induced power of the first unmanned aerial vehicle when hovering, the ground speed of the first unmanned aerial vehicle, and the average rotor induced speed; obtaining the profile drag power of the first unmanned aerial vehicle when hovering and the rotor tip speed of the first unmanned aerial vehicle; determining the profile drag power of the first unmanned aerial vehicle according to the profile drag power of the first unmanned aerial vehicle when hovering, the ground speed of the first unmanned aerial vehicle, and the rotor tip speed of the first unmanned aerial vehicle; obtaining the fuselage drag ratio of the first unmanned aerial vehicle, the air density, the rotor solidity of the first unmanned aerial vehicle, and the rotor disc area of the first unmanned aerial vehicle; obtaining the fuselage drag ratio of the first unmanned aerial vehicle, the air density, the rotor solidity of the first unmanned aerial vehicle, and the rotor disc area of the first unmanned aerial vehicle; and determining the power set corresponding to the first unmanned aerial vehicle according to the induced power of the first unmanned aerial vehicle, the profile drag power of the first unmanned aerial vehicle, the wake drag power of the first unmanned aerial vehicle, and the power of the communication and the carried load of the first unmanned aerial vehicle.

[0077] In some embodiments of the present application, the energy consumption of the (first) unmanned aerial vehicle during flight mainly includes induced power , profile drag power , and wake drag power , and the specific calculation formula is as follows:

[0078]

[0079]

[0080]

[0081] wherein, and are the profile power and induced power of the first unmanned aerial vehicle when hovering, respectively; represents the ground speed of the first unmanned aerial vehicle; is the average rotor induced speed of the first unmanned aerial vehicle when hovering; is the rotor tip speed of the first unmanned aerial vehicle; and are the fuselage drag ratio and rotor solidity of the first unmanned aerial vehicle, respectively, and are the air density and rotor disc area of the first unmanned aerial vehicle, respectively.

[0082] In the above step, the ground speed of the first unmanned aerial vehicle is determined by: obtaining the airspeed and flight direction of the first unmanned aerial vehicle, and obtaining the wind speed and wind direction; and determining the ground speed of the first unmanned aerial vehicle according to the airspeed, flight direction, wind speed and wind direction.

[0083] In some embodiments of the present application, the first unmanned aerial vehicle mainly flies at a uniform speed, and the acceleration and deceleration flight accounts for a small proportion which can be ignored, so the energy consumption generated by the acceleration and deceleration of the first unmanned aerial vehicle is not considered in the embodiments of the present application. Considering the influence of wind on the energy consumption of the first unmanned aerial vehicle, it is decomposed into wind speed and wind direction , and the ground speed of the first unmanned aerial vehicle can be represented as:

[0084]

[0085] wherein, and are the airspeed and flight direction of the first unmanned aerial vehicle, respectively.

[0086] In the above step, the induced power of the first unmanned aerial vehicle when hovering is determined by: obtaining the aircraft weight and increment correction coefficient of the first unmanned aerial vehicle; and determining the induced power of the first unmanned aerial vehicle when hovering according to the aircraft weight, increment correction coefficient, air density and rotor disc area.

[0087] In some embodiments of the present application, the induced power of the first unmanned aerial vehicle when hovering is calculated according to the following formula:

[0088]

[0089] wherein, is the aircraft weight of the first unmanned aerial vehicle, is the incremental correction coefficient.

[0090] In the above step, the form drag power of the first unmanned aerial vehicle in hovering is determined by: obtaining the drag coefficient, the rotor angular velocity of the first unmanned aerial vehicle and the rotor radius; and determining the form drag power of the first unmanned aerial vehicle in hovering according to the drag coefficient, the air density, the rotor solidity, the rotor disc area, the rotor angular velocity and the rotor radius.

[0091] In some embodiments of the present application, the form drag power of the first unmanned aerial vehicle in hovering is The calculation formula is as follows:

[0092]

[0093] wherein, is the drag coefficient, is the rotor angular velocity of the first unmanned aerial vehicle, is the rotor radius.

[0094] In the above step, the remaining battery capacity required for the first unmanned aerial vehicle to reach the landable airport from each waypoint in the preset flight route is determined by: obtaining the first remaining battery capacity of the first unmanned aerial vehicle at the previous waypoint, the full charge voltage, the nominal battery capacity and the slope of the linear curve of the voltage drop, wherein the slope is determined by the full charge voltage, the nominal battery capacity, the standard voltage and the nominal battery discharge coefficient; determining the voltage of the first unmanned aerial vehicle at the next waypoint according to the full charge voltage, the slope, the nominal battery capacity and the first remaining battery capacity; determining the current of the first unmanned aerial vehicle at the next waypoint according to the flight power of the first unmanned aerial vehicle and the voltage of the first unmanned aerial vehicle at the next waypoint; and determining the second remaining battery capacity of the first unmanned aerial vehicle at the next waypoint according to the current of the first unmanned aerial vehicle at the next waypoint, the rated discharge time, the nominal battery capacity and the time step, wherein the time step is the number of waypoints flown by the first unmanned aerial vehicle, and the iteration is stopped when the second remaining battery capacity meets the preset battery capacity condition, and the preset battery capacity condition is determined by the nominal battery capacity and the nominal battery discharge coefficient.

[0095] In some embodiments of the present application, the remaining battery capacity is predicted based on the iterative method, because the voltage, the current and the effective capacity dynamically change with the remaining battery capacity. Let the time step be , which is equivalent to that the first unmanned aerial vehicle uniformly flies for time on the route, and the step is the time iteration, and the time is . Assuming that the speed of the first unmanned aerial vehicle is , the preset flight route can be converted into discrete waypoints with an interval of The voltage, current and remaining battery capacity of the first unmanned aerial vehicle in the i-th iteration are calculated as follows:

[0096]

[0097]

[0098]

[0099] wherein, represents the remaining battery capacity of the first unmanned aerial vehicle in the i-th iteration, i.e., the first remaining battery capacity, represents the voltage of the first unmanned aerial vehicle in the i+1-th iteration, i.e., the voltage of the first unmanned aerial vehicle at the next waypoint, represents the current of the first unmanned aerial vehicle in the i+1-th iteration, i.e., the current of the first unmanned aerial vehicle at the next waypoint, represents the current of the first unmanned aerial vehicle at the next waypoint considering the Peukert law, represents the rated discharge time considering the Peukert law, represents the nominal battery capacity considering the Peukert law, represents the remaining battery capacity of the first unmanned aerial vehicle in the i+1-th iteration, i.e., the second remaining battery capacity of the first unmanned aerial vehicle at the next waypoint.

[0100] The second remaining battery capacity is iteratively calculated until the following condition is met:

[0101]

[0102] wherein, represents the preset battery capacity condition.

[0103] The first unmanned aerial vehicle remaining endurance is equivalent to the predicted first unmanned aerial vehicle remaining endurance, and the iteration can also be regarded as predicting the flight waypoint that the first unmanned aerial vehicle can actually fly to. Each iteration needs to synchronously calculate whether the first unmanned aerial vehicle can fly from the current flight waypoint and land at the next available airport, and the calculation logic and iteration process are consistent, i.e., whether the first unmanned aerial vehicle remaining endurance based on the battery capacity of the current flight waypoint can fly to the preset landing airport. If it can, this waypoint is placed in the set of available waypoints, and finally the closest available waypoint to the preset landing airport is selected from the set as the preset handover point. Figure 3 ​For the simulation map of the static pre-planning stage, the blue dots are discrete flight line points, the blue boxes are selected unmanned aerial vehicle airports, the red boxes are other airports, and the green triangles are calculated unmanned aerial vehicle handover points. When the current unmanned aerial vehicle (such as the first unmanned aerial vehicle) approaches the handover point, the replacement unmanned aerial vehicle in the blue airport takes off to the handover point to continue to perform the task along the flight line, and then the current unmanned aerial vehicle arrives at the handover point and flies to the blue airport for landing.

[0104] In the unmanned aerial vehicle cooperative operation method described above, the neural network model is obtained by training in the following manner: obtaining sample data, wherein the sample data includes multi-dimensional feature data of the unmanned aerial vehicle, and the multi-dimensional feature data includes battery state data, flight state data, environmental data, and task load data; processing the sample data using an initial neural network model to obtain a sample predicted remaining endurance time, wherein the initial neural network model includes a first model for processing time series data and a second model for processing spatial data; determining a fusion loss function based on the sample predicted remaining endurance time and the true remaining endurance time, and adjusting the parameters of the initial neural network model based on the loss value of the fusion loss function until the adjusted loss value meets a preset condition to stop iteration, thereby obtaining the neural network model.

[0105] In some embodiments of the present application, due to the many uncertain factors during unmanned aerial vehicle flight, the multi-dimensional features such as battery state, environmental changes, and flight state all affect the remaining endurance prediction. The environmental and flight factors cannot be comprehensively considered during static planning, and only a baseline reference can be provided. The AI neural network model is dynamically adjusted after static pre-planning in the embodiments of the present application, the multi-dimensional feature data is used to correct the static planning, and the remaining endurance time is predicted in real time. The neural network model part is described below.

[0106] LSTM is a commonly used time series prediction model that can handle long-term dependencies in time series data, but is not suitable for processing spatial data; CNN is a commonly used spatial data processing model that can handle spatial features in data, but is not suitable for processing time series data. Since the energy consumption of the unmanned aerial vehicle is affected by environmental spatial fields such as flight altitude, flight terrain elevation, and airspace restrictions, and the state of the unmanned aerial vehicle (position, attitude, battery state, etc.) changes continuously in time, the ConvLSTM combining LSTM (i.e., the first model described above) and CNN (i.e., the second model described above) architectures is used for real-time endurance prediction of the unmanned aerial vehicle in the embodiments of the present application. The spatial features affecting the energy consumption of the unmanned aerial vehicle are extracted through convolution operation, and the long-term influence of the state sequence of the unmanned aerial vehicle and its cumulative effect on energy consumption is modeled using the gating recurrent mechanism. The data processing process of the input gate , the forget gate , and the output gate is as follows:

[0107]

[0108]

[0109]

[0110] wherein, is an activation function, is a convolution kernel, different subscripts represent different convolution kernels, is a bias vector, different subscripts represent different bias vectors, is a hidden state at the last time, is a current input.

[0111] The neural network model uses more than 20 characteristic types including battery state data (voltage, current, remaining power, temperature, battery health), flight state data (altitude, speed, attitude angle, flight mode), environmental data (wind speed, temperature), task load data (unmanned aerial vehicle weight, hanging power consumption, hanging weight) as sample data during training. The initial neural network model processes the sample data to obtain the sample predicted remaining endurance time. According to the difference between the sample predicted endurance time and the real remaining endurance time, the loss value of the fusion loss function is calculated, and then the parameters of the neural network model are adjusted to reduce the loss value. Repeat the above process until the adjusted model parameters make the loss value meet the preset condition (such as the loss value reaches the minimum or the change amplitude is less than the threshold), and the training process stops, thereby obtaining the above neural network model. After about 25 rounds of iterative training, the loss function curve shown in FIG. 8 is obtained, and the endurance prediction error within 1 minute accounts for 93.5%. Figure 4

[0112] In the above steps, the fusion loss function is determined by the following methods: determining the mean square error function of the sample predicted remaining endurance time and the real remaining endurance time, and determining the mean square error function as the main loss; determining the asymmetric loss according to the first penalty coefficient, the second penalty coefficient, the sample predicted remaining endurance time and the real remaining endurance time; determining the multi-task learning loss according to the weight coefficient of the multiple auxiliary tasks and the corresponding mean square error loss function, wherein the multiple auxiliary tasks include voltage auxiliary prediction and flight state prediction; determining the fusion loss function according to the main loss, the asymmetric loss and the multi-task learning loss.

[0113] In some embodiments of the present application, in order to improve the prediction accuracy of the model, the present application defines a fusion loss function composed of a main loss , an asymmetric loss and a multi-task learning loss . Specifically as follows:

[0114] ​

[0115]

[0116]

[0117]

[0118] wherein, is the number of samples, is the true remaining endurance time, is the sample predicted remaining endurance time. The main loss in the loss function is the mean square error function, and the asymmetric loss introduces a penalty coefficient and ( ), wherein, is the first penalty coefficient, is the second penalty coefficient, and the penalty is heavier when the sample predicted remaining endurance time is greater than the true remaining endurance time (overestimation). is a sigmoid function, indicates that the penalty of the asymmetric loss is heavier when the sample predicted remaining endurance time is less, i.e., the power is lower, thereby enhancing the prediction accuracy at low power. Multi-task learning (also known as multi-assistant task) includes voltage auxiliary prediction and flight state prediction, and are the weight coefficients of different auxiliary tasks and the MSE (mean square error) loss function.

[0119] During flight, the neural network model dynamically predicts the remaining endurance time according to real-time and historical flight states, and simultaneously determines whether the preset handover point generated in the pre-planning stage is valid. If the neural network model continuously determines that the handover point is invalid for multiple times, a new handover point is selected from the set of available waypoints. Through the combination of static planning and dynamic correction, the system can real-time perceive the changes in the flight environment, automatically adjust the pre-planning scheme, and ensure the dynamic feasibility and safety of the selection of the handover point.

[0120] In step S206 in the above-mentioned method for cooperative operation of the unmanned aerial vehicle, the preset handover point and the preset landing airport are adjusted according to the predicted remaining endurance time to obtain a target handover point and a target landing airport, including: obtaining a reference time required for the unmanned aerial vehicle to reach the preset handover point; determining that the preset handover point needs to be adjusted when the predicted remaining endurance time is less than the reference time; determining the waypoint closest to the unmanned aerial vehicle in the set of available waypoints as the target handover point; and determining the target landing airport closest to the target handover point from the set of available airports.

[0121] In some embodiments of the present application, the reference time refers to the time required for the UAV to reach the preset handover point from the current position according to the theoretical calculation or historical data of the static planning stage. Comparing the predicted remaining endurance time with the reference time of reaching the preset handover point, if the predicted remaining endurance time is less than the reference time, it indicates that the UAV may not be able to reach the preset handover point as planned, and the strategy needs to be adjusted immediately. The flight point closest to the current position of the UAV is selected from the set of available flight points as the new target handover point, thereby ensuring that the UAV completes the handover within the safe range of power, and the airport closest to the target handover point is selected as the target landing airport.

[0122] The UAV cooperative operation method provided by the embodiments of the present application adopts a multi-UAV heterogeneous cabin relay take-off and landing mode under a multi-UAV network, breaks through the endurance limit, and can additionally release 30%-40% of the endurance of the UAV. Through precise endurance perception and relay mechanism, long-distance and long-time tasks can be reliably and continuously completed, and task interruption caused by power problems is avoided. The problems of low infrastructure utilization rate, long task cycle, poor flexibility, etc. existing in traditional multi-UAV cooperative schemes are solved. In addition, in the static pre-planning stage, the remaining endurance is predicted based on the independence of different UAVs flying and the physical energy consumption model, the global task is divided into multiple sub-tasks, and various energy consumptions such as induced power, profile drag power, and waste drag power, as well as wind speed, battery health, and other factors are comprehensively considered to optimize the utilization of the overall endurance resources of the UAV fleet. In the dynamic AI adjustment stage, the flight scene of the UAV is modeled and the loss function is fused, the mean square error (main loss), asymmetric loss, and multi-task learning loss are introduced into the space-time neural network, and the influence of sudden interference is reduced. At the same time, the system can perceive and respond to changes in flight conditions in real time by using rich real-time flight-related states (such as wind speed and attitude), automatically corrects the pre-planning scheme, and the endurance prediction error within 1 minute accounts for 93.5%, ensuring the dynamic safety and feasibility of the relay point selection, thereby greatly enhancing the adaptability and robustness of the system in uncertain environments.

[0123] Figure 5 is a structural diagram of a UAV cooperative operation device according to an embodiment of the present application, as shown in Figure 5 The device comprises:

[0124] The acquisition module 40 is configured to acquire a preset flight route of the UAV, wherein the preset flight route is determined by a plurality of discrete flight points.

[0125] The determination module 42 is configured to determine a preset handover point and a preset landing airport of the UAV according to the preset flight route.

[0126] The prediction module 44 is configured to predict the remaining endurance time of the UAV by using the neural network model, obtain a predicted remaining endurance time, and adjust the preset handover point and the preset landing airport according to the predicted remaining endurance time to obtain a target handover point and a target landing airport.

[0127] The execution module 46 is configured to perform the cooperative work of the plurality of UAVs according to the target handover point and the target landing airport, wherein the previous UAV lands at the target landing airport and continues to perform the work at the target handover point by the next UAV.

[0128] The acquisition module, the determination module, the prediction module and the execution module in the cooperative work device of the UAVs can expand the effective work range of the UAVs and optimize the resource allocation, thereby achieving the technical effect of improving the resource utilization rate and solving the technical problems of low resource utilization rate, poor task continuity and difficulty in coping with large-scale task demand in the related art.

[0129] In the determination module in the cooperative work device of the UAVs, the determination module is further configured to acquire a route starting point in the preset flight route, determine a first airport closest to the route starting point from a set of available airports, wherein each airport in the set of available airports contains a first UAV satisfying the preset flight condition, determine a first UAV flight power of the first UAV from the route starting point to other waypoints in the preset flight route, and determine a remaining battery capacity required by the first UAV at each waypoint in the preset flight route to a landable airport, determine, according to the first UAV flight power, a farthest waypoint reached by the first UAV under the condition of the remaining battery capacity as the preset handover point, and determine an available airport closest to the preset handover point as the preset landing airport.

[0130] In the determination module in the cooperative work device of the UAVs, the determination module is further configured to acquire a power set corresponding to the first UAV, wherein the power set includes induced power, form drag power, waste drag power and power of the first UAV communication and the carried load, and determine the first UAV flight power according to the induced power, the form drag power, the waste drag power, the power of the first UAV communication and the carried load and the energy efficiency coefficient.

[0131] In the determination module of the unmanned aerial vehicle cooperative operation device, the determination module is further configured to determine a power set, and specifically, the power set is determined by: obtaining an induced power of the first unmanned aerial vehicle when hovering, a ground speed of the first unmanned aerial vehicle, and an average rotor induced speed of the first unmanned aerial vehicle when hovering; determining an induced power of the first unmanned aerial vehicle according to the induced power of the first unmanned aerial vehicle when hovering, the ground speed of the first unmanned aerial vehicle, and the average rotor induced speed; obtaining a profile drag power of the first unmanned aerial vehicle when hovering and a rotor tip speed of the first unmanned aerial vehicle; determining a profile drag power of the first unmanned aerial vehicle according to the profile drag power of the first unmanned aerial vehicle when hovering, the ground speed of the first unmanned aerial vehicle, and the rotor tip speed of the first unmanned aerial vehicle; obtaining a fuselage drag ratio of the first unmanned aerial vehicle, an air density, a rotor solidity of the first unmanned aerial vehicle, and a rotor disc area of the first unmanned aerial vehicle; obtaining the fuselage drag ratio of the first unmanned aerial vehicle, the air density, the rotor solidity of the first unmanned aerial vehicle, and the rotor disc area of the first unmanned aerial vehicle; and determining the power set corresponding to the first unmanned aerial vehicle according to the induced power of the first unmanned aerial vehicle, the profile drag power of the first unmanned aerial vehicle, a waste drag power of the first unmanned aerial vehicle, and a power of communication and a carried load of the first unmanned aerial vehicle.

[0132] In the determination module of the unmanned aerial vehicle cooperative operation device, the determination module is further configured to determine a ground speed of the first unmanned aerial vehicle, and specifically, the ground speed of the first unmanned aerial vehicle is determined by: obtaining an airspeed of the first unmanned aerial vehicle and a flight direction, and obtaining a wind speed and a wind direction; and determining the ground speed of the first unmanned aerial vehicle according to the airspeed, the flight direction, the wind speed, and the wind direction.

[0133] In the determination module of the unmanned aerial vehicle cooperative operation device, the determination module is further configured to determine an induced power of the first unmanned aerial vehicle when hovering, and specifically, the induced power of the first unmanned aerial vehicle when hovering is determined by: obtaining an aircraft weight of the first unmanned aerial vehicle and an increment correction coefficient; and determining the induced power of the first unmanned aerial vehicle when hovering according to the aircraft weight, the increment correction coefficient, an air density, and a rotor disc area.

[0134] In the determination module of the unmanned aerial vehicle cooperative operation device, the determination module is further configured to determine a profile drag power of the first unmanned aerial vehicle when hovering, and specifically, the profile drag power of the first unmanned aerial vehicle when hovering is determined by: obtaining a drag coefficient, a rotor angular speed of the first unmanned aerial vehicle, and a rotor radius; and determining the profile drag power of the first unmanned aerial vehicle when hovering according to the drag coefficient, an air density, a rotor solidity, a rotor disc area, the rotor angular speed, and the rotor radius.

[0135] In the determination module of the unmanned aerial vehicle cooperative operation device, the determination module is further configured to obtain a first residual battery capacity of the first unmanned aerial vehicle at a previous waypoint, a full charge voltage, a nominal battery capacity, and a slope of a linear curve of voltage drop, wherein the slope is determined by the full charge voltage, the nominal battery capacity, a standard voltage, and a nominal battery discharge coefficient; determine a voltage of the first unmanned aerial vehicle at a next waypoint according to the full charge voltage, the slope, the nominal battery capacity, and the first residual battery capacity; determine a current of the first unmanned aerial vehicle at the next waypoint according to a flight power of the first unmanned aerial vehicle and the voltage of the first unmanned aerial vehicle at the next waypoint; and determine a second residual battery capacity of the first unmanned aerial vehicle at the next waypoint according to the current of the first unmanned aerial vehicle at the next waypoint, a rated discharge time, the nominal battery capacity, and a time step, wherein the time step is a number of waypoints flown by the first unmanned aerial vehicle, and iteration is stopped when the second residual battery capacity meets a preset battery capacity condition, and the preset battery capacity condition is determined by the nominal battery capacity and the nominal battery discharge coefficient.

[0136] In the unmanned aerial vehicle cooperative operation device, the training module 48 is further configured to train the neural network model, and specifically, the neural network model is trained in the following manner: obtaining sample data, wherein the sample data includes multi-dimensional feature data of the unmanned aerial vehicle, and the multi-dimensional feature data includes battery state data, flight state data, environment data, and task load data; processing the sample data using an initial neural network model to obtain a sample predicted remaining endurance time, wherein the initial neural network model includes a first model for processing time series data and a second model for processing spatial data; determining a fusion loss function according to the sample predicted remaining endurance time and a real remaining endurance time, and adjusting parameters of the initial neural network model according to a loss value of the fusion loss function until an adjusted loss value meets a preset condition to stop iteration, thereby obtaining the neural network model.

[0137] In the training module of the unmanned aerial vehicle cooperative operation device, the training module is further configured to determine a fusion loss function, and specifically, the fusion loss function is determined in the following manner: determining a mean square error function of the sample predicted remaining endurance time and the real remaining endurance time, and determining the mean square error function as a main loss; determining an asymmetric loss according to a first penalty coefficient, a second penalty coefficient, the sample predicted remaining endurance time, and the real remaining endurance time; determining a multi-task learning loss according to weight coefficients of multiple auxiliary tasks and corresponding mean square error loss functions, wherein the multiple auxiliary tasks include voltage auxiliary prediction and flight state prediction; and determining the fusion loss function according to the main loss, the asymmetric loss, and the multi-task learning loss.

[0138] In the prediction module of the unmanned aerial vehicle cooperative operation device, the prediction module is further configured to obtain a reference time required for the unmanned aerial vehicle to reach the preset handover point; determine that the preset handover point needs to be adjusted when the predicted remaining endurance time is less than the reference time; determine a target handover point as a waypoint in the set of available waypoints closest to the unmanned aerial vehicle; and determine a target landing airport closest to the target handover point from the set of available airports.

[0139] It should be noted that, Figure 5 The unmanned aerial vehicle cooperative operation device is configured to perform the unmanned aerial vehicle cooperative operation method. Figure 2 The unmanned aerial vehicle cooperative operation device is configured to perform the unmanned aerial vehicle cooperative operation method.

[0140] The electronic device is configured to perform the unmanned aerial vehicle cooperative operation method.

[0141] It should be noted that, Figure 2 The electronic device is configured to perform the unmanned aerial vehicle cooperative operation method.

[0142] The non-volatile storage medium includes a stored computer program, and a device in which the non-volatile storage medium is located performs the following unmanned aerial vehicle cooperative operation method by running the computer program: obtaining a preset flight route of an unmanned aerial vehicle, wherein the preset flight route is determined by a plurality of discrete waypoints; determining a preset handover point and a preset landing airport of the unmanned aerial vehicle according to the preset flight route; predicting a remaining endurance time of the unmanned aerial vehicle using a neural network model to obtain a predicted remaining endurance time, and adjusting the preset handover point and the preset landing airport according to the predicted remaining endurance time to obtain a target handover point and a target landing airport; and performing cooperative operation of a plurality of unmanned aerial vehicles according to the target handover point and the target landing airport, wherein a previous unmanned aerial vehicle lands at the target landing airport and continues to perform work at the target handover point by a next unmanned aerial vehicle.

[0143] It should be noted that the non-volatile storage medium is used to execute Figure 2 The non-volatile storage medium is used to execute the unmanned aerial vehicle cooperative operation method shown in the figure, and therefore the relevant explanations of the unmanned aerial vehicle cooperative operation method are also applicable to the non-volatile storage medium, which will not be repeated here.

[0144] The embodiments of the present application also provide a computer program product, which comprises computer instructions, and the computer instructions are executed by a processor to implement the steps of the unmanned aerial vehicle cooperative operation method in the embodiments of the present application.

[0145] The embodiments of the present application also provide a computer program, which is executed by a processor to implement the steps of the unmanned aerial vehicle cooperative operation method in the embodiments of the present application.

[0146] The above sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0147] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0148] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit described as the division is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0149] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0150] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware, or in the form of software functional unit.

[0151] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0152] The above only describes the preferred embodiments of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should also be considered as the protection scope of the present application.

Claims

1. A method for collaborative operation of unmanned aerial vehicles (UAVs), characterized in that, include: Obtain the preset flight path of the UAV, wherein the preset flight path is determined by multiple discrete waypoints; The preset handover point and preset landing airport of the UAV are determined based on the preset flight route; A neural network model is used to predict the remaining flight time of the UAV, and the predicted remaining flight time is obtained. Based on the predicted remaining flight time, the preset handover point and the preset landing airport are adjusted to obtain the target handover point and the target landing airport. Multiple drones coordinate operations based on the target handover point and the target landing airport. The previous drone lands at the target landing airport, and the next drone continues the operation at the target handover point. Before the previous drone arrives at the target landing airport, the next drone takes off and flies to the target landing airport. The neural network model is trained as follows: Sample data is acquired, including multi-dimensional feature data of the UAV, such as battery status data, flight status data, environmental data, and mission payload data; an initial neural network model is used to process the sample data to obtain the predicted remaining flight time, wherein the initial neural network model includes a first model for processing time-series data and a second model for processing spatial data; a fusion loss function is determined based on the predicted and actual remaining flight times, and the parameters of the initial neural network model are adjusted according to the loss value of the fusion loss function until the adjusted loss value meets a preset condition, at which point iteration stops, thus obtaining the neural network model. The fusion loss function is determined as follows: The mean squared error function of the predicted remaining flight time and the actual remaining flight time of the samples is determined, and this mean squared error function is determined as the main loss; an asymmetric loss is determined based on a first penalty coefficient, a second penalty coefficient, the predicted remaining flight time of the samples, and the actual remaining flight time; a multi-task learning loss is determined based on the weight coefficients of the multiple auxiliary tasks and their corresponding mean squared error loss functions, wherein the multiple auxiliary tasks include voltage-assisted prediction and flight status prediction; and the fusion loss function is determined based on the main loss, the asymmetric loss, and the multi-task learning loss.

2. The method according to claim 1, characterized in that, Determining the preset handover point and preset landing airport for the UAV based on the preset flight route includes: Obtain the starting point of the preset flight route; Determine the first airport closest to the origin of the route from the set of available airports, wherein each airport in the set of available airports contains a first UAV that meets preset flight conditions; Determine the first UAV's flight power from the starting point of the flight route to other waypoints in the preset flight route, and determine the remaining battery capacity required by the first UAV to reach a landable airport at each waypoint in the preset flight route. Based on the flight power of the first UAV, the furthest waypoint that the first UAV can reach under the condition that the remaining battery capacity allows is determined as the preset handover point; The available airport closest to the preset handover point is determined as the preset landing airport.

3. The method according to claim 2, characterized in that, Determining the flight power of the first UAV from the starting point of the flight route to other waypoints in the preset flight route includes: Obtain the power set corresponding to the first UAV, wherein the power set includes induced power, type resistance power, waste resistance power and the power of communication and payload of the first UAV; The flight power of the first UAV is determined based on the induced power, the type resistance power, the exhaust resistance power, the power of the first UAV communication and its payload, and the energy efficiency coefficient.

4. The method according to claim 3, characterized in that, The power set is determined in the following way: The induced power, ground speed, and average rotor induced speed of the first UAV during hovering are obtained. The induced power of the first UAV is determined based on the induced power of the first UAV when hovering, the ground speed of the first UAV, and the average rotor induced speed. Obtain the drag power of the first UAV when hovering and the rotor tip speed of the first UAV; The drag power of the first UAV is determined based on its hovering drag power, ground speed, and rotor tip speed. Obtain the fuselage drag ratio, air density, rotor solidity, and rotor disk area of ​​the first UAV; The exhaust drag power of the first UAV is determined based on the fuselage drag ratio, the air density, the rotor solidity, the rotor disk area, and the ground speed of the first UAV. Based on the induced power, the form resistance power, the exhaust resistance power, and the communication and payload power of the first UAV, the power set corresponding to the first UAV is determined.

5. The method according to claim 4, characterized in that, The ground speed of the first UAV is determined in the following way: Obtain the airspeed and flight direction of the first UAV, as well as the wind speed and wind direction; The ground speed of the first UAV is determined based on the airspeed, flight direction, wind speed, and wind direction.

6. The method according to claim 4, characterized in that, The induced power of the first UAV during hovering is determined in the following way: Obtain the aircraft weight and incremental correction coefficient of the first UAV; The induced power of the first UAV during hovering is determined based on the aircraft weight, the incremental correction coefficient, the air density, and the rotor disk area.

7. The method according to claim 4, characterized in that, The drag power of the first UAV during hovering is determined in the following way: Obtain the drag coefficient, the rotor angular velocity, and the rotor radius of the first UAV; The drag power of the first UAV when hovering is determined based on the drag coefficient, air density, rotor solidity, rotor disk area, rotor angular velocity, and rotor radius.

8. The method according to claim 2, characterized in that, Determining the remaining battery capacity required for the first UAV to reach a landable airport at each waypoint along the preset flight path includes: Obtain the slope of the linear curve of the first UAV's first remaining battery capacity, full charge voltage, nominal battery capacity, and voltage drop at the previous waypoint, wherein the slope is determined by the full charge voltage, the nominal battery capacity, the standard voltage, and the nominal battery discharge coefficient; Based on the full charge voltage, the slope, the nominal battery capacity, and the first remaining battery capacity, determine the voltage of the first UAV at the next waypoint; Based on the flight power of the first UAV and the voltage of the first UAV at the next waypoint, determine the current of the first UAV at the next waypoint; Based on the current, rated discharge time, nominal battery capacity, and time step of the first UAV at the next waypoint, the second remaining battery capacity of the first UAV at the next waypoint is determined, wherein the time step is the number of waypoints flown by the first UAV. The iteration stops when the second remaining battery capacity meets the preset battery capacity condition, which is determined by the nominal battery capacity and the nominal battery discharge coefficient.

9. The method according to claim 1, characterized in that, Adjusting the preset handover point and the preset landing airport based on the predicted remaining flight time to obtain the target handover point and the target landing airport includes: Obtain the reference time required for the drone to reach the preset handover point; If the predicted remaining battery life is less than the reference time, it is determined that the preset handover point needs to be adjusted. The waypoint closest to the UAV in the set of available waypoints is determined as the target handover point; Determine the target landing airport that is closest to the target handover point from the set of available airports.

10. A drone collaborative operation device, characterized in that, include: An acquisition module is used to acquire a preset flight path of the UAV, wherein the preset flight path is determined by multiple discrete waypoints; The determination module is used to determine the preset handover point and preset landing airport of the UAV based on the preset flight route; The prediction module is used to predict the remaining flight time of the UAV using a neural network model, obtain the predicted remaining flight time, and adjust the preset handover point and the preset landing airport based on the predicted remaining flight time to obtain the target handover point and the target landing airport. An execution module is used to perform collaborative operations of multiple drones based on the target handover point and the target landing airport. The previous drone lands at the target landing airport and the next drone continues to perform the operation at the target handover point. Before the previous drone arrives at the target landing airport, the next drone takes off and flies to the target landing airport. A training module is used to train a neural network model, which is obtained through the following methods: acquiring sample data, wherein the sample data includes multi-dimensional feature data of the UAV, including battery status data, flight status data, environmental data, and mission payload data; processing the sample data using an initial neural network model to obtain the sample predicted remaining flight time, wherein the initial neural network model includes a first model for processing time series data and a second model for processing spatial data; determining a fusion loss function based on the sample predicted remaining flight time and the actual remaining flight time, and adjusting the parameters of the initial neural network model based on the loss value of the fusion loss function. The iteration continues until the adjusted loss value meets a preset condition, at which point the iteration stops, thus obtaining the neural network model. The fusion loss function is determined as follows: the mean squared error function of the sample's predicted remaining flight time and the actual remaining flight time is determined, and the mean squared error function is determined as the main loss; the asymmetric loss is determined based on the first penalty coefficient, the second penalty coefficient, the sample's predicted remaining flight time, and the actual remaining flight time; the multi-task learning loss is determined based on the weight coefficients of the multiple auxiliary tasks and the corresponding mean squared error loss function, wherein the multiple auxiliary tasks include voltage-assisted prediction and flight status prediction; and the fusion loss function is determined based on the main loss, the asymmetric loss, and the multi-task learning loss.

11. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor, connected to the memory, is configured to execute program instructions to perform the following functions: acquiring a preset flight path of the UAV, wherein the preset flight path is determined by multiple discrete waypoints; determining a preset handover point and a preset landing airport of the UAV based on the preset flight path; predicting the remaining flight time of the UAV using a neural network model to obtain a predicted remaining flight time, and adjusting the preset handover point and the preset landing airport based on the predicted remaining flight time to obtain a target handover point and a target landing airport; performing collaborative operations of multiple UAVs based on the target handover point and the target landing airport, wherein the previous UAV lands at the target landing airport, and the next UAV continues the operation at the target handover point; before the previous UAV reaches the target landing airport, the next UAV takes off and flies to the target landing airport; the neural network model is trained by: acquiring sample data, wherein the sample data includes multi-dimensional feature data of the UAV, including battery status data, flight status data, environmental data, and mission payload data; and using an initial neural network model. The network model processes the sample data to obtain the sample predicted remaining flight time. The initial neural network model includes a first model for processing time-series data and a second model for processing spatial data. A fusion loss function is determined based on the sample predicted remaining flight time and the actual remaining flight time. The parameters of the initial neural network model are adjusted based on the loss value of the fusion loss function until the adjusted loss value meets a preset condition, at which point iteration stops, resulting in the neural network model. The fusion loss function is determined as follows: the mean squared error function of the sample predicted remaining flight time and the actual remaining flight time is determined, and this mean squared error function is used as the main loss. An asymmetric loss is determined based on a first penalty coefficient, a second penalty coefficient, the sample predicted remaining flight time, and the actual remaining flight time. A multi-task learning loss is determined based on the weight coefficients of multiple auxiliary tasks and their corresponding mean squared error loss functions. The multiple auxiliary tasks include voltage-assisted prediction and flight status prediction. The fusion loss function is determined based on the main loss, the asymmetric loss, and the multi-task learning loss.

12. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the UAV collaborative operation method according to any one of claims 1 to 9 by running the computer program.

13. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the UAV collaborative operation method according to any one of claims 1 to 9.

Images