Data collection-oriented multi-unmanned aerial vehicle dynamic path planning method, device, equipment and medium

Abstract

The application discloses a multi-unmanned aerial vehicle dynamic path planning method and device for data collection, equipment and medium, wherein the method comprises two stages, wherein the first stage, the off-line path pre-planning stage adopts a multi-agent deep deterministic policy gradient algorithm, assigns each unmanned aerial vehicle with its own Actor-Critic network for training and learning, and simultaneously, a general Critic network is separately designed for each unmanned aerial vehicle to observe global information and realize cooperative learning among the unmanned aerial vehicles; the second stage, the on-line path re-planning stage adopts a genetic algorithm to search and iterate the map containing static interest points and dynamic interest points at the beginning of each time slot, find the optimal path in the current time slot, and re-plan the path for the unmanned aerial vehicle; thus, through the combination of the two stages, the unmanned aerial vehicle cluster can maximize the total reward obtained by data collection under the consideration of both dynamic and static interest points.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle autonomous navigation and control, and particularly relates to a multi-unmanned aerial vehicle dynamic path planning method and device for data collection, equipment and medium. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicles, they play an increasingly important role in many fields such as agriculture, mapping, rescue, and surveillance. In data collection tasks such as geographic information collection and environmental monitoring, how to effectively maximize the total reward of data collection (including factors such as information age, which is distinguished from pure data volume) under the limitation of limited battery energy has become a research hotspot. In these applications, unmanned aerial vehicles usually need to fly between multiple pre-defined points of interest (POI), each POI has different data value or reward value, and completing the data collection task of each POI can obtain the corresponding reward.

[0003] In related technologies, heuristic algorithms, path finding and graph traversal search algorithms such as A* search algorithm and Dijkstra algorithm can well help unmanned aerial vehicles to find the shortest path from the starting point to a series of target points while avoiding obstacles and considering energy consumption constraints, but they need to process a large number of nodes and edges, and still require a large amount of computing resources when dealing with complex or large-scale environments, and are usually based on pre-defined static maps; for dynamic changes that may occur in the environment, such as temporary no-fly zones, POI points or the movement of other unmanned aerial vehicles, these algorithms may not be able to adapt or update the path in real time. SUMMARY

[0004] The present application aims to at least solve one of the above technical problems to some extent. To this end, one object of the present application is to propose a multi-unmanned aerial vehicle dynamic path planning method for data collection, which can maximize the reward value obtained by collecting fixed POI points, and when executing the fixed POI point collection task, it can consider the newly appeared dynamic POI points as a whole, and replan the path online, so that the unmanned aerial vehicle cluster obtains the maximum total reward value in the entire time slot.

[0005] To achieve the above object, the first aspect of the present application proposes a multi-UAV dynamic path planning method for data collection, comprising the following steps: in each time slot, obtaining the state information corresponding to each UAV in the data collection scene, wherein the state information comprises UAV self information, environment information and static interest point information; training the corresponding reinforcement learning network according to the state information, and obtaining the pre-planned path in offline time by using the trained reinforcement learning network; when starting online flight in each time slot, flying according to the pre-planned path, and obtaining the corresponding dynamic interest point information in the current time slot; adjusting the pre-planned path according to the dynamic interest point information by using the genetic algorithm to obtain the corresponding optimal planning path.

[0006] According to the above technical means, the embodiments of the present application apply the multi-agent reinforcement learning algorithm to the path planning problem of UAV data collection task, so that the platform plans a reasonable path for each UAV, which can maximize the reward value obtained by collecting fixed POI points; at the same time, when executing the fixed POI point collection task, the genetic algorithm is used to consider the newly appeared dynamic POI point, and the path is replanned online, so that the UAV cluster obtains the maximum total reward value in the whole time slot.

[0007] In addition, the multi-UAV dynamic path planning method for data collection proposed in the above embodiments of the present application can have the following additional technical features:

[0008] Further, training the corresponding reinforcement learning network according to the state information comprises: initializing the corresponding Actor-Critic network for each UAV; each UAV uses the Actor network to predict the corresponding flight action based on the current state information; after executing the flight action, the UAV receives the environment feedback to obtain new state information and reward; after each execution of the flight action, the current state information, the corresponding flight action, the new state information and the reward are stored as a batch of samples in the experience replay buffer; a batch of samples are randomly extracted from the experience replay buffer for training.

[0009] Further, the training process further comprises: using the Critic network to evaluate the calculated expected reward value of executing the flight action in the state, wherein the calculated expected reward value represents the expected cumulative reward of executing a certain action in the current state and following the current strategy; the Actor network calculates the policy gradient according to the feedback of the Critic network, and updates the parameters of the Actor network by using the gradient ascent method to optimize the future action prediction.

[0010] Further, the genetic algorithm is used to adjust the pre-planned path according to the dynamic interest point information to obtain a corresponding optimal planning path, including: defining a fitness function, and calculating the fitness according to the total energy consumed by the unmanned aerial vehicle in executing the assigned task, whether the dynamic interest point data collection task is completed, and the reward obtained after the dynamic interest point collection is completed; the genetic algorithm selects a better path for breeding, and performs cross and mutation operations to generate a new optimal planning path.

[0011] Further, whether the dynamic interest point data collection task is completed is judged according to the following formula:

[0012]

[0013] Wherein, a n,(k,t),t′ represents the allocation of the unmanned aerial vehicle for the dynamic interest point q (k,t) , q (k,t) represents the dynamic interest point numbered k appearing in the t th time slot, t' represents the time when the unmanned aerial vehicle executes the task, represents the total data amount of the dynamic interest point q (k,t) collected by all unmanned aerial vehicles in T time slots, wherein for any unmanned aerial vehicle u n ∈U, the data amount collected in each time slot is d n , D ′ k represents the data amount required by the dynamic interest point q (k,t) , if then it represents that the collection of this dynamic interest point is completed, and the reward r k can be obtained.

[0014] Further, the fitness function F is calculated according to the following formula:

[0015]

[0016] Wherein, represents the reward obtained after the dynamic interest point q (k,t′) collects the target data amount, E k is the total energy consumed by the unmanned aerial vehicle in executing the assigned task, including flight energy consumption and hovering energy consumption, k o is a constant depending on the size of the blade, m n represents the mass of the unmanned aerial vehicle u n , ρ represents the air density, X n represents the area of the unmanned aerial vehicle, represents the flight speed of the unmanned aerial vehicle, represents the rotor speed of the blade, represents the average induced velocity of the propeller in the forward combat mode, F is a constant depending on the drag coefficient of the UAV, and I represents the distance from the UAV to the dynamic point of interest q (k,t′) .

[0017] The second aspect embodiment of the present application provides a multi-UAV dynamic path planning device for data collection, comprising a first acquisition module, configured to acquire state information corresponding to each UAV in a data collection scene in each time slot, wherein the state information comprises UAV self information, environment information and static point of interest information; an offline path planning module, configured to train a corresponding reinforcement learning network according to the state information, and obtain a pre-planned path in an offline time by using the trained reinforcement learning network; a second acquisition module, configured to fly according to the pre-planned path when each time slot starts to fly online, and acquire corresponding dynamic point of interest information in the current time slot; and an online path re-planning module, configured to adjust the pre-planned path according to the dynamic point of interest information by using a genetic algorithm, so as to obtain a corresponding optimal planned path.

[0018] The third aspect embodiment of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-UAV dynamic path planning method for data collection according to the above-mentioned embodiments.

[0019] The fourth aspect embodiment of the present application provides a computer readable storage medium, which stores computer instructions for making the computer execute the multi-UAV dynamic path planning method for data collection according to the above-mentioned embodiments.

[0020] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 FIG. 1 is a flowchart of a multi-UAV dynamic path planning method for data collection according to an embodiment of the present application;

[0022] Figure 2 FIG. 2 is a schematic diagram of UAV path planning in a data collection scene according to an embodiment of the present application;

[0023] Figure 3 FIG. 3 is a time-sharing two-dimensional schematic map in a data collection scene according to an embodiment of the present application;

[0024] Figure 4 FIG. 4 is a block schematic diagram of a multi-UAV dynamic path planning device for data collection according to an embodiment of the present application;

[0025] Figure 5 An example diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Embodiments of the present application will be described in detail below with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations are used to denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and should not be understood as limiting the present application.

[0027] In order to better understand the above technical solutions, the exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0028] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0029] Please refer to Figure 1 , Figure 1 An example flowchart of a multi-unmanned aerial vehicle dynamic path planning method for data collection according to an embodiment of the present application is shown in Figure 1 The multi-unmanned aerial vehicle dynamic path planning method for data collection includes the following steps:

[0030] S101, in each time slot, state information corresponding to each unmanned aerial vehicle in the data collection scene is obtained, wherein the state information includes unmanned aerial vehicle self information, environment information and static point of interest information.

[0031] As an embodiment, the unmanned aerial vehicle self information includes the position, speed, direction and remaining power of the unmanned aerial vehicle in space; the environment information includes obstacles, positions of other unmanned aerial vehicles, etc.; and the static point of interest information includes static POI point marks that have been completed and static POI points to be completed.

[0032] It should be noted that POI points are divided into fixed and dynamic types; wherein the fixed type POI points are available for unmanned aerial vehicles to collect information at any time slot; the appearance and disappearance of dynamic POI points have fixed times, and only during the duration of dynamic POI points can information be collected; the above static POI points are fixed type POI points, wherein a time slot two-dimensional diagram of POI points is shown in Figure 3 .

[0033] S102, training the corresponding reinforcement learning network according to the state information, and obtaining the pre-planned path in the offline time by using the trained reinforcement learning network.

[0034] That is, an Actor-Critic network is initialized for each unmanned aerial vehicle; the Actor network receives the evaluation value of the Critic network, and directly outputs a determined action for the unmanned aerial vehicle using a deep deterministic policy gradient (DDPG) algorithm; the Critic network of each unmanned aerial vehicle is used to receive the state and action information of itself and all other unmanned aerial vehicles, and evaluate the utility of taking a certain action in a given state.

[0035] S103, when starting online flight at each time slot, flying according to the pre-planned path, and obtaining the corresponding dynamic point of interest information at the current time slot.

[0036] That is, each unmanned aerial vehicle flies according to the flight path determined in the offline path planning stage; when starting at the t'th time slot, each unmanned aerial vehicle re-plans the path in combination with the dynamic POI point position, the required data volume and the reward value at this time slot.

[0037] S104, adjusting the pre-planned path according to the dynamic point of interest information by using a genetic algorithm to obtain the corresponding optimal planning path.

[0038] It should be noted that, since each time slot is very small, the position of the unmanned aerial vehicle changes little, and considering the timeliness of dynamic real-time tasks, the duration of the task is not long, so the dynamic POI points generated in the previous time slot can be considered to be out of the consideration range; there are |U| unmanned aerial vehicles, each unmanned aerial vehicle travels according to the offline path planning scheme; a population of the genetic algorithm is initialized for each unmanned aerial vehicle, and each individual represents a possible flight path.

[0039] In summary, the embodiments of the present application apply the multi-agent reinforcement learning algorithm to the path planning problem of unmanned aerial vehicle data collection tasks, plan a reasonable path for each unmanned aerial vehicle by the platform, so that it can maximize the reward value obtained by collecting fixed POI points; at the same time, when executing the fixed POI point collection task, the genetic algorithm is used to consider the newly appeared dynamic POI points, and the path is re-planned online, so that the unmanned aerial vehicle cluster obtains the maximum total reward value in the whole time slot, as shown in the unmanned aerial vehicle path planning diagram. Figure 2

[0040] ​In order to implement the above embodiments, the multi-UAV dynamic path planning method for data collection proposed in this application includes two stages. The first stage is the offline path pre-planning stage, and the second stage is the online path re-planning stage. Environment initialization and parameter definition are also performed before the two stages.

[0041] When initializing the environment and defining the parameters, suppose there are |U| drones in the environment, and the set is U={u1,u2,…,u n ,…,u |U|}, where for any drone u n ∈U, the amount of data collected in each time slot is d n ; There are |P| fixed-class POIs, each of which is represented by p m ={(x m ,y m ),D m} means, where (x m ,y m ) represents the location of the fixed POI point, D m represents the amount of data required for fixed POIs; for dynamic POIs, at the beginning of the tth time slot, the system updates the newly appeared |Q t Dynamic POI points Each dynamic POI point is represented by q (k,t) ={r′ k ,τ k ,D′ k ,(x′ k ,y′ k )} means r′ k To complete the dynamic POI point q (k,t) The reward obtained from the collection task, τ k q (k,t) The number of time slots that last, (x′ k ,y′ k ) is X q Position, D′ k q (k,t) The amount of data required, (k, t) represents the point numbered k that appears in the tth time slot.

[0042] The offline path pre-planning phase includes the following steps:

[0043] Step 1: Network initialization and parameter definition

[0044] In the environment, set the existence of |U| drones. Initialize an Actor-Critic network for each drone; the Actor network is divided into two layers: the upper layer Actor is a deep neural network, responsible for generating real-time flight path; the lower layer Actor uses the DDPG algorithm to directly predict flight action; at the t-th time slot, define the state of the n-th drone as Including the position, speed, direction, remaining power of the drone in space, surrounding environment information (such as obstacles, the position of other drones, etc.) and task-related state information (such as the POI point mark that has completed the visit, the POI to be completed, etc.); the Critic network of each drone is used to receive the state and action information of itself and all other drones, and to evaluate the utility of taking a certain action in a given state.

[0045] Step 2: Environment interaction and action execution

[0046] At the beginning of each time slot t, each drone u n Based on its current state Use the Actor network to predict a flight action

[0047] Step 3: State update and reward calculation

[0048] After executing the action , the drone receives feedback from the environment, moves to a new state And get the reward

[0049] Step 4: Experience replay buffer update

[0050] After each action is executed, the Experience four-tuple is stored in the experience replay buffer.

[0051] Step 5: Learning and network update

[0052] Randomly sample a batch of samples from the experience replay buffer Training; the Critic network evaluates the execution of the action In the state The expected reward Q value, Q value represents the expected cumulative reward of executing a certain action in the current state and following the current policy; the Actor network calculates the policy gradient according to the feedback of the Critic network, and updates the parameters of the Actor network using gradient ascent method to optimize future action prediction.

[0053] Step 6: Repeat the learning steps of steps 2-5.

[0054] It should be noted that for any fixed POI point p m ∈P, assume that the UAV group cooperatively collects data amount Φ m at this point. Use a n,m,t to represent the distribution of UAVs, and

[0055]

[0056] Assume that there are T time slots in total, then

[0057]

[0058] If Φ m >D m , it is considered that the task is completed, and the platform can obtain a reward score r m ; Different fixed POI points have different reward scores; the platform plans the route for the UAV cluster in the path offline pre-planning stage, and the goal is to maximize the total reward obtained by the platform under the limited energy of each UAV.

[0059] The online path re-planning stage includes the following steps:

[0060] Step 1: initialization and parameter definition

[0061] Assume that the number of experiments is Ne=1000, and each experiment contains |T| = 500 time slots; at the beginning of each time slot, there are |P| fixed POI points P = {p1, p2, …, p m ,…, p |P|} on the map, and |Q t | new dynamic POI points appear Since each time slot is very small, the UAV position changes little, and considering the timeliness of dynamic real-time tasks, the task duration time slot is not long, so it can be considered that the dynamic POI generated in the previous time slot is not included in the consideration range; there are |U| UAVs, and each UAV travels according to the offline path pre-planning scheme; initialize the population of the genetic algorithm for each UAV, and each individual represents a possible flight path.

[0062] Step 2: define the fitness function

[0063] In the Nn experiment, each UAV flies according to the flight path determined in the path pre-planning stage; at the beginning of the t'th time slot, each UAV re-plans the path in combination with the dynamic POI point position, required data amount and reward value appearing in this time slot. Define the fitness function F, and the calculation formula is:

[0064]

[0065] wherein is the dynamic POI q (k,t′) the reward obtained after the amount of data collected reaches the target,

[0066]

[0067] E k is the total energy consumed by the UAV to perform the assigned task, which consists of two parts: flight energy consumption and hovering energy consumption.

[0068]

[0069] flies, k o is a constant depending on the blade size, m n is the mass of the UAV u n , p is the air density, X n is the area of the UAV.

[0070]

[0071] is the flight speed of the UAV; is the rotor speed of the blade; is the average induced speed of the propeller in the forward combat mode; F is a constant depending on the drag coefficient of the UAV; l is the distance from the UAV to the dynamic POI point q (k,t′) .

[0072] Step 3: Genetic algorithm operation

[0073] Standard operations of the genetic algorithm are performed, including selection, crossover and mutation. The selection operation is based on the fitness of the UAV path, and the better path is selected for breeding. The crossover operation allows two UAV paths to exchange their partial paths, generating new flight paths. The mutation operation randomly changes a part of the UAV path to explore new possibilities.

[0074] Step 4: Update UAV path

[0075] After each time slot, the flight path of the UAV is updated according to the new population generated by the genetic algorithm. Each UAV travels according to the new path.

[0076] Step 5: Check data collection status

[0077] At each time slot, check the data collection progress of all POIs. If the data collection amount of a certain POI reaches the set target Dm, the POI is marked as complete and the corresponding reward is calculated.

[0078] Step 6: Repeat execution and termination condition

[0079] Steps 2 to 5 are repeated until 500 time slots are reached, and one experiment is ended. The time when the data collection of each POI is completed and the flight path of the UAV are recorded in each experiment.

[0080] It should be noted that after the path is offline pre-planned, the UAV cluster travels according to the original planned path, and in this process, |Q t may appear at the beginning of each time slot. The platform needs to consider abandoning the collection task of some fixed POI points with low reward scores, and re-including a part of dynamic POI points for collection. Under the energy limitation of each UAV, whether the total reward score obtained is higher, and thus the path of each UAV is redistributed after online judgment.

[0081] Similar to the fixed POI, a n,(k,t),t′ represents the UAV allocation of the dynamic POI point q (k,t) . It should be noted that (k, t) represents that q (k,t) is the point numbered k appearing in the tth time slot. t' represents the time when the UAV performs the task. Among them,

[0082]

[0083] The total data amount of all UAVs collected in T time slots of the dynamic POI point q (k,t) is

[0084]

[0085] If , it means that the collection of this dynamic POI point is completed, and the platform can obtain the reward score r k .

[0086] Among them, at the beginning of each time slot, the above-mentioned judgment content of the UAV collection task is repeated, and the UAV cluster path is updated. After the energy of all UAVs is consumed, the task is completed. x m , respectively mark whether the data collection task of the fixed and dynamic POI points is completed, and the total reward after calculation is

[0087]

[0088] The task is completed.

[0089] In order to realize the above-mentioned embodiment, an embodiment of the present application proposes a multi-UAV dynamic path planning device for data collection, like Figure 4As shown, the multi-unmanned aerial vehicle dynamic path planning device for data collection provided in the present application comprises a first acquisition module 10, an offline path planning module 20, a second acquisition module 30 and an online path re-planning module 40.

[0090] The first acquisition module 10 is configured to acquire, in each time slot, state information corresponding to each unmanned aerial vehicle in the data collection scene, wherein the state information comprises unmanned aerial vehicle self information, environment information and static point of interest information.

[0091] The offline path planning module 20 is configured to train a corresponding reinforcement learning network according to the state information, and obtain a pre-planned path in an offline time by using the trained reinforcement learning network.

[0092] The second acquisition module 30 is configured to fly according to the pre-planned path when starting online flight in each time slot, and acquire corresponding dynamic point of interest information in the current time slot.

[0093] The online path re-planning module 40 is configured to adjust the pre-planned path according to the dynamic point of interest information by using a genetic algorithm to obtain a corresponding optimal planned path.

[0094] It should be noted that the foregoing explanation and description of the multi-unmanned aerial vehicle dynamic path planning method for data collection also applies to the multi-unmanned aerial vehicle dynamic path planning device for data collection of this embodiment, which will not be described here.

[0095] In summary, according to the multi-unmanned aerial vehicle dynamic path planning device for data collection of the present application, the first acquisition module acquires, in each time slot, state information corresponding to each unmanned aerial vehicle in the data collection scene, wherein the state information comprises unmanned aerial vehicle self information, environment information and static point of interest information; the offline path planning module trains a corresponding reinforcement learning network according to the state information, and obtains a pre-planned path in an offline time by using the trained reinforcement learning network; the second acquisition module flies according to the pre-planned path when starting online flight in each time slot, and acquires corresponding dynamic point of interest information in the current time slot; and the online path re-planning module adjusts the pre-planned path according to the dynamic point of interest information by using a genetic algorithm to obtain a corresponding optimal planned path; thereby, the reward value obtained by collecting fixed POI points can be maximized, and when performing a fixed POI point collection task, the newly appeared dynamic POI points are considered as a whole, the path is re-planned online, so that the unmanned aerial vehicle cluster obtains the maximum total reward value in the entire time slot.

[0096] Figure 5 The structure schematic diagram of the electronic device provided in the present application is shown. The electronic device can comprise:

[0097] The memory 501, the processor 502 and the computer program stored in the memory 501 and executable on the processor 502.

[0098] The processor 502 implements the data collection oriented multi-unmanned aerial vehicle dynamic path planning method provided in the above embodiments when executing the program.

[0099] Further, the electronic device further comprises:

[0100] The communication interface 503 is used for communication between the memory 501 and the processor 502.

[0101] The memory 501 is used for storing the computer program executable on the processor 502.

[0102] The memory 501 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0103] If the memory 501, the processor 502 and the communication interface 503 are independently implemented, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 5 In the figure, only one thick line is used to represent, but it does not mean that there is only one bus or one type of bus.

[0104] Optionally, in specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can complete communication between each other through an internal interface.

[0105] The processor 502 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0106] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the data collection-oriented multi-unmanned aerial vehicle dynamic path planning method.

[0107] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or N embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0108] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise explicitly and specifically limited.

[0109] Any process or method descriptions in flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for performing a specified logic function or process, and the various embodiments of the present application also include the possibility that the functions described can be implemented using a plurality of separate program components or objects to perform the described functions, and that these components or objects can be written in accordance with the present application and can be implemented with hardware that is specifically constructed to store and perform the executable instructions, or alternatively can be implemented with a general purpose receiver or a general purpose computer that is programmed to be a special purpose computer according to the present application.

[0110] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of executable instructions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a product of the manufacturing and / or processing. The computer-readable medium can include, but is not limited to, the following: an electronic connection (an electronic device with one or N wires), a portable computer diskette (a magnetic device), a RAM (random access memory), a ROM (read-only memory), an EPROM (erasable programmable ROM) or a Flash memory, an optical fiber, and a portable CD ROM. In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, since the program can be electronically captured, for example, by the optically scanning the paper or other suitable medium, then electronically captured, interpreted, or processed in a suitable manner if necessary, and stored in the computer storage.

[0111] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware and in another embodiment, any of the following technologies, known in the art, or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0112] Those skilled in the art can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer-readable storage medium. When the programs are executed, one or a combination of the steps of the method embodiments is included.

[0113] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0114] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A multi-UAV dynamic path planning method for data collection, characterized in that: The following steps are involved: In each time slot, the state information corresponding to each UAV in the data collection scene is obtained, wherein the state information includes the UAV's own information, environmental information, and static point of interest information; Training a corresponding reinforcement learning network according to the state information, and using the trained reinforcement learning network to obtain a pre-planned path for offline use; When starting online flight in each time slot, flying according to the pre-planned path and obtaining corresponding dynamic point of interest information in the current time slot; Adopting a genetic algorithm to adjust the pre-planned path according to the dynamic point of interest information to obtain a corresponding optimal planned path; Wherein, training a corresponding reinforcement learning network according to the state information includes: Initialize the corresponding Actor-Critic network for each drone; Each drone uses the Actor network to predict the corresponding flight action based on the current state information; After performing the flight action, the drone receives environmental feedback and obtains new status information and rewards; After each flight action is performed, the current state information, the corresponding flight action, the new state information and the reward are stored as a batch of samples in the experience replay buffer; Randomly draw a batch of samples from the experience replay buffer for training; The training process also includes: The Critic network is used to evaluate the calculated expected reward value of executing the flight action in the state, where the calculated expected reward value represents the expected cumulative reward of executing a certain action in the current state and following the current strategy; The Actor network calculates the policy gradient based on the feedback from the Critic network and uses the gradient ascent method to update the parameters of the Actor network to optimize future action predictions; The method of using a genetic algorithm to adjust the pre-planned path according to the dynamic point of interest information to obtain the corresponding optimal planned path includes: A fitness function is defined, and the fitness is calculated based on the total energy estimated to be consumed by the UAV in performing the assigned task, whether the dynamic point of interest data collection task is completed, and the reward obtained after the dynamic point of interest data collection is completed; The genetic algorithm selects a better path for reproduction according to the fitness, and performs crossover and mutation operations to generate a new optimal planning path; The following formula is used to determine whether the dynamic point of interest data collection task is completed: Among them, a n,(k,t),t′ Represents a dynamic interest point q (k,t) The distribution of drones, q (k,t) represents the dynamic point of interest numbered k that appears in the tth time slot, t′ represents the moment when the UAV performs the task, represents the dynamic interest points q collected by all drones in T time slots (k,t) The total amount of data, for any drone u n ∈U, the amount of data collected in each time slot is d n , D ′ k Represents a dynamic interest point q (k,t) The amount of data required, if It means that the dynamic interest points have been collected and r k rewards; Among them, the fitness function F is calculated according to the following formula: in, Represents a dynamic interest point q (k,t′) The reward obtained after collecting the target amount of data, E k is the total energy that the UAV is expected to consume when performing the assigned task, including flight energy consumption and hovering energy consumption, k o A constant that depends on the blade size, m n Indicates drone u n The mass of the air, ρ represents the air density, X n represents the area of ​​the drone, Indicates the flight speed of the drone. represents the rotor speed of the blade, represents the average induced speed of the propeller in forward combat mode, F depends on the constant of the UAV's drag coefficient, and l represents the time it takes for the UAV to reach the dynamic point of interest q (k,t′) distance.

2. A multi-UAV dynamic path planning device for data collection, used to implement the multi-UAV dynamic path planning method for data collection according to claim 1, characterized in that: include: A first acquisition module is configured to acquire, in each time slot, state information corresponding to each drone in the data collection scene, wherein the state information includes drone information, environment information, and static point of interest information; An offline path planning module, configured to train a corresponding reinforcement learning network according to the state information and use the trained reinforcement learning network to obtain a pre-planned path when offline; A second acquisition module is configured to fly according to the pre-planned path when starting online flight in each time slot, and acquire corresponding dynamic point of interest information in the current time slot; The online path re-planning module is used to adjust the pre-planned path according to the dynamic interest point information using a genetic algorithm to obtain a corresponding optimal planned path.

3. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-UAV dynamic path planning method for data collection as claimed in claim 1.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the multi-UAV dynamic path planning method for data collection as claimed in claim 1.

Images