Multi-unmanned aerial vehicle dynamic path planning method and device for data collection, equipment and medium

Through the combination of multi-agent reinforcement learning and genetic algorithms, dynamic planning of drone paths is realized, solving the problems of high computing resources consumption and inability to adapt to environmental changes in the existing technology, and improving the efficiency of drone data collection.

CN119065399AActive Publication Date: 2024-12-03XIAMEN UNIV

Patent Information

Application Number
CN202411015056.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-12-03
Estimated Expiration
2044-07-26

AI Technical Summary

Technical Problem

Existing UAV path planning algorithms consume a lot of computing resources when dealing with complex or large-scale environments, and cannot adapt to dynamic changes in the environment, such as temporary no-fly zones or emerging points of interest, resulting in the inability to update the path in real time, affecting data collection. efficiency.

Method used

Multi-agent reinforcement learning algorithm combined with genetic algorithms is adopted to train reinforcement learning network pre-planning paths and adjust the paths in real time in a dynamic environment to ensure that the drone cluster maximizes the acquisition of the total reward value in the time slot.

Benefits of technology

It effectively reduces computing resource consumption, improves adaptability to dynamic environments, ensures that drone clusters maximize data collection rewards under limited power, and improves data collection efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065399A_ABST
    Figure CN119065399A_ABST
Patent Text Reader

Abstract

The invention discloses a data collection-oriented multi-unmanned aerial vehicle dynamic path planning method, device and equipment and a medium, and the method comprises two stages: in the first stage of offline path pre-planning, a multi-agent depth deterministic strategy gradient algorithm is adopted, an Actor-Critic network is allocated for each unmanned aerial vehicle for training learning, and meanwhile, the multi-agent depth deterministic strategy gradient algorithm is adopted for the offline path pre-planning stage; an overall Critic network is independently designed for each unmanned aerial vehicle to observe global information, and cooperative learning among multiple unmanned aerial vehicles is realized; in the second online path re-planning stage, a genetic algorithm is adopted, a map containing static interest points and dynamic interest points is searched and iterated at the beginning of each time slot, the optimal path of the current time slot is found out, and path re-planning is performed on the unmanned aerial vehicle; therefore, through the combination of the two stages, the unmanned aerial vehicle cluster can maximize the total reward obtained by data collection under the condition that dynamic and static interest points are considered at the same time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of autonomous navigation and control technology for unmanned aerial vehicles, and in particular to a method, device, equipment and medium for dynamic path planning of multiple unmanned aerial vehicles for data collection. Background Art

[0002] With the rapid development of drone technology, they are playing an increasingly important role in agriculture, mapping, rescue, surveillance and other fields. In data collection tasks such as geographic information collection and environmental monitoring, how to effectively maximize the total amount of data collection rewards (including "information value" factors such as information age, as opposed to pure data volume) under the limited battery energy limit has become a research hotspot. In these applications, drones usually need to fly between multiple preset points of interest (POIs), each of which has different data value or reward value. Completing the data collection task at each point of interest can obtain corresponding rewards.

[0003] In the related technologies, heuristic algorithms, path finding and graph traversal search algorithms such as A* search algorithm and Dijkstra algorithm can help drones find the shortest path from a starting point to a series of target points while avoiding obstacles and considering energy consumption constraints. However, they need to process a large number of nodes and edges. When dealing with complex or large-scale environments, they may still require a lot of computing resources and are usually run based on pre-defined static maps. For dynamic changes that may occur in the environment, such as temporary no-fly zones, POI points or the movement of other drones, these algorithms may not be able to adapt or update the path in real time. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems in the above-mentioned technology to a certain extent. To this end, one purpose of the present application is to propose a multi-UAV dynamic path planning method for data collection, which can maximize the reward value obtained by collecting fixed POI points, and when performing fixed POI point collection tasks, comprehensively consider the newly emerged dynamic POI points, and re-plan the path online so that the UAV cluster can obtain the maximum total reward value in the entire time slot.

[0005] To achieve the above-mentioned objectives, the first aspect of the present application proposes a multi-UAV dynamic path planning method for data collection, comprising the following steps: in each time slot, obtaining the status information corresponding to each UAV in the data collection scene, wherein the status information includes the UAV's own information, environmental information and static point of interest information; training the corresponding reinforcement learning network according to the status information, and using the trained reinforcement learning network to obtain the pre-planned path when offline; when starting online flight in each time slot, flying according to the pre-planned path, and obtaining the corresponding dynamic point of interest information in the current time slot; using a genetic algorithm to adjust the pre-planned path according to the dynamic point of interest information to obtain the corresponding optimal planned path.

[0006] According to the above-mentioned technical means, the embodiment of the present application applies the multi-agent reinforcement learning algorithm to the path planning problem of the UAV data collection task, and the platform uniformly plans a reasonable path for each UAV so that it can maximize the reward value obtained by collecting fixed-type POI points; at the same time, when performing the fixed-type POI point collection task, the genetic algorithm is used to comprehensively consider the newly emerging dynamic POI points, and re-plan the path online, so that the UAV cluster can obtain the maximum total reward value in the entire time slot.

[0007] In addition, the multi-UAV dynamic path planning method for data collection proposed in the above embodiment of the present application may also have the following additional technical features:

[0008] Furthermore, the corresponding reinforcement learning network is trained according to the state information, including: initializing the corresponding Actor-Critic network for each drone; each drone uses the Actor network to predict the corresponding flight action based on the current state information; after executing the flight action, the drone receives environmental feedback to obtain new state information and rewards; after each flight action is executed, the current state information, the corresponding flight action, the new state information and the reward are stored as a batch of samples in the experience replay buffer; and a batch of samples are randomly extracted from the experience replay buffer for training.

[0009] Furthermore, the training process also includes: using a Critic network to evaluate the calculated expected reward value of executing a flight action in the state, wherein the calculated expected reward value represents the expected cumulative reward for executing an action in the current state and following the current strategy; the Actor network calculates the policy gradient based on the feedback of the Critic network, and uses the gradient ascent method to update the parameters of the Actor network to optimize future action predictions.

[0010] Furthermore, a genetic algorithm is used to adjust the pre-planned path according to the dynamic point of interest information to obtain a corresponding optimal planned path, including: defining a fitness function, and calculating the fitness based on the total energy estimated to be consumed by the drone to perform the assigned task, whether the dynamic point of interest data collection task is completed, and the reward obtained after the dynamic point of interest collection is completed; the genetic algorithm selects a better path for reproduction through the fitness, and performs crossover and mutation operations to generate a new optimal planned path.

[0011] Further, the following formula is used to determine whether the dynamic point of interest data collection task is completed:

[0012]

[0013]

[0014] Among them, a n,(k,t) , t′ represents the dynamic interest point q (k,t) The distribution of drones, q (k,t) represents the dynamic point of interest numbered k that appears in the tth time slot, t′ represents the moment when the drone performs the task, represents the dynamic interest points q collected by all drones in T time slots (k,t) The total amount of data, for any drone u n ∈U, the amount of data collected in each time slot is d n , D′ k Represents a dynamic interest point q k,t The amount of data required, if This means that the dynamic interest points have been collected and r k Reward.

[0015] Further, the fitness function F is calculated according to the following formula:

[0016]

[0017]

[0018]

[0019]

[0020] in, Represents a dynamic interest point q (k,t′) The reward obtained after collecting the target amount of data, E k is the total energy that the UAV is expected to consume when performing the assigned task, including flight energy and hovering energy. o A constant that depends on the blade size, m nIndicates drone u n The mass of the air, ρ represents the air density, X n represents the area of ​​the drone, Indicates the flight speed of the drone. represents the rotor speed of the blade, represents the average induced speed of the propeller in forward combat mode, F depends on the constant of the drag coefficient of the UAV, and l represents the arrival of the UAV at the dynamic point of interest q (k,t′) distance.

[0021] The second aspect of the present application provides a multi-UAV dynamic path planning device for data collection, including a first acquisition module, which is used to obtain the status information corresponding to each UAV in the data collection scene in each time slot, wherein the status information includes the UAV's own information, environmental information and static point of interest information; an offline path planning module, which is used to train the corresponding reinforcement learning network according to the status information, and use the trained reinforcement learning network to obtain a pre-planned path when offline; a second acquisition module, which is used to fly according to the pre-planned path when starting online flight in each time slot, and obtain the corresponding dynamic point of interest information in the current time slot; an online path re-planning module, which is used to use a genetic algorithm to adjust the pre-planned path according to the dynamic point of interest information to obtain the corresponding optimal planned path.

[0022] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-UAV dynamic path planning method for data collection as described in the above embodiment.

[0023] The fourth aspect of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the multi-UAV dynamic path planning method for data collection as described in the above embodiment.

[0024] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of a flow chart of a multi-UAV dynamic path planning method for data collection according to an embodiment of the present application;

[0026] Figure 2 A schematic diagram of drone path planning in a data collection scenario according to an embodiment of the present application;

[0027] Figure 3A two-dimensional schematic map of time slots in a data collection scenario according to an embodiment of the present application;

[0028] Figure 4 Schematic diagram of a multi-UAV dynamic path planning device for data collection according to an embodiment of the present application;

[0029] Figure 5 is an exemplary diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0031] In order to better understand the above technical solution, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0032] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0033] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a multi-UAV dynamic path planning method for data collection according to an embodiment of the present application, such as Figure 1 As shown, the multi-UAV dynamic path planning method for data collection includes the following steps:

[0034] S101, in each time slot, obtaining status information corresponding to each drone in the data collection scene, wherein the status information includes drone information, environment information and static interest point information.

[0035] As an embodiment, the drone's own information includes the drone's position, speed, direction, and remaining power in space; environmental information includes obstacles, the position of other drones, etc.; static point of interest information includes static POI point marks that have been visited and static POI points to be completed, etc.

[0036] It should be noted that POI points are divided into two categories: fixed and dynamic. Among them, fixed POI points can be used by drones to collect information at any time slot; dynamic POI points have fixed times for appearance and disappearance, and can only be used to collect information during the duration of the dynamic POI. The above-mentioned static POI points are fixed POI points, among which the two-dimensional schematic diagram of POI points divided into time slots is shown in Figure 3 shown.

[0037] S102, training a corresponding reinforcement learning network according to the state information, and using the trained reinforcement learning network to obtain a pre-planned path when offline.

[0038] That is to say, an Actor-Critic network is initialized for each drone; the Actor network receives the evaluation value of the Critic network and uses the deep deterministic policy gradient algorithm (DDPG) to directly output a determined action to the drone; the Critic network of each drone is used to receive the state and action information of itself and all other drones, and evaluate the utility of taking a certain action under a given state.

[0039] S103, when starting online flight in each time slot, flying according to the pre-planned path, and obtaining corresponding dynamic interest point information in the current time slot.

[0040] That is to say, each UAV flies according to the flight path established in the offline path pre-planning stage; at the beginning of the t'th time slot, each UAV replans the path based on the dynamic POI point location, required data volume and reward value appearing in this time slot.

[0041] S104, using a genetic algorithm to adjust the pre-planned path according to the dynamic interest point information to obtain a corresponding optimal planned path.

[0042] It should be noted that since each time slot is very small, the position of the drone does not change much, and from the perspective of the timeliness of the dynamic instant task, the task duration time slot is not long, so it can be considered that the dynamic POI points generated in the previous time slot are not taken into consideration; there are U drones, each of which moves according to the offline path pre-planning plan; the genetic algorithm population is initialized for each drone, and each individual represents a possible flight path.

[0043] In summary, the embodiment of the present application applies the multi-agent reinforcement learning algorithm to the path planning problem of the drone data collection task. The platform uniformly plans a reasonable path for each drone, so that it can maximize the reward value obtained by collecting fixed POI points; at the same time, when performing the fixed POI point collection task, the genetic algorithm is used to comprehensively consider the newly emerging dynamic POI points, and the path is re-planned online, so that the drone cluster obtains the maximum total reward value in the entire time slot. The drone path planning schematic diagram is shown in FIG. Figure 2 shown.

[0044] In order to implement the above-mentioned embodiments, the multi-UAV dynamic path planning method for data collection proposed in the present application includes two stages. The first stage is an offline path pre-planning stage, and the second stage is an online path re-planning stage. Environment initialization and parameter definition are also performed before the two stages.

[0045] When initializing the environment and defining parameters, suppose there are |U| drones in the environment, and the set is U = {u1, u2, ..., u n , ..., u |U|}, where for any drone u n ∈U, the amount of data collected in each time slot is d n ; Suppose there are P fixed POIs, each of which has p m ={(x m ,y m ), D m} means, where (x m ,y m ) represents the location of the fixed POI point, D m represents the amount of data required for fixed POIs; for dynamic POIs, at the beginning of the tth time slot, the system updates the newly appeared |Q t |Dynamic POI points Each dynamic POI point is represented by q k,t = {r′ k ,τk,D′ k , (x′ k , y′ k )} means, r′ k To complete the dynamic POI point q k,t The reward obtained from the collection task, τ k for q k,t The number of time slots that last, (x′ k , y′ k ) is X q The position of k for q k,t The amount of data required, (k, t) represents the point numbered k that appears in the tth time slot.

[0046] The offline path pre-planning phase includes the following steps:

[0047] Step 1: Network initialization and parameter definition

[0048] There are |U| drones in the environment. An Actor-Critic network is initialized for each drone. The Actor network is divided into two layers: the upper layer Actor is a deep neural network responsible for generating real-time flight paths; the lower layer Actor uses the DDPG algorithm to directly predict flight actions; at the tth time slot, the state of the nth drone is defined as It includes the drone’s position, speed, direction, remaining battery power, surrounding environment information (such as obstacles, the position of other drones, etc.) and task-related status information (such as POI point marks that have been visited, POIs to be completed, etc.); the Critic network of each drone is used to receive the status and action information of itself and all other drones, and evaluate the utility of taking a certain action under a given state.

[0049] Step 2: Environment interaction and action execution

[0050] At the beginning of each time slot t, each UAV u n Based on its current state Use the Actor network to predict a flight action

[0051] Step 3: Status update and reward calculation

[0052] In action After that, the drone receives feedback from the environment and moves to a new state. and get rewarded

[0053] Step 4: Experience Replay Buffer Update

[0054] After each action is completed, This experience quad is stored in the experience replay buffer.

[0055] Step 5: Learning and Network Update

[0056] Draw a random batch of samples from the experience replay buffer Training; Critic network evaluates execution actions In Status Calculate the expected reward under The Q value represents the expected cumulative reward for performing an action in the current state and following the current strategy. The Actor network calculates the policy gradient based on the feedback from the Critic network, and uses the gradient ascent method to update the parameters of the Actor network to optimize future action predictions.

[0057] Step 6: Repeat the learning steps from steps 2 to 5.

[0058] It should be noted that for any fixed POI point p m ∈P, let the drone group cooperate at this point to collect a total of Φ m The amount of data. n,m,t Indicates the distribution of drones.

[0059]

[0060] Assume there are T time slots in total, then

[0061]

[0062] If Φ m >D m , that is, the task is considered completed and the platform can obtain r m reward score; different fixed POI points have different reward scores; the platform plans routes for the drone cluster in the offline pre-planning stage of the path, with the goal of maximizing the total reward obtained by the platform under the constraint of limited energy of each drone.

[0063] The online path replanning phase includes the following steps:

[0064] Step 1: Initialization and parameter definition

[0065] Assume that the number of experiments is Ne = 1000, and each experiment contains |T| = 500 time slots. At the beginning of each time slot, there are |P| fixed POI points P = {p1, p2, ..., p m ,…,p |P|}, and the newly emerged |Q t |Dynamic POI points Since each time slot is very small, the position of the UAV does not change much, and considering the timeliness of the dynamic instant task, the task duration time slot is not long, so it can be considered that the dynamic POIs generated in the previous time slot are not taken into consideration; there are |U| UAVs, each of which moves according to the offline path pre-planning plan; the genetic algorithm population is initialized for each UAV, and each individual represents a possible flight path.

[0066] Step 2: Define the fitness function

[0067] In the Nnth experiment, each drone flies according to the flight path set in the path pre-planning stage; at the beginning of the t'th time slot, each drone replans the path based on the dynamic POI point position, required data volume and reward value appearing in this time slot. Define the fitness function F, and the calculation formula is:

[0068]

[0069] in It is a dynamic POI (k,t′) The reward for collecting the required amount of data.

[0070]

[0071] E k It is the total energy that the UAV is expected to consume when performing the assigned task, which is composed of two parts: flight energy consumption and hovering energy consumption.

[0072]

[0073] Fly in it, k o is a constant that depends on the blade size, m n It is UAVu n The mass of the air, ρ is the air density, X n is the area of ​​the drone.

[0074]

[0075] is the UAV flight speed; is the rotor speed of the blade; is the average induced speed of the propeller in forward combat mode; F is a constant that depends on the drag coefficient of the drone; l is the speed at which the drone reaches the dynamic POI point q (k,t′) distance.

[0076] Step 3: Genetic Algorithm Operation

[0077] Perform standard operations of genetic algorithms, including selection, crossover, and mutation. The selection operation selects the better path for reproduction based on the fitness of the drone paths. The crossover operation allows two drone paths to exchange parts of their paths to generate a new flight path. The mutation operation randomly changes parts of the drone paths to explore new possibilities.

[0078] Step 4: Update the drone path

[0079] After each time slot, the flight paths of the drones are updated according to the new population generated by the genetic algorithm. Each drone travels according to the new path.

[0080] Step 5: Check data collection status

[0081] The data collection progress of all POIs is checked in each time slot. If the data collection amount of a POI reaches the set target Dm, the POI is marked as completed and the corresponding reward is calculated.

[0082] Step 6: Repeat and Termination Conditions

[0083] Repeat steps 2 to 5 until 500 time slots are reached, and then end the experiment. Record the completion time of data collection for each POI and the flight path of the drone in each experiment.

[0084] It should be noted that after the offline pre-planning of the path is completed, the drone cluster moves along the original planned path. During this process, |q t | dynamic POI points. The platform needs to consider giving up the collection tasks of some fixed POI points with low reward scores, and re-incorporating some dynamic POI points for collection. Under the energy limit of each drone, whether the total reward score obtained is higher, after online judgment, the paths of each drone are reallocated.

[0085] Similar to fixed POI, use a n,(k,t),t′ Represents a dynamic POI point q (k,t) The distribution of drones. It should be noted that (k, t) represents q (k,t) is the point numbered k that appears in the tth time slot. t' represents the moment when the drone performs the task.

[0086]

[0087] Then the dynamic POI points q collected by all drones in T time slots (k,t) The total amount of data is

[0088]

[0089] like This means that the dynamic POI point has been collected and the platform can obtain r k of bonus points.

[0090] At the beginning of each time slot, the above judgment of the drone collection task is repeated and the drone cluster path is updated. After all drones have exhausted their energy, the task is completed. m , Mark whether the fixed and dynamic POI point data collection tasks are completed. The total reward after calculation is

[0091]

[0092] Mission accomplished.

[0093] In order to implement the above embodiment, the present application embodiment proposes a multi-UAV dynamic path planning device for data collection, such as Figure 4 As shown, the multi-UAV dynamic path planning device for data collection proposed in the present application includes: a first acquisition module 10, an offline path planning module 20, a second acquisition module 30 and an online path re-planning module 40.

[0094] The first acquisition module 10 is used to acquire the status information corresponding to each drone in the data collection scene in each time slot, wherein the status information includes the drone's own information, environmental information and static interest point information;

[0095] The offline path planning module 20 is used to train the corresponding reinforcement learning network according to the state information, and use the trained reinforcement learning network to obtain the pre-planned path when offline;

[0096] The second acquisition module 30 is used to fly according to the pre-planned path when starting the online flight in each time slot, and obtain the corresponding dynamic point of interest information in the current time slot;

[0097] The online path re-planning module 40 is used to adjust the pre-planned path according to the dynamic interest point information using a genetic algorithm to obtain a corresponding optimal planned path.

[0098] It should be noted that the aforementioned explanation of the embodiment of the multi-UAV dynamic path planning method for data collection is also applicable to the multi-UAV dynamic path planning device for data collection of this embodiment, and will not be repeated here.

[0099] In summary, according to the multi-UAV dynamic path planning device for data collection in the embodiment of the present application, the first acquisition module obtains the status information corresponding to each UAV in the data collection scene in each time slot, wherein the status information includes the UAV's own information, environmental information and static point of interest information; the offline path planning module trains the corresponding reinforcement learning network according to the status information, and uses the trained reinforcement learning network to obtain the pre-planned path when offline; the second acquisition module flies according to the pre-planned path when starting online flight in each time slot, and obtains the corresponding dynamic point of interest information in the current time slot; the online path re-planning module uses a genetic algorithm to adjust the pre-planned path according to the dynamic point of interest information to obtain the corresponding optimal planned path; thereby, the reward value obtained by collecting fixed POI points can be maximized, and when performing fixed POI point collection tasks, the newly emerging dynamic POI points are comprehensively considered, and the path is re-planned online so that the UAV cluster obtains the maximum total reward value in the entire time slot.

[0100] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0101] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0102] When the processor 502 executes the program, the multi-UAV dynamic path planning method for data collection provided in the above embodiment is implemented.

[0103] Furthermore, the electronic device further comprises:

[0104] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0105] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0106] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0107] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0108] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0109] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0110] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned multi-UAV dynamic path planning method for data collection.

[0111] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0112] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0113] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0114] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0115] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0116] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0117] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0118] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A multi-UAV dynamic path planning method for data collection, characterized in that: The following steps are involved: In each time slot, the state information corresponding to each drone in the data collection scene is obtained, wherein the state information includes the drone's own information, environmental information, and static interest point information; Training a corresponding reinforcement learning network according to the state information, and using the trained reinforcement learning network to obtain a pre-planned path when offline; When starting online flight in each time slot, flying according to the pre-planned path, and obtaining corresponding dynamic point of interest information in the current time slot; A genetic algorithm is used to adjust the pre-planned path according to the dynamic interest point information to obtain a corresponding optimal planned path.

2. The multi-UAV dynamic path planning method for data collection according to claim 1, characterized in that: Training a corresponding reinforcement learning network according to the state information includes: Initialize the corresponding Actor-Critic network for each drone; Each drone uses the Actor network to predict the corresponding flight action based on the current state information; After performing the flight action, the drone receives environmental feedback and obtains new status information and rewards; After each flight action is performed, the current state information, the corresponding flight action, the new state information and the reward are stored as a batch of samples in the experience replay buffer; A batch of samples are randomly drawn from the experience replay buffer for training.

3. The multi-UAV dynamic path planning method for data collection as claimed in claim 2, characterized in that: The training process also includes: The Critic network is used to evaluate the calculated expected reward value of executing the flight action in the state, where the calculated expected reward value represents the expected cumulative reward for executing a certain action in the current state and following the current strategy; The Actor network calculates the policy gradient based on the feedback from the Critic network and uses the gradient ascent method to update the parameters of the Actor network to optimize future action predictions.

4. The multi-UAV dynamic path planning method for data collection as claimed in claim 1, characterized in that: The pre-planned path is adjusted according to the dynamic interest point information by using a genetic algorithm to obtain a corresponding optimal planned path, including: A fitness function is defined, and the fitness is calculated based on the total energy that the UAV is expected to consume in performing the assigned task, whether the dynamic point of interest data collection task is completed, and the reward obtained after the dynamic point of interest collection is completed; The genetic algorithm selects a better path for reproduction through the fitness, and performs crossover and mutation operations to generate a new optimal planning path.

5. The multi-UAV dynamic path planning method for data collection as claimed in claim 4, characterized in that: The following formula is used to determine whether the dynamic point of interest data collection task is completed: Among them, a n,(k,t),t′ Represents a dynamic interest point q (k,t) The distribution of drones, q (k,t) represents the dynamic point of interest numbered k that appears in the tth time slot, t′ represents the moment when the drone performs the task, represents the dynamic interest points q collected by all drones in T time slots (k,t) The total amount of data, for any drone u n ∈U, the amount of data collected in each time slot is d n , D ′ k Represents a dynamic interest point q k,t The amount of data required, if This means that the dynamic interest points have been collected and r k Reward.

6. The multi-UAV dynamic path planning method for data collection as claimed in claim 5, characterized in that: The fitness function F is calculated according to the following formula: in, Represents a dynamic interest point q (k,t′) The reward obtained after collecting the required amount of data, e k is the total energy that the UAV is expected to consume when performing the assigned task, including flight energy and hovering energy. o A constant that depends on the blade size, m n Indicates drone u n The mass of the air, ρ represents the air density, X n represents the area of ​​the drone, Indicates the flight speed of the drone. represents the rotor speed of the blade, represents the average induced speed of the propeller in forward combat mode, F depends on the constant of the drag coefficient of the UAV, and l represents the arrival of the UAV at the dynamic point of interest q (k,t′) distance.

7. A multi-UAV dynamic path planning device for data collection, characterized in that: include: A first acquisition module is used to acquire the status information corresponding to each drone in the data collection scene in each time slot, wherein the status information includes the drone's own information, environmental information and static interest point information; An offline path planning module, used to train a corresponding reinforcement learning network according to the state information, and use the trained reinforcement learning network to obtain a pre-planned path when offline; A second acquisition module is used to fly according to the pre-planned path when starting the online flight in each time slot, and acquire the corresponding dynamic point of interest information in the current time slot; The online path re-planning module is used to adjust the pre-planned path according to the dynamic interest point information by using a genetic algorithm to obtain a corresponding optimal planned path.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-UAV dynamic path planning method for data collection as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the multi-UAV dynamic path planning method for data collection as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Data acquisition method based on multi-unmanned aerial vehicle crowd sensing

    CN113741530A

  • Unmanned aerial vehicle path dynamic planning method for minimizing information age

    CN114637330A

  • Intelligent path planning method for unmanned aerial vehicle cluster

    CN115562357A

  • Unmanned aerial vehicle data on-demand collection method based on deep reinforcement learning

    CN117278106A

  • Joint search method for UAV multiobjective path planning in urban low altitude environment

    US20180308371A1

Cited By

  • Unmanned aerial vehicle real-time communication and control method based on 5.5 G network

    CN119854749A