Coverage path planning method for multiple unmanned aerial vehicles in dam abnormal environment inspection

Abstract

The invention discloses a coverage path planning method for multiple unmanned aerial vehicles in dam abnormal environment inspection, and formally defines a coverage path planning problem of multiple unmanned aerial vehicles in dam abnormal environment inspection: firstly, defining an abnormal environment, namely a dam area after a disaster occurs; secondly, defining a coverage path planning problem of the multiple unmanned aerial vehicles in dam abnormal environment inspection as an optimization problem, wherein optimization targets are coverage rates and energy consumption of the multiple unmanned aerial vehicles in an abnormal environment; model design: aiming at the optimization problem defined in the previous step, constructing a DRL-based multi-unmanned aerial vehicle coverage path planning model; and updating parameters of the multi-unmanned aerial vehicle coverage path planning model and evaluating the performance of the model. Compared with the prior art, the method has the advantages of good practicability and the like.

Description

Technical Field

[0001] The present invention relates to a coverage path planning method for multiple unmanned aerial vehicles in abnormal dam environment inspection, which is used for coverage path planning when multiple unmanned aerial vehicles perform inspection tasks in a dam area after a natural disaster occurs, and belongs to the field of smart water conservancy technology. Background Art

[0002] Inspection is of great significance to the safe operation of dams. Comprehensive and effective dam inspection can timely detect potential structural problems such as cracks and leakage, and monitor the impact of natural disasters on the reservoir's ecological environment in real time. Recently, the rapid development of drone technology has provided new ideas for dam inspection. Multi-drone collaboration has the advantages of low cost, small size, and strong maneuverability, and has the potential to achieve comprehensive and efficient coverage in dam inspections. The key technology of multi-drone collaboration is drone coverage path planning.

[0003] In recent years, a large number of scholars have studied the coverage path planning of UAVs. The coverage path planning models of UAVs are mainly divided into two categories. The first category is the non-self-organizing control model, including cooperative control and random control. Cooperative control refers to the strategy of making multiple UAVs work together to achieve a common goal (such as covering an area). For example, in order to solve the problem of cooperative mechanism and distributed strategy in cooperative search of UAVs, some researchers have developed a distributed online heuristic method. However, the robustness of cooperative control is limited. If a few UAVs in the UAV cluster are damaged, the entire UAV network needs to be redeployed. Random control attempts to cover the target area by randomly selecting path points, thereby reducing computational overhead and time delay. For example, some scholars have designed a strategy that integrates random walks, artificial intelligence, and coverage control to deploy multiple helicopter-type UAVs in the target area. Compared with cooperative control, random control further improves robustness, but it exposes the problems of long coverage time and overlapping coverage.

[0004] To address these issues, a second type of UAV coverage path planning model was proposed, called the self-organizing control model. This model can achieve regional coverage tasks based on the decision-making of autonomous UAVs. Specifically, the researchers designed a multi-agent DRL (Deep Reinforcement Learning, DRL for short) model to monitor industrial production. The model is able to cover a larger area and minimize overlapping coverage. In order to meet the long-term knowledge and seamless coverage requirements of IoT nodes, the researchers developed a multi-rechargeable UAV collaborative solution to achieve system energy consumption optimization. Self-organizing control models have also been widely used in engineering inspections. For example, the researchers used a heat equation-driven regional coverage method to achieve trajectory planning for UAV inspections of three-dimensional infrastructure such as wind turbines and bridges. The method uses potential fields and distance fields to generate UAV flight trajectories and prevent collisions between UAVs, respectively.

[0005] However, during dam inspection, the dam area is inevitably in an abnormal environment due to the existence of natural disasters. The abnormal environment of the dam will cause two problems: 1) Natural disasters such as thunderstorms, strong winds and earthquakes will interfere with wireless communication signals, resulting in communication barriers between drones; 2) In the abnormal environment of the dam, the coverage and energy consumption of the drone are factors that cannot be ignored. For example, a larger drone coverage supports the acquisition of more disaster information. In addition, natural disasters can damage the energy supply device of the drone. The low energy consumption of the drone ensures that long-term inspection tasks can be performed under limited energy supply. Therefore, it is necessary to achieve joint optimization of coverage and energy consumption. At present, most self-organizing control models are designed to explore dam inspections under normal environments. When natural disasters occur in the dam area, these models do not take into account the above two problems. This is the driving force of the research of this invention. Summary of the invention

[0006] Purpose of the invention: In view of the problems and shortcomings of the prior art, the present invention provides a coverage path planning method for multiple UAVs in abnormal environment inspection of dams with good performance and strong practicality.

[0007] Technical solution: A coverage path planning method for multiple UAVs in abnormal dam environment inspection includes the following steps:

[0008] Step 1: Formalize the coverage path planning problem of multiple UAVs in dam abnormal environment inspection;

[0009] Step 2: Construct a multi-UAV coverage path planning model based on DRL;

[0010] Step 3: Update model parameters.

[0011] Preferably, the step 1 is specifically:

[0012] The formal definition of the coverage path planning problem of multiple UAVs in abnormal dam environment inspection can be divided into two steps, namely defining the abnormal environment and defining the optimization problem.

[0013] More preferably, the abnormal environment is specifically defined as:

[0014] Dam inspection is of great significance to the safe operation of the dam, especially after the dam area encounters natural disasters such as thunderstorms, strong winds, and earthquakes. Comprehensive and effective dam inspection can timely discover potential structural problems (such as cracks and leakage) and take corrective measures, while real-time monitoring the impact of natural disasters on the reservoir ecological environment. The present invention refers to the dam area after encountering natural disasters as the dam abnormal environment. Specifically, the dam abnormal environment is defined by the following components: the dam area including boundaries and obstacles, the disaster points in the dam area, the severity of the disaster at each disaster point, static sensors (deployed before the disaster occurs), multiple drones, and the coverage area of ​​each drone.

[0015] More preferably, the optimization problem is defined as follows:

[0016] After defining the abnormal environment of the dam, the present invention defines the coverage path planning problem of multiple UAVs in the abnormal environment inspection of the dam as an optimization problem. The optimization objectives include the coverage rate of multiple UAVs and the energy consumption of multiple UAVs, aiming to minimize the energy consumption of multiple UAVs and maximize the coverage area of ​​multiple UAVs.

[0017] Preferably, the step 2 is specifically:

[0018] According to the principles of deep learning and reinforcement learning, the present invention constructs a multi-UAV coverage path planning model based on DRL. The model can achieve the joint optimization of coverage and energy consumption of multiple UAVs during the inspection of abnormal dam environment.

[0019] More preferably, the DRL-based multi-UAV coverage path planning model is mainly composed of two parts, namely TP (Trace Pheromone, TP for short) and MADDPG (Multi-Agent DeepDeterministic Policy Gradient, MADDPG for short). TP establishes a communication medium between UAVs and realizes indirect communication of UAVs in abnormal dam environments. At the same time, TP can effectively manage the path information of UAVs and simplify the complex interactions between UAVs into low-dimensional representations. Then MADDPG is used to implement coverage path planning for multiple UAVs. In MADDPG, the reward function is used to simultaneously achieve the goals of maximizing the coverage rate of multiple UAVs and minimizing the energy consumption of multiple UAVs.

[0020] More preferably, the MADDPG is specifically:

[0021] Each drone has an independent MADDPG, and each MADDPG contains two networks, the actor network and the critic network. The actor network consists of two FC layers (Fully Connected Layers, FC layers for short) and a softmax layer. This network is designed to generate the probability of an action that a drone can perform (such as flight direction and flight angle). Based on these probabilities, the drone chooses the next action to be performed. The critic network consists of three FC layers and is mainly used to generate Q values.

[0022] Preferably, the step 3 is specifically:

[0023] MADDPG updates parameters by updating the actor network and critic network for each drone separately. Each drone has an independent actor-critic network pair, where the actor network is updated according to the gradient of the expected return, and the critic network updates the Q value according to the Bellman equation. During the training process, a centralized training method is used, that is, the critic network of each drone considers the actions of all drones. However, the execution is decentralized, that is, each drone acts independently according to its own strategy.

[0024] Finally, by comparing the performance of the model proposed in the present invention with some advanced DRL models in a simulated dam abnormal environment, it can be found that the model proposed in the present invention has a greater UAV coverage rate and lower UAV energy consumption in multi-UAV coverage path planning.

[0025] A computer device, characterized in that: the computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the coverage path planning method for multiple UAVs in abnormal environment inspection of dams as described above are implemented.

[0026] A computer-readable storage medium, characterized in that: the computer-readable storage medium stores a computer program for executing the coverage path planning method of multiple UAVs in abnormal dam environment inspection as described above.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] First, a new DRL-based model is proposed in this paper, which inherits the TP and MADDPG structures and is used for dam detection by multiple UAVs in abnormal environments. Specifically, inspired by ant colony pheromones, this paper develops a TP-based communication scheme to control the movement of UAVs. This scheme uses TP to establish a communication medium between UAVs and realizes indirect communication between UAVs in abnormal dam environments. At the same time, this scheme can effectively manage the path information of UAVs and simplify the complex interactions between UAVs into low-dimensional representations.

[0029] Second, the present invention introduces MADDPG to perform multi-UAV coverage path planning. In MADDPG, the reward function is designed to optimize the coverage and energy consumption of the UAV at the same time. Furthermore, the present invention introduces the penalty of overlapping coverage and boundary restrictions of the mission area into the reward function, thereby greatly improving the reliability of the UAV continuous motion control decision. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of coverage path planning for multiple UAVs in abnormal dam environment inspection according to an embodiment of the present invention;

[0031] Figure 2 This is an overall structural diagram of the DRL-based multi-UAV coverage path planning model in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0033] The present invention uses the learning ability of deep learning for big data and the decision-making ability of reinforcement learning to establish a multi-UAV coverage path planning model based on DRL. The coverage path planning method of multi-UAV in abnormal environment inspection of dam based on DRL is as follows: Figure 1 As shown, the following steps are included:

[0034] Step 1: Formalize the coverage path planning problem of multiple UAVs in dam abnormal environment inspection;

[0035] The formal definition of the coverage path planning problem of multiple UAVs in the abnormal environment inspection of the dam can be divided into two steps, namely defining the abnormal environment and defining the optimization problem. First, the abnormal environment of the dam is defined. Specifically, the entire dam area is divided into an L×W two-dimensional grid, which contains DP (Disaster Point, DP for short), boundaries and obstacles. When a natural disaster occurs, the entire dam area is regarded as the mission area of ​​the UAV. Before the disaster occurred, some static sensors had been deployed in the dam area. Normal static sensors can collect data and then transmit the data to the base station for observation and analysis. Given P UAVs and Q static sensors, the sets of UAVs and static sensors are represented as and Among them, x p ,p∈[1,P] and e q ,q∈[1,Q] are the pth UAV and the qth static sensor respectively. Since disasters may cause damage to static sensors in the mission area, abnormal static sensors cannot collect reliable data or transmit data to the base station. At the same time, due to natural disasters, the communication between UAVs may be interrupted. The set of DP is represented as where a d ,d∈[1,D] represents the dth DP. D represents the total number of DPs. The severity of the disaster for each DP is different and is quantified into a value, called the disaster value. For each DP, the set of disaster values ​​is represented as In addition, it is ensured that each DP is covered by at least one UAV in a certain time slot. More specifically, the coverage area of ​​each UAV is If a DP is located at least one UAV , it means that the DP has been covered by drones. Before performing the inspection task, all drones are parked at the same starting position.

[0036] After defining the abnormal environment of the dam, the coverage path planning problem of multiple UAVs in the abnormal environment inspection of the dam is defined as an optimization problem. Specifically, the coverage rate refers to the proportion of DPs that the UAV can cover within a period of time. If a DP is within the coverage area of ​​the UAV, If the DP is within , it means that the DP is covered. Each DP is represented as a corresponding disaster value, and the coverage rate of the Pth drone in the tth generation is calculated as:

[0037]

[0038] in, represents the sum of the disaster values ​​covered by the pth drone. p Represents the number of DPs within the coverage area of ​​the pth UAV. Represents the sum of the disaster values ​​of all DPs in the task area.

[0039] The highest coverage is achieved based on the energy consumption of the drones. Therefore, it is necessary to pay attention to the energy consumption of each drone in each generation, which includes the energy requirements for collecting data and continuous horizontal flight. Specifically, in the tth generation, the energy consumption of the pth drone is calculated as follows:

[0040]

[0041] in, Represents the moving distance of the pth drone. d is the energy consumption coefficient of the UAV’s moving distance. v Represents the energy consumption coefficient when the drone collects data.

[0042] In order to balance the coverage rate of drones and the energy consumption of drones, the coverage energy efficiency is designed considering the energy consumption of drones. It represents the number of DPs newly covered by drones in one generation. Specifically, in the tth generation, the coverage energy efficiency is represented as:

[0043]

[0044] where ω indicates whether each DP is fairly covered. is the incremental coverage of the pth drone. represents effective incremental coverage. This is because the UAV must ensure that the DP is fairly covered. represents the incremental energy consumption of the pth drone. In the coverage energy efficiency, the numerator is understood as the coverage benefit of the drone, and the denominator is understood as the coverage cost of the drone. Maximizing the energy coverage efficiency can ensure the joint optimization of the drone's coverage rate and energy consumption.

[0045] Step 2: Construct a multi-UAV coverage path planning model based on DRL;

[0046] According to the principles of deep learning and reinforcement learning, a multi-UAV coverage path planning model based on DRL is constructed. Figure 2As shown in the figure, the model mainly consists of two parts, namely TP and MADDPG. In nature, ants release chemical pheromones during foraging. These chemical pheromones generate a pheromone map showing the location information of food and ant nests. Ants use pheromone maps to exchange information with each other to obtain the specific location of food and ant nests. Inspired by this, a TP for drones is designed, which can save global state information such as the flight time and location of drones. These TPs generate trajectory tracking maps of drones in the mission area. The trajectory tracking map serves as a communication medium for drones in abnormal dam environments, solving the problem of communication interruption between drones caused by natural disasters. In addition, the concentration of TP shows dynamic changes. For example, the concentration of TP gradually decreases over time. At the same time, the concentration within a grid can be accumulated. In a period of time, the higher the concentration within a grid, the more drones patrol in the grid. In view of this, TP can be used to reduce the overlapping coverage of drones in the mission area.

[0047] Initialization of TP: Ants use chemical pheromones to guide their companions’ foraging routes. Driven by chemical pheromones, the present invention designs TP to record the flight routes of drones, aiming to achieve two goals: 1) to serve as a communication medium between drones in the abnormal environment of the dam, and 2) to reduce the repeated visits of drones to a grid within a period of time. Specifically, in the 0th generation, the TP concentration of the drone’s starting position is initialized as:

[0048]

[0049] Where (x, y) represents the starting coordinates of the drone. All drones have the same starting position. Through the trajectory tracking graph, each drone can sense the concentration of TP. The grid with high concentration is regarded as a negative influence of the neighboring drones, that is, the drones tend to fly to the grid with low concentration.

[0050] Evaporation of TP: Evaporation refers to the update of TP concentration, indicating that the previous TP concentration will dissipate over time. At the tth generation, the TP concentration that has evaporated in the i-th grid is expressed as:

[0051]

[0052] where ξ,ξ∈[0,1] represents the evaporation coefficient in each generation. In the tth generation, Represents the concentration of TP in the i-th grid.

[0053] Aggregation of TP: When the drone arrives at the i-th grid in the next generation (t+1 generation), the concentration of TP in that grid rises due to the deposition of pheromones. At the same time, as mentioned above, TP evaporates over time, causing the concentration to decrease. Therefore, in the t+1 generation, the concentration of TP in the i-th grid is calculated as:

[0054]

[0055] in, represents the concentration of TP left by the drone in the ith grid and is calculated as:

[0056]

[0057] in represents the concentration of TP left by the h-th UAV when it arrives at the i-th grid. i is the total number of drones in the i-th grid.

[0058] Finally, in order to avoid excessive fluctuations or negative concentrations of TP, the concentration of TP is limited to the interval Therefore, Updated to:

[0059]

[0060] In summary, the present invention uses TP to mark the flight trajectory of the UAV in the mission area. The UAV selects an action (such as selecting the next target grid) according to the concentration of TP.

[0061] During data collection, each drone interacts with the environment and accumulates rich experience in the mission area. This process is expressed as MDP (Markov Decision Process). In addition, drones can choose any flight distance and direction, so how to obtain multi-drone coverage path planning that meets the above optimization objectives is an MDP problem with a continuous action space. TP-based MADDPG has powerful decision-making and perception capabilities and is suitable for solving this problem. Specifically, according to the current state in the state space S, each drone adopts the strategy Select a suitable action from the action space A and then get the corresponding reward. Furthermore, in order to find high-reward actions, random noise is introduced when executing actions to ensure that the strategy is more divergent. The details of TP-based MADDPG are introduced as follows.

[0062] Design of states, actions, and rewards:

[0063] I. Environment state space: The state of each drone mainly includes the TP concentration of the grid, the coverage state of the DP, and the current energy consumption of the drone. Specifically, for the pth drone, the environment state s of the tth generation p (t) consists of the following three parts:

[0064] 1) TP concentration of the i-th grid;

[0065] 2) The coverage status of the dth DP. If the DP is covered, The value of is 1. Otherwise, The value of is 0.

[0066] 3) Energy consumption of the pth drone.

[0067] For the pth UAV, these three parts reflect the environmental information of the mission area. In the tth generation, the environmental state space of all P UAVs can be expressed as:

[0068]

[0069] In each generation, each drone acquires its coverage area Based on this information, the UAV makes a decision, i.e., selects the next target grid.

[0070] II. Action Space: Exploiting Strategies Each drone chooses an action to perform based on its current state space. The action consists of two components, the flight distance and flight direction of the drone. Specifically, in the tth generation, the action of the pth drone is expressed as:

[0071] 1) The flight distance of the pth drone. Normalize. 0 means the drone is in hovering state. Indicates that the drone has reached its maximum flight distance.

[0072] 2) The flight angle of the pth drone.

[0073] The action space a(t) is the flight distance and flight direction of all P drones in the tth generation, which can be expressed as:

[0074]

[0075] Since the decision variables of the UAV are continuous, the action space is defined as a continuous control task.

[0076] III. Reward: To solve practical problems in DRL, using a suitable reward function enables the drone to learn an excellent strategy. Therefore, in order to solve the multi-drone coverage path planning problem, a new reward function is designed. In particular, for the pth drone of the tth generation, a reward is given after the drone performs a task. This reward is considered as a state-action function, which aims to guide the p-th drone to choose the optimal strategy. It consists of the following parts:

[0077] 1) If DP is included in the coverage area Within, the reward Acquired by the pth drone. equal

[0078] 2) If the pth drone collides with an obstacle, it will receive a penalty The value of is related to the number of collisions.

[0079] 3) If the energy consumption of the pth drone reaches the threshold and the drone has returned to the base, the drone will be rewarded

[0080] 4) If the pth drone chooses the wrong flight direction, it means that the drone did not choose the direction with the lowest TP concentration. At this time, the pth drone will receive a penalty.

[0081] 5) Static sensors pre-deployed in the dam area can quickly collect disaster information and transmit it directly to the base station. However, after a natural disaster occurs, static sensors may be damaged. Abnormal static sensors can cause serious consequences such as misjudgment of disasters. Assuming that the abnormal probability of a static sensor is δ, δ<0.05 indicates that the static sensor functions normally. At this time, if the drone collects disaster information around a normal sensor, it will waste the drone's energy. This is because normal static sensors can directly transmit valid data back to the base station, and the drone does not need to collect this data again. In view of this, the drone needs to be punished. Conversely, 0.05<=δ<1 means that the static sensor is abnormal.

[0082] In the tth generation, the total reward of the pth drone is expressed as:

[0083]

[0084] where ω indicates whether each DP is fairly covered.

[0085] Actor Network: MADDPG contains two networks, namely the actor network and the critic network. In abnormal environments, each drone uses these two networks to perform coverage path planning in the dam abnormal environment. The structure of the actor network contains two FC layers and a softmax layer. The two FC layers choose ReLU as the activation function. In particular, for the pth drone, its state s p (t) is the input of the actor network. Two FC layers are used to extract s p (t). Then, a softmax layer is used to generate the likelihood of the corresponding action. Finally, the actor network selects the next action to be performed based on the likelihood of each action.

[0086] Critic network: The critic network contains 3 FC layers. Specifically, for the p-th drone, the actions a(t) and states s(t) of all P drones are used as the input of the p-th critic network. The first two FC layers use ReLU as the activation function to capture the hidden features in the input. The last FC layer produces a Q value, denoted as Q(s,a).

[0087] Step 3: Update the model parameters of MADDPG in step 2;

[0088] For an actor network, the set of policies is denoted as ψ = ψ1, ψ2, …, ψ P , where φ=φ1,φ2,…,φ P is the set of policy parameters for all P drones. At the beginning of each generation, each drone randomly selects Specifically, when the batch is input to the p-th actor network, the gradient of the expected reward is expressed as:

[0089]

[0090] Where m represents the index of the sample. Based on the observed state of strategy ψ, is the expected discounted reward after performing actions (a1, a2, …, a P ). represents the gradient of the policy parameters of the p-th actor network.

[0091] Afterwards, the weight parameters of the critic network are updated by the time difference error as:

[0092]

[0093] in, is the target Q-value of the p-th critic network and is defined as:

[0094]

[0095] in, Represents an approximate function. It is a target strategy. represents the reward value of the p-th critic network on the m-th sample. γ represents the weight coefficient. represents the policy function. Furthermore, the loss function l(φ p ) is represented as:

[0096]

[0097] Finally, the parameters of the target network are updated according to the actor network and the critic network. Soft update is a method used to update the target network. This update method can be expressed as:

[0098]

[0099] Among them, τ represents the moving distance from the target network to the evaluation network at each update. φ Q are the parameters of the trained critic network. Represents the policy parameters of the target network. Represents the parameters of the critic network in the target network. Finally, by comparing the performance of the model proposed in the present invention with some advanced DRL models in the simulated dam abnormal environment, it can be found that the model proposed in the present invention has a larger drone coverage rate and lower drone energy consumption in multi-drone coverage path planning.

[0100] This embodiment also relates to a storage medium, in which any one of the above-mentioned coverage path planning methods is stored.

[0101] Obviously, those skilled in the art should understand that the above-mentioned embodiments of the present invention, a method for planning a coverage path for multiple drones in abnormal dam environment inspection, can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A coverage path planning method for multiple UAVs in abnormal dam environment inspection, characterized in that: The steps include: Step 1: Formalize the coverage path planning problem of multiple UAVs in dam abnormal environment inspection; Step 2: Construct a multi-UAV coverage path planning model based on DRL; Step 3: Update model parameters.

2. The coverage path planning method for multiple UAVs in abnormal dam environment inspection according to claim 1 is characterized in that: The formal definition of the coverage path planning problem of multiple UAVs in the abnormal environment inspection of the dam is divided into two steps, namely, defining the abnormal environment and defining the optimization problem; The abnormal environment is specifically defined as follows: the dam area after a natural disaster is called the dam abnormal environment; the dam abnormal environment is defined by the following components: the dam area including boundaries and obstacles, the disaster points in the dam area, the severity of the disaster at each disaster point, static sensors, multiple drones, and the coverage area of ​​each drone. The definition optimization problem is specifically: After defining the abnormal environment of the dam, the coverage path planning problem of multiple UAVs in the abnormal environment inspection of the dam is defined as an optimization problem. The optimization objectives include the coverage rate of multiple UAVs and the energy consumption of multiple UAVs, aiming to minimize the energy consumption of multiple UAVs and maximize the coverage area of ​​multiple UAVs.

3. The coverage path planning method for multiple UAVs in abnormal dam environment inspection according to claim 1 is characterized in that: Construct a multi-UAV coverage path planning model based on DRL, which can achieve the joint optimization of coverage and energy consumption of multiple UAVs during the inspection process of dam abnormal environment; The DRL-based multi-UAV coverage path planning model mainly consists of two parts, namely TP and MADDPG; TP establishes a communication medium between UAVs, realizes indirect communication of UAVs in abnormal dam environments, manages the path information of UAVs, and simplifies the complex interactions between UAVs into low-dimensional representations; MADDPG is used to realize the coverage path planning of multi-UAVs; in MADDPG, the reward function is used to simultaneously achieve the goals of maximizing the coverage rate of multi-UAVs and minimizing the energy consumption of multi-UAVs.

4. The coverage path planning method for multiple UAVs in abnormal dam environment inspection according to claim 3 is characterized in that: The MADDPG is specifically: Each drone has an independent MADDPG. Each MADDPG consists of two networks, the actor network and the critic network. The actor network consists of two FC layers and a softmax layer. This network is designed to generate the probability of an action that a drone can perform. Based on the probability, the drone chooses the next action to be performed. The critic network consists of three FC layers to generate Q values. MADDPG updates parameters by updating the actor network and critic network for each drone separately; each drone has an independent actor-critic network pair, where the actor network is updated according to the gradient of the expected return, and the critic network updates the Q value according to the Bellman equation; during the training process, a centralized training method is used, that is, the critic network of each drone takes into account the actions of all drones; but the execution is decentralized, that is, each drone acts independently according to its own strategy.

5. According to the method for planning the coverage path of multiple UAVs in the inspection of abnormal dam environment in claim 3, in order to solve the problem of planning the coverage path of multiple UAVs, a reward function is designed; for the pth UAV of the tth generation, a reward is given after the UAV performs a task. This reward is regarded as a state-action function, which aims to guide the pth drone to choose the optimal strategy; It consists of the following parts: 1) If DP is included in the coverage area Within, the reward Acquired by the pth drone, equal 2) If the pth drone collides with an obstacle, it will receive a penalty The value of is related to the number of collisions; 3) If the energy consumption of the pth drone reaches the threshold and the drone has returned to the base, the drone will be rewarded 4) If the pth drone chooses the wrong flight direction, it means that the drone did not choose the direction with the lowest TP concentration, and the pth drone will be penalized. 5) Assume that the abnormal probability of the static sensor is δ, δ<set value, indicating that the static sensor functions normally; at this time, if the drone collects disaster information around the normal sensor, it will waste the drone's energy and the drone needs to be punished; on the contrary, 0.05<=set value<1 means that the static sensor is abnormal. In the tth generation, the total reward of the pth drone is expressed as:

6. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the coverage path planning method for multiple UAVs in abnormal environment inspection of dams as described in any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the coverage path planning method for multiple UAVs in abnormal dam environment inspection as described in any one of claims 1-5.

Images