Multi-unmanned aerial vehicle cooperative offshore wind turbine inspection route planning method and device

Abstract

The invention discloses a multi-unmanned aerial vehicle cooperative offshore wind turbine inspection route planning method and device, and belongs to the technical field of offshore wind turbine inspection route planning. The routing inspection path planning method comprises the following steps: S10, acquiring and analyzing wind speed information, establishing a synthetic speed model, and evaluating and calculating routing inspection flight time of an unmanned aerial vehicle among different fans; s20, generating global information based on the priori knowledge generated by the A * algorithm, and storing the priori knowledge generated by exploring an unknown environment into a priori playback pool; s30, distributing an inspection sub-area for each unmanned aerial vehicle; and S40, planning a routing inspection path for each unmanned aerial vehicle so as to realize a single-region routing inspection path within the shortest time. Based on an improved deep reinforcement learning algorithm, the optimal-A *-DQN algorithm is specifically adopted, and the multi-unmanned aerial vehicle cooperative offshore wind turbine inspection path planning efficiency, the convergence speed, the task completion rate, the target achievement rate and the dynamic environment adaptation capability are remarkably superior to those of a traditional algorithm in the aspects of multi-unmanned aerial vehicle cooperative offshore wind turbine inspection path planning efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of offshore wind turbine inspection path planning, and in particular relates to a method and device for planning an offshore wind turbine inspection path in collaboration with multiple unmanned aerial vehicles. Background Art

[0002] In recent years, drone technology has achieved rapid development and has been widely used in many fields. In particular, the role of drones has become increasingly prominent in the inspection of offshore wind farms. However, the number of wind turbines in offshore wind farms is large and widely distributed, and autonomous path planning of drones has become a key technical challenge. Traditional path planning algorithms such as artificial potential field algorithm, RRT algorithm, etc., as well as intelligent algorithms such as ant colony algorithm, gray wolf optimization algorithm, etc., have certain limitations when dealing with path planning problems in complex environments.

[0003] As an effective path planning method, deep reinforcement learning, especially the Deep Q-Network (DQN) algorithm, has shown great potential in the field of path planning. The DQN algorithm uses a deep neural network to fit the action value function Q of the Q-learning algorithm to generate an estimated Q value, which significantly improves the ability to process high-dimensional perceptual inputs and the overall learning efficiency. At the same time, the DQN algorithm introduces two key technologies: Experience Replay and Target Network. The experience replay mechanism maintains an experience replay pool to store the state, action, reward, and next state observed by the agent in the environment, and randomly extracts experience samples from it to train the network. The target network provides a relatively stable target Q value, reducing the training fluctuations caused by excessively frequent target Q values ​​during the learning process. This breaks the temporal correlation between samples, improves the stability of learning, and optimizes the long-term decision-making process.

[0004] However, the traditional DQN algorithm is inefficient, slow to converge, and wastes resources when dealing with unknown environments due to random action selection in the initial training phase. In addition, the single DQN algorithm is not general enough to deal with large-scale, dynamically changing scenarios such as offshore wind farms, and it is difficult to simultaneously ensure the optimal time and optimal path repetition rate. Summary of the invention

[0005] The embodiment of the present invention provides a method and device for multi-UAV coordinated offshore wind turbine inspection path planning, aiming to solve the problems that the traditional DQN algorithm is inefficient, slow in convergence speed, and wastes in resource utilization due to the random action selection in the initial training stage when dealing with unknown environments; and the single DQN algorithm is insufficient in generalization when dealing with large-scale, dynamically changing scenarios such as offshore wind farms, and it is difficult to simultaneously ensure the optimal time and optimal path repetition rate. The multi-UAV coordinated offshore wind turbine inspection path planning method of the present invention is based on an improved deep reinforcement learning algorithm, and specifically adopts the option-A*-DQN algorithm; wherein, the A* algorithm is used as a heuristic algorithm to optimize the initial action selection of the DQN algorithm, improve the training efficiency and convergence speed; the option concept is introduced to deal with the problem of complex UAV action space, by abstracting a series of actions into high-level actions, such as synthesizing the flying to the wind turbine and the collection action into an abstract action of flying to the wind turbine and collecting, thereby simplifying the decision-making process.

[0006] The technical solution of the present invention is achieved in this way:

[0007] A method for multi-UAV collaborative offshore wind turbine inspection path planning includes the following steps:

[0008] S10. Acquisition and analysis of wind speed information in the offshore wind turbine inspection area, establishment of a synthetic speed model using the wind speed information, and evaluation and calculation of the inspection flight time of the UAV between different wind turbines based on the synthetic speed model;

[0009] S20. Generate global information based on the prior knowledge generated by the A* algorithm, and store the prior knowledge generated by exploring the unknown environment in the prior replay pool;

[0010] S30. The UAV inspection command and dispatch center allocates an inspection sub-area to each UAV;

[0011] S40. After allocating the inspection sub-area to each drone, the drone inspection command and dispatch center plans an inspection path for each drone to achieve the shortest inspection path in a single area;

[0012] S50. Monitor the endurance status of each drone during the inspection mission, and provide feedback on the inspection route length and time after recalling the drone.

[0013] Preferably, in step S10,

[0014] The acquisition and analysis of the wind speed information is specifically as follows: the wind direction in the polar coordinate system of the wind speed information is expressed as:

[0015]

[0016] in, is the wind direction in the polar coordinate system, w=[w x ,w y ] is wind speed, w x and w y are the projections of wind speed on the x-axis and y-axis respectively; in the polar coordinate system, the phase is expressed in the counterclockwise direction.

[0017] The synthetic speed model, specifically the relationship between the synthetic speed of the drone, the drone speed and the wind speed, is as follows:

[0018]

[0019] in, For UAV U from wind turbine k 1 Fly to fan k 2 The synthesis speed of For UAV U from wind turbine k 1 Fly to fan k 2 The speed of the drone at the time, w is the wind speed. When the drone flies to the wind turbine for inspection, it may encounter two wind conditions, namely tailwind and headwind. The wind conditions need to be considered in the decision-making of the drone. The synthetic speed of the drone is used to evaluate and calculate the flight time between different wind turbines, which makes the evaluation and calculation more accurate.

[0020] The evaluation and calculation of the inspection flight time of the UAV between different wind turbines is performed using the following specific evaluation calculation formula:

[0021]

[0022] in, For the kth 1 The coordinates of the typhoon, For the kth 2 The coordinates of the typhoon, express Euclidean norm of a vector; For UAV U from wind turbine k 1 Fly to fan k 2 The synthesis speed of For UAV U from wind turbine k 1 To fan k 2 The inspection flight time between

[0023] Preferably, in step S20: the generation of global information adopts the option-A*-DQN algorithm, wherein the A* heuristic algorithm can generate static prior path information and Q table on the one hand, which as prior information reduces the initial exploration time and computing resources of the option-A*-DQN algorithm, and improves the efficiency and stability in the initial training; on the other hand, the weights of the prior knowledge Q value and the estimated Q value can be reasonably adjusted to generate the overall Q value of the system, realize collaborative optimization between intelligent agents, further optimize the action exploration strategy and multi-objective reward function, balance the exploration and utilization relationship, and enhance the autonomous decision-making ability of the drone, so as to be more effectively applied to the complex environment of multiple drones.

[0024] Step S20 specifically includes the following steps:

[0025] S21. Generate prior knowledge based on A* algorithm;

[0026] S22. Using the prior knowledge of exploration, the option-A*-DQN algorithm generates global information, designs dynamic weight parameters, and uses prior Q values ​​to optimize the multi-agent Q value function;

[0027] S23. Explore the unknown environment to generate prior knowledge and store it in the prior replay pool D*.

[0028] Preferably, in step S21, the A* algorithm evaluates the potential cost of each node through a heuristic function, thereby preferentially searching for paths that are more likely to lead to the target; in drone path planning, this can reduce the search space and speed up the process of finding a valid path. The scoring function (fScore) of the A* algorithm consists of two parts: the actual cost from the starting point to the current node (gScore) and the estimated cost from the current node to the target node (hScore):

[0029] f(n)=g(n)+h(n);

[0030] Where f(n) is the total score of node n, g(n) is the actual path cost from the starting point to node n, and h(n) is the heuristic estimated cost from node n to the target node.

[0031] In step S22, during the learning process of option-DQN, the A* algorithm can serve as a priori knowledge source to provide high-quality sample paths for the Q network, accelerate the learning process and improve the convergence of the strategy.

[0032] In step S23, the offshore environment may be affected by dynamic factors such as wind and waves. The A* algorithm is combined with option-DQN to dynamically adjust the heuristic function to adapt to environmental changes and plan the optimal path in real time.

[0033] Preferably, in step S30:

[0034] Step S30 specifically includes the following steps:

[0035] S31. Use the K-Means algorithm to divide the drone inspection area, and use the option-A*-DQN algorithm to generate inspection paths for the divided sub-areas; then convert the multi-drone collaborative inspection into a single-drone inspection in each sub-area, avoiding the duplication of multiple drone inspection paths in the traditional method, which leads to problems such as low task completion rate, and improving the efficiency of drone inspection;

[0036] S32. When a dynamic obstacle approaches during a drone inspection, the drone inspection device, i.e. the sensor carried by the drone, senses the distance between the drone and the dynamic obstacle. If it is found that the distance exceeds the safe distance, it will trigger automatic obstacle avoidance and adjust the path, thereby ensuring the stability and safety of the drone inspection.

[0037] The dynamic obstacles include other UAVs other than a certain cruising UAV, seabirds, airplanes and other flying objects on the cruising path.

[0038] In step S31, the K-Means algorithm divides the drone inspection area, specifically: minimizing the following objective function:

[0039]

[0040] Among them, ρ is the number of clusters, corresponding to the number of divisions of the inspection fan range, σ i is the i-th cluster, containing all wind turbines belonging to this cluster, is the centroid of the ith cluster, usually the mean of all points in the cluster, g is the coordinate of a wind turbine in the cluster, is the wind turbine coordinate g to the cluster centroid The square of the Euclidean distance.

[0041] Further preferably, in consideration of minimizing the number of drones, multi-scale clustering is used to improve the K-Means algorithm in step S31. Because multi-scale clustering allows resource allocation at different levels, the number of drones required can be effectively reduced while ensuring the completion of the task.

[0042] The multi-scale clustering described above first performs coarse-grained clustering to divide adjacent wind turbines into larger coarse-grained clusters, and then performs fine-grained clustering within the coarse-grained clusters. In this way, fewer drones can be used for inspection in larger clusters, and the drones can cover more wind turbines in the cluster.

[0043] The coarse-grained clustering: using a larger distance threshold for preliminary clustering,

[0044] C j ={x|d(x,m j )<=T_coarse}

[0045] Among them, C j is the jth cluster, m j It is cluster C j The centroid of T_coarse is the threshold of coarse-grained clustering. Fine-grained clustering: fine-grained clustering is performed within each coarse-grained cluster.

[0046] C ij ={x|d(x,m ij )<=T_fine}]

[0047] Among them, C ij is the i-th fine-grained cluster within the j-th coarse-grained cluster, m ij It is cluster C ij is the centroid of , and T_fine is the threshold for fine-grained clustering.

[0048] Multi-scale clustering helps reduce the overlap of drone inspection paths, thereby reducing unnecessary flight time and energy consumption. The distribution of offshore wind turbines may be uneven, and multi-scale clustering can better adapt to this change, ensuring that more resources are used in areas with dense wind turbines and less in sparse areas.

[0049] Preferably, in step S40:

[0050] The semi-Markov decision process (SMDP) is used to model the UAV inspection task. The six-tuple of the model is expressed as follows:<S,A,O,P,R,γ> , where S represents the state space, A represents the action space, O represents the option set, P is S×A→ΔS, which represents the state transition probability matrix, R is S×A×S→R, which represents the immediate reward function, and γ∈[0,1] represents the discount factor; the basic elements of the agent, state, action, option set, and immediate reward are modeled.

[0051] The basic elements of the agent, state, action, option set, and immediate reward are modeled as follows:

[0052] (1) Single agent: The drone is a single agent, and its option selection strategy is μ. At the beginning of the mission, the drone is based on its initial state s 0 Select an option, denoted as OP 0 , and according to OP 0 Internal strategy of 0 Perform a series of actions, in OP 0At the end of ξ, the drone gets the total instant reward r for this option. 0 , observe its own state information s again ξ , and select the next option to be executed according to μ, recorded as OP ξ , and the cycle continues until the drone mission is completed;

[0053] (2) State: Assume that the drone can fully observe its own state, that is, the state s at each time t t =(cr t ,p t ,e t ), s t ∈S, where cr t is a vector consisting of the percentage of each wind turbine inspected at time t, that is, p t is the position vector of the drone at time t, i.e. p t =(x t ,y t ,z t ), since the height of the drone is a constant H, then z t =H;e t The remaining battery power of the drone;

[0054] (3) Action: The basic action set A of the drone contains three basic actions, namely, the action of flying in a straight line towards a target. f 、Charging action a c And inspection action a cr , at each time t, perform action a f Make the drone fly to the specified target direction at a speed v 0 Fly in a straight line for one unit of time and perform action a c Make the drone fully charged and execute action a cr The amount of data collected by the drone inspection is C;

[0055] (4) Option set: The option set O executed by the drone includes inspecting a certain wind turbine SN i Data OP s,i (i∈N), return to charge OP c And end the mission OP p There are three options, namely O={OP s,1 ,OP s,2 …,OP s,N ,OP c ,OP p}; each option is a triple OP ,π OP ,β​OP >, the number of elements in option set O is |O|=N+2; the set of options that the drone can choose in any state is the entire option set O, that is The strategy π within each option OP are all set to known fixed strategies, and the termination condition β of each option OP All actions are executed. Specifically, for the inspection fan optionOP s,i , its strategy is to fly straight from the current position to the wind turbine SN i And collect the data of the fan i , until the collection is completed, the current option will not be exited; for return charging OP c The strategy is to fly straight to the smart hangar and charge it, and then exit OP after it is fully charged. c ; For ending the mission OP p , whose strategy is for the drone to fly straight to the end point, inform the environment that the mission is over and exit OP p ;

[0056] (5) Instant reward: The agent only receives the total instant reward r for each option at the end of each option. ξ . r ξ is the initial state of option s ξ and option action OP ξ The function of r ξ =r ξ (s ξ ,OP ξ ). Set the total instant reward of an option to be divided into power rewards Inspection Rewards and path rewards Among the three parts, the power reward is mainly used to punish the drone for running out of power during the execution of the option, that is,

[0057]

[0058] Among them, N e It is a negative constant. When the power is less than or equal to the minimum power B, a larger penalty value is given to the drone to force it to return. The minimum power B is the power required to support the drone to return and charge.

[0059] The inspection reward is used to punish the drone for repeatedly selecting the wind turbine option that has already been inspected.

[0060]

[0061] Among them, N cis a negative constant, Indicates the fans that have been inspected.

[0062] The path reward The distance l that the drone has flown within this option ξ It is inversely proportional to the speed of the UAV, which is used to guide the UAV to learn to fly the shortest possible path to inspect the wind turbine, which can be expressed as

[0063]

[0064] Among them, N l is a negative constant. Finally, the drone gets an immediate reward r after experiencing an option. ξ is the sum of the above three rewards, expressed as

[0065]

[0066] The state transfer matrix P in the SMDP model describes the dynamic rules of the environment in this scenario. It defines the probability distribution of the next state s′ of the drone when it takes action a in any state s:

[0067] Similar to the state action value function Q in MDP π (s,a), in the SMDP model, define the state-option value function To represent the state-option tuple (s, OP) under the strategy μ and option set O, the expected cumulative return can be obtained as:

[0068]

[0069] Among them, γ∈[0,1] represents the discount factor of the cumulative immediate reward, Satisfies the Bellman Expectation Equation. In order to solve the SMDP model, it is necessary to find the optimal state-option value function It should satisfy the Bellman Optimality Equation, as shown below. In order to solve the Bellman Optimality Equation, Monte Carlo algorithm, Temporal Difference (TD) algorithm, etc. can be used for solution.

[0070]

[0071] The option-A*-DQN algorithm execution architecture shows the reasoning process during algorithm execution. The interaction between the drone and the environment is from the previous optionOP ξ-1Exit, and get the overall instant reward of this option is r ξ-1 And the next status information is s ξ ,s ξ Includes the percentage vector cr of data collected by all wind turbines currently ξ , the location information of the drone ξ And the remaining battery of the drone ξ ; Then the current s ξ The value function neural network Q input to the option-A*-DQN algorithm ot In the figure, the network consists of an input layer, a hidden layer, and an output layer. The hidden layer consists of two fully connected layers. The first fully connected network contains 1024 neurons, and its activation function uses the ELU (Exponential Linear Unit) function. This activation function has the characteristic of exponential decay in the negative value domain, can provide a smooth gradient, and helps to accelerate the training process.

[0072] Among them, the output of the first layer network can be expressed as

[0073]

[0074] Among them, W 1 is the weight parameter of the first layer of neural network, b 1 is its bias parameter. The input of the second hidden layer is the output of the first hidden layer. The second hidden layer consists of 300 neurons. Its activation function is the same as the previous layer, and the ELU function is also used. The output of the second layer network can be expressed as

[0075]

[0076] Among them, W 2 and b 2 are the weight parameters and bias parameters of the second layer network respectively. The output layer receives the output X of the second layer network 1 , and use the Softmax activation function to output the |O|-dimensional vector q as

[0077]

[0078] Among them, W 3 and b 3 are the weight parameter and bias parameter of the output layer respectively. Softmax is a normalized exponential function that can convert the output of the neural network into a probability distribution.

[0079] Finally, the value function neural network Q ot The output q=(q 1 ,q 2 ,…,q |O|) is the probability of choosing each option, that is

[0080] The best option is found by using the ε-greedy algorithm. ε is a small value between 0 and 1. Each time, a random selection is made with a probability of ε, and a greedy algorithm is used to select with a probability of 1-ε, that is, the index of the largest value in q is selected as the option OP to be selected. ξ , the greedy algorithm can be expressed as:

[0081] OP ξ = arg max(q j )

[0082] Finally, select OP in option set O ξ The corresponding strategy π ξ and the termination condition β ξ And output corresponding actions to continue interacting with the environment.

[0083] The training algorithm of the option-A*-DQN algorithm is specifically:

[0084] When training the option-A*-DQN algorithm, set the drone experience memory Among them, e k is the kth experience vector, that is Indicates the current state. Indicates the option action obtained according to the current option-A*-DQN algorithm. is (s′ k ,o′ k ) to get the overall immediate feedback, and o′ k represents the next state and action of the drone after interacting with the environment, and D c Represents the maximum storage capacity of the memory. The value function neural network Q is trained by experience replay and experience random extraction ot , which can break the correlation between data and make the data satisfy the characteristics of independent and identical distribution as much as possible, thereby enhancing the stability of training. ot Also known as the evaluation network, in addition, set the target network Used to approximate the optimal evaluation network Right now

[0085] Preferably, in step S50:

[0086] Step S50 specifically includes:

[0087] S51. When the drone is performing an inspection task, if it detects that the fluctuation range of the wind speed and wind force at sea exceeds the threshold, the change information is fed back; the drone inspection command and dispatch center recalculates the cost of the path to ensure that each step of the inspection path is optimal;

[0088] S52. The UAV inspection command and dispatch center monitors the endurance status of each UAV in real time. If the endurance is insufficient, the UAV is recalled for charging. The center also makes dynamic adjustments based on the remaining energy consumption of each UAV, the distance between UAVs, and the workload, and generates a new inspection area allocation plan.

[0089] S53. If the inspection task is completed or the meteorological factors of wind speed and wind force change significantly, the drone will be recalled and the inspection path length and time will be fed back.

[0090] A device for multi-UAV coordinated offshore wind turbine inspection path planning, applied to the above inspection path planning method, comprising a UAV inspection device and a UAV inspection command and dispatch center;

[0091] The UAV inspection device includes an industry-level flight platform UAV, and ensures that the UAV has appropriate flight performance, endurance and carrying capacity, and can be used to detect and locate targets, and obtain sea conditions and environmental information;

[0092] The UAV inspection command and dispatch center is responsible for formulating inspection tasks, planning routes, and assigning tasks to UAVs. It can also receive and analyze data from UAVs.

[0093] Preferably, the industrial-grade flying platform UAV is equipped with a full-frame high-resolution gimbal camera, an infrared sensor, a multi-spectral sensor and a radar for detecting and locating targets, and acquiring sea conditions and environmental information.

[0094] The UAV inspection command and dispatch center is the hub of the entire device, and performs real-time monitoring and dispatching of tasks.

[0095] Compared with the prior art, the present invention has the following beneficial effects:

[0096] 1. The present invention constructs a synthetic speed model based on the wind speed information of the area to be inspected, updates the marine environment information in real time, and reasonably divides the area to be inspected, so that the UAV can choose a flight path that is more suitable for the current environment, which helps to improve the stability and safety of the UAV flight.

[0097] 2. The present invention adopts regional division in the coordinated inspection of multiple drones at sea to avoid the duplication of inspection paths of multiple drones in traditional methods, which leads to problems such as low task completion rate, and improves the efficiency of drone inspection.

[0098] 3. The present invention considers balancing the load of drones when allocating sub-areas to drones, thereby solving the problem of low efficiency caused by multiple round trips of drones due to insufficient endurance in traditional methods, and at the same time reduces the number of drones performing inspection tasks as much as possible to reduce costs.

[0099] 4. Through simulation experiments of multi-agent path planning, the present invention verifies that the option-A*-DQN algorithm is significantly superior to traditional algorithms in terms of path planning efficiency, convergence speed, task completion rate, goal achievement rate and ability to adapt to dynamic environments, proving its high efficiency and robustness in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 Flowchart of static and dynamic scenes in an embodiment of the present invention.

[0101] Figure 2 This is a drone inspection path diagram in an embodiment of the present invention. The horizontal and vertical coordinates are longitude and latitude. The diagram is divided into three sub-areas, showing the path trajectories of three drone inspections. The small gray circles in the diagram are randomly set obstacles.

[0102] Figure 3 This is a cycle reward diagram of the option-A*-DQN algorithm in an embodiment of the present invention, where the horizontal axis is the number of training cycles and the vertical axis is the cumulative overall timely reward. DETAILED DESCRIPTION

[0103] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0104] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, top, bottom...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0105] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0106] At present, when dealing with unknown environments, the traditional DQN algorithm has low efficiency, slow convergence speed, and waste of resources due to the random action selection in the initial training stage; and when dealing with large-scale, dynamically changing scenarios such as offshore wind farms, the single DQN algorithm has insufficient generalization and is difficult to simultaneously ensure the optimal time and optimal path repetition rate. In order to solve the above technical problems, the present invention proposes a multi-UAV collaborative offshore wind turbine inspection path planning method and device. The multi-UAV collaborative offshore wind turbine inspection path planning method of the present invention is based on an improved deep reinforcement learning algorithm, and specifically adopts the option-A*-DQN algorithm; wherein, the A* algorithm is used as a heuristic algorithm to optimize the initial action selection of the DQN algorithm, improve training efficiency and convergence speed; the option concept is introduced to deal with the problem of complex UAV action space, by abstracting a series of actions into high-level actions, such as synthesizing the flying to the wind turbine and the collection action into an abstract action of flying to the wind turbine and collecting, thereby simplifying the decision-making process.

[0107] Example

[0108] A device for multi-UAV coordinated offshore wind turbine inspection path planning, comprising a UAV inspection device and a UAV inspection command and dispatch center;

[0109] The drone inspection device includes an industry-level flight platform drone, and ensures that the drone has appropriate flight performance, endurance and carrying capacity. The drone is also equipped with various equipment, such as a full-frame high-resolution gimbal camera, infrared sensor, multi-spectral sensor, radar, etc., for detecting and locating targets, and obtaining sea conditions and environmental information.

[0110] The UAV inspection command and dispatch center is responsible for formulating inspection tasks, planning routes, and assigning tasks to UAVs. It can also receive and analyze data from UAVs. It is the hub of the entire system and monitors and dispatches tasks in real time.

[0111] This embodiment provides a method for planning a multi-UAV coordinated offshore wind turbine inspection path, comprising the following steps:

[0112] S10. Acquisition and analysis of wind speed information in the offshore wind turbine inspection area, establishment of a synthetic speed model using the wind speed information, and evaluation and calculation of the inspection flight time of the UAV between different wind turbines based on the synthetic speed model;

[0113] Initialize the UAV inspection command and dispatch center, set the mission scenario to an offshore wind farm, obtain and analyze the relevant wind speed information of the area to be inspected, and establish a wind speed model based on the information obtained.

[0114] The total number of wind turbines in the offshore wind farm is T, and the coordinates of the kth wind turbine are g = [x k ,y k ], wind speed is w=[w x ,w y ],w x and w y are the projections of wind speed on the x-axis and y-axis respectively.

[0115] The wind direction in the polar coordinate system containing wind speed information is expressed as:

[0116]

[0117] in, is the wind direction in the polar coordinate system, which is different from the wind direction in meteorological measurement. x ,w y ] is wind speed, w x and w y are the projections of wind speed on the x-axis and y-axis respectively. Therefore, the wind direction in meteorological measurement is expressed as In meteorological measurements, 0, π and represent the north, east, south, and west winds respectively. Therefore, the phase is expressed in a clockwise direction; however, in the polar coordinate system, the phase is expressed in a counterclockwise direction. Therefore, and The relationship between them is:

[0118]

[0119] When a drone flies to a wind turbine for inspection, it may encounter two wind conditions, namely tailwind and headwind, which need to be considered in the drone’s decision-making. Therefore, define the drone U from the wind turbine k 1 Fly to fan k 2 The synthesis speed for:

[0120]

[0121] Define UAV from wind turbine k 1Fly to fan k 2 Drone speed at for:

[0122]

[0123] in, and is the projection on the x-axis, and is the projection onto the y-axis.

[0124] The drone speed is the initial speed of the drone, and the composite speed is the speed affected by the wind speed. The relationship between the drone speed, wind speed, and composite speed is expressed as:

[0125]

[0126] use and Represents airspeed and groundspeed respectively. The maximum speed limit of drone U is generally, Refers to the maximum airspeed. However, if the drone flies at too high a ground speed, the structural capacity of the drone may decrease and the drone may not be able to remain stable. Therefore, both the airspeed and ground speed are limited to Specifically, for tailwind conditions, the ground speed limit is When the drone faces headwind, the airspeed is limited to and The angle between In addition, Used to represent w and For UAV U, the maximum wind speed resistance is express.

[0127] UAV U from wind turbine k 1 To fan k 2 The flight time can be calculated as:

[0128]

[0129] in, For the kth 1 The coordinates of the typhoon, For the kth 2 The coordinates of the typhoon, express Euclidean norm of a vector.

[0130] UAV U also has a maximum flight time, expressed as It represents the upper limit of the total flight time during the inspection process.

[0131] S20. Generate global information based on the prior knowledge generated by the A* algorithm, and store the prior knowledge generated by exploring the unknown environment in the prior replay pool D*;

[0132] In the option-A*-DQN algorithm of the present invention, the A* heuristic algorithm can generate static prior path information and Q table on the one hand, which can reduce the initial exploration time and computing resources of the option-A*-DQN algorithm as prior information, and improve the efficiency and stability in the initial training; on the other hand, it can reasonably adjust the weights of the prior knowledge Q value and the estimated Q value to generate the overall Q value of the system, realize collaborative optimization between intelligent agents, further optimize the action exploration strategy and multi-objective reward function, balance the exploration and utilization relationship, and enhance the autonomous decision-making ability of the drone, so as to be more effectively applied to the complex environment of multiple drones.

[0133] S21. Generate prior knowledge based on A* algorithm;

[0134] The A* algorithm uses a heuristic function to evaluate the potential cost of each node, thereby prioritizing the search for paths that are more likely to lead to the target; in drone path planning, this can reduce the search space and speed up the process of finding an effective path.

[0135] The scoring function (fScore) of the A* algorithm consists of two parts: the actual cost from the starting point to the current node (gScore) and the estimated cost from the current node to the target node (hScore):

[0136] f(n)=g(n)+h(n);

[0137] Where f(n) is the total score of node n, g(n) is the actual path cost from the starting point to node n, and h(n) is the heuristic estimated cost from node n to the target node.

[0138] S22. Using the prior knowledge of exploration, the option-A*-DQN algorithm generates global information, designs dynamic weight parameters, and uses prior Q values ​​to optimize the multi-agent Q value function;

[0139] In the learning process of option-DQN, the A* algorithm can serve as a source of prior knowledge, providing high-quality sample paths for the Q network, accelerating the learning process and improving the convergence of the strategy.

[0140] S23. Explore the unknown environment to generate prior knowledge and store it in the prior replay pool D*.

[0141] The offshore environment may be affected by dynamic factors such as wind and waves. The A* algorithm is combined with option-DQN to dynamically adjust the heuristic function to adapt to environmental changes and plan the optimal path in real time.

[0142] Specific application scenarios such as Figure 1 As shown:

[0143] According to the static and dynamic scenarios of offshore wind farms, static and dynamic problem models are established. In the early stage of training, the A* algorithm is used to provide an initial path for the drone, and the option-DQN algorithm is optimized on this basis.

[0144] When the drone encounters obstacles or other emergencies during a mission, the A* algorithm can quickly plan a local path to bypass the obstacle, and option-DQN is responsible for integrating these local paths to form a global optimal strategy.

[0145] When energy is limited, the A* algorithm can help the drone plan the most energy-saving path, and option-DQN can learn how to extend the drone's flight time as much as possible while ensuring mission completion.

[0146] S30. The UAV inspection command and dispatch center assigns each UAV an inspection sub-area (such as Figure 2 shown);

[0147] S31. The K-Means algorithm is used to divide the drone inspection area, and the option-A*-DQN algorithm generates inspection paths for the divided sub-areas, and then the multi-drone collaborative inspection is converted into a single-drone inspection in each sub-area, avoiding the duplication of multiple drone inspection paths in traditional methods, which leads to low task completion rate and other problems, and improving the efficiency of drone inspection.

[0148] Minimize the following objective function:

[0149]

[0150] Among them, ρ is the number of clusters, corresponding to the number of divisions of the inspection fan range, σ i is the i-th cluster, containing all wind turbines belonging to this cluster, is the centroid of the ith cluster, usually the mean of all points in the cluster, g is the coordinate of a wind turbine in the cluster, is the wind turbine coordinate g to the cluster centroid The square of the Euclidean distance.

[0151] Secondly, considering minimizing the number of drones, multi-scale clustering is used to improve the K-Means algorithm. This is because multi-scale clustering allows resource allocation at different levels, which can effectively reduce the number of drones required while ensuring the completion of the task.

[0152] By first performing coarse-grained clustering, adjacent wind turbines can be divided into larger clusters, and then fine-grained clustering can be performed within these clusters. This allows fewer drones to be used for inspection in larger clusters, and the drones can cover more wind turbines within the cluster.

[0153] Coarse-grained clustering: Use a larger distance threshold for preliminary clustering

[0154] C j ={x|d(x,m j )<=T_coarse}

[0155] Among them, C j is the jth cluster, m j It is cluster C j The centroid of , T_coarse is the threshold of coarse-grained clustering. Fine-grained clustering: fine-grained clustering is performed within each coarse-grained cluster

[0156] C ij ={x|d(x,m ij )<=T_fine}]

[0157] Among them, C ij is the i-th fine-grained cluster within the j-th coarse-grained cluster, m ij It is cluster C ij is the centroid of , and T_fine is the threshold for fine-grained clustering.

[0158] Multi-scale clustering helps reduce the overlap of drone inspection paths, thereby reducing unnecessary flight time and energy consumption. The distribution of offshore wind turbines may be uneven, and multi-scale clustering can better adapt to this change, ensuring that more resources are used in areas with dense wind turbines and less in sparse areas.

[0159] S32. When a drone is conducting an inspection, it is set to treat other drones as dynamic obstacles, including creatures such as seabirds. When a dynamic obstacle approaches, the drone inspection device, that is, the sensor carried by the drone, senses the distance to the dynamic obstacle. If it is found that the safe distance is exceeded, it will trigger automatic obstacle avoidance and adjust the path, thereby ensuring the stability and safety of the drone inspection.

[0160] The sensor on the drone can detect the distance of dynamic obstacles as d, and define a safe distance threshold as d safe .

[0161] d = distance_sensor(ds)

[0162] Among them, ds represents a dynamic obstacle.

[0163] When the detected distance d is less than the safety distance threshold d safe , the obstacle avoidance mechanism is triggered.

[0164] ifd <d safe thentriggeravoidance

[0165] The obstacle avoidance function is defined as avoidance_function(s,d), where s is the current state of the drone and d is the distance to the dynamic obstacle. The output of this function is an obstacle avoidance action a.

[0166] a=avoidance_function(s,d)

[0167] Calculate obstacle avoidance direction This direction points away from obstacles.

[0168]

[0169] in, is the location of the drone, is the location of the obstacle.

[0170] Based on the obstacle avoidance direction, an obstacle avoidance action a is determined, which can be a change in speed or direction.

[0171]

[0172] in, is the current velocity vector of the drone. The normalize function is used to normalize the vector to the appropriate velocity magnitude.

[0173] Perform obstacle avoidance maneuvers and adjust the drone's path.

[0174] s new =update_state(s,a)

[0175] Among them, s new It is the new state after performing the obstacle avoidance action.

[0176] S40. After allocating the inspection sub-area to each drone, the drone inspection command and dispatch center plans an inspection path for each drone to achieve the shortest inspection path in a single area; (see also Figure 3 Option-A*-DQN algorithm cycle reward diagram)

[0177] The research goal of the offshore wind power scenario model is to allow the drone to inspect all wind turbines in the mission in the shortest possible time and ensure that the remaining power of the drone is always above zero during the flight. i represents the i-th wind turbine, i∈N, N={1,2,…,N}, d i Indicates inspection SN i The amount of data collected, τ i Indicates collection d i The unit time required.

[0178] Assume that the total amount of data collected by the drone is D total , the total flight time is T units of time, the UAV Whether to collect SN at all times i By x t,i Indicates that x t,i ∈{0,1}, where 1 represents collection and 0 represents no collection. Since the drone needs to inspect all wind turbines in the mission, and each wind turbine only inspects τ i Unit time, we can get

[0179]

[0180] Among them, for the second formula, the specific meaning is: each fan i needs τ i Unit time for inspection, x t,i is whether the UAV inspects the i-th wind turbine at time t. By summing t, we can get the total time for the UAV to inspect wind turbine i during the entire flight process: In order to inspect each fan, Should be equal to the time τ that fan i needs to be collected i ,Right now

[0181] Set the UAV flight strategy to μ, and the remaining power of the UAV at time t to e t , then the optimization problem can be expressed as

[0182] minT

[0183]

[0184]

[0185] In deep reinforcement learning, the interaction between the agent and the environment can be described by a Markov decision process (MDP). An MDP is a five-tuple<S,A,R,γ> , where S represents the state space, A represents the action space, P:S×A→ΔS represents the state transition probability matrix, R:S×A×S→R represents the immediate reward function, and γ∈[0,1] represents the discount factor. At any time t in the MDP, the current state of the environment observed by the agent is s t (s t ∈S), and selects action a according to the strategy μ t (a t ∈A), the corresponding reward given by the environment to the agent is r t (r t ∈R), and enter the next state s according to the state transition matrix P of the environment t+1 (s t+1 ∈S), and repeat the above operation until the end.

[0186] Option can be seen as a holistic abstraction of a series of actions with a common purpose, defined as follows:

[0187] option is a triple<I,π,β> . Where I is the set of states that can execute the option, π is the internal strategy, and β is the termination condition. On an MDP, if a set O consisting of options is defined, it will become a semi-Markov decision process (SMDP, Semi-Markov Decision Process), as follows:

[0188] (MDP+option=SMDP), for any MDP and any set O of options defined on this MDP, the decision process that only selects options from the set and executes this option until the end is SMDP. SMDP can be represented by a six-tuple, that is,<S,A,O,P,R,γ> , where O represents the option set, and the remaining elements have the same meaning as the corresponding elements in the MDP quintuple. In SMDP, the agent generally needs to learn hierarchically the "high-level" strategy μ for selecting options and the "low-level" strategy π within each option.

[0189] SMDP is used to model the UAV inspection task. The six-tuple of the model is expressed as<S,A,O,P,R,γ> , where the basic elements such as agent, state, action, option set and immediate reward are modeled as follows:

[0190] (1) Single agent: The drone can be regarded as a single agent, and its option selection strategy is μ. At the beginning of the mission, the drone has its own initial state s 0 Select an option, denoted as OP 0 , and according to OP 0 Internal strategy of 0 Perform a series of actions, in OP 0 At the end of ξ, the drone gets the total instant reward r for this option. 0 , observe its own state information s again ξ , and select the next option to be executed according to μ, recorded as OP ξ , and the cycle continues until the drone mission is completed.

[0191] (2) State: Assume that the drone can fully observe its own state, that is, the state s at each time t t =(cr t ,p t ,e t ), s t ∈S, where cr t is a vector consisting of the percentage of each wind turbine inspected at time t, that is, p t is the position vector of the drone at time t, i.e. p t =(x t ,y t ,z t ), since the height of the drone is a constant H, then z t =H;e t The remaining battery power of the drone.

[0192] (3) Action: The basic action set A of the drone contains three basic actions, namely, the action of flying in a straight line towards a target. f 、Charging action a c And inspection action a cr At each time t, perform action a f Make the drone fly to the specified target direction at a speed v 0 Fly in a straight line for one unit of time and perform action a c Make the drone fully charged and execute action a cr The amount of data collected by the drone inspection is C.

[0193] (4) Option set: The option set O executed by the drone includes inspecting a certain wind turbine SN i Data OP s,i (i∈N), return to charge OP c And end the mission OPp There are three options, namely O={OP s,1 ,OP s,2 …,OP s,N ,OP c ,OP p}. Each option is a triple OP ,π OP ,β OP >, the number of elements in option set O is |O|=N+2. The set of options that the drone can choose in any state is the entire option set O, that is, The strategy π within each option OP are all set to known fixed strategies, and the termination condition β of each option OP Specifically, for the inspection fan option OP s,i , its strategy is to fly straight from the current position to the wind turbine SN i And collect the data of the fan i , until the collection is completed, the current option will not be exited; for return charging OP c The strategy is to fly straight to the smart hangar and charge it, and then exit OP after it is fully charged. c ; For ending the mission OP p , whose strategy is for the drone to fly straight to the end point, inform the environment that the mission is over and exit OP p .

[0194] (5) Instant reward: The agent only receives the total instant reward r for each option at the end of each option. ξ . r ξ is the initial state of option s ξ and option action OP ξ The function of r ξ =r ξ (s ξ ,OP ξ ). Set the total instant reward of an option to be divided into power rewards Inspection Rewards and path rewards Among the three parts, the power reward is mainly used to punish the drone for running out of power during the execution of the option, that is,

[0195]

[0196] Among them, N e ​It is a negative constant. When the power is less than or equal to the minimum power B, a larger penalty value is given to the drone to force it to return. The minimum power B is the power required to support the drone to return and charge.

[0197] The inspection reward is used to punish the drone for repeatedly selecting the wind turbine option that has already been inspected.

[0198]

[0199] Among them, N c is a negative constant, Indicates the fans that have been inspected.

[0200] The path reward The distance l that the drone has flown within this option ξ It is inversely proportional to the speed of the UAV, which is used to guide the UAV to learn to fly the shortest possible path to inspect the wind turbine, which can be expressed as

[0201]

[0202] Among them, N l is a negative constant. Finally, the drone gets an immediate reward r after experiencing an option. ξ is the sum of the above three rewards, expressed as

[0203]

[0204] The state transfer matrix P in the SMDP model describes the dynamic rules of the environment in this scenario. It defines the probability distribution of the next state s′ of the drone when it takes action a in any state s:

[0205] Similar to the state action value function Q in MDP π (s,a), in the SMDP model, define the state-option value function To represent the state-option tuple (s, OP) under the strategy μ and option set O, the expected cumulative return can be obtained as

[0206]

[0207] Among them, γ∈[0,1] represents the discount factor of the cumulative immediate reward, Satisfies the Bellman Expectation Equation. In order to solve the SMDP model, it is necessary to find the optimal state-option value function It should satisfy the Bellman Optimality Equation, as shown below. In order to solve the Bellman Optimality Equation, Monte Carlo algorithm, Temporal Difference (TD) algorithm, etc. can be used for solution.

[0208]

[0209] The option-A*-DQN algorithm execution architecture shows the reasoning process during algorithm execution. The interaction between the drone and the environment starts from the previous option OP ξ-1 Exit, and get the overall instant reward of this option is r ξ-1 And the next status information is s ξ ,s ξ Includes the percentage vector cr of data collected by all wind turbines currently ξ , the location information of the drone ξ And the remaining battery of the drone ξ Then the current s ξ The value function neural network Q input to the option-A*-DQN algorithm ot In the example, the network consists of an input layer, a hidden layer, and an output layer. The hidden layer consists of two fully connected layers. The first fully connected network contains 1024 neurons, and its activation function uses the ELU (Exponential Linear Unit) function. This activation function has an exponential decay characteristic in the negative domain, which can provide a smooth gradient and help accelerate the training process. The output of the first layer of the network can be expressed as

[0210]

[0211] Among them, W 1 is the weight parameter of the first layer of neural network, b 1 is its bias parameter. The input of the second hidden layer is the output of the first hidden layer. The second hidden layer consists of 300 neurons. Its activation function is the same as the previous layer, and it also uses the ELU function. The output of this layer can be expressed as

[0212]

[0213] Among them, W 2 and b 2 are the weight parameters and bias parameters of the second layer network respectively. The output layer receives the output X of the second layer network 1 , and use the Softmax activation function to output the |O|-dimensional vector q as

[0214]

[0215] Among them, W 3 and b 3 are the weight parameter and bias parameter of the output layer respectively. Softmax is a normalized exponential function that can convert the output of the neural network into a probability distribution.

[0216] Finally, the value function neural network Q ot The output q=(q 1 ,q 2 ,…,q |O| ) is the probability of choosing each option, that is

[0217] The best option is found by using the ε-greedy algorithm. ε is a small value between 0 and 1. Each time, a random selection is made with a probability of ε, and a greedy algorithm is used to select with a probability of 1-ε, that is, the index of the largest value in q is selected as the option OP to be selected. ξ , the greedy algorithm can be expressed as

[0218] OP ξ = arg max(q j )

[0219] Finally, select OP in option set O ξ The corresponding strategy π ξ and the termination condition β ξ And output corresponding actions to continue interacting with the environment.

[0220] The training algorithm of option-A*-DQN is as follows:

[0221] When training the option-A*-DQN algorithm, set the drone experience memory Among them, e k is the kth experience vector, that is Indicates the current state. Indicates the option action obtained according to the current option-A*-DQN algorithm. is (s′ k ,o′ k ) to get the overall immediate feedback, and o′ k represents the next state and action of the drone after interacting with the environment, and D c Represents the maximum storage capacity of the memory. The value function neural network Q is trained by experience replay and experience random extraction ot, which can break the correlation between data and make the data satisfy the characteristics of independent and identical distribution as much as possible, thereby enhancing the stability of training. ot Also known as the evaluation network, in addition, set the target network Used to approximate the optimal evaluation network Right now The loss function for evaluating the network can be expressed as

[0222]

[0223] Among them, θ represents the value function neural network Q ot The update rule for all parameters in is

[0224]

[0225] Among them, α is the learning rate, θ New and θ Old Respectively represent the parameters after evaluation network update and the parameters before update, the gradient of loss function It can be expressed as

[0226]

[0227] In order to obtain the overall immediate feedback, it means that the target network is updated by Soft-Update after interacting with the environment, that is, after a certain period of time, the parameters of the original target network and the current estimated network are used to update the target network together. The update rule is

[0228] θ Target,New =βθ Target,Old +(1-β)θ

[0229] Where β is the update rate, and β∈[0,1], θ Target,New and θ Target,Old Represent the target network The updated parameters and the parameters before and after the update, using the Soft-Update method for the target network can increase the robustness of the neural network training.

[0230] The specific steps of step S50 are as follows:

[0231] S51. When the drone is performing an inspection task, if it detects that the fluctuation range of the wind speed and wind force at sea exceeds the threshold, the change information will be fed back. The drone inspection command and dispatch center will recalculate the cost of the path to ensure that each step of the inspection path is optimal;

[0232] S52. The UAV inspection command and dispatch center monitors the endurance status of each UAV in real time. If the endurance is found to be insufficient, the UAV will be recalled for charging. The center will also make dynamic adjustments based on the remaining energy consumption status of each UAV, the distance between UAVs, and the workload to generate a new inspection area allocation plan.

[0233] S53. If the inspection task is completed or the meteorological factors such as wind speed and wind force change significantly, the drone will be recalled and the inspection path length and time will be returned.

[0234] A device for multi-UAV coordinated offshore wind turbine inspection path planning, applied to the above inspection path planning method, comprising a UAV inspection device and a UAV inspection command and dispatch center;

[0235] The drone inspection device includes an industry-level flight platform drone, and ensures that the drone has appropriate flight performance, endurance and carrying capacity. The drone is also equipped with various equipment, such as a full-frame high-resolution gimbal camera, infrared sensor, multi-spectral sensor, radar, etc., for detecting and locating targets, and obtaining sea conditions and environmental information.

[0236] The UAV inspection command and dispatch center is responsible for formulating inspection tasks, planning routes, and assigning tasks to UAVs. It can also receive and analyze data from UAVs. It is the hub of the entire system and monitors and dispatches tasks in real time.

[0237] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0238] 1. The present invention constructs a wind speed model according to the wind speed information of the area to be inspected, updates the marine environment information in real time, and reasonably divides the area to be inspected, so that the UAV can choose a flight path that is more suitable for the current environment, which helps to improve the stability and safety of the UAV flight.

[0239] 2. The present invention adopts regional division in the coordinated inspection of multiple drones at sea to avoid the duplication of inspection paths of multiple drones in traditional methods, which leads to problems such as low task completion rate, and improves the efficiency of drone inspection.

[0240] 3. The present invention considers balancing the load of drones when allocating sub-areas to drones, thereby solving the problem of low efficiency caused by multiple round trips of drones due to insufficient endurance in traditional methods, and at the same time reduces the number of drones performing inspection tasks as much as possible to reduce costs.

[0241] 4. Through simulation experiments of multi-agent path planning, the present invention verifies that the option-A*-DQN algorithm is significantly superior to traditional algorithms in terms of path planning efficiency, convergence speed, task completion rate, goal achievement rate and ability to adapt to dynamic environments, proving its high efficiency and robustness in complex environments.

[0242] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for multi-UAV coordinated offshore wind turbine inspection path planning, characterized in that: The following steps are involved: S10. Acquisition and analysis of wind speed information in the offshore wind turbine inspection area, establishment of a synthetic speed model using the wind speed information, and evaluation and calculation of the inspection flight time of the UAV between different wind turbines based on the synthetic speed model; S20. Generate global information based on the prior knowledge generated by the A* algorithm, and store the prior knowledge generated by exploring the unknown environment in the prior replay pool; S30. The UAV inspection command and dispatch center allocates an inspection sub-area to each UAV; S40. After allocating the inspection sub-area to each drone, the drone inspection command and dispatch center plans an inspection path for each drone to achieve the shortest inspection path in a single area; S50. Monitor the endurance status of each drone during the inspection mission, and provide feedback on the inspection route length and time after recalling the drone.

2. The multi-UAV collaborative offshore wind turbine inspection path planning method according to claim 1 is characterized by: In step S10, The acquisition and analysis of the wind speed information is specifically as follows: the wind direction in the polar coordinate system of the wind speed information is expressed as: in, is the wind direction in the polar coordinate system, w=[w x ,w y ] is wind speed, w x and w y are the projections of wind speed on the x-axis and y-axis respectively; in the polar coordinate system, the phase is expressed in the counterclockwise direction.

3. The multi-UAV collaborative offshore wind turbine inspection path planning method according to claim 1 is characterized by: In step S10, The synthetic speed model, specifically the relationship between the synthetic speed of the drone, the drone speed and the wind speed, is as follows: in, is the composite speed of UAV U when it flies from wind turbine k1 to wind turbine k2, is the speed of the drone U when it flies from wind turbine k1 to wind turbine k2, and w is the wind speed; The evaluation and calculation of the inspection flight time of the UAV between different wind turbines is performed using the following specific evaluation calculation formula: in, is the coordinate of the k1th wind turbine, is the coordinate of the k2th wind turbine, express Euclidean norm of a vector; is the composite speed of UAV U when it flies from wind turbine k1 to wind turbine k2; is the inspection flight time of UAV U from wind turbine k1 to wind turbine k2.

4. The multi-UAV collaborative offshore wind turbine inspection path planning method according to claim 1 is characterized by: In step S20: the global information is generated using the option-A*-DQN algorithm.

5. The multi-UAV collaborative offshore wind turbine inspection path planning method according to claim 1 is characterized by: Step S20 specifically includes the following steps: S21. Generate prior knowledge based on A* algorithm; S22. Using the prior knowledge of exploration, the option-A*-DQN algorithm generates global information, designs dynamic weight parameters, and uses prior Q values ​​to optimize the multi-agent Q value function; S23. Explore the unknown environment to generate prior knowledge and store it in the prior replay pool D*; In step S21, the A* algorithm evaluates the potential cost of each node through a heuristic function; the scoring function of the A* algorithm is: f(n)=g(n)+h(n); Where f(n) is the total score of node n, g(n) is the actual path cost from the starting point to node n, and h(n) is the heuristic estimated cost from node n to the target node.

6. The multi-UAV coordinated offshore wind turbine inspection path planning method according to claim 1 is characterized by: Step S30 specifically includes the following steps: S31. Use the K-Means algorithm to divide the drone inspection area, and use the option-A*-DQN algorithm to generate inspection paths for the divided sub-areas; S32. When a drone is conducting an inspection and a dynamic obstacle approaches, the drone inspection device, i.e. the sensor carried by the drone, senses the distance between the drone and the dynamic obstacle. If it is found that the distance exceeds the safe distance, it will trigger automatic obstacle avoidance and adjust the path.

7. The multi-UAV coordinated offshore wind turbine inspection path planning method according to claim 6 is characterized by: In step S31, the K-Means algorithm divides the drone inspection area, specifically: minimizing the following objective function: Among them, ρ is the number of clusters, corresponding to the number of divisions of the inspection fan range, σ i is the i-th cluster, containing all wind turbines belonging to this cluster, is the centroid of the ith cluster, usually the mean of all points in the cluster, g is the coordinate of a wind turbine in the cluster, is the wind turbine coordinate g to the cluster centroid The square of the Euclidean distance; In step S31, multi-scale clustering is used to improve the K-Means algorithm; The multi-scale clustering described above first performs coarse-grained clustering, which can divide adjacent wind turbines into larger coarse-grained clusters, and then performs fine-grained clustering within the coarse-grained clusters; The coarse-grained clustering: using a larger distance threshold for preliminary clustering, C j ={x|d(x,m j )<=T_coarse} Among them, C j is the jth cluster, m j It is cluster C j The centroid of , T_coarse is the threshold of coarse-grained clustering; The fine-grained clustering: fine-grained clustering is performed within each coarse-grained cluster. C ij ={x|d(x,m ij )<=T_fine}] Among them, C ij is the i-th fine-grained cluster within the j-th coarse-grained cluster, m ij It is cluster C ij is the centroid of , and T_fine is the threshold for fine-grained clustering.

8. The multi-UAV coordinated offshore wind turbine inspection path planning method according to claim 1 is characterized by: In step S40: The semi-Markov decision process is used to model the UAV inspection task. The six-tuple of the model is expressed as<S,A,O,P,R,γ> , where S represents the state space, A represents the action space, O represents the option set, P is S×A→ΔS, which represents the state transition probability matrix, R is S×A×S→R, which represents the immediate reward function, and γ∈[0,1] represents the discount factor; the basic elements of the agent, state, action, option set, and immediate reward are modeled.

9. The multi-UAV coordinated offshore wind turbine inspection path planning method according to claim 1 is characterized by: Step S50 specifically includes: S51. When the drone performs an inspection task, if it detects that the fluctuation range of the wind speed and wind force at sea exceeds the threshold, the change information is fed back; the drone inspection command and dispatch center recalculates the cost of the path; S52. The UAV inspection command and dispatch center monitors the endurance status of each UAV in real time. If the endurance is insufficient, the UAV is recalled for charging. The center also makes dynamic adjustments based on the remaining energy consumption of each UAV, the distance between UAVs, and the workload, and generates a new inspection area allocation plan. S53. If the inspection task is completed or the meteorological factors of wind speed and wind force change significantly, the drone will be recalled and the inspection path length and time will be fed back.

10. A device for collaborative inspection path planning of multiple UAVs for offshore wind turbines, applied to the inspection path planning method according to any one of claims 1 to 9, characterized in that: Including drone inspection devices and drone inspection command and dispatch center; The UAV inspection device includes an industry-level flight platform UAV, and ensures that the UAV has appropriate flight performance, endurance and carrying capacity, and can be used to detect and locate targets, and obtain sea conditions and environmental information; The UAV inspection command and dispatch center is responsible for formulating inspection tasks, planning routes, and assigning tasks to UAVs. It can also receive and analyze data from UAVs. The industry-grade flying platform drone is equipped with a full-frame high-resolution gimbal camera, infrared sensor, multi-spectral sensor and radar, which are used to detect and locate targets, and obtain sea conditions and environmental information.

Images