A method and system for multi-UAV inspection trajectories and mobile energy storage terminal scheduling and inspection

By optimizing the UAV inspection trajectory using the DDQN algorithm with a multi-head self-attention mechanism and optimizing the charging schedule using the bidirectional heuristic A* algorithm for mobile energy storage, the problems of endurance, obstacle avoidance, and trajectory optimization in UAV inspection are solved, thereby improving inspection efficiency and safety.

CN119937625BActive Publication Date: 2025-11-14STATE GRID HUBEI ELECTRIC POWER INFORMATION & TELECOMMUNICATION COMPANY +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411890637.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-20
Publication Date
2025-11-14
Estimated Expiration
2044-12-20

AI Technical Summary

Technical Problem

Existing drone inspection technology faces problems such as insufficient endurance, the need for a large amount of human intervention, difficulty in autonomous obstacle avoidance and trajectory optimization, and low efficiency, especially in large-scale inspection tasks.

Method used

The DDQN algorithm with multi-head self-attention mechanism is used to optimize the inspection trajectory of UAVs, and the bidirectional heuristic A* algorithm of mobile energy storage terminal is combined for charging scheduling. The flight speed is selected by greedy algorithm and ε-greedy strategy, and the inspection points are allocated by density-aware K-Means++ algorithm.

Benefits of technology

It improves the efficiency and safety of drone inspections, reduces human intervention, optimizes trajectory planning, shortens inspection time, and enhances the stability and solution efficiency of the algorithm.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119937625B_ABST
    Figure CN119937625B_ABST
Patent Text Reader

Abstract

This invention proposes a method and system for multi-UAV inspection trajectories and mobile energy storage terminal scheduling inspections. The method first acquires the location information of each UAV's inspection points, then allocates inspection points to each UAV based on the distribution of inspection points and the number of UAVs. Finally, based on the inspection tasks of each UAV, it optimizes the inspection trajectories of each UAV using a multi-head self-attention mechanism (DDQN) algorithm and optimizes the charging scheduling strategy for the mobile energy storage terminal. This invention not only improves the inspection efficiency of UAVs and reduces the total inspection time, but also enhances the safety of UAVs during the inspection process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) inspection technology, specifically relating to a method and system for multi-UAV inspection trajectories and mobile energy storage terminal scheduling and inspection. Background Technology

[0002] With the rapid development of China's economy, various inspection tasks have emerged in the industrial sector. Currently, most areas still rely primarily on manual inspections, supplemented by machine-assisted inspections. Traditional manual inspections have several significant drawbacks when facing large, dispersed, or hazardous environments. First, manual inspections typically require a large amount of human resources, especially in vast or inaccessible areas (such as high-voltage power lines, oil pipelines, and wind power facilities). A large number of inspection personnel means high labor costs and time investment, while also making it difficult to respond quickly to emergencies. Second, manual inspections are slow, limited by human physical strength, working hours, and movement speed. Especially in areas with complex terrain or harsh conditions, inspection personnel cannot quickly cover large areas, resulting in long inspection cycles and hindering the timely detection and handling of problems. Furthermore, in the event of equipment failure or emergencies, traditional inspection methods struggle to quickly locate problems and respond. The long inspection cycle often means that problems are only discovered during the next inspection, delaying troubleshooting and potentially leading to more serious consequences. Furthermore, in extreme weather (such as strong winds, heavy rain, heavy snow, etc.) or harsh terrain (such as mountains, deserts, etc.), the difficulty and risk of manual inspection work increase significantly, and sometimes it is even impossible to carry out, which limits the coverage and continuity of inspection.

[0003] Drone inspection technology has gained increasing attention across various industries in recent years, particularly in power, petroleum, agriculture, and infrastructure, demonstrating significant application potential due to its flexibility, efficiency, and intelligence. Drones can rapidly cover large areas in a short time, making them especially suitable for inspecting widely distributed infrastructure such as power lines, oil pipelines, and wind power equipment. Drone inspection enables real-time data collection, significantly reducing inspection time and eliminating the need for manual physical inspections of hazardous areas. Furthermore, drones can replace humans in entering dangerous or inaccessible environments, such as high-altitude, high-pressure, and highly corrosive areas. This effectively avoids human contact with hazardous facilities and environments, reducing accidents and safety hazards during inspections and ensuring the safety of inspection personnel. Moreover, drones are equipped with advanced devices such as high-resolution cameras, infrared sensors, thermal imagers, and lidar, enabling them to capture more precise images and data. Through high-definition images, videos, 3D modeling, and sensor data, drones can generate detailed inspection reports in real time, improving the comprehensiveness and accuracy of data collection. However, the process of drone-based inspection also faces several challenges that need to be addressed.

[0004] Firstly, one of the challenges faced by drones in actual inspections is their battery life. Most commercial drones have limited battery life, typically between 30 minutes and 1 hour. This short flight time restricts their coverage area, especially in large-scale inspection tasks. Secondly, although drones possess autonomous flight capabilities, many inspection tasks still require significant human intervention. For example, drones need to avoid obstacles and no-fly zones during inspections to ensure flight safety. Therefore, enabling drones to autonomously identify and avoid obstacles / no-fly zones and optimizing their inspection trajectories are key to improving drone inspection efficiency. Furthermore, the complexity of the algorithm for planning drone trajectories increases exponentially with the number of inspection points. Therefore, rationally designing multiple drone trajectories based on the distribution of inspection points is also crucial for improving inspection efficiency. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems in the existing technology by providing a method and system for multi-UAV inspection trajectories and mobile energy storage terminal scheduling and inspection.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] In a first aspect, this invention proposes a method for multi-UAV inspection trajectories and mobile energy storage terminal scheduling inspection, including:

[0008] S1. Obtain the location information of each drone inspection point;

[0009] S2. Assign inspection points to each drone based on the distribution of inspection points and the number of drones.

[0010] S3. Based on the inspection points of each UAV, the DDQN algorithm based on the multi-head self-attention mechanism is used to optimize the inspection trajectory of each UAV and optimize the charging scheduling strategy of the mobile energy storage terminal.

[0011] S3 includes:

[0012] S31. Determine the optimal inspection point for the UAV based on a greedy algorithm;

[0013] S32. Select the UAV's flight speed v according to the ε-greedy strategy in the current time slot. n And calculate the reward r for the current time slot according to the following formula. n :

[0014]

[0015] In the above formula, r a As a punishment for drones flying into no-fly zones, d o,n d represents the distance between the drone's current location and the inspection point. o,n+1Let r be the distance between the next time slot location of the UAV and the inspection point, η be a constant for reducing the UAV inspection time slot, and r be the distance between the next time slot location of the UAV and the inspection point. a Both η and η are negative constants;

[0016] S33, the result of the transfer (q) n ,v n ,r n ,q n+1 Saved to the experience pool, where q n q n+1 These represent the drone's position in the current time slot and the next time slot, respectively.

[0017] S34. Randomly select N1 samples from the experience pool and use gradient descent to reduce the loss of the neural network, thereby optimizing the inspection trajectory of the UAV and obtaining a greater reward. Finally, the optimal inspection trajectory of each UAV is obtained.

[0018] The loss function is:

[0019]

[0020] In the above formula, λ is the discount factor, Q(q) n ,v n |θ) represents the location q of the drone in the current dueling network. n Take action v n Q value, For the target dueling network, the drone is at position q n+1 Take action Q value, θ, θ * Factors that influence the parameters of a neural network model;

[0021] The dueling network introduces a multi-head self-attention mechanism to enhance the state value function and the advantage function. The enhanced Q function is:

[0022]

[0023] V(q)=f V (h att (q))

[0024] A(q,v)=f A (h att (q),v)

[0025]

[0026] Q i =W Q h(q)

[0027] K j =WK h(q)

[0028] V j =W V h(q)

[0029] In the above formula, V(q) and A(q,v) are the state value function and advantage function, respectively; q and v are the location and action of the UAV, respectively; B is the number of selectable actions; f V h used to enhance the state features of attention att (q) Calculate the state value V(q), h att (q) represents the state features after attention enhancement, f A The advantage function A(q,v) is used to calculate the advantage function from the state features after attention enhancement. i,j For attention weights, V j Q is a value vector. i For query vector, Let be the transpose of the key vector, i be the index of the query vector, j be the indices of the key and value vectors, and d be the transpose of the key vector. k W represents the dimension of the key vector. Q W is the linear transformation matrix that maps h(q) to the query vector space. K W is the linear transformation matrix that maps h(q) to the bond vector space. V h(q) is the linear transformation matrix that maps h(q) to the value vector space, where h(q) is the eigenvector.

[0030] S35. Determine whether the current battery energy of the drone is less than the set threshold. If it is less, the mobile energy storage terminal charges the drone according to the charging scheduling strategy. If it is not less, proceed to S36.

[0031] S36. Determine whether the drone has completed the inspection task of the inspection point. If not, return to S32; if completed, proceed to S37.

[0032] S37. Determine whether the drone has completed the inspection tasks of all inspection points. If not, return to S31; if completed, proceed to S38.

[0033] S38. Determine if the maximum number of iterations has been reached. If not, return to S31 to optimize the inspection trajectory of the next drone.

[0034] S31 includes:

[0035] S311. The drone assigns numbers to all the inspection points it is assigned.

[0036] S312. Calculate the distance between the current position of the drone and each unfinished inspection point, delete the points that have been inspected, and sort the remaining inspection points in descending order of distance.

[0037] S313. Determine whether the inspection point with the smallest distance has been inspected. If it has been inspected, return to S312 for the next loop calculation. If it has not been inspected, output the number of the inspection point as the next target.

[0038] The charging scheduling strategy for the mobile energy storage terminal is determined using a bidirectional heuristic A* algorithm, specifically including:

[0039] S351. Initialize open_list and closed_list, and set the starting point w of the mobile energy storage terminal. s Add to open_list;

[0040] S352. Simultaneously perform forward and backward searches. The forward search starts from the starting point and searches for a path to the target, while the backward search starts from the target and searches for a path to the starting point. Calculate the F-value of each node in the open_list and select the node with the smallest F-value as the current node w. c The F-value of each node is calculated using the following formula:

[0041] F = f forward (b)+f backward (b)

[0042] f forward (b) = g start (b)+h goal (b)

[0043] f backward (b) = g goal (b)+h start (b)

[0044] In the above formula, f forward (b), f backward (b) represents the expected total path cost for forward and reverse searches, respectively, g. start (b) is from the starting point w s The actual path cost to the current node b, h goal (b) is the route from the current node b to the target node w. g The estimated cost, g goal (b) represents the total path cost of the reverse search from the target to the current node b, h. start (b) is to return from the current node b to the starting point w. s heuristic value;

[0045] S353, Move the current node w cMove it from open_list to closed_list;

[0046] S354. Determine the current node w c If a neighboring node is a new node or a shorter path has been found, then add it to the open_list.

[0047] S355, Determine the current node w c Is it the target node w? g If open_list is empty, output the optimal path for the mobile energy storage terminal, i.e., the charging scheduling scheme for the mobile energy storage terminal; otherwise, return S352.

[0048] S2 uses a density-aware K-Means++ algorithm to allocate inspection points for each UAV, specifically including:

[0049] S21. Selecting initial cluster centers based on distance and density weights, including:

[0050] S211. Randomly select one inspection point from all inspection points as the first cluster center;

[0051] S212. Calculate the distance D(w) between each of the other inspection points and the cluster center. i ), and the density weights ρ(w) of each other inspection point. i );

[0052] S213. Calculate the probability of each other inspection point being selected as the next cluster center based on distance and density weights, and select the inspection point with the highest probability value as the next cluster center:

[0053]

[0054] In the above formula, L(w) i ) represents the i-th inspection point w i The probability of being selected as the next cluster center, D(w) i ) for w i Distance from the cluster center, ρ(w) i ) for w i The density weight, where I is the number of inspection points;

[0055] S214. Determine whether the number of cluster centers has reached the target value, i.e. the number of drones. If it has, obtain the initial cluster centers; if not, return to S212 for the next round of screening.

[0056] S22. Assign each inspection point to the nearest cluster center;

[0057] S23. Update cluster centers based on density weights:

[0058]

[0059] In the above formula, For the updated j-th cluster center, S j For all inspection points belonging to the j-th cluster center;

[0060] S24. Repeat S22-S23 in a loop until the iteration termination condition is met.

[0061] In S212, D(w) i The following formula is used to calculate:

[0062]

[0063] In the above formula, d(w) i ,e j ) represents the i-th inspection point w i To the j-th cluster center e j The Euclidean distance;

[0064] ρ(w i The following formula is used to calculate:

[0065]

[0066] In the above formula, δ is a small positive number.

[0067] Secondly, this invention proposes a multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection system, including an information acquisition module, an inspection point allocation module, an inspection trajectory optimization module, and a mobile energy storage terminal charging scheduling module;

[0068] The information acquisition module is used to acquire the location information of each UAV inspection point;

[0069] The inspection point allocation module is used to allocate inspection points to each drone based on the distribution of inspection points and the number of drones.

[0070] The inspection trajectory optimization module is used to optimize the inspection trajectory of each UAV based on the inspection points of each UAV using the DDQN algorithm based on the multi-head self-attention mechanism.

[0071] The mobile energy storage terminal charging scheduling module is used to optimize and determine the charging scheduling strategy of the mobile energy storage terminal and control the mobile energy storage terminal to charge the drone.

[0072] The inspection trajectory optimization module optimizes the inspection trajectory of each UAV according to the following steps:

[0073] A1. Determine the optimal inspection point for the UAV based on a greedy algorithm;

[0074] A2. Select the UAV's flight speed v according to the ε-greedy strategy in the current time slot. n And calculate the reward r for the current time slot according to the following formula. n :

[0075]

[0076] In the above formula, r a As a punishment for drones flying into no-fly zones, d o,n d represents the distance between the drone's current location and the inspection point. o,n+1 Let r be the distance between the next time slot location of the UAV and the inspection point, η be a constant for reducing the UAV inspection time slot, and r be the distance between the next time slot location of the UAV and the inspection point. a Both η and η are negative constants;

[0077] A3. The result of the transfer (q) n ,v n ,r n ,q n+1 Saved to the experience pool, where q n q n+1 These represent the drone's position in the current time slot and the next time slot, respectively.

[0078] A4. Randomly select N1 samples from the experience pool and use gradient descent to reduce the loss of the neural network, thereby optimizing the inspection trajectory of the UAV and obtaining a larger reward. Finally, the optimal inspection trajectory for each UAV is obtained.

[0079] The loss function is:

[0080]

[0081] In the above formula, λ is the discount factor, Q(q) n ,v n |θ) represents the location q of the drone in the current dueling network. n Take action v n Q value, For the target dueling network, the drone is at position q n+1 Take action Q value, θ, θ * Factors that influence the parameters of a neural network model;

[0082] The dueling network introduces a multi-head self-attention mechanism to enhance the state value function and the advantage function. The enhanced Q function is:

[0083]

[0084] V(q)=f V (hatt (q))

[0085] A(q,v)=f A (h att (q),v)

[0086]

[0087] Q i =W Q h(q)

[0088] K j =W K h(q)

[0089] V j =W V h(q)

[0090] In the above formula, V(q) and A(q,v) are the state value function and advantage function, respectively; q and v are the location and action of the UAV, respectively; B is the number of selectable actions; f V h used to enhance the state features of attention att (q) Calculate the state value V(q), h att (q) represents the state features after attention enhancement, f A The advantage function A(q,v) is used to calculate the advantage function from the state features after attention enhancement. i,j For attention weights, V j Q is a value vector. i For query vector, Let be the transpose of the key vector, i be the index of the query vector, j be the indices of the key and value vectors, and d be the transpose of the key vector. k W represents the dimension of the key vector. Q W is the linear transformation matrix that maps h(q) to the query vector space. K W is the linear transformation matrix that maps h(q) to the bond vector space. V h(q) is the linear transformation matrix that maps h(q) to the value vector space, where h(q) is the eigenvector.

[0091] A5. Determine if the current battery energy of the drone is less than the set threshold. If it is less, the mobile energy storage terminal charges the drone according to the charging scheduling strategy. If it is not less, proceed to A6.

[0092] A6. Determine whether the drone has completed the inspection task for this inspection point. If not, return to A2; if completed, proceed to A7.

[0093] A7. Determine whether the drone has completed the inspection tasks of all inspection points. If not, return to A1; if completed, proceed to A8.

[0094] A8. Determine if the maximum number of iterations has been reached. If not, return to A1 to optimize the inspection trajectory of the next drone.

[0095] The mobile energy storage terminal charging scheduling module uses a bidirectional heuristic A* algorithm to optimize and determine the charging scheduling strategy for the mobile energy storage terminal. The algorithm's process includes:

[0096] B1. Initialize open_list and closed_list, and set the starting point w of the mobile energy storage terminal. s Add to open_list;

[0097] B2. Simultaneously perform forward and backward searches. The forward search starts from the starting point and searches for a path to the target, while the backward search starts from the target and searches for a path to the starting point. Calculate the F-value of each node in the open_list and select the node with the smallest F-value as the current node w. c The F-value of each node is calculated using the following formula:

[0098] F = f forward (b)+f backward (b)

[0099] f forward (b) = g start (b)+h goal (b)

[0100] f backward (b) = g goal (b)+h start (b)

[0101] In the above formula, f forward (b), f backward (b) represents the expected total path cost for forward and reverse searches, respectively, g. start (b) is from the starting point w s The actual path cost to the current node b, h goal (b) is the route from the current node b to the target node w. g The estimated cost, g goal (b) represents the total path cost of the reverse search from the target to the current node b, h. start (b) is to return from the current node b to the starting point w. s heuristic value;

[0102] B3. Move the current node w c Move it from open_list to closed_list;

[0103] B4. Determine the current node w cIf a neighboring node is a new node or a shorter path has been found, then add it to the open_list.

[0104] B5. Determine the current node w c Is it the target node w? g If open_list is empty, output the optimal path for the mobile energy storage terminal, i.e., the charging scheduling scheme for the mobile energy storage terminal; otherwise, return B2.

[0105] The inspection point allocation module uses a density-aware K-Means++ algorithm to allocate inspection points for each UAV. The algorithm's process includes:

[0106] C1. Initial cluster centers are selected based on distance and density weights, including:

[0107] C11. Randomly select one inspection point from all inspection points as the first cluster center;

[0108] C12. Calculate the distance D(w) between each of the other inspection points and the cluster center using the following formula. i ) and the density weights ρ(w) of each other inspection point i ):

[0109]

[0110] In the above formula, d(w) i ,e j ) represents the i-th inspection point w i To the j-th cluster center e j The Euclidean distance, δ is a small positive number, and I is the number of inspection points;

[0111] C13. Calculate the probability of each other inspection point being selected as the next cluster center based on distance and density weights, and select the inspection point with the highest probability value as the next cluster center:

[0112]

[0113] In the above formula, L(w) i ) represents the i-th inspection point w i The probability of being selected as the next cluster center, D(w) i ) for w i Distance from the cluster center, ρ(w) i ) for w i Density weights;

[0114] C14. Determine whether the number of cluster centers has reached the target value, i.e., the number of drones. If it has, obtain the initial cluster centers; if not, return to S212 for the next round of screening.

[0115] C2. Assign each inspection point to the nearest cluster center;

[0116] C3. Update cluster centers based on density weights:

[0117]

[0118] In the above formula, For the updated j-th cluster center, S j For all inspection points belonging to the j-th cluster center;

[0119] C4. Repeat C2-C3 in a loop until the iteration termination condition is met.

[0120] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0121] 1. This invention discloses a multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection method. First, it acquires the location information of each UAV inspection point. Then, it allocates inspection points to each UAV based on the distribution of inspection points and the number of UAVs. Finally, based on the inspection tasks of each UAV, it optimizes the inspection trajectory of each UAV using a DDQN algorithm based on a multi-head self-attention mechanism and optimizes the charging scheduling strategy of the mobile energy storage terminal. On the one hand, this method reduces task complexity by allocating inspection tasks. By designing the UAV inspection trajectory based on reinforcement learning and combining it with the charging scheduling of the mobile energy storage terminal, it not only improves the inspection efficiency of the UAVs and reduces the total inspection time, but also enhances the safety of the UAVs during the inspection process (efficiently avoiding obstacles / no-fly zones). On the other hand, by introducing a multi-head self-attention mechanism into the DDQN algorithm, this method effectively enhances the dueling network's attention to key states and actions, improving decision-making ability. Simultaneously, it considers no-fly zones, making it more consistent with the actual scenario of UAV inspection.

[0122] 2. The present invention provides a multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection method. It adopts a bidirectional heuristic A* algorithm to optimize and determine the charging scheduling scheme of the mobile energy storage terminal. Compared with the traditional A* algorithm, this algorithm searches from two directions simultaneously, namely forward search and reverse search. The forward search starts from the starting point and finds the path to the target, while the reverse search starts from the target and finds the path to the starting point. Based on this algorithm, the charging scheduling optimization of the mobile energy storage terminal can reduce the inspection time of UAVs, accelerate the convergence speed of the algorithm, and improve the stability of the algorithm.

[0123] 3. The present invention provides a multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection method. It uses the density-aware K-Means++ algorithm to allocate inspection points for each UAV. This algorithm considers both distance and density weights for cluster center selection, which can prevent getting trapped in local optima and improve the solution efficiency of the algorithm. Attached Figure Description

[0124] Figure 1 This is an overall flowchart of the method described in Example 1.

[0125] Figure 2 This is a flowchart of the inspection point allocation process for the UAV in Example 1.

[0126] Figure 3 This is a flowchart of the drone inspection trajectory optimization process in Example 1.

[0127] Figure 4 This is a flowchart of the mobile energy storage terminal scheduling optimization process in Example 1.

[0128] Figure 5 The image shows the simulation results.

[0129] Figure 6 This is a structural diagram of the system described in Example 2. Detailed Implementation

[0130] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0131] Example 1:

[0132] A method for multi-UAV inspection trajectories and mobile energy storage terminal scheduling inspection, such as Figure 1 As shown, the specific steps are as follows:

[0133] 1. The ground service center dispatches drones equipped with LiDAR and visual sensors to the inspection area to obtain inspection point cloud data. Based on the acquired point cloud data, the coordinates of inspection points and the locations of obstacles are obtained through feature extraction. The key to extracting inspection points is to find significant landmarks or markers and to distinguish between smooth surfaces and high-curvature points based on curvature calculations.

[0134] For the inspection point location w o curvature k o It can be estimated using the covariance matrix C of its neighborhood:

[0135]

[0136] In the above formula, λ min λ is the minimum value among the eigenvalues. i Let M be the i-th eigenvalue. For w o The mean of all points in the neighborhood, where O is the number of points to be selected.

[0137] Since points with high curvature are often located at edges and corners, this embodiment extracts the coordinates of key inspection points and the location of obstacles by setting a curvature threshold, i.e., k > ι.

[0138] 2. The ground service center uses a density-aware K-means++ algorithm to assign inspection tasks to each drone based on the distribution of inspection points and the number of drones. Figure 2 As shown, the algorithm's flow includes:

[0139] 2.1. Initial cluster centers are selected based on distance and density weights to ensure that the initial cluster centers are related to the density of inspection points, including:

[0140] 2.1.1 Randomly select one inspection point from all inspection points as the first cluster center.

[0141] 2.1.2 Calculate the distance D(w) between each of the other inspection points and the cluster center using the following formula. i ), and the density weights ρ(w) of each other inspection point. i ):

[0142]

[0143] In the above formula, d(w) i ,e j ) represents the i-th inspection point w i To the j-th cluster center e j The Euclidean distance, δ is a small positive number to avoid the denominator being zero, and I is the number of inspection points.

[0144] 2.1.3 Calculate the probability of each other inspection point being selected as the next cluster center based on distance and density weights, and select the inspection point with the highest probability value as the next cluster center:

[0145]

[0146] In the above formula, L(w) i ) represents the i-th inspection point w i The probability of being selected as the next cluster center, D(w) i ) for w i Distance from the cluster center, ρ(w) i ) for w i Density weights.

[0147] 2.1.4 Determine whether the number of cluster centers has reached the target value, i.e. the number of drones. If it has, the initial cluster centers are obtained; if not, return to 2.1.2 for the next round of screening.

[0148] 2.2 Assign each inspection point to the nearest cluster center.

[0149] 2.3 To optimize the inspection route of the UAV, the cluster centers are updated based on density weights:

[0150]

[0151] In the above formula, For the updated j-th cluster center, S j This refers to all inspection points belonging to the j-th cluster center.

[0152] 2.4 Repeat steps 2.2-2.3 until the cluster centers no longer change significantly, thus obtaining the inspection points for each UAV.

[0153] 3. Determine the optimal inspection point for the UAV based on a greedy algorithm. The specific process includes:

[0154] 3.1 The drone assigns numbers to all the inspection points it is assigned.

[0155] 3.2 Calculate the distance between the current position of the drone and each uncompleted inspection point, delete the points that have been inspected, and sort the remaining inspection points in descending order of distance.

[0156] 3.3 Determine whether the inspection point with the smallest distance has been inspected. If it has been inspected, return to S312 for the next loop calculation; if it has not been inspected, output the number of the inspection point as the next target.

[0157] 4. The drone is currently at location q. n The distance to the selected inspection point and the distance to the no-fly zone are sensed through sensor devices or flight control acquisition modules.

[0158] 5. The DDQN algorithm based on a multi-head self-attention mechanism is used to optimize the inspection trajectory of each UAV, such as... Figure 3 As shown, the algorithm's flow includes:

[0159] 5.1 Select the UAV's flight speed v according to the ε-greedy strategy in the current time slot. n It can also sense the distance between the next time slot and the no-fly zone, and the distance between the next time slot and the inspection point.

[0160]

[0161] In the above formula, q n+1 The location of the drone in the next time slot. The duration of a time slot.

[0162] 5.2 Calculate the reward r for the current time slot using the following formula. n :

[0163]

[0164] In the above formula, r aAs a punishment for drones flying into no-fly zones, d o,n d represents the distance between the drone's current location and the inspection point. o,n+1 Let r be the distance between the next time slot location of the UAV and the inspection point, η be a constant for reducing the UAV inspection time slot, and r be the distance between the next time slot location of the UAV and the inspection point. a Both η and η are negative constants.

[0165] 5.3. Transfer the result (q) n ,v n ,r n ,q n+1 Saved to the experience pool, where q n q n+1 These represent the drone's position in the current time slot and the next time slot, respectively.

[0166] 5.4 Randomly select N1 samples (i.e., samples from N1 time slots) from the experience pool, and use gradient descent to reduce the loss of the neural network, thereby optimizing the inspection trajectory of the UAV and obtaining a greater reward. Finally, the optimal inspection trajectory for each UAV is obtained.

[0167] The loss function is:

[0168]

[0169] In the above formula, λ is the discount factor, Q(q) n ,v n |θ) represents the location q of the drone in the current dueling network. n Take action v n Q value, For the target dueling network, the drone is at position q n+1 Take action Q value, θ, θ * These are factors that affect the parameters of a neural network model.

[0170] Dueling networks, by calculating state value and advantage function separately, make value estimation more stable. However, traditional structures may not be able to effectively focus on important features in complex environments. Therefore, this embodiment introduces a multi-head self-attention mechanism into the dueling network to enhance the state value function and advantage function, thereby improving the network's focus on key states and actions. The specific process includes:

[0171] First, a multi-head self-attention mechanism is used to extract key state features from the environment:

[0172]

[0173] After introducing a multi-head self-attention mechanism, the state q (i.e., the location q of the drone) will be... nGiven a convolutional or fully connected network, the extracted feature vector h(q) can be expressed as:

[0174] h(q)=f encoder (q)

[0175] Then use h(q) to generate Q. i K j V j ,Right now

[0176] Q i =W Q h(q)

[0177] K j =W K h(q)

[0178] V j =W V h(q);

[0179] Next, the attention weights are calculated:

[0180]

[0181] The enhanced state obtained by weighting the attention weights is represented as follows:

[0182]

[0183] The state-value function and advantage function enhanced by multi-head self-attention mechanism are as follows:

[0184] V(q)=f V (h att (q))

[0185] A(q,v)=f A (h att (q),v)

[0186] Finally, the state value function and the advantage function are integrated to form the enhanced Q function, namely:

[0187]

[0188] V(q)=f V (h att (q))

[0189] A(q,v)=f A (h att (q),v)

[0190]

[0191] Q i =WQ h(q)

[0192] K j =W K h(q)

[0193] V j =W V h(q)

[0194] In the above formula, V(q) and A(q,v) are the state value function and advantage function, respectively; q and v are the location and action of the UAV, respectively; B is the number of selectable actions; f V h used to enhance the state features of attention att (q) Calculate the state value V(q), h att (q) represents the state features after attention enhancement, f A The advantage function A(q,v) is used to calculate the advantage function from the state features after attention enhancement. i,j For attention weights, V j Q is a value vector. i For query vector, Let be the transpose of the key vector, i be the index of the query vector, j be the indices of the key and value vectors, and d be the transpose of the key vector. k W represents the dimension of the key vector. Q W is the linear transformation matrix that maps h(q) to the query vector space. K W is the linear transformation matrix that maps h(q) to the bond vector space. V Let h(q) be the linear transformation matrix that maps h(q) to the value vector space, where h(q) is the eigenvector.

[0195] 5.5. Based on the onboard energy consumed by the drone, determine whether the drone's current battery energy is less than a set threshold. If it is, the mobile energy storage unit charges the drone according to the charging scheduling strategy; if it is not less, proceed to step 5.6, where...

[0196] The method for calculating the onboard energy consumed by a drone is as follows:

[0197] Considering that the energy consumed by the drone during flight is greater than the energy it needs to charge within the same time frame, the drone waits on the nearest road before the mobile energy storage device arrives (considering that in real-world scenarios, the mobile energy storage device can only move on roads).

[0198] Assume that the total time consumed by drone i after completing the inspection task is T. tot T tot It consists of three parts: the charging time T of the drone during the inspection process. ch Flight time T during drone inspection fAnd the waiting time T for drones to dispatch mobile energy storage terminals when there is no power. a That is, T tot =T ch +T f +T a Then at time T tot The onboard energy E consumed by the drone can be expressed as:

[0199]

[0200] In the above formula, P(V) represents the instantaneous energy consumption of the UAV during flight, and P0 and P... i Here are two constants, representing the blade profile power and induced power of the UAV in hovering state, respectively, where V is the flight speed and U is the induced power. tip denoted as the tip velocity of the rotor blade, v0 as the average rotor induced velocity in hover, d0 as the fuselage drag ratio, ρ as the air density, s as the rotor solidity, and A as the rotor disk area.

[0201] The charging scheduling strategy for the mobile energy storage terminal is determined using a bidirectional heuristic A* algorithm. Figure 4 As shown, the algorithm's flow includes:

[0202] A. Initialize the node: Set the starting point of the mobile energy storage terminal to w s The target point of the drone is set to w g Initialize open_list and closed_list, and set the starting point w of the mobile energy storage terminal. s Add to open_list.

[0203] B. Selecting the optimal node: The traditional A* algorithm only searches from the starting point to the target. This algorithm performs both forward and reverse searches simultaneously. The forward search starts from the starting point and searches for a path to the target, while the reverse search starts from the target and searches for a path to the starting point. Finally, when the forward and reverse searches meet at a certain node, a complete path is formed.

[0204] Calculate the F-value of each node in open_list, and select the node with the smallest F-value as the current node w. c The F-value of each node is calculated using the following formula:

[0205] F = f forward (b)+f backward (b)

[0206] f forward (b) = g start (b)+h goal (b)

[0207] f backward (b) = ggoal (b)+h start (b)

[0208] In the above formula, f forward (b), f backward (b) represents the expected total path cost for forward and reverse searches, respectively, g. start (b) is from the starting point w s The actual path cost to the current node b is represented by the path length, h. goal (b) is the route from the current node b to the target node w. g The estimated cost is also represented by distance, g. goal (b) represents the total path cost of the reverse search from the target to the current node b, h. start (b) is to return from the current node b to the starting point w. s The heuristic value.

[0209] C. Move the current node w c Move it from open_list to closed_list.

[0210] D. Check neighboring nodes: Determine the current node w c If the four neighboring nodes (up, down, left, and right) are new nodes or shorter paths have been found, then update g(b) and f(b) and add them to open_list.

[0211] E. Determine the current node w c Is it the target node w? g If open_list is empty, output the optimal path for the mobile energy storage terminal, i.e., the charging scheduling scheme for the mobile energy storage terminal; otherwise, return B.

[0212] 5.6 Determine whether the drone has completed the inspection task for the inspection point. If not, return to 5.1; if completed, proceed to S37.

[0213] 5.7 Determine whether the drone has completed the inspection tasks of all inspection points. If not, return to step 3; if completed, proceed to step 5.8.

[0214] 5.8 Determine if the maximum number of iterations has been reached. If not, return to step 3 to optimize the inspection trajectory of the next UAV.

[0215] To verify the effectiveness of the present invention, a simulation was conducted using the method described in Example 1 (in this simulation, the number of inspection points was set to 5 (X in the figure), the number of drones was 1, the number of no-fly zones was 3 (blue squares), the gray area represented roads, the green pentagrams represented the initial positions of the drones, the green solid lines represented the trajectories of the drones, and the yellow solid lines represented the trajectories of the mobile energy storage terminals. The onboard energy carried by the drones was set to 50KJ). The results are as follows. Figure 5 As shown.

[0216] As can be seen, the method described in this invention can ensure that the drone completes its inspection mission while avoiding no-fly zones. Furthermore, it can be seen that when the drone's battery is low or the mission is completed, the drone will schedule a mobile energy storage device to recharge it.

[0217] Example 2:

[0218] A multi-drone inspection trajectory and mobile energy storage terminal scheduling and inspection system, such as Figure 6 As shown, it includes an information acquisition module, an inspection point allocation module, an inspection trajectory optimization module, and a mobile energy storage terminal charging scheduling module.

[0219] The information acquisition module is used to acquire the location information of each UAV inspection point.

[0220] The inspection point allocation module is used to allocate inspection points for each drone based on the distribution of inspection points and the number of drones, using a density-aware K-Means++ algorithm. The algorithm's process includes:

[0221] C1. Initial cluster centers are selected based on distance and density weights, including:

[0222] C11. Randomly select one inspection point from all inspection points as the first cluster center;

[0223] C12. Calculate the distance D(w) between each of the other inspection points and the cluster center using the following formula. i ) and the density weights ρ(w) of each other inspection point i ):

[0224]

[0225] In the above formula, d(w) i ,e j ) represents the i-th inspection point w i To the j-th cluster center e j The Euclidean distance, δ is a small positive number, and I is the number of inspection points;

[0226] C13. Calculate the probability of each other inspection point being selected as the next cluster center based on distance and density weights, and select the inspection point with the highest probability value as the next cluster center:

[0227]

[0228] In the above formula, L(w) i ) represents the i-th inspection point w i The probability of being selected as the next cluster center, D(w) i ) for w i Distance from the cluster center, ρ(w) i ) for w i Density weights;

[0229] C14. Determine whether the number of cluster centers has reached the target value, i.e., the number of drones. If it has, obtain the initial cluster centers; if not, return to S212 for the next round of screening.

[0230] C2. Assign each inspection point to the nearest cluster center;

[0231] C3. Update cluster centers based on density weights:

[0232]

[0233] In the above formula, For the updated j-th cluster center, S j For all inspection points belonging to the j-th cluster center;

[0234] C4. Repeat C2-C3 in a loop until the iteration termination condition is met.

[0235] The inspection trajectory optimization module is used to optimize the inspection trajectory of each UAV based on the inspection points of each UAV using the DDQN algorithm based on a multi-head self-attention mechanism. The algorithm process includes:

[0236] A1. Determine the optimal inspection point for the UAV based on a greedy algorithm;

[0237] A2. Select the UAV's flight speed v according to the ε-greedy strategy in the current time slot. n And calculate the reward r for the current time slot according to the following formula. n :

[0238]

[0239] In the above formula, r a As a punishment for drones flying into no-fly zones, d o,n d represents the distance between the drone's current location and the inspection point. o,n+1Let r be the distance between the next time slot location of the UAV and the inspection point, η be a constant for reducing the UAV inspection time slot, and r be the distance between the next time slot location of the UAV and the inspection point. a Both η and η are negative constants;

[0240] A3. The result of the transfer (q) n ,v n ,r n ,q n+1 Saved to the experience pool, where q n q n+1 These represent the drone's position in the current time slot and the next time slot, respectively.

[0241] A4. Randomly select N1 samples from the experience pool and use gradient descent to reduce the loss of the neural network, thereby optimizing the inspection trajectory of the UAV and obtaining a larger reward. Finally, the optimal inspection trajectory for each UAV is obtained.

[0242] The loss function is:

[0243]

[0244] In the above formula, λ is the discount factor, Q(q) n ,v n |θ) represents the location q of the drone in the current dueling network. n Take action v n Q value, For the target dueling network, the drone is at position q n+1 Take action Q value, θ, θ * Factors that influence the parameters of a neural network model;

[0245] The dueling network introduces a multi-head self-attention mechanism to enhance the state value function and the advantage function. The enhanced Q function is:

[0246]

[0247] V(q)=f V (h att (q))

[0248] A(q,v)=f A (h att (q),v)

[0249]

[0250] Q i =W Q h(q)

[0251] K j =WK h(q)

[0252] V j =W V h(q)

[0253] In the above formula, V(q) and A(q,v) are the state value function and advantage function, respectively; q and v are the location and action of the UAV, respectively; B is the number of selectable actions; f V h used to enhance the state features of attention att (q) Calculate the state value V(q), h att (q) represents the state features after attention enhancement, f A The advantage function A(q,v) is used to calculate the advantage function from the state features after attention enhancement. i,j For attention weights, V j Q is a value vector. i For query vector, Let be the transpose of the key vector, i be the index of the query vector, j be the indices of the key and value vectors, and d be the transpose of the key vector. k W represents the dimension of the key vector. Q W is the linear transformation matrix that maps h(q) to the query vector space. K W is the linear transformation matrix that maps h(q) to the bond vector space. V h(q) is the linear transformation matrix that maps h(q) to the value vector space, where h(q) is the eigenvector.

[0254] A5. Determine if the current battery energy of the drone is less than the set threshold. If it is greater than the threshold, the mobile energy storage terminal will charge the drone according to the charging scheduling strategy. If it is not less than the threshold, proceed to A6.

[0255] A6. Determine whether the drone has completed the inspection task for this inspection point. If not, return to A2; if completed, proceed to A7.

[0256] A7. Determine whether the drone has completed the inspection tasks of all inspection points. If not, return to A1; if completed, proceed to A8.

[0257] A8. Determine if the maximum number of iterations has been reached. If not, return to A1 to optimize the inspection trajectory of the next drone.

[0258] The mobile energy storage terminal charging scheduling module uses a bidirectional heuristic A* algorithm to optimize and determine the charging scheduling strategy of the mobile energy storage terminal, and controls the mobile energy storage terminal to charge the drone. The process of the bidirectional heuristic A* algorithm includes:

[0259] B1. Initialize open_list and closed_list, and set the starting point w of the mobile energy storage terminal.s Add to open_list;

[0260] B2. Simultaneously perform forward and backward searches. The forward search starts from the starting point and searches for a path to the target, while the backward search starts from the target and searches for a path to the starting point. Calculate the F-value of each node in the open_list and select the node with the smallest F-value as the current node w. c The F-value of each node is calculated using the following formula:

[0261] F = f forward (b)+f backward (b)

[0262] f forward (b) = g start (b)+h goal (b)

[0263] f backward (b) = g goal (b)+h start (b)

[0264] In the above formula, f forward (b), f backward (b) represents the expected total path cost for forward and reverse searches, respectively, g. start (b) is from the starting point w s The actual path cost to the current node b, h goal (b) is the route from the current node b to the target node w. g The estimated cost, g goal (b) represents the total path cost of the reverse search from the target to the current node b, h. start (b) is to return from the current node b to the starting point w. s heuristic value;

[0265] B3. Move the current node w c Move it from open_list to closed_list;

[0266] B4. Determine the current node w c If a neighboring node is a new node or a shorter path has been found, then add it to the open_list.

[0267] B5. Determine the current node w c Is it the target node w? g If open_list is empty, output the optimal path for the mobile energy storage terminal, i.e., the charging scheduling scheme for the mobile energy storage terminal; otherwise, return B2.

Claims

1. A method for multi-UAV inspection trajectories and mobile energy storage terminal scheduling and inspection, characterized in that, The method includes: S1. Obtain the location information of each drone inspection point; S2. Assign inspection points to each drone based on the distribution of inspection points and the number of drones. S3. Based on the inspection points of each UAV, the DDQN algorithm based on a multi-head self-attention mechanism is used to optimize the inspection trajectory of each UAV, and the charging scheduling strategy of the mobile energy storage terminal is optimized and determined, including: S31. Determine the optimal inspection point for the UAV based on a greedy algorithm; S32. Select the UAV's flight speed v according to the ε-greedy strategy in the current time slot. n And calculate the reward r for the current time slot according to the following formula. n : In the above formula, r a As a punishment for drones flying into no-fly zones, d o,n d represents the distance between the drone's current location and the inspection point. o,n+1 Let r be the distance between the next time slot location of the UAV and the inspection point, η be a constant for reducing the UAV inspection time slot, and r be the distance between the next time slot location of the UAV and the inspection point. a Both η and η are negative constants; S33, the result of the transfer (q) n ,v n ,r n ,q n+1 Saved to the experience pool, where q n q n+1 These represent the drone's position in the current time slot and the next time slot, respectively. S34. Randomly select N1 samples from the experience pool and use gradient descent to reduce the loss of the neural network, thereby optimizing the inspection trajectory of the UAV and obtaining a greater reward. Finally, the optimal inspection trajectory of each UAV is obtained. The loss function is: In the above formula, λ is the discount factor, Q(q) n ,v n |θ) represents the location q of the drone in the current dueling network. n Take action v n Q value, For the target dueling network, the drone is at position q n+1 Take action Q value, θ, θ * Factors that influence the parameters of a neural network model; The dueling network introduces a multi-head self-attention mechanism to enhance the state value function and the advantage function. The enhanced Q function is: V(q)=f V (h att (q)) A(q,v)=f A (h att (q),v) Q i =W Q h(q) K j =W K h(q) V j =W V h(q) In the above formula, V(q) and A(q,v) are the state value function and the advantage function, respectively; q and v are the location and action of the UAV, respectively; B is the number of selectable actions; f V Used to extract state features h from attention enhancement att (q) Calculate the state value V(q), h att (q) represents the state features after attention enhancement, f A The advantage function A(q,v) is used to calculate the advantage function from the state features after attention enhancement. i,j For attention weights, V j Q is a value vector. i For query vector, Let be the transpose of the key vector, i be the index of the query vector, j be the indices of the key and value vectors, and d be the transpose of the key vector. k W represents the dimension of the key vector. Q W is the linear transformation matrix that maps h(q) to the query vector space. K W is the linear transformation matrix that maps h(q) to the bond vector space. V h(q) is the linear transformation matrix that maps h(q) to the value vector space, where h(q) is the eigenvector. S35. Determine whether the current battery energy of the drone is less than the set threshold. If it is less, the mobile energy storage terminal charges the drone according to the charging scheduling strategy. If it is not less, proceed to S36. S36. Determine whether the drone has completed the inspection task of the inspection point. If not, return to S32; if completed, proceed to S37. S37. Determine whether the drone has completed the inspection tasks of all inspection points. If not, return to S31; if completed, proceed to S38. S38. Determine if the maximum number of iterations has been reached. If not, return to S31 to optimize the inspection trajectory of the next drone.

2. The method for multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection according to claim 1, characterized in that, S31 includes: S311. The drone assigns numbers to all the inspection points it is assigned. S312. Calculate the distance between the current position of the drone and each unfinished inspection point, delete the points that have been inspected, and sort the remaining inspection points in descending order of distance. S313. Determine whether the inspection point with the smallest distance has been inspected. If it has been inspected, return to S312 for the next loop calculation. If it has not been inspected, output the number of the inspection point as the next target.

3. The method for multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection according to claim 1, characterized in that, The charging scheduling strategy for the mobile energy storage terminal is determined using a bidirectional heuristic A* algorithm. The algorithm's process includes: S351. Initialize open_list and closed_list, and set the starting point w of the mobile energy storage terminal. s Add to open_list; S352. Simultaneously perform forward and backward searches. The forward search starts from the starting point and searches for a path to the target, while the backward search starts from the target and searches for a path to the starting point. Calculate the F-value of each node in the open_list and select the node with the smallest F-value as the current node w. c The F-value of each node is calculated using the following formula: F=f forward (b)+f backward (b) f forward (b)=g start (b)+h goal (b) f backward (b)=g goal (b)+h start (b) In the above formula, f forward (b), f backward (b) represents the expected total path cost for forward and reverse searches, respectively, g. start (b) is from the starting point w s The actual path cost to the current node b, h goal (b) is the route from the current node b to the target node w. g The estimated cost, g goal (b) represents the total path cost of the reverse search from the target to the current node b, h. start (b) is to return from the current node b to the starting point w. s heuristic value; S353, Move the current node w c Move it from open_list to closed_list; S354. Determine the current node w c If a neighboring node is a new node or a shorter path has been found, then add it to the open_list. S355, Determine the current node w c Is it the target node w? g If open_list is empty, output the optimal path for the mobile energy storage terminal, i.e., the charging scheduling scheme for the mobile energy storage terminal; otherwise, return S352.

4. The method for multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection according to claim 1, characterized in that, The S2 algorithm uses a density-aware K-Means++ algorithm to allocate inspection points for each UAV. The algorithm's process includes: S21. Selecting initial cluster centers based on distance and density weights, including: S211. Randomly select one inspection point from all inspection points as the first cluster center; S212. Calculate the distance D(w) between each of the other inspection points and the cluster center. i ), and the density weights ρ(w) of each other inspection point. i ); S213. Calculate the probability of each other inspection point being selected as the next cluster center based on distance and density weights, and select the inspection point with the highest probability value as the next cluster center: In the above formula, L(w) i ) represents the i-th inspection point w i The probability of being selected as the next cluster center, D(w) i ) for w i Distance from the cluster center, ρ(w) i ) for w i The density weight, where I is the number of inspection points; S214. Determine whether the number of cluster centers has reached the target value, i.e. the number of drones. If it has, obtain the initial cluster centers; if not, return to S212 for the next round of screening. S22. Assign each inspection point to the nearest cluster center; S23. Update cluster centers based on density weights: In the above formula, For the updated j-th cluster center, S j For all inspection points belonging to the j-th cluster center; S24. Repeat S22-S23 in a loop until the iteration termination condition is met.

5. The multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection method according to claim 4, characterized in that, In S212, D(w) i The following formula is used to calculate: In the above formula, d(w) i ,e j ) represents the i-th inspection point w i To the j-th cluster center e j The Euclidean distance; ρ(w i The following formula is used to calculate: In the above formula, δ is a small positive number.

6. A multi-UAV inspection trajectory and mobile energy storage terminal scheduling and inspection system, characterized in that, The system includes an information acquisition module, an inspection point allocation module, an inspection trajectory optimization module, and a mobile energy storage terminal charging scheduling module. The information acquisition module is used to acquire the location information of each UAV inspection point; The inspection point allocation module is used to allocate inspection points to each drone based on the distribution of inspection points and the number of drones. The inspection trajectory optimization module is used to optimize the inspection trajectory of each UAV based on the inspection points of each UAV using the DDQN algorithm based on a multi-head self-attention mechanism. The specific steps are as follows: A1. Determine the optimal inspection point for the UAV based on a greedy algorithm; A2. Select the UAV's flight speed v according to the ε-greedy strategy in the current time slot. n And calculate the reward r for the current time slot according to the following formula. n : In the above formula, r a As a punishment for drones flying into no-fly zones, d o,n d represents the distance between the drone's current location and the inspection point. o,n+1 Let r be the distance between the next time slot location of the UAV and the inspection point, η be a constant for reducing the UAV inspection time slot, and r be the distance between the next time slot location of the UAV and the inspection point. a Both η and η are negative constants; A3. The result of the transfer (q) n ,v n ,r n ,q n+1 Saved to the experience pool, where q n q n+1 These represent the drone's position in the current time slot and the next time slot, respectively. A4. Randomly select N1 samples from the experience pool and use gradient descent to reduce the loss of the neural network, thereby optimizing the inspection trajectory of the UAV and obtaining a larger reward. Finally, the optimal inspection trajectory for each UAV is obtained. The loss function is: In the above formula, λ is the discount factor, Q(q) n ,v n |θ) represents the location q of the drone in the current dueling network. n Take action v n Q value, For the target dueling network, the drone is at position q n+1 Take action Q value, θ, θ * Factors that influence the parameters of a neural network model; The dueling network introduces a multi-head self-attention mechanism to enhance the state value function and the advantage function. The enhanced Q function is: V(q)=f V (h att (q)) A(q,v)=f A (h att (q),v) Q i =W Q h(q) K j =W K h(q) V j =W V h(q) In the above formula, V(q) and A(q,v) are the state value function and the advantage function, respectively; q and v are the location and action of the UAV, respectively; B is the number of selectable actions; f V Used to extract state features h from attention enhancement att (q) Calculate the state value V(q), h att (q) represents the state features after attention enhancement, f A The advantage function A(q,v) is used to calculate the advantage function from the state features after attention enhancement. i,j For attention weights, V j Q is a value vector. i For query vector, Let be the transpose of the key vector, i be the index of the query vector, j be the indices of the key and value vectors, and d be the transpose of the key vector. k W represents the dimension of the key vector. Q W is the linear transformation matrix that maps h(q) to the query vector space. K W is the linear transformation matrix that maps h(q) to the bond vector space. V h(q) is the linear transformation matrix that maps h(q) to the value vector space, where h(q) is the eigenvector. A5. Determine if the current battery energy of the drone is less than the set threshold. If it is greater than the threshold, the mobile energy storage terminal will charge the drone according to the charging scheduling strategy. If it is not less than the threshold, proceed to A6. A6. Determine whether the drone has completed the inspection task for this inspection point. If not, return to A2; if completed, proceed to A7. A7. Determine whether the drone has completed the inspection tasks of all inspection points. If not, return to A1; if completed, proceed to A8. A8. Determine if the maximum number of iterations has been reached. If not, return to A1 to optimize the inspection trajectory of the next drone. The mobile energy storage terminal charging scheduling module is used to optimize and determine the charging scheduling strategy of the mobile energy storage terminal and control the mobile energy storage terminal to charge the drone.

7. The multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection system according to claim 6, characterized in that, The mobile energy storage terminal charging scheduling module uses a bidirectional heuristic A* algorithm to optimize and determine the charging scheduling strategy for the mobile energy storage terminal. The algorithm's process includes: B1. Initialize open_list and closed_list, and set the starting point w of the mobile energy storage terminal. s Add to open_list; B2. Simultaneously perform forward and backward searches. The forward search starts from the starting point and searches for a path to the target, while the backward search starts from the target and searches for a path to the starting point. Calculate the F-value of each node in the open_list and select the node with the smallest F-value as the current node w. c The F-value of each node is calculated using the following formula: F=f forward (b)+f backward (b) f forward (b)=g start (b)+h goal (b) f backward (b)=g goal (b)+h start (b) In the above formula, f forward (b), f backward (b) represents the expected total path cost for forward and reverse searches, respectively, g. start (b) is from the starting point w s The actual path cost to the current node b, h goal (b) is the route from the current node b to the target node w. g The estimated cost, g goal (b) represents the total path cost of the reverse search from the target to the current node b, h. start (b) is to return from the current node b to the starting point w. s heuristic value; B3. Move the current node w c Move it from open_list to closed_list; B4. Determine the current node w c If a neighboring node is a new node or a shorter path has been found, then add it to the open_list. B5. Determine the current node w c Is it the target node w? g If open_list is empty, output the optimal path for the mobile energy storage terminal, i.e., the charging scheduling scheme for the mobile energy storage terminal; otherwise, return B2.

8. The multi-UAV inspection trajectory and mobile energy storage terminal scheduling inspection system according to claim 6, characterized in that, The inspection point allocation module uses a density-aware K-Means++ algorithm to allocate inspection points for each UAV. The algorithm's process includes: C1. Initial cluster centers are selected based on distance and density weights, including: C11. Randomly select one inspection point from all inspection points as the first cluster center; C12. Calculate the distance D(w) between each of the other inspection points and the cluster center using the following formula. i ) and the density weights ρ(w) of each other inspection point i ): In the above formula, d(w) i ,e j ) represents the i-th inspection point w i To the j-th cluster center e j The Euclidean distance, δ is a small positive number, and I is the number of inspection points; C13. Calculate the probability of each other inspection point being selected as the next cluster center based on distance and density weights, and select the inspection point with the highest probability value as the next cluster center: In the above formula, L(w) i ) represents the i-th inspection point w i The probability of being selected as the next cluster center, D(w) i ) for w i Distance from the cluster center, ρ(w) i ) for w i Density weights; C14. Determine whether the number of cluster centers has reached the target value, i.e., the number of drones. If it has, obtain the initial cluster centers; if not, return to S212 for the next round of screening. C2. Assign each inspection point to the nearest cluster center; C3. Update cluster centers based on density weights: In the above formula, For the updated j-th cluster center, S j For all inspection points belonging to the j-th cluster center; C4. Repeat C2-C3 in a loop until the iteration termination condition is met.

Citation Information

Patent Citations

  • Unmanned aerial vehicle electric power inspection method based on deep reinforcement learning algorithm

    CN117250979A

  • Transmission, transformation and distribution unmanned aerial vehicle inspection task allocation method, system and equipment based on deep reinforcement learning, and medium

    CN118298336A