Unmanned aerial vehicle three-dimensional visual coverage path planning method based on elastic charging strategy

Abstract

The invention discloses an unmanned aerial vehicle three-dimensional visual coverage path planning method based on an elastic charging strategy, and the method comprises the steps: building a to-be-covered target waypoint according to a task terrain, deploying the position of a ground charging station, and obtaining a waypoint set containing the target waypoint, and coordinates; establishing an optimization target of a nonlinear electric vehicle path problem based on a charging power consumption model of the unmanned aerial vehicle, and setting constraint conditions; establishing a framework of a self-attention deep reinforcement learning algorithm; and the unmanned aerial vehicle calls the trained network model to complete a coverage task of a three-dimensional task scene, calculates corresponding flight time and charging time, and evaluates a self-attention deep reinforcement learning algorithm. The invention also discloses a corresponding system, electronic equipment and a computer readable storage medium, and solves the technical problems that the target value evaluation of the whole task is not accurate due to the lack of an accurate model in the charging link in the unmanned aerial vehicle coverage task, the charging efficiency is reduced by adopting a fixed charging strategy, and a more flexible charging strategy is needed to optimize the charging efficiency.

Description

Technical Field

[0001] This invention relates to the field of UAV path planning and battery management technology, specifically to a UAV three-dimensional vision coverage path planning method and system based on an elastic charging strategy. Background Technology

[0002] The Vehicle Routing Problem (CVRP) with limited capacity, in simple terms, involves delivering goods located in a warehouse to a series of customers who have ordered such goods. Each customer has a certain demand for goods, and the truck's carrying capacity is limited. When the remaining goods in a truck are insufficient to deliver to the customers, the truck needs to return to the warehouse to replenish its stock. The goal of the problem is to minimize the total cost of vehicle routing by determining which customers each vehicle delivers to and the order in which these deliveries are made. In a single-vehicle, multi-warehouse logistics delivery scenario, the core task becomes how a truck with limited capacity can efficiently move between multiple warehouses (replenishment points) distributed in different geographical locations and numerous customers waiting to be served. Unlike the traditional single-warehouse model, in this case, the vehicle has more decision-making freedom when its stock is depleted or insufficient to meet the demand of the next node—it can choose one of the existing k warehouses for replenishment and then continue its delivery mission. The goal of this problem is to establish a route with the lowest total cost by scientifically planning the order in which customers visit and when and which warehouse to replenish. When the problem size is small, we can still use simple enumeration to lock in the optimal solution.

[0003] However, with the increase in customer scale With the expansion of the scope and the introduction of the number of selectable warehouses k, the difficulty of solving the problem will increase exponentially. Given a scenario where one customer is paired with k warehouses, the potential solution space complexity can be expressed by the formula... To describe it. Specifically, in the formula It reveals the complete permutation of all possible access sequences that can be formed by all client nodes; and This reflects the complex branch of route decision-making—that is, after completing the service for each customer, the vehicle not only has to decide whether to continue to the next customer, but also faces the question of whether to interrupt the delivery to the next customer. The decision to replenish stock in one of several warehouses presents a formidable challenge. Given this massive number of possible combinations, attempting to manually sift through a vast array of possibilities to find the shortest and most feasible solution that satisfies the constraints is virtually impossible. Therefore, to address this highly complex challenge and improve decision-making efficiency, it is essential to introduce specialized mathematical optimization models and algorithms for solving the problem.

[0004] Unmanned aerial vehicles (UAVs), especially multi-rotor UAVs, have been widely used in tasks involving the coverage and monitoring of wide-area 3D terrain. The nonlinear trolley path problem (EVRP-NL) is a variant extension of VRP, namely the nonlinear trolley path problem. Using the basic framework of VRP, the "truck" in the problem is actually a UAV performing a visual coverage task. The client sets up a series of shooting points based on the terrain, and the cargo required at each client point is the energy consumed by the UAV flying to any node. The client points are the aerial shooting waypoints that the UAV needs to visit. The UAV's battery capacity is limited, with a fixed amount of energy stored at takeoff. This stored energy is continuously consumed while visiting shooting points. When the energy level is low, the UAV needs to land at a pre-set charging station (one or more) in the problem scenario to recharge. Unlike the vehicle path problem, UAV energy replenishment is a continuous process; charging takes time, and the time consumed is the UAV's flight time. One of the core technical challenges of this type of task is planning one or more flight paths that can cover all target areas, while optimizing these paths to minimize total operation time and energy consumption. However, the single-operation capability of drones is severely limited by the energy capacity of their onboard batteries when performing the aforementioned tasks. Therefore, existing technologies typically employ a solution of deploying ground charging stations within the mission area to provide energy replenishment.

[0005] Existing technical solutions mainly include three methods:

[0006] (1) The decision trajectory is determined directly by using fixed geometric motion rules, among which the mainstream Z-type traversal method can quickly generate feasible solutions, but the quality of the solutions is usually not very good.

[0007] (2) Heuristic iterative algorithms, such as genetic algorithms and ant colony algorithms inspired by biotechnology, are inspired by the biological world. They only need to set specific evaluation rules and iterative rules for the solution, without having to pay attention to some characteristics of the problem itself, and can obtain high-performance solutions.

[0008] (3) Deep reinforcement learning algorithm, which uses reinforcement learning to train the neural network model without using prior data, combining the fast response of the algorithm with the solution performance of the bio-inspired algorithm.

[0009] In trajectory planning that incorporates charging needs, existing technologies have the following two main shortcomings:

[0010] First, the modeling of the battery charging process is overly simplistic. Many existing trajectory planning algorithms reduce the battery charging process to a linear model, where the amount of charge is directly proportional to the charging time. Such linear models fail to accurately reflect the inherent nonlinear characteristics of battery charging. Specifically, as the battery approaches full charge, its charging efficiency drops significantly, meaning the time required to charge a unit of charge (i.e., the marginal charging time) increases dramatically. Existing algorithms ignore this nonlinear effect, leading to biased estimations of charging time and thus affecting the global optimality of trajectory planning.

[0011] Secondly, the charging strategy lacks flexibility. To address the complexity of the charging process, existing technologies generally employ a fixed, non-optimized charging strategy, requiring the drone to fully charge its battery every time it enters a charging station. While this "fixed full charge" strategy simplifies the decision-making process, it frequently triggers the inefficient phase at the end of the charging process, causing unnecessary long waits at charging stations and severely impacting overall mission efficiency. Although partial charging can reduce waiting time at stations, this may increase the frequency of drone trips to and from charging stations and incur additional flight energy consumption. Summary of the Invention

[0012] The purpose of this invention is to provide a method and system for UAV 3D vision coverage path planning based on an elastic charging strategy. This addresses the technical problem that the lack of an accurate model for the charging process in UAV coverage missions leads to inaccurate evaluation of the overall mission objective value, requiring specific modeling based on actual battery characteristic curves. It also addresses the technical problem that the fixed charging strategy used in existing solutions reduces charging efficiency, necessitating a more flexible charging strategy to further optimize the charging efficiency of UAVs.

[0013] The first aspect of the present invention is to provide a method for planning three-dimensional visual coverage path of a UAV based on an elastic charging strategy, comprising:

[0014] S1, establish the target waypoints to be covered based on the mission terrain and deploy the ground charging stations, and obtain the waypoint set containing the target waypoints and their coordinates;

[0015] S2, Establish the optimization objective and set constraints for the nonlinear trolleybus routing problem based on the charging and power consumption model of UAVs;

[0016] S3, establishes the framework for self-attention deep reinforcement learning algorithms;

[0017] S4, the UAV calls the trained network model to complete the 3D task scene coverage task, calculates the corresponding flight time and charging time, and evaluates the self-attention deep reinforcement learning algorithm.

[0018] Preferably, S1 includes:

[0019] S11, Obtain task environment information according to the requirements of the covered task;

[0020] S12, establish a topology graph consisting of takeoff points, waypoints, and charging stations, as well as an abstract graph representation of the mission, including a set of all nodes covering the mission. and the set of all edges , and the set of actual spatial locations corresponding to the nodes As shown in equation (1):

[0021] (1);

[0022] This includes the set consisting of all nodes. As shown in equation (2):

[0023] (2);

[0024] Each node is assigned a unique number using a serial number system. The serial numbers for the takeoff point, charging station, and shooting waypoint are defined as follows (3):

[0025] (3);

[0026] in, Indicates the nodes for drone takeoff and landing; This represents the set of deployed charging stations; This represents the set of waypoints used for ground-based photography coverage.

[0027] express The set of three-dimensional coordinates of the actual spatial location of the node in the task scene is shown in Equation (4):

[0028] (4);

[0029] Represents a node The set of all edges formed by connecting any two nodes in the array is shown in equation (5):

[0030] (5).

[0031] Preferably, S2 includes:

[0032] S21. Establish the energy model required for the nonlinear trolley path problem. The energy model includes the UAV's battery charging model and the UAV's power consumption model, and determine the input information required for the problem, including the UAV's flight speed, upper limit of battery capacity, and lower limit of battery capacity.

[0033] S22, Establish the optimization objective and constraints for the nonlinear trolley path problem.

[0034] Preferably, S21 includes:

[0035] (1) Establish a battery charging model for the UAV, which is used to calculate the charging time of the UAV given the battery level of the UAV entering the charging station and the battery level of the UAV leaving the charging station; including: by measuring real battery charging data and fitting it with a piecewise linear model, a calculation formula (6) is constructed to estimate the charging time of the UAV:

[0036] (6);

[0037] in, The maximum error of the fit, yes One slope parameter, This represents the number of charging data sampling points that each slope will pass through, indicating that there are... A set of straight lines describing the lithium battery charging model; each set of lines calculates the moment when charging begins. and the moment of stopping charging and leaving That is, to obtain One candidate time to enter the charging station and time of leaving the charging station Finally, the charging time is calculated by taking the maximum value of each of the two candidate times according to formula (7) and calculating the difference. As shown below:

[0038] (7);

[0039] in, Indicates that the drone is at the charging station Charging time; This indicates that the drone is at the mission point or charging station. Battery status at that time; This indicates the time when the drone corresponds to the charging model based on the battery state when entering and leaving the charging station.

[0040] (2) Establish a power consumption model for the UAV. The power consumption model for the UAV is used to obtain the relationship between flight distance and energy consumption under the known flight speed of the UAV and the total payload of the UAV. An empirical model is adopted, as shown in the following formula (8):

[0041] (8);

[0042] in, The relationship curves between drone speed, payload, and power are derived from existing experiments and can be obtained through sampling in real-world scenarios. It is the random error introduced by the empirical model that satisfies ; Indicates that the drone is from the node Fly to the node Path length; Represents a node Fly to the node The increment of the battery level at that location, with a value range of [0,1].

[0043] Preferably, S22 includes:

[0044] (1) Determine the optimization objective of the model, as shown in equation (9):

[0045] (9);

[0046] objective function This represents the minimum flight time for each flight path of the drone coverage mission and the minimum charging time for each charging station, assuming a constant speed, and the corresponding flight time and length of each path. Proportional to the number of drones at charging stations Charging time Battery state upon entering and leaving the charging station Related; Indicates that the drone is from the node Fly to the node Time required; Used to indicate that the drone has passed through the node. To the node The path is represented by 1, indicating that the path was visited, and 0, indicating that the path was not visited. Indicates that the drone is at the charging station Charging time;

[0047] (2) Determine the constraints related to the problem, including: determining the corresponding constraints based on the coverage task and the characteristics of the UAV itself:

[0048] Constraint 1 is used to guarantee coverage of each waypoint, as shown in equation (10):

[0049] (10);

[0050] Constraint 2 is used to ensure that the drone does not fly continuously to the charging station, as shown in equation (11):

[0051] (11);

[0052] Constraint 3 is used to balance the flight path, ensuring that there is only one complete UAV path loop, as shown in equation (12):

[0053] (12);

[0054] Constraint 4 ensures that the power consumption of the planned path will not exceed the maximum power of the drone before the next charging, as shown in equation (13):

[0055] (13);

[0056] Constraint 5 is used to ensure that the drone receives sufficient charging at the charging station, as shown in equation (14):

[0057] (14);

[0058] Constraint 6 is used to ensure the energy supply of the UAV throughout the entire mission, as shown in equation (15):

[0059] (15);

[0060] Constraint 7 is used to ensure that the battery power of the UAV during flight does not exceed the threshold, as shown in equation (16):

[0061] (16);

[0062] in, This indicates the minimum battery level of the drone during the entire coverage process, ensuring battery safety and providing the drone with a certain amount of energy redundancy. This indicates the highest battery level of the drone during the entire coverage process, ensuring that the battery will not be damaged due to overcharging; This indicates that the drone is at the mission point or entering and leaving the charging station. Battery status at that time; Is the plan in the current state? The amount of electricity charged, i.e., the amount of charge the drone receives at the charging point, is measured in units of energy. It is a variable whose value depends on the power of the charging point and the charging time. Battery characteristics; the definition of the strategy path. This represents a sequence of node visits consisting of a series of node numbers. This indicates the maximum time step in the node's access sequence; Indicates the drone's location Fly to position The energy consumed.

[0063] Preferably, S3 includes:

[0064] S31, Establish a Markov decision model based on the task, including a state space. Action space ,award and termination conditions Defined quadruple The formal description of information at each time step simplifies the action space using a critical greedy charging approach, including:

[0065] (1) Define the state space Including: at the current moment The state space of the intelligent agent is composed of It means that among them yes In the task graph of the agent at any given moment The point in the middle, It indicates its charging status or remaining power.

[0066] (2) Define the action space Including: in the state Below, the action space of an intelligent agent is defined as ;in Is it an intelligent agent? The nodes that need to be accessed at all times. Is the plan in the current state? The amount of electricity charged, i.e., the amount of electricity charged in time step. The amount of charge a drone receives at a charging point is measured in units of energy. It is a variable whose value depends on the power of the charging point and the charging time. Battery characteristics; when the drone is at a charging point, this charge will be added to its remaining battery power from the previous moment. middle;

[0067] State transition for the agent from state Transition to state It is based on the current time of the drone. status and actions ; Location updated to In Battery status The transfer conforms to the calculation formula (17):

[0068] (17);

[0069] in, Indicates the drone's location Move (fly) to location The energy consumed;

[0070] For action space It includes the UAV's decision target point and the charge amount at the target point. When making decisions simultaneously with the flight node, there is a coupling between charging decisions and flight decisions. Therefore, the minimum charging criterion is adopted to simplify the action space, which is then simplified to: ,in The estimated energy consumption for the drone's flight from the current charging point to the next charging point in the decision path; the simplified action space makes decisions on the drone's flight and charging simultaneously only through the target point of the drone's decision, which greatly reduces the spatial complexity and decision difficulty of the Markov model.

[0071] (3) Define rewards This includes: the objective of the drone coverage mission is to minimize the total mission time, and the reinforcement learning objective is to maximize the cumulative reward obtained throughout the Markov process. ,to this end, The reward for each moment is It is defined as the negative value of the time cost incurred during the state transition process, which includes flight time and time spent at the node. The charging time and the corresponding reward function are shown in equation (18):

[0072] (18);

[0073] in, Indicates from position Move to position The required flight time or travel time is a key cost in path planning, which typically depends on the Euclidean distance or actual path distance between two points, the flight speed of the drone, and environmental factors such as wind, obstacles, and airspace restrictions. Indicates at node The charging time or service time is a location-dependent time cost: when It is a charging point ( )hour, This indicates the time required to charge at that point. Charging time depends on the charging power, battery capacity, current battery level, target battery level, and charging efficiency. It is the work point ( )hour,

[0074] (4) Define the termination condition This includes: the Markov process reaching its termination condition when the drone has visited all the waypoints and returned to the takeoff point. The cumulative reward is calculated as shown in equation (19):

[0075] (19);

[0076] S32, the UAV performs the coverage task from the takeoff point according to the waypoint flight path. Based on the Markov decision model described in step S31, the takeoff point is used as the initial position in the model, and the fully charged state is used as the initial battery state in the model; including:

[0077] The graph data is input into the encoder module of the deep network corresponding to the Markov decision model for encoding, thus encoding the initial state of the UAV. In the input decoder module, the deep network processes the decoder output to obtain the target point for the first flight step; wherein, the encoding operation is completed by formula (20):

[0078] (20);

[0079] The encoder uses three different sets of parameters to perform three rounds of encoding operations to extract high-dimensional information from the graph and obtain the encoded output. ;

[0080] Formula (20) is the mathematical expression for the multi-head attention mechanism, used to encode input features, where, The input feature matrix has a dimension of . ,in This represents the number of nodes in the graph, including takeoff points, waypoints, and charging stations. The dimension of the feature vector; Indicates the encoder at the 1st The input features of the layer; for the first layer encoder, These are the initial characteristics of the nodes; : No. One attention point ( The learnable parameter matrix is ​​used to linearly transform the input features into query, key, and value matrices, respectively. , , usually set ; is the scaling factor, representing the dimension of the key vector, used to scale the dot product result and prevent gradient vanishing or exploding in the softmax function; The softmax function normalizes each row of the matrix so that the sum of the elements in each row is 1. This is used to calculate the attention weights. , ; The concatenation function concatenates the output matrices of multiple attention heads along the feature dimensions. If there are a total of... There are 1 head, and the output dimension of each head is 1. The output dimension after concatenation is ; The number of attention heads, i.e., the number of parallel self-attention mechanisms in multi-head attention; The output of the multi-head attention function, i.e., for the input The feature matrix obtained after multi-head self-attention calculation;

[0081] S33: Based on the drone's remaining battery power, calculate the expected battery power for the next waypoint using the power consumption model. If the expected battery power is sufficient to reach at least one charging station, estimate the possible battery power for the next state based on the current remaining battery power and decide whether the drone should enter S34 for charging or repeat S33; otherwise, enter S34 for charging; if the drone has traversed all waypoints, return to the takeoff point and execute S35; S33 includes:

[0082] (1) According to the first Step state Decision No. The algorithm checks if the battery has enough energy to reach a charging station. If it does, the algorithm decides to fly based on its battery status; otherwise, the algorithm stops at the decision stage. During the flight maneuver, all flight shooting points were excluded, and only charging was decided upon.

[0083] (2) Using a masking method, waypoints and charging stations that do not meet the constraints are masked by the constraints in EVRP-NL and the current battery status of the UAV. The candidate values ​​of the remaining nodes are calculated by formula (21) of the following decoding operation, thereby obtaining the specific representation of the candidate values, that is, the nodes in the node set. Obtain candidate scores :

[0084] (twenty one);

[0085] in, For nodes The candidate score is a scalar value used to characterize the node selection in the current state. As a measure of suitability or attractiveness for the next visit target, the higher the score, the greater the probability of being selected. It is an adjustable constant used for scaling. The output range of the function controls the magnitude of the candidate scores; The hyperbolic tangent activation function maps the input to the interval (-1, 1); This represents the learnable parameter matrix used to generate the query vector, with dimensions [missing information]. This belongs to the decoder and maps the decoder's context information to the query space; Indicates the decoder at time step The context vector, with dimensions of It encodes the decision history and drone status information up to the current moment, and is the hidden state of the decoder; The query vector representing the current time step is obtained through... right The result is obtained by transposing after a linear transformation, with dimensions of ; This represents the learnable parameter matrix used to generate the key vector (Key), with dimensions [missing information]. This maps the node features output by the encoder to the key space; Represents a node The encoded feature vector, with dimension The final encoded output from the encoder The middle corresponds to the node The first line contains the high-dimensional semantic information of the node in the task graph; Represents a node The key vector, through right We obtain it by performing a linear transformation, with dimension . ; Scaling factor It is the dimension of the query and key vector; This is a very large negative value, i.e., negative infinity. In the masking mechanism, it is assigned to those infeasible nodes, and in the subsequent Softmax calculation, =0, thus ensuring that the probability of these nodes being selected is 0;

[0086] (3) The obtained candidate values ​​are mapped through a hybrid expert network, and then the softmax function is used to output the next target node. The following formula (22) shows the process of calculating the final node, as follows:

[0087] (twenty two)

[0088] Formula (22) is used to output the probability distribution of the final selection of the next node through a hybrid expert network based on the candidate scores, and Formula (21) is used to obtain the candidate scores. The data is simultaneously fed into two different expert networks for nonlinear transformation, and the gating network is adjusted according to the current power level. and historical charging behavior The mixed weights of the two expert outputs are dynamically calculated, and the final mixed result is then processed by Softmax to obtain the probability of selecting each node. Let V be a probability distribution vector with dimension |V|, which represents the total number of nodes in the graph. Let V be the _th_ node of the vector. element Indicates the selection of a node As The probability of accessing the target at any given time; The Softmax function transforms a vector of arbitrary real values ​​into a probability distribution vector, where the sum of all elements is 1. For a gated network, the input is and The output is a weight vector (e.g., [α, 1-α]) used to mix the outputs of the two expert networks; Indicates the drone at time step The remaining battery charge (normalized to the [0,1] range); A binary flag indicates whether the drone has already visited a charging station during the current mission; This is a vector concatenation operation that concatenates the output vectors of two expert networks along the feature dimension. The network is trained to produce learnable parameter matrices for two expert networks. These are two parallel fully connected layers with different initialization parameters. Let V be the candidate score vector, with dimension |V|, which is the candidate score of all nodes calculated by formula (21). The vector formed;

[0089] S34, charging the drone;

[0090] S35, the UAV ends its coverage mission and returns to the takeoff point, and calculates the objective function to train the network; it determines whether the current number of training rounds has reached the set termination round number. If the termination round number has been reached, the training ends, the trained model is output, and step S4 is executed; otherwise, the network model parameters in the self-attention deep reinforcement learning algorithm are optimized according to the result of the task execution, and the process returns to S32.

[0091] Preferably, S35 includes:

[0092] (1) When training network parameters, the baseline parameters are first randomly initialized. The baseline index is obtained through network output. ;

[0093] (2) During the initial training, As training parameters Based on the current parameters and the overall trajectory reward obtained. The REINFORCE method is used to obtain the parameters. The update gradient, for Update according to the following calculation formula (23):

[0094] (twenty three);

[0095] Updated ;

[0096] in, For gradient operators, it means that with respect to parameters Find the partial derivative; Let be the objective function, representing the parameters The expected cumulative reward is given below, and the training objective is to maximize it. ; The approximation is equal to, indicating that Monte Carlo sampling is used to estimate the expected gradient; The batch size is the number of independent trajectories (episodes) sampled in each training batch. To index all trajectories in a batch Summation; Indicates the index of the trajectory in the batch. ; For the first A trajectory is the sequence of nodes visited by a drone during a complete mission; For trajectory In drone coverage missions, the cumulative reward is usually a negative time cost, so maximizing the cumulative reward is equivalent to minimizing the total mission time. For the first Each task graph instance contains a set of nodes. Edge set and node position set It describes the specific task scenario; For the baseline function, by parameters Parameterization, used to estimate a given graph An approximation of the expected cumulative reward is used to reduce the variance of the gradient estimate; These are the parameters of the baseline network, which are also updated during training; Let be the policy probability, representing the probability given a graph. Under the condition that the parameter is The strategy network generates trajectories The probability is the product of the probabilities of the strategy choosing an action at each time step; Let be the logarithmic probability of the trajectory; The score function is the logarithmic probability with respect to the parameter. The gradient indicates how to adjust To increase or decrease the trajectory The probability of its occurrence; The estimate of the dominance function represents the trajectory. The cumulative reward is evaluated relative to the baseline prediction. If the result is positive, it indicates that the trajectory is better than average and its probability should be increased; if the result is negative, its probability should be decreased.

[0097] (3) The baseline is updated using a one-sided t-test in the test set and the baseline metric. Only when more than 60% of the examples in the dataset have a target value better than the baseline metric will the update be performed. Updated to When the parameters in step S32 are returned to... ;

[0098] (4) Repeat the above parameter update method until the number of iterations reaches the set value; after training, the baseline parameters are... As the final output network parameters.

[0099] A second aspect of the present invention provides a UAV 3D vision coverage path planning system based on an elastic charging strategy, for implementing the method of the first aspect, comprising:

[0100] The information acquisition module (101) is used to establish the target waypoints to be covered and the location of the ground charging station based on the task terrain, and to acquire the set of waypoints containing the target waypoints and their coordinates;

[0101] The optimization objective establishment module (102) is used to establish the optimization objective and set the constraints for the nonlinear trolleybus path problem based on the charging and power consumption model of the UAV.

[0102] The self-attention deep reinforcement learning module (103) is used to establish the framework of the self-attention deep reinforcement learning algorithm;

[0103] The path planning module (104) is used by the UAV to call the trained network model to complete the 3D task scene coverage task, calculate the corresponding flight time and charging time, and evaluate the self-attention deep reinforcement learning algorithm.

[0104] A third aspect of the present invention provides an electronic device including a processor and a memory, the memory storing a plurality of instructions, the processor being configured to read the instructions and execute the method as described in the first aspect.

[0105] A fourth aspect of the present invention provides a computer-readable storage medium storing a plurality of instructions which can be read by a processor and executed as described in the first aspect.

[0106] The beneficial effects of the method and system of the present invention are as follows:

[0107] This invention establishes a battery measurement platform and combines a multi-segment linear model to fit the battery charging curve; it designs a hybrid expert network based on the nonlinear charging characteristics of UAV batteries to flexibly adjust the charging strategy during the decision-making process; it solves the technical problem of extending the UAV charging strategy using an expert network model as the final output layer of the network based on deep reinforcement learning algorithms or other sequential decision-making algorithms in UAV 3D vision coverage tasks. The beneficial effects are as follows:

[0108] 1. This invention introduces the actual charging characteristics of batteries into the visual coverage problem of UAVs and designs a corresponding nonlinear trolley path problem, so that the planned task objectives are more consistent with reality.

[0109] 2. This invention uses a deep reinforcement learning algorithm with an elastic charging strategy to solve the established combinatorial optimization problem. It adopts an attention-pointer network as the network model and introduces an expert fusion network into the structure, enabling the model to have a more flexible charging strategy and optimize the charging decision in the drone coverage mission.

[0110] 3. This invention combines deep learning and reinforcement learning, achieving faster decision-making speeds to meet the dynamic decision-making needs of UAVs, while also exhibiting strong generalization capabilities. Furthermore, the trained network possesses excellent scalability, enabling it to easily solve larger-scale and more complex problems. Attached Figure Description

[0111] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of the present invention, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0112] Figure 1(a) is a flowchart of a UAV 3D vision coverage path planning method based on an elastic charging strategy according to an embodiment of the present invention; Figure 1(b) is a flowchart of the reinforcement learning algorithm training and verification process in the UAV 3D vision coverage path planning method based on an elastic charging strategy according to an embodiment of the present invention.

[0113] Figure 2 This is a system architecture diagram of a UAV 3D vision coverage path planning system based on an elastic charging strategy according to an embodiment of the present invention;

[0114] Figure 3 This is a schematic diagram of a coverage task scenario provided according to an embodiment of the present invention;

[0115] Figure 4 This is a schematic diagram of a platform for measuring the state of charge of a battery according to an embodiment of the present invention;

[0116] Figure 5 This invention provides a diagram showing the results of battery data collection using a charging platform constructed according to an embodiment of the invention.

[0117] Figure 6 This is a structural diagram of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0118] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0119] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0120] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0121] Example 1

[0122] This embodiment provides a UAV 3D visual coverage path planning method based on an elastic charging strategy. The core task of visual coverage in a 3D terrain mission scenario is for a multi-rotor UAV equipped with an automatic gimbal camera to completely capture a 3D region of interest while satisfying a series of measurement quality constraints, specifically including equidistant constraints, frontal constraints, and overlap constraints. A set of aerial shooting points, also known as the UAV's mission waypoint set, is generated from 3D terrain point clouds acquired via satellite digital elevation images or radar, according to the measurement quality constraints. To ensure energy supply during the UAV coverage mission, one or more charging stations need to be deployed in flat, open terrain areas so that the UAV can directly land at a charging station to recharge when its battery is low.

[0123] The goal of its drone 3D vision coverage planning is to minimize the total time the drone spends on the mission, including cumulative flight and charging time, while ensuring complete coverage of the mission site by traversing waypoints. Furthermore, the drone must adhere to a critical safety constraint: its battery SoC must always be above a safe threshold to prevent the drone from running out of power due to low battery levels.

[0124] Referring to Figures 1(a) and 1(b), this embodiment provides a UAV coverage trajectory planning method based on an elastic charging strategy. The UAV achieves optimal trajectory planning for the total mission time in a wide-area three-dimensional environment containing a set of waypoints to be covered and multiple ground charging stations, including:

[0125] S1, establish the target waypoints to be covered based on the mission terrain and deploy the ground charging stations, and obtain the set of waypoints containing the target waypoints and their coordinates.

[0126] In a preferred embodiment, S1 includes:

[0127] S11, Obtain task environment information according to the requirements of the covered task;

[0128] S12, establish a topology graph consisting of takeoff points, waypoints, and charging stations, as well as an abstract graph representation of the mission, including a set of all nodes covering the mission. and the set of all edges , and the set of actual locations in space corresponding to the nodes As shown in equation (1):

[0129] (1);

[0130] This includes the set consisting of all nodes. As shown in equation (2):

[0131] (2);

[0132] Each node is assigned a unique number using a serial number system. The serial numbers for takeoff point, charging station, and shooting waypoint are defined as follows (3):

[0133] (3);

[0134] in, Indicates the nodes for drone takeoff and landing; This represents the set of deployed charging stations; This represents the set of waypoints used for ground-based photography coverage.

[0135] express The set of three-dimensional coordinates of the actual spatial location of the node in the task scene is shown in Equation (4):

[0136] (4);

[0137] Represents a node The set of all edges formed by connecting any two nodes in the array is shown in equation (5):

[0138] (5).

[0139] S2. Establish the optimization objective and set constraints for the nonlinear trolley route problem based on the charging and power consumption model of UAVs.

[0140] In a preferred embodiment, S2 includes:

[0141] S21. Establish the energy model required for the nonlinear trolley path problem. The energy model includes the UAV's battery charging model and the UAV's power consumption model, and determine the input information required for the problem, including the UAV's flight speed, upper limit of battery capacity, and lower limit of battery capacity.

[0142] In a preferred embodiment, S21 includes:

[0143] (1) Establish a battery charging model for the UAV. The battery charging model for the UAV is used to calculate the charging time of the UAV given the battery level of the UAV entering the charging station and the battery level of the UAV leaving the charging station.

[0144] This invention measures real battery charging data, and the platform for measuring this real data is referenced. Figure 4 This includes the battery under test, a smart charger, a digital meter, and a host computer data acquisition system. Data from a 2200mAh drone battery was collected and fitted using a piecewise linear model. A calculation formula (6) was constructed to estimate the drone's charging time:

[0145] (6);

[0146] in, The maximum error of the fit, yes One slope parameter, This represents the number of charging data sampling points that each slope will pass through, indicating that there are... A set of straight lines describing the lithium battery charging model; each set of lines calculates the moment when charging begins. and the moment of stopping charging and leaving That is, to obtain One candidate time to enter the charging station and time of leaving the charging station Finally, the charging time is calculated by taking the maximum value of each of the two candidate times according to formula (7) and calculating the difference. As shown below:

[0147] (7);

[0148] in, : Indicates that the drone is at the charging station Charging time; This indicates that the drone is at the mission point or charging station. Battery status at that time; This indicates the time when the drone corresponds to the charging model based on the battery state when entering and leaving the charging station.

[0149] (2) Establish a power consumption model for the UAV. The power consumption model for the UAV is used to obtain the relationship between flight distance and energy consumption under the known flight speed of the UAV and the total payload of the UAV. An empirical model is adopted, as shown in the following formula (8):

[0150] (8);

[0151] in, The relationship curves between drone speed, payload, and power are derived from existing experiments and can be obtained through sampling in real-world scenarios. It is the random error introduced by the empirical model that satisfies ; Indicates that the drone is from the node Fly to the node Path length; Represents a node Fly to the node The increment of the battery level at that location, with a value range of [0,1].

[0152] In this embodiment, based on actual sampling, the drone was controlled at a speed of 6 m / s, with a rated operating power of 226 W.

[0153] S22, Establish the optimization objective and constraints for the nonlinear trolley path problem.

[0154] In a preferred embodiment, S22 includes:

[0155] (1) Determine the optimization objective of the model, as shown in equation (9):

[0156] (9);

[0157] objective function This represents the minimum flight time for each flight path of the drone coverage mission and the minimum charging time for each charging station, assuming a constant speed, and the corresponding flight time and length of each path. Proportional to the number of drones at charging stations Charging time Battery state upon entering and leaving the charging station Related; Indicates that the drone is from the node Fly to the node Time required; Used to indicate that the drone has passed through the node. To the node The path is represented by 1, indicating that the path was visited, and 0, indicating that the path was not visited. Indicates that the drone is at the charging station The time required for charging.

[0158] (2) Determine the constraints related to the problem, including: determining the corresponding constraints based on the coverage task and the characteristics of the UAV itself:

[0159] Constraint 1 is used to guarantee coverage of each waypoint, as shown in equation (10):

[0160] (10);

[0161] Constraint 2 is used to ensure that the drone does not fly continuously to the charging station, as shown in equation (11):

[0162] (11);

[0163] Constraint 3 is used to balance the flight path, ensuring that there is only one complete UAV path loop, as shown in equation (12): (12);

[0164] Constraint 4 ensures that the power consumption of the planned path will not exceed the maximum power of the drone before the next charging, as shown in equation (13):

[0165] (13);

[0166] Constraint 5 is used to ensure that the drone receives sufficient charging at the charging station, as shown in equation (14):

[0167] (14);

[0168] Constraint 6 is used to ensure the energy supply of the UAV throughout the entire mission, as shown in equation (15):

[0169] (15);

[0170] Constraint 7 is used to ensure that the battery power of the UAV during flight does not exceed the threshold, as shown in equation (16):

[0171] (16);

[0172] in, This indicates the minimum battery level of the drone during the entire coverage process, ensuring battery safety and providing the drone with a certain amount of energy redundancy. This indicates the highest battery level of the drone during the entire coverage process, ensuring that the battery will not be damaged due to overcharging; This indicates that the drone is at the mission point or entering and leaving the charging station. Battery status at that time; Is the plan in the current state? The amount of electricity charged, i.e., the amount of charge the drone receives at the charging point, is measured in units of energy. It is a variable whose value depends on the power of the charging point and the charging time. Battery characteristics; the definition of the strategy path. This represents a sequence of node visits consisting of a series of node numbers. This indicates the maximum time step in the node's access sequence; Indicates the drone's location Fly to position The energy consumed.

[0173] S3, establishes the framework for self-attention deep reinforcement learning algorithms;

[0174] In a preferred embodiment, S3 includes:

[0175] S31, Establish a Markov decision model based on the task, including a state space. Action space ,award and termination conditions Defined quadruple The form describes the information for each time step and simplifies the action space by using critical greedy charging.

[0176] In this embodiment, step S31 includes:

[0177] (1) Define the state space Including: at the current moment The state space of the intelligent agent is composed of It means that among them yes In the task graph of the agent at any given moment The point in the middle, It indicates its charging status or remaining power.

[0178] (2) Define the action space Including: in the state Below, the action space of an intelligent agent is defined as ;in Is it an intelligent agent? The nodes that need to be accessed at all times. Is the plan in the current state? The amount of electricity charged, i.e., the amount of electricity charged in time step. The amount of charge a drone receives at a charging point is measured in units of energy, such as joules (J) or milliampere-hours (mAh). This is a variable, and its value may depend on factors such as the charging point's power, charging time, and battery characteristics. When the drone is at a charging point, this charge is added to its remaining battery level from the previous moment. middle;

[0179] State transition for the agent from state Transition to state It is based on the current time of the drone. status and actions ; Location updated to In Battery status The transfer conforms to the calculation formula (17):

[0180] (17);

[0181] in, Indicates the drone's location Move (fly) to location The energy consumed;

[0182] For action space It includes the UAV's decision target point and the charge amount at the target point. When making decisions simultaneously with the flight node, there is a coupling between charging decisions and flight decisions. Therefore, the minimum charging criterion is adopted to simplify the action space, which is then simplified to: ,in The estimated energy consumption for the drone's flight from the current charging point to the next charging point in the decision path; the simplified action space makes decisions on the drone's flight and charging simultaneously only through the target point of the drone's decision, which greatly reduces the spatial complexity and decision difficulty of the Markov model.

[0183] (3) Define rewards This includes: the objective of the drone coverage mission is to minimize the total mission time, and the reinforcement learning objective is to maximize the cumulative reward obtained throughout the Markov process. ,to this end, The reward for each moment is It is defined as the negative value of the time cost incurred during the state transition process, which includes flight time and time spent at the node. The charging time and the corresponding reward function are shown in equation (18):

[0184] (18);

[0185] in, Indicates from position Move to position The required flight time or travel time is a key cost in path planning, which typically depends on the Euclidean distance or actual path distance between two points, the flight speed of the drone, and environmental factors such as wind, obstacles, and airspace restrictions. Indicates at node The charging time or service time is a location-dependent time cost: when It is a charging point ( )hour, This indicates the time required to charge at that point. Charging time depends on the charging power, battery capacity, current charge level, target charge level, and charging efficiency. It is the work point ( )hour, Because it is not charging at that point.

[0186] Define rewards Intention:

[0187] Reward design: Reward for each step = -(flight time + charging time). The agent (drone) is penalized for wasting time, thus incentivizing it to find the time-optimal path and charging strategy.

[0188] Reinforcement learning objective: Maximize cumulative rewards That is, to minimize the total mission time (total flight time + total charging time).

[0189] Correlation: This reward function works in conjunction with the battery state transition equation (Equation 17): the battery equation (6) ensures energy constraints (avoiding battery depletion); the reward function (18) optimizes time efficiency while satisfying energy constraints.

[0190] Furthermore, if the task execution time is not negligible and is not included In practice, it might be necessary to extend the reward function, for example, by adding a task time factor; in actual implementation, and It may be pre-computed or computed in real time as part of the environment model; this negative time cost reward design is a common reinforcement learning modeling approach for minimizing time problems, and these two parameters are the core time cost elements in the joint optimization of UAV path and charging.

[0191] (4) Define the termination condition This includes: the Markov process reaching its termination condition when the drone has visited all the waypoints and returned to the takeoff point. The cumulative reward is calculated as shown in equation (19):

[0192] (19).

[0193] S32, the UAV performs the coverage task from the takeoff point according to the waypoint flight path. According to the Markov decision model described in step S31, the takeoff point is taken as the initial position in the model, and the fully charged state is taken as the initial battery state in the model.

[0194] In a preferred embodiment, S32 specifically includes:

[0195] The graph data is input into the encoder module of the deep network corresponding to the Markov decision model for encoding, thus encoding the initial state of the UAV. In the input decoder module, the deep network processes the decoder output to obtain the target point for the first flight step; wherein, the encoding operation is completed by formula (20):

[0196] (20);

[0197] The encoder uses three different sets of parameters to perform three rounds of encoding operations to extract high-dimensional information from the graph and obtain the encoded output. .

[0198] Formula (20) is the mathematical expression for the Multi-Head Attention (MHA) mechanism, used to encode input features, where, The input feature matrix has a dimension of . ,in This represents the number of nodes in the graph, including takeoff points, waypoints, and charging stations. The dimension of the feature vector; Indicates the encoder at the 1st The input features of the layer; for the first layer encoder, The initial features of the node, such as node coordinates and type, are embedded in the representation; : No. One attention point ( The learnable parameter matrix is ​​used to linearly transform the input features into query, key, and value matrices, respectively. , , usually set ; is the scaling factor, representing the dimension of the key vector (Key), used to scale the dot product result and prevent gradient vanishing or exploding in the softmax function; The softmax function normalizes each row of the matrix so that the sum of the elements in each row is 1, which is used to calculate the attention weights. Specifically, for the matrix... ,have ; The concatenation function concatenates the output matrices of multiple attention heads along the feature dimensions. If there are a total of... There are 1 head, and the output dimension of each head is 1. The output dimension after concatenation is ; The number of attention heads, i.e., the number of parallel self-attention mechanisms in multi-head attention; The output of the multi-head attention function, i.e., for the input The feature matrix obtained after multi-head self-attention calculation; in formula (20), for each attention head Calculate the query matrix Key matrix Sum matrix Calculate the attention score matrix , dimension This indicates the correlation between nodes; for The attention weight matrix is ​​obtained by applying softmax to each row. ; Calculate the output of this head , dimension .

[0199] S33: Based on the current remaining battery power of the drone, calculate the expected battery power after flying to the next waypoint according to the power consumption model. If the expected battery power is sufficient to reach at least one charging station, estimate the possible battery power for the next state based on the current remaining battery power and decide whether the drone should enter S34 to charge or repeat S33; otherwise, enter S34 to charge; if the drone has traversed all waypoints, return to the takeoff point and execute S35.

[0200] In this embodiment, step S33 specifically includes:

[0201] (1) According to the first Step state Decision No. The algorithm checks if the battery has enough energy to reach a charging station. If it does, the algorithm decides to fly based on its battery status; otherwise, the algorithm stops at the decision stage. During the flight maneuver, all flight shooting points were excluded, and only charging was decided upon.

[0202] (2) Using a masking method, waypoints and charging stations that do not meet the constraints are masked by the constraints in EVRP-NL and the current battery status of the UAV. The candidate values ​​of the remaining nodes are calculated by formula (21) of the following decoding operation, thereby obtaining the specific representation of the candidate values, that is, the nodes in the node set. Obtain candidate scores :

[0203] (twenty one);

[0204] Formula (21) calculates the matching degree (dot product similarity) between the current decoder state (Query) and the feature (Key) of each node in the graph. The higher the matching degree, the higher the candidate score. At the same time, the problem constraints are forced to be met through a mask mechanism (setting the score of infeasible nodes to -∞). For nodes The candidate score is a scalar value used to characterize the node selection in the current state. As a measure of "suitability" or "attractiveness" for the next visit target, the higher the score, the greater the probability of being selected. It is an adjustable constant, for example Used for scaling The output range of the function controls the magnitude of the candidate scores; The hyperbolic tangent activation function maps the input to the (-1,1) interval, introducing nonlinearity to ensure that the network can learn more complex patterns; This represents the learnable parameter matrix used to generate the query vector, with dimensions [missing information]. This belongs to the decoder and maps the decoder's context information to the query space; Indicates the decoder at time step The context vector, with dimensions of It encodes information such as the decision history up to the current moment and the drone's status (such as battery level), and is the hidden state of the decoder; The query vector representing the current time step is obtained through... right The result is obtained by transposing after a linear transformation, with dimensions of ; This represents the learnable parameter matrix used to generate the key vector (Key), with dimensions [missing information]. This maps the node features output by the encoder to the key space; Represents a node The encoded feature vector, with dimension The final encoded output from the encoder The middle corresponds to the node That line contains the high-dimensional semantic information of the node in the task graph; Represents a node The key vector, through right We obtain it by performing a linear transformation, with dimension . ; Scaling factor The dimensions of the query and key vectors are scaled to prevent the dot product result from becoming too large, which would cause the gradient of the Softmax function to vanish and help stabilize the training. This is a very large negative value, i.e., negative infinity. In the masking mechanism, it is assigned to infeasible nodes (e.g., nodes with insufficient power to fly there, or nodes that violate problem constraints). In subsequent Softmax calculations, =0, thus ensuring that the probability of these nodes being selected is 0.

[0205] (3) The obtained candidate values ​​are mapped through a hybrid expert network, and then the softmax function is used to output the next target node. The following formula (22) shows the process of calculating the final node, as follows:

[0206] (twenty two)

[0207] Formula (22) is used to output the probability distribution of the final selection of the next node through a hybrid expert network based on the candidate scores, and Formula (21) is used to obtain the candidate scores. The data is simultaneously fed into two different expert networks for nonlinear transformation, and the gating network is adjusted according to the current power level. and historical charging behavior The model dynamically calculates the weighted mixture of the two expert outputs, and then performs a softmax operation on the final mixture to obtain the probability of selecting each node. This approach allows the model to flexibly adjust its charging and flight decision-making strategies based on real-time conditions (especially battery level), rather than using fixed rules (such as requiring the battery to be fully charged). Let V be a probability distribution vector with dimension |V|, which represents the total number of nodes in the graph. Let V be the _th_ node of the vector. element Indicates the selection of a node As The probability of accessing the target at any given time; The Softmax function transforms a vector of arbitrary real values ​​into a probability distribution vector, where the sum of all elements is 1. A gating network is a small neural network (such as a single-layer fully connected network) whose function is to determine which "expert" should be relied upon more based on the current state. The input is... and The output is a weight vector (e.g., [α, 1-α]) used to mix the outputs of the two expert networks; Indicates the drone at time step The remaining battery charge (normalized to the [0,1] range); A binary flag indicates whether the drone has reached the charging station. It indicates whether the drone has visited the charging station in the current mission (e.g., 1 indicates visited, 0 indicates not visited). This information helps the network learn to adopt different charging strategies at different stages of the mission. This is a vector concatenation operation that concatenates the output vectors of two expert networks along the feature dimension. The network is trained to provide learnable parameter matrices for two expert networks. These are two parallel fully connected layers with different initialization parameters. Each expert network attempts to learn from candidate score vectors. It learns and extracts different decision-making patterns (for example, one expert may be biased towards "aggressive flight" and another towards "conservative charging"). Let V be the candidate score vector, with dimension |V|, which is the candidate score of all nodes calculated by formula (21). The vector formed by the vector.

[0208] S34, charging the drone;

[0209] S35, the UAV ends its coverage mission and returns to the takeoff point, and calculates the objective function to train the network; it determines whether the current number of training rounds has reached the set termination round number. If the termination round number has been reached, the training ends, the trained model is output, and step S4 is executed; otherwise, the network model parameters in the self-attention deep reinforcement learning algorithm are optimized according to the result of the task execution, and the process returns to S32.

[0210] In a preferred embodiment, S35 specifically includes:

[0211] (1) When training network parameters, the baseline parameters are first randomly initialized. The baseline index is obtained through network output. ;

[0212] (2) During the initial training, As training parameters Based on the current parameters and the overall trajectory reward obtained. The REINFORCE method is used to obtain the parameters. The update gradient, for Update according to the following calculation formula (23):

[0213] (twenty three);

[0214] Updated ;

[0215] in, For gradient operators, it means that with respect to parameters Find the partial derivative; Let be the objective function, representing the parameters The expected cumulative reward is calculated based on the given information. The training objective is to maximize... ; The approximation is equal to, indicating that Monte Carlo sampling is used to estimate the expected gradient; Batch Size is the number of independent trajectories (episodes) sampled in each training batch. To index all trajectories in a batch Summation; Indicates the index of the trajectory in the batch. ; For the first A trajectory is the sequence of nodes visited by a drone during a complete mission (including takeoff point, waypoint, charging station, and return to takeoff point). For trajectory The cumulative reward (total return) in drone coverage missions is typically a negative time cost (e.g., Therefore, maximizing the cumulative reward is equivalent to minimizing the total task time; For the first Each task graph instance contains a set of nodes. Edge set and node position set It describes the specific mission scenario (waypoints, charging station distribution, etc.); The baseline function is defined by parameters. Parameterization (usually a neural network) is used to estimate a given graph. It is an approximation of the expected cumulative reward, and its function is to reduce the variance of the gradient estimate; These are the parameters of the baseline network, which are also updated during training; Let be the policy probability, representing the probability given a graph. Under the condition that the policy network (parameters are) Generate trajectory The probability is the product of the probabilities of the policy choosing an action at each time step (based on the Markov property). The logarithm is the probability of the trajectory. Taking the logarithm makes it easier to calculate the gradient and avoids numerical underflow. The score function is the logarithmic probability with respect to the parameters. The gradient indicates how to adjust To increase (or decrease) the trajectory The probability of its occurrence; The advantage function is the estimate of the trajectory. The cumulative reward is the measure of the trajectory’s superiority over the baseline prediction. If it is positive, it means that the trajectory is better than average and its probability should be increased; if it is negative, its probability should be decreased.

[0216] Formula (23) uses the policy gradient theorem to estimate the objective function by sampling a batch of trajectories. The gradient. The gradient direction is estimated by the dominance. Weighting increases the probability of trajectories that yield higher rewards (relative to the baseline) and decreases the probability of those that yield lower rewards. Baseline The introduction of this method can significantly reduce the variance of gradient estimation, thereby stabilizing the training process.

[0217] (3) The baseline is updated using a one-sided t-test in the test set and the baseline metric. The update is only performed when the target value of the vast majority (e.g., more than 60%) of the examples in the dataset is better than the baseline metric. Updated to When the parameters in step S32 are returned to... ;

[0218] (4) Repeat the above parameter update method until the number of iterations reaches the set value; after training, the baseline parameters are... As the final output network parameters.

[0219] To verify the beneficial effects of this embodiment, a numerical simulation experiment was conducted.

[0220] In this simulation environment, 80 target waypoints and 3 charging stations are set up, randomly distributed within a 100x100 three-dimensional area. The UAV's battery capacity is set to 2200mAh, and its power consumption and charging model are set according to the model established in step 201. The battery measurement data is specifically provided by... Figure 4 exhibit.

[0221] The drone achieves optimal trajectory planning for the total mission time in a wide-area three-dimensional environment containing a set of target waypoints to be covered and multiple ground charging stations.

[0222] S4, the UAV calls the trained network model to complete the 3D task scene coverage task, calculates the corresponding flight time and charging time, and evaluates the self-attention deep reinforcement learning algorithm.

[0223] Example 2

[0224] like Figure 2 As shown, this embodiment provides a UAV 3D vision coverage path planning system based on an elastic charging strategy, used to implement the method of Embodiment 1, including:

[0225] Information acquisition module 101 is used to establish the target waypoints to be covered and the location of the ground charging station based on the task terrain, and to acquire a set of waypoints containing the target waypoints and their coordinates;

[0226] The optimization objective establishment module 102 is used to establish the optimization objective and set the constraints for the nonlinear trolleybus path problem based on the charging and power consumption model of the UAV.

[0227] Self-attention deep reinforcement learning module 103 is used to establish the framework of self-attention deep reinforcement learning algorithm;

[0228] The path planning module 104 is used by the UAV to call the trained network model to complete the 3D task scene coverage task, calculate the corresponding flight time and charging time, and evaluate the self-attention deep reinforcement learning algorithm.

[0229] Application Examples

[0230] To verify the beneficial effects of this embodiment, a numerical simulation experiment was conducted. For example... Figure 3 As shown, in this simulation environment, 80 target waypoints and 3 charging stations are set up, randomly distributed within a 100x100 three-dimensional area. The UAV's battery capacity is set to 2200mAh, and its power consumption and charging model are set according to the model established in S21. The battery measurement data are specifically provided by... Figure 5 exhibit. Figure 3 The demonstration showed that the drone traversed a series of shooting waypoints in a mission area with three-dimensional terrain. The drone's energy would decrease as it flew, requiring it to land at a ground-based charging station to recharge. Figure 4 It includes the drone battery under test, a smart charger capable of current control, a digital meter capable of continuously measuring and recording current and voltage data, and a host computer program that works with the meter. Figure 5 The curves showing the specific measurements of the battery's state of charge (SoC) over time are displayed.

[0231] Using the technical solution of this embodiment, through planning that includes a fusion charging strategy, the total task time, including elastic charging decisions, is ultimately generated to be 3015 seconds. The total number of decision time steps for the UAV within this trajectory is... The value was 85, and two charging tasks were performed, at time steps respectively. Batteries 36 and 60 were charged to 67% and 77% of their respective battery capacities. For reference, a fixed charging strategy requiring a full charge each time would result in a total planned task time of 6111 seconds under the same task environment. Therefore, this embodiment, by introducing flexible charging decisions, effectively avoids unnecessary charging waits, reducing the total task time by approximately 50.6% compared to a fixed charging strategy, significantly improving task execution efficiency.

[0232] The present invention also provides a memory that stores multiple instructions for implementing the method as described in Embodiment 1.

[0233] like Figure 6 As shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301. The memory 302 stores a plurality of instructions, which can be loaded and executed by the processor to enable the processor to perform methods as described in Embodiments 2 and 3.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for UAV 3D visual coverage path planning based on an elastic charging strategy, characterized in that, include: S1, establish the target waypoints to be covered based on the mission terrain and deploy the ground charging stations, and obtain the waypoint set containing the target waypoints and their coordinates; S2, Establish the optimization objective and set constraints for the nonlinear trolleybus routing problem based on the charging and power consumption model of UAVs; S3, establishes the framework for self-attention deep reinforcement learning algorithms; S4, the UAV calls the trained network model to complete the 3D task scene coverage task, calculates the corresponding flight time and charging time, and evaluates the self-attention deep reinforcement learning algorithm.

2. The UAV 3D visual coverage path planning method based on an elastic charging strategy according to claim 1, characterized in that, S1 includes: S11, Obtain task environment information according to the requirements of the covered task; S12, establish a topology graph consisting of takeoff points, waypoints, and charging stations, as well as an abstract graph representation of the mission, including a set of all nodes covering the mission. and the set of all edges , and the set of actual locations in space corresponding to the nodes As shown in equation (1): （1）； This includes the set consisting of all nodes. As shown in equation (2): （2）； Each node is assigned a unique number using a serial number system. The serial numbers for takeoff point, charging station, and shooting waypoint are defined as follows (3): （3）； in, Indicates the nodes for drone takeoff and landing; This represents the set of deployed charging stations; This represents the set of waypoints used for ground-based photography coverage. express The set of three-dimensional coordinates of the actual spatial location of the node in the task scene is shown in Equation (4): （4）； Represents a node The set of all edges formed by connecting any two nodes in the array is shown in equation (5): （5）。 3. The UAV 3D visual coverage path planning method based on elastic charging strategy according to claim 2, characterized in that, S2 includes: S21. Establish the energy model required for the nonlinear trolley path problem. The energy model includes the UAV's battery charging model and the UAV's power consumption model, and determine the input information required for the problem, including the UAV's flight speed, upper limit of battery capacity, and lower limit of battery capacity. S22, Establish the optimization objective and constraints for the nonlinear trolley path problem.

4. The UAV 3D visual coverage path planning method based on elastic charging strategy according to claim 3, characterized in that, S21 includes: (1) Establish a battery charging model for the UAV, which is used to calculate the charging time of the UAV given the battery level of the UAV entering the charging station and the battery level of the UAV leaving the charging station; including: by measuring real battery charging data and fitting it with a piecewise linear model, a calculation formula (6) is constructed to estimate the charging time of the UAV: （6）； in, The maximum error of the fit, yes One slope parameter, This represents the number of charging data sampling points that each slope will pass through, indicating that there are... A set of straight lines describing the lithium battery charging model; each set of lines calculates the moment when charging begins. and the moment of stopping charging and leaving That is, to obtain One candidate time to enter the charging station and time of leaving the charging station Finally, the charging time is calculated by taking the maximum value of each of the two candidate times according to formula (7) and calculating the difference. As shown below: （7）； in, Indicates that the drone is at the charging station Charging time; This indicates that the drone is at the mission point or charging station. Battery status at that time; This indicates the time when the drone corresponds to the charging model based on the battery state when entering and leaving the charging station. (2) Establish a power consumption model for the UAV. The power consumption model for the UAV is used to obtain the relationship between flight distance and energy consumption under the known flight speed of the UAV and the total payload of the UAV. An empirical model is adopted, as shown in the following formula (8): （8）； in, The relationship curves between drone speed, payload, and power are derived from existing experiments and can be obtained through sampling in real-world scenarios. It is the random error introduced by the empirical model that satisfies ; Indicates that the drone is from the node Fly to the node Path length; Represents a node Fly to the node The increment of the battery level at that location, with a value range of [0,1].

5. The UAV 3D visual coverage path planning method based on an elastic charging strategy according to claim 4, characterized in that, S22 includes: (1) Determine the optimization objective of the model, as shown in equation (9): （9）； objective function This represents the minimum flight time for each flight path of the drone coverage mission and the minimum charging time for each charging station, assuming a constant speed, and the corresponding flight time and length of each path. Proportional to the number of drones at charging stations Charging time Battery state upon entering and leaving the charging station Related; Indicates that the drone is from the node Fly to the node Time required; Used to indicate that the drone has passed through the node. To the node The path is represented by 1, indicating that the path was visited, and 0, indicating that the path was not visited. Indicates that the drone is at the charging station Charging time; (2) Determine the constraints related to the problem, including: determining the corresponding constraints based on the coverage task and the characteristics of the UAV itself: Constraint 1 is used to guarantee coverage of each waypoint, as shown in equation (10): （10）； Constraint 2 is used to ensure that the drone does not fly continuously to the charging station, as shown in equation (11): （11）； Constraint 3 is used to balance the flight path, ensuring that there is only one complete UAV path loop, as shown in equation (12): （12）； Constraint 4 ensures that the power consumption of the planned path will not exceed the maximum power of the drone before the next charging, as shown in equation (13): （13）； Constraint 5 is used to ensure that the drone receives sufficient charging at the charging station, as shown in equation (14): （14）； Constraint 6 is used to ensure the energy supply of the UAV throughout the entire mission, as shown in equation (15): （15）； Constraint 7 is used to ensure that the battery power of the UAV during flight does not exceed the threshold, as shown in equation (16): （16）； in, This indicates the minimum battery level of the drone during the entire coverage process, ensuring battery safety and providing the drone with a certain amount of energy redundancy. This indicates the highest battery level of the drone during the entire coverage process, ensuring that the battery will not be damaged due to overcharging; This indicates that the drone is at the mission point or entering and leaving the charging station. Battery status at that time; Is the plan in the current state? The amount of electricity charged, i.e., the amount of charge the drone receives at the charging point, is measured in units of energy. It is a variable whose value depends on the power of the charging point and the charging time. Battery characteristics; the definition of the strategy path. This represents a sequence of node visits consisting of a series of node numbers. This indicates the maximum time step in the access sequence of this node; Indicates the drone's location Fly to position The energy consumed.

6. The UAV 3D visual coverage path planning method based on an elastic charging strategy according to claim 5, characterized in that, S3 includes: S31, Establish a Markov decision model based on the task, including a state space. Action space ,award and termination conditions Defined quadruple The formal description of information at each time step simplifies the action space using a critical greedy charging approach, including: (1) Define the state space Including: at the current moment The state space of the intelligent agent is composed of It means that among them yes In the task graph of the agent at any given moment The point in the middle, It indicates its charging status or remaining power. (2) Define the action space Including: in the state Below, the action space of an intelligent agent is defined as ;in Is it an intelligent agent? The nodes that need to be accessed at all times. Is the plan in the current state? The amount of electricity charged, i.e., the amount of electricity charged in time step. The amount of charge a drone receives at a charging point is measured in units of energy. It is a variable whose value depends on the power of the charging point and the charging time. Battery characteristics; when the drone is at a charging point, this charge will be added to its remaining battery power from the previous moment. middle; State transition for the agent from state Transition to state It is based on the current time of the drone. status and actions ; Location updated to In Battery status The transfer conforms to the calculation formula (17): （17）； in, Indicates the drone's location Move to position The energy consumed; For action space It includes the UAV's decision target point and the charge amount at the target point. When making decisions simultaneously with the flight node, there is a coupling between charging decisions and flight decisions. Therefore, the minimum charging criterion is adopted to simplify the action space, which is then simplified to: ,in The estimated energy consumption for the drone's flight from the current charging point to the next charging point in the decision path; the simplified action space makes decisions on the drone's flight and charging simultaneously only through the target point of the drone's decision, which greatly reduces the spatial complexity and decision difficulty of the Markov model. (3) Define rewards This includes: the objective of the drone coverage mission is to minimize the total mission time, and the reinforcement learning objective is to maximize the cumulative reward obtained throughout the Markov process. ,to this end, The reward for each moment is It is defined as the negative value of the time cost incurred during the state transition process, which includes flight time and time spent at the node. The charging time and the corresponding reward function are shown in equation (18): （18）； in, Indicates from position Move to position The required flight time or travel time is a key cost in path planning, which typically depends on the Euclidean distance or actual path distance between two points, the flight speed of the drone, and environmental factors such as wind, obstacles, and airspace restrictions. Indicates at node The charging time or service time is a location-dependent time cost: when It is a charging point ( )hour, This indicates the time required to charge at that point. Charging time depends on the charging power, battery capacity, current battery level, target battery level, and charging efficiency. It is the work point ( )hour, ; (4) Define the termination condition This includes: the Markov process reaching its termination condition when the drone has visited all the waypoints and returned to the takeoff point. The cumulative reward is calculated as shown in equation (19): （19）； S32, the UAV performs the coverage task from the takeoff point according to the waypoint flight path. Based on the Markov decision model described in step S31, the takeoff point is used as the initial position in the model, and the fully charged state is used as the initial battery state in the model; including: The graph data is input into the encoder module of the deep network corresponding to the Markov decision model for encoding, thus encoding the initial state of the UAV. In the input decoder module, the deep network processes the decoder output to obtain the target point for the first flight step; wherein, the encoding operation is completed by formula (20): （20）； The encoder uses three different sets of parameters to perform three rounds of encoding operations to extract high-dimensional information from the graph and obtain the encoded output. ; Formula (20) is the mathematical expression for the multi-head attention mechanism, used to encode input features, where, The input feature matrix has a dimension of . ,in This represents the number of nodes in the graph, including takeoff points, waypoints, and charging stations. The dimension of the feature vector; Indicates the encoder at the 1st The input features of the layer; for the first layer encoder, These are the initial characteristics of the nodes; : No. The learnable parameter matrices of each attention head are used to linearly transform the input features into query, key, and value matrices, respectively. , , usually set ; is the scaling factor, representing the dimension of the key vector, used to scale the dot product result and prevent gradient vanishing or exploding in the softmax function; The softmax function normalizes each row of the matrix so that the sum of the elements in each row is 1. This is used to calculate the attention weights. , ; The concatenation function concatenates the output matrices of multiple attention heads along the feature dimensions. If there are a total of... There are 1 head, and the output dimension of each head is 1. The output dimension after concatenation is ; The number of attention heads, i.e., the number of parallel self-attention mechanisms in multi-head attention; The output of the multi-head attention function, i.e., for the input The feature matrix obtained after multi-head self-attention calculation; S33: Based on the drone's remaining battery power, calculate the expected battery power for the next waypoint using the power consumption model. If the expected battery power is sufficient to reach at least one charging station, estimate the possible battery power for the next state based on the current remaining battery power and decide whether the drone should enter S34 for charging or repeat S33; otherwise, enter S34 for charging; if the drone has traversed all waypoints, return to the takeoff point and execute S35; S33 includes: (1) According to the first Step state Decision No. The algorithm checks if the battery has enough energy to reach a charging station. If it does, the algorithm decides to fly based on its battery status; otherwise, the algorithm stops at the decision stage. During the flight maneuver, all flight shooting points were excluded, and only charging was decided upon. (2) Using a masking method, waypoints and charging stations that do not meet the constraints are masked by the constraints in EVRP-NL and the current battery status of the UAV. The candidate values ​​of the remaining nodes are calculated by formula (21) of the following decoding operation, thereby obtaining the specific representation of the candidate values, that is, the nodes in the node set. Obtain candidate scores : （21）； in, For nodes The candidate score is a scalar value used to characterize the node selection in the current state. As a measure of suitability or attractiveness for the next visit target, the higher the score, the greater the probability of being selected. It is an adjustable constant used for scaling. The output range of the function controls the magnitude of the candidate scores; The hyperbolic tangent activation function maps the input to the interval (-1, 1); This represents the learnable parameter matrix used to generate the query vector, with dimensions [missing information]. This belongs to the decoder and maps the decoder's context information to the query space; Indicates the decoder at time step The context vector, with dimensions of It encodes the decision history and drone status information up to the current moment, and is the hidden state of the decoder; The query vector representing the current time step is obtained through... right The result is obtained by transposing after a linear transformation, with dimensions of ; This represents the learnable parameter matrix used to generate the key vectors, with dimensions of . This maps the node features output by the encoder to the key space; Represents a node The encoded feature vector, with dimension The final encoded output from the encoder The middle corresponds to the node The first line contains the high-dimensional semantic information of the node in the task graph; Represents a node The key vector, through right We obtain it by performing a linear transformation, with dimension . ; Scaling factor It is the dimension of the query and key vector; This is a very large negative value, i.e., negative infinity. In the masking mechanism, it is assigned to those infeasible nodes, and in the subsequent Softmax calculation, =0, thus ensuring that the probability of these nodes being selected is 0; (3) The obtained candidate values ​​are mapped through a hybrid expert network, and then the softmax function is used to output the next target node. The following formula (22) shows the process of calculating the final node, as follows: （22） Formula (22) is used to output the probability distribution of the final selection of the next node through a hybrid expert network based on the candidate scores, and Formula (21) is used to obtain the candidate scores. The data is simultaneously fed into two different expert networks for nonlinear transformation, and the gating network is adjusted according to the current power level. and historical charging behavior The mixed weights of the two expert outputs are dynamically calculated, and the final mixed result is then processed by Softmax to obtain the probability of selecting each node. Let V be a probability distribution vector with dimension |V|, which represents the total number of nodes in the graph. Let V be the _th_ node of the vector. element Indicates the selection of a node As The probability of accessing the target at any given time; The Softmax function transforms a vector of arbitrary real values ​​into a probability distribution vector, where the sum of all elements is 1. For a gated network, the input is and The output is a weight vector used to mix the outputs of the two expert networks; Indicates the drone at time step The remaining battery power is normalized to the [0,1] interval; It indicates whether the drone has reached the charging station, and is a binary flag that indicates whether the drone has visited the charging station in the current mission; This is a vector concatenation operation that concatenates the output vectors of two expert networks along the feature dimension. The network is trained to produce learnable parameter matrices for two expert networks. These are two parallel fully connected layers with different initialization parameters. Let V be the candidate score vector, with dimension |V|, which is the candidate score of all nodes calculated by formula (21). The vector formed; S34, charging the drone; S35, the UAV ends its coverage mission and returns to the takeoff point, and calculates the objective function to train the network; it determines whether the current number of training rounds has reached the set termination round number. If the termination round number has been reached, the training ends, the trained model is output, and step S4 is executed; otherwise, the network model parameters in the self-attention deep reinforcement learning algorithm are optimized according to the result of the task execution, and the process returns to S32.

7. The UAV 3D visual coverage path planning method based on an elastic charging strategy according to claim 6, characterized in that, The S35 includes: (1) When training network parameters, the baseline parameters are first randomly initialized. The baseline index is obtained through network output. ; (2) During the initial training, As training parameters Based on the current parameters and the overall trajectory reward obtained. The REINFORCE method is used to obtain the parameters. The update gradient, for Update according to the following calculation formula (23): （23）； Updated ; in, For gradient operators, it means that with respect to parameters Find the partial derivative; Let be the objective function, representing the parameters The expected cumulative reward is given below, and the training objective is to maximize it. ; The approximation is equal to, indicating that Monte Carlo sampling is used to estimate the expected gradient; This is the batch size, which is the number of independent trajectories sampled in each training batch; To index all trajectories in a batch Summation; Indicates the index of the trajectory in the batch. ; For the first A trajectory is the sequence of nodes visited by a drone during a complete mission; For trajectory In drone coverage missions, the cumulative reward is usually a negative time cost, so maximizing the cumulative reward is equivalent to minimizing the total mission time. For the first Each task graph instance contains a set of nodes. Edge set and node position set It describes the specific task scenario; For the baseline function, by parameters Parameterization, used to estimate a given graph An approximation of the expected cumulative reward is used to reduce the variance of the gradient estimate; These are the parameters of the baseline network, which are also updated during training; Let be the policy probability, representing the probability given a graph. Under the condition that the parameter is The strategy network generates trajectories The probability is the product of the probabilities of the strategy choosing an action at each time step; Let be the logarithmic probability of the trajectory; The score function is the logarithmic probability with respect to the parameter. The gradient indicates how to adjust To increase or decrease the trajectory The probability of its occurrence; The estimate of the dominance function represents the trajectory. The cumulative reward is evaluated relative to the baseline prediction. If the result is positive, it indicates that the trajectory is better than average and its probability should be increased; if the result is negative, its probability should be decreased. (3) The baseline is updated using a one-sided t-test in the test set and the baseline metric. Only when more than 60% of the examples in the dataset have a target value better than the baseline metric will the update be performed. Updated to When the parameters in step S32 are returned to... ; (4) Repeat the above parameter update method until the number of iterations reaches the set value; after training, the baseline parameters are... As the final output network parameters.

8. A UAV three-dimensional vision coverage path planning system based on an elastic charging strategy, used to implement the method according to any one of claims 1-7, characterized in that, include: The information acquisition module (101) is used to establish the target waypoints to be covered and the location of the ground charging station based on the task terrain, and to acquire the set of waypoints containing the target waypoints and their coordinates; The optimization objective establishment module (102) is used to establish the optimization objective and set the constraints for the nonlinear trolleybus path problem based on the charging and power consumption model of the UAV. The self-attention deep reinforcement learning module (103) is used to establish the framework of the self-attention deep reinforcement learning algorithm; The path planning module (104) is used by the UAV to call the trained network model to complete the 3D task scene coverage task, calculate the corresponding flight time and charging time, and evaluate the self-attention deep reinforcement learning algorithm.

9. An electronic device comprising a processor and a memory, the memory storing a plurality of instructions, the processor being configured to read the instructions and execute the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a plurality of instructions, the plurality of instructions being readable by a processor and executing the method of any one of claims 1-7.

Images