An unmanned aerial vehicle path planning method and system based on confidence awareness expert fusion and teacher strategy distillation

Abstract

The application relates to a UAV path planning method and system based on confidence awareness expert fusion and teacher strategy distillation, and belongs to the technical field of UAV autonomous navigation and intelligent decision-making. The method acquires a local environment observation and a flight state of a UAV, constructs a state feature, and determines an actionable set; a plurality of differentiated expert strategies are used to generate an action distribution, after invalid action masking, an expert fusion router is trained according to a normalized distribution entropy, and a teacher action distribution is obtained; a lightweight student strategy is trained by taking the teacher action distribution as a soft supervision signal, and is deployed on a UAV on-board or an edge computing device to output a discrete topological action. The application can realize adaptive expert fusion under a plurality of constraint states, and reduce online reasoning and deployment overheads.

Description

Technical Field

[0001] This invention belongs to the field of autonomous navigation and intelligent decision-making technology for unmanned aerial vehicles (UAVs), and involves deep reinforcement learning, hybrid expert models, action feasibility constraints, confidence-aware expert fusion, and policy distillation techniques. Specifically, it relates to a UAV path planning method and system based on confidence-aware expert fusion and teacher policy distillation. Background Technology

[0002] With the development of low-altitude intelligent networks, industrial IoT, and unmanned systems technologies, drones have been widely used in tasks such as urban inspection, logistics transportation, environmental monitoring, emergency rescue, and operations in hazardous areas. Path planning, as a key component of autonomous drone flight, requires determining reasonable flight maneuvers in real time based on information such as the drone's current location, target location, obstacle distribution, controlled flight path boundaries, and local wind fields.

[0003] In a dynamic, controlled flight path environment, UAVs not only need to reach the target area but also simultaneously meet requirements such as flight path compliance, static obstacle avoidance, dynamic obstacle avoidance, wind disturbance adaptation, and path efficiency. The degree to which various constraints affect path decision-making varies significantly under different flight conditions. For example, near the boundary of a controlled flight path, path planning strategies need to focus on preventing the UAV from deviating from the controlled flight path; in areas with strong local wind fields, path planning strategies need to focus on the impact of wind disturbance on the actual landing point of the UAV; and in areas with dense obstacles, path planning strategies need to increase the priority of obstacle avoidance actions.

[0004] Existing deep reinforcement learning-based UAV path planning methods typically employ a single-policy network, directly mapping the UAV's observed states to flight actions. This single-policy network needs to balance multiple objectives within the same parameter space, such as flight path compliance, wind disturbance adaptation, and obstacle avoidance, which can easily lead to interference between different decision-making objectives. Furthermore, when flight path structure, wind conditions, and obstacle distribution change, the single-policy network struggles to maintain stable path planning performance under different constraints.

[0005] To improve path decision-making capabilities in complex environments, some existing technologies employ hybrid expert models to construct multiple expert strategies with different decision preferences, and select the appropriate expert through gating networks or routing networks. However, existing expert routing methods typically rely on manually defined expert switching rules, scene category labels, or manually labeled routing samples, making them difficult to adapt to UAV path planning tasks where the dominant constraints continuously change with environmental conditions.

[0006] Furthermore, some existing expert fusion methods directly evaluate the output of each expert policy based on the complete action space, without fully considering the infeasibility of actions caused by controlled flight path boundaries, obstacle-occupied areas, and wind disturbance offsets. Actions with high output probabilities in the complete action space may actually be impossible to execute, which may lead the router to make incorrect judgments about the degree of matching between the expert policy and the current flight state.

[0007] Meanwhile, hybrid expert models typically require simultaneous invocation of multiple expert policies and expert routing networks during the online inference phase, resulting in a large number of model parameters and computational overhead, making them difficult to deploy directly on UAV-based computing platforms or edge computing devices with limited computing, storage, and energy resources. Traditional knowledge distillation methods usually use a single action label as the supervision target, making it difficult to fully preserve the action probability relationships formed by multiple expert policies under different constraints.

[0008] Therefore, how to evaluate the matching degree between different expert strategies and the current flight state within the actionable domain, achieve adaptive fusion of the states of multiple expert strategies without relying on manual route labels, and transfer the comprehensive path planning capability of the hybrid expert teacher model to the lightweight student strategy are the technical problems that need to be solved for UAV path planning in dynamic controlled airway environments. Summary of the Invention

[0009] In view of this, the present invention provides a method and system for UAV path planning based on confidence-aware expert fusion and teacher policy distillation.

[0010] This invention performs invalid action masking on the expert action distribution output by each expert policy based on the current set of actionable actions, calculates the normalized distribution entropy of each expert policy in the actionable action domain, and uses the normalized distribution entropy to construct a self-supervised routing loss, thereby training a confidence-aware expert fusion router without the need for manually formulating expert switching rules and providing expert category labels.

[0011] This invention further uses the teacher action distribution obtained by fusing multiple expert policies as a soft supervision signal. By training a lightweight student policy through teacher policy distillation, the UAV can complete discrete topology action decision-making in a dynamic controlled flight path environment without having to call multiple expert policies and confidence-aware expert fusion routers simultaneously during the online execution phase.

[0012] To achieve the above objectives, the present invention provides the following technical solution: A method for UAV path planning based on confidence-aware expert fusion and teacher policy distillation includes the following steps.

[0013] S1. Obtain local environmental observation and flight status information of the UAV at the current decision moment, construct state features based on the local environmental observation and flight status information, and determine the current set of actionable actions based on the controlled flight path range, obstacle-occupied area, and UAV state transition relationship.

[0014] S2, input the state features into multiple expert policies with different flight constraint preferences to obtain the expert action distribution corresponding to each expert policy, and perform invalid action masking and renormalization on each expert action distribution according to the current set of actionable actions to obtain the feasible expert action distribution of each expert policy in the actionable action domain.

[0015] S3, calculate the normalized distribution entropy of each feasible expert action distribution within the feasible action domain, construct a self-supervised routing loss based on the normalized distribution entropy, train a confidence-aware expert fusion router using the self-supervised routing loss, enable the confidence-aware expert fusion router to generate expert fusion weights corresponding to each expert policy based on the state features, and perform weighted fusion of multiple feasible expert action distributions based on the expert fusion weights to obtain the teacher action distribution.

[0016] S4, the local environment observations and their corresponding teacher action distributions are combined into teacher knowledge entries, a teacher knowledge base is constructed based on multiple teacher knowledge entries, and the teacher action distributions are used as soft supervision signals to train lightweight student policies through teacher policy distillation loss.

[0017] S5. The trained lightweight student policy is deployed on the UAV's onboard computing platform or an edge computing device that communicates with the UAV. The lightweight student policy generates a student action distribution based on real-time local environment observations. The student action distribution is then subjected to invalid action masking and renormalization based on the real-time determined set of currently actionable actions. The discrete topology action for the UAV's next decision cycle is determined based on the renormalized student action distribution.

[0018] Furthermore, in S1, the local environment observation and flight status information are represented as follows: (1) In the formula, For the first t Local environmental observations and flight status information at each decision-making moment; This refers to local airspace observation information within the perception range of the UAV; This refers to the topology cell where the drone is currently located. The drone's current position relative to the target's position; The flight speed of the drone; This provides information on local wind fields.

[0019] The local airspace observation information includes at least static obstacle information, dynamic obstacle prediction information, and controlled airway boundary information.

[0020] The local airspace observation information is obtained through a preset environmental perception network. Encode to obtain environmental features and the environmental characteristics Drone flight speed Local wind field information and the relative position of the target Mapping to a unified feature space yields state features: (2) In the formula, State characteristics; This is the environmental feature mapping matrix; This is the flight speed feature mapping matrix; This is the local wind field feature mapping matrix; The target's relative position feature mapping matrix; This is a bias term.

[0021] The preset environment perception network can be a convolutional neural network, a graph neural network, a topological feature coding network, or other network structures with spatial feature extraction capabilities.

[0022] Furthermore, a discrete topology action space is pre-defined. The discrete topological action space It includes six horizontal adjacency actions, ascent actions, descent actions, and hovering actions.

[0023] For discrete topological action space any candidate action in Based on the UAV's current location, the displacement corresponding to the candidate actions, local wind field information, and the decision cycle, the UAV's candidate actions are predicted. Predicting the next position: (3) In the formula, This is the current location of the drone; Candidate actions Local wind field The corresponding prediction of the next position under the influence; Candidate actions The corresponding displacement; This is the wind disturbance impact coefficient; The decision cycle is the period between two adjacent decision moments.

[0024] Based on the predicted next location and the continuous motion segments traversed by the drone within the current decision-making cycle, the current set of possible actions is determined according to the following formula: (4) In the formula, Create a set of currently available actions; For the next moment, it is a space suitable for flight; This refers to the conservatively occupied area of ​​obstacles within the current decision-making cycle. For the drone from its current location Move to the predicted next position The continuous line segments traversed; This represents the empty set.

[0025] When candidate actions If the predicted next position is within the flyable space at the next moment, and the continuous motion segment does not intersect with the conservatively occupied area of ​​the obstacle, then the candidate action is selected. Add to the current set of available actions Otherwise, the candidate action will be... It was determined to be an inaction.

[0026] Furthermore, in S2, the multiple expert strategies with different flight constraint preferences include at least a route compliance expert, a wind disturbance resistance expert, and an obstacle avoidance expert.

[0027] Each expert strategy shares the state features and discrete topology action space, and each has its own independent policy network parameters and different policy training reward preferences.

[0028] The channel compliance expert increased the weight of the channel deviation penalty in the strategy training rewards, so that when the drone approaches the boundary of the controlled channel, it will prioritize the action of staying inside the controlled channel.

[0029] The anti-wind disturbance expert increases the weight of wind disturbance impact and energy consumption items in the strategy training rewards, enabling the UAV to adjust its flight maneuver preferences according to local wind speed and direction, thereby reducing the impact of local wind fields on the actual landing point and flight energy consumption of the UAV.

[0030] The obstacle avoidance expert increases the weight of the near-distance penalty for static and dynamic obstacles in the strategy training reward, so that the UAV will prioritize actions that keep away from the area occupied by obstacles in areas with dense obstacles.

[0031] Each expert policy can be pre-trained using a proximal policy optimization algorithm. After the expert policies are pre-trained, the policy network parameters of each expert policy are frozen.

[0032] No. i The distribution of expert actions output by each expert strategy is represented as follows: (5) In the formula, For the first i An expert strategy in local environmental observation and flight status information Next, select candidate actions a The probability of; For the first i The strategy network corresponding to each expert strategy; For the first i The policy network parameters of an expert policy.

[0033] Based on the current set of actionable actions Define the action feasibility indicator function: (6) According to the following formula, the first i The distribution of expert actions output by each expert strategy is analyzed by performing an invalid action mask and renormalization: (7) In the formula, For the first i The distribution of feasible expert actions corresponding to each expert strategy; For summation action variables in the discrete topological action space; This is a numerically stable term used to avoid a denominator of zero.

[0034] By using an invalid action mask, the probability of non-actions is set to zero, and the probability of actionable actions is renormalized, thereby evaluating the action output of each expert strategy within the current actionable domain.

[0035] Furthermore, in S3, the first i The distribution entropy of the feasible expert action distribution corresponding to each expert strategy is expressed as: (8) Considering that the number of available actions may differ at different decision points, the distribution entropy is normalized based on the current number of available actions to obtain the first... i Normalized distribution entropy corresponding to each expert strategy: (9) In the formula, For the first i The normalized distribution entropy corresponding to each expert strategy; Set up current actionable actions The number of actions in the process; This indicates taking the maximum value.

[0036] By adopting The number of actions in the normalization term can prevent the logarithmic term from being zero when the current number of available actions is one.

[0037] The smaller the normalized distribution entropy, the more concentrated the distribution of feasible expert actions of the corresponding expert strategy, and the clearer the expert strategy's judgment on the optimal action in the current state; the larger the normalized distribution entropy, the more dispersed the distribution of feasible expert actions of the corresponding expert strategy.

[0038] Confidence-aware expert fusion routers based on state features As input, output the expert matching score, which is the same as the number of expert strategies: (10) In the formula, A confidence-aware expert fusion router; Parameters of the fusion router are used for confidence-aware experts; Match score vectors to experts.

[0039] The expert matching scores are normalized using a normalized exponential function to obtain the expert fusion weights corresponding to each expert strategy: (11) In the formula, For the first i The expert matching score corresponding to each expert strategy; N The number of expert strategies; j The summation number for the expert strategy; For the first i Each expert strategy corresponds to an expert fusion weight, and satisfies the following: .

[0040] Construct a self-supervised routing loss based on the normalized distribution entropy corresponding to each expert strategy: (12) In the formula, This is the loss for self-supervised routing; For route training batches; This represents the number of samples in the routing training batch.

[0041] When training the confidence-aware expert fusion router, the policy network parameters of each expert's policy are frozen, and the self-supervised routing loss is used as the basis for the training. Update the parameters of the confidence-aware expert fusion router .

[0042] Minimize self-supervised routing loss This allows the confidence-aware expert fusion router to assign larger expert fusion weights to expert policies with smaller normalized distribution entropy, thereby achieving adaptive matching between each expert policy and the current flight state without the need for manual routing labels.

[0043] The distributions of multiple feasible expert actions are weighted and fused according to the fusion weights of each expert to obtain the expert fusion action distribution: (13) In the formula, For expert-integrated action distribution.

[0044] The expert fusion action distribution is then subjected to invalid action masking and renormalization again to obtain the teacher action distribution: (14) In the formula, Distribution of teacher actions.

[0045] The teacher action distribution not only includes information on the priority action to be performed in the current state, but also retains the probabilistic relationship between multiple possible actions.

[0046] Furthermore, in S4, local environmental observations and corresponding teacher action distributions are combined to form teacher knowledge entries: (15) In the formula, For the first t One teacher knowledge item.

[0047] A teacher knowledge base is constructed based on multiple teacher knowledge items: (16) In the formula, For teachers' knowledge base; M This refers to the number of teacher knowledge entries in the teacher knowledge base.

[0048] Local environmental observation samples are collected under different initial positions, target positions, obstacle distributions, controlled channel structures, and local wind field conditions. These local environmental observation samples are then input into a hybrid expert teacher model composed of multiple expert strategies and confidence-aware expert fusion routers to obtain the corresponding teacher action distribution.

[0049] The obtained local environmental observations and teacher action distributions are stored in the teacher knowledge base, and the teacher knowledge base is divided into training set and validation set according to a preset ratio.

[0050] The lightweight student strategy takes local environment observations as input and outputs a distribution of student actions corresponding to a discrete topological action space. (17) In the formula, The distribution of student actions output for a lightweight student strategy; A strategy for lightweight students; Parameters for lightweight student strategies.

[0051] Using teacher action distribution as a soft supervision signal, a teacher strategy distillation loss is constructed: (18) In the formula, Distillation loss for teacher strategies; Distillation training batches obtained by sampling from the teacher knowledge base; This represents the number of samples in the distillation training batch.

[0052] By minimizing the distillation loss of the teacher's strategy This allows the student action distribution to approximate the teacher action distribution, thereby transferring the action probability relationship formed by the hybrid expert teacher model under different constraint states such as channel compliance, wind disturbance adaptation, and obstacle avoidance to the lightweight student strategy.

[0053] After training the lightweight student strategy, fix the parameters of the lightweight student strategy. .

[0054] Furthermore, in S5, the trained lightweight student policies are deployed on the drone's onboard computing platform or on edge computing devices that communicate with the drone.

[0055] During the online execution phase, the hybrid expert teacher model, multiple expert strategies, confidence-aware expert fusion router, and teacher knowledge base do not participate in online inference.

[0056] The drone acquires real-time local environmental observations at the current decision-making moment and outputs the student action distribution through a lightweight student strategy. The current set of possible actions is recalculated based on real-time controlled channel boundaries, obstacle occupancy, and local wind field information.

[0057] Perform invalid action masking and renormalization on the student action distribution according to the following formula: (19) In the formula, This represents the student action distribution after invalid action masking and renormalization.

[0058] The action with the highest probability is selected from the renormalized student action distribution and used as the discrete topological action for the next decision cycle of the UAV: (20) In the formula, This refers to the discrete topological actions of the UAV in the next decision cycle.

[0059] This invention also provides a UAV path planning system based on confidence-aware expert fusion and teacher strategy distillation, including a state feature construction module, a feasible action determination module, a differentiated expert strategy module, an invalid action masking and renormalization module, a confidence-aware expert fusion routing module, a teacher action distribution generation module, a teacher knowledge base construction module, a teacher strategy distillation module, and an edge path decision module.

[0060] The state feature construction module is used to obtain local environmental observation and flight state information of the UAV at the current decision moment, and construct state features based on the local environmental observation and flight state information.

[0061] The action determination module is used to determine the current set of actionable actions based on the controlled flight path range, the area occupied by obstacles, and the state transition relationship of the UAV.

[0062] The differentiated expert strategy module includes multiple expert strategies with different flight constraint preferences, which are used to generate corresponding expert action distributions based on state characteristics.

[0063] The invalid action masking and renormalization module is used to perform invalid action masking and renormalization on the distribution of each expert action based on the current set of possible actions, so as to obtain the feasible expert action distribution corresponding to each expert strategy.

[0064] The confidence-aware expert fusion routing module is used to calculate the normalized distribution entropy of each feasible expert action in the feasible action domain, construct a self-supervised routing loss based on the normalized distribution entropy, and generate expert fusion weights corresponding to each expert strategy based on the state characteristics.

[0065] The teacher action distribution generation module is used to perform weighted fusion of multiple feasible expert action distributions according to the expert fusion weights to obtain the teacher action distribution.

[0066] The teacher knowledge base construction module is used to combine local environmental observations and their corresponding teacher action distributions into teacher knowledge entries, and to construct a teacher knowledge base based on multiple teacher knowledge entries.

[0067] The teacher policy distillation module is used to train lightweight student policies using the teacher action distribution as a soft supervision signal and through teacher policy distillation loss.

[0068] The edge path decision module is used to generate a student action distribution based on real-time local environment observations using a trained lightweight student policy, perform invalid action masking and renormalization on the student action distribution based on the real-time determined set of currently actionable actions, and determine the discrete topology action of the UAV in the next decision cycle based on the renormalized student action distribution.

[0069] The output of the state feature construction module is connected to the differentiated expert strategy module and the confidence-aware expert fusion routing module, respectively; the output of the actionable action determination module is connected to the invalid action masking and renormalization module, the teacher action distribution generation module, and the edge path decision module, respectively; the differentiated expert strategy module is connected to the teacher action distribution generation module through the invalid action masking and renormalization module; the confidence-aware expert fusion routing module is connected to the teacher action distribution generation module; the teacher action distribution generation module is connected to the teacher strategy distillation module through the teacher knowledge base construction module; and the teacher strategy distillation module is connected to the edge path decision module.

[0070] The differentiated expert strategy module, invalid action masking and renormalization module, confidence-aware expert fusion routing module, teacher action distribution generation module, teacher knowledge base construction module, and teacher strategy distillation module are configured to run during the offline training phase.

[0071] The edge path decision module is configured to call the trained lightweight student policy to make discrete topology action decisions during the online execution phase, and to stop calling the differentiated expert policy module, the confidence-aware expert fusion routing module, and the teacher knowledge base construction module during the online execution phase.

[0072] The present invention also provides an electronic device for UAV path planning, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the above-mentioned UAV path planning method based on confidence-aware expert fusion and teacher policy distillation.

[0073] The beneficial effects of this invention are as follows: (1) Perform invalid action masking and renormalization on the distribution of each expert action based on the current set of actionable actions, and calculate the normalized distribution entropy of the expert action distribution within the actionable action domain. This can reduce the interference of unactionable actions on the judgment of the degree of matching of expert strategies and improve the consistency between the expert confidence evaluation results and the actual actionable actions of the UAV.

[0074] (2) The self-supervised routing loss is constructed by utilizing the normalized distribution entropy of each expert strategy in the action domain. There is no need to manually formulate expert switching rules, nor is there a need to provide expert category labels or manually labeled routing training samples. It can adaptively generate expert fusion weights based on the current flight status of the UAV.

[0075] (3) By generating teacher action distribution through multiple expert strategies with different flight constraint preferences, the action preferences of airway compliance experts, wind disturbance experts and obstacle avoidance experts can be integrated to improve the path decision adaptability of UAVs in controlled airway boundaries, strong wind disturbance areas and dense obstacle areas.

[0076] (4) Using teacher action distribution as a soft supervision signal to train lightweight student strategies can transfer the action probability relationship formed by the fusion of multiple expert strategies to lightweight student strategies. Compared with using only a single action label for supervision, it can retain more teacher strategy decision information.

[0077] (5) Multiple expert strategies, confidence-aware expert fusion routers and teacher knowledge bases are set up for use in the offline training phase. In the online execution phase, only the trained lightweight student strategies are called, which can reduce the model storage and online inference overhead of UAV onboard computing platforms or edge computing devices.

[0078] (6) In the online execution phase, the present invention redetermines the current set of actionable actions and performs invalid action masking and renormalization on the student action distribution, which can avoid the direct execution of non-actionable actions output by the lightweight student strategy and improve the executability of the UAV path planning results.

[0079] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0080] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the overall architecture of the UAV path planning system based on confidence-aware expert fusion and teacher policy distillation of the present invention; Figure 2 This is a schematic diagram of the process for generating differentiated expert strategies and teacher action distribution in this invention. Figure 3 This is a schematic diagram illustrating the teacher knowledge base construction, lightweight student strategy distillation, and edge-end online decision-making process of this invention. Detailed Implementation

[0081] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0082] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0083] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0084] Example 1: Unmanned Aerial Vehicle (UAV) Path Planning Method like Figure 1 As shown, this embodiment provides a UAV path planning method based on confidence-aware expert fusion and teacher policy distillation. The method includes an offline training phase and an online execution phase.

[0085] The offline training phase includes state feature construction, action determination, differential expert policy pre-training, confidence-aware expert fusion router training, teacher action distribution generation, teacher knowledge base construction, and lightweight student policy distillation.

[0086] During the online execution phase, only the trained lightweight student policy is invoked, and the current set of actionable actions determined in real time is used to make discrete topology action decisions for the UAV.

[0087] S1: State Feature Construction and Action Determination At the current decision-making moment, local airspace observation information within the drone's perception range can be obtained through onboard sensing devices or environmental perception devices connected to the drone.

[0088] The local airspace observation information includes at least static obstacle information, dynamic obstacle prediction information, and controlled airway boundary information. The static obstacle information may include the location and occupied area of ​​buildings, poles, mountains, or other fixed obstacles. The dynamic obstacle prediction information may include the predicted location and predicted occupied area of ​​other UAVs or moving objects within the current decision-making period.

[0089] At the same time, it acquires the topology cell where the drone is currently located, the drone's current position, the target position, the drone's flight speed, and local wind field information.

[0090] Construct local environmental observation and flight status information according to formula (1) .

[0091] Local airspace observation information is obtained through a pre-set environmental sensing network. Encode to obtain environmental features According to formula (2), environmental features, flight speed, local wind field information, and target relative position are mapped to a unified feature space to obtain state features. .

[0092] In this embodiment, the discrete topology action space includes six horizontal adjacency actions, ascent actions, descent actions, and hovering actions.

[0093] For any candidate action, based on the current position of the UAV, the displacement corresponding to the candidate action, the local wind field, and the decision cycle, the predicted next position of the UAV after executing the candidate action is predicted according to formula (3).

[0094] The flyable space for the next moment is determined based on the controlled flight path boundary and the conservatively occupied area of ​​obstacles. If the predicted next position corresponding to a candidate action is within the flyable space for the next moment, and the continuous motion line segment traversed by the UAV from the current position to the predicted next position does not intersect with the conservatively occupied area of ​​obstacles, then the candidate action is added to the current set of actionable actions.

[0095] If the predicted next position is outside the controlled flight path, or if the predicted next position falls into the obstacle conservatively occupied area, or if the continuous movement line of the UAV intersects the obstacle conservatively occupied area, then the corresponding candidate action will be determined as an unplayable action.

[0096] S2: Differentiated Expert Strategy Pre-training like Figure 2 As shown, this embodiment constructs a waterway compliance expert, a wind disturbance resistance expert, and an obstacle avoidance expert.

[0097] Each expert policy uses the same state features as input and outputs an expert action distribution corresponding to the discrete topological action space. However, each expert policy uses independent policy network parameters and different policy training reward preferences.

[0098] During the training of navigation compliance experts, the weight of navigation deviation penalties is increased, so that when a drone approaches the boundary of a controlled navigation channel, navigation compliance experts will assign a higher probability of actions that are far from the boundary of the controlled navigation channel or remain within the controlled navigation channel.

[0099] During the training of wind disturbance experts, the weights of wind disturbance impact and energy consumption items are increased, enabling wind disturbance experts to adjust the probability of different flight maneuvers based on the wind speed and direction of the local wind field, thereby reducing the impact of wind disturbance on the actual landing point and flight energy consumption of the UAV.

[0100] During the training of obstacle avoidance experts, the weight of the close-range penalty for both static and dynamic obstacles is increased, so that when the drone approaches the area conservatively occupied by the obstacle, the obstacle avoidance expert will assign a higher probability of actions that move away from the obstacle.

[0101] Each expert policy can be pre-trained using a proximal policy optimization algorithm. After the expert policies are pre-trained, the policy network parameters of each expert policy are frozen.

[0102] Input the state characteristics into the waterway compliance expert, wind disturbance expert and obstacle avoidance expert respectively, and obtain the expert action distribution corresponding to each expert strategy according to formula (5).

[0103] Based on the current set of actionable actions determined by S1, the action feasibility indicator function is defined according to formula (6), and invalid action masking and renormalization are performed on the distribution of each expert action according to formula (7) to obtain the distribution of feasible expert actions of each expert strategy in the current actionable action domain.

[0104] S3: Confidence-Aware Expert Fusion Calculate the distribution entropy of each feasible expert action distribution according to formula (8), and normalize the distribution entropy according to the current number of feasible actions according to formula (9).

[0105] Under the same decision-making state, if the normalized distribution entropy of an expert strategy is small, it indicates that the action distribution of the expert strategy within the current actionable domain is relatively concentrated, and it has a clear selection tendency for one or a few actions; if the normalized distribution entropy of an expert strategy is large, it indicates that the selection tendency of the expert strategy for multiple actions under the current state is relatively dispersed.

[0106] The confidence-aware expert fusion router takes state features as input. In this embodiment, the confidence-aware expert fusion router can employ a two-layer fully connected network. The first layer is used to extract state-related routing features, and the second layer outputs expert matching scores equal to the number of expert policies.

[0107] The expert matching score vector is obtained according to formula (10), and the expert matching score is normalized according to formula (11) to obtain the expert fusion weight corresponding to each expert strategy.

[0108] The routing training batch is constructed from the state samples collected during the expert strategy pre-training stage, and the self-supervised routing loss is calculated according to formula (12).

[0109] When training the confidence-aware expert fusion router, keep the parameters of the environment-aware network, the state feature mapping layer, and each expert policy fixed, and only update the parameters of the confidence-aware expert fusion router.

[0110] To minimize the self-supervised routing loss, the confidence-aware expert fusion router tends to assign larger expert fusion weights to expert policies with smaller normalized distribution entropy.

[0111] For example, when a drone is located near the boundary of a controlled airway and the airway compliance expert outputs a relatively concentrated action distribution within the current actionable domain, the confidence-aware expert fusion router can improve the expert fusion weight of the airway compliance expert.

[0112] When a drone is in a strong local wind field and the anti-wind disturbance expert outputs a relatively concentrated action distribution within the current actionable domain, the confidence perception expert fusion router can improve the expert fusion weight of the anti-wind disturbance expert.

[0113] When the drone is in an area with dense obstacles, and the obstacle avoidance expert outputs a relatively concentrated action distribution within the current actionable domain, the confidence perception expert fusion router can improve the expert fusion weight of the obstacle avoidance expert.

[0114] The expert fusion action distribution is obtained by weighting and fusing multiple feasible expert action distributions according to formula (13). The invalid action mask and renormalization are then performed on the expert fusion action distribution again according to formula (14) to obtain the teacher action distribution.

[0115] S4: Teacher Knowledge Base Construction and Teacher Strategy Distillation like Figure 3 As shown, local environmental observation samples were collected under different initial positions, target positions, obstacle distributions, controlled channel structures, and local wind field conditions.

[0116] Input each local environmental observation sample into the hybrid expert teacher model to obtain the teacher action distribution corresponding to each local environmental observation sample.

[0117] According to formula (15), each local environmental observation and the corresponding teacher action distribution are combined into teacher knowledge items, and according to formula (16), multiple teacher knowledge items constitute a teacher knowledge base.

[0118] The lightweight student strategy takes local environment observations as input and outputs the student action distribution corresponding to the discrete topological action space.

[0119] Distillation training batches are sampled from the teacher knowledge base, the teacher strategy distillation loss is calculated according to formula (18), and the parameters of the lightweight student strategy are updated based on the teacher strategy distillation loss.

[0120] Through multiple rounds of iterative training, the student action distribution output by the lightweight student strategy is made to approximate the teacher action distribution output by the hybrid expert teacher model.

[0121] The lightweight student strategy model has fewer parameters and less computational cost than the hybrid expert teacher model, which is composed of multiple expert strategies and a confidence-aware expert fusion router.

[0122] S5: Edge-end Online Path Decision The trained lightweight student strategies are deployed on drone-borne computing platforms or edge computing devices that communicate with drones.

[0123] During the online execution phase, the drone acquires observations of the current local environment and outputs the distribution of student actions through a lightweight student strategy.

[0124] The current set of possible actions is redefined based on the real-time controlled channel boundary, obstacle occupancy, and local wind field, and invalid action masking and renormalization are performed on the student action distribution according to formula (19).

[0125] According to formula (20), the action with the highest probability is selected from the renormalized student action distribution and used as the discrete topological action of the UAV in the next decision cycle.

[0126] The UAV executes the determined discrete topology actions and, upon entering the next decision cycle, reacquires local environmental observations and repeats the above online path decision-making process until the UAV reaches the target area or meets the preset mission termination conditions.

[0127] Example 2: Unmanned Aerial Vehicle (UAV) Path Planning System like Figure 1As shown, this embodiment provides a UAV path planning system based on confidence-aware expert fusion and teacher policy distillation, including a state feature construction module, a feasible action determination module, a differentiated expert policy module, an invalid action masking and renormalization module, a confidence-aware expert fusion routing module, a teacher action distribution generation module, a teacher knowledge base construction module, a teacher policy distillation module, and an edge path decision module.

[0128] The state feature construction module constructs state features based on UAV local environment observation and flight state information, and sends the state features to the differentiated expert strategy module and the confidence perception expert fusion routing module, respectively.

[0129] The action determination module determines the current set of actionable actions based on the controlled flight path range, obstacle-occupied area, local wind field, and UAV state transition relationship, and sends the current set of actionable actions to the invalid action mask and renormalization module, the teacher action distribution generation module, and the edge path decision module.

[0130] The differentiated expert strategy module includes waterway compliance experts, wind disturbance resistance experts, and obstacle avoidance experts, which are used to generate corresponding expert action distributions based on state characteristics.

[0131] The invalid action masking and renormalization module performs invalid action masking and renormalization on each expert action distribution based on the current set of feasible actions, and sends the obtained feasible expert action distributions to the confidence-aware expert fusion routing module and the teacher action distribution generation module, respectively.

[0132] The confidence-aware expert fusion routing module calculates the corresponding normalized distribution entropy based on the action distribution of each feasible expert, generates expert fusion weights based on state features, and uses the normalized distribution entropy to construct a self-supervised routing loss to train the confidence-aware expert fusion router.

[0133] The teacher action distribution generation module performs weighted fusion of multiple feasible expert action distributions based on expert fusion weights, and performs invalid action masking and renormalization on the fusion result to obtain the teacher action distribution.

[0134] The teacher knowledge base construction module combines local environmental observations and their corresponding teacher action distributions into teacher knowledge entries, and constructs a teacher knowledge base based on multiple teacher knowledge entries.

[0135] The teacher strategy distillation module samples teacher knowledge items from the teacher knowledge base and uses the distribution of teacher actions as a soft supervision signal to train lightweight student strategies.

[0136] The edge path decision module is equipped with a trained lightweight student policy, which outputs the student action distribution based on real-time local environment observations and combines it with the current set of actionable actions to determine the discrete topological action of the UAV in the next decision cycle.

[0137] During the offline training phase, the differentiated expert policy module, invalid action masking and renormalization module, confidence-aware expert fusion routing module, teacher action distribution generation module, teacher knowledge base construction module, and teacher policy distillation module are run to complete multiple expert policy pre-training, expert fusion router training, and lightweight student policy distillation.

[0138] During the online execution phase, the edge path decision module calls the trained lightweight student policy to make drone path decisions, without having to call the differentiated expert policy module, the confidence-aware expert fusion routing module, and the teacher knowledge base construction module.

[0139] Example 3: Unmanned Aerial Vehicle (UAV) Path Planning Electronic Equipment This embodiment provides an electronic device for unmanned aerial vehicle (UAV) path planning, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor.

[0140] The memory is used to store computer programs, UAV local environment observation data, flight status information, lightweight student strategy parameters, and intermediate data generated during path planning.

[0141] When the processor executes the computer program stored in the memory, it executes the UAV path planning method based on confidence-aware expert fusion and teacher policy distillation as described in Embodiment 1.

[0142] The electronic device can be installed on the UAV's onboard computing platform or on an edge computing device that communicates with the UAV.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A UAV path planning method based on confidence-aware expert fusion and teacher strategy distillation, characterized in that, Includes the following steps: S1, acquire local environmental observation and flight status information of the UAV at the current decision moment, construct state features based on the local environmental observation and flight status information, and determine the current set of actionable actions based on the controlled flight path range, obstacle-occupied area and UAV state transition relationship; S2, input the state features into multiple expert policies with different flight constraint preferences to obtain the expert action distribution corresponding to each expert policy, and perform invalid action masking and renormalization on each expert action distribution according to the current set of actionable actions to obtain the feasible expert action distribution of each expert policy in the actionable action domain. S3, calculate the normalized distribution entropy of each feasible expert action distribution within the feasible action domain, construct a self-supervised routing loss based on the normalized distribution entropy, train a confidence-aware expert fusion router using the self-supervised routing loss, enable the confidence-aware expert fusion router to generate expert fusion weights corresponding to each expert strategy based on the state features, and perform weighted fusion of multiple feasible expert action distributions based on the expert fusion weights to obtain the teacher action distribution; S4, the local environment observations and their corresponding teacher action distributions are combined into teacher knowledge entries, a teacher knowledge base is constructed based on multiple teacher knowledge entries, and the teacher action distributions are used as soft supervision signals to train lightweight student policies through teacher policy distillation loss. S5. The trained lightweight student policy is deployed on the UAV's onboard computing platform or an edge computing device that communicates with the UAV. The lightweight student policy generates a student action distribution based on real-time local environment observations. The student action distribution is then masked and renormalized based on the real-time determined set of currently available actions. The discrete topology action for the UAV's next decision cycle is determined based on the renormalized student action distribution. 2.The method of claim 1, wherein, The local environmental observation and flight status information are represented as follows: wherein, a local airspace observation information within a perception range of the UAV, the local airspace observation information comprising at least static obstacle information, dynamic obstacle prediction information, and controlled airway boundary information; a topological element in which the UAV is currently located; a target relative position of a current position of the UAV relative to a target position; a flight speed of the UAV; a local wind field information; The local space observation information is encoded by a preset environment perception network to obtain environment features and the state features are constructed according to the following formula: wherein, is the state feature, , , and are the corresponding feature mapping matrices, is the bias term. 3.The UAV path planning method of claim 1, wherein, The current set of possible actions is determined according to the following formula: wherein, is a current actionable set, is a discrete topological action space, is a candidate action, is a current position of the UAV, is a candidate action under a local wind field corresponding predicted next position, is a flyable space at the next time, is a conservative occupancy area of the obstacle within the current decision period, is a continuous motion segment of the UAV from the current position to the predicted next position; The predicted next position is determined according to the following formula: wherein, is a candidate action a corresponding displacement, is a wind disturbance influence coefficient, is a decision period; The discrete topology action space includes six horizontal adjacency actions, ascent actions, descent actions, and hovering actions. 4.The method of claim 3, wherein, The multiple expert strategies with different flight constraint preferences include at least a route compliance expert, a wind disturbance resistance expert, and an obstacle avoidance expert. In the strategy training reward of the route compliance expert, the weight of the route deviation penalty item is greater than the weight of the corresponding penalty item in other expert strategies. In the strategy training reward of the wind disturbance resistance expert, the weight of the wind disturbance impact item and the energy consumption item is greater than the weight of the corresponding item in other expert strategies. In the strategy training reward of the obstacle avoidance expert, the weight of the static obstacle and dynamic obstacle proximity penalty item is greater than the weight of the corresponding penalty item in other expert strategies. Define an action feasibility indication function based on the current set of possible actions: and invalid action masking and renormalization is applied to the expert action distribution output by the nth expert policy according to the following formula: i expert action distribution = expert action distribution * (1 - invalid action mask) wherein, is the i expert policy based on the local environment observation output candidate action a probability, is the i feasible expert action distribution corresponding to the is a numerical stabilizer. 5.The UAV path planning method of claim 4, wherein, No. i The normalized distribution entropy corresponding to each expert strategy is determined according to the following formula: Where |A_t| is the number of actions in the current set of available actions; The confidence-aware expert fusion router generates expert matching scores based on the state features, and generates expert fusion weights corresponding to each expert strategy according to the following formula: wherein, is the confidence-aware expert fusion router, is a parameter for the confidence-aware expert fusion router, is an expert match score vector, is an expert match score for the i th expert policy, N is a number of expert policies, is an expert fusion weight for the i th expert policy; The self-supervised routing loss is determined according to the following formula: wherein, For the routing training batch, the parameters of each expert policy are frozen when training the confidence-aware expert fusion router, and the parameters of the confidence-aware expert fusion router are updated according to the self-supervised routing loss. 6.The UAV path planning method of claim 5, wherein, The distributions of multiple feasible expert actions are weighted and fused according to the fusion weights of each expert to obtain the expert fusion action distribution: The teacher action distribution is obtained by performing invalid action masking and renormalization on the expert fusion action distribution according to the following formula: wherein, is the expert fusion action distribution, is the teacher action distribution; The aforementioned local environmental observations and teacher action distributions are combined to form teacher knowledge entries: And a teacher knowledge base is constructed based on multiple teacher knowledge items: wherein, is the teacher knowledge base, M is the number of teacher knowledge items in the teacher knowledge base. 7.The UAV path planning method of claim 6, wherein, The lightweight student strategy outputs the distribution of student actions based on local environment observations: wherein, is the light student policy, is a parameter of the light student policy; The teacher strategy distillation loss is determined according to the following formula: wherein, is a distilled training batch obtained from sampling from the teacher knowledge base; During the online execution phase, the student action distribution is masked for invalid actions and renormalized according to the following formula: The discrete topological action of the UAV in the next decision cycle is determined according to the following formula: wherein, is a discrete topological action for the next decision cycle of the UAV, and the plurality of expert policies, the confidence-aware expert fusion router, and the teacher knowledge base are not invoked in the online execution phase.

8. A UAV path planning system based on confidence-aware expert fusion and teacher strategy distillation, characterized in that, include: The state feature construction module is used to acquire local environmental observation and flight state information of the UAV at the current decision moment, and construct state features based on the local environmental observation and flight state information; The action determination module is used to determine the current set of actionable actions based on the controlled flight path range, the area occupied by obstacles, and the state transition relationship of the UAV; The differentiated expert strategy module includes multiple expert strategies with different flight constraint preferences, used to generate corresponding expert action distributions based on the state characteristics; The invalid action masking and renormalization module is used to perform invalid action masking and renormalization on the distribution of each expert action according to the current set of actionable actions, so as to obtain the feasible expert action distribution corresponding to each expert strategy. The confidence-aware expert fusion routing module is used to calculate the normalized distribution entropy of each feasible expert action in the feasible action domain, construct a self-supervised routing loss based on the normalized distribution entropy, and generate expert fusion weights corresponding to each expert strategy based on the state features. The teacher action distribution generation module is used to weight and fuse multiple feasible expert action distributions according to the expert fusion weights to obtain the teacher action distribution; The teacher knowledge base construction module is used to compose local environmental observations and their corresponding teacher action distributions into teacher knowledge entries, and to construct a teacher knowledge base based on multiple teacher knowledge entries. The teacher strategy distillation module is used to train lightweight student strategies using the teacher action distribution as a soft supervision signal and through teacher strategy distillation loss. The edge path decision module is used to generate a student action distribution based on real-time local environment observations using a trained lightweight student policy, perform invalid action masking and renormalization on the student action distribution based on the real-time determined set of currently actionable actions, and determine the discrete topology action of the UAV in the next decision cycle based on the renormalized student action distribution.

9. The UAV path planning system of claim 8, wherein, The differentiated expert strategy module, invalid action masking and renormalization module, confidence-aware expert fusion routing module, teacher action distribution generation module, teacher knowledge base construction module, and teacher strategy distillation module are configured to run during the offline training phase; the edge path decision module is configured to call the trained lightweight student strategy to make discrete topology action decisions during the online execution phase, and to stop calling the differentiated expert strategy module, confidence-aware expert fusion routing module, and teacher knowledge base construction module during the online execution phase. 10.An unmanned aerial vehicle path planning electronic device, comprising: The unmanned aerial vehicle path planning method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and the processor implements the unmanned aerial vehicle path planning method according to any one of claims 1 to 7 when the computer program is executed.

Images