Proximity policy optimization (ppo) based distribution network field survey path planning method

Abstract

The application provides a distribution network field survey path planning method based on a proximal policy optimization (PPO), and relates to the technical field of unmanned aerial vehicle path planning.Through inputting survey area coordinates and a topographic map in an unmanned aerial vehicle, a deployment point of the unmanned aerial vehicle is initialized, and an environment state containing position coordinates, target distance, recognition confidence and an identified target index is constructed; the environment state is input into a pre-trained PPO model, the model adopts a moving action network structure, a discrete action network outputs a device recognition instruction, a continuous action network outputs a flight speed vector, and a critic network evaluates a state value; a comprehensive reward function is designed to guide the unmanned aerial vehicle to efficiently complete a survey task while avoiding repeated identification.The application significantly improves the intelligent level and operation efficiency of distribution network survey through collaborative processing of discrete identification decisions and continuous flight control.

Description

Technical Field

[0001] This invention relates to a method for planning the path of a distribution network field survey based on near-end strategy optimization of PPO, and belongs to the field of UAV path planning technology. Background Technology

[0002] With the rapid development of drone technology, its application in power distribution network field surveying is becoming increasingly widespread. Utilizing drones for automated surveying can effectively overcome terrain obstacles, improve data acquisition efficiency, and reduce labor costs. However, power distribution network environments are typically complex, containing various elements such as power poles, transformers, and meter boxes, and their distribution is irregular. Achieving fully automated path planning for drones, enabling them not only to fly safely and avoid obstacles, but also to intelligently identify, locate, and prioritize the survey of critical equipment, is an extremely challenging task. Traditional path planning methods often fall short when dealing with such complex decision-making problems involving multiple objectives and constraints.

[0003] In recent years, deep reinforcement learning, especially the Proximal Policy Optimization (PPO) algorithm, has been introduced into UAV path planning due to its outstanding performance in complex sequential decision-making problems. Existing technologies, such as the Chinese invention patent application CN117193378B, disclose a multi-UAV path planning method based on an improved PPO algorithm. By introducing methods such as replay buffers and V-trace, it effectively improves sample utilization and training stability, focusing on solving the problem of multiple UAVs coordinating obstacle avoidance and reaching the target point in space. However, the action space design of the above-mentioned technical solutions is relatively simple (mainly velocity vectors), and the state space and reward function focus on evaluating physical information such as distance and collision during flight. Its single policy network output cannot simultaneously handle the discrete decision of selecting and recognizing targets and the continuous decision of controlling flight attitude, resulting in insufficient adaptability in complex survey scenarios requiring precise operation and intelligent decision-making.

[0004] Therefore, there is an urgent need for a path planning method that can deeply integrate discrete device identification and decision-making with continuous flight control, and perform refined state and reward design for the characteristics of power distribution network survey scenarios. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a distribution network field survey route planning method based on near-end strategy optimization of PPO.

[0006] The technical solution of the present invention is as follows: A method for planning the route of a distribution network field survey based on near-end strategy optimization of PPO, the method comprising: Input the pre-set information for field survey into the UAV, including the latitude and longitude coordinates of the field survey space and the topographic map of the field survey area; after the UAV analyzes the topographic map of the field survey area to obtain ground feature information, it observes and constructs the initial environmental state; The initial environmental state is input into the pre-trained parallel action network structure PPO model to obtain a hybrid action including target priority decision command and flight speed vector command; The UAV reaches the target location according to the hybrid action execution path, records the target coordinates based on the discrete actions in the hybrid action, re-observes the environment to update the initial environmental state, inputs the updated environmental state into the PPO model to generate a new hybrid action, reaches the next target location according to the new hybrid action execution path and obtains the corresponding environmental state of the next target location, updates the hybrid action based on the corresponding environmental state of the next target location and records the target coordinates, until the final target location is reached and the survey is completed.

[0007] Preferably, the Parallel Action Network (PPO) model includes a performer network and a critic network. The performer network includes a discrete action network and a continuous action network. The discrete action network is used to output decision commands related to target identification of distribution network elements, and the continuous action network is used to output speed commands related to UAV flight control. The critic network is used to evaluate the action value of the UAV in the current environmental state and update the parameters of the performer network based on the action value.

[0008] Preferably, the process of pre-training the PPO model includes: The pre-training phase consists of multiple training loops, each containing multiple time periods. During each time period, the UAV observes the environmental state, selects and executes a hybrid action, and stores the memory containing the current environmental state, the selected action, the obtained reward value, and the next environmental state in the experience replay pool. After all time periods of a training loop are completed, the parameters of the discrete action network, continuous action network, and critic network in the performer network are updated by sampling the importance of the memories in the experience replay pool. The training loop is repeated until the action decision accuracy and path planning efficiency of the PPO model meet the preset requirements, thus completing the pre-training.

[0009] Preferably, the build environment states include: The longitude, latitude, and elevation coordinates provided by the UAV's GPS module are converted into three-dimensional spatial coordinates to form the UAV's position vector; the weighted product of the relative distance from the UAV to each identified target and the corresponding target identification confidence level is calculated to form the target distance and confidence level vector; the index numbers of the currently successfully identified distribution network element targets are used to construct the identified target index vector; the UAV position vector, the target distance and confidence level vector, and the identified target index vector are concatenated to form the initial environmental state.

[0010] Preferably, the reward value is expressed by the formula: ; In the formula, For the first The reward value per episode; , , and These are empirical parameters; For the first Before the start of the episode, the drone reached the non-repeating identification stage. The weighted product of the relative distance to each distribution network survey target and the confidence level of the target identification; To explore completion status; For time-related parameters; For drones in the Total energy consumption for performing hybrid actions within an Episode; For the first Before the start of the episode, the drone was identified as having repeated issues. The weighted product of the relative distance to each distribution network survey target and the confidence level of the target identification; for The total number of distribution network elements repeatedly identified within a single episode.

[0011] Preferably, the loss function for updating the parameters of the discrete action network. Expressed as a formula: ; in: ; ; In the formula, These are the parameters of the discrete action network; For the first The expectation of each episode; For new discrete action strategies And the old discrete action strategy The ratio between; For the first The status of each episode; for Clipping function; For the generalized advantage estimator (GAE); These are the hyperparameters of the clipping function; and These are discrete action strategies and their corresponding weights; For parameters Defined discrete action strategy; For the first Discrete actions of each episode.

[0012] Preferably, the loss function for updating the continuous action network. Expressed as a formula: ; In the formula, The ratio between the new continuous action strategy and the old continuous action strategy; and These are the continuous action strategies and their corresponding weights; For continuous action network parameters; For parameters Defined continuous action strategy.

[0013] Preferably, the loss function for updating the critic network. Expressed as a formula: ; In the formula, Parameters for the critic network; In the state The predicted value below; This is the target value.

[0014] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the distribution network field survey path planning method based on near-end strategy optimization PPO as described in the present invention.

[0015] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the distribution network field survey path planning method based on near-end strategy optimization PPO as described in the present invention.

[0016] The present invention has the following beneficial effects: 1. This invention relates to a method for path planning in distribution network field surveys based on Proximal Point Optimization (PPO). It constructs and trains a PPO model with a parallel action network structure. This PPO model decomposes the performer network into a discrete action network and a continuous action network. The parallel action network structure satisfies the mixed action requirements of decision-making and control in distribution network surveys, enabling the simultaneous output of the optimal equipment identification strategy and the smoothest flight path, thus enhancing the algorithm's adaptability in complex scenarios. Furthermore, the discrete action network is specifically responsible for outputting decision commands related to the identification of distribution network elements. The UAV actively judges and executes identification operations for key distribution network equipment through an independent discrete action network, making path planning serve the survey business itself and improving the intelligence and accuracy of survey decision-making. The continuous action network is specifically responsible for outputting velocity vector commands related to UAV flight control. 2. This invention is a method for planning the path of power distribution network field survey based on near-end strategy optimization (PPO). It models the field survey task of UAVs as a Markov decision process with a complex state space, which includes the UAV's position, relative distance, target recognition confidence, and index of identified targets. When designing the reward function, it comprehensively considers the exploration completion degree, time consumption, energy consumption, and repeated recognition penalty, thereby improving the efficiency of UAVs in learning the optimal strategy, avoiding ineffective hovering and repetitive work, and improving the efficiency of field survey operations. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 A schematic diagram of the performer network structure. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0020] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0022] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0023] Example 1: like Figure 1 As shown, this embodiment provides a method for distribution network field survey route planning based on near-end strategy optimization of PPO. The method includes: S1. Input the pre-set information for field survey into the UAV, including the latitude and longitude coordinates of the field survey area and the topographic map of the field survey area, wherein: After receiving the pre-set information for field survey, the UAV determines the boundary outline of the survey area based on the latitude and longitude coordinates of the field survey space. The flight trajectory of the UAV must not exceed the range defined by the latitude and longitude coordinates. Meanwhile, the UAV processes the topographic map of the field survey area using its built-in topographic map analysis function. Based on the features such as the outlines and symbols of the features on the topographic map, it extracts and identifies the location information of trees, mountains, rivers, fields, and houses (such as the concentrated distribution area of ​​trees, the scattered distribution range of houses, and the quadrant of the mountains). This forms a list of features, enabling the UAV to establish a preliminary understanding of the geographical environment of the survey area. Based on the list of features, the UAV plans its subsequent path to avoid obstacles and selects the target observation angle. Furthermore, after the UAV analyzes the topographic map of the field survey area to obtain ground feature information, it combines the latitude and longitude coordinates of the field survey area to select locations within the field survey area that meet the initial deployment conditions as initial deployment points for subsequent observations. Specifically: Exclude areas marked in the land cover information list that have concentrated tree distribution, mountainous areas (slope > 15°), areas within 3m of building complexes, and areas within 5m of riverbanks to ensure that the initial deployment point has no obvious obstruction or safety risks; at the same time, the initial deployment point must be at least 20m away from the boundary line of the survey area to avoid the drone's field of view exceeding the survey range during subsequent initial observations; the three-dimensional coordinates of the final determined initial deployment point are pre-recorded by the drone's built-in GPS module; After the initial deployment point is determined, the UAV activates its sensor module at that point to observe and construct the initial environmental state. This includes preliminary detection of surrounding power distribution network survey targets (pole towers, transformers, metering boxes, pole-mounted equipment, and guy wires on concrete poles) using a visual sensor, obtaining the relative distance between the UAV and each target, as well as the recognition confidence level of each target (output by the target detection model on the visual sensor); simultaneously, it retrieves the three-dimensional coordinates of the initial deployment point pre-recorded by the GPS module as the UAV's position parameters in the initial environmental state; since no surveying action has been performed yet, the index of identified targets in the initial environmental state is set to an empty set. Finally, the relative distances of the UAV to each target, the confidence level of each target, the UAV's position parameters, and the index of identified targets in the air set are integrated to form a complete initial environmental state; S2. In this embodiment, the field survey and exploration of the UAV is modeled as a Markov decision process. The initial environmental state is input into the pre-trained parallel action network structure PPO model to obtain a hybrid action including target priority decision command and flight speed vector command. S21. The PPO model includes a performer network and a critic network, such as... Figure 2 As shown, the performer network includes a discrete action network and a continuous action network. The discrete action network, continuous action network, and critic network all consist of an input layer, a hidden layer, and an output layer. The hidden layer contains two fully connected layers. The number of neurons in the input layer and the two fully connected layers are 256, 128, and 64, respectively. The hidden layer and output layer of the discrete action network are divided into... One branch, The number of discrete action types; Among them, the discrete action network is used to output decision commands related to target identification of distribution network elements, the continuous action network is used to output speed commands related to UAV flight control, and the critic network is used to evaluate the action value of the UAV in the current environmental state and update the parameters of the performer network based on the action value. S22. After modeling the UAV's field survey and exploration as a Markov decision process, the PPO model can be trained, wherein: The Markov decision process consists of quadruples Definition: Where This represents the set of all possible system states, that is, the set of all possible values ​​of the environmental state observed during field surveying and exploration. In UAV field surveying and exploration, a fixed time period is defined as an Episode. In this Episode, the UAV will observe the current environment and take actions. The completion of the actions is considered the end of an Episode. The environmental state obtained from the observation is expressed by the formula: ; in: ; ; ; ; In the formula, For the first The environmental conditions are obtained by observing the environment before each episode begins; For the first The three-dimensional coordinates of the drone's position parameters before each episode begins , The spatial x-coordinate is the result of longitude conversion. The vertical coordinate is the spatial coordinate after latitude conversion. The vertical coordinates are the spatial coordinates after elevation transformation; For the first The relative distances of the drones to each target before the start of each episode, where, For the first Before each episode begins, the drone identifies the target. Relative distance and target identification The weighted product of confidence levels. To identify the total number of targets, For drones to identify targets The relative distance corresponds to the weight. For the first Before each episode begins, the drone identifies the target. The relative distance, To identify the target The recognition confidence level corresponds to the weight. For the first Identify the target before each episode begins. The degree of certainty in identification; when hour For an empty set, when hour In the first The index of the distribution network elements successfully identified by the drone within each episode is the index of the identified target, which is numbered starting from 1. In the specific settings of this embodiment, the current field environment records a total of 15 pole / tower locations (including branch poles and transformer double poles), 17 meter box locations, 6 pole-mounted devices, and 21 concrete pole guy wires, which constitutes the number of identified target types. ; In the quadruple This represents the set of all possible actions that the drone may take, which in this embodiment is a hybrid action space consisting of discrete and continuous actions; This represents the state transition probability function, which describes the probability distribution of the environment transitioning to another state after taking a certain action in a specific state; The reward value function fed back to the drone represents the value of each action in a certain environment; This modeling transforms the complex survey route planning problem into a sequential decision problem that can find the optimal strategy through reinforcement learning. S23. The process of pre-training the PPO model includes: S231. Initialize the parameters of the PPO model and the corresponding experience replay pool; S232. The pre-training phase is set to include multiple training cycles, and each training cycle includes multiple time periods (i.e., episodes). In this embodiment, the training phase is set to run for a total of 600 cycles, and each cycle consists of 200 episodes. S233, Regarding the first The first episode will be the first one. Each episode drone serves as the current episode, observing the environment and obtaining its state before the current episode begins. The environmental state is input into the PPO model to obtain the hybrid action selected by the UAV within the current episode. ; Furthermore, the process of obtaining mixed actions is specifically as follows: S2331, the drone will be in the... The environmental state obtained from observations before the start of each episode The input is fed into the discrete action network, and the discrete action network's... Each of the branches outputs a discrete action, and the discrete action network's _th _ ... The discrete actions of each branch output are... ,in, , In this embodiment, This is a discrete action space, which contains optional discrete actions, such as hovering, shooting, high-precision search, and return to home. These actions can be omitted in actual use if they are not needed. S2332, the drone will be in the... The environmental state obtained from observations before the start of each episode Input is fed into a continuous motion network, and output is a continuous motion space. Continuous actions , ,in, , and These are the lateral movement velocity vector, the longitudinal movement velocity vector, and the vertical movement velocity vector, which determine the direction vector of the UAV's flight. S2333, the drone in the Within an episode, from the discrete action space Discrete actions selected for each mobile device and from the continuous action space The continuous actions selected for each mobile device are combined to form the drone's actions in the first step. The blend action selected from the blend action space within each episode , ; S234. The drone performs a hybrid maneuver within the current episode. After the mixed actions are completed, the environmental state changes according to the state transition probabilities in the Markov decision process, and the drone receives a reward value within the current episode. And observe the environment at the end of the current episode and before the start of the next episode to obtain the environmental state. ; Furthermore, the reward value Expressed as a formula: ; In the formula, , , and As an empirical parameter, it assigns importance to the components of different reward values; at this point... For the first Before the start of the episode, the drone reached the non-repeating identification stage. The weighted product of the relative distance to each distribution network survey target and the confidence level of the target identification; To determine the degree of completion, for example, if the number of poles and towers in the field survey is known, the degree of completion can be the number of poles and towers that have been identified and located divided by the total number of poles and towers, or it can be the degree of exploration for that field area. This is a time-related parameter; the shorter the time elapsed during the drone's operation, the larger this parameter will be. For drones in the The total energy consumption of performing mixed actions within an episode (such as flight energy consumption and equipment operation energy consumption). For the first Before the start of the episode, the drone was identified as having repeated issues. The weighted product of the relative distance to each distribution network survey target and the confidence level of the target identification; for The total number of repeatedly identified distribution network elements within a single episode; S235. The drone stores its own memories in the experience replay pool within the current episode. In the middle, the content to be memorized is ; S236. After all episodes of a training cycle have ended, the memories in the experience replay pool are sampled for importance, and the network parameters of the discrete action network, continuous action network, and critic network in the performer network of the PPO algorithm are updated, where: S2361, Discrete action strategy is defined as follows: The continuous action strategy is defined as follows: In the formula, and These are the parameters for discrete action networks and the parameters for continuous action networks, respectively. For the index of the action subspace; Loss function of discrete action networks Expressed as a formula: ; in: ; ; In the formula, For the first The expectation of each episode; For new discrete action strategies And the old discrete action strategy The ratio between; for Clipping function; This is the Generalized Advantage Estimator (GAE), used to estimate the advantage of an action, i.e., how good or bad the action is relative to the average action. This is a hyperparameter of the pruning function, used to avoid excessively large policy updates; in this embodiment, it is set to 0.1. and These are discrete action strategies and their corresponding weights; S2362, Loss Function of Continuous Action Networks Expressed as a formula: ; In the formula, The ratio between the new continuous action strategy and the old continuous action strategy; and These are the continuous action strategies and their corresponding weights; S2363. The goal of the critic network is to minimize the difference between predicted and observed values ​​to evaluate the current state and assist the performer network in updating its parameters. Its parameter update loss function... Expressed as a formula: ; In the formula, Parameters for the critic network; This is a predicted value; The target value; It is worth noting that, for ease of understanding Markov decision-making, the formula in step S236 is... Indicates the first The state of an episode, in this embodiment, is essentially the environment state. ; S237. Repeat the training loop until the action decision accuracy and path planning efficiency of the PPO model meet the preset requirements to complete the pre-training. Preferably, in this embodiment, the optimizer is the Adam optimizer. S3. The UAV conducts field surveys and explorations based on the aforementioned hybrid actions, specifically: The UAV reaches the target location according to the hybrid action execution path, records the target coordinates based on the discrete actions in the hybrid action, re-observes the environment to update the initial environmental state, inputs the updated environmental state into the PPO model to generate a new hybrid action, reaches the next target location according to the new hybrid action execution path and obtains the corresponding environmental state of the next target location, updates the hybrid action based on the corresponding environmental state of the next target location and records the target coordinates, until the final target location is reached and the survey is completed.

[0024] Example 2: This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the distribution network field survey path planning method based on near-end strategy optimization PPO as described in any embodiment of the present invention.

[0025] Example 3: This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the distribution network field survey path planning method based on near-end strategy optimization PPO as described in any embodiment of the present invention.

[0026] It is worth noting that the system, electronic device, and computer-readable storage medium described in this invention are all based on the same inventive concept as the method described in this invention, and will not be described again here.

[0027] In this embodiment of the invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0028] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0029] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0030] In several embodiments provided by this invention, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0031] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for distribution network field survey route planning based on near-end strategy optimization of PPO, characterized in that, The method includes: Input the pre-set information for field survey into the UAV, including the latitude and longitude coordinates of the field survey space and the topographic map of the field survey area; after the UAV analyzes the topographic map of the field survey area to obtain ground feature information, it observes and constructs the initial environmental state; The initial environmental state is input into a pre-trained Parallel Action Network (PPO) model to obtain a hybrid action including target priority decision commands and flight velocity vector commands. The pre-training process of the PPO model includes: The pre-training phase consists of multiple training loops, each containing multiple time periods. During each time period, the UAV observes the environmental state, selects and executes a hybrid action, and stores the memory containing the current environmental state, the selected action, the obtained reward value, and the next environmental state in the experience replay pool. After all time periods of a training loop are completed, the parameters of the discrete action network, continuous action network, and critic network in the performer network are updated by sampling the importance of the memories in the experience replay pool. The training loop is repeated until the action decision accuracy and path planning efficiency of the PPO model meet the preset requirements, thus completing the pre-training. The UAV reaches the target position according to the hybrid action execution path, records the target coordinates based on the discrete actions in the hybrid action, re-observes the environment to update the initial environmental state, inputs the updated environmental state into the PPO model to generate a new hybrid action, reaches the next target position according to the new hybrid action execution path and obtains the environmental state corresponding to the next target position, updates the hybrid action based on the environmental state corresponding to the next target position and records the target coordinates, until the final target position is reached and the survey is completed. Among them, the loss function for updating the parameters of the discrete action network. Expressed as a formula: ； in: ； ； In the formula, These are the parameters of the discrete action network; For the first The expectation of each episode; For new discrete action strategies And the old discrete action strategy The ratio between; For the first The status of each episode; for Clipping function; For the generalized advantage estimator (GAE); These are the hyperparameters of the clipping function; and These are discrete action strategies and their corresponding weights; For parameters Defined discrete action strategy; For the first Discrete actions of each episode.

2. The distribution network field survey route planning method based on near-end strategy optimization of PPO as described in claim 1, characterized in that, The Parallel Action Network (PPO) model includes a performer network and a critic network. The performer network comprises a discrete action network and a continuous action network. The discrete action network outputs decision commands related to target identification of distribution network elements, while the continuous action network outputs speed commands related to UAV flight control. The critic network evaluates the action value of the UAV under the current environmental conditions and updates the parameters of the performer network based on the action value.

3. The distribution network field survey route planning method based on near-end strategy optimization of PPO as described in claim 1, characterized in that, The build environment includes: The longitude, latitude, and elevation coordinates provided by the UAV's GPS module are converted into three-dimensional spatial coordinates to form the UAV's position vector; the weighted product of the relative distance from the UAV to each identified target and the corresponding target identification confidence level is calculated to form the target distance and confidence level vector; the index numbers of the currently successfully identified distribution network element targets are used to construct the identified target index vector; the UAV position vector, the target distance and confidence level vector, and the identified target index vector are concatenated to form the initial environmental state.

4. The distribution network field survey route planning method based on near-end strategy optimization of PPO as described in claim 1, characterized in that, The reward value is expressed by the formula: ； In the formula, For the first The reward value per episode; , , and These are empirical parameters; For the first Before the start of the episode, the drone reached the non-repeating identification stage. The weighted product of the relative distance to each distribution network survey target and the confidence level of the target identification; To explore completion status; For time-related parameters; For drones in the Total energy consumption for performing hybrid actions within an Episode; For the first Before the start of the episode, the drone was identified as having repeated issues. The weighted product of the relative distance to each distribution network survey target and the confidence level of the target identification; for The total number of distribution network elements repeatedly identified within a single episode.

5. The distribution network field survey route planning method based on near-end strategy optimization of PPO as described in claim 1, characterized in that, Update the loss function of the continuous action network Expressed as a formula: ； In the formula, The ratio between the new continuous action strategy and the old continuous action strategy; and These are the continuous action strategies and their corresponding weights; For continuous action network parameters; For parameters Defined continuous action strategy.

6. The distribution network field survey route planning method based on near-end strategy optimization of PPO as described in claim 1, characterized in that, Update the loss function of the critic network Expressed as a formula: ； In the formula, Parameters for the critic network; In the state The predicted value below; This is the target value.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the distribution network field survey route planning method based on near-end strategy optimization of PPO as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the distribution network field survey route planning method based on near-end strategy optimization of PPO as described in any one of claims 1 to 6.

Images