Method for automatically generating digital construction scheme of power transmission line project

Abstract

An automatic generation method for a digital construction scheme of a power transmission line project comprises the steps that the digital construction scheme of the power transmission line project is decomposed, an automatic generation model of the digital construction scheme of the power transmission line project is constructed, and the automatic generation model comprises a state space, an action space, an intelligent agent and a reward function; the intelligent agent comprises a strategy network and a value network, the strategy network is used for outputting action probability distribution, the value network is used for outputting a state value, and the reward function comprises an efficiency reward, a quality reward and a security penalty; training the intelligent agent by using a PPO algorithm, and optimizing parameters of the strategy network by minimizing a loss function; and then obtaining an optimal digital construction scheme of the power transmission line project based on the trained automatic generation model. According to the method, deep reinforcement learning and automatic generation of the digital construction scheme of the power transmission line project are combined, and the obtained optimal scheme can improve the construction efficiency, improve the project quality and ensure the construction safety.

Description

Technical Field

[0001] The invention belongs to the technical field of digital construction of power transmission lines, and in particular relates to a method for automatically generating a digital construction plan for a power transmission line project. Background Art

[0002] Power transmission is one of the basic infrastructures of modern society, and the construction of transmission lines is a key link in the development of power systems. Traditional transmission line construction scheme design usually relies on engineers' experience and specifications, and has problems such as low design efficiency, difficulty in cost control, and difficulty in safety assessment. However, with the continuous development of artificial intelligence technology, deep reinforcement learning, as a method that can learn optimal strategies through interaction with the environment, provides new possibilities for the generation of digital construction schemes for transmission line projects.

[0003] The document "Application of multidimensional information model in the design of transmission line construction scheme" (Guo Ting, Zhang Liang, Liao Shuang, et al. Application of multidimensional information model in the design of transmission line construction scheme [J]. Automation and Instrumentation, 2022 (11): 194-198.) designed a transmission line construction scheme based on the multidimensional information model, which improved the economic efficiency while ensuring the construction quality of the transmission line. The document "Research on an intelligent autonomous optimization method for construction parameters of a cutter suction dredger based on reinforcement learning" (Lu Jiajun, Yang Bo, Xu Ting. Research on an intelligent autonomous optimization method for construction parameters of a cutter suction dredger based on reinforcement learning [J]. China Harbor Construction, 2022, 42 (08): 71-75.) proposed an autonomous optimization method for construction parameters of a cutter suction dredger based on reinforcement learning, using machine learning methods to seek the optimal operating parameters of the dredging process and realize autonomous analysis and decision-making of the dredging process. Literature "Research and Development of Decision-making Support System for Live Crossing Construction Scheme of Transmission Line" (Chen Yi, Li Xiaobin, Lin Guanglong, etc. Research and Development of Decision-making Support System for Live Crossing Construction Scheme of Transmission Line [J]. Communication Power Technology, 2019, 36(03): 31-35.) In view of the fact that the existing decision-making support system for the construction scheme of transmission line crossing is restricted by the weight value of the affected factors and cannot accurately decide the optimal crossing construction scheme, the entropy weight method is used to solve the inaccuracy of the weight value, and a fuzzy comprehensive evaluation model based on the entropy weight method is constructed. The decision-making support system for the construction scheme of transmission line crossing is developed, thereby realizing the decision-making support for the construction scheme of crossing. However, the above existing technologies rarely combine deep reinforcement learning with the automatic generation of digital construction schemes for transmission line projects to improve construction efficiency, ensure construction safety and improve project quality. Summary of the invention

[0004] The purpose of the present invention is to provide a method for automatically generating a digital construction plan for a power transmission line project, which can improve construction efficiency, ensure construction safety and improve project quality, in view of the above-mentioned problems existing in the prior art.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A method for automatically generating a digital construction plan for a power transmission line project, the method comprising the following steps:

[0007] S1. Decompose the digital construction plan of the transmission line project and construct an automatic generation model of the digital construction plan of the transmission line project; the automatic generation model includes a state space, an action space, an intelligent agent, and a reward function. The intelligent agent includes a strategy network and a value network. The strategy network is used to output the action probability distribution, and the value network is used to output the state value. The reward function includes efficiency reward, quality reward, and safety penalty;

[0008] S2. Use the PPO algorithm to train the agent and optimize the parameters of the policy network by minimizing the loss function;

[0009] S3. Obtain the optimal digital construction plan for the transmission line project based on the trained automatic generation model.

[0010] The step S2 comprises the following steps:

[0011] S21, establishing an experience replay buffer for the agent;

[0012] S22, initializing the parameters of the strategy network;

[0013] S23, for each training cycle, initialize the state, the agent interacts with the environment, selects actions and calculates reward functions according to the action probability distribution output by the policy network, stores the current state, action, reward and new state in the experience replay buffer; calculates the advantage function and loss function of the current state-action pair using the PPO algorithm, and minimizes the loss function using the gradient descent method to update the parameters of the policy network;

[0014] S24. Repeat step S23 until the loss function converges.

[0015] The loss function is the loss function improved by the dynamic maximum entropy strategy, specifically:

[0016]

[0017] In the above formula, Lppo(θ) represents the loss function; is the mathematical expectation; π θ (a t |st ) is the probability distribution of the action output by the strategy network, indicating that according to the strategy parameter π θ In status t Take action a t The probability distribution of , the strategy parameters refer to the parameters of the strategy network; According to the old policy parameters before the update In status t Take action a t The probability of For state s t Take action a t The advantage function; clip is the clipping function; ε is the clipping coefficient; V θ (s t ) is the state value function output by the value network, indicating that in state s t The expected reward of following the strategy. is the target value function, indicating that in state s t Follow the strategy and execute action a t The expected return obtained; H(π θ (·|s t )) is the strategy π θ In status t The entropy value of the selected action; v , H are the value function loss coefficient and entropy term coefficient respectively; R t+k represents the reward from the current time step t to the future time step k; γ k is the attenuation factor, γ k ∈(0,1).

[0018] The PPO algorithm is an improved PPO algorithm, specifically, the learning rate of the PPO algorithm is improved by using the cosine annealing algorithm. The improved learning rate is:

[0019]

[0020] In the above formula, x t is the learning rate of the current time step; is the minimum learning rate; is the initial learning rate; T mult is the annealing period; π is a constant.

[0021] The reward function is specifically:

[0022] R t =αE(s t ,a t )+βC(s t ,a t )-δw t ;

[0023]

[0024] In the above formula, R t is the reward function of the current time step t; E(s t ,a t ) is in state s t Take action a t efficiency reward; C(s t ,a t ) is in state s t Take action a t Quality reward; w t is the safety penalty during the construction process; α, β, δ are weight coefficients, representing the importance of construction efficiency, project quality and construction safety in the reward function respectively; W c (s t ,a t ) is the construction volume completed within a given construction time; W t (s t ,a t ) is the total construction volume within a given construction time; N d (s t ,a t ) is the number of defects detected; N t (s t ,a t ) is the total number of tests; S i Indicates the severity of the ith accident; N is the number of safety accidents that occurred.

[0025] The strategy performance index is calculated in each training cycle, and the entropy term coefficient in the loss function is dynamically adjusted according to the strategy performance index; the entropy term coefficient in the loss function is dynamically adjusted according to the following strategy:

[0026] If the strategy performance indicator does not improve significantly over multiple periods and fluctuates greatly, increase the entropy coefficient to increase exploration:

[0027] λ H ←λ H +α;

[0028] In the above formula, α is the adjustment step size;

[0029] If the strategy performance indicator is significantly improved and the volatility is small, reduce the entropy coefficient to increase stability and increase exploration:

[0030] λ H ←λ H -α;

[0031] The calculation formula of the strategy performance indicator is:

[0032]

[0033] In the above formula, R i is the reward value of the i-th round, which means the sum of the rewards of all time steps in the i-th round; M is the number of rounds.

[0034] The policy network and the value network share a Bi-GRU layer, through which features are extracted from the input data. At the same time, the policy network and the value network each have an independent fully connected layer as the output layer. The fully connected layer of the policy network uses softmax as the activation function, and the fully connected layer of the value network uses Linear as the activation function.

[0035] The state space includes construction progress, resource status, environmental conditions, safety risks, and construction quality. The expression of the state vector is:

[0036] S={P,T,N e ,N m ,W,K h ,Q};

[0037] In the above formula, P is the percentage of current construction progress, which is used to reflect the completion status of the transmission line project; T is the current construction stage; N e is the number of available mechanical equipment; N m is the amount of available construction materials; W is the current weather conditions; K h is the number of existing safety hazards; Q is the quality assessment of the completed construction project.

[0038] The action space includes basic engineering construction, installation of poles and towers, installation of conductors, laying of cables, construction progress update, and inspection and acceptance. The expression of the action vector is:

[0039] A={A b ,A t ,A w ,A c ,A p ,A q};

[0040] In the above formula, A b For basic engineering construction; t To install the tower; A w To lay the wire; A c To lay cables; A p To update the construction progress; A q To conduct quality inspection and acceptance of the completed construction parts.

[0041] In step S1, the decomposition of the digital construction plan of the transmission line project is specifically as follows: first, the digital construction plan of the transmission line project is decomposed into equipment elements, operation elements, operation intentions, and operation sequences, wherein the equipment elements represent the construction equipment involved in the digital construction plan of the transmission line project, the operation elements represent the basic operations performed using the equipment elements, and the operation intentions refer to the operation purposes to be achieved by combining the equipment elements and the operation elements, and then the equipment elements and the operation elements are combined to form operation intentions, and different operation intentions are superimposed in the operation sequence to form an action sequence of the digital construction plan of the transmission line project.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention discloses a method for automatically generating a digital construction plan for a power transmission line project. The method decomposes the digital construction plan for the power transmission line project and constructs an automatic generation model for the digital construction plan for the power transmission line project. The automatic generation model includes a state space, an action space, an intelligent agent, and a reward function. The intelligent agent includes a strategy network and a value network. The strategy network is used to output the probability distribution of actions, and the value network is used to output the state value. The reward function includes an efficiency reward, a quality reward, and a safety penalty. Then, the intelligent agent is trained using a PPO algorithm to optimize the parameters of the strategy network by minimizing the loss function. The optimal digital construction plan for the power transmission line project is obtained based on the trained automatic generation model. On the one hand, the design can combine deep reinforcement learning with the automatic generation of the digital construction plan for the power transmission line project, and provide a new path for the generation of the digital construction plan for the power transmission line project. On the other hand, a reward function is constructed according to the efficiency reward, the quality reward, and the safety penalty. The intelligent agent is guided to gradually optimize the parameters of the strategy network through the reward function, so that after the training of the automatic generation model is completed, the optimal digital construction plan for the power transmission line project given by the automatic generation model can improve the construction efficiency, improve the engineering quality, and ensure the construction safety. Therefore, the present invention combines deep reinforcement learning with the automatic generation of digital construction plans for transmission line projects, providing a new path for the generation of digital construction plans. The resulting digital construction plans can improve construction efficiency, improve project quality, and ensure construction safety.

[0044] 2. The present invention provides a method for automatically generating a digital construction plan for a power transmission line project, which uses a dynamic maximum entropy strategy to improve the loss function of the PPO algorithm; specifically, an entropy term is added to the loss function, and the entropy term coefficient is dynamically adjusted during the training process. The dynamic adjustment is based on the strategy performance index, and the strategy performance index is obtained by calculating the average reward of the strategy in each training cycle; at the beginning of a training cycle, the initial entropy term coefficient and the adjustment step size are set, and the strategy performance index is calculated after the training cycle ends. If the strategy performance index has not been significantly improved in multiple cycles and has large fluctuations, the entropy coefficient is increased to increase the exploration, and if the strategy performance index has been significantly improved and has low volatility, the entropy coefficient is reduced to increase stability; this design applies the dynamically adjusted entropy term coefficient to the loss function to balance the exploration and development of the algorithm, which helps the algorithm to jump out of the local optimal solution, enhance the exploration and robustness of the strategy, and thus find the global optimal solution, effectively improving the convergence speed and overall performance of the algorithm. Therefore, the present invention can effectively enhance the exploration and robustness of the strategy by dynamically adjusting the entropy term coefficient.

[0045] 3. The present invention discloses a method for automatically generating a digital construction plan for a power transmission line project, which uses a cosine annealing algorithm to improve the learning rate of the PPO algorithm. The traditional PPO algorithm usually uses a fixed learning rate, which may cause instability in the training process or slow convergence in some environments, and is difficult to adapt to the dynamic changes of different environments and tasks. However, the present design uses the smoothness of the cosine function to dynamically adjust the learning rate of the PPO algorithm during the training process. In the early stages of training, cosine annealing provides a higher learning rate to quickly promote algorithm exploration and accelerate convergence. As the training progresses, the learning rate gradually decreases according to the shape of the cosine function, helping the algorithm to dig out potential optimal solutions. When approaching the end of a cycle, the learning rate increases again, which helps the algorithm to jump out of the local optimum that it may fall into, and re-accelerate the search. Therefore, the present invention uses the cosine annealing algorithm to improve the learning rate, which can accelerate the convergence of the algorithm and help the algorithm jump out of the local optimum. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0047] The present invention is further described in detail below in conjunction with specific implementations and drawings.

[0048] Example:

[0049] See also Figure 1 , a method for automatically generating a digital construction plan for a power transmission line project, which is performed in the following steps:

[0050] S1. Decompose the digital construction plan of the transmission line project;

[0051] The purpose of decomposition is to decompose the equipment and operations involved in the construction plan and convert them into quantitative parameters that can be recognized by computers, so as to achieve quantitative analysis of the construction plan and facilitate the recognition and inductive learning of the deep reinforcement learning model;

[0052] The specific steps of decomposition are: firstly, decomposing the digital construction plan of the transmission line project into equipment element, operation element, operation intention, and operation sequence, wherein the equipment element represents the construction equipment involved in the digital construction plan of the transmission line project, the operation element represents the basic operation performed by using the equipment element, and the operation intention refers to the operation purpose to be achieved by combining the equipment element and the operation element, and then combining the equipment element and the operation element to form the operation intention, superimposing different operation intentions according to the operation sequence, and forming the action sequence of the digital construction plan of the transmission line project;

[0053] S2. constructing an automatic generation model of a digital construction plan for a transmission line project based on the decomposition results; the automatic generation model includes a state space, an action space, an intelligent agent, and a reward function;

[0054] The state space includes construction progress, resource status, environmental conditions, safety risks, and construction quality. The expression of the state vector is:

[0055] S={P,T,N e ,N m ,W,K h ,Q};

[0056] In the above formula, P is the percentage of current construction progress, which is used to reflect the completion status of the transmission line project; T is the current construction stage; N e is the number of available mechanical equipment; N m is the amount of available construction materials; W is the current weather conditions; K h is the number of existing safety hazards; Q is the quality assessment of the completed construction project;

[0057] The action space includes basic engineering construction, installation of poles and towers, installation of conductors, laying of cables, construction progress update, and inspection and acceptance. The expression of the action vector is:

[0058] A={A b ,A t ,A w ,A c ,A p ,A q};

[0059] In the above formula, A b For basic engineering construction; t To install the tower; A w To lay the wire; Ac To lay cables; A p To update the construction progress; A q To conduct quality inspection and acceptance of completed construction parts;

[0060] The intelligent agent includes a policy network and a value network; the policy network and the value network share a Bi-GRU layer, and extract features from the input data through the Bi-GRU layer. This is because the bidirectional processing mechanism of Bi-GRU makes it particularly suitable for processing complex time series tasks. By adopting this Bi-GRU architecture, the policy network and the value network can more effectively process time series data and capture the context dependencies, thereby improving the optimization effect of the digital construction plan of the transmission line project. Sharing a Bi-GRU layer can reduce the amount of calculation; at the same time, the policy network and the value network each have an independent fully connected layer as the output layer, and the extracted features are further processed by their own independent fully connected layers. Finally, the policy network outputs the action probability distribution, and the value network outputs the state value; with respect to the output dimension, the output dimension of the policy network is equal to the size of the action space, so its output dimension is 6, and the purpose of the value network is to output the value of the selected action, so its output dimension is 1; with respect to the activation function, the policy network uses softmax as the activation function, the output of each action is between 0 and 1, and the sum of the outputs of all actions is 1, and the value network uses Linear as the activation function;

[0061] S3. Using the PPO algorithm to train the agent; the training is performed according to the following steps:

[0062] S31, initialize the environment, establish an experience replay buffer for the agent, and the experience replay buffer is used to store experience samples collected from the environment;

[0063] S32, initializing the parameters of the strategy network;

[0064] S33. For each training cycle, the state is initialized, the agent interacts with the environment, the policy network uses the features extracted by the Bi-GRU layer to output the action probability distribution, selects and executes a specific action, calculates the reward function, stores the current state, action, reward and new state in the experience replay buffer, and the value network outputs the state value; the PPO algorithm is used to calculate the advantage function and loss function of the current state-action pair, the gradient descent method is used to minimize the loss function to update the parameters of the policy network, and the mean square error is minimized to update the parameters of the value network;

[0065] S34, repeat step S23 until the loss function converges;

[0066] The reward function is constructed with the goal of improving construction efficiency, ensuring construction safety and improving project quality, and is specifically:

[0067] R t =αE(s t ,a t )+βC(s t ,a t )-δw t ;

[0068]

[0069] In the above formula, R t is the reward function of the current time step t; E(s t ,a t ) is in state s t Take action a t efficiency reward; C(s t ,a t ) is in state s t Take action a t Quality reward; w t is the safety penalty during the construction process; α, β, δ are weight coefficients, representing the importance of construction efficiency, project quality and construction safety in the reward function respectively; W c (s t ,a t ) is the construction volume completed within a given construction time; W t (s t ,a t ) is the total construction volume within a given construction time; N d (s t ,a t ) is the number of defects detected; N t (s t ,a t ) is the total number of tests; S i Indicates the severity of the ith accident; N is the number of safety accidents that occurred;

[0070] The PPO algorithm is an improved PPO algorithm, and the improvement is:

[0071] (1) The learning rate of the PPO algorithm is improved by using the cosine annealing algorithm. The improved learning rate is:

[0072]

[0073] In the above formula, x t is the learning rate of the current time step; is the minimum learning rate; is the initial learning rate; T mult is the annealing period; π is a constant, approximately 3.14;

[0074] (2) The dynamic maximum entropy strategy is used to improve the loss function of the PPO algorithm. The improved loss function is as follows:

[0075]

[0076] In the above formula, Lppo(θ) represents the loss function; is the mathematical expectation; π θ (a t |s t ) is the probability distribution of the action output by the strategy network, indicating that according to the strategy parameter π θ In status t Take action a t The probability distribution of , the strategy parameters refer to the parameters of the strategy network; According to the old policy parameters before the update In status t Take action a t The probability of For state s t Take action a t The advantage function; clip is the clipping function; ε is the clipping coefficient; V θ (s t ) is the state value function output by the value network, indicating that in state s t The expected reward obtained by following the strategy; V t target is the target value function, indicating that in state s t Follow the strategy and execute action a t The expected return obtained; H(π θ (·|s t )) is the strategy π θ In status t The entropy value of the selected action; v , H are the value function loss coefficient and entropy term coefficient respectively; R t+k represents the reward from the current time step t to the future time step k; γ k is the attenuation factor, γ k ∈(0,1);

[0077] In the above loss function This is the newly added entropy term. The strategy performance index is calculated in each training cycle, and the entropy term coefficient is dynamically adjusted according to the following strategy:

[0078] If the strategy performance indicator does not improve significantly over multiple periods and fluctuates greatly, increase the entropy coefficient to increase exploration:

[0079] λ H ←λH +α;

[0080] In the above formula, α is the adjustment step size;

[0081] If the strategy performance indicator is significantly improved and the volatility is small, reduce the entropy coefficient to increase stability and increase exploration:

[0082] λ H ←λ H -α;

[0083] The calculation formula of the strategy performance indicator is:

[0084]

[0085] In the above formula, R i is the reward value of the ith round, which means the sum of rewards of all time steps in the ith round; M is the number of rounds; each training cycle generally contains multiple rounds, and a round refers to a complete interaction process in reinforcement learning, that is, the agent starts from the initial state in the environment and goes through a series of actions until it reaches a certain terminal state, and the training cycle refers to the number of parameter updates of the model. Each training cycle will experience multiple rounds of accumulation and processing. t refers to the instantaneous reward at time step t, which represents the single-step reward obtained by the agent after performing an action at a certain moment, and R i represents the cumulative reward of the i-th round, covering the sum of rewards of all time steps in a complete round (i.e., from the initial state to the terminal state);

[0086] S3. Obtain the optimal digital construction plan for the transmission line project based on the trained automatic generation model.

[0087] Performance Verification:

[0088] Four schemes are set up to verify the effectiveness of the method for automatically generating digital construction plans for power transmission line projects proposed in the present invention. Scheme 1: Automatic generation method for digital construction plans for power transmission lines based on GA; Scheme 2: Automatic generation method for digital construction plans for power transmission lines based on DDPG; Scheme 3: Automatic generation method for digital construction plans for power transmission lines based on traditional PPO; Scheme 4: Automatic generation method for digital construction plans for power transmission lines proposed in the present invention. The generated digital construction plans for power transmission lines are compared in terms of construction time, resource utilization, construction cost, engineering quality qualification rate, safety accident rate and plan generation time. The comparison results are shown in Table 1:

[0089] Table 1 Comparison results

[0090]

[0091] It can be seen from Table 1 that the construction time of Scheme 4 is 12.3h, which is 30.1%, 20.1% and 15.2% lower than that of Schemes 1, 2 and 3 respectively; the resource utilization rate of Scheme 4 is 86.4%, which is 16.5%, 8.5% and 4.1% higher than that of Schemes 1, 2 and 3 respectively; the construction cost of Scheme 4 is 18665 yuan, which is 16.3%, 7.46% and 4.37% lower than that of Schemes 1, 2 and 3 respectively; the engineering quality qualification rate of Scheme 4 is 92.4%, which is 12.4%, 8.3% and 3.7% higher than that of Schemes 1, 2 and 3 respectively; the safety accident rate of Scheme 4 is 0.32 / 1000h, which is 39.3%, 26% and 17.8% lower than that of Schemes 1, 2 and 3 respectively; the scheme generation time of Scheme 4 is 30.4s, which is 37.5%, 25.5% and 15.5% lower than that of Schemes 1, 2 and 3 respectively. It can be concluded that the method proposed in the present invention not only shortens the construction time, improves resource utilization, and reduces construction costs, but also significantly improves the engineering quality qualification rate, reduces the safety accident rate and solution generation time, fully demonstrating its advantages in optimizing construction efficiency, safety assurance and decision-making efficiency.

Claims

1. A method for automatically generating a digital construction plan for a power transmission line project, characterized in that: The automatic generation method comprises the following steps: S1. Decompose the digital construction plan of the transmission line project and construct an automatic generation model of the digital construction plan of the transmission line project; the automatic generation model includes a state space, an action space, an intelligent agent, and a reward function. The intelligent agent includes a strategy network and a value network. The strategy network is used to output the action probability distribution, and the value network is used to output the state value. The reward function includes efficiency reward, quality reward, and safety penalty; S2. Use the PPO algorithm to train the agent and optimize the parameters of the policy network by minimizing the loss function; S3. Obtain the optimal digital construction plan for the transmission line project based on the trained automatic generation model.

2. The method for automatically generating a digital construction plan for a power transmission line project according to claim 1, characterized in that: The step S2 comprises the following steps: S21, establishing an experience replay buffer for the agent; S22, initializing the parameters of the strategy network; S23, for each training cycle, initialize the state, the agent interacts with the environment, selects actions and calculates reward functions according to the action probability distribution output by the policy network, stores the current state, action, reward and new state in the experience replay buffer; calculates the advantage function and loss function of the current state-action pair using the PPO algorithm, and minimizes the loss function using the gradient descent method to update the parameters of the policy network; S24. Repeat step S23 until the loss function converges.

3. A method for automatically generating a digital construction plan for a power transmission line project according to claim 1 or 2, characterized in that: The loss function is the loss function improved by the dynamic maximum entropy strategy, specifically: In the above formula, L ppo (θ) represents the loss function; is the mathematical expectation; π θ (a t |s t ) is the probability distribution of the action output by the strategy network, indicating that according to the strategy parameter π θ In status t Take action a t The probability distribution of , the strategy parameters refer to the parameters of the strategy network; According to the old policy parameters before the update In status t Take action a t probability; For state s t Take action a t The advantage function; clip is the clipping function; ε is the clipping coefficient; V θ (s t ) is the state value function output by the value network, indicating that in state s t The expected reward of following the strategy. is the target value function, indicating that in state s t Follow the strategy and execute action a t The expected return obtained; H(π θ (·|s t )) is the strategy π θ In status t The entropy value of the selected action; v , H are the value function loss coefficient and entropy term coefficient respectively; R t+k represents the reward from the current time step t to the future time step k; γ k is the attenuation factor, γ k ∈(0,1).

4. A method for automatically generating a digital construction plan for a power transmission line project according to claim 1 or 2, characterized in that: The PPO algorithm is an improved PPO algorithm, specifically, the learning rate of the PPO algorithm is improved by using the cosine annealing algorithm. The improved learning rate is: In the above formula, x t is the learning rate of the current time step; is the minimum learning rate; is the initial learning rate; T mult is the annealing period; π is a constant.

5. The method for automatically generating a digital construction plan for a power transmission line project according to claim 2, characterized in that: The reward function is specifically: R t =αE(s t ,a t )+βC(s t ,a t )-δw t ; In the above formula, R t is the reward function of the current time step t; E(s t ,a t ) is in state s t Take action a t efficiency reward; C(s t ,a t ) is in state s t Take action a t Quality reward; w t is the safety penalty during the construction process; α, β, δ are weight coefficients, representing the importance of construction efficiency, project quality and construction safety in the reward function respectively; W c (s t ,a t ) is the construction volume completed within a given construction time; W t (s t ,a t ) is the total construction volume within a given construction time; N d (s t ,a t ) is the number of defects detected; N t (s t ,a t ) is the total number of tests; S i Indicates the severity of the ith accident; N is the number of safety accidents that occurred.

6. The method for automatically generating a digital construction plan for a power transmission line project according to claim 3, characterized in that: Calculate the strategy performance index in each training cycle, and dynamically adjust the entropy term coefficient in the loss function based on the strategy performance index; The entropy term coefficient in the loss function is dynamically adjusted according to the following strategy: If the strategy performance indicator does not improve significantly over multiple periods and fluctuates greatly, increase the entropy coefficient to increase exploration: l H ←l H +a; In the above formula, α is the adjustment step size; If the strategy performance indicator is significantly improved and the volatility is small, reduce the entropy coefficient to increase stability and increase exploration: l H ←l H -a; The calculation formula of the strategy performance indicator is: In the above formula, R i is the cumulative reward of the i-th round, which means the sum of the rewards of all time steps in the i-th round; M is the number of rounds.

7. A method for automatically generating a digital construction plan for a power transmission line project according to claim 1 or 2, characterized in that: The policy network and the value network share a Bi-GRU layer, through which features are extracted from the input data. At the same time, the policy network and the value network each have an independent fully connected layer as the output layer. The fully connected layer of the policy network uses softmax as the activation function, and the fully connected layer of the value network uses Linear as the activation function.

8. The method for automatically generating a digital construction plan for a power transmission line project according to claim 1 or 2, characterized in that: The state space includes construction progress, resource status, environmental conditions, safety risks, and construction quality. The expression of the state vector is: S={P,T,N e ,N m ,W,K h ,Q}; In the above formula, P is the percentage of current construction progress, which is used to reflect the completion status of the transmission line project; T is the current construction stage; N e is the number of available mechanical equipment; N m is the amount of available construction materials; W is the current weather conditions; K h is the number of existing safety hazards; Q is the quality assessment of the completed construction project.

9. A method for automatically generating a digital construction plan for a power transmission line project according to claim 1 or 2, characterized in that: The action space includes basic engineering construction, installation of poles and towers, installation of conductors, laying of cables, construction progress update, and inspection and acceptance. The expression of the action vector is: A={A b ,A t ,A w ,A c ,A p ,A q }; In the above formula, A b For basic engineering construction; t To install the tower; A w To lay the wire; A c To lay cables; A p To update the construction progress; A q To conduct quality inspection and acceptance of the completed construction parts.

10. A method for automatically generating a digital construction plan for a power transmission line project according to claim 1 or 2, characterized in that: In step S1, the decomposition of the digital construction plan of the transmission line project is specifically as follows: first, the digital construction plan of the transmission line project is decomposed into equipment elements, operation elements, operation intentions, and operation sequences, wherein the equipment elements represent the construction equipment involved in the digital construction plan of the transmission line project, the operation elements represent the basic operations performed using the equipment elements, and the operation intentions refer to the operation purposes to be achieved by combining the equipment elements and the operation elements, and then the equipment elements and the operation elements are combined to form operation intentions, and different operation intentions are superimposed in the operation sequence to form an action sequence of the digital construction plan of the transmission line project.

Images