Digital twinborn model construction method of hydropower station maintenance teaching system

Abstract

The invention discloses a digital twinborn model construction method for a hydropower station maintenance teaching system, and belongs to the technical field of hydropower station simulation maintenance teaching, and the method comprises the following steps: S1, obtaining historical operation state data of hydropower station equipment, and building a training set; s2, constructing a digital twinborn model according to the training set; and S3, simulating the operation process characteristics of the hydropower station in real time through the digital twinborn model, and completing the construction of the digital twinborn model of the hydropower station maintenance teaching system. According to the reinforcement learning-based hydropower station equipment actual operation process simulation logic, the digital twin model can simulate reasonable equipment faults in a teaching simulation scene of equipment fault maintenance and equipment daily maintenance, and the effect of restoring real situations can be achieved. The maintenance training cost of a traditional hydropower station is too high, a three-dimensional digital model of the hydropower station equipment can be built for a user to learn through the method, the experience feeling of the user is further improved, and the training cost is reduced.

Description

Technical Field

[0001] This invention belongs to the field of teaching technology for simulated maintenance of hydropower stations, and specifically relates to a method for constructing a digital twin model of a hydropower station maintenance teaching system. Background Technology

[0002] Maintenance plays a crucial role in ensuring the safe, stable, and efficient operation of hydropower station equipment. Hydropower station equipment maintenance involves a series of procedural activities, including disassembling and handling components. Due to the numerous and complex structures of hydropower station equipment components, relevant technical personnel must continuously improve their professional theoretical knowledge and maintenance capabilities. Traditional hydropower station equipment maintenance training methods are time-consuming, resource-intensive, and sometimes require on-site visits for learning, undoubtedly resulting in high operating costs for hydropower stations. These traditional training methods have gradually come under criticism within the industry. Therefore, researching the key technologies, theories, and methods for digitizing hydropower station equipment maintenance, and constructing a digital twin model for virtual maintenance of hydropower station equipment, has significant engineering application value for improving the maintenance level of hydropower station equipment. Summary of the Invention

[0003] To address the aforementioned shortcomings in existing technologies, this invention provides a digital twin model construction method for a hydropower station maintenance training system, which solves the problem of excessively high maintenance training costs in traditional hydropower stations.

[0004] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: a method for constructing a digital twin model of a hydropower station maintenance teaching system, comprising the following steps: S1. Obtain historical operating status data of hydropower station equipment and establish a training set; S2. Construct a digital twin model based on the training set; S3. Simulate the characteristics of hydropower station operation in real time using a digital twin model to complete the construction of a digital twin model for the hydropower station maintenance teaching system.

[0005] Furthermore: In S1, the historical operating status data of the hydropower station equipment includes fault data of various components in the turbine and governor during historical operating status.

[0006] Furthermore: S2 includes the following steps: S21. Taking each piece of equipment in the hydropower station as an agent, the training set is input into the Q network of the agent to establish a multi-agent Markov decision model that simulates the operation of hydropower station equipment. S22. Continuously select actions according to the agent's strategy, apply the agent's actions to the environment, generate experience data, and store it in the agent's experience pool. S23. During the interaction between the agent and the environment, the parameters of the target network are updated according to the experience pool, and the parameters of the target network are copied to the policy network after a preset time interval. S24. When the maximum time step is reached, the optimal policy network is obtained, and the construction of the digital twin model is completed. The Q value output by the digital twin model is the health label of the device.

[0007] The beneficial effects of the above-mentioned further solutions are as follows: Based on the idea of ​​reinforcement learning, each intelligent agent of the device interacts with the environment in an autonomous learning manner, continuously tries and fails, and updates the Q network using random uniform experience samples in the experience pool, thereby autonomously completing the learning of the device operation process simulation.

[0008] Further: In S21, the agent's Q network includes a feature extraction subnetwork and a feature classification subnetwork connected in sequence. The feature extraction subnetwork includes a first convolutional layer, a first pooling layer and a second convolutional layer connected in sequence. The feature classification subnetwork includes an adaptive pooling layer, a fully connected layer and an output layer connected in sequence.

[0009] Furthermore, the activation function of the feature classification subnetwork is the Log-Softmax function.

[0010] Furthermore: In S21, the Markov-related variables are specifically: state space S This represents all states of the environment. ,in, In time t status It is based on the training center equipment x In time t The state is obtained; Action space A This represents all types of identification actions, which correspond to the device's health tag. , This indicates that the equipment is working properly. K For the device's first K Types of fault labels, in time t action ; Transition probability P , indicating state Execute action After transitioning to state The probability of; reward function R , , Indicates the state Execute action The expected value of immediate rewards; In the formula, l The fault label corresponding to the action.

[0011] Furthermore: In S22, the expression for the action selected by the agent is specifically as follows: In the formula, To find the maximum Q function In state s Choose the action a , This represents the probability of randomly selecting an action.

[0012] The beneficial effect of the above further scheme is that, in order to prevent the agent from getting trapped in a local optimum too early, the strategy of this invention adopts... The probability of randomly selecting an action, 1- The probability of Q The function determines the action, and as the number of iterations increases, it tends to use the function more and more. Q Actions with larger function values.

[0013] Furthermore: In S23, the expression for the loss function used to update the agent is as follows: In the formula, The target Q-value is the output of the target network. For network parameters, For the policy network in state s Next action a Q-value estimation, For mathematical expectation; In the formula, In the state Take action below Expected returns For learning rate, In the state Take action below Instant rewards As a discount factor for rewards, In the new state The maximum expected return that can be obtained by taking different actions. In order to take new action The new state that is obtained later.

[0014] The beneficial effects of the above-mentioned further scheme are as follows: During the autonomous learning process of the agent, a small batch of experience data is randomly extracted from the experience pool, and the loss function of the current Q value and the target Q value is calculated through the policy network and the target network. The calculated loss function is used to update the target network. Then, after a certain number of delays, the parameters of the target network are copied to the policy network to update the policy network and obtain the optimal policy network.

[0015] Furthermore: In S24, the digital twin model uses the particle swarm optimization algorithm to output the Q-value, specifically: S241. Initialize the position and velocity of the particle swarm, and calculate the fitness of each particle. ; In the formula, For the first i One target output, n For the number of outputs, For the policy network reward function R The i One output; S242. Update the velocity and position of each particle, including considering both individual and global optima; S243. Repeat S241~S242 until the maximum number of iterations is reached or the fitness of the global optimal solution is lower than the preset fitness threshold, and obtain the Q value calculated by the policy network. This is used as the Q-value output by the digital twin model; In the formula, For parameters The policy network for input The prediction.

[0016] Furthermore, in S3, the digital twin model generates health tags for each piece of equipment during the operation of the hydropower station, constructs corresponding features based on the health tags, and uses a three-dimensional digital model to model the features, simulating normal and fault behaviors during the operation of the hydropower station. The beneficial effects of this invention are as follows: (1) This invention provides a method for constructing a digital twin model of a hydropower station maintenance teaching system. Based on the simulation logic of the actual operation process of hydropower station equipment using reinforcement learning, the digital twin model can simulate reasonable equipment faults in the teaching simulation scenarios of equipment fault repair and daily equipment maintenance, and can achieve a more realistic effect.

[0017] (2) The maintenance training cost of traditional hydropower stations is too high. The method of the present invention can build a three-dimensional digital model of hydropower station equipment for users to learn, which further improves the user experience and reduces the training cost. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a method for constructing a digital twin model of a hydropower station maintenance teaching system according to the present invention.

[0019] Figure 2 This is a schematic diagram of a Q-network for an intelligent agent. Detailed Implementation

[0020] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0021] like Figure 1 As shown, in one embodiment of the present invention, a method for constructing a digital twin model of a hydropower station maintenance teaching system includes the following steps: S1. Obtain historical operating status data of hydropower station equipment and establish a training set; S2. Construct a digital twin model based on the training set; S3. Simulate the characteristics of hydropower station operation in real time using a digital twin model to complete the construction of a digital twin model for the hydropower station maintenance teaching system.

[0022] In S1, the historical operating status data of the hydropower station equipment includes fault data of various components in the turbine and governor during historical operation.

[0023] In this embodiment, the turbine and governor are composed of multiple components. In order to ensure the accuracy and realism of the digital twin model simulation, it is necessary to obtain the health status of each component during the working process in detail before constructing the digital twin model, so as to truly reflect the working status of the hydropower station equipment and provide support for the fault diagnosis and analysis of the hydropower station maintenance teaching system.

[0024] S2 includes the following steps: S21. Taking each piece of equipment in the hydropower station as an agent, the training set is input into the Q network of the agent to establish a multi-agent Markov decision model that simulates the operation of hydropower station equipment. In this embodiment, the agent's Q-network includes a target network and a policy network, and the target network and policy network have the same structure.

[0025] S22. Continuously select actions according to the agent's strategy, apply the agent's actions to the environment, generate experience data, and store it in the agent's experience pool. S23. During the interaction between the agent and the environment, the parameters of the target network are updated according to the experience pool, and the parameters of the target network are copied to the policy network after a preset time interval. S24. When the maximum time step is reached, the optimal policy network is obtained, and the construction of the digital twin model is completed. The Q value output by the digital twin model is the health label of the device.

[0026] In this embodiment, the present invention is based on the idea of ​​reinforcement learning. Each intelligent agent of the device interacts with the environment in an autonomous learning manner, continuously tries and fails, and updates the Q network using random uniform experience samples in the experience pool, thereby autonomously completing the learning of the device operation process simulation.

[0027] like Figure 2 As shown in S21, the agent's Q network includes a feature extraction subnetwork and a feature classification subnetwork connected in sequence. The feature extraction subnetwork includes a first convolutional layer, a first pooling layer and a second convolutional layer connected in sequence. The feature classification subnetwork includes an adaptive pooling layer, a fully connected layer and an output layer connected in sequence.

[0028] The activation function for the feature classification subnetwork is the Log-Softmax function.

[0029] In S21, the Markov-related variables are specifically: state space S This represents all states of the environment. ,in, In time t status It is based on the training center equipment x In time t The state is obtained; Action space A This represents all types of identification actions, which correspond to the device's health tag. , This indicates that the equipment is working properly. K For the device's first K Types of fault labels, in time t action ; Transition probability P , indicating state Execute action After transitioning to state The probability of; reward function R , , Indicates the state Execute action The expected value of immediate rewards; In the formula, l The fault label corresponding to the action.

[0030] The reward function is used to evaluate the quality of the agent's actions, thereby guiding the agent to learn strategies and make better corresponding actions.

[0031] In S22, the expression for the action selected by the agent is as follows: In the formula, To find the maximum Q function In state s Choose the action a , This represents the probability of randomly selecting an action.

[0032] In this embodiment, to prevent the agent from getting trapped in local optima too early, the strategy of this invention adopts... The probability of randomly selecting an action, 1- The probability of Q The function determines the action, and as the number of iterations increases, it tends to use the function more and more. Q Actions with larger function values.

[0033] In S23, the expression for the loss function used to update the agent is as follows: In the formula, The target Q-value is the output of the target network. For network parameters, For the policy network in state s Next action a Q-value estimation, For mathematical expectation; In the formula, In the state Take action below Expected returns For learning rate, In the state Take action below Instant rewards As a discount factor for rewards, In the new state The maximum expected return that can be obtained by taking different actions. In order to take new action The new state that is obtained later.

[0034] In this embodiment, during the simulation process of the present invention, each time the agent selects an action, it will use the corresponding [action] based on its current state and the selected action. Update the agent's target Q-value. During the agent's autonomous learning process, a small batch of experience data is randomly sampled from the experience pool. The loss function for the current Q-value and the target Q-value is calculated through the policy network and the target network. The calculated loss function is used to update the target network. Then, after a certain delay, the parameters of the target network are copied to the policy network to update the policy network and obtain the optimal policy network.

[0035] In S24, the digital twin model uses the particle swarm optimization algorithm to output the Q-value, specifically: S241. Initialize the position and velocity of the particle swarm, and calculate the fitness of each particle. ; In the formula, For the first i One target output, n For the number of outputs, For the policy network reward function R The i One output; S242. Update the velocity and position of each particle, including considering both individual and global optima; S243. Repeat S241~S242 until the maximum number of iterations is reached or the fitness of the global optimal solution is lower than the preset fitness threshold, and obtain the Q value calculated by the policy network. This is used as the Q-value output by the digital twin model; In the formula, For parameters The policy network for input The prediction.

[0036] In S3, the digital twin model generates health tags for each piece of equipment during the operation of the hydropower station. Based on the health tags, corresponding features are constructed, and the features are modeled using a three-dimensional digital model to simulate the normal and fault behaviors during the operation of the hydropower station. In this embodiment, the hydropower station maintenance teaching system is built using SolidWorks 2015 software. The system models the equipment based on its health tags, perfectly replicating the fault conditions that occur during the actual operation of the hydropower station. Users can learn entirely based on the corresponding content of the hydropower station maintenance teaching system, thus enhancing the user experience.

[0037] The beneficial effects of this invention are as follows: This invention provides a method for constructing a digital twin model of a hydropower station maintenance teaching system. Based on the simulation logic of the actual operation process of hydropower station equipment using reinforcement learning, the digital twin model can simulate reasonable equipment faults in teaching simulation scenarios of equipment fault repair and daily equipment maintenance, and can achieve a more realistic effect.

[0038] Traditional hydropower station maintenance training costs are too high. The method of this invention can build a three-dimensional digital model of hydropower station equipment for users to learn from, which further improves the user experience and reduces training costs.

[0039] In the description of this invention, it should be understood that the terms "center," "thickness," "upper," "lower," "horizontal," "top," "bottom," "inner," "outer," and "radial," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying the relative importance or the number of technical features implicitly specified. Therefore, a feature defined by "first," "second," and "third" may explicitly or implicitly include one or more of that feature.

Claims

1. A method for constructing a digital twin model of a hydropower station maintenance teaching system, characterized in that, Includes the following steps: S1. Obtain historical operating status data of hydropower station equipment and establish a training set; S2. Construct a digital twin model based on the training set; S3. Simulate the characteristics of hydropower station operation in real time using a digital twin model to complete the construction of a digital twin model for the hydropower station maintenance teaching system.

2. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 1, characterized in that, In S1, the historical operating status data of the hydropower station equipment includes fault data of various components in the turbine and governor during historical operation.

3. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 1, characterized in that, S2 includes the following steps: S21. Taking each piece of equipment in the hydropower station as an agent, the training set is input into the Q network of the agent to establish a multi-agent Markov decision model that simulates the operation of hydropower station equipment. S22. Continuously select actions according to the agent's strategy, apply the agent's actions to the environment, generate experience data, and store it in the agent's experience pool. S23. During the interaction between the agent and the environment, the parameters of the target network are updated according to the experience pool, and the parameters of the target network are copied to the policy network after a preset time interval. S24. When the maximum time step is reached, the optimal policy network is obtained, and the construction of the digital twin model is completed. The Q value output by the digital twin model is the health label of the device.

4. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 3, characterized in that, In S21, the agent's Q network includes a feature extraction subnetwork and a feature classification subnetwork connected in sequence. The feature extraction subnetwork includes a first convolutional layer, a first pooling layer and a second convolutional layer connected in sequence. The feature classification subnetwork includes an adaptive pooling layer, a fully connected layer and an output layer connected in sequence.

5. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 4, characterized in that, The activation function for the feature classification subnetwork is the Log-Softmax function.

6. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 3, characterized in that, In S21, the Markov-related variables are specifically: state space S This represents all states of the environment. ,in, In time t status It is based on the training center equipment x In time t The state is obtained; Action space A This represents all types of identification actions, which correspond to the device's health tag. , This indicates that the equipment is working properly. K For the device's first K Types of fault labels, in time t action ; Transition probability P , indicating state Execute action After transitioning to state The probability of; reward function R , , Indicates the state Execute action The expected value of immediate rewards; In the formula, l The fault label corresponding to the action.

7. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 6, characterized in that, In S22, the expression for the action selected by the agent is as follows: In the formula, To find the maximum Q function In state s Choose the action a , This represents the probability of randomly selecting an action.

8. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 7, characterized in that, In S23, the expression for the loss function used to update the agent is as follows: In the formula, The target Q-value is the output of the target network. For network parameters, For the policy network in state s Next action a Q-value estimation, For mathematical expectation; In the formula, In the state Take action below Expected returns For learning rate, In the state Take action below Instant rewards As a discount factor for rewards, In the new state The maximum expected return that can be obtained by taking different actions. In order to take new action The new state that is obtained later.

9. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 8, characterized in that, In S24, the digital twin model uses the particle swarm optimization algorithm to output the Q-value, specifically: S241. Initialize the position and velocity of the particle swarm, and calculate the fitness of each particle. ; In the formula, For the first i One target output, n For the number of outputs, For the policy network reward function R The i One output; S242. Update the velocity and position of each particle, including considering both individual and global optima; S243. Repeat S241~S242 until the maximum number of iterations is reached or the fitness of the global optimal solution is lower than the preset fitness threshold, and obtain the Q value calculated by the policy network. This is used as the Q-value output by the digital twin model; In the formula, For parameters The policy network for input The prediction.

10. The method for constructing a digital twin model of a hydropower station maintenance teaching system according to claim 1, characterized in that, In S3, the digital twin model generates health tags for each piece of equipment during the operation of the hydropower station. Based on the health tags, corresponding features are constructed, and the features are modeled using a three-dimensional digital model to simulate the normal and fault behaviors during the operation of the hydropower station.

Images