Operation and maintenance strategy generation methods, apparatus, equipment, storage media, and program products

Abstract

This application relates to a method, apparatus, computer device, computer-readable storage medium, and computer program product for generating operation and maintenance strategies. The method includes: acquiring operating status data of multiple power devices in a power system, and fusing the operating status data to obtain a composite state vector corresponding to each power device; determining the corresponding regulation document for each power device from a power operation and maintenance regulation knowledge graph based on the device type and the corresponding composite state vector; and inputting the corresponding composite state vector and the corresponding regulation document into a pre-trained operation and maintenance strategy generation model to obtain the power system's operation and maintenance strategy output by the model. The power system's operation and maintenance strategy includes the operation and maintenance strategies corresponding to each power device. This method can match actual operational needs and effectively ensure the stable operation of the power system.

Description

Technical Field

[0001] This application relates to the field of power technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for generating operation and maintenance strategies. Background Technology

[0002] The power system is a vital infrastructure for ensuring industrial production and people's lives. The stability of the power system's operation is directly related to the reliability of power supply and the normal operation of society.

[0003] In existing technologies, most methods use fixed procedures to generate power equipment operation and maintenance strategies, and then carry out operation and maintenance work according to the generated strategies to ensure the normal operation of the power system.

[0004] However, existing methods for generating operation and maintenance strategies cannot match actual operational needs and are insufficient to effectively ensure the stable operation of the power system. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for generating operation and maintenance strategies that can match actual operational needs and effectively ensure the stable operation of the power system, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for generating operation and maintenance strategies, including:

[0007] The system acquires the operating status data of multiple power devices in the power system and performs fusion processing on the operating status data to obtain the composite state vector corresponding to each power device.

[0008] Based on the equipment type of each power equipment and the composite state vector corresponding to each power equipment, the corresponding procedure documents for each power equipment are determined from the power operation and maintenance procedure knowledge graph.

[0009] The composite state vectors corresponding to each power device and the corresponding procedure documents are input into a pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model. The operation and maintenance strategy of the power system includes the operation and maintenance strategy corresponding to each power device.

[0010] In one embodiment, the training method for the operation and maintenance strategy generation model includes: acquiring power operation and maintenance source data, and determining a set of operating conditions and a set of risk constraints based on the power operation and maintenance source data; constructing a simulation system of the power system, and determining a reward function based on the set of operating conditions and the set of risk constraints; and training an initial operation and maintenance strategy generation model based on the simulation system and the reward function to obtain the operation and maintenance strategy generation model.

[0011] In one embodiment, determining the set of operating conditions and the set of risk constraints based on power operation and maintenance source data includes: performing semantic parsing based on power operation and maintenance source data to construct a power operation and maintenance procedure knowledge graph; and determining the set of operating conditions and the set of risk constraints according to the logical relationships of the power operation and maintenance procedure knowledge graph.

[0012] In one embodiment, determining the reward function based on the set of operating conditions and the set of risk constraints includes: constructing a constraint satisfaction function based on the set of operating conditions and the set of risk constraints; and constructing a reward function based on the constraint satisfaction function and the power system's operation and maintenance benefit reward.

[0013] In one embodiment, the initial operation and maintenance strategy generation model is trained based on a simulation system and a reward function to obtain the operation and maintenance strategy generation model, including: determining multiple candidate operation and maintenance strategies based on the simulation system, the power operation and maintenance procedure knowledge graph, and the initial operation and maintenance strategy generation model; and performing group rule-based strategy optimization reinforcement learning iterative training on the initial operation and maintenance strategy generation model based on the simulation system, the reward function, and the multiple candidate operation and maintenance strategies to obtain the operation and maintenance strategy generation model.

[0014] In one embodiment, the initial operation and maintenance strategy generation model is subjected to iterative training of group rule policy optimization reinforcement learning based on a simulation system, a reward function, and multiple candidate operation and maintenance strategies to obtain the operation and maintenance strategy generation model. This includes: executing multiple candidate operation and maintenance strategies using the simulation system to obtain the execution results corresponding to the multiple candidate operation and maintenance strategies, and determining the reward values ​​corresponding to the multiple candidate operation and maintenance strategies based on the reward function and the execution results; iteratively updating the initial operation and maintenance strategy generation model based on the reward values ​​to obtain an updated initial operation and maintenance strategy generation model; and training the initial operation and maintenance strategy generation model based on the simulation system and the reward function based on the updated initial operation and maintenance strategy generation model until the updated candidate operation and maintenance strategy generation model meets preset conditions, at which point the updated candidate operation and maintenance strategy generation model is determined as the operation and maintenance strategy generation model.

[0015] Secondly, this application also provides an operation and maintenance strategy generation apparatus, comprising:

[0016] The acquisition module is used to acquire the operating status data of multiple power devices in the power system and perform fusion processing on the operating status data to obtain the composite state vector corresponding to each power device.

[0017] The determination module is used to determine the corresponding procedure documents for each power equipment from the power operation and maintenance procedure knowledge graph based on the equipment type of each power equipment and the composite state vector corresponding to each power equipment.

[0018] The execution module is used to input the composite state vectors corresponding to each power device and the corresponding procedure documents of each power device into the pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model. The operation and maintenance strategy of the power system includes the operation and maintenance strategy corresponding to each power device.

[0019] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the embodiments of the first aspect above.

[0020] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the embodiments of the first aspect above.

[0021] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the embodiments of the first aspect above.

[0022] The aforementioned operation and maintenance strategy generation method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire operating status data of multiple power devices in a power system, and perform fusion processing on the operating status data to obtain composite state vectors corresponding to each power device. Based on the equipment type and the corresponding composite state vectors of each power device, the corresponding regulation documents for each power device are determined from a power operation and maintenance regulation knowledge graph. The composite state vectors and corresponding regulation documents of each power device are input into a pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model. The operation and maintenance strategy of the power system includes the operation and maintenance strategy corresponding to each power device. The operation and maintenance strategy generation method provided in this application, by fusing operating status data of multiple power devices to obtain composite state vectors, and then inputting the composite state vectors and corresponding regulation documents of each power device into the operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model, enables the generated operation and maintenance strategy to match actual operating needs, thereby effectively ensuring the stable operation of the power system. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the operation and maintenance strategy generation method in one embodiment;

[0025] Figure 2 This is a flowchart illustrating the training method for the operation and maintenance strategy generation model in one embodiment.

[0026] Figure 3 This is a flowchart illustrating a method for determining the set of operating conditions and the set of risk constraints in one embodiment.

[0027] Figure 4 This is a flowchart illustrating a method for determining a reward function based on a set of operating conditions and a set of risk constraints in one embodiment.

[0028] Figure 5 This is a flowchart illustrating a method for obtaining an operation and maintenance strategy generation model in one embodiment.

[0029] Figure 6 This is a flowchart illustrating a method for obtaining an operation and maintenance strategy generation model in another embodiment;

[0030] Figure 7 This is a flowchart illustrating the operation and maintenance strategy generation method in another embodiment;

[0031] Figure 8 This is a structural block diagram of the operation and maintenance strategy generation device in one embodiment;

[0032] Figure 9 This is an internal structural diagram of a computer device in one embodiment;

[0033] Figure 10 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0036] The power system is a vital infrastructure for ensuring industrial production and people's lives. The stability of the power system's operation is directly related to the reliability of power supply and the normal operation of society.

[0037] In existing technologies, most methods use fixed procedures to generate power equipment operation and maintenance strategies, and then carry out operation and maintenance work according to the generated strategies to ensure the normal operation of the power system.

[0038] However, existing methods for generating operation and maintenance strategies cannot match actual operational needs and are insufficient to effectively ensure the stable operation of the power system.

[0039] In view of this, this application provides a method for generating operation and maintenance strategies. The method acquires operating status data from multiple power devices in a power system and fuses this data to obtain composite state vectors for each power device. Based on the device type and the corresponding composite state vectors, it determines the corresponding operational and maintenance (O&M) documents from a power O&M procedure knowledge graph. The composite state vectors and corresponding O&M documents are then input into a pre-trained O&M strategy generation model to obtain the O&M strategy for the power system output by the model. The O&M strategy for the power system includes the O&M strategies for each power device. The O&M strategy generation method provided in this application fuses operating status data from multiple power devices to obtain composite state vectors. Then, it inputs the composite state vectors and corresponding O&M documents into the O&M strategy generation model to obtain the O&M strategy for the power system output by the model. This ensures that the generated O&M strategy matches actual operational needs, thereby effectively guaranteeing the stable operation of the power system.

[0040] The operation and maintenance strategy generation method provided in this application can be executed by a computer device, which can be a terminal or a server.

[0041] In one exemplary embodiment, such as Figure 1 As shown, a method for generating operation and maintenance strategies is provided, which includes the following steps:

[0042] Step 101: Obtain the operating status data of multiple power devices in the power system, and perform fusion processing on the operating status data to obtain the composite state vector corresponding to each power device.

[0043] Optionally, multiple electrical devices in the power system may include transformers, switchgear, oil draining and nitrogen filling fire extinguishing devices, dehumidifiers, non-excitation tap changers, fan motors, etc.

[0044] Optionally, the operating status data of multiple power devices can be multi-source heterogeneous operating condition and health monitoring data. For example, the operating status data may include equipment temperature, vibration amplitude, partial discharge, insulation resistance value, oil level index, operating load, historical defect records, current alarm information, ambient temperature and humidity, and cumulative operating time, etc.

[0045] Optionally, the composite state vector can be used to characterize the real-time integrated operating status of power equipment.

[0046] In some exemplary embodiments, a computer device can acquire operating status data of multiple power devices in a power system.

[0047] Specifically, computer equipment can obtain multi-source operating status data of multiple power devices in the power system from the power system's monitoring platform, equipment ledger database, historical operation record system, and field environmental sensing terminal.

[0048] Furthermore, after acquiring the operating status data of multiple power devices in the power system, the computer equipment can perform fusion processing on the operating status data to obtain the composite state vector corresponding to each power device.

[0049] Specifically, the computer equipment can first perform data preprocessing on the operating status data, such as standardization, normalization, and redundancy removal. For each power device, the computer equipment can perform feature fusion processing on the preprocessed operating status data of the power device to obtain the composite state vector of the power device.

[0050] Step 102: Based on the equipment type of each power equipment and the composite state vector corresponding to each power equipment, determine the corresponding procedure document for each power equipment from the power operation and maintenance procedure knowledge graph.

[0051] Optionally, the power operation and maintenance procedure knowledge graph can be a structured knowledge base built using natural language processing and knowledge graph technology, based on power industry standard documents, operation and maintenance procedure manuals, and expert experience bases.

[0052] For example, the power operation and maintenance procedure knowledge graph can be stored according to equipment type. The power operation and maintenance procedure knowledge graph can contain the operation and maintenance standards, maintenance procedures, safety specifications, operating condition sets and risk constraint sets corresponding to various types of power equipment.

[0053] Optionally, the procedure document can be an operation and maintenance compliance guidance document that matches the corresponding power equipment. For example, the procedure document can include formal constraint information such as operating conditions, risk thresholds, and maintenance requirements for the corresponding power equipment.

[0054] In some exemplary embodiments, after obtaining the composite state vectors corresponding to each power device, the computer device can determine the corresponding procedure documents for each power device from the power operation and maintenance procedure knowledge graph based on the device type of each power device and the composite state vectors corresponding to each power device.

[0055] Specifically, the computer equipment can first select the corresponding procedure knowledge entries from the power operation and maintenance procedure knowledge graph based on the equipment type of each power equipment. Then, based on the real-time health status, operating conditions and environmental conditions reflected by the composite state vector of each power equipment, it can match the procedure documents that are suitable for the current equipment status from the selected procedure knowledge entries corresponding to the power equipment type.

[0056] Step 103: Input the composite state vectors corresponding to each power device and the corresponding procedure documents into the pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model.

[0057] Optionally, the operation and maintenance strategy generation model can be trained based on a group rule strategy optimization reinforcement learning algorithm.

[0058] Optionally, the operation and maintenance strategy of the power system may include the operation and maintenance strategy corresponding to each power device. The operation and maintenance strategy of the power system can be used to guide the daily inspection, live-line testing, maintenance and repair of power devices.

[0059] In some exemplary embodiments, after obtaining the composite state vectors and procedure documents corresponding to each power device, the computer device can input the composite state vectors and procedure documents corresponding to each power device into a pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model.

[0060] The aforementioned operation and maintenance strategy generation method acquires operating status data from multiple power devices in a power system and fuses this data to obtain composite state vectors for each power device. Based on the device type and the corresponding composite state vectors, it determines the corresponding operational and maintenance (O&M) documents from a power O&M procedure knowledge graph. The composite state vectors and corresponding O&M documents are then input into a pre-trained O&M strategy generation model to obtain the O&M strategy for the power system output by the model. This O&M strategy includes the O&M strategies for each power device. The O&M strategy generation method provided in this application fuses operating status data from multiple power devices to obtain composite state vectors. Then, it inputs the composite state vectors and corresponding O&M documents into the O&M strategy generation model to obtain the O&M strategy for the power system output by the model. This ensures that the generated O&M strategy matches actual operational needs, thereby effectively guaranteeing the stable operation of the power system.

[0061] In one exemplary embodiment, such as Figure 2 As shown, the training method for the operation and maintenance strategy generation model includes the following steps:

[0062] Step 201: Obtain power operation and maintenance source data, and determine the set of operating conditions and risk constraints based on the power operation and maintenance source data.

[0063] Optionally, power operation and maintenance source data can be unstructured text data obtained from power industry standard documents, operation and maintenance manuals, and expert experience databases. For example, power operation and maintenance source data can include original materials such as operation and maintenance standards, maintenance procedures, and safety specifications for various types of power equipment, including various equipment types such as transformers and switchgear, and operation and maintenance scenarios, in various forms such as text descriptions, tables, and diagrams.

[0064] Optionally, the set of operating conditions can consist of a series of logical propositions that must be satisfied, and is a compound predicate formula concerning equipment state variables and operation and maintenance action variables.

[0065] Optionally, the risk constraint set can be the hazardous areas that the equipment needs to avoid during operation and the safety prerequisites for operation and maintenance, expressed in the form of inequality constraints.

[0066] In some exemplary embodiments, the computer device can acquire power operation and maintenance source data.

[0067] Specifically, computer equipment can obtain and summarize unstructured text data containing equipment operation and maintenance standards, maintenance procedures, safety specifications, etc., from preset power industry standard document libraries, operation and maintenance procedure manual libraries, and expert experience knowledge bases to obtain power operation and maintenance source data.

[0068] Furthermore, after acquiring power operation and maintenance source data, computer equipment can determine the set of operating conditions and the set of risk constraints based on the power operation and maintenance source data.

[0069] Specifically, computer equipment can perform semantic analysis based on power operation and maintenance source data to determine the set of operating conditions and risk constraints.

[0070] Step 202: Construct a simulation system for the power system and determine the reward function based on the set of operating conditions and the set of risk constraints.

[0071] Optionally, the power system simulation system can be a digital twin simulation environment for power equipment built on a rule base. For example, the power system simulation system uses a state transition function that combines determinism and stochasticity to simulate the state evolution of power equipment under various operation and maintenance actions.

[0072] Optionally, the reward function can be used to quantify the overall performance of the operation and maintenance strategy.

[0073] In some exemplary embodiments, computer devices can construct simulation systems for power systems.

[0074] Specifically, computer equipment can construct a digital twin simulation environment, which includes a state transition function that combines determinism and randomness, based on the actual operating mechanism, operation and maintenance specifications, and state change logic of power equipment. This is also known as a power system simulation system.

[0075] Furthermore, after constructing a simulation system for the power system, the computer equipment can also determine the reward function based on the set of operating conditions and the set of risk constraints.

[0076] Specifically, computer equipment can construct a constraint satisfaction function based on the set of operating conditions and the set of risk constraints, and then determine the reward function based on the constraint satisfaction function.

[0077] Step 203: Train the initial operation and maintenance strategy generation model based on the simulation system and reward function to obtain the operation and maintenance strategy generation model.

[0078] In some exemplary embodiments, after obtaining the simulation system and the reward function, the computer device can train the initial operation and maintenance strategy generation model based on the simulation system and the reward function to obtain the operation and maintenance strategy generation model.

[0079] In one exemplary embodiment, such as Figure 3 As shown, the determination of the set of operating conditions and the set of risk constraints based on power operation and maintenance source data includes the following steps:

[0080] Step 301: Perform semantic parsing based on power operation and maintenance source data to construct a knowledge graph of power operation and maintenance procedures.

[0081] In some exemplary embodiments, computer devices can perform semantic parsing based on power operation and maintenance source data to construct a knowledge graph of power operation and maintenance procedures.

[0082] Specifically, computer equipment can use pre-trained power domain language models to perform deep semantic analysis on power operation and maintenance source data. By using named entity recognition technology, four key entities—equipment / components, attributes / status, operational behavior, and values / ranges—can be extracted from the power operation and maintenance source data. Then, by using dependency parsing technology, conditional relationships, attribute associations, and negation / prohibition relationships between entities can be extracted. Based on the identified entities and semantic relationships, a power operation and maintenance procedure knowledge graph can be constructed, which includes "equipment object-operation behavior-safety conditions-risk constraints" and "equipment-attribute-status / value" and "equipment-condition-operation" as core triples.

[0083] Step 302: Determine the set of operating conditions and the set of risk constraints based on the logical relationships of the power operation and maintenance regulations knowledge graph.

[0084] In some exemplary embodiments, after constructing a knowledge graph of power operation and maintenance procedures, the computer device can determine the set of operating conditions and the set of risk constraints based on the logical relationships of the knowledge graph.

[0085] Specifically, computer equipment can, based on the logical types and constraint attributes of each knowledge unit in the power operation and maintenance procedure knowledge graph, group together compound predicate formula knowledge units that represent the operation and maintenance operations must satisfy and are connected by logical operators to form an operation condition set; and group together inequality constraint knowledge units that represent the dangerous areas that equipment operation must avoid and the safety prerequisites that operation and maintenance operations must satisfy, expressed as prohibitions or strong triggering conditions, to form a risk constraint set.

[0086] In one exemplary embodiment, such as Figure 4 As shown, the reward function is determined based on the set of operating conditions and the set of risk constraints, including the following steps:

[0087] Step 401: Construct a constraint satisfaction function based on the set of operating conditions and the set of risk constraints.

[0088] Optionally, the constraint satisfaction function can be a composite indicator function used to verify whether any device state and operation and maintenance action simultaneously satisfy all operating conditions and risk constraints.

[0089] In some exemplary embodiments, the computer device may construct a constraint satisfaction function based on a set of operating conditions and a set of risk constraints.

[0090] Specifically, the computer device constructs a constraint satisfaction function for each operational condition constraint in the set of operational conditions, each risk constraint function in the set of risk constraints, and the corresponding safety threshold. This constraint satisfaction function can be expressed as follows: ,in, This represents the i-th operation condition constraint. Let j represent the risk constraint function. This represents the risk threshold for the j-th risk. The indicator function is used to ensure that action 'a' is in the specified state. The constraint satisfaction function ensures that action 'a' is in the specified state through the indicator function. Simultaneously satisfying all operational conditions and risk constraints forms a hard boundary for strategy search.

[0091] The constraint satisfaction function takes the form of a product of indicator functions. It performs a validity check on each operational condition constraint and a threshold compliance check on each risk constraint. The output value of the constraint satisfaction function is 1 if and only if all the aforementioned constraints are satisfied, otherwise it is 0.

[0092] Step 402: Construct a reward function based on the constraint satisfaction function and the power system operation and maintenance efficiency reward.

[0093] Optionally, the power system operation and maintenance efficiency reward is used to quantify the economic benefits of performing operation and maintenance actions, such as cost savings, improved equipment health, and reduced failure risk.

[0094] In some exemplary embodiments, the computer device can construct a reward function based on the constraint satisfaction function and the operation and maintenance efficiency reward of the power system.

[0095] Specifically, computer equipment can employ a weighted combination approach to construct a reward function, integrating operational efficiency rewards and constraint satisfaction functions. This reward function can be expressed as follows: ,in, It is an economic benefit reward function, which can be used to quantify the state. The following operation and maintenance actions are adopted. The economic benefits are determined by the change in operational costs performed in the simulation environment. It is a constraint satisfaction function of procedural knowledge, which can be used to represent strategies. The degree to which procedural constraints are met. These are constraint satisfaction weighting coefficients, which can be used to balance economy and compliance.

[0096] In one exemplary embodiment, such as Figure 5As shown, the initial operation and maintenance strategy generation model is trained based on the simulation system and reward function to obtain the operation and maintenance strategy generation model, including the following steps:

[0097] Step 501: Determine multiple candidate operation and maintenance strategies based on the simulation system, the power operation and maintenance procedure knowledge graph, and the initial operation and maintenance strategy generation model.

[0098] In some exemplary embodiments, the computer device can determine multiple candidate operation and maintenance strategies based on a simulation system, a power operation and maintenance procedure knowledge graph, and an initial operation and maintenance strategy generation model.

[0099] Specifically, the computer equipment can synchronously input the composite state vector of the power equipment output by the simulation system and the matching procedure constraint information in the power operation and maintenance procedure knowledge graph into the initial operation and maintenance strategy generation model. The initial operation and maintenance strategy generation model is based on the group relative strategy optimization algorithm. Within the safe and feasible domain defined by the formal constraints of the procedure, it generates a sequence of candidate operation and maintenance actions for the current equipment state, forming a set of candidate operation and maintenance strategies to be evaluated. Each candidate operation and maintenance strategy corresponds to a different combination of operation and maintenance operations.

[0100] Step 502: Based on the simulation system, reward function, and multiple candidate operation and maintenance strategies, perform group rule policy optimization reinforcement learning iterative training on the initial operation and maintenance strategy generation model to obtain the operation and maintenance strategy generation model.

[0101] In some exemplary embodiments, after determining multiple candidate operation and maintenance strategies based on a simulation system, a power operation and maintenance procedure knowledge graph, and an initial operation and maintenance strategy generation model, the computer device can perform group rule-based strategy optimization reinforcement learning iterative training on the initial operation and maintenance strategy generation model based on the simulation system, the reward function, and the multiple candidate operation and maintenance strategies to obtain the operation and maintenance strategy generation model.

[0102] In one exemplary embodiment, such as Figure 6 As shown, the initial operation and maintenance strategy generation model is iteratively trained using group rule-based policy optimization reinforcement learning based on a simulation system, reward function, and multiple candidate operation and maintenance strategies to obtain the operation and maintenance strategy generation model. This includes the following steps:

[0103] Step 601: Execute multiple candidate operation and maintenance strategies using a simulation system to obtain the execution results corresponding to the multiple candidate operation and maintenance strategies. Based on the reward function and the execution results corresponding to the multiple candidate operation and maintenance strategies, determine the reward value corresponding to the multiple candidate operation and maintenance strategies.

[0104] In some exemplary embodiments, the computer device can use a simulation system to execute multiple candidate operation and maintenance strategies respectively to obtain the execution results corresponding to the multiple candidate operation and maintenance strategies, and determine the reward value corresponding to the multiple candidate operation and maintenance strategies based on the reward function and the execution results corresponding to the multiple candidate operation and maintenance strategies.

[0105] Specifically, computer equipment can input multiple candidate operation and maintenance strategies one by one into the power system digital twin simulation system. The simulation system simulates the execution of each candidate operation and maintenance strategy through its built-in state transition function, and obtains the execution results such as equipment state evolution, operation and maintenance cost consumption, and constraint satisfaction status for each strategy. Then, the execution results are substituted into the reward function to calculate the operation and maintenance benefit reward and constraint satisfaction score for each candidate operation and maintenance strategy. The weighted sum is then used to obtain the total reward value for each candidate operation and maintenance strategy, forming a group reward vector.

[0106] Step 602: Iteratively update the initial operation and maintenance strategy generation model based on the reward value to obtain the updated initial operation and maintenance strategy generation model. Based on the updated initial operation and maintenance strategy generation model, perform the step of training the initial operation and maintenance strategy generation model based on the simulation system and reward function until the updated candidate operation and maintenance strategy generation model meets the preset conditions. Then, the updated candidate operation and maintenance strategy generation model is determined as the operation and maintenance strategy generation model.

[0107] In some exemplary embodiments, the computer device may iteratively update the initial operation and maintenance strategy generation model based on the reward value to obtain an updated initial operation and maintenance strategy generation model, and based on the updated initial operation and maintenance strategy generation model, perform a step of training the initial operation and maintenance strategy generation model based on the simulation system and the reward function until the updated candidate operation and maintenance strategy generation model meets the preset conditions, and then determine the updated candidate operation and maintenance strategy generation model as the operation and maintenance strategy generation model.

[0108] Specifically, the computer equipment can first normalize the reward vectors within the group, calculate the mean and standard deviation of the rewards, and use the normalization result as the group relative advantage estimate of each candidate operation and maintenance strategy. Then, based on the group rule strategy optimization reinforcement learning algorithm objective function, a KL divergence regularization term is introduced to constrain the policy update magnitude. Combined with the advantage estimate, the parameters of the initial operation and maintenance strategy generation model are iteratively updated to obtain the updated model. Subsequently, the process of candidate strategy generation, simulation execution, reward calculation, and parameter update is repeated with the updated model until the operation and maintenance strategy output by the model continuously meets the procedural constraints, the economic benefit index is stably met, and the convergence accuracy meets the preset threshold. Finally, the model is determined as the trained operation and maintenance strategy generation model.

[0109] In one exemplary embodiment, such as Figure 7 As shown, another method for generating operation and maintenance strategies is provided, which includes the following steps:

[0110] Step 701: Obtain power operation and maintenance source data, and determine the set of operating conditions and risk constraints based on the power operation and maintenance source data; construct a simulation system for the power system, and construct a constraint satisfaction function based on the set of operating conditions and risk constraints; construct a reward function based on the constraint satisfaction function and the power system's operation and maintenance benefit reward; determine multiple candidate operation and maintenance strategies based on the simulation system, the power operation and maintenance procedure knowledge graph, and the initial operation and maintenance strategy generation model.

[0111] Step 702: Execute multiple candidate operation and maintenance strategies using a simulation system to obtain execution results for each strategy. Determine reward values ​​for each strategy based on the reward function and the execution results. Iterate and update the initial operation and maintenance strategy generation model based on the reward values ​​to obtain an updated model. Train the model using the simulation system and reward function until the updated model meets the preset conditions. If the updated model meets the preset conditions, the updated model is then determined as the operation and maintenance strategy generation model.

[0112] Step 703: Obtain the operating status data of multiple power devices in the power system, and perform fusion processing on the operating status data to obtain the composite state vector corresponding to each power device; based on the equipment type of each power device and the composite state vector corresponding to each power device, determine the corresponding procedure document of each power device from the power operation and maintenance procedure knowledge graph.

[0113] Step 704: Input the composite state vectors corresponding to each power device and the corresponding procedure documents of each power device into the pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model. The operation and maintenance strategy of the power system includes the operation and maintenance strategy corresponding to each power device.

[0114] The operation and maintenance strategy generation method provided in this application addresses the problems of low efficiency in strategy exploration and high risk of safety constraint violation. By introducing a group rule-based strategy optimization reinforcement learning algorithm framework, it eliminates the complex Critic model in traditional PPO and instead uses the intra-group relative reward mechanism as a baseline, significantly reducing memory consumption and computational overhead during training. At the same time, through the formal safety constraint function built into the digital twin simulation environment, the action space is rigidly limited to the compliance range in the early stage of strategy exploration, thereby avoiding the generation of invalid or dangerous exploration samples, improving sample efficiency and convergence speed, and fundamentally ensuring the safety and compliance of the strategy generation process.

[0115] Furthermore, to address the problem of domain knowledge being difficult to effectively formalize and embed into the policy optimization process, natural language processing technology is combined with knowledge graph construction. Through deep semantic parsing, the set of operational conditions and risk constraints are extracted from unstructured procedural texts and transformed into machine-computable formal logical constraints (such as predicate formulas and inequalities). These constraints are directly integrated into the reward function design of the group rule policy optimization reinforcement learning algorithm, enabling domain knowledge to rigidly guide policy optimization in an interpretable and verifiable manner, avoiding the signal sparsity and multi-objective trade-off problems in traditional reward function design.

[0116] Finally, to address the issues of insufficient convergence stability and generalization ability of the algorithm in high-dimensional complex spaces, the variance of policy updates is effectively reduced by using the intra-group normalized advantage estimation mechanism and KL divergence regularization term in group rule policy optimization reinforcement learning, which prevents policy drift and local optimum traps. At the same time, combined with multi-dimensional state simulation in a high-fidelity digital twin environment, the robustness and scalability of the generated policy under dynamic conditions are ensured.

[0117] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0118] Based on the same inventive concept, this application also provides an operation and maintenance strategy generation apparatus for implementing the operation and maintenance strategy generation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more operation and maintenance strategy generation apparatus embodiments provided below can be found in the limitations of the operation and maintenance strategy generation method described above, and will not be repeated here.

[0119] In one exemplary embodiment, such as Figure 8 As shown, an operation and maintenance strategy generation device 800 is provided, including: an acquisition module 801, a determination module 802, and an execution module 803, wherein:

[0120] The acquisition module 801 is used to acquire the operating status data of multiple power devices in the power system and perform fusion processing on the operating status data to obtain the composite state vector corresponding to each power device.

[0121] The determination module 802 is used to determine the corresponding procedure documents for each power equipment from the power operation and maintenance procedure knowledge graph based on the equipment type of each power equipment and the composite state vector corresponding to each power equipment.

[0122] The execution module 803 is used to input the composite state vectors corresponding to each power device and the procedure documents corresponding to each power device into the pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model. The operation and maintenance strategy of the power system includes the operation and maintenance strategy corresponding to each power device.

[0123] In one embodiment, the execution module 803 is further configured to acquire power operation and maintenance source data, and determine the set of operating conditions and risk constraints based on the power operation and maintenance source data; construct a simulation system for the power system, and determine a reward function based on the set of operating conditions and risk constraints; and train an initial operation and maintenance strategy generation model based on the simulation system and the reward function to obtain an operation and maintenance strategy generation model.

[0124] In one embodiment, the execution module 803 is specifically used to perform semantic parsing based on power operation and maintenance source data to construct a power operation and maintenance procedure knowledge graph; and to determine the set of operating conditions and the set of risk constraints according to the logical relationships of the power operation and maintenance procedure knowledge graph.

[0125] In one embodiment, the execution module 803 is specifically used to construct a constraint satisfaction function based on the set of operating conditions and the set of risk constraints; and to construct a reward function based on the constraint satisfaction function and the power system's operation and maintenance benefit reward.

[0126] In one embodiment, the execution module 803 is specifically used to determine multiple candidate operation and maintenance strategies based on the simulation system, the power operation and maintenance procedure knowledge graph, and the initial operation and maintenance strategy generation model; and to perform group rule strategy optimization reinforcement learning iterative training on the initial operation and maintenance strategy generation model based on the simulation system, the reward function, and the multiple candidate operation and maintenance strategies to obtain the operation and maintenance strategy generation model.

[0127] In one embodiment, the execution module 803 is specifically used to execute multiple candidate operation and maintenance strategies using a simulation system to obtain execution results corresponding to the multiple candidate operation and maintenance strategies, and to determine the reward values ​​corresponding to the multiple candidate operation and maintenance strategies based on the reward function and the execution results corresponding to the multiple candidate operation and maintenance strategies; to iteratively update the initial operation and maintenance strategy generation model based on the reward values ​​to obtain an updated initial operation and maintenance strategy generation model, and to perform a training step on the initial operation and maintenance strategy generation model based on the simulation system and the reward function based on the updated initial operation and maintenance strategy generation model, until the updated candidate operation and maintenance strategy generation model meets the preset conditions, and then the updated candidate operation and maintenance strategy generation model is determined as the operation and maintenance strategy generation model.

[0128] Each module in the aforementioned operation and maintenance strategy generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0129] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an operation and maintenance strategy generation method.

[0130] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an operation and maintenance strategy generation method.

[0131] Those skilled in the art will understand that Figure 9 and Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the above embodiments.

[0133] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0134] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0136] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0138] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating operation and maintenance strategies, characterized in that, The method includes: The operating status data of multiple power devices in the power system are acquired, and the operating status data are fused to obtain a composite state vector corresponding to each power device. Based on the equipment type of each power equipment and the composite state vector corresponding to each power equipment, the corresponding procedure document for each power equipment is determined from the power operation and maintenance procedure knowledge graph. The composite state vectors corresponding to each of the power devices and the procedure documents corresponding to each of the power devices are input into a pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model. The operation and maintenance strategy of the power system includes the operation and maintenance strategy corresponding to each of the power devices.

2. The method according to claim 1, characterized in that, The training method for the operation and maintenance strategy generation model includes: Acquire power operation and maintenance source data, and determine the set of operating conditions and risk constraints based on the power operation and maintenance source data; A simulation system for the power system is constructed, and a reward function is determined based on the set of operating conditions and the set of risk constraints. The initial operation and maintenance strategy generation model is trained based on the simulation system and the reward function to obtain the operation and maintenance strategy generation model.

3. The method according to claim 2, characterized in that, The determination of the set of operating conditions and the set of risk constraints based on the power operation and maintenance source data includes: Semantic parsing is performed based on the power operation and maintenance source data to construct a knowledge graph of power operation and maintenance procedures; Based on the logical relationships of the power operation and maintenance procedure knowledge graph, the set of operating conditions and the set of risk constraints are determined.

4. The method according to claim 2, characterized in that, The determination of the reward function based on the set of operating conditions and the set of risk constraints includes: Construct a constraint satisfaction function based on the set of operating conditions and the set of risk constraints; Based on the constraint satisfaction function and the power system operation and maintenance efficiency reward, the reward function is constructed.

5. The method according to claim 2, characterized in that, The step of training the initial operation and maintenance strategy generation model based on the simulation system and the reward function to obtain the operation and maintenance strategy generation model includes: Based on the simulation system, the power operation and maintenance procedure knowledge graph, and the initial operation and maintenance strategy generation model, multiple candidate operation and maintenance strategies are determined. Based on the simulation system, the reward function, and the multiple candidate operation and maintenance strategies, the initial operation and maintenance strategy generation model is subjected to group rule policy optimization reinforcement learning iterative training to obtain the operation and maintenance strategy generation model.

6. The method according to claim 5, characterized in that, The step of iteratively training the initial operation and maintenance strategy generation model using group rule-based policy optimization reinforcement learning based on the simulation system, the reward function, and the multiple candidate operation and maintenance strategies to obtain the operation and maintenance strategy generation model includes: The simulation system is used to execute the multiple candidate operation and maintenance strategies respectively to obtain the execution results corresponding to the multiple candidate operation and maintenance strategies, and the reward value corresponding to the multiple candidate operation and maintenance strategies is determined based on the reward function and the execution results corresponding to the multiple candidate operation and maintenance strategies. The initial operation and maintenance strategy generation model is iteratively updated based on the reward value to obtain an updated initial operation and maintenance strategy generation model. Based on the updated initial operation and maintenance strategy generation model, the step of training the initial operation and maintenance strategy generation model based on the simulation system and the reward function is performed until the updated candidate operation and maintenance strategy generation model meets the preset conditions. Then, the updated candidate operation and maintenance strategy generation model is determined as the operation and maintenance strategy generation model.

7. An operation and maintenance strategy generation device, characterized in that, The device includes: The acquisition module is used to acquire the operating status data of multiple power devices in the power system, and to perform fusion processing on the operating status data to obtain a composite state vector corresponding to each power device. The determination module is used to determine the corresponding procedure document for each of the power equipment based on the equipment type of each of the power equipment and the composite state vector corresponding to each of the power equipment from the power operation and maintenance procedure knowledge graph. The execution module is used to input the composite state vectors corresponding to each of the power devices and the procedure documents corresponding to each of the power devices into a pre-trained operation and maintenance strategy generation model to obtain the operation and maintenance strategy of the power system output by the operation and maintenance strategy generation model. The operation and maintenance strategy of the power system includes the operation and maintenance strategy corresponding to each of the power devices.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images