Overhaul plan and power generation scheduling collaborative optimization system and optimization method based on deep reinforcement learning

Abstract

The present application relates to the technical field of management platform, especially to a maintenance plan and power generation scheduling collaborative optimization system and method based on deep reinforcement learning. The method is: collecting power grid multi-source heterogeneous data, standardizing and extracting features of the multi-source heterogeneous data, and constructing a multi-dimensional state feature vector; constructing a deep reinforcement learning model; inputting the real-time generated multi-dimensional state feature vector into the trained deep reinforcement learning model to output a comprehensive decision vector; generating a maintenance work order and a power generation scheduling instruction based on the comprehensive decision vector, and issuing and executing; collecting power grid actual operation data after the instruction is issued and executed, calculating the evaluation results based on the preset evaluation index system, and adjusting the parameters of the deep reinforcement learning model according to the evaluation results to drive the continuous optimization of the model. The present application improves the collaboration of maintenance plan and power generation scheduling, improves the efficiency of power grid operation, guarantees the safety and stability of power supply, and reduces the operation and maintenance cost.

Description

Technical Field

[0001] This invention relates to the field of management platform technology, and in particular to a collaborative optimization system and method for maintenance planning and power generation dispatch based on deep reinforcement learning. Background Technology

[0002] As the power system accelerates its transformation towards an integrated "generation, grid, load, and storage" system, the operating environment of the power grid is undergoing profound changes. The number of grid-connected devices is growing exponentially, maintenance plans are numerous, and the workload of distribution network dispatching is becoming increasingly heavy with urban development. Traditional models are no longer sufficient to handle the management and control needs of multiple concurrent tasks. At the same time, the randomness of new energy power generation and the uncertainty of load fluctuations combine to make the grid's operating status constantly changing. The disconnect between maintenance plans and power generation dispatching can easily lead to power supply gaps or equipment overload risks. Therefore, the need for coordination in complex grid scenarios is becoming a significant driving force for industry development.

[0003] For a long time, maintenance plans have mostly adopted a rolling "annual, quarterly, monthly, weekly, daily" compilation model, but they remain in a state of "each fighting their own battle" with power generation dispatch. The maintenance department focuses on equipment health and resource allocation, while the dispatch department focuses on grid load balance. This data fragmentation across departments leads to a persistently high rate of repeated power outages. Traditional methods such as issuing orders by telephone and manual verification are inefficient, and maintenance plans take a lot of time from submission to implementation, and are prone to affecting dispatch safety due to human error. The existence of this collaboration barrier makes the limitations of the traditional management model increasingly apparent.

[0004] With the advancement of digital transformation in the power industry, data connectivity technology has laid a solid foundation for collaborative optimization. State Grid Tongshan District Company has achieved data connectivity between the OMS and PMS systems through cross-system interfaces, enabling "one-click reporting and multi-system flow" of maintenance information, reducing manual workload by 67%. Yinchuan Power Supply Company has built an intelligent dispatching interaction system, realizing integrated information interaction between local dispatching, distribution dispatching, and on-site operation and maintenance. The establishment of a full-scale dispatching information resource pool further enables the fusion analysis of equipment operation data, power outage event data, and dispatching data, providing data support for collaborative decision-making. However, traditional optimization methods still have significant shortcomings. While heuristic algorithms such as genetic algorithms and simulated annealing can solve single-objective optimization problems, they are prone to getting trapped in local optima and have slow convergence speeds when facing multi-dimensional dynamic constraints such as maintenance resource constraints, grid load fluctuations, and equipment health status. Deep reinforcement learning, with its inherent advantages in handling high-dimensional dynamic systems, learns optimal strategies through continuous interaction between the agent and the environment, accurately capturing the temporal correlation between maintenance plans and power generation dispatch, opening up a new path for solving multi-objective collaborative optimization problems.

[0005] The current pain points of collaborative mechanisms are further forcing technological breakthroughs. While existing systems can perform risk verification, they struggle to respond in real time to complex constraints—the combined effects of load fluctuations during peak electricity demand, random fluctuations in renewable energy output, and sudden equipment warnings often lead to the failure of pre-set maintenance plans and power dispatch schemes. For example, a region experienced a short-term power shortage due to a maintenance plan implemented without considering a sudden drop in wind power output; the root cause was the lack of an intelligent decision-making mechanism capable of dynamically balancing multiple constraints. Furthermore, the coordination of maintenance plans and power dispatch must consider three major objectives: equipment reliability, power supply stability, and operation and maintenance costs. Traditional manual decision-making or static algorithms struggle to achieve optimal multi-objective results. While State Grid Chongqing Electric Power's "Intelligent Maintenance Plan Manager" has shortened the process time, it still relies on pre-set rules for verification and cannot dynamically adjust priorities based on real-time operating conditions. In addition, many existing systems lack a closed-loop mechanism of "execution-evaluation-optimization." While some systems can evaluate the effectiveness of maintenance plan execution, they cannot automatically feed the evaluation results back to the optimization model. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide a collaborative optimization system for maintenance planning and power generation scheduling based on deep reinforcement learning, which improves the synergy between maintenance planning and power generation scheduling and the efficiency of power grid operation, ensures safe and stable power supply, and reduces operation and maintenance costs.

[0007] This invention provides a method for collaborative optimization of maintenance planning and power generation scheduling based on deep reinforcement learning, comprising the following steps: Step 1: Collect multi-source heterogeneous data from the power grid, perform standardization processing and feature extraction on the multi-source heterogeneous data, and construct a multi-dimensional state feature vector; Step 2: Construct a deep reinforcement learning model; The collaborative optimization problem of maintenance planning and power generation scheduling is constructed as a Markov decision process, defining a state space, action space, and reward function. The state space is composed of the multi-dimensional state feature vector generated in step 1. An intelligent agent is constructed, which includes a CNN module for extracting grid topology spatial features, an LSTM module for extracting temporal features, and a fusion layer for fusing spatial features, weighted temporal features, equipment health state vectors, and maintenance resource state vectors to output a comprehensive decision vector. The intelligent agent is trained to obtain a trained deep reinforcement learning model. Step 3: Input the multi-dimensional state feature vector generated in real time into the trained deep reinforcement learning model and output a comprehensive decision vector; generate a maintenance work order based on the maintenance actions in the comprehensive decision vector, generate a power generation dispatch instruction based on the dispatch actions in the comprehensive decision vector, and issue the maintenance work order and the power generation dispatch instruction for execution. Step 4: Collect actual power grid operation data after the command is issued and executed, calculate the evaluation results based on the preset evaluation index system, and adjust the parameters of the deep reinforcement learning model according to the evaluation results to drive the continuous optimization of the model.

[0008] As a further technical solution, step 1 specifically includes: Step 1-1: Extract structured data from the power management system, including equipment ledgers, maintenance work orders, and defect records, using ETL tools, and configure an incremental synchronization mechanism; receive real-time grid topology changes, node voltages, line power flow, and power generation output measurement data from the energy management system via message queues; deploy monitoring equipment at renewable energy power plants to collect real-time power and meteorological data for wind and solar power, with remote plants using LoRaWAN backhaul and supporting breakpoint resume. Load and power generation data are collected through smart meters and power transmitters; refined weather forecasts are obtained by connecting to meteorological department APIs; smart sensors are deployed on key equipment in substations and aggregated to data acquisition terminals via industrial Ethernet; distributed fiber optic sensors are deployed on transmission lines to monitor icing and galloping conditions. Steps 1-2: Use outlier detection algorithms to clean the collected data, remove noisy and abnormal fluctuation data, and transform it into data with unified standards; Steps 1-3: Extract key features from maintenance data and scheduling data, perform dimensionality reduction optimization on the key features, and construct the multidimensional state feature vector that is finally input into the deep reinforcement learning model.

[0009] As a further technical solution, the dimensionality reduction optimization method is as follows: Principal component analysis is used to reduce the dimensionality of data containing multiple dimensions, retaining several principal components with a cumulative contribution rate of 95%, and using the principal component scores as new features to input into the deep reinforcement model in subsequent steps; then, mutual information feature selection is used to calculate the mutual information value between each original feature and the two target variables of equipment failure and overload limit, and features with mutual information greater than 0.1 are selected to further reduce dimensionality.

[0010] As a further technical solution, step 2 specifically includes: Step 2-1: Construct the collaborative optimization problem of maintenance plan and power generation dispatch as a Markov decision process, and define the agent and environment model; Step 2-2: Build the network architecture of the intelligent agent; specifically including: Step 2-2-1: Construct a CNN module to receive adjacency matrix data representing the power grid topology, and extract local spatial connectivity features in the adjacency matrix through multi-layer two-dimensional convolution operations, and output spatial feature vectors; Step 2-2-2: Construct an LSTM module to receive time series data sliced ​​at preset time intervals, extract time-series dependency features from the time series data through a cyclic connection structure, and output an initial time-series feature vector; Step 2-2-3: Introduce a temporal attention layer to weight temporal features; Step 2-2-4: Construct a fusion layer for feature stitching and output; Step 2-2-5: Connect the modules that have been constructed above according to the data flow and encapsulate them into a unified intelligent agent; Steps 2-3: Use an experience replay mechanism to store historical experience data generated by the agent's interaction with the environment, and randomly select data for training; combine the near-end policy optimization algorithm and the deep Q-network algorithm to train the model, and continuously adjust the network parameters so that the agent learns the optimal decision-making strategy; at the same time, update the target network regularly. As a further technical solution, the defined environment model includes refined state space construction, action space discretization and constraint processing, and dynamic adjustment of multi-objective weights of the reward function; The refined state space construction includes four dimensions: power grid operation status, equipment health status, maintenance resource status, and new energy output prediction. Action space discretization and constraint handling include maintenance plan adjustment and power generation dispatch scheme optimization; The dynamic adjustment of multi-objective weights in the reward function includes: adopting a hybrid weight calculation method based on entropy weighting and analytic hierarchy process, dynamically adjusting the weights of each objective according to the power grid operation scenario, and setting a penalty mechanism including equipment failure penalty, repeated power outage penalty, and resource over-allocation penalty.

[0011] As a further technical solution, the LSTM module is implemented using a Long Short-Term Memory (LSTM) network. The LSTM network adopts a two-layer structure. The first layer outputs the hidden state sequence of the complete time step, and the second layer outputs only the hidden state of the last time step, or outputs the hidden state sequence of all time steps for subsequent attention layer processing. The hidden state dimension of each layer of the LSTM network is set according to the input data scale and computing resources.

[0012] As a further technical solution, in step 2-2-3, a time attention layer is connected after the output of the time series feature extraction sub-network; the time attention layer is used to dynamically calculate the importance weight of the hidden state at each time point according to the time interval between each historical time point and the current decision time point in the time series data, and to perform weighted summation of the hidden states at each time point according to the weights to generate a weighted time series feature vector.

[0013] As a further technical solution, in step 2-2-4, the fusion layer is connected to the output of the CNN module, the output of the LSTM module, and the provider of the auxiliary state vector, respectively; Before the device health state vector is input into the fusion layer for splicing, a self-attention mechanism is applied to the device health state vector.

[0014] This invention also provides a collaborative optimization system for maintenance planning and power generation dispatch based on deep reinforcement learning, comprising: The data fusion and preprocessing module is used to collect data from all aspects of the power grid, standardize the data, and construct feature data. A deep reinforcement learning model is used to collaboratively optimize maintenance plans and power generation dispatch schemes under multi-dimensional constraints of grid operation, equipment health, and new energy forecasting. The system parameter adjustment module is used to receive feedback instructions generated by the effect evaluation and feedback application module, and dynamically adjust the reward function weight coefficients, training hyperparameters or model instances of the deep reinforcement learning module according to the feedback instructions, so as to drive the continuous evolution of the strategy of the deep reinforcement learning module. The effect evaluation and feedback application module is used to collect the actual operation data of the power grid after the decision support module issues the instructions, and to quantitatively evaluate the decision effect based on the preset multi-dimensional evaluation index system, and generate feedback instructions based on the evaluation results. The decision support module, connected to the output of the deep reinforcement learning module, is used to convert the optimization scheme output by the deep reinforcement learning model into executable maintenance work orders and power generation dispatch instructions, and to provide a visual interactive decision-making environment.

[0015] As a further technical solution, the deep reinforcement learning module internally incorporates an intelligent agent. The network architecture of the intelligent agent consists of three sub-modules. CNN module: Used to receive adjacency matrix data representing the topology of the power grid, extract local spatial connectivity features in the adjacency matrix through convolution operations, and output spatial feature vector; LSTM module: used to receive time series data sliced ​​at preset time intervals, extract time-dependent features from the time series data through a recurrent neural network structure, and output a time-series feature vector; Fusion layer: Connected to the output of the CNN module and the output of the LSTM module respectively, it is used to concatenate and fuse the spatial feature vector, the temporal feature vector and the auxiliary state vector representing the health status and maintenance resource status of the equipment, and output a decision vector containing maintenance actions and scheduling actions after being processed by a fully connected layer.

[0016] As a further technical solution, the output of the LSTM module is also connected to a time attention layer, which is used to dynamically allocate the weights of the features at each time point in the time series according to the time interval between each time point in the time series and the current decision time point.

[0017] As a further technical solution, before inputting the device health state vector into the LSTM module for concatenation, a self-attention mechanism is first applied to the device health state vector; the self-attention mechanism performs weighted aggregation processing on the device health state vector by calculating the correlation weights between each device health state vector.

[0018] As a further technical solution, the system parameter adjustment module is divided into three functional sub-units: The reward function weight dynamic adjustment subunit is used to adjust the weight coefficients of each optimization objective of the reward function in the deep reinforcement learning module; The online hyperparameter fine-tuning subunit is used to adjust the training-related hyperparameters of the agent network in the deep reinforcement learning module, as well as some weight parameters of the fusion layer. The Model Retraining Triggering and Scheduling Subunit is used to adjust the overall model instance of the deep reinforcement learning module.

[0019] As a further technical solution, the effect evaluation and feedback application module includes: an online evaluation and comparison unit, used to use an online comparison testing framework to proportionally divide the actual operating traffic into the current decision scheme and the comparison decision scheme, compare the cumulative reward value and constraint violation number of the two schemes under the same boundary conditions in real time, and generate an evaluation report including a risk heatmap and a scheme economic comparison table; a closed-loop feedback unit, used to push the quantitative indicators in the evaluation report to the system parameter adjustment module to trigger the recalculation of reward function weights or hyperparameter adjustment, and push the high-quality decision sample data generated during the evaluation process to the data management module of the deep reinforcement learning module as supplementary samples for online fine-tuning; and an automatic retraining trigger unit, used to automatically start the offline retraining process of the deep reinforcement learning module when the average cumulative reward within a continuous preset period is lower than a preset percentage threshold of the historical best value, or when the deterioration of key safety indicators exceeds a preset tolerance threshold, and the new model replaces the online model after gray-scale release verification.

[0020] As a further technical solution, the decision support module includes: a maintenance plan collaborative optimization unit, used to call the maintenance actions output by the deep reinforcement learning module, perform N-1 safety verification in conjunction with the power system simulation model, and convert the verified maintenance actions into structured maintenance work orders, which include safety measures, process flow, required tools and equipment, and personnel qualification requirements; a generation dispatch collaborative optimization unit, used to update the grid operation status according to a preset cycle, call the dispatch actions output by the deep reinforcement learning module, generate automatic generation control commands and automatic voltage control commands, and send the commands to the generation side and energy storage side control systems through a communication protocol conforming to IEC 60870-5-104 or IEC 61850 standards; and a visualization interactive decision unit, used to construct a virtual grid visualization interface based on three-dimensional digital twin technology, and overlay and display equipment health status layers, power flow distribution layers, and maintenance plan layers on the interface, while supporting dispatchers to adjust decision preference parameters through human-computer interaction and triggering the deep reinforcement learning module to recalculate the optimal solution in real time.

[0021] Compared with existing technologies, the maintenance planning and power generation scheduling collaborative optimization system based on deep reinforcement learning of the present invention has the following beneficial effects: (1) Through the data fusion and preprocessing module, multi-source data such as power management system (PMS) and energy management system (EMS) are integrated to break down information silos and realize automatic data interaction and sharing. The deep reinforcement learning model can analyze information such as power grid operation status and equipment health status in real time, quickly generate collaborative optimization schemes, and significantly shorten the time for plan formulation and adjustment. Taking maintenance plan as an example, the system can automatically optimize maintenance time window to avoid conflict with peak power grid load periods, reduce the workload of repeated manual coordination and adjustment, significantly improve overall work efficiency, and reduce manpower and time costs.

[0022] (2) By leveraging the ability of deep reinforcement learning to process high-dimensional dynamic systems, the grid operation status, changes in new energy output and equipment health status can be perceived in real time, and maintenance plans and power generation scheduling schemes can be dynamically adjusted to ensure that the grid supply and demand can be effectively balanced under various complex operating conditions, avoid chain reactions caused by equipment failures, and ensure the safety and stability of power supply.

[0023] (3) Integrating multi-objective requirements into the learning process of the intelligent agent. In terms of equipment maintenance, the system can rationally arrange maintenance time and resources based on characteristics such as equipment health index and maintenance urgency index, avoiding over-maintenance and reducing maintenance costs. In terms of power generation dispatch, by optimizing unit combination and output allocation, the system can improve the capacity for new energy consumption, reduce fossil energy consumption, and lower power generation costs. At the same time, ensuring power supply stability can reduce economic losses caused by power outages, ultimately reducing the overall operation and maintenance costs of the power system and improving economic efficiency.

[0024] (4) A complete closed-loop optimization system was established by utilizing the incremental learning characteristics of deep reinforcement learning. The effect evaluation and feedback application module of the application layer uses a multi-dimensional effect evaluation index system to quantitatively evaluate the collaborative optimization effect of maintenance plans and power generation dispatch. The evaluation results are fed back to the deep reinforcement learning model and the system parameter adjustment module. The agent continuously learns and improves its strategies based on the feedback, and continuously optimizes the decision-making model. This self-learning capability enables the system to continuously adapt to changes in the power grid's operating status, transforming from the traditional "experience-driven" decision-making mode to a new "data-driven, intelligent decision-making" mode, thereby improving the intelligent management level of the power system.

[0025] (5) By comprehensively analyzing the status of maintenance resources and power generation resources, the system can rationally arrange the combination of maintenance equipment, optimize the power output allocation of units, and set up reserve capacity. In terms of maintenance, based on the health status of equipment and the status of maintenance resources, manpower and material resources are centrally allocated to improve the utilization efficiency of maintenance resources. In terms of power generation dispatch, new energy power generation is fully utilized, and the operation mode of traditional energy units such as thermal power and hydropower is optimized to make power generation resources more rationally allocated, avoid resource waste, and improve the overall resource utilization efficiency. Detailed Implementation

[0026] To further understand the present invention, embodiments of the present invention are described below in conjunction with examples. However, it should be understood that these descriptions are only for further illustrating the features and advantages of the present invention, and not for limiting the present invention.

[0027] An embodiment of the present invention discloses a collaborative optimization system for maintenance planning and power generation dispatch based on deep reinforcement learning, comprising: The data fusion and preprocessing module is used to collect multi-source heterogeneous data from all aspects of the power grid, perform standardization processing on it, and construct feature data. Specifically, the data across the entire power grid includes maintenance-related data such as equipment ledgers, maintenance history, and defect records obtained from the power management system; dispatch data such as actual power grid topology, node voltage, line power flow, and power generation output collected from the energy management system; real-time power prediction data for wind and solar power collected through monitoring equipment deployed at renewable energy power plants, such as physical quantities like equipment temperature, vibration, current, voltage, and partial discharge; power grid load data and power generation data collected through smart meters, power transmitters, and other equipment; and weather forecast data from meteorological departments, such as irradiance, wind speed, and wind direction, providing a comprehensive data foundation for the system.

[0028] The standardization process includes: using outlier detection algorithms, such as the 3σ principle and the isolated forest algorithm, to clean the collected data and remove noisy and abnormally fluctuating data. A unified data standard is established to convert and unify the encoding of data from different sources and in different formats, such as standardizing information like device serial numbers and geographical locations, to ensure data consistency and compatibility, facilitating subsequent analysis and processing. The construction of feature data includes: extracting key features based on the power system's operating mechanism and business needs; constructing feature data such as equipment health index and maintenance urgency index for maintenance data; and extracting feature data such as load peak-valley difference, renewable energy penetration rate, and weak links in the power grid for dispatch data. Principal component analysis (PCA) and mutual information methods are used to reduce the dimensionality of the feature data, thereby reducing data redundancy and improving model training efficiency.

[0029] A deep reinforcement learning model is used to collaboratively optimize maintenance plans and power generation dispatch schemes under multi-dimensional constraints of grid operation, equipment health, and new energy forecasting. The deep reinforcement learning module contains an intelligent agent. The network architecture of the intelligent agent consists of three sub-modules. CNN module: Used to receive adjacency matrix data representing the topology of the power grid, extract local spatial connectivity features in the adjacency matrix through convolution operations, and output spatial feature vector; For example, the adjacency matrix representing the power grid topology is used as input. The adjacency matrix has a dimension of N×N, where N is the number of power grid nodes, and the matrix elements represent the connection relationships between nodes and the line admittance values. The CNN module is implemented using a convolutional neural network, containing three convolutional layers and two pooling layers. The first convolutional layer uses a 3×3 kernel with a stride of 1 and 32 output channels; the second convolutional layer uses a 3×3 kernel with a stride of 1 and 64 output channels; the third convolutional layer uses a 3×3 kernel with a stride of 1 and 128 output channels. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function layer. The pooling layer uses 2×2 max pooling. After convolution and pooling processing, a spatial feature vector of dimension D1 is output.

[0030] LSTM module: used to receive time series data sliced ​​at preset time intervals, extract time-dependent features from the time series data through a recurrent neural network structure, and output a time series feature vector; preferably, the output of the LSTM module is also connected to a time attention layer, used to dynamically allocate the weights of the features at each time point in the time series and the current decision time point according to the time interval between each time point in the time series and the current decision time point.

[0031] For example, the system receives time-series data sliced ​​at 15-minute intervals. This time-series data includes load curve data from the past 24 hours, renewable energy output forecast data for the next 48 hours, and historical trend data of equipment health indicators. The LSTM module is implemented using a two-layer Long Short-Term Memory (LSTM) network, with each layer having a hidden state dimension of 128. The first LSTM layer outputs the complete time-step sequence, while the second LSTM layer only outputs the hidden state of the last time step. A temporal attention layer is connected after the second LSTM layer. This temporal attention layer calculates the correlation score between the hidden state at each historical moment and the current decision moment using a trainable weight matrix, and then performs a weighted sum of the features at each moment after normalization using the Softmax function, outputting a weighted temporal feature vector with dimension D2.

[0032] Fusion layer: Connected to the output of the CNN module and the output of the LSTM module respectively, it is used to concatenate and fuse the spatial feature vector, the temporal feature vector and the auxiliary state vector representing the health status and maintenance resource status of the equipment, and output a decision vector containing maintenance actions and scheduling actions after being processed by the fully connected layer. Preferably, before inputting the device health state vector into the LSTM module for concatenation, a self-attention mechanism is applied to the device health state vector. This self-attention mechanism calculates the correlation weights between the device health state vectors and performs weighted aggregation processing on them to enhance the expression strength of devices with abnormal health states in the fused features, thereby improving the agent network's sensitivity to potential device failure risks.

[0033] For example, the spatial feature vector of dimension D1 output by the CNN module, the weighted temporal feature vector of dimension D2 output by the LSTM module, and the auxiliary state vector of dimension D3 are concatenated to obtain a fusion vector of dimension (D1+D2+D3). The auxiliary state vector includes the equipment health state vector and the maintenance resource state vector. Preferably, before concatenation, a self-attention mechanism is applied to the equipment health state vector to enhance the expression intensity of abnormal equipment states in the fusion vector. The concatenated fusion vector is input into a decision network containing two fully connected layers. The first fully connected layer contains 256 neurons, and the second fully connected layer contains 128 neurons, both using the ReLU activation function. The output layer of the decision network is divided into two branches: the first branch outputs the probability distribution of maintenance actions, using the Softmax activation function; the second branch outputs the Q-value estimate of the power generation scheduling level, using a linear activation function.

[0034] The system parameter adjustment module is connected to the effect evaluation and feedback application module and the deep reinforcement learning module, respectively. It is used to receive feedback instructions generated by the effect evaluation and feedback application module, and dynamically adjust the reward function weight coefficients, training hyperparameters or model instances of the deep reinforcement learning module according to the feedback instructions, so as to drive the continuous evolution of the strategy of the deep reinforcement learning module. The system parameter adjustment module receives feedback instructions from the effect evaluation and feedback application module. These instructions include: deviation of the quantification evaluation index, cumulative reward decay rate, statistical values ​​of constraint violation counts, and control signals to trigger retraining. The adjustment actions are applied to the deep reinforcement learning module, specifically affecting: the multi-objective weight coefficient vector of the reward function, hyperparameters related to agent training, sampling policy parameters of the experience replay pool, and the start signal to trigger offline model retraining.

[0035] Based on the different objects being adjusted and the response delay, the system parameter adjustment module is divided into three functional sub-units: The reward function weight dynamic adjustment subunit is used to adjust the weight coefficients of each optimization objective of the reward function in the deep reinforcement learning module; Triggering condition: The effect evaluation module detects short-term indicator deviations, such as the new energy consumption rate being lower than the target value by 5% for two consecutive hours, but the key safety indicators remain normal. This is determined to be a shift in the distribution of the operating scenario rather than model degradation.

[0036] Adjustment Mechanism: A hybrid weight calculation program combining entropy weighting and analytic hierarchy process (AHP) is invoked. Using real-time power grid operation data as input, the objective weights (entropy weighting) and subjective weights (AHP) of each objective are recalculated and weighted to generate a new weight vector. These objectives include: power supply stability, renewable energy consumption, equipment safe operation, maintenance economy, and network loss optimization. The updated weight vector is pushed to the reward calculation interface of the deep reinforcement learning module, causing the agent's reward sensitivity to specific objectives to shift during subsequent inference, thus automatically adapting its action output to the current operating conditions.

[0037] The online hyperparameter fine-tuning subunit is used to adjust the training-related hyperparameters of the agent network in the deep reinforcement learning module, as well as some weight parameters of the fusion layer.

[0038] Triggering condition: The performance evaluation module detects a slow but continuous decline in model performance, such as the average cumulative reward over seven consecutive days being less than 90% of the historical best value, but not yet reaching the safety threshold.

[0039] Adjustment mechanism: Limited online fine-tuning: The actual operation and interaction data of the previous day stored in the data management module are used as supplementary samples to perform small-batch, multi-round supervised fine-tuning of the fully connected weights of the fusion layer of the agent network.

[0040] Maximum update step size constraint: Sets an upper norm limit on the change in the network weight vector by a single fine-tuning, to prevent the core safety strategy learned by the network during the simulation training phase from being violated by fitting short-term noisy data.

[0041] Learning rate decay reconfiguration: Adjust the optimizer's learning rate decay factor based on recent fluctuations in the loss function.

[0042] The Model Retraining Triggering and Scheduling Subunit is used to adjust the overall model instance of the deep reinforcement learning module. Triggering conditions include hard triggering conditions and soft triggering conditions; Hard trigger condition: Key safety indicators deteriorate beyond the preset tolerance threshold, such as the N-1 pass rate dropping by more than 5%.

[0043] Soft trigger condition: The average cumulative reward within a consecutive preset period is lower than the preset percentage of the historical best value.

[0044] Adjustment mechanism: Automatic retraining process initiated: A retraining command is sent to the algorithm training platform, which automatically extracts recent full historical data from the data warehouse and mixes it with typical fault samples generated in the simulation environment to construct a retraining dataset.

[0045] Siamese network training: Start a Siamese network with the same architecture as the online model in the background parallel computing cluster, and use the aforementioned training and optimization strategies to train from scratch or fine-tune the training based on the current weights until the cumulative reward of the new model on the validation set recovers to the historical high level.

[0046] Canary release control: After the new model is trained, only a portion of non-critical decision traffic is routed to the new model, and its performance is closely monitored by the effect evaluation module.

[0047] Hot replacement command issuance: When the comprehensive indicators of the new model are significantly and consistently better than the old model during the gray-scale operation cycle, a hot replacement command is issued to seamlessly switch all decision traffic to the new model.

[0048] The effect evaluation and feedback application module is used to collect the actual operation data of the power grid after the decision support module issues instructions, and to quantitatively evaluate the decision effect based on a preset multi-dimensional evaluation index system, and generate feedback instructions based on the evaluation results.

[0049] The multidimensional evaluation index system includes technical indicators, economic indicators, and safety indicators.

[0050] Technical indicators include mean time between failures (MTBF), N-1 pass rate, and voltage qualification rate; economic indicators include maintenance cost savings rate, renewable energy consumption rate, and grid loss reduction rate; safety indicators include repeated power outages, resource idle rate, and fault underreporting rate. The indicators are automatically collected through a data warehouse, and future trends are predicted using the ARIMA time series analysis algorithm.

[0051] The effect evaluation and feedback application module includes: an online evaluation and comparison unit, which uses an online comparison testing framework to proportionally divide the actual operating traffic into the current decision scheme and the comparison decision scheme, compares the cumulative reward value and constraint violation number of the two schemes under the same boundary conditions in real time, and generates an evaluation report including a risk heatmap and a scheme economic comparison table; a closed-loop feedback unit, which pushes the quantitative indicators in the evaluation report to the system parameter adjustment module to trigger the recalculation of reward function weights or hyperparameter adjustment, and pushes the high-quality decision sample data generated during the evaluation process to the data management module of the deep reinforcement learning module as supplementary samples for online fine-tuning; and an automatic retraining trigger unit, which automatically starts the offline retraining process of the deep reinforcement learning module when the average cumulative reward within a continuous preset period is lower than a preset percentage threshold of the historical best value, or when the deterioration of key safety indicators exceeds a preset tolerance threshold. The new model replaces the online model after gray-scale release verification.

[0052] This module forms a closed loop with the deep reinforcement learning model, the system parameter adjustment module, and the decision support module: it receives the actual execution results issued by the decision support module and outputs evaluation reports and feedback instructions.

[0053] The decision support module is used to transform the optimization schemes output by the deep reinforcement learning model into executable maintenance work orders and power generation dispatch instructions, and provides a visual interactive decision-making environment. The decision support module is connected to the output of the deep reinforcement learning module. The decision support module includes: a maintenance plan collaborative optimization unit, used to call the maintenance actions output by the deep reinforcement learning module, perform N-1 safety checks in conjunction with a power system simulation model, and convert the checked maintenance actions into structured maintenance work orders, which include safety measures, process flows, required tools and equipment, and personnel qualification requirements; a generation dispatch collaborative optimization unit, used to update the grid operating status according to a preset cycle, call the dispatch actions output by the deep reinforcement learning module, generate automatic generation control commands and automatic voltage control commands, and send the commands to the generation-side and energy storage-side control systems through a communication protocol conforming to IEC 60870-5-104 or IEC 61850 standards; and a visualization interactive decision unit, used to construct a virtual grid visualization interface based on 3D digital twin technology, and overlay and display equipment health status layers, power flow distribution layers, and maintenance plan layers on the interface, while also supporting dispatchers to adjust decision preference parameters through human-computer interaction and triggering the deep reinforcement learning module to recalculate the optimal solution in real time.

[0054] In summary, the data fusion and preprocessing module collects and processes multi-source data to construct features; the agent in the deep reinforcement learning model outputs maintenance and scheduling decisions based on the current state; the decision support module converts the decisions into executable instructions and issues them; the effect evaluation module monitors the execution results and feeds them back to the system parameter adjustment module; the system parameter adjustment module dynamically optimizes hyperparameters and reward weights, driving the continuous evolution of the deep reinforcement learning model. This forms a complete closed loop of "perception-decision-execution-evaluation-optimization," achieving collaborative intelligent optimization of power grid maintenance and power generation scheduling.

[0055] Embodiments of the present invention also disclose a method for collaborative optimization of maintenance planning and power generation dispatch based on deep reinforcement learning, comprising the following steps: Step 1: Collect multi-source heterogeneous data from all aspects of the power grid, perform standardization processing and feature extraction, and construct a multi-dimensional state feature vector; The multi-source heterogeneous data across the entire power grid includes maintenance-related data such as equipment ledgers, maintenance history, and defect records obtained from the power management system; dispatch data such as actual power grid topology, node voltage, line power flow, and generation output collected from the energy management system; and real-time power prediction data for wind and solar power collected through monitoring equipment deployed at renewable energy power plants, including physical quantities such as equipment temperature, vibration, current, voltage, and partial discharge. The monitoring equipment can be intelligent sensors, deployed on key equipment, with a data acquisition frequency of no less than 10kHz, and aggregated to a data acquisition terminal via industrial Ethernet. Distributed fiber optic sensors are deployed on transmission lines to monitor icing and galloping conditions, with a positioning accuracy of up to 100 meters.

[0056] The system collects grid load data and power generation data through devices such as smart meters and power transmitters; it also accesses weather forecast data from meteorological departments, such as light intensity, wind speed and direction, to provide a comprehensive data foundation for the system.

[0057] Establish a secure data channel with the power management system through Web Service interfaces or database middleware; Based on the CIM interface of the IEC 61970 standard, real-time data interaction with the energy management system is realized; a message queue is deployed to receive grid topology changes and real-time measurement data, and key node data is refreshed once per second to ensure the real-time status of grid operation.

[0058] Edge computing gateways are deployed at new energy power plants such as wind farms and photovoltaic power stations to collect inverter power and meteorological monitoring data via protocols such as Modbus / TCP and DL / T645. LoRaWAN low-power wide area network technology is used to transmit data from remote power plants back to the central server, supporting breakpoint resume functionality. The meteorological monitoring instrument acquires refined meteorological forecast data with a spatial resolution of 1km×1km and a temporal resolution of 15 minutes. It focuses on collecting data on solar irradiance, wind speed and direction, temperature and humidity, which affect renewable energy power generation, while also incorporating historical meteorological data for model training. Methods for standardizing data include: Outlier detection algorithms, such as the 3σ rule and the isolated forest algorithm, are used to clean the collected data, removing noisy and abnormally fluctuating data. A unified data standard is established to convert and standardize the encoding of data from different sources and in different formats. For example, information such as device serial numbers and geographical locations are standardized to ensure data consistency and compatibility, facilitating subsequent analysis and processing. The 3σ principle refers to calculating the mean μ and standard deviation σ of historical data for electrical quantities such as current and voltage that follow a normal distribution, and marking data outside the range of [μ-3σ, μ+3σ] as outliers. A sliding window is used to dynamically update the mean and standard deviation to adapt to changes in power grid operation. The Isolation Forest algorithm refers to constructing an ensemble model of multiple isolated trees to detect anomalies in non-stationary data such as equipment temperature and vibration. An anomaly score threshold of 0.8 is set, and data exceeding this threshold undergoes secondary verification. Interpolation repair is performed using data from adjacent measurement points, such as filling missing values ​​with cubic spline interpolation. Key features are extracted from the standardized data. Specifically, for maintenance data, derived data such as hydrogen growth rate and acetylene percentage are calculated using transformer oil chromatography data; the proportion of high-frequency components in the circuit breaker vibration spectrum is extracted as a mechanical wear feature. Based on this multi-source information, an equipment health index is constructed, and a maintenance urgency index is further calculated. For scheduling data, the sliding window method is used to calculate the load peak-valley difference rate, the real-time renewable energy penetration rate is calculated, and N-1 verification is performed through topology analysis to mark weak links in the power grid.

[0059] The extracted key features are subjected to dimensionality reduction using principal component analysis, retaining principal components whose cumulative contribution rate reaches a preset threshold; at the same time, mutual information algorithm is used to screen features whose correlation strength with the target variable is greater than a preset threshold, forming the multidimensional state feature vector that is finally input into the deep reinforcement learning model.

[0060] The principal component analysis specifically involves performing principal component analysis on data containing multiple temperatures, retaining several principal components with a cumulative contribution rate reaching 95%. The principal component scores are then used as new features and input into the deep reinforcement model in subsequent steps, reducing computational load while avoiding information loss.

[0061] The mutual information feature selection is specifically as follows: calculate the correlation strength between each original feature and the two targets "equipment failure" and "load exceeding limit", retain only 30 key features with a correlation strength greater than 0.1, and remove the rest of useless or weakly correlated features, thereby achieving dimensionality reduction.

[0062] The multidimensional state feature vector specifically includes: an adjacency matrix encoding the power grid topology into node attributes including voltage amplitude, phase angle, and normalized power flow data; an equipment health state vector including real-time equipment monitoring data, historical defect codes, and predicted remaining service life; a maintenance resource state vector represented by multidimensional vector quantization, including maintenance personnel skill level, number of available tools, and vehicle dispatch status; and a new energy output prediction vector including predicted power curves and prediction confidence intervals for a future preset time period.

[0063] It also includes: feature updates; Feature updates are implemented by automatically refreshing features through daily scheduled updates and event triggers, while a version control tool records each change in the feature calculation logic to ensure that the feature definitions and processing methods used are consistent regardless of when the model is trained or online inference is performed.

[0064] Step 2: Build and train a deep reinforcement learning model; Specifically, it includes: Step 2-1: Construct the collaborative optimization problem of maintenance plan and power generation dispatch as a Markov decision process, and define the agent and environment model; An intelligent agent is a learner or controller responsible for making decisions. It observes the state of the environment, selects actions according to policies, and improves its decisions through the rewards it receives.

[0065] The environment model is the part outside the agent that is affected by actions and provides feedback. It receives actions from the agent, updates its own state, and generates a reward signal and the next state.

[0066] By defining the state space, action space, and reward function in detail, the actual power grid maintenance and scheduling problem is formalized into a Markov decision process, thereby clarifying the interaction boundary and optimization objective between the agent and the environment. Specifically, this includes: refined state space construction, action space discretization and constraint handling, and dynamic adjustment of multi-objective weights in the reward function; The refined state space construction includes four dimensions: power grid operation status, equipment health status, maintenance resource status, and new energy output prediction. The power grid operating status encodes the power grid topology as an adjacency matrix, with node attributes including voltage amplitude, phase angle, load type, and other information. Line power flow data is collected at 15-minute intervals and normalized to the [-1, 1] interval. Power grid vulnerability indicators, such as line overload margin calculated based on the DC power flow method, are introduced as a supplementary dimension to the state space. For each type of equipment, a health status vector is established, which includes real-time monitoring data, historical defect record codes, and predicted remaining service life. A health status decay factor is set to dynamically adjust the data weights as the equipment operates. The maintenance resource status quantifies information such as the skill level of maintenance personnel, the quantity of available tools and equipment, and the scheduling status of maintenance vehicles, and uses a multi-dimensional vector representation. By establishing a resource constraint matrix, the allocation of maintenance tasks with resource conflicts within the same time period is restricted. The power output forecast for new energy sources is integrated with numerical weather prediction (NWP) data, combined with the ramp rate and forecast error statistics of wind farms or photovoltaic power plants, to construct a state vector containing the predicted power curve for the next 48 hours. Forecast uncertainty is modeled, and a forecast confidence interval is added as a state feature. Action space discretization and constraint handling include maintenance plan adjustment and power generation dispatch scheme optimization; The maintenance plan adjustment involves discretizing the maintenance time window into decision points in hours. Equipment combination adjustments use binary coding, where 1 indicates maintenance is selected and 0 indicates no maintenance. For critical equipment such as the main transformer in a hub substation, mandatory maintenance constraints are set to ensure that maintenance is prioritized when safety thresholds are met.

[0067] The optimization of the power generation dispatch scheme involves dividing the unit output range into N discrete levels, each corresponding to a different power generation efficiency curve. A tiered adjustment strategy is adopted for reserve capacity settings, dynamically adjusting the reserve ratio based on load forecast errors. Unit start-up and shutdown constraints are established to avoid losses caused by frequent start-ups and shutdowns.

[0068] The reward function dynamically adjusts the weights of multiple objectives, specifically including: A hybrid weighting method based on entropy weighting and the analytic hierarchy process (AHP) is adopted to dynamically adjust the weights of each target according to the power grid operation scenario. During peak electricity consumption periods, the weight of power supply stability is increased; during periods of high renewable energy generation, the weight of renewable energy consumption is emphasized. Add a penalty term to the reward function and dynamically adjust the weights of multiple objectives. The penalty term is a negative reward. When the agent makes a bad decision, it is penalized immediately, so that the agent learns to avoid these behaviors.

[0069] For example, an immediate penalty of -1000 is imposed for equipment failure; a penalty of -500 is imposed for each repeated power outage; and over-allocation of maintenance resources is deducted according to the resource idle rate. The weighting parameters are reassessed hourly by monitoring the power grid status in real time. Step 2-2: Build the network architecture for the intelligent agents; The network architecture of the intelligent agent includes a CNN module, an LSTM module, and a fusion layer; Specifically, the following steps are included: Step 2-2-1: Construct a CNN module, which is a convolutional neural network layer group, used to receive adjacency matrix data representing the power grid topology, and extract local spatial connection features in the adjacency matrix through multi-layer two-dimensional convolution operations, and output spatial feature vector; The adjacency matrix has a dimension of N×N, where N is the number of power grid nodes, and the matrix elements represent the connection relationships between nodes and the line admittance value or the normalized branch impedance value. The convolutional neural network layer group contains multiple convolutional layers and pooling layers stacked alternately. Each convolutional layer uses a convolutional kernel of a preset size to perform sliding convolution operations on the input feature map, extracting the topological connection patterns within the node and its neighborhood. Each convolutional layer is followed by a nonlinear activation function layer to enhance the network's nonlinear expressive power. The pooling layer uses max pooling or average pooling operations to downsample and reduce the dimensionality of the feature map output by the convolutional layers, reducing the computational cost of subsequent layers while preserving significant topological features.

[0070] Preferably, the number of convolutional layers in the CNN module is configured differently according to the power grid voltage level. For example, for a 110kV power grid, three convolutional layers and two pooling layers are stacked alternately; for a 500kV power grid, five convolutional layers and four pooling layers are stacked alternately. The convolutional kernel size is preferably 3×3 with a stride of 1, and the number of output channels of each convolutional layer increases with the network depth.

[0071] Step 2-2-2: Construct an LSTM module, which is a recurrent neural network layer group, used to receive time series data sliced ​​at preset time intervals, extract the temporal dependency features in the time series data through a recurrent connection structure, and output an initial temporal feature vector; The time series data includes load curve data for a preset past period, renewable energy output forecast data for a preset future period, and historical trend data of equipment health indicators. The time series data is sliced ​​at fixed time intervals to form a continuous time step sequence.

[0072] The LSTM module is preferably implemented using a Long Short-Term Memory (LSTM) network, which effectively captures long-term dependencies in the time series through a gating mechanism. The LSTM network can employ a two-layer structure: the first layer outputs the hidden state sequence for the complete time step, and the second layer outputs only the hidden state of the last time step, or outputs the hidden state sequence for all time steps for subsequent attention layers. The hidden state dimension of each LSTM layer is set according to the input data size and computational resources, for example, 128 dimensions.

[0073] Step 2-2-3: Introduce a temporal attention layer to weight temporal features; A temporal attention layer is connected after the output of the temporal feature extraction subnetwork. The temporal attention layer is used to dynamically calculate the importance weight of the hidden state at each time step based on the time interval between each historical time step and the current decision time step in the time series data, and to perform a weighted summation of the hidden states at each time step according to the weights to generate a weighted temporal feature vector.

[0074] Specifically, the temporal attention layer maps the hidden state at each time step to an attention score using a trainable weight matrix. The score reflects the relevance of historical information at that time step to the current decision. For the new energy output prediction curve, recent prediction data is usually assigned a higher attention weight due to its smaller prediction error, while long-term prediction data is assigned a lower weight due to increased uncertainty. The attention scores are processed by a normalized exponential function to form a weight distribution, which is then multiplied and accumulated with the hidden state at the corresponding time step to obtain the final weighted temporal feature vector.

[0075] Step 2-2-4: Construct a fusion layer for feature stitching and output; The fusion layer is connected to the output of the CNN module, the output of the LSTM module, and the provider of the auxiliary state vector. The fusion layer is used to concatenate the spatial feature vector output by the CNN module, the weighted temporal feature vector output by the LSTM module, the equipment health state vector representing the equipment health status, and the maintenance resource state vector representing the maintenance resource status to generate a fused feature vector.

[0076] Preferably, before inputting the device health status vector into the fusion layer for concatenation, a self-attention mechanism is applied to the device health status vector. This self-attention mechanism calculates the correlation weights between different device health status vectors at the same time, performing weighted aggregation on the device health status vectors to enhance the feature representation strength of devices in abnormal states, enabling the fusion layer to more sensitively perceive potential device failure risks.

[0077] The concatenation operation links the aforementioned feature vectors end-to-end along their feature dimensions, forming a composite vector whose dimension is the sum of the dimensions of each vector. This composite vector is then input into a decision network for processing. The decision network contains at least one fully connected layer, each followed by a nonlinear activation function layer. The final output layer of the decision network is divided into two branches according to task requirements: the first branch outputs the probability distribution of maintenance actions, which can use a normalized exponential function as the activation function; the second branch outputs the value estimate of discrete power generation scheduling positions, which can use a linear activation function. The outputs of the decision network collectively constitute the composite decision vector of the agent network.

[0078] Step 2-2-5: Connect the modules that have been constructed above according to the data flow and encapsulate them into a unified intelligent agent.

[0079] Define a standardized input interface for the intelligent agent network. The input interface receives a multi-dimensional state feature vector generated by the data fusion and preprocessing module. The multi-dimensional state feature vector includes at least: a power grid topology adjacency matrix, time-series load and prediction data slices, equipment health state vectors, and maintenance resource state vectors.

[0080] The output specification of the agent network is defined, which corresponds to the definition of the action space of deep reinforcement learning: for maintenance plan decision, the output is the probability distribution of each device performing maintenance action in each discrete time window; for power generation dispatch decision, the output is the selection probability or value assessment value of each generator set at each discrete output level.

[0081] Through the above steps, the agent network architecture in the deep reinforcement learning model is constructed. This network architecture can effectively extract the spatial structural features of the power grid topology and the temporal evolution features of the operational data, and organically integrate multi-dimensional heterogeneous state information, providing structural support for subsequent reinforcement learning training and online inference.

[0082] Steps 2-3: Randomly select historical experience data to train and optimize the deep reinforcement learning model; The training process includes: adopting a priority experience replay strategy, calculating the sampling priority of historical experience samples based on the time difference error, and extracting samples according to the priority ratio; dividing the experience replay pool into a normal operation sub-pool, a fault warning sub-pool, and an emergency sub-pool for partition management; using a deep Q-network algorithm to explore the action space in the early stage of training, and switching to a proximal policy optimization algorithm for policy optimization when the number of training steps reaches a preset proportion of the total number of steps; and using a soft update strategy to periodically update the target network parameters, and dynamically adjusting the target network update frequency according to the fluctuation amplitude of the training loss function to avoid value function oscillation problems during training. Step 3: Input the real-time multidimensional state feature vector generated in Step 1 into the deep reinforcement learning model trained in Step 2. The agent network outputs a comprehensive decision vector containing the probability distribution of maintenance actions and the selection of generation dispatch levels. For maintenance actions in the comprehensive decision vector, the power system simulation model is called to perform N-1 safety verification. The maintenance actions that pass the verification are transformed into structured maintenance work orders, which include safety measures, process flow, required tools and equipment, and personnel qualification requirements. For dispatch actions in the comprehensive decision vector, automatic generation control commands and automatic voltage control commands are generated and sent to the generation and energy storage control systems through a communication protocol conforming to IEC 60870-5-104 or IEC 61850 standards.

[0083] Step 4: Collect actual power grid operation data after the instructions in Step S3 are issued and executed. Calculate indicator values ​​based on a preset multi-dimensional evaluation indicator system that includes technical, economic, and safety indicators, and use a time-series analysis algorithm to predict future trends in the indicators. Using an online comparative testing framework, the actual operating flow is proportionally divided into the current decision scheme and the comparison decision scheme. The cumulative reward value and constraint violation count of the two schemes under the same boundary conditions are compared in real time, and an evaluation report is generated. A tiered feedback operation is performed based on the quantitative results of the evaluation report: when the deviation of short-term indicators exceeds a preset threshold, the mixed weights of the reward function are recalculated, and the weight coefficients of each optimization objective are adjusted; when the model performance shows continuous degradation, the agent network is fine-tuned online using recent actual operating data, with a maximum update step size constraint; when the deterioration of key safety indicators exceeds a preset tolerance threshold or the cumulative reward value is lower than a preset percentage of the historical best value for multiple consecutive preset periods, the offline retraining process is automatically initiated, and the new model replaces the online model after gray-scale release verification.

[0084] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0085] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A collaborative optimization method for maintenance planning and power generation dispatch based on deep reinforcement learning, characterized in that, Includes the following steps: Step 1: Collect multi-source heterogeneous data from the power grid, perform standardization processing and feature extraction on the multi-source heterogeneous data, and construct a multi-dimensional state feature vector; Step 2: Construct a deep reinforcement learning model; The collaborative optimization problem of maintenance planning and power generation scheduling is constructed as a Markov decision process, defining a state space, action space, and reward function. The state space is composed of the multi-dimensional state feature vector generated in step 1. An intelligent agent is constructed, which includes a CNN module for extracting grid topology spatial features, an LSTM module for extracting temporal features, and a fusion layer for fusing spatial features, weighted temporal features, equipment health state vectors, and maintenance resource state vectors to output a comprehensive decision vector. The intelligent agent is trained to obtain a trained deep reinforcement learning model. Step 3: Input the multi-dimensional state feature vector generated in real time into the trained deep reinforcement learning model and output a comprehensive decision vector; generate a maintenance work order based on the maintenance actions in the comprehensive decision vector, generate a power generation dispatch instruction based on the dispatch actions in the comprehensive decision vector, and issue the maintenance work order and the power generation dispatch instruction for execution. Step 4: Collect actual power grid operation data after the command is issued and executed, calculate the evaluation results based on the preset evaluation index system, and adjust the parameters of the deep reinforcement learning model according to the evaluation results to drive the continuous optimization of the model.

2. The method for collaborative optimization of maintenance planning and power generation dispatch based on deep reinforcement learning according to claim 1, characterized in that, Step 1 specifically includes: Step 1-1: Extract structured data from the power management system, including equipment ledgers, maintenance work orders, and defect records, using ETL tools, and configure an incremental synchronization mechanism; receive real-time grid topology changes, node voltages, line power flow, and power generation output measurement data from the energy management system via message queues; deploy monitoring equipment at renewable energy power plants to collect real-time power and meteorological data for wind and solar power, with remote plants using LoRaWAN backhaul and supporting breakpoint resume. Load and power generation data are collected through smart meters and power transmitters; refined weather forecasts are obtained by connecting to meteorological department APIs; smart sensors are deployed on key equipment in substations and aggregated to data acquisition terminals via industrial Ethernet; distributed fiber optic sensors are deployed on transmission lines to monitor icing and galloping conditions. Steps 1-2: Use outlier detection algorithms to clean the collected data, remove noisy and abnormal fluctuation data, and transform it into data with unified standards; Steps 1-3: Extract key features from maintenance data and scheduling data, perform dimensionality reduction optimization on the key features, and construct the multidimensional state feature vector that is finally input into the deep reinforcement learning model.

3. The method for collaborative optimization of maintenance planning and power generation scheduling based on deep reinforcement learning according to claim 2, characterized in that, The dimensionality reduction optimization method is as follows: Principal component analysis is used to reduce the dimensionality of data containing multiple dimensions, retaining several principal components with a cumulative contribution rate of 95%, and using the principal component scores as new features to input into the deep reinforcement model in subsequent steps; then, mutual information feature selection is used to calculate the mutual information value between each original feature and the two target variables of equipment failure and overload limit, and features with mutual information greater than 0.1 are selected to further reduce dimensionality.

4. The method for collaborative optimization of maintenance planning and power generation dispatch based on deep reinforcement learning according to claim 1, characterized in that, Step 2 specifically includes: Step 2-1: Construct the collaborative optimization problem of maintenance plan and power generation dispatch as a Markov decision process, and define the agent and environment model; Step 2-2: Build the network architecture of the intelligent agent; specifically including: Step 2-2-1: Construct a CNN module to receive adjacency matrix data representing the power grid topology, and extract local spatial connectivity features in the adjacency matrix through multi-layer two-dimensional convolution operations, and output spatial feature vectors; Step 2-2-2: Construct an LSTM module to receive time series data sliced ​​at preset time intervals, extract time-series dependency features from the time series data through a cyclic connection structure, and output an initial time-series feature vector; Step 2-2-3: Introduce a temporal attention layer to weight temporal features; Step 2-2-4: Construct a fusion layer for feature stitching and output; Step 2-2-5: Connect the modules that have been constructed above according to the data flow and encapsulate them into a unified intelligent agent; Steps 2-3: Use an experience replay mechanism to store historical experience data generated by the agent's interaction with the environment, and randomly select data for training; combine the near-end policy optimization algorithm and the deep Q-network algorithm to train the model, and continuously adjust the network parameters so that the agent learns the optimal decision-making strategy; at the same time, update the target network regularly.

5. The method for collaborative optimization of maintenance planning and power generation scheduling based on deep reinforcement learning according to claim 4, characterized in that, The defined environment model includes refined state space construction, action space discretization and constraint handling, and dynamic adjustment of multi-objective weights of the reward function. The refined state space construction includes four dimensions: power grid operation status, equipment health status, maintenance resource status, and new energy output prediction. Action space discretization and constraint handling include maintenance plan adjustment and power generation dispatch scheme optimization; The dynamic adjustment of multi-objective weights in the reward function includes: adopting a hybrid weight calculation method based on entropy weighting and analytic hierarchy process, dynamically adjusting the weights of each objective according to the power grid operation scenario, and setting a penalty mechanism including equipment failure penalty, repeated power outage penalty, and resource over-allocation penalty.

6. The method for collaborative optimization of maintenance planning and power generation scheduling based on deep reinforcement learning according to claim 4, characterized in that, The LSTM module is implemented using a Long Short-Term Memory (LSTM) network. The LSTM network has a two-layer structure. The first layer outputs the hidden state sequence of the complete time step, and the second layer outputs only the hidden state of the last time step, or outputs the hidden state sequence of all time steps for subsequent attention layers to process. The hidden state dimension of each LSTM layer is set according to the input data scale and computing resources.

7. The method for collaborative optimization of maintenance planning and power generation scheduling based on deep reinforcement learning according to claim 4, characterized in that, In step 2-2-3, a temporal attention layer is connected after the output of the temporal feature extraction sub-network. The temporal attention layer is used to dynamically calculate the importance weight of the hidden state at each time point based on the time interval between each historical time point and the current decision time point in the time series data, and to perform a weighted summation of the hidden states at each time point based on the weights to generate a weighted temporal feature vector.

8. The method for collaborative optimization of maintenance planning and power generation scheduling based on deep reinforcement learning according to claim 4, characterized in that, In step 2-2-4, the fusion layer is connected to the output of the CNN module, the output of the LSTM module, and the provider of the auxiliary state vector, respectively. Before the device health state vector is input into the fusion layer for splicing, a self-attention mechanism is applied to the device health state vector.

9. A collaborative optimization system for maintenance planning and power generation dispatching based on deep reinforcement learning, characterized in that, include: The data fusion and preprocessing module is used to collect multi-source heterogeneous data from all aspects of the power grid, perform standardization processing on it, and construct feature data. A deep reinforcement learning model is used to collaboratively optimize maintenance plans and power generation dispatch schemes under multi-dimensional constraints of grid operation, equipment health, and new energy forecasting. The system parameter adjustment module is used to receive feedback instructions generated by the effect evaluation and feedback application module, and dynamically adjust the reward function weight coefficients, training hyperparameters or model instances of the deep reinforcement learning module according to the feedback instructions, so as to drive the continuous evolution of the strategy of the deep reinforcement learning module. The effect evaluation and feedback application module is used to collect the actual operation data of the power grid after the decision support module issues instructions, and to quantitatively evaluate the decision effect based on a preset multi-dimensional evaluation index system, and generate feedback instructions based on the evaluation results. The decision support module is connected to the output of the deep reinforcement learning model. It is used to convert the optimization scheme output by the deep reinforcement learning model into executable maintenance work orders and power generation dispatch instructions, and to provide a visual interactive decision-making environment.

10. The maintenance planning and power generation dispatch collaborative optimization system based on deep reinforcement learning according to claim 9, characterized in that, The deep reinforcement learning module contains an intelligent agent. The network architecture of the intelligent agent consists of three sub-modules. CNN module: Used to receive adjacency matrix data representing the topology of the power grid, extract local spatial connectivity features in the adjacency matrix through convolution operations, and output spatial feature vector; LSTM module: used to receive time series data sliced ​​at preset time intervals, extract time-dependent features from the time series data through a recurrent neural network structure, and output a time-series feature vector; Fusion layer: Connected to the output of the CNN module and the output of the LSTM module respectively, it is used to concatenate and fuse the spatial feature vector, the temporal feature vector and the auxiliary state vector representing the health status and maintenance resource status of the equipment, and output a decision vector containing maintenance actions and scheduling actions after being processed by a fully connected layer.

11. The maintenance planning and power generation dispatch collaborative optimization system based on deep reinforcement learning according to claim 10, characterized in that, The output of the LSTM module is also connected to a time attention layer, which is used to dynamically allocate the weights of the features at each time point in the time series based on the time interval between each time point in the time series and the current decision time point.

12. The maintenance planning and power generation dispatch collaborative optimization system based on deep reinforcement learning according to claim 10, characterized in that, Before the device health state vector is input into the LSTM module for concatenation, a self-attention mechanism is applied to the device health state vector; the self-attention mechanism performs weighted aggregation processing on the device health state vector by calculating the correlation weight between each device health state vector.

13. The maintenance planning and power generation dispatch collaborative optimization system based on deep reinforcement learning according to claim 9, characterized in that, The system parameter adjustment module is divided into three functional sub-units: The reward function weight dynamic adjustment subunit is used to adjust the weight coefficients of each optimization objective of the reward function in the deep reinforcement learning module; The online hyperparameter fine-tuning subunit is used to adjust the training-related hyperparameters of the agent network in the deep reinforcement learning module, as well as some weight parameters of the fusion layer. The Model Retraining Triggering and Scheduling Subunit is used to adjust the overall model instance of the deep reinforcement learning module.

14. The maintenance planning and power generation dispatch collaborative optimization system based on deep reinforcement learning according to claim 9, characterized in that, The effect evaluation and feedback application module includes: an online evaluation and comparison unit, which uses an online comparison testing framework to proportionally divide the actual operating traffic into the current decision scheme and the comparison decision scheme, compares the cumulative reward value and constraint violation number of the two schemes under the same boundary conditions in real time, and generates an evaluation report including a risk heatmap and a scheme economic comparison table; a closed-loop feedback unit, which pushes the quantitative indicators in the evaluation report to the system parameter adjustment module to trigger the recalculation of reward function weights or hyperparameter adjustment, and pushes the high-quality decision sample data generated during the evaluation process to the data management module of the deep reinforcement learning module as supplementary samples for online fine-tuning; and an automatic retraining trigger unit, which automatically starts the offline retraining process of the deep reinforcement learning module when the average cumulative reward within a continuous preset period is lower than a preset percentage threshold of the historical best value, or when the deterioration of key safety indicators exceeds a preset tolerance threshold. The new model replaces the online model after gray-scale release verification.

15. The maintenance planning and power generation dispatch collaborative optimization system based on deep reinforcement learning according to claim 9, characterized in that, The decision support module includes: a maintenance plan collaborative optimization unit, used to call the maintenance actions output by the deep reinforcement learning module, perform N-1 safety checks in conjunction with the power system simulation model, and convert the checked maintenance actions into structured maintenance work orders, which include safety measures, process flow, required tools and equipment, and personnel qualification requirements; a generation dispatch collaborative optimization unit, used to update the grid operation status according to a preset cycle, call the dispatch actions output by the deep reinforcement learning module, generate automatic generation control commands and automatic voltage control commands, and send the commands to the generation-side and energy storage-side control systems through a communication protocol conforming to IEC60870-5-104 or IEC 61850 standards; and a visualization interactive decision unit, used to construct a virtual grid visualization interface based on three-dimensional digital twin technology, and overlay and display equipment health status layers, power flow distribution layers, and maintenance plan layers on the interface, while supporting dispatchers to adjust decision preference parameters through human-computer interaction and triggering the deep reinforcement learning module to recalculate the optimal solution in real time.