A power distribution network planning and operation method combining digital twinning with reinforcement learning

Abstract

The application provides a power distribution network planning and operation method combining digital twinning and reinforcement learning, first acquires power distribution network structure, technical configuration and load state data through digital twinning; second extracts typical load mode by using a clustering method, and constructs a system state matrix; then extracts high-dimensional features by using a convolutional neural network, and maps to a planning and operation strategy space; then iteratively trains a reinforcement learning model by using a fitting Q iteration algorithm, and optimizes decision cost; finally outputs an optimal network reconstruction and technical investment scheme adapting to system dynamic changes. The application can effectively reduce investment cost, simultaneously reduce optimization calculation time consumption, and has excellent scalability and engineering practicability.

Description

Technical Field

[0001] This invention relates to the field of power system technology, specifically to a distribution network planning and operation method that combines digital twins with reinforcement learning. Background Technology

[0002] With the development of economy and technology, electricity has become an indispensable secondary energy source in people's production and life, bringing endless convenience. With the diversification of power generation, load and storage equipment in the power grid and their interaction, the uncertainty and randomness of distribution network operation are increasing, and the operating scenarios are becoming increasingly complex.

[0003] The core flaw of traditional benchmark optimization and single-stage optimization methods is their lack of dynamic adaptability. While benchmark optimization can yield a theoretically optimal solution, it fails to consider the dynamic changes in distribution network load patterns and network structure. Single-stage optimization solves each stage independently, ignoring the interrelationships between decisions across multiple stages, making it difficult to adapt to system state fluctuations after a high proportion of distributed generation is integrated. Furthermore, the computational complexity of these methods increases exponentially with the scale of the power grid, resulting in low solution efficiency and poor real-time performance, failing to meet the immediate decision-making needs of distribution network operation. Moreover, they heavily rely on accurate mathematical modeling and complete system parameters, lacking robustness to uncertainties such as renewable energy output and load demand. When actual operating conditions deviate from modeling assumptions, the optimization effect significantly decreases. Additionally, the tendency to treat some indicators as constraints makes it difficult to achieve a globally optimal balance across multiple objectives.

[0004] The independent and multi-technology integrated application of digital twins has significant shortcomings in terms of functionality and data. Existing technologies mostly focus on single functional scenarios such as prediction, protection, and design, lacking integrated application across the entire process of distribution network planning and operation, and failing to support collaborative decision-making between planning investment and real-time operation. This approach is highly dependent on the topology, measurement, and operational data of the physical power grid. However, the monitoring coverage of a large number of traditional assets in low-voltage distribution networks is low, and missing or poor-quality data directly reduces the mapping accuracy between the digital twin and the physical system, affecting the reliability of decision-making. At the same time, the technical standards for digital twins are not unified across different scenarios, resulting in poor cross-system interoperability and compatibility. Furthermore, industrial-grade products are mostly customized, requiring a large amount of equipment and professional technical teams for construction and maintenance, leading to high implementation costs and weak portability.

[0005] The main problems with traditional reinforcement learning's single and improved application methods are insufficient feature fusion and scenario adaptation capabilities. Most methods focus only on single optimization objectives such as network reconstruction and equipment investment, failing to achieve multi-dimensional feature collaborative modeling of network structure, load patterns, and technology combinations, and thus cannot fully reflect the complex operating state of the distribution network. Some technical branches have high computational complexity and long training time, and model training is highly dependent on the quantity and quality of historical data, resulting in limited generalization ability when facing new scenarios such as new load access. Some methods also fail to consider multi-stage cumulative costs, only optimizing single-stage objectives, and lacking long-term foresight in planning decisions, making it difficult to cope with the long-term dynamic transition of the distribution network. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention provides a distribution network planning and operation method that combines digital twins with reinforcement learning. This method adapts to dynamic changes, achieves multi-objective global optimization, improves feature extraction and decision-making performance, and enhances generalization capabilities.

[0007] The technical solution provided by this invention: a distribution network planning and operation method combining digital twins and reinforcement learning, comprising the following steps:

[0008] (1) Collect data on the distribution network structure, technical configuration, and load status through digital twins;

[0009] (2) Typical load patterns are extracted using clustering methods to construct the system state matrix;

[0010] (3) Use convolutional neural networks to extract high-dimensional features and map them to the planning and operation strategy space;

[0011] (4) Optimize decision-making costs by iteratively training the reinforcement learning model using the Q-iteration algorithm;

[0012] (5) Output the optimal network reconstruction and technology investment scheme to adapt to the dynamic changes of the system.

[0013] Furthermore, step (1) includes:

[0014] (1-1) Operational Data Acquisition

[0015] Digital twins, as virtual copies of physical power distribution networks, collect real-time operational status data of the physical power distribution network through sensors, smart meters, and related terminal equipment deployed on the network. They also integrate historical operational data with information related to future energy scenarios to ensure that the data covers both static topology and dynamic operational characteristics.

[0016] (1-2) Network structure data acquisition and representation

[0017] Network structure data focuses on the topological connections and electrical parameters of the distribution network. The data collected includes the connection status of buses and feeders, the total length of feeders, and is standardized and represented through matrix modeling and electrical parameter derivation.

[0018] (1-3) Data Acquisition and Characterization of Technical Configuration

[0019] Technical configuration data targets key equipment in the distribution network used to improve operational stability. The data collected includes the installation locations and quantities of static var compensators, capacitor banks, and energy storage systems. Standardized representation is achieved through matrix modeling to confirm the technical installation status matrix. Its dimensions are ,in For distribution network busbar collection, A collection of technical equipment, matrix elements A non-negative integer, representing the busbar. The technical equipment connected above The total number directly reflects the technical deployment status of the distribution network under a certain stage k;

[0020] (1-4) Load status data acquisition and characterization

[0021] Load status data focuses on the electricity consumption characteristics of heterogeneous users. The data collected includes real-time power consumption and user-side distributed energy operation data. Standardized representation is achieved through data normalization and matrix modeling.

[0022] Furthermore, steps (1-2) include:

[0023] Connection State Matrix Construction: Defining the Network Configuration State Matrix Its dimensions are ,in For distribution network busbar collection, For the feeder set, matrix elements For binary variables, satisfying:

[0024] ,

[0025] This matrix intuitively reflects a certain stage. The topology connection status of the sub-distribution network;

[0026] Electrical parameter derivation: Based on the assumption of uniform feeder length distribution, the resistance and reactance between adjacent buses are calculated to support subsequent power loss calculations.

[0027] Feeder branch length calculation:

[0028] ,

[0029] In the formula, From the feeder head to the busbar branch length, For feeder Total length, For feeder The set of connected busbars;

[0030] Calculation of resistance between adjacent busbars:

[0031] ,

[0032] In the formula, busbar With busbar Branch resistance between, The resistance per unit length of the feeder.

[0033] Calculation of reactance between adjacent busbars:

[0034] ,

[0035] In the formula, busbar With busbar Branch reactance between, This is the reactance per unit length of the feeder.

[0036] Furthermore, steps (1-4) collect real-time electricity metering data from heterogeneous users, normalize it to per-unit active power values, and eliminate the impact of load amplitude differences among different users on pattern recognition; simultaneously, they collect the output power of rooftop photovoltaic panels. Electric vehicle charging power Power consumption of air source heat pump Distributed energy data, etc.

[0037] Furthermore, step (2) includes:

[0038] (2-1) Extraction of typical load patterns using CPM clustering method

[0039] (2-2) Integration of future energy scenarios and net load calculation

[0040] Distributed energy sources such as rooftop solar panels, electric vehicles, and air-source heat pumps, along with flexible loads, are incorporated into typical load patterns. Net load demand is calculated to reflect the actual system load status, as shown in the following formula:

[0041] ,

[0042] In the formula, Let be the net load requirement for the y-th cluster at time step t. For photovoltaic output power, The charging power for electric vehicles, For the power consumption of the heat pump, , , It is a binary variable.

[0043] For temperature-controlled loads such as heat pumps, the building's thermal inertia is modeled using a 1R1C thermal model, and the equipment is automatically controlled to start and stop to maintain a comfortable indoor temperature, ensuring the accuracy of the net load calculation without changing the core clustering characteristics of typical load patterns.

[0044] (2-3) Construction of system load state matrix

[0045] Based on typical load patterns and net load calculation results, a standardized state matrix is ​​constructed. To achieve a structured representation of the load state:

[0046] Matrix definition: A matrix has dimensions of 1. ,in For distribution network busbar collection, This is a typical load cluster set.

[0047] Element meaning: Matrix element For binary variables, satisfying:

[0048] ,

[0049] Each busbar corresponds to only one typical cluster, ensuring the uniqueness and consistency of the matrix, intuitively reflecting the load characteristic distribution of each node in the distribution network, and laying the foundation for subsequent convolutional neural network extraction of spatial correlation features.

[0050] Furthermore, step (2-1) includes:

[0051] (2-1-1) Data Collection and Normalization

[0052] Real-time electricity metering data of heterogeneous users is collected from the digital twin system and converted into per-unit active power values ​​to eliminate the interference of load amplitude differences between different users on pattern recognition and ensure that clustering focuses on the shape characteristics of load curves.

[0053] (2-1-2) Probability density function fitting

[0054] The nonparametric kernel density estimation method is used to fit the power consumption probability density function at each time step based on historical load samples. It can accurately reflect the load uncertainty distribution without the need for preset distribution parameters. The formula is as follows:

[0055] ,

[0056] In the formula, Let be the fitted probability density function. Let t be the load uncertainty variable at time step t. For the h-th historical sample of the load variable, For historical sample sets, For bandwidth smoothing parameters, The kernel function is used to achieve PDF fitting by superimposing the kernel functions of all historical samples.

[0057] (2-1-3) Initial cluster generation

[0058] The cumulative distribution function (CDF) is derived based on the fitted power consumption probability density function. Initial clusters are generated using Latin hypercube sampling, and the initial clusters are calculated using the inverse CDF function, as shown in the following formula:

[0059] ,

[0060] In the formula, For time step Next The load values ​​of each cluster, The cumulative distribution function is... Let be a random variable that follows a uniform distribution. For the cluster set, the probability of each initial cluster occurring is 1. ;

[0061] (2-1-4) Cluster Merging Optimization

[0062] Euclidean distance is used to measure the distance between any two clusters. and The similarity is used to eliminate low-probability clusters and merge high-similarity clusters to screen typical load patterns. The core formula is as follows:

[0063] Similarity calculation:

[0064] ,

[0065] In the formula, The distance is the Euclidean distance function; the smaller the distance, the higher the cluster similarity.

[0066] Minimum similarity filtering:

[0067] ,

[0068] In the formula, For clustering The minimum Euclidean distance to all other clusters.

[0069] Low-value cluster identification and merging:

[0070] ,

[0071] Identify low-probability, highly similar clusters The probability of occurrence is summed up and added to the most similar cluster. Post-removal Iterate this process until a predetermined number of typical clusters are obtained;

[0072] (2-1-5) Determination of typical load patterns

[0073] The final clusters are the typical load patterns, whose curve characteristics reflect the electricity consumption patterns of different user groups and include information on load time-series changes and probability distribution.

[0074] Furthermore, step (3) includes:

[0075] (3-1) Determine the CNN input set and data preprocessing

[0076] The constructed system state matrix and the planned candidate action matrix are fused into a high-dimensional tensor, which serves as the input dataset for the convolutional neural network (CNN). At the same time, the input data is standardized to eliminate differences in dimensions and magnitudes, ensuring the effectiveness of feature extraction.

[0077] Input set composition: The input is the distribution network configuration state matrix. Technical installation status matrix Load mode state matrix And the fusion tensor of the candidate action matrix for planning and operation, the tensor dimension matches the scale of the distribution network bus node and the number of equipment / clusters, and fully represents the high-dimensional correlation information between system state and candidate action;

[0078] Data preprocessing: Z-score normalization is used to standardize the input tensor, eliminating the difference in the units of data of different dimensions, avoiding the dominance of features with large amplitudes in the extraction results, and improving the CNN's ability to learn key features;

[0079] (3-2) High-dimensional feature extraction of convolutional layers + pooling layers

[0080] We design a dual-layer convolutional layer-pooling layer cascaded structure. A sliding filter captures the spatial correlation features of the high-dimensional input, and pooling layers achieve feature dimensionality reduction, preserving key features while reducing computational complexity. This includes:

[0081] (3-2-1) Convolutional Layer Feature Extraction: Multi-size filters are used to perform convolution operations on the input tensor. Local spatial features are mined through linear transformations and nonlinear activations to generate multi-channel feature maps. The formula is as follows:

[0082] ,

[0083] In the formula, The feature map generated by the convolution operation. Input tensors to the CNN, Here is the filter weight matrix. This is a two-dimensional convolution operation. For bias terms, The ReLU activation function is used to introduce non-linear features and improve the network's ability to fit complex features; the convolutional layers use the same padding strategy to ensure that the size of the feature map after convolution matches the input and avoids the loss of edge features;

[0084] (3-2-2) Pooling layer feature dimensionality reduction: Each convolutional layer is followed by a pooling layer. Downsampling is used to compress the dimension of the feature map, remove redundant features and retain the key information of local features.

[0085] (3-2-3) Multi-filter feature fusion: By setting multiple sets of filters with different weights, multi-channel feature maps are generated, and then all channel feature maps are stacked into a global feature map to fully characterize the global spatial correlation features of the distribution network's "state-action".

[0086] (3-3) Flattening of global features and nonlinear transformation of fully connected layers

[0087] The stacked global feature map is flattened to transform the two-dimensional spatial feature map into a one-dimensional feature vector, eliminating the influence of spatial dimension on subsequent mapping; then, the one-dimensional feature vector is subjected to continuous nonlinear transformation through multiple fully connected layers.

[0088] (3-4) Mapping from high-dimensional feature space to planning and operation strategy space

[0089] The globally fused features output by the fully connected layer are mapped to an action value function through the output layer, achieving a precise quantitative mapping from the high-dimensional feature space to the planning and execution strategy space. The core formula is:

[0090] ,

[0091] In the formula, The optimal action value function predicted by CNN. This is the parameterized regression function for a CNN, whose parameters are composed of filter weights, fully connected layer weights, and biases. The system state matrix, To plan and run the candidate action matrix.

[0092] Furthermore, step (4) includes:

[0093] (4-1) Constructing a Markov decision process for distribution network planning and operation

[0094] The planning and operation decision-making of the entire life cycle of the distribution network is abstracted into the MDP model, which clarifies the four core elements of state, action, state transition and cost function, and establishes a theoretical framework for FQI iterative training.

[0095] (4-2) Generate FQI batch training tuples

[0096] The training data is preprocessed based on the MDP model to generate batch data of state-action-next state-cost quadruples, providing data support for the initial training and iterative updates of FQI. The specific process is as follows:

[0097] (4-2-1) Based on the initial state of the distribution network , With full-stage load status As input, randomly generate planned operation actions that satisfy power grid operation constraints. ;

[0098] (4-2-2) Calculate the state of the next stage based on the state transition function. ;

[0099] (4-2-3) Solve the multi-objective optimization problem of the distribution network to obtain the immediate decision cost. Finally, batch training tuples are generated. The dataset is divided into training and testing sets according to a set ratio to provide data for the initial training of the CNN's Q function.

[0100] (4-3) Iterative training process of FQI based on Bellman optimal equation

[0101] FQI uses the Bellman optimal equation as its core update criterion and combines it with CNN to achieve iterative prediction and parameter optimization of the Q function. Through multiple batches of iterations, the Q function converges to its optimal value, ultimately resulting in a trained reinforcement learning model.

[0102] (4-4) Quantitative optimization of multi-objective decision-making costs

[0103] The four objectives of distribution network planning and operation—investment cost, power loss, load reduction, and renewable energy curtailment—are integrated into a normalized comprehensive decision cost function, which serves as the cost basis and optimization objective for FQI iterative training, achieving global collaborative optimization of multiple objectives.

[0104] The comprehensive decision cost function is obtained by minimizing and maximizing the four objectives to eliminate dimensional differences, and then weighting them equally to obtain the comprehensive cost, which serves as the immediate cost of the MDP. The formula is:

[0105] ,

[0106] In the formula, For the target weight, For investment costs, For power loss, For renewable energy curtailment, To reduce load, various indicators Its theoretical minimum / maximum value;

[0107] (4-5) Optimization strategies for model training

[0108] The Adam optimizer is used to update the CNN parameters through gradient descent, and the learning rate is dynamically adjusted according to the accuracy of the test set through the learning rate scheduler. L2 regularization and Dropout mechanism are introduced to suppress model overfitting and ensure that the model still has stable Q function prediction and cost optimization capabilities in unseen distribution network conditions.

[0109] Furthermore, step (4-3) includes:

[0110] (4-3-1) Confirming the Bellman optimal equation: The optimal Q-function satisfies a recursive relationship. The optimal Q-value of the current state is the expected sum of the immediate cost and the minimum optimal Q-value of the next state. This is the core basis for iterative updates, and the formula is:

[0111] ,

[0112] In the formula, Find the minimum optimal Q value for all possible actions in the next state;

[0113] (4-3-2) Initial Training: Without a pre-trained model, assume the initial optimal Q value is 0, i.e. Using batch training tuples For input, immediate cost For output, train the CNN to obtain the initial Q-function prediction model. ;

[0114] (4-3-3) Batch Iterative Update: The preprocessed batch data is used as the iterative unit to complete the Q-function update and model retraining batch by batch.

[0115] Based on the CNN model trained in the previous round Enter the next state And iterate through all possible actions. The formula for predicting and searching for the minimum optimal Q value of the next state is:

[0116] ,

[0117] Calculate the optimal Q-value for the current state based on the Bellman optimality equation. Complete the recursive update of the Q value.

[0118] In the current state Given the input, the updated optimal Q-value For the output, the CNN is retrained to obtain a new Q-function prediction model. ;

[0119] (4-3-4) Convergence Determination: Repeat the batch iteration process described above. When the Q-value prediction error of the CNN model in two adjacent training rounds on the test set is less than a preset threshold, or when the number of iterations reaches a preset value, the model is determined to have converged, and the final reinforcement learning model is obtained. .

[0120] Furthermore, step (5) relies on the training of a convergent convolutional neural network and the fitting of a Q-iterative reinforcement learning model, combined with the real-time state perception and virtual simulation capabilities of the digital twin, to solve for the optimal action combination from the planned operation action space, and after verification by physical constraints, it is transformed into an engineering solution. At the same time, a dynamic update mechanism is established to adapt to system changes and form a data closed loop.

[0121] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0122] 1. Adapt to dynamic changes and achieve global optimization for multiple objectives.

[0123] To address the issues of poor dynamic adaptability and multi-objective imbalance in traditional mathematical programming optimization, this invention models decision-making as a Markov decision process. It iteratively optimizes the cumulative cost across all stages by fitting Q, and combines a digital twin real-time perception and dynamic update mechanism to quickly adapt to changes in system state. At the same time, it integrates multiple optimization objectives to achieve global optimal balance, breaking through the limitations of traditional methods.

[0124] 2. Integrated digital decision-making lowers the barriers to implementation.

[0125] To address the issues of fragmented functions, high data dependence, and poor portability in existing digital twins, this invention deeply integrates digital twins with reinforcement learning to achieve integrated decision-making throughout the planning and operation process. It reduces data dependence through clustering algorithms, improves interoperability through standardized matrices, and reduces deployment costs by relying on convolutional neural networks to adapt to power distribution networks of different scales.

[0126] 3. Improve feature extraction and decision-making performance, and enhance generalization ability.

[0127] To address the problems of insufficient feature fusion, low efficiency, and short-sighted decision-making in traditional reinforcement learning, this invention achieves multi-dimensional feature fusion through a customized convolutional neural network, improves training efficiency and decision stability by fitting the Q-iteration algorithm, enhances the model's generalization ability by constructing a data closed loop, avoids short-sighted decision-making through multi-stage modeling, and adapts to long-term dynamic transitions. Attached Figure Description

[0128] Figure 1This is a flowchart of the method of the present invention;

[0129] Figure 2 This is a comparison chart of investment costs in embodiments of the present invention. Detailed Implementation

[0130] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0131] like Figure 1 As shown, a distribution network planning and operation method combining digital twins and reinforcement learning is presented. First, data on the distribution network structure, technical configuration, and load status are collected using digital twins. Second, a clustering method is used to extract typical load patterns and construct a system state matrix. Next, a convolutional neural network is used to extract high-dimensional features and map them to the planning and operation strategy space. Then, a reinforcement learning model is iteratively trained using a Q-iteration algorithm to optimize decision costs. Finally, the optimal network reconfiguration and technical investment scheme adapted to the dynamic changes of the system is output. This invention has been verified by IEEE 18-node and IEEE 33-node distribution network examples, effectively reducing investment costs and shortening optimization computation time, demonstrating excellent scalability and engineering practicality.

[0132] The method specifically includes the following steps:

[0133] Step 1: Collect data on the distribution network structure, technical configuration, and load status using digital twins.

[0134] The first step involves collecting data on the distribution network structure, technical configuration, and load status through digital twins. The core of this approach is to leverage the precise mapping between digital twins and the physical distribution network, integrating measurement devices such as sensors and smart meters to comprehensively collect three types of key status data and transform them into standardized matrix representations. This lays the data foundation for subsequent feature extraction and decision optimization. The specific implementation is as follows:

[0135] 1. Data collection methods

[0136] Digital twins, as virtual replicas of physical power distribution networks, collect real-time operational status data of the physical power grid through terminal devices such as sensors and smart meters deployed on the distribution network. At the same time, they integrate historical operational data with information related to future energy scenarios, ensuring that the data covers both static topology and dynamic operational characteristics. The data collection process has high timeliness and high reliability, and can correct measurement errors through multi-source data fusion.

[0137] 2. Network Structure Data Acquisition and Representation

[0138] Network structure data focuses on the topological connections and electrical parameters of the distribution network. The data collected includes basic information such as the connection status of buses and feeders, and the total length of feeders. A standardized representation is achieved through matrix modeling and electrical parameter derivation.

[0139] Connection State Matrix Construction: Defining the Network Configuration State Matrix Its dimensions are ,in For distribution network busbar collection, For the feeder set. Matrix elements. For binary variables, satisfying:

[0140]

[0141] This matrix intuitively reflects a certain stage. The topology connection status of the sub-distribution network.

[0142] Electrical parameter derivation: Based on the assumption of uniform feeder length distribution, the resistance and reactance between adjacent buses are calculated to support subsequent power loss calculations.

[0143] Feeder branch length calculation:

[0144]

[0145] In the formula, From the feeder head to the busbar branch length, For feeder Total length, For feeder The set of connected buses.

[0146] Calculation of resistance between adjacent busbars:

[0147]

[0148] In the formula, busbar With busbar Branch resistance between, The resistance per unit length of the feeder.

[0149] Calculation of reactance between adjacent busbars:

[0150]

[0151] In the formula, busbar With busbar Branch reactance between, This is the reactance per unit length of the feeder.

[0152] 3. Technical Configuration Data Acquisition and Characterization

[0153] Technical configuration data targets key equipment in the distribution network used to improve operational stability. The data collected includes the installation locations and quantities of static var compensators (SVCs), capacitor banks, and energy storage systems, and is standardized using matrix modeling.

[0154] Define the technology installation status matrix Its dimensions are ,in This refers to a collection of technical equipment (including SVCs, capacitor banks, and energy storage systems). Matrix elements. A non-negative integer, representing the busbar. The technical equipment connected above The total number directly reflects the technical deployment status of the distribution network at a certain stage k.

[0155] 4. Load Status Data Acquisition and Characterization

[0156] Load status data focuses on the electricity consumption characteristics of heterogeneous users (commercial, residential, and industrial). The data collected includes real-time power consumption and operational data of user-side distributed energy resources (photovoltaics, heat pumps, and electric vehicles). Standardized representation is achieved through data normalization and matrix modeling.

[0157] Raw data acquisition and preprocessing: Real-time electricity metering data from heterogeneous users is collected and normalized to per-unit active power values ​​to eliminate the impact of load amplitude differences among users on pattern recognition. Based on the correspondence between historical load samples and time steps, the normalized electricity metering data is matrix-organized to form a standardized load sample matrix. This standardized matrix serves as the unified input data for subsequent CPM clustering analysis. Simultaneously, the output power of rooftop photovoltaic panels is collected. Electric vehicle charging power Power consumption of air source heat pump Distributed energy data, etc.

[0158] Step 2: Extract typical load patterns and construct the system state matrix using clustering methods.

[0159] The second step uses clustering methods to extract typical load patterns and construct a system state matrix. The core of this step is to identify typical load characteristics from historical load data using the Cluster Generation-Merge (CPM) statistical method, integrate the impact of distributed energy resources in conjunction with future energy scenarios, and finally represent the load state in a standardized matrix form. This provides structured input for subsequent feature extraction and decision optimization. The specific implementation is as follows:

[0160] 1. Typical load pattern extraction (CPM clustering method)

[0161] A five-step method is used to perform cluster analysis on load data, accurately capturing the temporal patterns and peak diversity of load. The specific steps are as follows:

[0162] Step 1: Obtaining Standardized Load Sample Data

[0163] Obtain the standardized matrix from the first step, and use the per-unit active power values ​​of each historical load sample at continuous time steps as input for probability density function fitting and subsequent CPM cluster analysis.

[0164] Step 2: Probability density function fitting

[0165] A nonparametric kernel density estimation method is used to fit the power consumption probability density function (PDF) for each time step based on historical load samples. This method can accurately reflect the load uncertainty distribution without pre-setting distribution parameters. The formula is as follows:

[0166]

[0167] In the formula, Let be the fitted probability density function. Let t be the load uncertainty variable at time step t. For the h-th historical sample of the load variable, For historical sample sets, For bandwidth smoothing parameters, The kernel function is used to achieve PDF fitting by superimposing the kernel functions of all historical samples.

[0168] Step 3: Initial Cluster Generation

[0169] Based on the fitted PDF, the cumulative distribution function (CDF) is derived, and initial clusters are generated using Latin hypercube sampling. This approach both augments small sample data to avoid overfitting and covers potential variations in loading uncertainty. The initial clusters are calculated using the inverse CDF function, as shown in the following formula:

[0170]

[0171] In the formula, For time step Next The load values ​​of each cluster, The cumulative distribution function is... Let be a random variable that follows a uniform distribution. For the cluster set, the probability of each initial cluster occurring is 1. .

[0172] Step 4: Clustering and Merging Optimization

[0173] Euclidean distance is used to measure the distance between any two clusters. and The similarity is used to eliminate low-probability clusters and merge high-similarity clusters to screen typical load patterns. The core formula is as follows:

[0174] 1. Similarity calculation:

[0175]

[0176] In the formula, The distance is the Euclidean distance function; the smaller the distance, the higher the cluster similarity.

[0177] 2. Minimum similarity filtering:

[0178]

[0179] In the formula, For clustering The minimum Euclidean distance to all other clusters.

[0180] 3. Low-value cluster identification and merging:

[0181]

[0182] Identify low-probability, highly similar clusters The probability of occurrence is summed up and added to the most similar cluster. Post-removal This process is iterated until a predetermined number of typical clusters are obtained.

[0183] Step 5: Determine typical load patterns

[0184] The final clusters are the typical load patterns, whose curve characteristics reflect the electricity consumption patterns of different user groups and include information on load time-series changes and probability distribution.

[0185] 2. Future Energy Scenarios Integration and Net Load Calculation

[0186] Distributed energy sources such as rooftop solar panels, electric vehicles, and air-source heat pumps, along with flexible loads, are incorporated into typical load patterns. Net load demand is calculated to reflect the actual system load status, as shown in the following formula:

[0187]

[0188] In the formula, Let be the net load requirement for the y-th cluster at time step t. For photovoltaic output power, The charging power for electric vehicles, For the power consumption of the heat pump, , , It is a binary variable (1 indicates that the corresponding device has it installed, and 0 indicates that it has not been installed).

[0189] For temperature-controlled loads such as heat pumps, the building's thermal inertia is modeled using a 1R1C thermal model, and the start and stop of equipment are automatically controlled to maintain a comfortable indoor temperature, ensuring the accuracy of the net load calculation without changing the core clustering characteristics of typical load patterns.

[0190] 3. Construction of System Load State Matrix

[0191] Based on typical load patterns and net load calculation results, a standardized state matrix is ​​constructed. To achieve a structured representation of the load state:

[0192] Load state matrix construction: Define the load mode state matrix The matrix dimension is ,in For distribution network busbar collection, This is a typical load cluster set.

[0193] Element meaning: Matrix element For binary variables, satisfying:

[0194]

[0195] Each busbar corresponds to only one typical cluster, ensuring the uniqueness and consistency of the matrix, intuitively reflecting the load characteristic distribution of each node in the distribution network, and laying the foundation for subsequent convolutional neural network extraction of spatial correlation features.

[0196] Step 3: Extract high-dimensional features using a convolutional neural network and map them to the planning and execution strategy space.

[0197] The third step utilizes convolutional neural networks to extract high-dimensional features and map them to the planning and operation strategy space. The core of this step is to construct a customized convolutional neural network (CNN) architecture adapted to the high-dimensional matrix data of the distribution network. Through hierarchical feature extraction using convolution and pooling, spatial correlation feature mining and dimensionality reduction of high-dimensional state and action data are achieved. Then, a fully connected layer nonlinearly maps the extracted global features into an action value function (Q-function), realizing a precise mapping from the high-dimensional feature space of the distribution network to the planning and operation strategy space. This provides a quantitative basis for action value in subsequent reinforcement learning model training. The specific implementation is as follows:

[0198] 1. Determine the CNN input set and data preprocessing.

[0199] The system state matrix constructed previously and the candidate action matrix for planning are fused into a high-dimensional tensor, which serves as the input dataset for the CNN. Simultaneously, the input data is standardized to eliminate differences in units and magnitudes, ensuring the effectiveness of feature extraction.

[0200] Input set composition: The input is the distribution network configuration state matrix. Technical installation status matrix Load mode state matrix And the planning and execution of candidate action matrices (network reconstruction action matrix) Technology Investment Action Matrix The fusion tensor of the distribution network bus node is matched with the number of equipment / clusters, and the tensor dimension is matched with the size of the distribution network bus node and the number of equipment / clusters, which fully represents the high-dimensional correlation information of system state-candidate action;

[0201] Data preprocessing: Z-score normalization is used to standardize the input tensor, eliminating the difference in the dimensions of data of different dimensions, avoiding the dominance of features with large amplitudes in the extraction results, and improving the CNN's ability to learn key features.

[0202] 2. High-dimensional feature extraction using convolutional layers and pooling layers

[0203] We design a dual-convolutional-pooling layer cascaded structure. A sliding filter captures the spatial correlation features of the high-dimensional input, and the pooling layer achieves feature dimensionality reduction, preserving key features while reducing computational complexity. The core implementation is as follows:

[0204] (1) Convolutional layer feature extraction: Multi-size filters are used to perform convolution operations on the input tensor. Local spatial features are mined through linear transformation and nonlinear activation to generate multi-channel feature maps. The formula is as follows:

[0205]

[0206] In the formula, The feature map generated by the convolution operation. Input tensors to the CNN, Here is the filter weight matrix. This is a two-dimensional convolution operation. For bias terms, The ReLU activation function is used to introduce non-linear features and improve the network's ability to fit complex features; the convolutional layers use the same padding strategy to ensure that the size of the feature map after convolution matches the input and avoids the loss of edge features.

[0207] (2) Pooling layer feature dimensionality reduction: Each convolutional layer is followed by a pooling layer. The feature map is compressed by downsampling, redundant features are removed and key information of local features is retained, which reduces the computational complexity of the subsequent network and improves the translation invariance of features, ensuring the stability of feature extraction results when the topological position of the distribution network node changes slightly.

[0208] (3) Multi-filter feature fusion: By setting multiple sets of filters with different weights, multi-channel feature maps are generated, and then all channel feature maps are stacked into a global feature map to fully characterize the global spatial correlation features of the distribution network's "state-action".

[0209] 3. Flattening of global features and nonlinear transformation of fully connected layers

[0210] The stacked global feature map is flattened to transform the two-dimensional spatial feature map into a one-dimensional feature vector, eliminating the influence of spatial dimension on subsequent mapping. Then, the one-dimensional feature vector is continuously transformed nonlinearly through multiple fully connected layers, gradually compressing the feature dimension while strengthening the deep correlation between features, realizing the transformation from local features to global fused features, and laying a nonlinear feature foundation for subsequent policy space mapping.

[0211] 4. Mapping from high-dimensional feature space to planning and operational strategy space

[0212] The globally fused features output by the fully connected layer are mapped to an action value function (Q-function) through the output layer, achieving a precise quantitative mapping from the high-dimensional feature space to the planning and execution strategy space. The core formula is as follows:

[0213]

[0214] In the formula, The optimal action value function (Q function) for CNN prediction. This is the parameterized regression function for a CNN, whose parameters are composed of filter weights, fully connected layer weights, and biases. The system state matrix, To plan and run the candidate action matrix.

[0215] The core logic of this mapping is as follows: the Q-function value is the expected cumulative decision cost of taking a certain planned action under a certain system state. The planned operation strategy space consists of all feasible network reconstruction actions and technology investment actions. Different action combinations correspond to different decision points in the strategy space. The CNN completes the quantitative evaluation of each decision point by outputting the Q-function value, realizing a one-to-one mapping from the high-dimensional feature space to the planned operation strategy space. The magnitude of the Q-function value directly quantifies the merits of the corresponding planned operation action, providing the core basis for judging the value of the action for subsequent training of the reinforcement learning model using the Q-iteration algorithm.

[0216] Step 4: Iteratively train the reinforcement learning model using the Q-iteration algorithm to optimize decision cost.

[0217] The fourth step involves iteratively training the reinforcement learning model and optimizing the decision cost using the fitted Q-iteration algorithm. The core of this step is to model the distribution network planning and operation decision as a Markov Decision Process (MDP), using the fitted Q-iteration (FQI) algorithm as the core batch reinforcement learning algorithm. This combines the high-dimensional features extracted from previous iterations with the Q-function prediction capability of CNNs, and updates the Q-function parameters through multiple iterations using the Bellman optimal equation. Simultaneously, it achieves quantitative optimization of the decision cost based on a normalized multi-objective cost function, ultimately resulting in a converged reinforcement learning model that provides a quantitative basis for optimal planning and operation decisions. The specific implementation is as follows:

[0218] 1. Construct a Markov Decision Process (MDP) for distribution network planning and operation.

[0219] The planning and operation decision-making of the entire life cycle of the distribution network is abstracted into an MDP model, which clarifies the four core elements: state, action, state transition, and cost function. This establishes a theoretical framework for FQI iterative training. The core formulas and definitions are as follows:

[0220] 1. State transition: The distribution network transitions from a phase... To the stage The state change is determined by the current state and the action taken, as shown in the formula:

[0221]

[0222] In the formula, The overall state of the distribution network (including network configuration) at stage k. Technical installation Load mode ), For the stage Planned operational actions taken (including network reconfiguration) Technology investment ), This is the state transition function. This is a complete set of stages.

[0223] 2. Accumulated Costs: The accumulated future costs after taking a series of actions are the sum of discounted immediate costs at each stage, as shown in the formula:

[0224]

[0225] In the formula, For the stage The initial accumulation of future costs, As a discount factor, Let $k$ be the immediate decision cost at stage $k$.

[0226] 3. Definition of Q-function: The action value function (Q-function) quantifies the expected cumulative cost of taking an action in a given state. It is the core objective of FQI training, and its formula is:

[0227]

[0228] In the formula, For strategy The Q function below, For mathematical expectation, strategy Implement the mapping from state to action. ); Optimal Q function , which is the minimum Q-function under all strategies, is the ultimate goal of iterative training.

[0229] 2. Generate FQI batch training tuples (data preprocessing)

[0230] The training data is preprocessed based on the MDP model to generate batch data of state-action-next state-cost quadruples, providing data support for the initial training and iterative updates of FQI. The specific process is as follows:

[0231] (1) Based on the initial state of the distribution network , With full-stage load status As input, randomly generate planned operation actions that satisfy power grid operation constraints. ;

[0232] (2) Calculate the state of the next stage based on the state transition function. ;

[0233] (3) Solve the multi-objective optimization problem of the distribution network to obtain the real-time decision cost. Finally, batch training tuples are generated. The dataset is divided into training and testing sets according to a set ratio, providing data for the initial training of the CNN's Q function.

[0234] 3. Iterative Training Process of FQI Based on Bellman Optimal Equation

[0235] FQI uses the Bellman optimal equation as its core update criterion and combines it with CNN to achieve iterative prediction and parameter optimization of the Q function. Through multiple batches of iterations, the Q function converges to its optimal value, and finally, a trained reinforcement learning model is obtained. The steps are as follows:

[0236] (1) Bellman optimality equation: The optimal Q-function satisfies a recursive relationship. The optimal Q-value of the current state is the expected sum of the immediate cost and the minimum optimal Q-value of the next state. This is the core basis for iterative updates. The formula is:

[0237]

[0238] In the formula, Find the minimum optimal Q value for all possible actions in the next state.

[0239] (2) Initial Training: Without a pre-trained model, assume the initial optimal Q value is 0, i.e. Using batch training tuples For input, immediate cost For output, train the CNN to obtain the initial Q-function prediction model. .

[0240] (3) Batch Iterative Update: The preprocessed batch data is used as the iterative unit to complete the Q-function update and model retraining batch by batch.

[0241] Step 1: Based on the CNN model trained in the previous round Enter the next state And iterate through all possible actions. The formula for predicting and searching for the minimum optimal Q value of the next state is:

[0242]

[0243] In the formula, The value of the Q-function predicted by the CNN.

[0244] Step 2: Calculate the optimal Q-value for the current state based on the Bellman optimality equation. This completes the recursive update of the Q value.

[0245] Step 3: In the current state Given the input, the updated optimal Q-value For the output, the CNN is retrained to obtain a new Q-function prediction model. .

[0246] (4) Convergence determination: Repeat the batch iteration process described above. When the Q-value prediction error of the CNN model in two adjacent training rounds on the test set is less than a preset threshold, or the number of iterations reaches a preset value, the model is determined to have converged, and the final reinforcement learning model is obtained. (B represents the total number of batches).

[0247] 4. Quantitative optimization of multi-objective decision-making costs

[0248] The four objectives of distribution network planning and operation—investment cost, power loss, load reduction, and renewable energy curtailment—are integrated into a normalized comprehensive decision cost function, which serves as the cost basis and optimization objective for FQI iterative training. This achieves global collaborative optimization across multiple objectives. The core formula and implementation are as follows:

[0249] (1) Comprehensive decision cost function: Minimum-maximum normalization is applied to the four objectives to eliminate dimensional differences. The comprehensive cost is obtained after equal weighting and is used as the immediate cost of the MDP. The formula is:

[0250]

[0251] In the formula, For the target weight, For investment costs, For power loss, For renewable energy curtailment, To reduce load, various indicators It is its theoretical minimum / maximum value; this function guarantees the fairness and synergy of the four objectives, and avoids a single objective dominating the optimization result.

[0252] (2) Cost optimization logic: The core objective of FQI iterative training is to minimize the optimal Q function. The optimal Q function is the cumulative expectation of the comprehensive decision-making cost. Therefore, the iterative process of the model is essentially a quantitative optimization of the comprehensive decision-making cost of the entire distribution network. During the training process, the parameters of the CNN are adjusted so that the predicted value of the Q function gradually approaches the true optimal cost, and finally the global optimal cost of planning and operation decision-making is achieved.

[0253] 5. Optimization strategies for model training

[0254] To improve model training efficiency and generalization ability, a series of optimization strategies are introduced during the FQI iterative training process: the Adam optimizer is used to update the gradient descent of CNN parameters, and the learning rate is dynamically adjusted according to the accuracy of the test set through the learning rate scheduler; L2 regularization and Dropout mechanism are introduced to suppress model overfitting and ensure that the model still has stable Q function prediction and cost optimization capabilities under unseen distribution network conditions.

[0255] Step 5: Output the optimal network reconstruction and technology investment plan to adapt to the dynamic changes of the system.

[0256] The core of the fifth step is to rely on a convergent convolutional neural network and a fitted Q-iteration (CNN+FQI) reinforcement learning model, combined with the real-time state awareness and virtual simulation capabilities of digital twins, to solve for the optimal action combination from the planned action space. After verification by physical constraints, it is transformed into an engineering solution. At the same time, a dynamic update mechanism is established to adapt to system changes and form a data closed loop. The specific implementation is as follows:

[0257] 1. Real-time sensing and standardized representation of the dynamic state of the distribution network

[0258] Digital twins continuously collect real-time data on distribution network topology, technical equipment operation, load patterns, and distributed energy output / operation to determine the current state of the system and transform this data into a standardized network configuration state matrix according to predetermined rules. Technical installation status matrix Load mode state matrix This ensures that the state representation format is completely consistent with the training input of the reinforcement learning model, providing a unified input basis for model invocation.

[0259] 2. Global prediction of the Q-function of the convergent model

[0260] The standardized real-time system state matrix is ​​imported into a CNN model that has been trained and converged using FQI. For all combinations of actions that satisfy physical constraints in the planning and operation strategy space, the optimal action value function is predicted globally. The core formula is:

[0261]

[0262] In the formula, The optimal Q-function value predicted by the model represents a network reconstruction action taken in the current state. Technology investment activities The expected cumulative decision cost of the combination; after traversing all possible action combinations, a set of global Q function values ​​is formed.

[0263] 3. Solving for the optimal planning and execution actions

[0264] With the core objective of minimizing cumulative decision cost, and based on the Bellman optimality criterion, the action combination corresponding to the minimum Q-function value is selected from the global Q-function value set as the optimal action for the current system state. The core solution formula is:

[0265]

[0266] In the formula, The optimal network reconstruction action matrix has binary elements that explicitly define the connection adjustment strategies for buses and feeders, satisfying bus topology constraints, and the network configuration state transitions follow... ; The optimal technology investment action matrix has binary elements that clearly define the installation strategies for new SVCs, capacitor banks, and energy storage systems on each bus, and the updates to the technology installation status follow an accumulation rule. For scenarios involving drastic load mode switching during the transition from summer to winter, this solution logic can quickly adapt to system state changes caused by the doubling of load peaks due to thermal electrification, and output the optimal decision that balances short-term operational constraints and long-term investment costs.

[0267] 4. Verification of physical constraints and cost of optimal action

[0268] The optimal action combination obtained from the solution is substituted into the digital twin distribution network physical simulation model for comprehensive verification and validation. On the one hand, it verifies whether the scheme meets core physical constraints such as power flow, bus voltage, line current, and energy storage charging and discharging rate to ensure the feasibility of the project. On the other hand, it verifies the investment cost, power loss, and other indicators after the actual implementation of the scheme to verify the consistency between the actual comprehensive cost and the model's predicted Q value. If the optimal action combination does not meet the constraints or the cost deviation exceeds the standard, the action combinations corresponding to the second smallest Q function value are sequentially screened and re-verified until the optimal feasible action combination that meets all physical constraints is determined.

[0269] Verification has shown that the optimal solution can reduce investment costs by 50% compared to single-stage optimization methods in scenarios where the distribution network system transitions from summer to winter and the peak load doubles due to thermal electrification. At the same time, it can stably control the deviation between the full-cycle power loss and load shedding and the benchmark optimization method within 5%, achieving significant optimization of investment costs while ensuring power supply reliability and system operating efficiency.

[0270] 5. Build a dynamic adaptive update mechanism for the solution.

[0271] To address the dynamic changes in the distribution network status, a threshold-triggered dynamic update mechanism is established to solve the decision lag problem: the digital twin monitors the change amplitude of the status matrix in real time. When changes such as load mode switching and topology adjustment exceed the preset threshold, the scheme re-solution process is triggered. Based on the new status data, the CNN model is finely tuned online, and the optimal solution under the new status can be quickly solved without full retraining. If there is a phased state transition such as seasonal load switching, the FQI small batch iteration is triggered to update the model's Q function parameters, ensuring the long-term optimality of the model's decision.

[0272] 6. Standardized solution output and end-to-end data closed loop

[0273] The validated optimal combination of feasible actions is transformed into a standardized engineering solution that conforms to the field operation specifications of the distribution network. The network reconfiguration solution specifies the topology adjustment targets, timing, and execution equipment, while the technology investment solution specifies the equipment installation locations, quantities, and switching priorities. The digital twin distributes the standardized solution to the field control terminal, enabling automated execution of network reconfiguration actions and precise deployment of technology investment equipment. Simultaneously, actual operational and cost data during solution execution are collected and added to the reinforcement learning model's training dataset, forming a closed-loop data process encompassing data acquisition, model training, solution output, field execution, and data feedback. This continuously improves the model's decision-making accuracy and the solution's adaptability to dynamic system changes.

[0274] This invention deeply integrates digital twins with reinforcement learning to achieve standardized collection and representation of multi-dimensional data on distribution network structure, technical configuration, and load status. At the same time, relying on the real-time status perception and virtual simulation capabilities of digital twins, it completes the physical constraint verification and dynamic adaptation of the optimal solution.

[0275] This invention employs a customized convolutional neural network to extract spatial correlation features from high-dimensional state data of power distribution networks and accurately maps them to the planning and operation strategy space, solving the problem of inefficient feature extraction of high-dimensional data and providing a quantitative basis for action value for reinforcement learning model training.

[0276] This invention constructs a reinforcement learning model based on the fitting Q-iteration algorithm combined with Markov decision process, performs multi-round iterative training and multi-objective cost optimization for distribution network planning and operation decisions, and establishes a threshold-triggered dynamic update mechanism to achieve real-time adaptation of optimal network reconfiguration and technical investment schemes to the dynamic state of the distribution network.

[0277] Table 1 shows the computation time of the method of this invention and the benchmark optimization under different power grid sizes.

[0278] Table 1 Computation time of the proposed model and benchmark optimization under different power grid sizes.

[0279] Calculation time (s) The model proposed in this invention Benchmark optimization IEEE 18-node distribution network 12218 38402 IEEE 33-node distribution network 12246 71578

[0280] like Figure 2 The figure shows a comparison of the investment costs of the method of this invention with other methods. Verified through distribution network examples, this method reduces the optimization computation time by 68.18% in an IEEE 18-node distribution network and by 82.89% in an IEEE 33-node distribution network compared to traditional benchmark optimization methods. Furthermore, the model of this invention exhibits superior performance compared to single-stage optimization, Monte Carlo tree search, and deep deterministic policy gradient: its investment cost is reduced by approximately 50% compared to single-stage optimization when the system transitions to high winter loads; due to consideration of multi-stage cumulative costs and adaptation to dynamic system state transitions, cost changes are stable without abrupt changes, and the deviation between the total investment cost and the theoretical optimal value of benchmark optimization is minimal. Compared to other reinforcement learning methods, this invention adapts to the discrete action characteristics of distribution networks, resulting in superior accuracy, stability, and optimization performance in cost control. It possesses scalability and engineering practicality, and can be adapted to the planning, operation, and optimization needs of low-voltage distribution networks.

[0281] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A power distribution network planning and operation method combining digital twinning with reinforcement learning, characterized in that, The method comprises the following steps: (1) Collecting power distribution network structure, technical configuration and load state data through digital twinning; (2) Extracting typical load patterns by clustering method and constructing system state matrix; (3) Extracting high-dimensional features by using convolutional neural network and mapping to planning operation strategy space; (4) Iterative training of reinforcement learning model by fitting Q iterative algorithm to optimize decision cost; (5) Outputting optimal network reconstruction and technical investment scheme adapting to system dynamic changes.

2. The power distribution network planning and operation method of claim 1, wherein The step (1) comprises: (1-1) Operation data collection Digital twinning, as a virtual copy of the physical power distribution network, collects real-time operation state data of the physical power grid through sensors, smart meters and related terminal devices deployed in the power distribution network, integrates historical operation data and future energy scenario correlation information, and ensures data coverage of static topology and dynamic operation characteristics; (1-2) Network structure data collection and representation Network structure data focuses on the topological connection relationship and electrical parameters of the power distribution network, and the collection content includes the connection state of bus and feeder, the total length of feeder, and the standardized representation is completed through matrix modeling and electrical parameter derivation; (1-3) Technical configuration data collection and representation Technical configuration data targets key equipment in the distribution network used to improve operational stability. The data collected includes the installation locations and quantities of static var compensators, capacitor banks, and energy storage systems. Standardized representation is achieved through matrix modeling to confirm the technical installation status matrix. Its dimensions are ,in For distribution network busbar collection, A collection of technical equipment, matrix elements A non-negative integer, representing the busbar. The technical equipment connected above The total number directly reflects the technical deployment status of the distribution network under a certain stage k; (1-4) Load state data collection and representation Load state data focuses on the power consumption characteristics of heterogeneous users, and the collection content includes real-time power consumption and user-side distributed energy operation data, and the standardized representation is completed through data normalization and matrix modeling.

3. The power distribution network planning and operation method of claim 2, wherein, The step (1-2) comprises: Connection status matrix construction: define network configuration status matrix with dimension where is the set of buses of the distribution network, is the set of feeders, the matrix element is a binary variable satisfying: ， The matrix intuitively reflects the state of a certain stage the topology connection state of the lower distribution network; Electrical parameter derivation: based on the assumption that the feeder length is uniformly distributed, the resistance and reactance between adjacent buses are calculated to support subsequent power loss calculation: Feeder branch length calculation: ， In the formula, From the feeder head to the busbar branch length, For feeder Total length, For feeder The set of connected busbars; Adjacent bus resistance calculation: ， wherein is the busbar is the branch resistance between the busbar is the branch resistance between the busbar is the feeder unit length resistance, Adjacent bus reactance calculation: ， wherein is the bus and the branch reactance between the bus is the feeder per unit length reactance.

4. The power distribution network planning and operation method of claim 2, wherein, The step (1-4) collects real-time power consumption metering data of heterogeneous users, normalizes the active power standard value, eliminates the influence of different user load amplitude differences on pattern recognition; At the same time, the distributed energy data is collected, including the output power of the roof photovoltaic panel , electric vehicle charging power , air source heat pump power consumption Distributed energy data.

5. The power distribution network planning and operation method of claim 1, wherein, The step (2) comprises: (2-1) Extracting typical load patterns by CPM clustering method (2-2) Future energy scenario integration and net load calculation Integrate rooftop photovoltaic panels, electric vehicles, air source heat pumps and other distributed energy and flexible loads into typical load patterns, calculate net load demand to reflect the actual system load state, and the formula is as follows: ， wherein is the net load demand of the yth cluster at time step t, is the photovoltaic output power, is the electric vehicle charging power, is the heat pump electricity consumption, , , is a binary variable, For temperature-controlled loads such as heat pumps, model the building thermal inertia through a 1R1C thermal model, automatically control the start and stop of devices to maintain indoor comfort temperature, and ensure the authenticity of net load calculation, but do not change the core clustering characteristics of typical load patterns; (2-3) System load state matrix construction Based on the typical load pattern and the net load calculation results, a standardized state matrix is constructed , realizing the structural characterization of load state: Matrix definition: matrix dimension is wherein is a set of distribution network buses, is a set of typical load clusters; Element Meaning: Matrix element Is a binary variable satisfying: ， Each bus corresponds to only one typical cluster, ensuring the uniqueness and consistency of the matrix, and directly reflecting the load characteristic distribution of each node of the power distribution network, laying a foundation for subsequent convolutional neural network to extract spatial correlation features.

6. The power distribution network planning and operation method of claim 5, wherein, The step (2-1) comprises: (2-1-1) Data collection and normalization Collect real-time power consumption data of heterogeneous users from the digital twinning system, convert it into active power per unit, eliminate the interference of different user load amplitude differences on pattern recognition, and ensure that clustering focuses on load curve shape characteristics; (2-1-2) Probability density function fitting Using non-parametric kernel density estimation method, fitting the power consumption probability density function of each time step based on historical load samples, without presetting distribution parameters, the load uncertainty distribution can be accurately reflected, and the formula is as follows: ， wherein, is the fitted probability density function, is the load uncertainty variable at time step t, is the hth historical sample of the load variable, is the set of historical samples, is the bandwidth smoothing parameter, is the kernel function, the PDF fitting is achieved by kernel function superposition over all historical samples, (2-1-3) Initial cluster generation Cumulative distribution function (CDF) is derived based on the fitted power consumption probability density function, and initial clusters are generated by Latin hypercube sampling, which are calculated by the inverse function of CDF as follows: ， wherein is the time step The next cluster's load value, is the cumulative distribution function, is a random variable that follows a uniform distribution, is the set of clusters, all initial clusters have the same probability of occurring; (2-1-4) Cluster merging optimization The Euclidean distance is used to measure the similarity of any two clusters With The low probability clusters are removed and the high similarity clusters are merged, and the typical load mode is screened. The core formula is as follows: Similarity calculation: ， In the formula, is the Euclidean distance function, the smaller the distance, the higher the similarity of the cluster. Minimum similarity screening: ， wherein is the cluster minimum euclidean distance to all other clusters, Low-value cluster identification and merging: ， Identifying low probability and high similarity clusters Stacking their occurrence probability to the most similar cluster Post-pruning Iterating this process until a preset number of typical clusters is obtained; (2-1-5) Determination of typical load mode The final retained cluster is the typical load mode, and its curve characteristics reflect the power consumption rules of different user groups and contain load time sequence changes and probability distribution information.

7. The power distribution network planning and operation method of claim 1, wherein, The step (3) comprises: (3-1) Determining the CNN input set and data preprocessing Fuse the constructed system state matrix and the planning operation candidate action matrix into a high-dimensional tensor as the input data set of the convolutional neural network (CNN), and standardize the input data to eliminate the dimension and amplitude differences and ensure the effectiveness of feature extraction; Input set composition: input is the power distribution network network configuration state matrix , the technology installation state matrix , the load mode state matrix , and the fusion tensor of the planning operation candidate action matrix, the tensor dimension matches the bus node scale and the number of equipment / clusters of the power distribution network, and the high-dimensional correlation information of the system state-candidate action is completely characterized. Data preprocessing: Z-score normalization is used to standardize the input tensor, eliminate the dimension differences of different dimensions of data, avoid the dominance of features with large amplitudes in the extraction results, and improve the learning ability of CNN to key features; (3-2) High-dimensional feature extraction of convolutional layer + pooling layer A double-layer convolutional layer-pooling layer series structure is designed to capture the spatial correlation features of high-dimensional input through sliding filters, and to realize feature dimension reduction through the pooling layer to retain key features while reducing computational complexity, including: (3-2-1) Convolutional layer feature extraction: multiple size filters are used to perform convolution operation on the input tensor to extract local spatial features through linear transformation and nonlinear activation, and generate multi-channel feature maps, with the formula being: ， wherein, is a feature map generated by a convolution operation, is a CNN input tensor, is a filter weight matrix, is a two-dimensional convolution operation, is a bias term, is a ReLU activation function, used to introduce nonlinear features and improve the network's fitting ability for complex features; the convolution layer adopts the same padding strategy to ensure that the feature map size matches the input after convolution, avoiding the loss of edge features; (3-2-2) Pooling layer feature dimension reduction: a pooling layer is connected in series after each convolutional layer to compress the dimension of the feature map through downsampling operation, eliminate redundant features and retain the key information of local features; (3-2-3) Multi-filter feature fusion: multiple sets of filters with different weights are set to generate multi-channel feature maps, and then all channel feature maps are stacked into a global feature map to fully represent the global spatial correlation features of "state-action" of the power distribution network; (3-3) Flattening of global features and nonlinear transformation of fully connected layer The stacked global feature map is flattened to convert the two-dimensional spatial feature map into a one-dimensional feature vector to eliminate the influence of spatial dimension on subsequent mapping; then the one-dimensional feature vector is subjected to continuous nonlinear transformation through multiple fully connected layers; (3-4) Mapping from high-dimensional feature space to planning operation strategy space The global fusion features output by the fully connected layer are mapped to the action value function through the output layer to realize precise quantitative mapping from the high-dimensional feature space to the planning operation strategy space, with the core formula being: ， wherein is the optimal action value function predicted by the CNN, is a regression function parameterized by the CNN, whose parameters consist of filter weights, fully connected layer weights and biases, is the system state matrix, is the matrix of candidate actions for planning.

8. The power distribution network planning and operation method of claim 1, wherein The step (4) comprises: (4-1) Constructing Markov decision process for power distribution network planning and operation Abstract the planning and operation decisions of the whole life cycle of the power distribution network into the MDP model to clearly define the four core elements of state, action, state transition, and cost function, and establish a theoretical framework for FQI iterative training; (4-2) Generating FQI batch training tuples The pre-processing of the training data is completed based on the MDP model to generate state-action-next state-cost quadruple batch data, which provides data support for the initial training and iterative update of FQI. The specific process is as follows: (4-2-1) initial state of distribution network , and full-phase load state as input, randomly generate planning operation actions that satisfy power grid operation constraints ; (4-2-2) Calculating the next stage state according to the state transition function ; (4-2-3) Solving the multi-objective optimization problem of the power distribution network to obtain the real-time decision cost The final batch of training tuples is generated And proportionally divided into training set and test set, providing data for the initialization of CNN Q function training; (4-3) FQI iterative training process based on Bellman optimal equation FQI takes the Bellman optimal equation as the core update criterion, combines CNN to realize the iterative prediction and parameter optimization of Q function, and makes the Q function converge to the optimal value through multiple batch iterations, finally obtains the trained reinforcement learning model, (4-4) Quantitative optimization of multi-objective decision cost The investment cost, power loss, load reduction, and renewable energy curtailment of the distribution network planning and operation are integrated into a normalized comprehensive decision cost function as the cost basis and optimization target of FQI iterative training, realizing the global collaborative optimization of multi-objective, Comprehensive decision cost function: the minimum-maximum normalization eliminates the dimensional difference of the four targets, and the equal weight is weighted to obtain the comprehensive cost as the immediate cost of MDP , the formula is: ， wherein, is the target weight, is the investment cost, is the power loss, is the renewable energy curtailment, is the load reduction, the is the theoretical minimum / maximum value thereof; (4-5) Optimization strategy of model training The Adam optimizer is used to realize the gradient descent update of CNN parameters, and the learning rate scheduler is used to dynamically adjust the learning rate according to the test set accuracy; L2 regularization and Dropout mechanism are introduced to suppress model overfitting, and ensure that the model still has stable Q function prediction and cost optimization ability under unseen distribution network states.

9. The power distribution network planning and operation method of claim 8, wherein, The step (4-3) comprises: (4-3-1) Confirm the Bellman optimal equation: the optimal Q function satisfies the recursive relationship, and the optimal Q value of the current state is the sum of the immediate cost and the expected minimum optimal Q value of the next state, which is the core basis of iterative update, and the formula is: ， wherein is the minimum optimal Q-value for all actionable actions for the next state; (4-3-2) Initial training: when there is no pre-trained model, it is assumed that the initial optimal Q value is 0, that is , the input in the batch training tuple , the output is the instantaneous cost , and the initial Q function prediction model is obtained by training the CNN ; (4-3-3) Batch iterative update: the pre-processed batch data is used as the iteration unit to complete the Q function update and model retraining batch by batch: CNN model based on previous round of training , input next state and traverse all actionable , predict and search the minimum optimal Q value of next state, formula: ， According to Bellman's optimality equation, the optimal Q value of the current state is calculated , the recursive update of the Q value is completed, with the current state updated optimal Q value retraining the CNN to obtain a new Q function prediction model ; (4-3-4) Convergence determination: repeat the above batch iteration process, when the Q value prediction error of the CNN model of adjacent two rounds of training on the test set is less than the preset threshold, or the iteration number reaches the preset value, determine that the model converges, and obtain the final reinforcement learning model .

10. The power distribution network planning and operation method of claim 1, wherein, The step (5) relies on the trained convolutional neural network and the fitted Q iterative reinforcement learning model, combines the real-time state perception and virtual simulation capability of digital twinning, solves the optimal action combination from the planning and operation action space, and after physical constraint verification, it is converted into an engineering scheme, and at the same time, a dynamic update mechanism is established to adapt to system changes and form a data closed loop.

Images