Distributed photovoltaic spatio-temporal data recovery and unsupervised clustering method and system

Abstract

The invention relates to the technical field of power system automation, and discloses a distributed photovoltaic spatio-temporal data recovery and unsupervised clustering method and system.The distributed photovoltaic spatio-temporal data recovery and unsupervised clustering method comprises the steps that observation data of a distributed photovoltaic system are collected, and the observation data comprise historical distributed photovoltaic output time sequences; the collected historical distributed photovoltaic output time sequence is used as an input signal to be transmitted to each module of the GRAIL framework so as to drive a network to jointly learn time sequence dynamic characteristics and a clustering structure; wherein the GRAIL framework is a closed-loop iterative structure, and comprises a clustering sensing data generation module, a latent feature representation module and a self-adaptive clustering and refining module. The method can be widely applied to distributed photovoltaic station operation monitoring and abnormity identification scenes, and intelligent regional state recovery and mode grouping are realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of power system automation, and relates to a method and system for distributed photovoltaic space-time data recovery and unsupervised clustering. BACKGROUND

[0002] With the continuous expansion of the installed capacity of distributed photovoltaic systems in power systems, how to realize fine perception, data repair and structure identification of the operation state has become a key difficulty in distributed energy management. Especially in the widely deployed scenarios of household photovoltaic systems and industrial and commercial photovoltaic systems, due to the differences in terminal access capability, unstable communication, frequent equipment failure and other factors, photovoltaic operation data generally have problems such as missing, interference and false reporting, which brings great challenges to subsequent data-driven modeling, operation optimization and fault identification.

[0003] In actual operation, due to wide geographical distribution, multiple component types and large differences in meteorological environment, the power generation characteristics of distributed photovoltaic systems have obvious time dynamics and spatial heterogeneity, showing significant multi-scale and non-stationary characteristics. On the one hand, data missing will make the operation state judgment incomplete and not timely, affecting the regulation and control response and economic benefits of the distributed photovoltaic system; on the other hand, the data often mix the mixed distribution of different climate zones, inclination orientations and operation conditions, making it difficult for traditional clustering and modeling methods to accurately distinguish different operation modes, and weakening the effectiveness of strategy adaptation and performance evaluation.

[0004] Existing research mainly focuses on the following two aspects: one is to reconstruct missing photovoltaic data based on interpolation, matrix completion and other methods, but it often ignores the time sequence continuity and device spatial attributes, and it is difficult to ensure the physical rationality and prediction stability of the repair results; the other is to classify and manage the distributed photovoltaic system based on clustering or partitioning methods, but there are defects such as low clustering accuracy, sensitivity to data noise and the need to preset the number of clusters, making it difficult to realize high robustness and scalability of classification and identification.

[0005] Therefore, there is an urgent need for an end-to-end method framework that integrates space-time data missing recovery and clustering division. SUMMARY

[0006] The purpose of the present application is to provide a method and system for distributed photovoltaic space-time data recovery and unsupervised clustering, which can realize missing data repair and unsupervised clustering identification of distributed photovoltaic systems, and is suitable for distributed power supply scenarios with serious photovoltaic operation data anomalies and missing.

[0007] To solve the above technical problems, the present application is realized by using the following technical scheme.

[0008] In a first aspect, the present application provides a method for distributed photovoltaic space-time data recovery and unsupervised clustering, comprising:

[0009] collecting observation data of a distributed photovoltaic system, the observation data comprising historical distributed photovoltaic output time series;

[0010] inputting the collected historical distributed photovoltaic output time series as input signals into modules of a GRAIL framework to drive network joint learning of time series dynamic characteristics and clustering structures;

[0011] The GRAIL framework is a closed-loop iterative structure, comprising a clustering-aware data generation module, a latent feature representation module, and an adaptive clustering and refinement module.

[0012] In combination with the first aspect, further, the collected historical distributed photovoltaic output time series are inputted as input signals into modules of a GRAIL framework to drive network joint learning of time series dynamic characteristics and clustering structures, comprising:

[0013] preprocessing the collected observation data to obtain a plurality of observation segments and corresponding mask matrices, inputting each observation segment into a sequence generator, and outputting a reconstructed complete time series;

[0014] inputting the reconstructed complete time series into an autoencoder to extract low-dimensional latent features;

[0015] performing density clustering on the extracted low-dimensional latent features to obtain initial clustering labels, performing neighborhood reclassification on noise points in the obtained initial clustering labels to obtain final clustering labels, encoding the final clustering labels into clustering embedding vectors, and feeding the clustering embedding vectors back to the sequence generator for next round optimization until the clustering structure and the reconstruction error are both converged.

[0016] In combination with the first aspect, further, the collected observation data are preprocessed to obtain a plurality of observation segments and corresponding mask matrices, each observation segment is inputted into a sequence generator, and a reconstructed complete time series is outputted, comprising:

[0017] based on the historical distributed photovoltaic output time series, using clustering embedding vectors to reconstruct the missing part of each observation segment; wherein, represents a clustering embedding vector of a category to which the i-th sample belongs, represents the length of a time window, i.e., the number of time steps in a photovoltaic window, represents the feature dimension of each time point, represents the length of a time window, i.e., the number of time steps in a photovoltaic window, represents the feature dimension of each time point, The observation segment with missing values ​​observed in the photovoltaic window, the first The sample and the first Each photovoltaic window corresponds one-to-one;

[0018] Given a time series window and its corresponding binary mask ,in, Representing a time series window Missing; Time Series Window For the first Observation segments corresponding to each photovoltaic window;

[0019] Embedding matrix Cluster labels Mapped to a continuous clustering embedding vector Then, with each observation segment in the time step Concatenate the sequences to form an enhanced sequence for use in the input sequence generator:

[0020] ;

[0021] in, This represents the reconstruction result output by the sequence generator, i.e., the first... The complete time series of each sample is reconstructed by the sequence generator. This represents a sequence generator implemented using an LSTM network structure.

[0022] In conjunction with the first aspect, further, the step of inputting the reconstructed complete time series into an autoencoder to extract low-dimensional latent features includes:

[0023] The automatic encoder includes an encoder and a decoder;

[0024] Based on the reconstructed complete time series encoder Map it to latent coded features Then by the decoder Reconstructing the time series :

[0025] ;

[0026] in, Indicates the first Each sample is a complete time series segment reconstructed by a sequence generator; express The latent coded features output after encoder processing, i.e., low-dimensional latent features; express After being processed by the decoder, the first decoded reconstruction time series of the sample; latent encoding feature dimension.

[0027] With reference to the first aspect, further, the density clustering operation is performed on the extracted low-dimensional latent feature to obtain an initial clustering label, i.e., in the following formula (1) ; the noise points in the obtained initial clustering label are subjected to neighbor reclassification to obtain a final clustering label; the final clustering label is encoded into a clustering embedding vector, and the clustering embedding vector is fed back to the sequence generator for the next round of optimization until the clustering structure and the reconstruction error are both converged, including:

[0028] Based on the extracted latent encoding feature , an HDBSCAN algorithm is used to extract a potential group structure:

[0029] (1);

[0030] wherein, denotes a photovoltaic window assigned to a clustering label, -1 denotes that the sample is determined to be noise, denotes an HDBSCAN algorithm;

[0031] For each cluster , the stability is defined as:

[0032] ;

[0033] wherein, denotes the stability of a cluster , denotes the inverse density, denotes the density, denotes the number of samples of a cluster at a corresponding level in a hierarchical clustering structure, denotes the minimum inverse density threshold value at which the cluster appears, denotes the maximum inverse density threshold value at which the cluster disappears, and finally a clustering configuration is selected that maximizes the overall stability to obtain an initial clustering label; wherein, denotes the total number of samples;

[0034] For any sample marked as noise, i.e., the sample , the initial clustering label of the sample will be re-assigned to the majority class of its nearest non-noise neighbors to obtain a re-assigned clustering label ​i.e. the final cluster labels, i.e. the following formula (2) :

[0035] (2) ;

[0036] wherein, denotes the th sample, denotes the th neighboring sample set of the th sample, denotes the mode function, returning the cluster label with the highest frequency, denotes the photovoltaic window assigned cluster label; each re-assigned cluster label will be mapped to a continuous cluster embedding vector , using an embedding matrix

[0037] :

[0038] The cluster embedding vector will be fed back to the sequence generator.

[0039] In combination with the first aspect, further, the encoder and the decoder are jointly trained by a composite objective function:

[0040] ;

[0041] wherein, denotes the total loss function for jointly training the encoder and the decoder, denotes the encoder reconstruction error loss, being the first sub-objective, encouraging the preservation of the signal features of the time series in the encoding process; denotes the cluster compactness loss, being the second sub-objective, encouraging the clustering of samples of the same class in the latent space; denotes the cluster loss weight coefficient, controlling the balance between the two, being a hyperparameter.

[0042] The encoder reconstruction error loss uses a mean squared error loss function to achieve the reconstruction of the time series:

[0043] ;

[0044] wherein, denotes the total number of samples;

[0045] The cluster compactness loss uses a penalty on the intra-class variance in the latent space:

[0046] ;

[0047] wherein, represents the cluster label is the class of the sample set, is the latent space centroid corresponding to the cluster label is the class; represents the cluster has clusters in common.

[0048] In a second aspect, the present application provides a system for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering, comprising:

[0049] a data acquisition module configured to acquire observation data of a distributed photovoltaic system, the observation data comprising a historical distributed photovoltaic output time series;

[0050] a GRAIL framework module configured to input the acquired historical distributed photovoltaic output time series as an input signal into each module of the GRAIL framework to drive network joint learning of temporal dynamic features and clustering structure;

[0051] wherein the GRAIL framework is a closed-loop iterative structure comprising a clustering-aware data generation module, a latent feature representation module, and an adaptive clustering and refinement module.

[0052] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering described above.

[0053] In a fourth aspect, the present application provides a computer device, comprising:

[0054] a memory for storing a computer program;

[0055] a processor for executing the computer program to implement the steps of the method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering described above.

[0056] In a fourth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering described above.

[0057] The application is based on constructing a missing mask based on historical time sequence data of a photovoltaic power station, and time sequence completion of missing observation data is realized through a generation model integrated with an LSTM; the completed data is input into an automatic encoder to extract a latent feature representation and optimize reconstruction accuracy and clustering discrimination ability; a density-based unsupervised clustering algorithm is constructed to complete pattern recognition of an implied operating state, and a closed-loop optimization iteration strategy is used to enhance the generation and clustering collaboration ability. Through a feedback mechanism, the clustering label is fed back to the generation module to realize collaborative training and feature enhancement of multiple modules, thereby effectively improving the reconstruction accuracy and clustering stability in a large-scale photovoltaic data missing and heterogeneous scene.

[0058] Compared with the prior art, the application has the following beneficial effects:

[0059] (1) The application automatically mines potential operating modes based on original observation data, realizes unsupervised clustering division and interpretable state reconstruction, improves the observability and response reliability of a distributed photovoltaic system under low-quality data conditions, and serves intelligent operation and maintenance and scheduling optimization of a distributed photovoltaic station.

[0060] (2) The application proposes a unified learning framework, closely couples data generation for clustering and clustering in a latent space, realizes closed-loop optimization through an iterative feedback mechanism, promotes mutual enhancement between generation and division, and effectively alleviates the error propagation problem commonly seen in a traditional decoupling process.

[0061] (3) The application develops an LSTM generator based on a clustering condition, and combines a mask-guided loss function to explicitly emphasize the recovery ability of missing data. This design improves the accuracy of data filling and supports robust time sequence modeling in a widely sparse data scene.

[0062] (4) The application uses an HDBSCAN algorithm to detect potential clustering structures without presetting the number of clusters. In addition, the application further introduces a neighborhood voting mechanism for reassigning samples labeled as noise, thereby enhancing spatial consistency and improving robustness to outliers. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 A method flowchart for distributed photovoltaic space-time data recovery and unsupervised clustering in the embodiment 1 of the application;

[0064] Figure 2 A time sequence block residual heat map of photovoltaic output in the embodiment 2 of the application;

[0065] Figure 3 An average photovoltaic output graph divided according to clustering in the embodiment 2 of the application;

[0066] Figure 4 This is a graph showing the hourly average photovoltaic power divided by clustering in Embodiment 2 of the present invention;

[0067] Figure 5 This is a convergence curve of the clustering and reconstruction metrics during the external iteration process in Embodiment 2 of the present invention, wherein... Figure 5 'a' represents the convergence curve of the silhouette score during the external iteration process. Figure 5 b is the convergence curve of the variance ratio criterion (Calinski–Harabasz score) during the external iteration process. Figure 5 c represents the convergence curve of the Davies-Bouldin score during the external iteration process. Figure 5 d represents the convergence curve of the normalized clustering score during the external iteration process. Figure 5 e represents the convergence curve of the encoder reconstruction error index during the external iteration process. Detailed Implementation

[0068] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0069] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0070] Example 1

[0071] like Figure 1 As shown, the steps of the distributed photovoltaic spatiotemporal data recovery and unsupervised clustering method in this embodiment are as follows:

[0072] Step S1: Collect observation data from the distributed photovoltaic system to provide input for subsequent data recovery and clustering. Observation data refers to the historical distributed photovoltaic power output time series, which is sequential data that changes over time. For missing points in the above observation data, a binary mask marker is generated simultaneously to indicate whether each observation point is missing.

[0073] Historical distributed photovoltaic (PV) power output time series data are input signals to various modules of the GRAIL (Generative Reconstruction and Adaptive Identification via Latents) framework. These signals serve as crucial inputs during network training (the joint training process of all modules within the GRAIL framework), driving the network to jointly learn temporal dynamic features and clustering structures. The GRAIL framework comprises three modules: cluster-aware data generation, latent feature representation, and adaptive clustering and refinement, integrated into a closed-loop iterative structure.

[0074] Step S2: Construct Module I of the GRAIL framework, namely the cluster-aware data generation module. The collected historical distributed photovoltaic power output time series is windowed and standardized to obtain multiple observation segments and corresponding mask matrices. Each observation segment is input into the cluster-aware sequence generator (i.e., the cluster-aware data generation module), and data completion is performed by combining the current cluster embedding vector, outputting the reconstructed complete time series. The sequence generator is based on a Long Short-Term Memory (LSTM) network structure and uses a mask-aware mean squared error loss function, performing supervised learning only on data at missing locations. Details are as follows.

[0075] Based on the historical distributed photovoltaic power output time series obtained in step S1, a specific clustering embedding vector is used. For the missing part of each observation segment Refactoring is performed. Among other things... Indicates the first Clustering embedding vectors of the categories to which each sample belongs. This indicates the length of the time window, specifically the number of time steps within a photovoltaic window. The characteristic dimension representing each point in time is typically the photovoltaic power output characteristic, which is one-dimensional. express It is a dimension A real matrix, Indicates the first The observation segments with missing values ​​observed in the photovoltaic window are time series segments. The first photovoltaic window and the first Each sample is an equivalent concept; in model training (model training refers to the training process of a sequence generator to model and recover missing time series), each photovoltaic window constitutes a sample.

[0076] Given a time series window and its corresponding binary mask (in, Representing a time series window The GRAIL framework divides the historical distributed photovoltaic output time series into fixed-length windows, where each time series window is associated with an observation segment of the th photovoltaic window. Each photovoltaic window is assigned a discrete cluster label inferred by Module III, the adaptive clustering and refinement module, where -1 indicates that the sample is determined to be noise, i.e., the photovoltaic window constitutes a sample that is not assigned to any cluster, and where the number of clusters is common to all clusters.

[0077] The embedding matrix maps the cluster labels to a continuous cluster embedding vector which is then concatenated with each observation segment in the time step to form an augmented sequence for input to the sequence generator:

[0078] ;

[0079] where denotes the reconstruction result output by the sequence generator, i.e., the complete time series (as a complete synthetic segment) of the th sample generated by the sequence generator, denotes the sequence generator implemented by an LSTM network structure.

[0080] This conditional mechanism allows the LSTM network-based sequence generator to adaptively adjust its generation behavior based on the prior information at the cluster level (e.g., geographic location and weather). To avoid reconstruction bias from the observed data, a mask-aware loss function is introduced, which only supervises the missing parts:

[0081] ;

[0082] where denotes the mask-aware loss function used to train the sequence generator; denotes the observation mask of the th sample in time , dimension ; denotes the observed time series segment with missing values of the th sample in time , dimension ; denotes the observation mask of the a sequence generator outputs on time , dimension .

[0083] This design of the present application ensures that the gradient only propagates through the unobserved part, thus avoiding trivial reconstruction and facilitating semantically more meaningful missing imputation. In practical implementation, the sequence generator is implemented as a sequence model to capture local temporal dependencies, we use LSTM for implementation. The combination of clustered embedding and selective gradient flow enables the sequence generator to learn latent generative dynamics that are robust to noise and sparsity. The model (referring to the sequence generator) parameters are updated by backpropagation through the masked reconstruction loss, where denotes the entire trainable parameters of the sequence generator, and denotes the time-series dynamic feature.

[0084] Module I implements a cluster-aware, mask-conditioned sequence generator that can synthesize complete photovoltaic output time series windows from incomplete inputs, transforming partial observation inputs into semantically rich, structurally consistent reconstructed data. These generated outputs not only improve data quality, but also serve as inputs for the next stage of the encoder for joint representation learning.

[0085] Step S3: Module II of the GRAIL framework, i.e. the latent feature representation module, is constructed. The reconstructed complete time series is input into the autoencoder module (i.e. the latent feature representation module) to extract low-dimensional latent space representation; the autoencoder module includes an encoder and a decoder, the encoder is a deep neural network structure for extracting latent space features, and the decoder is used to reconstruct the original time series and shares the optimization objective with the sequence generator output. Specifically as follows.

[0086] Based on the complete time series obtained in step S2, the encoder maps it to latent encoding features , which are restored to reconstructed time series by the decoder :

[0087] ;

[0088] wherein, denotes the complete time series segment of the th sample generated by the sequence generator, denotes the latent encoding features output after the encoder processing, denotes After being processed by the decoder, the first Decoding and reconstructing the time series of each sample, Representing latent coded features Dimensions.

[0089] The encoder-decoder pair is jointly trained using a composite objective function:

[0090] ;

[0091] in, This represents the total loss function for jointly training the encoder and decoder. This represents the encoder reconstruction error loss, which is the first sub-objective, encouraging the preservation of time-series signal characteristics during the encoding process. The cluster compactness loss represents the second sub-objective, which encourages the clustering of samples of the same class in the latent space. This represents the clustering loss weight coefficient, which controls the balance between the two and is a hyperparameter.

[0092] To ensure sufficient information is retained in the latent representation, the first sub-target encoder reconstruction error loss... Using the mean squared error (MSE) loss function, accurate reconstruction of time series can be achieved:

[0093] ;

[0094] in, This represents the total number of samples.

[0095] To improve semantic consistency and structural separability, the second sub-objective clustering compactness loss is used. Penalize the intra-class variance in the latent space:

[0096] ;

[0097] in, Indicates that the cluster label is the first A sample set of classes The cluster label is the first The latent space centroid corresponding to the class; Indicates clustering common Clusters.

[0098] Composite objective function The guided latent space possesses both good reconstruction and clustering discriminative capabilities; the trade-off between its two sub-objectives is determined by hyperparameters. Control. The latent feature representation module is trained in an incremental manner in external iterations, rather than being reset at each round. The cumulative optimization mechanism enables the latent space to evolve in coordination with the sequence generator and the adaptive clustering and refinement module, stabilizing the learning of latent representations, avoiding collapse, and gradually improving semantic granularity. Unlike general dimensionality reduction techniques, the latent feature representation module explicitly supports both reconstruction fidelity and clustering alignment, thereby bridging the gap between low-level temporal dynamics and high-level structural semantics.

[0099] Step S4: Constructing Module III of the GRAIL framework, namely the adaptive clustering and refinement module. First, perform a density clustering operation on the latent space representation, using the HDBSCAN algorithm to generate initial clustering labels without pre-setting the number of clusters; then perform nearest neighbor reclassification on the noise points in the clustering results to obtain the final refined clustering labels; finally, encode the final clustering labels into clustering embedding vectors and feed them back to the sequence generator for the next round of optimization, realizing closed-loop joint training of clustering-reconstruction-representation.

[0100] The HDBSCAN algorithm used in the density clustering operation on the latent space representation supports irregular cluster structures and can automatically identify the number of clusters; in the closed-loop joint training process, multiple modules (referring to the three modules of the GRAIL framework) are updated alternately in an iterative manner, and the updated results of the clustering labels are used as control variables for the input of the sequence generator in the next round, until both the clustering structure and the reconstruction error converge. Among them, clustering structure convergence refers to the clustering results of the previous and next two iterations being basically unchanged, and reconstruction error convergence refers to the value of the first round minus the value of the first round is less than a threshold value , which meets the convergence condition. Specifically as follows.

[0101] Based on the latent encoding features obtained in step S3 , the GRAIL framework uses a density-based spatial clustering algorithm (HDBSCAN algorithm) to extract the latent group structure:

[0102] ;

[0103] wherein, represents the clustering label to which the photovoltaic window is assigned, -1 indicates that the sample is determined to be noise, represents the HDBSCAN algorithm.

[0104] In order to automatically select the optimal clustering scale, the HDBSCAN algorithm performs a grid search on a set of candidate minimum clustering scale values , wherein, denotes the number of candidate minimum cluster size values. For each setting (i.e. each candidate minimum cluster size value ), the HDBSCAN algorithm constructs a mutual reachable graph and computes the distance as follows:

[0105] ;

[0106] where, denotes the mutual reachable distance between samples and measuring the strength of dense connection between two samples, denotes the distance of sample to its th nearest neighbor. This adaptive metric can handle density-heterogeneous datasets.

[0107] The resulting hierarchical clustering structure (which is built internally by HDBSCAN) is evaluated by a stability selection criterion. For each cluster its stability is defined as:

[0108] ;

[0109] where denotes the stability of cluster , denotes the inverse density, denotes the density, denotes the number of samples of cluster at the corresponding level in the hierarchical clustering structure, denotes the minimum inverse density threshold at which cluster appears, denotes the maximum inverse density threshold at which cluster disappears. The final selected clustering configuration .

[0110] To enhance the completeness of the clustering, the GRAIL framework introduces a pseudo-label refinement method based on nearest-neighbor reclassification. For any sample labeled as noise (i.e. ), its cluster label will be reassigned as the majority class of its nearest non-noisy neighbors:

[0111] ;

[0112] where, denotes the set of nearest neighbors of the th sample, denotes the mode function, returning the label with the most occurrences, this modification helps to enhance the spatial continuity and completeness of cluster assignments, denotes the photovoltaic window assigned cluster labels. Each re- assigned cluster label will be mapped to a continuous cluster embedding vector , using the embedding matrix Implementation:

[0113] ;

[0114] cluster embedding vector will be fed back to module I to guide the subsequent data generation. Different from traditional clustering methods, module III not only generates pseudo labels (i.e. cluster labels ) for downstream tasks, but also realizes iterative closed-loop optimization by feeding back the cluster labels to the sequence generator.

[0115] Embodiment 2

[0116] The data set used in this embodiment is composed of multi-site spatio-temporal records collected from horizontal photovoltaic components deployed in 12 different locations in the northern hemisphere, covering a 14-month observation period, namely: location 1, location 2, location 3, …, location 12. Each observation data contains 17 original features, covering spatial coordinates, time indicators and meteorological variables. To verify the effectiveness of the spatio-temporal data recovery and unsupervised clustering identification method proposed in this invention, experiments are conducted with different degrees of missing ratio (20%, 40%, 60%) for simulation, and performance evaluation is conducted in terms of recovery accuracy and clustering recognition ability. To balance the time resolution and computational efficiency, the original photovoltaic data is sampled at 15 minutes, which is aggregated into hourly time windows through mean pooling, finally forming 18,312 hourly samples. The marginal distribution of the resampled data on all numerical predictors is highly consistent with the original photovoltaic data. This consistency shows that the aggregation process performs well in preserving the time and meteorological pattern characteristics, and is suitable for subsequent hourly modeling tasks.

[0117] In addition, there is significant heterogeneity in time coverage and data completeness across different sites. To mitigate this time misalignment issue and guarantee consistent time coverage, we select a uniform time sub-window: [2017-12-01 00:00:00 to 2018-06-01 15:00:00], resulting in a total of 7192 hourly observations. Nonetheless, the processed data still suffers from high sparsity. Compared to the theoretically expected 13,176 hourly records (within the selected time sub-window), there are only 7192 valid observations, with a missing rate of 45.4%. This high proportion of missingness indicates that this dataset constructs a highly challenging and representative experimental scenario.

[0118] To systematically evaluate the performance of clustering algorithms on distributed PV spatio-temporal data, this embodiment constructs a comprehensive benchmark that covers eight representative methods. These methods include: K-means, DBSCAN, HDBSCAN, Deep Embedding K-means (DE-KMeans), Deep Embedding DBSCAN (DE-DBSCAN), Deep Embedding HDBSCAN (DE-HDBSCAN), Deep Embedding Clustering using HDBSCAN Head (DEC-HDBSCAN), and the proposed GRAIL framework.

[0119] Unlike traditional approaches that separate representation learning from clustering, the GRAIL framework of the present invention performs a unified end-to-end optimization procedure. Each experiment consists of 20 outer iterations, each iteration including three consecutive stages. In the first stage, an LSTM model (i.e., a sequence generator, a type of recurrent neural network) is used to reconstruct the time series with missing values. Each time series is composed of a fixed-length input window of 32 time steps. Each time series is further augmented to a trainable embedding of 8 dimensions, encoding the spatial identity of the PV site. By applying a binary mask to distinguish unobserved locations, the LSTM model (sequence generator) is optimized using a mean squared error objective only for the masked entries. The LSTM model (sequence generator) is trained for 10 epochs (an epoch refers to the process of the LSTM model completely traversing the training dataset) in each iteration, using the Adam optimizer with a learning rate of 10 -3 and a weight decay of 10 -5, batch size of 16. In the second stage, the padded time series input a self-encoder consisting of four fully connected layers with ReLU activation function, and the encoder projects it into a 10-dimensional latent space representation. The decoder mirrors the structure to reconstruct the original features. The training is performed for 10 epochs per round using a combined loss function that incorporates the reconstruction error and a cluster consistency regularizer to encourage samples belonging to the same cluster to occupy contiguous regions in the latent space. Importantly, the network parameters of the LSTM model (sequence generator) remain unchanged across iterations, which helps refine the latent structure. In the third stage, a density-based spatial clustering algorithm HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) is performed on the generated embedding representation. A grid search is performed on candidate values of the minimum cluster size, which are distributed in a logarithmic interval from 0.1% to 50% of the data proportion to accommodate clusters of different densities. To address the presence of noise labeled points and improve the allocation coverage of samples, a 5-neighbor majority voting mechanism is used to reassign samples that have not been clustered previously.

[0120] To systematically evaluate the performance of the clustering algorithm under different configurations, the embodiment uses a multi-index evaluation scheme that combines four widely used and complementary clustering evaluation indicators: Silhouette Score (S), Calinski-Harabasz Score (C), Davies-Bouldin Score (D), and Normalized Cluster Score (H). To unify these indicators into a single clustering ranking standard, a comprehensive score is obtained by min-max normalization and weighted fusion, as shown in Table 1.

[0121] Table 1

[0122]

[0123] To further study the impact of the generator on the enhancement of clustering performance, the embodiment conducts experiments without the generator. The GRAIL framework of the present invention achieves a 0.06 improvement in the Silhouette coefficient, a 15-fold improvement in the Calinski-Harabasz index (up to 3.877 × 10 6 ), a 0.09 improvement in compactness (68% improvement), and a 0.041 improvement in normalized entropy, indicating that it not only performs well in geometric clustering but also achieves more fairness in sample distribution. These findings highlight the advantages of the GRAIL framework of the present invention in achieving feedback-coupled representation learning and clustering under the guidance of the generator.

[0124] To evaluate the generation accuracy of GRAIL in reconstructing incomplete photovoltaic sequences, Figure 2 A sequence plot residual heat map is shown, where each row represents a photovoltaic site and each column represents an hourly time step. The heat map encodes the absolute difference between the observed and reconstructed values, thereby visually quantifying the reconstruction accuracy in both spatial and temporal dimensions. The residuals are mainly concentrated in the dark blue areas, indicating that the generation module of GRAIL can stably and accurately capture the temporal dynamics of photovoltaic output. Specifically, sites such as Site 1, Site 2, and Site 12 have consistently low reconstruction errors at all time steps, highlighting the ability of the GRAIL to consistently capture the photovoltaic power dynamics of specific sites. Although light blue and red areas are observed at Site 4 and Site 5 locations, these deviations are temporally local and spatially limited, indicating that these errors are mainly caused by meteorological disturbances rather than systematic biases in the model itself. In addition, GRAIL maintains temporal stability over multiple hours of prediction period, with no significant drift or cumulative performance degradation observed, emphasizing the robustness of the model in maintaining integrity over long-term sequences. More importantly, the lack of significant residual concentration areas for specific sites further verifies the generalization ability of GRAIL in heterogeneous geographical locations.

[0125] At the same time, the clustering results of GRAIL are further verified by the photovoltaic time series power curves. Figure 3 Hourly average photovoltaic power output from 10:00 to 15:00 for each cluster is shown. The clustering results clearly divide into two groups, corresponding to low and high power generation scenarios, respectively. Cluster0 (Cluster 0) exhibits a suppressed power curve, with output consistently below 4.5kW. Specifically, its average output starts at 4.1kW at 10:00 and further decreases to 3.3kW at 15:00. This flat trajectory indicates the presence of persistent atmospheric attenuation, such as overcast days, heavy cloud cover, and snow accumulation. In contrast, Cluster 1 (Cluster 1) exhibits a typical sunny clock curve, with output starting at 9.5kW at 10:00, reaching a peak of 13.5kW at 13:00, and then slowly decreasing to 11.5kW at 15:00. Its fluctuation amplitude is about three times that of Cluster0, indicating that it is in a clear sky state with optimal light-to-electricity conversion efficiency. Hourly curves within each cluster are almost parallel, reflecting low intra-class differences and high consistency.

[0126] In addition, Figure 4The average environmental characteristics associated with each cluster are shown, with six important meteorological indicators compared using radar plots, including relative humidity, ambient temperature, wind speed, visibility, surface pressure, and cloud top height, and normalized to the [0, 1] interval for comparison. These two clusters exhibit significantly different environmental characteristics. Cluster 0 (red line) is dominated by humidity, representing overcast or saturated conditions, often involving thick cloud cover, reduced visibility, and low-pressure atmospheric boundaries, which collectively impair incident solar irradiance and suppress photovoltaic performance. In contrast, Cluster 1 (blue line) exhibits higher normalized scores in ambient temperature, wind speed, visibility, pressure, and cloud top height, corresponding to sunny, dry, high-pressure, highly visible, and high-cloud scenarios, which are conducive to maximizing solar irradiance and heat dissipation. These complementary meteorological attributes of the two clusters are consistent with the approximately 9 kW difference in power generation per class, with Cluster 1 corresponding to high-production periods and Cluster 0 reflecting weather-affected low-production periods. These meteorological differences emphasize the semantic validity of the clustering results in the GRAIL framework. Figure 3

[0127] Figure 5 The convergence behavior of the GRAIL alternating optimization is shown, where Figure 5 a is a plot of the silhouette score convergence curve over the outer iteration process, Figure 5 b is a plot of the Calinski-Harabasz criterion index convergence curve over the outer iteration process, Figure 5 c is a plot of the Davies-Bouldin index convergence curve over the outer iteration process, Figure 5 d is a plot of the normalized cluster score index convergence curve over the outer iteration process, Figure 5 ​e is the convergence curve of the encoder reconstruction error indicator during the external iteration process. The silhouette coefficient sharply rises from 0.13 in the 2nd iteration to 0.9 in the 4th iteration and stabilizes thereafter. This early stabilization indicates that the latent space is quickly organized into compact, well-separated clusters with minimal intra-cluster dispersion. The trajectory of the Calinski-Harabasz index is similar, although the initial change is gradual, it grows by three orders of magnitude between the 5th and 7th iterations. This growth continues after the silhouette coefficient stabilizes, indicating that the differences between clusters in the latent space continue to increase, reflecting a more discriminative representation. In contrast, the Davies-Bouldin index exhibits a monotonic decreasing trend, falling below 0.05 in the 6th iteration. This trend further supports the reduction in cluster overlap and the formation of well-separated substructures. The normalized entropy suddenly rises in the 3rd iteration and remains at the theoretical maximum thereafter, indicating that the distribution of samples across clusters tends to balance and stabilize. The reconstruction error of the autoencoder changes little initially and starts to decrease after the 5th iteration, reflecting the shift in GRAIL's ability from global cluster geometry stabilization to local reconstruction tasks, further improving data generation quality. These patterns illustrate the computational efficiency of GRAIL, which typically converges within six external iterations, significantly reducing training costs and outperforming typical deep clustering models. This accelerated optimization not only improves scalability but also enhances the model's responsiveness in scenarios where training resources are limited.

[0128] Embodiment 3

[0129] Based on the same inventive concept as Embodiment 1, this embodiment introduces a system for distributed photovoltaic spatio-temporal data recovery and unsupervised clustering, comprising:

[0130] A data acquisition module configured to acquire observation data of a distributed photovoltaic system, the observation data including historical distributed photovoltaic output time series;

[0131] A GRAIL framework module configured to input the acquired historical distributed photovoltaic output time series as an input signal into each module of the GRAIL framework to drive joint learning of temporal dynamic features and clustering structure.

[0132] The GRAIL framework is a closed-loop iterative structure, including a clustering-aware data generation module, a latent feature representation module, and an adaptive clustering and refinement module.

[0133] Embodiment 4

[0134] Based on the same inventive concept as other embodiments, this embodiment introduces a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the distributed photovoltaic spatio-temporal data recovery and unsupervised clustering method described above.

[0135] Embodiment 5

[0136] Based on the same inventive concept as other embodiments, this embodiment introduces a computer device, comprising: a memory, configured to store a computer program; and a processor, configured to execute the computer program to implement the steps of the method for distributed photovoltaic spatio-temporal data recovery and unsupervised clustering.

[0137] Embodiment 6

[0138] Based on the same inventive concept as other embodiments, this embodiment introduces a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method for distributed photovoltaic spatio-temporal data recovery and unsupervised clustering.

[0139] Those skilled in the art should understand that embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0140] The present application is described with reference to the flowcharts and / or block diagrams of the method, device (system) and computer program product according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as a combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device implemented in accordance with the flowcharts and / or block diagrams. Figure 1 The function specified in a flow or multiple flows and / or blocks Figure 1 The function specified in a flow or multiple flows and / or blocks

[0141] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that implement the flowcharts and / or block diagrams. Figure 1 The function specified in a flow or multiple flows and / or blocks Figure 1 The function specified in a flow or multiple flows and / or blocks

[0142] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the functions specified in the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the block Figure 1 Figure 1 one flow or multiple flows and / or the functions specified in the block

[0143] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and those skilled in the art can make many forms under the inspiration of the present application, which are all within the protection of the present application.

Claims

1. A method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering, characterized in that, include: Collect observation data of distributed photovoltaic systems, including historical time series of distributed photovoltaic power output; The collected historical distributed photovoltaic power output time series is used as input signals to various modules of the GRAIL framework to drive the network to jointly learn time series dynamic features. With clustering structure; The GRAIL framework is a closed-loop iterative structure, including a cluster-aware data generation module, a latent feature representation module, and an adaptive clustering and refinement module.

2. The method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering according to claim 1, characterized in that, The collected historical distributed photovoltaic power output time series is used as input signals to various modules of the GRAIL framework to drive the network to jointly learn time-series dynamic features. Clustering structures, including: The collected observation data is preprocessed to obtain several observation segments and their corresponding mask matrices. Each observation segment is input into the sequence generator, and the reconstructed complete time series is output. The reconstructed complete time series is input into an autoencoder to extract low-dimensional latent features; Density clustering is performed on the extracted low-dimensional latent features to obtain initial cluster labels; noise points in the obtained initial cluster labels are then processed. Nearest neighbor reclassification is performed to obtain the final cluster label. The final cluster label is encoded into a cluster embedding vector, and the cluster embedding vector is fed back to the sequence generator for the next round of optimization until the cluster structure and reconstruction error converge.

3. The method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering according to claim 2, characterized in that, The collected observation data is preprocessed to obtain several observation segments and their corresponding mask matrices. Each observation segment is input into a sequence generator, which outputs a reconstructed complete time series, including: Based on historical distributed photovoltaic power output time series, clustering embedding vectors are used. For the missing part of each observation segment Reconstruction is required; among which, Indicates the first Clustering embedding vectors of the categories to which each sample belongs. This indicates the length of the time window, specifically the number of time steps within a photovoltaic window. This represents the feature dimension at each time point. Indicates the first The observation segment with missing values ​​observed in the photovoltaic window, the first The sample and the first Each photovoltaic window corresponds one-to-one; Given a time series window and its corresponding binary mask ,in, Representing a time series window Missing; Time Series Window For the first Observation segments corresponding to each photovoltaic window; Embedding matrix Cluster labels Mapped to a continuous clustering embedding vector Then, with each observation segment in the time step Concatenate the sequences to form an enhanced sequence for use in the input sequence generator: ； in, This represents the reconstruction result output by the sequence generator, i.e., the first... The complete time series of each sample is reconstructed by the sequence generator. This represents a sequence generator implemented using an LSTM network structure.

4. The method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering according to claim 2, characterized in that, The process of inputting the reconstructed complete time series into an autoencoder to extract low-dimensional latent features includes: The automatic encoder includes an encoder and a decoder; Based on the reconstructed complete time series encoder Map it to latent coded features Then by the decoder Reconstructing the time series : ； in, Indicates the first Each sample is a complete time series segment reconstructed by a sequence generator; express The latent coded features output after encoder processing, i.e., low-dimensional latent features; express After being processed by the decoder, the first Decoding and reconstructing the time series of each sample; Representing latent coded features Dimensions.

5. The method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering according to claim 2, characterized in that, The extracted low-dimensional latent features are subjected to density clustering to obtain initial cluster labels, which are defined in the following formula (1). ; Perform the following steps on the noise points in the obtained initial cluster labels Nearest neighbor reclassification yields the final cluster labels; The final cluster labels are encoded into cluster embedding vectors, and these vectors are fed back into the sequence generator for the next round of optimization, until both the cluster structure and reconstruction error converge, including: Based on the extracted latent coded features The HDBSCAN algorithm is used to extract latent group structure: （1）； in, Indicates photovoltaic window The assigned cluster label, -1 indicates that the sample It was determined to be noise. This represents the HDBSCAN algorithm; For each cluster Its stability is defined as: ； in, Cluster stability, Indicates inverse density, Indicates density, This represents the number of samples in a cluster at a corresponding level in a hierarchical clustering structure. Cluster The minimum inverse density threshold that appears, Cluster The maximum inverse density threshold for disappearance is used to ultimately select the clustering configuration that maximizes overall stability. The initial cluster labels are obtained; where, Indicates the total number of samples; For any sample labeled as noise, i.e. For samples, their initial cluster labels will be reassigned to their... The majority class of the nearest non-noisy neighbors is used to obtain the reassigned cluster labels. This yields the final cluster label, which is the value in formula (2) below. : （2）； in, Indicates the first one sample A set of neighboring samples, The mode function returns the cluster label that appears most frequently. Indicates photovoltaic window The assigned cluster labels; each reassigned cluster label It will be mapped to a continuous clustering embedding vector Using embedding matrix accomplish: ； Clustering embedding vectors It will be sent back to the sequence generator.

6. The method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering according to claim 4, characterized in that, The encoder and decoder are jointly trained using a composite objective function: ； in, This represents the total loss function for jointly training the encoder and decoder. This represents the encoder reconstruction error loss, which is the first sub-objective, encouraging the preservation of time-series signal characteristics during the encoding process. This represents the cluster compactness loss, which is the second sub-objective, encouraging the clustering of samples of the same class in the latent space. This represents the clustering loss weight coefficient, which controls the balance between the two and is a hyperparameter. Encoder reconstruction error loss The mean squared error loss function is used to reconstruct the time series: ； in, Indicates the total number of samples; Cluster compactness loss Penalize the intra-class variance in the latent space: ； in, Indicates that the cluster label is the first A sample set of classes The cluster label is the first The latent space centroid corresponding to the class; Indicates clustering common Clusters.

7. A system for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering, characterized in that, include: The data acquisition module is configured to collect observation data of the distributed photovoltaic system, including historical distributed photovoltaic power output time series. The GRAIL framework module is configured to input the collected historical distributed photovoltaic power output time series as input signals into each module of the GRAIL framework, so as to drive the network to jointly learn the time series dynamic features and clustering structure. The GRAIL framework is a closed-loop iterative structure, including a cluster-aware data generation module, a latent feature representation module, and an adaptive clustering and refinement module.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the distributed photovoltaic spatiotemporal data recovery and unsupervised clustering method according to any one of claims 1 to 6.

9. A computer device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that: When executed by a processor, the computer program implements the steps of the method for distributed photovoltaic spatiotemporal data recovery and unsupervised clustering as described in any one of claims 1 to 6.

Images