A Method and System for Anomaly Detection of Irradiance Data Based on Hybrid Neural Networks

Abstract

This invention belongs to the technical field of anomaly data detection. To address the inaccuracy of existing irradiance data anomaly detection methods, it proposes a method and system for irradiance data anomaly detection based on a hybrid neural network. The method employs an autocorrelation function to determine the optimal temporal window length; adaptive dynamic time warping based on the physical constraints of cloud propagation and the local cost function based on the irradiance change rate determines the optimal time delay parameters between stations and performs time delay alignment; bidirectional gated recurrent units are used to extract the temporal dependency patterns of the target station; a time delay graph convolutional network is used to extract spatial correlation features between the reference and target stations considering time delay characteristics based on the time delay parameters; and a spatiotemporal attention mechanism is used for deep feature fusion to obtain the simulated irradiance value of the target station. The residual sequence is then used to determine whether the data at the current moment is an anomaly. This invention can accurately identify abnormal irradiance data.

Description

Technical Field

[0001] This invention belongs to the technical field of abnormal data detection, and in particular relates to a method and system for detecting abnormal irradiance data based on a hybrid neural network. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In practical engineering applications, the planning, operation, and maintenance of photovoltaic power plants all heavily rely on high-precision irradiance measurement data. During the planning phase, irradiance data is the foundation for solar resource assessment and photovoltaic power plant site selection. During grid-connected operation, accurate irradiance measurement significantly improves the accuracy of ultra-short-term photovoltaic power prediction, which is crucial for ensuring the safe and stable operation of the power grid. Furthermore, irradiance data can support the diagnosis of power generation faults and the optimization of operation and maintenance strategies for photovoltaic power plants. However, in actual measurement processes, due to various factors such as instrument malfunctions, communication transmission anomalies, installation calibration deviations, data storage overflows, and environmental weather disturbances, the original irradiance measurement data inevitably contains problems such as missing values, noise, or outliers.

[0004] In recent years, with the development of big data and artificial intelligence technologies, researchers have begun to explore the use of deep learning models for data quality control. Related research mainly focuses on using time series analysis algorithms to identify and correct anomalies in single-site data. By mining the time autocorrelation in historical observations of a single measurement point, isolated jump points, trend anomalies, or periodic deviations can be identified. However, single measurement points cannot verify the rationality of the data, making it difficult to effectively distinguish outliers caused by local sensor malfunctions or environmental interference. Furthermore, in cases of continuous data loss or prolonged anomalies, single measurement points can only rely on autoregressive prediction, and errors accumulate rapidly over time. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides a method and system for detecting anomalies in irradiance data based on a hybrid neural network. This method can accurately identify and effectively correct abnormal irradiance data, providing high-quality data support for photovoltaic power plant power prediction and meteorological data analysis.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for detecting anomalies in irradiance data based on a hybrid neural network, comprising: The autocorrelation function method was used to calculate the autocorrelation coefficient of the historical irradiance sequence of the target site under different delay orders, and the optimal time window length was determined. The optimal time delay parameters of each reference station relative to the target station are determined based on adaptive dynamic time warping, and time delay alignment is performed on the irradiance time series data of multiple stations; wherein, the adaptive dynamic time warping introduces physical law constraints of cloud propagation and local cost function based on the rate of change of irradiance. The target site's own irradiance sequence and time-delay-aligned multi-site irradiance time-series data are input into a hybrid neural network model. A bidirectional gated recurrent unit is used to learn the variation law of the target site's irradiance sequence and extract the time-series features of the target site's irradiance sequence. Based on the optimal time delay parameter, a time-delay graph convolutional network is used to extract spatial correlation features with propagation time delay characteristics between each reference site and the target site. The time-series features and spatial correlation features are deeply fused through a spatiotemporal attention mechanism to obtain the simulated irradiance value of the target site at the current moment. When the residual between the measured irradiance and the simulated irradiance at the target site at the current moment exceeds the dynamic threshold, the data at the current moment is determined to be an outlier. For data points identified as anomalous, simulated irradiance values ​​are used as correction results to replace the original anomalous data.

[0007] Secondly, the present invention provides an irradiance data anomaly detection system based on a hybrid neural network, comprising: The time-series window analysis module is configured to: use the autocorrelation function method to calculate the autocorrelation coefficient of the historical irradiance sequence of the target site under different delay orders, and determine the optimal time window length; The multi-site time delay alignment module is configured to: determine the optimal time delay parameters of each reference site relative to the target site based on adaptive dynamic time warping, and perform time delay alignment operation on the multi-site irradiance time series data; wherein, the adaptive dynamic time warping introduces physical law constraint terms of cloud propagation and local cost function based on irradiance change rate; The simulation module is configured to: input the target site's own irradiance sequence and time-delay-aligned multi-site irradiance time-series data into a hybrid neural network model; use bidirectional gated recurrent units to learn the variation law of the target site's irradiance sequence and extract the time-series features of the target site's irradiance sequence; based on the optimal time-delay parameters, use a time-delay graph convolutional network to extract spatial correlation features with propagation time-delay characteristics between each reference site and the target site; and deeply fuse the time-series features and spatial correlation features through a spatiotemporal attention mechanism to obtain the simulated irradiance value of the target site at the current moment. The anomaly detection module is configured to determine the current data as an anomaly when the residual between the measured irradiance and the simulated irradiance value at the target site at the current moment exceeds the dynamic threshold. The data correction module is configured to use simulated irradiance values ​​as the correction result for data points identified as anomalous, replacing the original anomalous data.

[0008] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0009] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0010] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0011] The above one or more technical solutions have the following beneficial effects: In this invention, the irradiance time lag correlation between stations is calculated based on adaptive dynamic time warping, the optimal time delay parameters are determined and time delay alignment is performed. By introducing physical law constraints of cloud propagation and a local cost function based on the rate of change of irradiance into the adaptive dynamic time warping, the time lag parameters of each reference station relative to the target station can be accurately estimated. The optimal time delay parameters are explicitly embedded into the time delay graph convolutional network processing, which can accurately model spatial correlations with physical propagation time delays. Through bidirectional gated recurrent units, time delay graph convolutional networks and spatiotemporal attention mechanisms, the spatiotemporal features of irradiance between different stations are collaboratively modeled, which significantly improves the anomaly detection accuracy and correction effect of irradiance data.

[0012] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0013] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0014] Figure 1 This is a graph showing the autocorrelation coefficients of the solar radiation time series in an embodiment of the present invention; Figure 2 This is a site alignment path diagram in an embodiment of the present invention; Figure 3 This is a structural diagram of the gated loop unit in an embodiment of the present invention; Figure 4 The following is a comparison of the simulation results of the models in the embodiments of the present invention; wherein, (a) is the simulation result of the BiGRU model, (b) is the simulation result of the spatial model, and (c) is the simulation result of the hybrid model; Figure 5 This is a box plot of the simulation error in an embodiment of the present invention; Figure 6 This is a violin plot showing the simulation error of the model in this embodiment of the invention; Figure 7 This is an image showing the anomaly detection results in an embodiment of the present invention; Figure 8 This is a comparison chart of data correction results in an embodiment of the present invention; Figure 9 This is a flowchart of the irradiance data anomaly detection method based on a hybrid neural network in an embodiment of the present invention. Detailed Implementation

[0015] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0016] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0017] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0018] Example 1 like Figure 9 As shown, this embodiment discloses a method for detecting anomalies in irradiance data based on a hybrid neural network, including: Step 1: Calculate the autocorrelation coefficient of the historical irradiance sequence of the target site under different delay orders using the autocorrelation function method, and determine the optimal time window length; Step 2: Determine the optimal time delay parameters of each reference station relative to the target station based on adaptive dynamic time warping, and perform time delay alignment operation on the multi-station irradiance time series data; wherein, the adaptive dynamic time warping introduces physical law constraint terms of cloud propagation and local cost function based on irradiance change rate; Step 3: Input the target site's own irradiance sequence and time-delay aligned multi-site irradiance time-series data into the hybrid neural network model. Use bidirectional gated recurrent units to learn the variation law of the target site's irradiance sequence and extract the time-series features of the target site's irradiance sequence. Based on the optimal time delay parameters, use a time-delay graph convolutional network to extract spatial correlation features with propagation time delay characteristics between each reference site and the target site. Use a spatiotemporal attention mechanism to deeply fuse the time-series features and spatial correlation features to obtain the simulated irradiance value of the target site at the current moment. Step 4: When the residual between the measured irradiance and the simulated irradiance value at the target site at the current moment exceeds the dynamic threshold, the data at the current moment is determined to be an outlier. Step 5: For data points identified as anomalous, use simulated irradiance values ​​as correction results to replace the original anomalous data.

[0019] In this embodiment, the autocorrelation function method is used to determine the optimal temporal window length; an adaptive dynamic time warping technique guided by physical constraints is used to calculate the irradiance time lag correlation between stations, determine the optimal time delay parameters, and perform time delay alignment; a bidirectional gated recurrent unit is used to extract the temporal dependency pattern of the target station itself; a time delay graph convolutional network is used to extract spatial correlation features with propagation time delay characteristics between the reference station and the target station based on the optimal time delay parameters; and a spatiotemporal attention mechanism is used to deeply fuse temporal and spatial features to obtain the simulated irradiance value of the target station. The residual sequence between the simulated and measured irradiance values ​​is calculated. When the residual between the measured irradiance and the simulated irradiance value of the target station at the current moment exceeds a dynamic threshold, the data at the current moment is determined to be an anomaly. For the data points determined to be anomalies, the simulated irradiance value is used as the correction result to replace the original abnormal data, generating a high-quality multi-site irradiance dataset. This embodiment effectively solves the problem that traditional methods ignore spatial propagation time delay characteristics and are difficult to achieve spatiotemporal feature collaborative modeling, significantly improving the anomaly detection accuracy and correction effect of irradiance data.

[0020] The following is a detailed description of the irradiance data anomaly detection method based on a hybrid neural network proposed in this embodiment, specifically including: Step 1: Calculate the autocorrelation coefficient of the historical irradiance sequence of the target site under different delay orders using the autocorrelation function method, and determine the optimal time window length.

[0021] Autocorrelation analysis is performed on the irradiance time series of the target site to determine a reasonable input sequence length for the subsequent gated cyclic unit module, thereby achieving a balance between capturing long-term dependencies and avoiding redundant calculations.

[0022] The optimal length of the input sequence is determined by the classic autocorrelation function (ACF) method in time series analysis. The autocorrelation coefficient measures the linear correlation of the time series under different time lags.

[0023] For a time length of irradiance sequence lag Autocorrelation coefficient of order Defined as: (1) In the formula, The duration is The mean of the irradiance sequence samples, The lag order; This represents the irradiance at time t.

[0024] Strong autocorrelation means that historical data has a significant impact on the current value, while the decay law of autocorrelation indicates the length boundary of effective historical information.

[0025] Figure 1 This is an autocorrelation coefficient plot of a solar radiation time series. The horizontal axis represents the lag order, with 15-minute intervals, and the vertical axis represents the autocorrelation coefficient, ranging from -1 to 1. The autocorrelation function of the solar radiation time series maintains significant autocorrelation within a lag of 13 (approximately 3.25 hours), with a correlation coefficient higher than 0.5, indicating that historical data within the short term has the strongest reference value for current irradiance. Subsequently, the coefficient turns from positive to negative and exhibits periodic fluctuations.

[0026] Step 2: Based on the physical constraint-guided adaptive dynamic time warping technique, determine the optimal time delay parameters of each reference station relative to the target station, and perform time delay alignment operation on the multi-station irradiance time series data.

[0027] Considering that factors such as cloud movement can cause differences in the timing of irradiance changes at different stations, traditional dynamic time warping techniques only align based on the similarity of sequence morphology, without taking into account the physical constraints of meteorological propagation, which can easily lead to non-physical time inversion alignment or time delay estimation bias.

[0028] Therefore, this embodiment improves the traditional dynamic time warping technique in the following three aspects and proposes a physically-constrained adaptive dynamic time warping (PCA-DTW) technique.

[0029] (1) Adaptive cost function based on irradiance change rate sequence.

[0030] The local cost function d(i,j) no longer uses the Euclidean distance based on the original irradiance sequence in traditional DTW, but instead uses a weighted Euclidean distance based on the irradiance change rate sequence: (2) In the formula, i is the time index of the target station; j is the time index of the reference station; The first-order difference of the irradiance of the target site at time i reflects the local rate of change; The first-order difference of the irradiance at time j is the reference station; The adaptive weighting factor is defined as follows: (3) In the formula, is the local standard deviation of the irradiance change rate sequence of the target site at time i, is the threshold for distinguishing normal fluctuations from drastic changes, which can be set according to historical data statistics.

[0031] (2) Introduce physical constraint terms.

[0032] To make the time delay estimation result conform to the physical law of cloud propagation, two types of physical constraints are introduced.

[0033] ① Regularization term guided by geographical distance: The time delay generated by the physical propagation process should be of the same order of magnitude as the time scale obtained by dividing the geographical distance between sites by the typical cloud movement speed. Based on this, the regularization term guided by geographical distance is defined as: (4) In the formula, is the geographical distance between the target site p and the reference site q, is the typical cloud movement speed estimated based on historical meteorological data, is the sampling time interval of the irradiance data, is the time offset corresponding to the current alignment point.

[0034] The core idea of this regularization term guided by geographical distance is that the time delay generated by the physical propagation process should be of the same order of magnitude as the time scale obtained by dividing the geographical distance between sites by the typical cloud movement speed. When the time offset of the path point (i, j) is much larger than this physical time scale, this match is unlikely to be caused by physical propagation and should be penalized; conversely, when the time offset is close to the physical time scale, the penalty is smaller. The path point (i, j) represents aligning the i-th moment of the target site p with the j-th moment of the reference site q.

[0035] ② Propagation direction constraint penalty term: Considering that cloud movement is driven by the prevailing wind direction, the time delay alignment should conform to the physical propagation direction from the upwind to the downwind. Define the propagation direction constraint penalty term as: (5) Among them, the specific situations violating the physical propagation direction include: the reference site is upwind of the target site and j > i, that is, the reference site time lags behind the target site; or the reference site is downwind of the target site and j < i, that is, the reference site time is ahead of the target site. The physical meaning of this design is that the greater the reverse time offset, the more serious the violation of the physical propagation direction and the greater the penalty.

[0036] (3) Improve the objective function of DTW: The aforementioned local cost function, geographical distance-guided regularization term, and propagation direction constraint penalty term are integrated into the objective function.

[0037] For a regular path , where K is the total number of steps on the path. The k-th path node on the path. This indicates the time at which the point will reach the target station p. The time relative to the reference site q Align them.

[0038] Define the overall objective function for: (6) In the formula, path point The corresponding weighting coefficients are used to avoid excessively large local spans in the path; The defined local cost function at the path point The value at; This represents the intensity coefficient of geographic distance regularization; The regularization term defined for formula (4) at path points The value at; These are the weighting coefficients for directional constraints, used to balance the importance of morphological similarity and physical propagation direction constraints; The directional penalty function at the path point The value at that location.

[0039] The objective function requires minimizing the morphological differences between sequences while satisfying the physical constraints of the propagation direction.

[0040] (4) Solving by dynamic programming: The above minimization problem can be solved using dynamic programming. Definition Let represent the minimum cumulative distance from the starting point (1,1) to the current position (i,j), expressed by the formula: (7) DTW requires regular paths to satisfy endpoint constraints, continuity constraints, and monotonicity constraints. Let the length of the irradiance sequence at target station p be N, and the length of the irradiance sequence at reference station q be M. Assume the sequence length N = M, starting from the starting point... Following formula (7) recursively to the endpoint D(N,M), the value stored in the endpoint cell is the total cost of the optimal regularized path. : (8) The path that reaches the minimum value by backtracking is the optimal regular path.

[0041] Specifically, taking station p as the target station, the improved DTW alignment path and normalization distance between the irradiance sequence of reference station q and the sequence of station p are calculated. Through the alignment path, it is possible to identify which historical time sequence of reference station q is most similar in shape to the current time sequence of target station p and satisfies the physical propagation constraints. This allows for the calculation of the maximum cross-correlation coefficient considering time delay adjustment, and the time offset is extracted from the optimal normalization path as the optimal time delay parameter between stations. Figure 2 As shown.

[0042] Step 3: Input the target site's own irradiance sequence and time-delay-aligned multi-site irradiance time-series data into the hybrid neural network model. Use bidirectional gated recurrent units to learn the variation law of the target site's irradiance sequence and extract the time-series features of the target site's irradiance sequence. Based on the optimal time delay parameters, use a time-delay graph convolutional network to extract the spatial correlation features with propagation time delay characteristics between each reference site and the target site. Use a spatiotemporal attention mechanism to deeply fuse the time-series features and spatial correlation features to obtain the simulated irradiance value of the target site at the current moment.

[0043] In this embodiment, the hybrid neural network model includes a bidirectional gated recurrent unit, a time-delay graph convolutional network, and a spatiotemporal attention mechanism.

[0044] Traditional recurrent neural networks (RNNs) allow information to pass between different time steps in the network, thereby enabling the mining and extraction of temporal information. However, traditional RNNs suffer from the problems of vanishing or exploding gradients when processing long-term sequences.

[0045] To address this technical bottleneck, Long Short-Term Memory (LSTM) neural networks introduce three key gates: the forget gate, the input gate, and the output gate. These three gates work together to control information forgetting, information updating, and state output. To further improve computational efficiency, the gated recurrent unit (GRU) integrates the forget gate and input gate of the LTM neural network into a single update gate and introduces a reset gate to control the input of historical information. It also simplifies the output mechanism, resulting in fewer training parameters and faster inference speed. Its network structure is as follows: Figure 3 As shown.

[0046] The specific working process of the gated loop unit is as follows: (1) Reset the door calculate: (9) In the formula, It is the Sigmoid activation function, which compresses the value to (0,1). and The weight matrix represents the hidden states from the previous time step. Input at the current time Connect to the reset door; The bias vector; the output of the reset gate. express The importance of the corresponding element to the candidate state; This represents the input at time t.

[0047] (2) Update the door calculate: (10) In the formula, and This is the weight matrix; This is the bias vector. The calculation process of the update gate is similar to that of the reset gate, but its output... Used to control the hidden state at the current moment. The latent state of the previous time step and candidate states The mixing ratio.

[0048] (3) Candidate hidden states calculate: (11) In the formula, tanh is the hyperbolic tangent activation function, which compresses the value to the interval (-1, 1) to generate candidate hidden states. ; The Hadamard product represents the element-wise multiplication of two vectors. and This is the weight matrix. This is the bias vector.

[0049] (4) Final hidden state calculate: (12) In the formula, The degree to which the final hidden state depends on the old state and the new candidate state; This represents the implicit state at the previous moment.

[0050] One-way gated loop units can only learn historical sequence patterns at the current time step. However, in some application scenarios, such as irradiance measurement data anomaly identification, future irradiance measurement information can be obtained simultaneously in offline scenarios. Therefore, it is also necessary to incorporate it into time series pattern learning.

[0051] The working process of the bidirectional gated loop unit is shown in the following formula: (13) In the formula, and These represent the forward-gated loop unit and the backward-gated loop unit, respectively; [;] represents the vector concatenation operation; This indicates the implicit state at the previous moment; This represents the input at time t, and in this embodiment, it represents the irradiance of the target site at time t.

[0052] To more accurately model the spatial relationships with time delay characteristics among multiple sites, this embodiment proposes to employ a time-delay graph convolutional network. Specifically, the multi-site system is constructed as a graph structure G=(V,E), where the node set V corresponds to each irradiance measurement site, and the edge set E represents the spatial relationships between sites. The adjacency matrix A of the graph is constructed based on the optimal time delay parameters determined by the improved dynamic time warping technique and the geographical distance, and is specifically defined as follows: (14) In the formula, The geographical distance between site p and site q. The optimal delay parameters are determined by the adaptive dynamic time warping technique. and This is the scale parameter. This definition gives higher weight to geographically proximate site pairs with lower latency.

[0053] The time-delay graph convolution operation is defined as follows: (15) In the formula, Adjacency matrix for adding self-loops; This is the corresponding degree matrix. For the features of the l-th layer nodes, Let σ be the learnable weight matrix, and σ be the activation function.

[0054] To overcome the limitations of using simple splicing or equal-weighted summation when fusing features in bidirectional gated recurrent units and time-delayed convolutional networks, this embodiment further introduces a spatiotemporal attention mechanism, which includes a temporal attention module and a spatial attention module.

[0055] The time attention module operates on the hidden state sequence output by the bidirectional gated loop unit. Let the hidden state output by the bidirectional gated loop unit at time step t be... Time attention weight Calculated in the following way: (16) In the formula, The attention score at time step t. and Here, T is the learnable parameter, T is the time window length, and u is the summation index. This is used to iterate through all time steps. Let be the attention score for time step u.

[0056] The weighted time series features are represented as follows: .

[0057] The spatial attention module is applied to the multi-site features extracted by the time-delay graph neural network. Let the features of the reference station p be... Spatial attention weights The calculation is as follows: (17) In the formula, and Here, S is a learnable parameter, and S is the number of reference sites. Let q be the spatial characteristics of the site.

[0058] Weighted spatial correlation features Represented as: .

[0059] By fusing the weighted temporal features and the weighted spatial correlation features, such as through a splicing operation, the simulated irradiance value of the target site at the current moment is obtained through a fully connected layer.

[0060] To enhance the physical consistency and anomaly sensitivity during hybrid neural network model training, this embodiment designs a composite loss function that integrates physical constraints and anomaly perception. Total Loss Function It consists of three parts: simulated loss Physical constraint loss and anomaly perception loss .

[0061] (1) Simulated loss : The mean squared error is used to measure the simulated value at time i. The true value at time i Differences between them: (18) (2) Physical constraint loss : ①Nonnegativity constraint

[0062] For simulated values Penalize negative samples: (19) ②Time smoothing constraint : Limit the simulated values ​​at adjacent time points, i.e., time point i+1. With time i Rate of change: (20) Physical constraint loss for: (twenty one) In the formula, , These are the weighting coefficients corresponding to non-negativity constraints and time smoothing constraints.

[0063] (3) Loss of anomaly perception : Dynamically adjust weights based on sample residuals This allows the model to focus on samples that are difficult to fit: (twenty two) In the formula, σ is the standard deviation of the residuals of the current batch, and n is the sample size.

[0064] This mechanism enables hybrid neural network models to adaptively enhance their ability to fit outliers during training, thereby improving the accuracy of anomaly detection.

[0065] The total loss function is: (twenty three) In the formula, and To balance the hyperparameters of each loss term.

[0066] Step 4: When the residual between the measured irradiance and the simulated irradiance value at the target site at the current moment exceeds the dynamic threshold, the data at the current moment is determined to be an outlier.

[0067] The three-standard-deviation rule is an anomaly detection method based on the assumption of a normal distribution. It assumes that the residual sequence follows a normal distribution. According to the probabilistic statistical properties of the normal distribution, the probability of a residual falling within the mean ± three standard deviations is 99.73%, while the probability of it falling outside this range is only 0.27%. Therefore, sample points outside this range are considered outliers.

[0068] Let the residual sequence be The formulas for calculating its mean μ and standard deviation σ are as follows: (twenty four) (25) The anomaly detection threshold is: (26) The rules for anomaly detection are as follows: (27) In the formula, Let be the i-th residual.

[0069] The three-standard-deviation rule has the advantages of being computationally simple and easy to implement, and is suitable for scenarios where the residuals approximately follow a normal distribution. However, this method is quite sensitive to outliers, as their presence can affect the calculation of the mean and standard deviation, potentially leading to a masking effect.

[0070] To overcome the limitations of the fixed threshold method, this embodiment further employs a dynamic threshold algorithm for anomaly detection. The dynamic threshold algorithm updates the statistical characteristics of the residuals in real time through a sliding window, enabling the threshold to adapt to the local fluctuations in the residual sequence.

[0071] Specifically, for the residual at the current time t Based on its first L historical residuals Calculate local mean and local standard deviation : (28) (29) The dynamic threshold is then: (30) In the formula, is the threshold coefficient, usually taken as a value between 2.5 and 3.5; L is the sliding window length, which is set according to the data sampling frequency and fluctuation characteristics. This represents the local mean at time t; This represents the local standard deviation at time t.

[0072] Dynamic thresholding algorithms can effectively capture local variation features of residual sequences, improving the adaptability of anomaly detection.

[0073] Step 5: For data points identified as anomalous, use the simulated irradiance value as the correction result to replace the original anomalous data; for data points identified as normal, retain the original measurement value.

[0074] This embodiment employs a dynamic threshold determination method based on a sliding window. It dynamically constructs anomaly detection thresholds by statistically analyzing the local mean and standard deviation of historical residuals, effectively addressing the poor adaptability of traditional fixed threshold methods. Experimental results demonstrate that the hybrid neural network model incorporating time delay characteristics exhibits superior performance in irradiance simulation tasks, accurately identifying and effectively correcting abnormal irradiance data, thus providing high-quality data support for photovoltaic power plant power prediction and meteorological data analysis.

[0075] The hybrid model constructed in this embodiment is evaluated, including: first, the simulation accuracy evaluation, and second, the anomaly detection effect evaluation.

[0076] 1. Simulation accuracy evaluation indicators.

[0077] (1) Root Mean Squared Error (RMSE) RMSE is essentially the square root of the mean of the squared errors, and it can intuitively reflect the accuracy of the simulation mixture model. The calculation formula is: (31) In the formula, n is the number of samples; For the first The actual irradiance value of each sample; For the first Simulated irradiance values ​​for each sample.

[0078] (2) Mean Absolute Error (MAE).

[0079] MAE represents the average absolute error between the true value and the simulated value, and its mathematical expression is as follows: (32) Both RMSE and MAE are error indicators that directly reflect how closely the true value is similar to the simulated value. The closer the simulated value is to the true value, the closer the obtained value is to 0, and vice versa. Therefore, the smaller the values ​​of RMSE and MAE, the higher the accuracy of the model.

[0080] (3) Coefficient of determination (R) 2 ).

[0081] (33) In the formula, The average value of the true irradiance; n is the number of samples; For the first The actual irradiance value of each sample; For the first Simulated irradiance values ​​for each sample.

[0082] The coefficient of determination reflects the trend error between the true and simulated values, and its value ranges from 0 to 1. The closer the value is to 1, the higher the accuracy and the better the model's fit; conversely, when the value is lower, the accuracy decreases. When the value approaches 0, it indicates a large deviation between the simulated value and the true value, and the model fit is poor.

[0083] 2. Indicators for anomaly detection effectiveness.

[0084] Anomaly detection is treated as a binary classification problem and evaluated using confusion matrix-related metrics. True Positive (TP): The number of samples that are actually outliers and are correctly detected as outliers by the model.

[0085] False Positive (FP): The number of samples that are actually normal values ​​but are incorrectly identified as outliers by the model.

[0086] True Negative (TN): The number of samples that are actually normal values ​​and are correctly identified as normal values ​​by the model.

[0087] False Negative (FN): The number of samples that are actually outliers but are missed by the model (judged as normal).

[0088] Based on the above definition, the following evaluation indicators are further calculated: (1) Accuracy (ACC).

[0089] Accuracy is the proportion of correctly classified samples out of the total number of samples, reflecting the overall accuracy of the model's classification. Its calculation formula is: (34) In the formula, TP represents a true negative instance and TN represents a true negative instance.

[0090] (2) Precision (PRE).

[0091] Precision is the proportion of samples that the model classifies as anomalous, but which are actually anomalous. It reflects the reliability of the model's anomaly detection results. Its calculation formula is: (35) In the formula, FP represents a false positive and TP represents a true positive.

[0092] (3) Recall (REC).

[0093] Recall is the proportion of all true outliers that are correctly detected by the model, reflecting the model's sensitivity to outliers. Its calculation formula is: (36) In the formula, FN represents a false negative.

[0094] (4) F1 score (F1-Score, F1).

[0095] The F1 score is the harmonic mean of precision and recall, comprehensively reflecting the model's performance in balancing precision and recall. When precision and recall conflict, the F1 score provides a holistic evaluation metric. Its calculation formula is: (37) In the formula, PRE represents precision; REC represents recall.

[0096] To evaluate the simulation capabilities of the hybrid model, it was compared with two benchmark models: a bidirectional gated recurrent unit model using only temporal information and a time-delay graph convolutional neural network model using only spatial information. By comparing the simulation accuracy of the different models on the same test set, it can be determined whether spatiotemporal feature fusion brings about a performance improvement.

[0097] By displaying the simulated and actual observation curves over two consecutive days, the fitting ability of the three models on different time scales can be intuitively compared, such as... Figure 4 As shown in (a)-(c), the BiGRU model can track the diurnal cycle variation trend of irradiance well, but it shows obvious response lag during periods of rapid weather change. The spatial model is more sensitive to synchronous spatial correlation changes and can respond quickly to regional weather changes, but its simulated values ​​are generally low at the peak of irradiance, making it difficult to reach the peak level observed in reality. This indicates that relying solely on spatial information cannot fully reflect the irradiance characteristics of the target site itself. The hybrid model proposed in this embodiment effectively integrates the continuity of time series with the consistency of space. It not only closely matches the real curve in terms of overall trend, but also responds promptly to rapid fluctuation events. At the same time, it can more accurately reproduce the actual peak value of irradiance, significantly improving the fitting accuracy and detail restoration capability.

[0098] Table 1 Comparison of Simulation Accuracy Evaluation Indicators

[0099] Analysis of Table 1 shows that the hybrid model outperforms the single time-series or spatial model in all error metrics, with a coefficient of determination closer to 1. This indicates that fusing spatiotemporal features can more comprehensively characterize the variation pattern of irradiance and reduce simulation bias caused by missing information. Furthermore, from the perspective of error distribution, the hybrid model exhibits more concentrated and smaller errors, demonstrating stronger robustness.

[0100] Table 2 Comparison of Anomaly Detection Performance Indicators

[0101] Table 2 shows the performance comparison results of different models in the anomaly detection task of irradiance data. As can be seen from the table, the hybrid model proposed in this invention performs best in all evaluation metrics, significantly outperforming the single BiGRU model and the spatial model. This indicates that fusing the time delay characteristics of multiple sites effectively enhances the model's ability to identify anomalous patterns, achieving significant improvements in reducing both false alarm and false negative rates. The hybrid model's F1 score of 0.8279 is 11.8%–30.1% higher than the comparative models, verifying the comprehensive advantages of the proposed method in anomaly detection tasks.

[0102] Figure 5 The box plots clearly show the distribution range, median, and dispersion of the simulation errors for the three models. The mixed model has the lowest error box position and the shortest length, indicating that its simulation results are more concentrated and stable. Figure 6 The violin plot shown further reveals the density pattern of the error distribution, confirming from a probability distribution perspective that the hybrid model can output high-precision simulations more stably and significantly reduce the risk of large errors.

[0103] Based on the high-precision simulation results, Figure 7 This further demonstrates the practical effectiveness of anomaly detection. By setting dynamic thresholds to identify outliers and replacing the original outlier observations with model-simulated values, automated quality control of irradiance data is achieved. The analysis of the correction results not only focuses on the number and proportion of outliers identified, but also emphasizes evaluating the improvement in physical consistency and spatiotemporal coherence of the corrected data.

[0104] Data order is as follows Figure 8 As shown, the proposed method can effectively identify obvious anomalous data, and the corrected data curves are smoother and more reasonable. The correction process did not interfere with normal data, demonstrating the accuracy of the method in distinguishing between anomalous fluctuations and natural changes. The corrected dataset can provide a more reliable data foundation for subsequent applications such as resource assessment and power prediction.

[0105] Example 2 The purpose of this embodiment is to provide an anomaly detection system for irradiance data based on a hybrid neural network, including: The time-series window analysis module is configured to: use the autocorrelation function method to calculate the autocorrelation coefficient of the historical irradiance sequence of the target site under different delay orders, and determine the optimal time window length; The multi-site time delay alignment module is configured to: determine the optimal time delay parameters of each reference site relative to the target site based on adaptive dynamic time warping, and perform time delay alignment operation on the multi-site irradiance time series data; wherein, the adaptive dynamic time warping introduces physical law constraint terms of cloud propagation and local cost function based on irradiance change rate; The simulation module is configured to: input the target site's own irradiance sequence and time-delay-aligned multi-site irradiance time-series data into a hybrid neural network model; use bidirectional gated recurrent units to learn the variation law of the target site's irradiance sequence and extract the time-series features of the target site's irradiance sequence; based on the optimal time-delay parameters, use a time-delay graph convolutional network to extract spatial correlation features with propagation time-delay characteristics between each reference site and the target site; and deeply fuse the time-series features and spatial correlation features through a spatiotemporal attention mechanism to obtain the simulated irradiance value of the target site at the current moment. The anomaly detection module is configured to determine the current data as an anomaly when the residual between the measured irradiance and the simulated irradiance value at the target site at the current moment exceeds the dynamic threshold. The data correction module is configured to use simulated irradiance values ​​as the correction result for data points identified as anomalous, replacing the original anomalous data.

[0106] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When executed by the processor, the computer instructions perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0107] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0108] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0109] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0110] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0111] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0112] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0113] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0114] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0115] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for detecting anomalies in irradiance data based on a hybrid neural network, characterized in that, include: The autocorrelation function method was used to calculate the autocorrelation coefficient of the historical irradiance sequence of the target site under different delay orders, and the optimal time window length was determined. The optimal time delay parameters of each reference station relative to the target station are determined based on adaptive dynamic time warping, and time delay alignment is performed on the irradiance time series data of multiple stations; wherein, the adaptive dynamic time warping introduces physical law constraints of cloud propagation and local cost function based on the rate of change of irradiance. The target site's own irradiance sequence and time-delay-aligned multi-site irradiance time-series data are input into a hybrid neural network model. A bidirectional gated recurrent unit is used to learn the variation law of the target site's irradiance sequence and extract the time-series features of the target site's irradiance sequence. Based on the optimal time delay parameter, a time-delay graph convolutional network is used to extract spatial correlation features with propagation time delay characteristics between each reference site and the target site. The time-series features and spatial correlation features are deeply fused through a spatiotemporal attention mechanism to obtain the simulated irradiance value of the target site at the current moment. When the residual between the measured irradiance and the simulated irradiance at the target site at the current moment exceeds the dynamic threshold, the data at the current moment is determined to be an outlier. For data points identified as anomalous, simulated irradiance values ​​are used as correction results to replace the original anomalous data.

2. The method for detecting anomalies in irradiance data based on a hybrid neural network as described in claim 1, characterized in that, Based on adaptive dynamic time warping, the optimal time delay parameters of each reference station relative to the target station are determined, and time delay alignment is performed on the multi-station irradiance time series data, specifically as follows: Construct a local cost function based on the irradiance change rate sequence; Based on the geographical distance between the target site and the reference site, the time offset corresponding to the current alignment point, and the sampling time interval of typical cloud movement speed and irradiance data, a geographical distance-guided regularization term is determined. The propagation direction constraint penalty is determined based on whether the reference station time lags behind the target station time; where a penalty is imposed if the reference station time lags behind the target station time. A total objective function is constructed based on the local cost function, the regularization term guided by geographical distance, and the propagation direction constraint penalty term. The total objective function is solved by dynamic programming to obtain the optimal time delay parameters between stations, thereby achieving time delay alignment of multi-site irradiance time series data.

3. The method for detecting anomalies in irradiance data based on a hybrid neural network as described in claim 1, characterized in that, A graph structure is constructed based on the spatial relationships between each irradiance measurement station and the measurement stations. An adjacency matrix is ​​jointly constructed based on the geographical distance between the measurement stations and the optimal time delay parameter. In the message passing of the time delay graph convolutional network, the spatial relationship features with propagation time delay characteristics between each reference station and the target station are modeled.

4. The method for detecting anomalies in irradiance data based on a hybrid neural network as described in claim 1, characterized in that, The determination of the dynamic threshold is specifically as follows: calculate the local mean and local standard deviation based on multiple historical residuals, and perform weighted fusion based on the calculated local mean and local standard deviation to obtain the dynamic threshold.

5. The method for detecting anomalies in irradiance data based on a hybrid neural network as described in claim 1, characterized in that, The loss function of the hybrid neural network model includes simulation loss, physical constraint loss, and anomaly perception loss; the physical constraint loss includes a non-negative irradiance constraint term and a time smoothing constraint term; the anomaly perception loss dynamically adjusts the weights based on the sample residuals, enabling the hybrid neural network model to pay more attention to outliers during training.

6. The method for detecting anomalies in irradiance data based on a hybrid neural network as described in claim 1, characterized in that, The temporal and spatial correlation features are deeply integrated through a spatiotemporal attention mechanism, specifically: The hidden state output by the bidirectional gated recurrent unit is weighted using temporal attention weights to obtain the weighted temporal features; Spatial association features with propagation delay characteristics are weighted using spatial attention weights to obtain weighted spatial association features. The weighted temporal features and weighted spatial correlation features are fused together, and the simulated irradiance value of the target site at the current moment is obtained through a fully connected layer.

7. A system for detecting anomalies in irradiance data based on a hybrid neural network, characterized in that, include: The time-series window analysis module is configured to: use the autocorrelation function method to calculate the autocorrelation coefficient of the historical irradiance sequence of the target site under different delay orders, and determine the optimal time window length; The multi-site time delay alignment module is configured to: determine the optimal time delay parameters of each reference site relative to the target site based on adaptive dynamic time warping, and perform time delay alignment operation on the multi-site irradiance time series data; wherein, the adaptive dynamic time warping introduces physical law constraint terms of cloud propagation and local cost function based on irradiance change rate; The simulation module is configured to: input the target site's own irradiance sequence and time-delay-aligned multi-site irradiance time-series data into a hybrid neural network model; use bidirectional gated recurrent units to learn the variation law of the target site's irradiance sequence and extract the time-series features of the target site's irradiance sequence; based on the optimal time-delay parameters, use a time-delay graph convolutional network to extract spatial correlation features with propagation time-delay characteristics between each reference site and the target site; and deeply fuse the time-series features and spatial correlation features through a spatiotemporal attention mechanism to obtain the simulated irradiance value of the target site at the current moment. The anomaly detection module is configured to determine the current data as an anomaly when the residual between the measured irradiance and the simulated irradiance value at the target site at the current moment exceeds the dynamic threshold. The data correction module is configured to use simulated irradiance values ​​as the correction result for data points identified as anomalous, replacing the original anomalous data.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.

Images