A method for off-grid photovoltaic array fault diagnosis based on ConvTran

Abstract

A kind of off-grid photovoltaic array fault diagnosis method based on ConvTran, comprising the following steps: constructing Transformer model;Convolution module is used on the Transformer architecture, before input embedding vector is input into Transformer module, position embedding vector generated by tAPE is added to input embedding vector, after obtaining the final output of Transformer module, global average pooling and full connection layer are applied to process, obtain more translation invariance model, finally, the result of classification prediction is obtained by applying Softmax function;Collect the multivariate time series data of four channels of photovoltaic array, input ConvTran network model, and carry out fault diagnosis to photovoltaic array.The method is more suitable for the fault type classification of multivariate time series, and shows more excellent applicability and accuracy in off-grid photovoltaic array fault diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of online monitoring and fault diagnosis of power equipment, and particularly relates to a method for off-grid photovoltaic array fault diagnosis based on ConvTran. BACKGROUND

[0002] At present, the fault diagnosis methods for photovoltaic arrays can be divided into two categories: visual imaging and electrical characteristic parameters. Visual imaging requires the use of specific detection instruments in a specific environment to diagnose the faults of photovoltaic modules, and it is also unable to monitor large photovoltaic arrays in real time, has a high use threshold, and is difficult to obtain a large number of fault samples. The photovoltaic array fault diagnosis methods for electrical characteristic parameters include the circuit structure method, the mathematical model method, and the machine learning method, etc. The circuit structure method configures voltage and current sensors in a corresponding embedding mode according to different connection structures of photovoltaic arrays to locate the approximate range of faults. However, this method needs to arrange a large number of sensors at the photovoltaic array, increases the complexity of wiring, and has a high cost, which is not suitable for large photovoltaic power stations. The mathematical model method first constructs a mathematical model of the expected output of the photovoltaic array, and then imports the measured voltage and current data into the pre-constructed mathematical model to estimate the working state of the photovoltaic array. However, due to the variability and complexity of the operating environment of the photovoltaic array, the constructed mathematical model cannot accurately reflect the actual operating conditions of the photovoltaic array, which weakens the adaptability of the method in actual working conditions and further affects the accuracy of fault diagnosis.

[0003] The machine learning method is the current mainstream photovoltaic array fault diagnosis method, which establishes the mapping relationship between the fault characteristic values and the fault types in the photovoltaic array through a machine learning algorithm to realize fault diagnosis. Compared with the traditional fault diagnosis method, the machine learning method exhibits faster speed and higher accuracy in identifying and classifying photovoltaic array faults.

[0004] Since the output characteristics of the photovoltaic array are related to meteorological conditions such as solar irradiance and temperature, the changes in electrical signals are closely related to time, so the fault diagnosis method of time-series voltage and current is selected, i.e., the voltage and current of each branch of the photovoltaic array are measured at the direct-current combiner box of the photovoltaic power generation system. This fault detection method can diagnose during the operation of the photovoltaic system. The fault recognition and positioning based only on the time-series changes of the voltage and current of the photovoltaic array is the most efficient fault detection method, which greatly reduces the workload of collecting and preprocessing the data set. Therefore, how to use a multivariate time-series classification method to diagnose the faults of the photovoltaic array is a problem to be solved by those skilled in the art. SUMMARY

[0005] In view of this, the application provides an off-grid photovoltaic array fault diagnosis method based on ConvTran, which combines the advantages of the ConvTran network model to improve the applicability and stability of photovoltaic array fault diagnosis. The specific method steps are as follows:

[0006] S1: Construct a Transformer model, including an input module, an encoder module, a decoder module and an output module;

[0007] S2: Use a convolution module on the Transformer architecture to shorten the length of the time series and capture the local information present in the original time series;

[0008] S3: Before inputting the input embedding vector into the Transformer module, add the position embedding vector generated by tAPE to the input embedding vector, so that the model can capture the time sequence of the time series;

[0009] S4: After obtaining the final output of the Transformer module, apply global average pooling and a fully connected layer for processing to obtain a model with more translation invariance, and finally apply the Softmax function to obtain the classification prediction result, and finally construct the ConvTran model;

[0010] S5: Collect the voltages UA and UB and branch currents IA and IB on the DC output side of the photovoltaic array, form a four-channel multivariate time series data, collect a time point every 1s, and pack a group of samples every 30s, input into the ConvTran network model, and perform fault diagnosis on the photovoltaic array.

[0011] Further, the input of the Transformer architecture is sequence data, which is first converted into a feature vector by an embedding algorithm (Embedding). If the input is a sentence, the word embedding algorithm such as word2vec, GloVe, one-hot encoding can be used to convert the input sentence into a word vector after word segmentation. After embedding operation on the input sequence, the feature vector also needs to be positionally encoded (Positional Encoding), and the positional encoding has the same dimension as the input embedding, so as to add the positional encoding information and the encoding of the corresponding position sequence vector, so that the self-attention mechanism can consider the sequence order information and all input sequence information. For multivariate time series, Transformer trains all input time series at the same time, and the self-attention layer cannot preserve the time sequence position information in the Transformer architecture, so position encoding is needed to help it understand the sequence order. The method of position encoding includes absolute position encoding and relative position encoding to enhance the time background of time series input.

[0012] The Encoder module is composed of multiple stacks of the same structure, each Encoder is composed of two sub-layers, Self-Attention and Fully Connected Feed-forward Network, each sub-layer adopts Residual Connection, and then Layer Normalization. It should be noted that although the structure of each Encoder is exactly the same, the weight parameters are different, the parameters are trained independently, and the output of the Encoder module is input to each Decoder to perform multi-head attention mechanism.

[0013] The Decoder module is also composed of multiple stacks of the same structure, in addition to the Self-Attention layer and the Feed-forward network, the Decoder has a third sub-layer, that is, the multi-head attention mechanism is performed on the output obtained by the Encoder stack. Unlike the Encoder, the Self-Attention layer of the Decoder adds Masking to ensure that the prediction of position i can only rely on the known output located before position i, preventing the current position from being affected by subsequent positions. Like the Encoder module, each sub-layer of the Decoder module also performs Residual Connection and Layer Normalization operation, although the structure of each Decoder is exactly the same, the weight parameters are different, and the parameters are trained independently.

[0014] The final output module is composed of a linear layer and a Softmax layer. The linear layer is a simple fully connected neural network that maps the output vector of the decoding module to a longer vector, i.e. the logits vector. The Softmax layer converts the attention score of each sequence segment into a probability distribution between 0 and 1, and selects the result corresponding to the highest probability as the output of this time step.

[0015] The attention mechanism can be described as a process of mapping a query and a set of key-value pairs to an output, where the query, key, value, and output are all vectors.

[0016] For a d x dimensional input sequence x t , xt={x1,x2,…,x L}, where L is the length of the time series, and the d z dimensional output sequence obtained after self-attention calculation is z t , zt={z1,z2,…,z L}, where z i is calculated by the weighted sum of input elements, as shown in equation (1).

[0017]

[0018] The weight of each coefficient α i,j is calculated by the Softmax function, as shown in equation (2).

[0019]

[0020] In the equation, e ij is the attention weight from position j to position i, which is calculated by the scaled dot product, as shown in equation (3). The higher the correlation between position j and position i, the larger the result after dot product operation. In the equation, the projection is a parameter matrix, which is different for each layer.

[0021]

[0022] Therefore, the attention mechanism is essentially to calculate the weight coefficient of the corresponding value in the input elements through the query and key in the input elements, and then to weight and sum the value according to the weight coefficient.

[0023] Multi-head attention (MHA) replaces the original self-attention mode of calculating only once. It uses h different learned linear transformations to linearly map the query, key and value respectively, then performs scaled dot product attention calculation at the same time, and finally linearly transforms the concatenated output value to generate the final result.

[0024] The core idea of the multi-head attention mechanism is to maintain the total parameter size unchanged, and to independently perform attention calculation in different high-dimensional subspaces by mapping the same query, key and value in different subspaces, and then to fuse the attention information in different subspaces. In this way, the dimension of a single vector in each attention calculation is reduced, which prevents overfitting in a certain sense. Since the attention has different distribution patterns in different subspaces, multi-head attention actually explores the correlation between sequences from different angles. Through different self-attention layers, multiple feature representations are obtained, and the features captured in different subspaces are spliced together.

[0025] The original self-attention considers the absolute position, and adds the absolute position embedding P = (p1, …, p L ) in the input embedding x, as shown in equation (4).

[0026] x i = x i + p i(4)

[0027] where the position embedding There are several ways to encode absolute position, including fixed position encoding by sine and cosine functions of different frequencies, called Vanilla APE, or learnable encoding by trainable parameters.

[0028] By using fixed position encoding by sine and cosine functions, the d model dimensional position embedding at the i-th time step can be represented by equation (5).

[0029]

[0030] where d model is the embedding dimension, which is the same as the dimension of the sequence feature vector; k is the dimension, which is in the range of ; w k is the frequency term. Each position gets a combination of sine and cosine functions of different periods in the embedding dimension, resulting in unique texture position information, and finally making the Transformer model learn the dependency and timing characteristics between different positions.

[0031] In addition to absolute position embedding, relative position considers the pairwise relationship between input elements. This method encodes the relative distance between input elements x i and x j into a vector Embedding the encoding vector into the self-attention module, equations (1) and (3) are modified to obtain equations (6) and (7). By this method, the pairwise position relationship is trained in the training process of the Transformer.

[0032]

[0033] Relative position information is provided to the model from two levels of value and key. First, the relative position information is included in the model as an additional component of the key, and the Softmax operation is shown in equation (3), which is the same as Vanilla self-attention. Finally, the relative position information is provided again as a sub-component of the value matrix. In addition, considering that the effectiveness of the relative position information will decrease significantly after a certain distance, in order to improve the efficiency and performance of the model, a clip function is introduced to limit and reduce the number of required parameters. When calculating the attention between position i and position j, the distance between them is considered, and the encoding calculation is shown in equations (8) to (10).

[0034]

[0035] clip(x, k) = max(-k, min(k, x)) (10)

[0036] where p V and p K are trainable weights for the relative position encoding of values and keys, respectively. where The scalar k is the maximum relative distance.

[0037] As can be seen from equation (8), it requires O(L 2 d) memory due to the additional relative position encoding. There is a new method for calculating relative position encoding, called vector method, which reduces the intermediate memory requirement from O(L 2 d) to O(Ld) using the offset operation, giving up the additional relative position embedding corresponding to the value item, only focusing on the key component. The encoding calculation is shown in equation (11) and equation (12). Where the Skew procedure uses padding, reshaping and slicing to reduce the memory requirement.

[0038]

[0039] S rel = Skew(W Q P) (12)

[0040] When applied to time series data, the Transformer model requires effective position encoding to capture the ordering of time series data. The present technology applies a new time series absolute position encoding method (tAPE), which integrates time series length and input embedding dimension into absolute position encoding. An efficient relative position encoding method (eRPE) is also applied. These two position encoding methods are simple and effective, and can be easily integrated into the Transformer module to improve the generalization ability of time series. The ConvTran network model combines tAPE, eRPE and convolution-based input encoding to improve the position and data embedding of time series data, and achieves excellent results in multivariate time series classification tasks.

[0041] Absolute position encoding was originally proposed for language modeling tasks, usually using 512 or 1024 high embedding dimensions for position embeddings of 512 length input, higher embedding dimensions can better reflect the similarity between different positions. When using lower embedding dimensions for position encoding, the similarity between two positions calculated by the dot product does not always decrease with the increase of the distance between the two positions, and the distance perception attribute disappears.

[0042] Although high embedding dimensions will show an ideal monotonically decreasing trend with the increase of the distance between two positions, it is not suitable for encoding time series data sets, because most time series data sets have low data dimensions, and higher embedding dimensions may reduce model throughput due to additional parameters, increasing the risk of model overfitting. On the other hand, at low embedding dimensions, the similarity between two random embedding vectors is high, which is called anisotropy phenomenon, so the embedding vector space cannot be fully utilized to distinguish two positions, and position encoding fails at low embedding dimensions.

[0043] Therefore, the algorithm requires position embedding of time series to have distance awareness while being isotropic. In order to incorporate distance awareness, the length of time series is used in equation (5). In this equation, w k refers to the frequency of the sine and cosine functions that generate the embedding vector. If not modified, as the sequence length L increases, the dot product between positions will become more and more irregular, resulting in the loss of distance perception. After introducing the length parameter into the frequency term of the sine and cosine functions in equation (5), the dot product will maintain a monotonically smooth trend.

[0044] As the embedding dimension d model increases, the vector embedding is more likely to sample from the low frequency sine function, resulting in anisotropy phenomenon. In order to alleviate this problem, d model The parameter is introduced into the frequency term of the sine and cosine functions in equation (5). Using the new time series based absolute position encoding method (tAPE), where The input embedding dimension d model and the length of time series L are considered, as shown in equation (13):

[0045]

[0046] Compared with Vanilla absolute position encoding, the dot product representing the similarity between two positions has a more stable monotonic decreasing trend as the distance between the two positions in the time series increases, and the similarity between the embedding vectors of tAPE decreases. This is because tAPE can provide isotropic encoding in the embedding space, maintain distance-aware characteristics, and better utilize the embedding vector space to distinguish between two positions.

[0047] The input embedding is the basis of all previous relative position encoding methods, which add or multiply the position matrix with the query, key and value matrix. In this paper, an efficient relative position embedding (eRPE) model independent of the input embedding is introduced.

[0048] The calculation formula used in the eRPE model is shown in equation (14).

[0049]

[0050] In the formula, L is the sequence length, e i,j is the attention weight, w i-j is a learnable scalar representing the relative position weight between position i and position j,

[0051] is each attention module in the multi-head attention mechanism, creating a trainable parameter w of size 2L-1, because the maximum distance is 2L-1. For the indices i and j of two positions, the corresponding relative scalar is w i-j+L , where the index starts from 1 instead of 0, and L 2 elements need to be indexed from the 2L-1 vectors.

[0052] First, the relative position embedding w i-j is a static parameter independent of the input, and the attention weight e i,j is dynamically determined by the representation of the input sequence. Attention adapts to the input sequence through an input-adaptive weighting strategy, enabling the model to capture complex relationships between different time points, which is exactly the feature needed when extracting high-level concepts from time series, such as seasonal components in time series. However, when the data size is limited, using attention will face greater risk of overfitting.

[0053] Second, the relative position embedding w i-j considers the relative displacement between positions i and j, rather than their values. This is similar to the translational invariance of convolution, which has been proven to enhance generalization ability, so w i-jConsider as scalar instead of vector to achieve translation invariance without increasing the number of parameters. Moreover, the value of w i-j all (i,j) can be included in the paired dot product attention function, minimizing the additional amount of calculation, and this efficient relative position encoding method is eRPE.

[0054] eRPE first applies the Softmax function to the attention matrix, and then adds the relative position information to the model. Because the position value without the Softmax function is clearer, the performance of the attention model will be better. Compared with the existing model in which the Softmax is applied to the relative position embedding, the clearer position embedding of eRPE is more conducive to the time series classification task.

[0055] From the foregoing analysis, the complexity of global attention is quadratic to the length of the sequence. If the attention proposed in equation (14) is directly applied to the original time series, the calculation speed will be too slow for long time series. Therefore, the ConvTran network model is introduced. First, the length of the time series is shortened using the convolution module, and then the feature map is reduced to a size with lower calculation intensity, and then the new position encoding method is applied.

[0056] Using the convolution module on the Transformer architecture not only reduces the calculation intensity and improves the network training speed, but the convolution operation is also very suitable for capturing local features. Therefore, the convolution is taken as the first module in the ConvTran model architecture, which is convenient for capturing any local information with distinction in the original time series.

[0057] In the convolution module, M time convolution kernels are first applied to the input multivariate time series data, so that the ConvTran network model extracts the time information in the input sequence. Then, the output of the time convolution kernel is convolved with d model d x x M size spatial convolution kernel to capture the correlation between variables in the original time series and construct d model size input embedding. This disjoint time-space convolution first expands the number of input channels and then compresses them. One key reason for this choice is that the feedforward network (FFN) in the Transformer also expands the size of the input, and then projects the expanded hidden state back to the original size to capture the spatial interaction.

[0058] Before inputting the input embedding vector into the Transformer module, the position embedding vector generated by tAPE is added to the input embedding vector, so that the model can capture the time order of the time series. The size of the tAPE embedding vector is d modelThe same as the input embedding vector. In multi-head attention, first use a linear layer to convert the Lxd model dimension input into Lxd z dimension, that is, Lxd z dimension q(query), k(key), v(value) matrix, where d z represents the dimension of the model, which is a custom parameter. These q, k, v matrices are reshaped to h x L x d z / h to represent the hth attention head. Each attention head is responsible for capturing different patterns in the time series, for example, one attention head focuses on non-noise data, another attention head focuses on seasonal components, and another attention head focuses on trends. After obtaining the q, k, v matrices, finally, attention calculation is performed within the multi-head attention module using equation (4-14).

[0059] The feedforward network in the Transformer model is a multi-layer perceptron module composed of two linear layers and a Gaussian Error Linear Units (GELUs) activation function. GELUs introduces the idea of random regularization in the activation function, which is a probabilistic description of the input of a neuron, a combination of dropout, zoneout, and ReLU, which can improve the generalization ability of the model. Assuming the input is X and the mask is m, then m follows a Bernoulli distribution F(x) = P(X < x), where X follows a standard normal distribution, that is, X ~ N(0, 1). The mathematical expression of GELUs is shown in equation (15).

[0060] GELU(x) = xΦ(x) = xP(X < x) (15)

[0061] The same as the Transformer basic architecture, the multi-head attention layer and the feedforward network layer in the ConvTran network also apply residual connection and layer normalization to obtain the final output of the Transformer module. Then apply max pooling and global average pooling (GAP) to the output of the ELU activation function in the last layer to obtain a more translation-invariant model. Finally, apply the Softmax function to obtain the classification prediction result.

[0062] The present application can achieve the following beneficial effects:

[0063] Compared with other improved algorithms based on CNN such as FCN, MC-DCNN, ResNet, MLSTM-FCN and improved algorithm GTN based on Transformer, the recognition ability of the ConvTran algorithm to various faults of the photovoltaic array is more balanced, and the fault diagnosis accuracy of the off-grid photovoltaic array is the highest, especially under the fault of partial shadow shielding, the diagnosis accuracy is significantly higher than that of other algorithms. The improved algorithm combining convolution and Transformer, the ConvTran network model improves the position and data embedding of time series data in the Transformer architecture, has better classification effect in the multi-element time series classification task, can distinguish the slight difference between short circuit fault and partial shadow shielding, is more suitable for fault type classification of multi-element time series, and has more excellent applicability and accuracy in off-grid photovoltaic array fault diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 The Transformer model architecture of the application is shown in the figure;

[0065] Figure 2 The residual connection and layer normalization schematic diagram of the application is shown in the figure;

[0066] Figure 3 The composition structure diagram of multi-head attention of the application is shown in the figure;

[0067] Figure 4 The self-attention module diagram of relative position encoding of the application is shown in the figure;

[0068] Figure 5 The overall architecture diagram of the ConvTran model of the application is shown in the figure;

[0069] Figure 6 The GELU activation function curve diagram of the application is shown in the figure;

[0070] Figure 7 The overall structure diagram of the grid-connected photovoltaic array fault simulation test platform of the application is shown in the figure;

[0071] Figure 8 The photovoltaic array fault recognition confusion matrix diagram based on ConvTran of the application is shown in the figure;

[0072] Figure 9 The off-grid photovoltaic array fault recognition confusion matrix diagram of the comparative model algorithm of the application is shown in the figure;

[0073] Figure 10 The accuracy diagram of the off-grid photovoltaic array fault recognition of each algorithm of the application is shown in the figure;

[0074] Figure 11 The total accuracy diagram of the off-grid photovoltaic array fault recognition of each algorithm of the application is shown in the figure. DETAILED DESCRIPTION

[0075] With reference to the drawings accompanying the specification Figures 1-11 The application provides a ConvTran-based off-grid photovoltaic array fault diagnosis method,

[0076] S1: Constructing a Transformer model, including four parts of an input module, an encoder module, a decoder module and an output module;

[0077] S2: Using a convolution module on the Transformer architecture to shorten the length of the time series and capture the local information existing in the original time series;

[0078] S3: Adding the position embedding vector generated by tAPE to the input embedding vector before inputting the input embedding vector into the Transformer module, so that the model can capture the time sequence of the time series;

[0079] S4: After obtaining the final output of the Transformer module, applying global average pooling and a fully connected layer for processing to obtain a model with more translation invariance, and finally applying a Softmax function to obtain the classification prediction result, and finally constructing a ConvTran model;

[0080] S5: Collecting the voltages UA and UB and branch currents IA and IB on the DC output side of the photovoltaic array, forming a four-channel multivariate time series data, collecting a time point every 1s, and packaging a group of samples every 30s, inputting the ConvTran network model, and performing fault diagnosis on the photovoltaic array.

[0081] Further, the input of the Transformer architecture is sequence data, which is first converted into a feature vector through an embedding algorithm (Embedding). If the input is a sentence, the input sentence can be converted into a word vector after word segmentation by using a word embedding algorithm such as word2vec, GloVe, one-hot encoding, etc. After the embedding operation on the input sequence, the feature vector also needs to be positionally encoded (Positional Encoding). The position encoding has the same dimension as the input embedding, so as to add the position encoding information and the encoding of the corresponding position sequence vector, so that the self-attention mechanism can consider both the sequence order information and all input sequence information. For multivariate time series, the Transformer simultaneously trains all input time series, and the self-attention layer cannot retain the position information of the time series in the Transformer architecture, so position encoding is needed to help it understand the sequence order. The method of position encoding includes absolute position encoding and relative position encoding, to enhance the time background of the time series input.

[0082] Encoder module is composed of multiple encoder stacks with the same structure. Each encoder is composed of two sub-layers, self-attention layer and fully connected feed-forward network. Each sub-layer adopts residual connection, and then layer normalization. It is worth noting that although the structure of each encoder is exactly the same, the weight parameters are different, and the parameters are trained independently. The output of the encoder module is input into each decoder to perform multi-head attention mechanism.

[0083] Decoder module is also composed of multiple decoder stacks with the same structure. In addition to self-attention layer and feed-forward network, the decoder has a third sub-layer, which performs multi-head attention mechanism on the output of the encoder stack. Unlike the encoder, the self-attention layer of the decoder adds masking to ensure that the prediction of position i can only rely on the known output before position i, preventing the current position from being affected by subsequent positions. Like the encoder module, each sub-layer of the decoder module also performs residual connection and layer normalization. Although the structure of each decoder is exactly the same, the weight parameters are different, and the parameters are trained independently.

[0084] The final output module is composed of a linear layer and a softmax layer. The linear layer is a simple fully connected neural network that maps the output vector of the decoding module to a longer vector, i.e. logits vector. The softmax layer converts the attention score of each sequence segment into a probability distribution between 0 and 1, and selects the result corresponding to the highest probability as the output of this time step.

[0085] Attention mechanism can be described as a process of mapping a query and a set of key-value pairs to an output, where the query, key, value and output are all vectors.

[0086] For a d x dimensional input sequence x t , xt={x1,x2,…,x L}, where L is the length of the time series, and the d z dimensional output sequence obtained after self-attention calculation is z t , zt={z1,z2,…,z Lwherein z i is calculated by the weighted sum of input elements, as shown in equation (1).

[0087]

[0088] The weight of each coefficient is α i,j is calculated by the Softmax function, as shown in equation (2).

[0089]

[0090] In the formula, e ij is the attention weight from position j to position i, which is calculated by the scaled dot product, as shown in equation (3). The higher the correlation between position j and position i, the larger the result after dot product operation. In the formula, the projection is a parameter matrix, which is different for each layer.

[0091]

[0092] Therefore, the attention mechanism is essentially to calculate the weight coefficient of the corresponding value in the input elements through the query and key in the input elements, and then to weight and sum the value according to the weight coefficient.

[0093] Multi-head attention (MHA) replaces the original self-attention mode of calculating only once. It uses h different learned linear transformations to linearly map the query, key and value respectively, then performs scaled dot product attention calculation at the same time, and finally linearly transforms the concatenated output value to generate the final result.

[0094] The core idea of the multi-head attention mechanism is to maintain the overall parameter size unchanged, and to independently perform attention calculation in different high-dimensional subspaces by mapping the same query, key and value in different subspaces, and then to fuse the attention information in different subspaces. In this way, the dimension of a single vector in each attention calculation is reduced, which prevents overfitting in a certain sense. Since the attention has different distribution patterns in different subspaces, multi-head attention actually explores the correlation between sequences from different angles. Through different self-attention layers, multiple feature representations are obtained, and the features captured in different subspaces are spliced together.

[0095] The original self-attention considers the absolute position, and adds the absolute position embedding P = (p1, …, p L ) in the input embedding x, as shown in equation (4).

[0096] x i = x i + pi (4)

[0097] where the position embedding There are several ways to encode absolute position, including fixed position encoding by sine and cosine functions of different frequencies, called Vanilla APE, or learnable encoding by trainable parameters.

[0098] By using fixed position encoding by sine and cosine functions, the d model dimensional embedding of position i at time step t can be represented by equation (5).

[0099]

[0100] where d model is the embedding dimension, which is the same as the dimension of the sequence feature vector; k is the dimension, which is in the range of ; w k is the frequency term. Each position gets a combination of sine and cosine functions of different periods in the embedding dimension, resulting in unique texture position information, and finally making the Transformer model learn the dependency and timing characteristics between different positions.

[0101] In addition to absolute position embedding, relative position considers the pairwise relationship between input elements. This method encodes the relative distance between input elements x i and x j into a vector Embedding the encoding vector into the self-attention module, equations (1) and (3) are modified to obtain equations (6) and (7). By this method, the pairwise position relationship is trained in the training process of the Transformer.

[0102]

[0103] Relative position information is provided to the model from two levels of value and key. First, the relative position information is included in the model as an additional component of the key, and the Softmax operation is shown in equation (3), which is the same as Vanilla self-attention. Finally, the relative position information is provided again as a sub-component of the value matrix. In addition, considering that the effectiveness of the relative position information will decrease significantly after a certain distance, in order to improve the efficiency and performance of the model, a clip function is introduced to limit and reduce the number of required parameters. When calculating the attention between position i and position j, the distance between them is considered, and the encoding calculation is shown in equations (8) to (10).

[0104]

[0105] clip(x, k) = max(-k, min(k, x)) (10)

[0106] where p V and p K are trainable weights for the relative position encoding of values and keys, respectively. where The scalar k is the maximum relative distance.

[0107] However, as can be seen from equation (8), it requires O(L 2 d) memory due to the additional relative position encoding. There is a new method for calculating relative position encoding, called vector method, which reduces the intermediate memory requirement from O(L 2 d) to O(Ld) using the skew operation, giving up the additional relative position embedding corresponding to the value item and only focusing on the key components. The encoding calculation is shown in equation (11) and equation (12). Where the Skew procedure uses padding, reshaping and slicing to reduce the memory requirement.

[0108]

[0109] S rel = Skew(W Q P) (12)

[0110] When applied to time series data, the Transformer model requires effective position encoding to capture the ordering of time series data. The present technology applies a new time series absolute position encoding method (tAPE), which integrates time series length and input embedding dimension into absolute position encoding. An efficient relative position encoding method (eRPE) is also applied. These two position encoding methods are simple and effective, and can be easily integrated into the Transformer module to improve the generalization ability of time series. The ConvTran network model combines tAPE, eRPE and convolution-based input encoding to improve the position and data embedding of time series data, and achieves excellent results in multivariate time series classification tasks.

[0111] Absolute position encoding was originally proposed for language modeling tasks, usually using 512 or 1024 high embedding dimensions for position embedding of 512 length input, higher embedding dimensions can better reflect the similarity between different positions. When using lower embedding dimensions for position encoding, the similarity between two positions calculated by the dot product does not always decrease with the increase of the distance between the two positions, and the distance perception attribute disappears.

[0112] Although high embedding dimensions will show an ideal monotonically decreasing trend with the increase of the distance between two positions, it is not suitable for encoding time series data sets, because most time series data sets have low data dimensions, and higher embedding dimensions may reduce model throughput due to additional parameters, increasing the risk of model overfitting. On the other hand, at low embedding dimensions, the similarity between two random embedding vectors is high, which is called anisotropy phenomenon, so the embedding vector space cannot be fully utilized to distinguish two positions, and position encoding fails at low embedding dimensions.

[0113] Therefore, the algorithm requires position embedding of time series to have distance awareness while being isotropic. In order to incorporate distance awareness, the length of time series is used in equation (5). In this equation, w k refers to the frequency of the sine and cosine functions that generate the embedding vector. If not modified, as the sequence length L increases, the dot product between positions will become more and more irregular, resulting in the loss of distance perception. After introducing the length parameter into the frequency term of the sine and cosine functions in equation (5), the dot product will maintain a monotonically smooth trend.

[0114] As the embedding dimension d model increases, the vector embedding is more likely to sample from the low frequency sine function, resulting in anisotropy phenomenon. In order to alleviate this problem, d model The parameter is introduced into the frequency term of the sine and cosine functions in equation (5) at the same time. Using the new time Absolute Position Encoding (tAPE) based on time series, where The input embedding dimension d model and the length of time series L are considered, as shown in equation (13):

[0115]

[0116] Compared with Vanilla absolute position encoding, the point product representing the similarity between two positions has a more stable monotonic decreasing trend as the distance between the two positions in the time series increases, and the similarity between the embedding vectors of tAPE decreases. This is because tAPE can provide isotropic encoding in the embedding space, maintain distance-aware characteristics, and better utilize the embedding vector space to distinguish between two positions.

[0117] The input embedding is the basis of all previous relative position encoding methods, which add or multiply the position matrix with the query, key and value matrix. In this paper, an efficient relative position embedding (eRPE) model independent of the input embedding is introduced.

[0118] The calculation formula used in the eRPE model is shown in equation (14).

[0119]

[0120] In the formula, L is the sequence length, e i,j is the attention weight, w i-j is a learnable scalar representing the relative position weight between position i and position j,

[0121] In the multi-head attention mechanism, each attention module creates a trainable parameter w of size 2L-1, because the maximum distance is 2L-1. For the indices i and j of two positions, the corresponding relative scalar is w i-j+L , where the index starts from 1 instead of 0, and L 2 elements need to be indexed from the 2L-1 vectors.

[0122] First, the relative position embedding w i-j is a static parameter independent of the input, and the attention weight e i,j is dynamically determined by the representation of the input sequence. Attention adapts to the input sequence through an input-adaptive weighting strategy, enabling the model to capture complex relationships between different time points, which is exactly the feature needed when extracting high-level concepts from time series, such as seasonal components in time series. However, when the data size is limited, using attention will face greater risk of overfitting.

[0123] Second, the relative position embedding w i-j considers the relative displacement between positions i and j, rather than their values. This is similar to the translational invariance of convolution, which has been proven to enhance generalization ability, so w i-jConsider as scalar instead of vector to achieve translation invariance without increasing the number of parameters. Moreover, the value of w i-j all (i,j) can be included in the paired dot product attention function, minimizing the additional amount of calculation, and this efficient relative position encoding method is eRPE.

[0124] eRPE first applies the Softmax function to the attention matrix and then adds the relative position information to the model. The position value without the Softmax function is clearer, and the performance of the attention model is better. Compared with the existing model in which the Softmax is applied to the relative position embedding, the clearer position embedding of eRPE is more conducive to the time series classification task.

[0125] From the previous analysis, the complexity of global attention is quadratic to the length of the sequence. If the attention proposed in equation (14) is directly applied to the original time series, the calculation speed will be too slow for long time series. Therefore, the ConvTran network model is introduced. First, the length of the time series is shortened using the convolution module, and then the feature map is reduced to a size with lower calculation intensity, and then the new position encoding method is applied.

[0126] Using the convolution module on the Transformer architecture not only reduces the calculation intensity and improves the network training speed, but the convolution operation is also very suitable for capturing local features. Therefore, the convolution is used as the first module in the ConvTran model architecture, which is convenient for capturing any local information that exists in the original time series.

[0127] In the convolution module, M time convolution kernels are first applied to the input multivariate time series data, so that the ConvTran network model extracts the time information in the input sequence. Then, the output of the time convolution kernel is convolved with d model d x x M size spatial convolution kernel to capture the correlation between variables in the original time series and construct d model size input embedding. This disjoint time-space convolution first expands the number of input channels and then compresses them. One of the key reasons for this choice is that the feedforward network (FFN) in the Transformer also expands the size of the input, and then projects the expanded hidden state back to the original size to capture the spatial interaction.

[0128] Before inputting the input embedding vector into the Transformer module, the position embedding vector generated by tAPE is added to the input embedding vector, so that the model can capture the time order of the time series. The size of the tAPE embedding vector is d modelThe same as the input embedding vector. In multi-head attention, first use a linear layer to convert the Lxd model dimension input to Lxd z dimension, that is, Lxd z dimension q(query), k(key), v(value) matrix, where d z represents the dimension of the model, which is a custom parameter. These q, k, v matrices are reshaped to h x L x d z / h to represent the hth attention head. Each attention head is responsible for capturing different patterns in the time series, for example, one attention head focuses on non-noise data, another attention head focuses on seasonal components, and another attention head focuses on trends. After obtaining the q, k, v matrices, finally, the attention calculation is performed within the multi-head attention module using equation (4-14).

[0129] The feedforward network in the Transformer model is a multi-layer perceptron module composed of two linear layers and a Gaussian Error Linear Units (GELUs) activation function. GELUs introduce the idea of random regularization in the activation function, which is a probabilistic description of the input of a neuron, a combination of dropout, zoneout, and ReLU, which can improve the generalization ability of the model. Assuming the input is X and the mask is m, then m follows a Bernoulli distribution F(x) = P(X < x), where X follows a standard normal distribution, i.e. X ~ N(0, 1). The mathematical expression of GELUs is shown in equation (15).

[0130] GELU(x) = xΦ(x) = xP(X < x) (15)

[0131] The same as the Transformer basic architecture, the multi-head attention layer and the feedforward network layer in the ConvTran network also apply residual connection and layer normalization to obtain the final output of the Transformer module. Then apply max pooling and global average pooling (GAP) to the output of the ELU activation function in the last layer to obtain a more translation-invariant model. Finally, apply the Softmax function to obtain the classification prediction result.

[0132] Figure 7 The grid-connected photovoltaic array fault simulation test platform is shown, and the multivariate time series of the output voltage and current of each branch of the off-grid photovoltaic array in the training set sample under normal operation is shown in Table 1, the training set sample when open circuit fault occurs in A road is shown in Table 2, the training set sample when short circuit fault occurs in B road is shown in Table 3, and the training set sample when shadow shading occurs in B road is shown in Table 4.

[0133] Table 1. Example of multivariate time series training set samples of off-grid photovoltaic array output in normal operation

[0134]

[0135]

[0136] Table 2. Example of multivariate time series training set samples of off-grid photovoltaic array output in open-circuit fault

[0137]

[0138] Table 3. Example of multivariate time series training set samples of off-grid photovoltaic array output in short-circuit fault

[0139]

[0140] Table 4. Example of multivariate time series training set samples of off-grid photovoltaic array output in shadowing fault

[0141]

[0142] In this embodiment, 1371 groups of multivariate time series sample data were collected in the off-grid photovoltaic array fault simulation test, and the 1371 groups of data were randomly divided into 1034 groups of training set and 337 groups of test set. The label corresponding to the grid-connected photovoltaic array operating state of each sample was marked. To facilitate the ConvTran network to classify different operating conditions of the grid-connected photovoltaic array, the labels of the sample set and the corresponding photovoltaic array operating states are shown in Table 5.

[0143] Table 5. Correspondence table between sample set labels and actual photovoltaic array operating states

[0144]

[0145] The training set was input into the ConvTran photovoltaic array fault diagnosis model for training. The training process of the training set refers to the ConvTran network model framework of Figure 5 Since the input of the ConvTran network model is four-channel multivariate time series data, i.e., the branch voltage and branch current of the two branches A and B of the grid-connected photovoltaic array, dx in Figure 5 is equal to 4, and L is the length of the multivariate time series.

[0146] The parameters of the convolution module in the ConvTran network model are as follows: the number of time and space convolution kernels is set to 64, the length of the time convolution kernel is set to 8, and the width of the space convolution kernel is equal to the dimension of the input. The parameters of the Transformer module in the ConvTran network model are as follows: 8 attention modules are used in the multi-head attention mechanism to capture the attention changes in the input sequence.Figure 5 The dimensions dmodel and dz of the Transformer encoding are both set to 64. The feed-forward network of the Transformer expands the size of the input by a factor of 4, and then projects the 4 times wider hidden states back to the original size. In addition to that, Adam optimization and early stopping based on validation loss are used.

[0147] After the multivariate time series data of the training set is input into the ConvTran network model, the time convolution kernel in the convolution module is first input to extract the time information in the input sequence, and then the output of the time convolution kernel is convolved with the spatial convolution kernel to capture the correlation between variables in the original time series and construct an input embedding of size d model After that, the position embedding vector generated by tAPE is superimposed on the input embedding vector to input the Transformer module, so that the ConvTran network can capture the time sequence of the time series. In the multi-head attention layer, the input Lxd model is first converted into 3 matrices of size Lxd z , namely q (query), k (key), and v (value) matrices, and 8 attention heads perform attention calculation respectively. The feed-forward network in the Transformer module is a multi-layer perceptron module composed of two linear layers and a GELU activation function, which improves the generalization ability of the network model. Each layer of multi-head attention layer and feed-forward network layer applies residual connection and layer normalization to obtain the final output of the Transformer module. Then, max-pooling and global average pooling (GAP) are applied to the output of the ELU activation function in the last layer to obtain a model with more translation invariance. Finally, the Softmax function is applied to obtain the classification prediction result. Compare it with the label input of the training set to update the parameters of the network model and train the model.

[0148] The multivariate time series of the output voltage and current of each branch of the off-grid photovoltaic array in the test set sample under normal operation is shown in Table 6, the test set sample when the open circuit fault occurs in the A path is shown in Table 7, the test set sample when the short circuit fault occurs in the B path is shown in Table 8, and the test set sample when the shadow shielding occurs in the B path is shown in Table 9.

[0149] Table 6 Example of multivariate time series test set sample of off-grid photovoltaic array output under normal operation

[0150]

[0151] Table 7 Example of multivariate time series test set sample of off-grid photovoltaic array output under open circuit fault

[0152]

[0153] Table 8. Example of multivariate time series test set samples of off-grid photovoltaic array output in short-circuit fault

[0154]

[0155]

[0156] Table 9. Example of multivariate time series test set samples of off-grid photovoltaic array output in shadow-shading fault

[0157]

[0158] The test set is input into the trained ConvTran model suitable for photovoltaic array fault diagnosis, and finally the off-grid photovoltaic array fault classification result is output.

[0159] In order to compare and analyze the effectiveness and accuracy of the ConvTran model used in this paper, the same 1034 training set and 337 test set are used as data samples, and the improved algorithms FCN, MC-DCNN, ResNet and MLSTM-FCN of CNN are used as comparison models of ConvTran network, and the Gated Transformer Networks (GTN) model based only on the Transformer architecture is additionally supplemented as the comparison model of ConvTran network.

[0160] The 337 test set samples are composed of 189 normal operation samples, 50 open-circuit fault samples, 49 short-circuit fault samples and 49 shadow-shading fault samples. The confusion matrix of the photovoltaic array fault recognition based on ConvTran is shown in Figure 8 As shown in the figure, only one sample is misclassified in the 337 test set samples, and the total accuracy of classification is as high as 99.7%.

[0161] As shown in the confusion matrix of Figure 8 , the photovoltaic array fault diagnosis model based on ConvTran does not classify the abnormal state of the photovoltaic array as the normal operation state, and the accuracy of abnormal diagnosis is 100%, which fully plays the role of fault warning, and the recognition accuracy of open-circuit fault, short-circuit fault and shadow-shading fault is also 100%, realizing the accurate recognition of faults.

[0162] The confusion matrix of the four comparison model algorithms FCN, MC-DCNN, ResNet and MLSTM-FCN based only on CNN and the comparison model algorithm GTN based only on the Transformer architecture mentioned in the last section are shown in Figure 9

[0163] As shown in the confusion matrix of Figure 9 ​The confusion matrix of each algorithm shows that the other improved algorithms based on CNN have misdiagnosed the abnormal state of the photovoltaic array, and cannot fully achieve the warning function. Among them, the FCN algorithm and the ResNet algorithm cannot distinguish between normal operation state and shadow shielding fault state, the MC-DCNN algorithm and the MLSTM-FCN algorithm are prone to confusion between short circuit and shadow shielding state, and the GTN algorithm has a certain degree of confusion for normal operation state, short circuit state and shadow shielding state, which is not conducive to judging the severity of the fault and affects the subsequent response measures.

[0164] The accuracy of the ConvTran network model and the above method in off-grid photovoltaic array fault diagnosis is compared, and the results are shown in Table 10, Figure 10 and Figure 11 shown.

[0165] Table 10 Accuracy of off-grid photovoltaic array fault diagnosis under different algorithms

[0166]

[0167]

[0168] As can be seen from Table 10, compared with other improved algorithms based on CNN such as FCN, MC-DCNN, ResNet, MLSTM-FCN and improved algorithm GTN based on Transformer, the recognition ability of ConvTran algorithm for various faults of photovoltaic array is more balanced, and the accuracy of fault diagnosis of off-grid photovoltaic array is the highest, especially in the fault of partial shadow shielding, the accuracy of diagnosis is significantly higher than that of other algorithms.

[0169] This shows that as an improved algorithm combining convolution and Transformer, the ConvTran network model improves the position and data embedding of time series data in the Transformer architecture, has better classification effect in the multi-element time series classification task, can distinguish the slight difference between short circuit fault and partial shadow shielding, is more suitable for fault type classification of multi-element time series, and has more excellent applicability and accuracy in off-grid photovoltaic array fault diagnosis.

[0170] The above only describes the embodiments of the present application, and does not limit the patent scope of the present application; any equivalent method or structure made by using the content of the present application is included in the patent protection scope of the present application.

Claims

1. A ConvTran-based off-grid photovoltaic array fault diagnosis method, characterized in that: The steps include: S1: Build the Transformer model, which includes four parts: input module, encoder module, decoder module, and output module; S2: Use convolutional modules on the Transformer architecture to shorten the length of the time series and capture local information in the original time series; S3: Before the input embedding vector is fed into the Transformer module, the position embedding vector generated by tAPE is added to the input embedding vector so that the model can capture the temporal order of the time series, where tAPE is the time series absolute position encoding method; S4: After obtaining the final output of the Transformer module, global average pooling and fully connected layers are applied to obtain a model with greater translation invariance. Finally, the Softmax function is applied to obtain the classification prediction result. S5: Collect the multivariate time series data of the four channels of the photovoltaic array, input it into the ConvTran network model, and perform fault diagnosis on the photovoltaic array; the steps of training the model include: after the multivariate time series data of the training set is input into the ConvTran network model, first input the time convolution kernel in the convolution module to extract the time information in the input sequence, then convolve the output of the time convolution kernel with the spatial convolution kernel to capture the correlation between the variables in the original time series, and construct an input embedding of the size of dmodel, then superimpose the position embedding vector generated by tAPE on the input embedding vector and input it into the Transformer module, so that the ConvTran network can capture the time order of the time series. In the multi-head attention layer, first use the linear layer to The input of the L×dmodel is converted into 3 matrices of size L×dz, where L is the length of the multivariate time series. The 8 attention heads perform attention calculations respectively. The feedforward network in the Transformer module is a multi-layer perceptron module, consisting of two linear layers and a GELU activation function. Each multi-head attention layer and feedforward network layer applies residual connections and layer normalization to obtain the final output of the Transformer module. Then, maximum pooling and global average pooling are applied to the output of the ELU activation function of the last layer to obtain a model with more translation invariance. Finally, the Softmax function is applied to obtain the classification prediction result, which is compared with the label input of the training set, and the parameters of the network model are updated for model training.

2. The off-grid photovoltaic array fault diagnosis method based on ConvTran according to claim 1, characterized in that: The input module in step S1 receives sequence data as input. The input sequence is converted into a feature vector through an embedding algorithm. After the embedding operation is completed on the input sequence, the feature vector is positionally encoded. The position encoding has the same dimension as the input embedding.

3. The off-grid photovoltaic array fault diagnosis method based on ConvTran according to claim 2, characterized in that: The encoder module in step S1 is composed of multiple encoder stacks with the same structure. Each encoder consists of two sub-layers: a self-attention layer and a fully connected feedforward network. Each sub-layer adopts residual connection and then performs layer normalization.

4. The off-grid photovoltaic array fault diagnosis method based on ConvTran according to claim 3, characterized in that: The decoder module in step S1 consists of multiple decoder stacks with the same structure. In addition to the self-attention layer and the feedforward network, the decoder has a third sub-layer that performs a multi-head attention mechanism on the output of the encoder stack.

5. The off-grid photovoltaic array fault diagnosis method based on ConvTran according to claim 4, characterized in that: The output module in step S1 consists of a linear layer and a softmax layer. The linear layer is a simple fully connected neural network that maps the output vector of the decoding module to a longer vector, namely the logits vector. The softmax layer converts the attention score of each sequence segment into a probability distribution between 0 and 1, and selects the result corresponding to the highest probability as the output of this time step.

6. The off-grid photovoltaic array fault diagnosis method based on ConvTran according to claim 4, characterized in that: The multi-head attention mechanism uses h different learned linear transformations to linearly map queries, keys, and values ​​while maintaining the overall parameter scale unchanged, then performs scaled dot product attention calculations simultaneously, and finally concatenates the generated output values ​​and performs linear transformations to produce the final result.

7. The off-grid photovoltaic array fault diagnosis method based on ConvTran according to claim 1, characterized in that: In step S5, the multivariate time series data of the four channels of the photovoltaic array are collected, specifically, one time point is collected every 1s, and a sample is packaged into a group every 30s. A total of 1371 groups of multivariate time series sample data are collected, and the 1371 groups of data are randomly divided into 1034 training sets and 337 test sets.

8. The off-grid photovoltaic array fault diagnosis method based on ConvTran according to claim 7, characterized in that: The multivariate time series data of the four channels of the photovoltaic array are the voltages and currents of the two branches A and B on the DC output side.

9. The off-grid photovoltaic array fault diagnosis method based on ConvTran according to claim 8, characterized in that: The parameters of the convolution module in the ConvTran network model are as follows: the number of temporal and spatial convolution kernels is set to 64, and the length of the temporal convolution kernel is set to 8, and the width of the spatial convolution kernel is equal to the dimension of the input; the parameters of the Transformer module in the ConvTran network model are as follows: 8 attention modules are used in the multi-head attention mechanism to capture the attention changes in the input sequence, the dimensions dmodel and dz of the Transformer encoding are both set to 64, and the Transformer feedforward network expands the size of the input by 4 times, and then projects the 4 times wider hidden state back to the original size.

Images