CVT operating status evaluation method and system based on improved decoding layer model

Abstract

The present invention is a CVT operating state evaluation method and system based on an improved decoding layer model, and relates to the technical field of online monitoring of electric power metering, wherein the method comprises acquiring CVT data for hysteresis operation; constructing a CVT operating state evaluation model, comprising an embedding layer, an improved transformer decoding layer, a CloFormer layer and a distribution estimation layer; the improved transformer decoding layer comprises a first normalization layer, an attention layer, a second normalization layer and a feedforward neural layer; an embedded variable is generated through the embedding layer, and is input into the improved transformer decoding layer after normalization; a CloFormer layer is used for feature extraction; the distribution estimation layer is a Gaussian distribution model, and an optimal parameter group is determined by using an EM algorithm and a BIC criterion, and a model training is performed to obtain a final CVT operating state evaluation model; and real-time CVT data is input into the final model to obtain the CVT operating state.

Description

Technical Field

[0001] The present invention relates to the technical field of online monitoring of electric power metering, and mainly to a CVT operating state evaluation method and system based on an improved decoding layer model. Background Art

[0002] As an important part of the electric energy metering device, the accuracy and reliability of the transformer's metering performance is directly related to economic and social benefits, and plays an important role in ensuring the healthy and stable operation of the power grid. Capacitor Voltage Transformers (CVT) are divided by series capacitors, and then reduced and isolated by electromagnetic transformers. As an instrument for converting voltage, capacitor voltage transformers can also couple carrier frequencies to transmission lines for long-distance communication, selective line high-frequency protection, remote control and other functions. Compared with conventional electromagnetic voltage transformers, capacitor voltage transformers have many advantages in terms of economy and safety in addition to high impulse insulation strength, simple manufacturing, small size, and light weight. In the actual operation of CVT, the transformer error will fail within its working life due to the influence of the acquisition principle and harsh environment. Therefore, it is necessary to quickly identify when its operating state is abnormal so that the relevant operation and maintenance personnel can arrange inspection and maintenance work. If the transformer abnormality is not discovered in time, it will affect the operation of the power grid.

[0003] For example, the Chinese invention patent with the publication number of "CN114062993A" discloses "A CVT error state prediction method based on a time convolutional network", which specifically discloses "A CVT error state prediction method based on a time convolutional network, the method comprising the following steps: calculating the CVT characteristic parameter value and generating a time series; using the Hampel filter to process the abnormal value; using the ADF test on the processed sequence to determine its stationarity, if the time series is non-stationary, it is converted into a stationary time series by differential exponential smoothing; using the training sample to train the time convolutional network; in the test phase, inputting the test sample to the saved network model, outputting the CVT error prediction value, comparing it with the true value, and checking the network The effectiveness of the model; applying the well-trained temporal convolutional network model to the state prediction of CVT can determine whether there is a possibility of out-of-tolerance in the future error of CVT. However, the method using the Hampel filter to process outliers may not be sufficient to deal with all types of anomalies, which may cause the model to fail to detect abnormal conditions of the transformer in time when facing complex or unknown abnormal patterns; in addition, the generalization ability of the temporal convolutional network model used in the method is highly dependent on the quality and quantity of training samples. If the training samples are insufficient or not representative, the generalization ability of the model will be affected; at the same time, the performance of the temporal convolutional network used in the method may be affected by the length of the input sequence. Too long a sequence may lead to excessive computational complexity and affect the efficiency of the model. Summary of the invention

[0004] In order to solve the above-mentioned problems existing in the prior art, the present application provides a CVT operating status evaluation method and system based on an improved decoding layer model.

[0005] The technical solution of this application is as follows:

[0006] On the one hand, the present invention proposes a CVT operating state evaluation method based on an improved decoding layer model, the method comprising:

[0007] Acquire historical CVT data within a preset time period and perform data cleaning, wherein the historical CVT data includes secondary side voltage, secondary load and temperature; perform a hysteresis operation on the cleaned historical CVT data to obtain covariates corresponding to each historical CVT data, and concatenate the historical CVT data with the corresponding covariates to obtain a connection variable;

[0008] Construct a CVT operation status evaluation model, including the embedding layer, An improved transformer decoding layer, a CloFormer layer and a distribution estimation layer, wherein: the improved transformer decoding layer includes a first RMSNorm normalization layer, an attention layer, a second RMSNorm normalization layer and a feedforward neural layer; the feedforward neural layer includes three linear layers and a SwiGLU activation function layer;

[0009] The connection variables are embedded by using the embedding layer to generate embedded variables; the normalized embedded variables are input into the first improved transformer decoding layer, and the output of the previous improved transformer decoding layer is used as the input of the next improved transformer decoding layer, and the The final decoding variable is obtained; wherein the attention layer obtains the normalized embedded variable to obtain the output variable of the attention layer; the output variable of the attention layer is Input the second RMSNorm normalization layer to get the normalized variable of the attention layer ;

[0010] The feedforward neural layer includes three linear layers and a SwiGLU activation function layer, wherein the linear layer includes a first linear layer, a second linear layer and a third linear layer; the normalized variables of the attention layer are input into the feedforward neural layer to obtain intermediate decoding variables, which are expressed as follows:

[0011] ;

[0012] ;

[0013] In the formula, represents the intermediate decoding variable; represents a feed-forward neural network; represents the first activation function; represents the third activation function; Indicates that the parameter is The second activation function of represents the parameters of the first linear layer; represents the parameters of the second linear layer; represents the parameters of the third linear layer; Indicates adjustable parameters;

[0014] The intermediate decoding variable and the output variable of the attention layer are residually fused to obtain the decoding variable , expressed as:

[0015] ;

[0016] in, represents residual fusion;

[0017] The decoded variable is used as the input of the next transformer decoding layer, iteratively times, generating the final decoded variable ;

[0018] The CloFormer layer is used to extract the features of the final decoded variable and generate the feature vector of the final decoded variable;

[0019] The characteristic vector of the final decoded variable is used as the input of the distribution estimation layer, and the distribution estimation layer is specifically a mixed Gaussian distribution GMM model, wherein the optimal parameter group of the mixed Gaussian distribution GMM model is determined by using the expectation maximization EM algorithm and the Bayesian information BIC criterion; the CVT operation status evaluation model is trained using historical CVT data to obtain a trained CVT operation status evaluation model;

[0020] The current CVT data is obtained, and the current CVT data is input into a trained CVT operation status evaluation model to obtain the current CVT operation status.

[0021] Preferably, the steps of performing a hysteresis operation on the cleaned historical CVT data to obtain covariates corresponding to each historical CVT data, and splicing the historical CVT data with the corresponding covariates are specifically as follows:

[0022] Get historical CVT data after cleaning, including voltage , Secondary load and temperature ,in Indicates the time, represents the matrix transpose operator, Indicates Historical CVT data, express Moment Historical CVT data; time-lag operation of voltage, secondary load and temperature;

[0023] Preset an ordered set of hysteresis indices , Indicates the total number of time points, defined The hysteresis operation at time is expressed as:

[0024] ;

[0025] ;

[0026] ;

[0027] In the formula, express Historical CVT data at all times; express moment covariate data; Indicates covariates of historical CVT data; Represents a function mapping; Indicates the covariate type serial number; represents the set of real numbers; express Moment covariates of historical CVT data; Indicates The index value of the historical CVT data; get the voltage covariate , secondary load covariate and the temperature covariate ;

[0028] The historical CVT data are spliced ​​with the corresponding covariates and expressed as follows:

[0029] ;

[0030] ;

[0031] ;

[0032] In the formula, represents the voltage connection variable; represents the secondary load connection variable; represents the temperature connection variable; express Voltage connection variable at the moment; express Secondary load connection variable at moment; express The temperature at the moment is connected to the variable; represents the connection function; Indicates a connection symbol.

[0033] Preferably, the embedding layer is used to embed the connection variable to generate an embedded variable, and the embedded variable is normalized, which is expressed as:

[0034] ;

[0035] In the formula, represents a collection of embedded variables; represents the embedded function; represents the voltage embedded variable; represents the secondary load embedded variable; represents the temperature embedded variable;

[0036] The embedded variables are normalized using the first RMSNorm normalization layer. The first RMSNorm normalization layer specifically reconstructs the output normalization of the transformer decoding layer into the input normalization, and replaces the layerNorm normalization function in the transformer decoding layer with the RMSNorm normalization function, which is expressed as:

[0037] , ;

[0038] , ;

[0039] , ;

[0040] In the formula, Indicates a preset positive number; represents the normalized voltage embedded variable; represents the normalized secondary loading embedded variable; represents the normalized temperature embedded variable; Indicates voltage embedded variables; Indicates A secondary load embedding variable; Indicates temperature embedded variables; Indicates the number of voltage covariate data; Indicates The index value of the voltage covariate data; Indicates the number of secondary load covariate data; Indicates The index value of the secondary loading covariate data; Indicates the number of temperature data covariate data; Indicates The index value of the temperature data.

[0041] Preferably, the attention layer obtains the normalized embedded variables, segments the normalized embedded variables and identifies outliers to obtain the output variables of the attention layer, specifically:

[0042] The attention layer reconstructs the attention mechanism of the transformer and uses the SwiGLU activation function to replace the ReLU activation function in the attention mechanism of the transformer;

[0043] The normalized embedded variable is segmented according to the preset segment number and segment length to obtain the segmentation variable, which is expressed as:

[0044] ;

[0045] ;

[0046] ;

[0047] In the formula, Segmental variable representing the normalized voltage embedded variable; Segmental variable representing the normalized secondary loading embedded variable; Segmental variable representing the normalized embedded temperature variable; Indicates The segment length is The normalized voltage is embedded in the segmented variable; Indicates The segment length is The normalized quadratic loadings of the segmented variables are embedded in the variables; Indicates The segment length is The normalized quadratic loadings of the segmented variables are embedded in the variables; express Length; express Length; express Length; express The number of segments; express The number of segments; express The number of segments; express The segment length; express The segment length; express The segment length is ; Indicates The index value of the segmentation variable; Indicates The index value of the segmentation variable; Indicates The index value of the segmentation variable;

[0048] The specific method of outlier identification is to calculate the quartiles and interquartile ranges of the segmented variables, which can be expressed as follows:

[0049] ;

[0050] ;

[0051] ;

[0052] ;

[0053] In the formula, Indicates The interquartile range of the segmental variables; Indicates The segmentation variable value at one quarter of the segmentation variables; Indicates The segmentation variable values ​​that are at three-quarters of the segmentation variables; Indicates The number of segments for a segmentation variable; Indicates The index value of the segmentation variable;

[0054] The segmentation variable is input into the linear layer built into the attention layer, which is expressed as:

[0055] , ;

[0056] ;

[0057] ;

[0058] In the formula, Indicates The query value of a segmentation variable; Indicates The key of each segmentation variable; Indicates The value of the segmentation variable; Represents the query value linear transformation layer; represents the key linear transformation layer; Representation value linear transformation layer;

[0059] Calculate the output of the attention layer, expressed as:

[0060] , ;

[0061] ;

[0062] ;

[0063] ;

[0064] ;

[0065] In the formula, Represents the intermediate output variable of the attention layer; Represents the output variable of the attention layer; represents a set of embedded variables; Represents the query value of the segmentation variable; The key representing the segmentation variable; Represents the value of the segmentation variable; Indicates the number of segmentation variables; represents residual fusion; represents the built-in normalization function of the attention layer; Indicates the preset projection dimension; Represents the attention mechanism.

[0066] Preferably, the CloFormer layer is used to extract the feature vector of the final decoded variable, specifically:

[0067] The CloFormer layer includes a convolution stem Convolution Stem, CloBlock and a convolution feedforward network ConvFFN. The CloFormer layer obtains the final decoding variable and performs a convolution operation using the Convolution Stem, which is expressed as follows:

[0068] ;

[0069] In the formula, Represents the output of Convolution Stem; Indicates the preset convolution kernel size; Indicates the preset convolution stride; Indicates the preset convolution padding; Indicates the preset number of output channels; Represents the convolution operation;

[0070] The CloBlock includes a local branch and a global branch. The local branch is locally enhanced using the AttnConv module, which is expressed as:

[0071] ;

[0072] ;

[0073] ⊙ ;

[0074] In the formula, represents context-aware local enhancement; represents context-aware weight; represents depthwise separable convolution; express The query vector of express Keys; express The value of represents a fully connected layer; represents the fourth activation function; represents the second activation function; Indicates the preset dimension; ⊙ indicates the XOR operation;

[0075] The global branch uses the attention mechanism for global enhancement, which is expressed as:

[0076] ;

[0077] In the formula, Indicates global enhancement; represents the attention mechanism; represents the pooling layer;

[0078] Combine local enhancement and global enhancement and input them into the convolutional feedforward network , expressed as:

[0079] ;

[0080] In the formula, represents the output of the CloFormer layer, which is the feature vector of the final decoded variable; represents the connection function; Represents a convolutional feed-forward network.

[0081] Preferably, the optimal parameter group of the GMM model is determined using the expectation maximization EM algorithm and the BIC criterion; and the Gaussian distribution model with the optimal parameter group is trained using the historical CVT data set to obtain the final CVT operating state evaluation model, specifically:

[0082] The probability distribution of the feature vector of the final decoded variable under the Gaussian distribution model is expressed as follows:

[0083] ;

[0084] ;

[0085] ;

[0086] In the formula, The average value of the feature vector representing the final decoded variable; represents the probability density function; The variance of the eigenvector representing the final decoded variable; represents pi; Indicates The parameters of the Gaussian distribution sub-model, where Indicates The expected value of the Gaussian distribution sub-model; Indicates The variance of the Gaussian distribution sub-model; Indicates The weight coefficients of the Gaussian distribution sub-models; represents probability; Indicates The index value of the Gaussian distribution sub-model; Indicates the number of Gaussian distribution sub-models; represents the exponential function;

[0087] The Gaussian distribution model parameters are optimized using the expectation maximization EM algorithm and the Bayesian information BIC criterion. The expectation maximization EM algorithm includes an E step and an M step, and the E step and the M step are iterated until a preset optimization termination condition is reached; the optimization termination condition is expressed as follows:

[0088] ;

[0089] ;

[0090] ;

[0091] In the formula, represents the log-likelihood function; Indicates The log-likelihood value after iterations; Indicates the maximum number of iterations; Indicates the total number of segments; Indicates The index value of the iteration; represents the logarithmic function;

[0092] The E step specifically calculates the probability that each decoded variable value comes from the Gaussian distribution submodel , expressed as:

[0093] , , ;

[0094] In the formula, Indicates The feature vector of the final decoded variable; Indicates The index value of the feature vector of the final decoded variable; The number of feature vectors representing the final decoded variables;

[0095] The M step is specifically to calculate the parameters of the new iterative Gaussian distribution sub-model, which is expressed as follows:

[0096] ;

[0097] ;

[0098] ;

[0099] The BIC criterion is used to determine the optimal parameter group of the Gaussian distribution model, and the optimal Gaussian distribution model is obtained, which is expressed as follows:

[0100] ;

[0101] ;

[0102] In the formula, represents the maximum likelihood estimate of the Gaussian distribution model; Represents the number of parameters of the Gaussian distribution sub-model; Indicates the function for obtaining the maximum value of the log-likelihood function; Represents the Bayesian Information Criterion value.

[0103] On the other hand, the present invention also proposes a CVT operating state evaluation system based on an improved decoding layer model, the system comprising a data acquisition module, a state evaluation module and a state output module, specifically:

[0104] The data acquisition module is used to acquire historical CVT data within a preset time period and perform data cleaning, wherein the historical CVT data includes secondary side voltage, secondary load and temperature; perform a hysteresis operation on the cleaned historical CVT data to obtain covariates corresponding to each historical CVT data, and splice the historical CVT data with the corresponding covariates to obtain connection variables; transmit the spliced ​​CVT data and its corresponding covariates to the state evaluation module;

[0105] The state assessment module has a built-in CVT operation state assessment model, including an embedding layer, Improved transformer decoding layer, CloFormer layer and distribution estimation layer, where:

[0106] The embedding layer is used to receive the spliced ​​CVT data and its corresponding covariates to generate embedded variables;

[0107] The improved transformer decoding layer includes a first RMSNorm normalization layer, an attention layer, a second RMSNorm normalization layer and a feedforward neural layer; the feedforward neural layer includes three linear layers and a SwiGLU activation function layer; the normalized embedded variables are input into the first improved transformer decoding layer, the output of the previous improved transformer decoding layer is used as the input of the next improved transformer decoding layer, and the iteration is repeated. The final decoding variable is obtained; wherein the attention layer obtains the normalized embedded variable to obtain the output variable of the attention layer; the output variable of the attention layer is Input the second RMSNorm normalization layer to get the normalized variable of the attention layer ;

[0108] The feedforward neural layer includes three linear layers and a SwiGLU activation function layer, wherein the linear layer includes a first linear layer, a second linear layer and a third linear layer; the normalized variables of the attention layer are input into the feedforward neural layer to obtain intermediate decoding variables, which are expressed as follows:

[0109] ;

[0110] ;

[0111] In the formula, represents the intermediate decoding variable; represents a feed-forward neural network; represents the first activation function; represents the third activation function; Indicates that the parameter is The second activation function of represents the parameters of the first linear layer; represents the parameters of the second linear layer; represents the parameters of the third linear layer; Indicates adjustable parameters;

[0112] The intermediate decoding variable and the output variable of the attention layer are residually fused to obtain the decoding variable , expressed as:

[0113] ;

[0114] in, represents residual fusion;

[0115] The decoded variable is used as the input of the next transformer decoding layer, iteratively times, generating the final decoded variable ;

[0116] The CloFormer layer is used to capture the contextual features of the final decoded variable and generate a feature vector of the final decoded variable;

[0117] The distribution estimation layer is a mixed Gaussian distribution GMM model, wherein the optimal parameter group of the mixed Gaussian distribution GMM model is determined by using the expectation maximization EM algorithm and the Bayesian information BIC criterion; the CVT operation status evaluation model is trained using historical CVT data to obtain a trained CVT operation status evaluation model; the current CVT data is obtained, and the current CVT data is input into the trained CVT operation status evaluation model to obtain the current CVT operation status;

[0118] The status output module is used to output the operating status of the CVT.

[0119] On the other hand, the present invention also proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for evaluating the CVT operating status based on the improved decoding layer model as described in any embodiment of the present invention is implemented.

[0120] On the other hand, the present invention further proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the CVT operating status assessment method based on the improved decoding layer model as described in any embodiment of the present invention.

[0121] Compared with the prior art, the present invention has the following beneficial effects:

[0122] 1) The present invention provides a CVT operating state evaluation method and system based on an improved decoding layer model, which obtains covariates corresponding to each historical CVT data by performing a hysteresis operation; it can better capture the dynamic changes and trends in the data, enhance the temporal nature and correlation between the data, and improve the accuracy of the model's prediction of the CVT operating state;

[0123] 2) The present invention provides a CVT operating status evaluation method and system based on an improved decoding layer model. The Transformer decoding model is improved by introducing an embedding layer, an improved transformer decoding layer, a CloFormer layer, and a distribution estimation layer. The prediction ability of the model is improved, and it can better adapt to different scenarios and data sets, and the scalability and heterogeneity of the model are enhanced. The introduction of the CloFormer layer further enhances the model's ability to process time series data.

[0124] 3) The present invention provides a CVT operating status evaluation method and system based on an improved decoding layer model, constructs a distribution estimation layer based on a Gaussian distribution model, uses the expectation maximization EM algorithm for parameter optimization, and obtains the optimal parameter group based on the Bayesian information BIC criterion, thereby improving the estimation accuracy of the model parameters and enhancing the stability and reliability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0125] Figure 1 is a method flow chart of an embodiment of the present invention;

[0126] Figure 2 It is an overall architecture diagram of the CVT operating status evaluation model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0127] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0128] The present invention provides the following technical solution: a CVT operating status evaluation method and system based on an improved decoding layer model.

[0129] Example 1

[0130] See also Figure 1 This embodiment provides a CVT operating state evaluation method based on an improved decoding layer model, and the specific steps include:

[0131] S1. Acquire historical CVT data within a preset time period and perform data cleaning, wherein the historical CVT data includes secondary side voltage, secondary load and temperature; perform a hysteresis operation on the cleaned historical CVT data to obtain covariates corresponding to each historical CVT data, and splice the historical CVT data with the corresponding covariates to obtain a connection variable;

[0132] S11. Obtain historical CVT data after cleaning, including voltage , Secondary load and temperature ,in Indicates the time, represents the matrix transpose operator, Indicates Historical CVT data, express Moment Historical CVT data; time-lag operation of voltage, secondary load and temperature;

[0133] Preset an ordered set of hysteresis indices , Indicates the total number of time points, defined The hysteresis operation at time is expressed as:

[0134] ;

[0135] ;

[0136] ;

[0137] In the formula, express Historical CVT data at all times; express moment covariate data; Indicates covariates of historical CVT data; Represents a function mapping; Indicates the covariate type serial number; represents the set of real numbers; express Moment covariates of historical CVT data; Indicates The index value of the historical CVT data; get the voltage covariate , secondary load covariate and the temperature covariate ;

[0138] S12. The historical CVT data is concatenated with the corresponding covariates, and the formula is:

[0139] ;

[0140] ;

[0141] ;

[0142] In the formula, represents the voltage connection variable; represents the secondary load connection variable; represents the temperature connection variable; express Voltage connection variable at the moment; express Secondary load connection variable at moment; express The temperature at the moment is connected to the variable; represents the connection function; Indicates a connection symbol;

[0143] S2. Construct a CVT operation status evaluation model. For details, see Figure 2 The CVT operation status evaluation model includes an embedding layer, An improved transformer decoding layer, CloFormer layer, and distribution estimation layer;

[0144] S21, embedding the connection variable using the embedding layer to generate an embedded variable, and normalizing the embedded variable, which is expressed as:

[0145] ;

[0146] In the formula, represents a set of embedded variables; represents the embedded function; represents the voltage embedded variable; represents the secondary load embedded variable; represents the temperature embedded variable;

[0147] S22, the improved transformer decoding layer includes a first RMSNorm normalization layer, an attention layer, a second RMSNorm normalization layer and a feedforward neural layer;

[0148] S221, using the first RMSNorm normalization layer to normalize the embedded variable, the first RMSNorm normalization layer specifically reconstructs the output normalization of the transformer decoding layer into the input normalization, and uses the RMSNorm normalization function to replace the layerNorm normalization function in the transformer decoding layer, which is expressed as:

[0149] , ;

[0150] , ;

[0151] , ;

[0152] In the formula, Indicates a preset positive number; represents the normalized voltage embedded variable; represents the normalized secondary loading embedded variable; represents the normalized temperature embedded variable; Indicates voltage embedded variables; Indicates A secondary load embedding variable; Indicates temperature embedded variables; Indicates the number of voltage covariate data; Indicates The index value of the voltage covariate data; Indicates the number of secondary load covariate data; Indicates The index value of the secondary loading covariate data; Indicates the number of temperature data covariate data; Indicates The index value of the temperature data;

[0153] S222, the attention layer obtains the normalized embedded variables, segments and identifies outliers on the normalized embedded variables, and obtains the output variables of the attention layer; the attention layer reconstructs the attention mechanism of the transformer, and changes the focus from all variables at the same time point to each single variable, specifically, all variables at the same time point are no longer regarded as a whole, but each single variable in the multivariate sequence is regarded as an independent attention mechanism object, and the ReLU activation function in the attention mechanism of the transformer is replaced by the SwiGLU activation function;

[0154] The normalized embedded variable is segmented according to the preset segment number and segment length to obtain the segmentation variable, which is expressed as:

[0155] ;

[0156] ;

[0157] ;

[0158] In the formula, Segmental variable representing the normalized voltage embedded variable; Segmental variable representing the normalized secondary loading embedded variable; Segmental variable representing the normalized embedded temperature variable; Indicates The segment length is The normalized voltage is embedded in the segmented variable; Indicates The segment length is The normalized quadratic loadings of the segmented variables are embedded in the variables; Indicates The segment length is The normalized quadratic loadings of the segmented variables are embedded in the variables; express Length; express Length; express Length; express The number of segments; express The number of segments; express The number of segments; express The segment length; express The segment length; express The segment length is ; Indicates The index value of the segmentation variable; Indicates The index value of the segmentation variable; Indicates The index value of the segmentation variable;

[0159] The specific method of outlier identification is to calculate the quartiles and interquartile ranges of the segmented variables, which can be expressed as follows:

[0160] ;

[0161] ;

[0162] ;

[0163] ;

[0164] In the formula, Indicates The interquartile range of the segmental variables; Indicates The segmentation variable value at one quarter of the segmentation variables; Indicates The segmentation variable values ​​that are at three-quarters of the segmentation variables; Indicates The number of segments for a segmentation variable; Indicates The index value of the segmentation variable;

[0165] The segmentation variable is input into the linear layer built into the attention layer, which is expressed as:

[0166] , ;

[0167] ;

[0168] ;

[0169] In the formula, Indicates The query value of a segmentation variable; Indicates The key of the segmentation variable; Indicates The value of the segmentation variable; Represents the query value linear transformation layer; represents the key linear transformation layer; Representation value linear transformation layer;

[0170] Calculate the output of the attention layer, expressed as:

[0171] , ;

[0172] ;

[0173] ;

[0174] ;

[0175] ;

[0176] In the formula, Represents the intermediate output variable of the attention layer; Represents the output variable of the attention layer; represents a collection of embedded variables; Represents the query value of the segmentation variable; The key representing the segmentation variable; Represents the value of the segmentation variable; Indicates the number of segmentation variables; represents residual fusion; represents the built-in normalization function of the attention layer; Indicates the preset projection dimension; represents the attention mechanism;

[0177] S223, the feedforward neural layer includes three linear layers and a SwiGLU activation function layer, wherein the linear layer includes a first linear layer, a second linear layer and a third linear layer; the normalized variable of the attention layer is input into the feedforward neural layer to obtain an intermediate decoding variable, which is expressed as:

[0178] ;

[0179] ;

[0180] In the formula, represents the intermediate decoding variable; represents a feed-forward neural network; represents the first activation function; represents the third activation function; Indicates that the parameter is The second activation function of represents the parameters of the first linear layer; represents the parameters of the second linear layer; represents the parameters of the third linear layer; Indicates adjustable parameters;

[0181] The intermediate decoding variable and the output variable of the attention layer are residually fused to obtain the decoding variable , expressed as:

[0182] ;

[0183] in, represents residual fusion;

[0184] The decoded variable is used as the input of the next transformer decoding layer, iteratively times, generating the final decoded variable ;

[0185] S23. Use the CloFormer layer to extract the feature vector of the final decoded variable, specifically:

[0186] The CloFormer layer includes a convolution stem Convolution Stem, CloBlock and a convolution feedforward network ConvFFN. The CloFormer layer obtains the final decoding variable and performs a convolution operation using the Convolution Stem, which is expressed as follows:

[0187] ;

[0188] In the formula, Represents the output of Convolution Stem; Indicates the preset convolution kernel size; Indicates the preset convolution stride; Indicates the preset convolution padding; Indicates the preset number of output channels; Represents the convolution operation;

[0189] The CloBlock includes a local branch and a global branch. The local branch is locally enhanced using the AttnConv module, which is expressed as:

[0190] ;

[0191] ;

[0192] ⊙ ;

[0193] In the formula, represents context-aware local enhancement; represents the context-aware weight; represents depthwise separable convolution; express The query vector of express Keys; express The value of represents a fully connected layer; represents the fourth activation function; represents the second activation function; Indicates the preset dimension; ⊙ indicates the XOR operation;

[0194] The global branch uses the attention mechanism for global enhancement, which is expressed as:

[0195] ;

[0196] In the formula, Indicates global enhancement; represents the attention mechanism; represents the pooling layer;

[0197] Combine local enhancement and global enhancement and input them into the convolutional feedforward network It is expressed as:

[0198] ;

[0199] In the formula, represents the output of the CloFormer layer, which is the feature vector of the final decoded variable; represents the connection function; represents a convolutional feed-forward network;

[0200] S24, the distribution estimation layer is specifically a mixed Gaussian distribution GMM model, and the probability distribution of the feature vector of the final decoded variable under the Gaussian distribution model is expressed as follows:

[0201] ;

[0202] ;

[0203] ;

[0204] In the formula, The average value of the feature vector representing the final decoded variable; represents the probability density function; The variance of the eigenvector representing the final decoded variable; represents pi; Indicates The parameters of the Gaussian distribution sub-model, where Indicates The expected value of the Gaussian distribution sub-model; Indicates The variance of the Gaussian distribution sub-model; Indicates The weight coefficients of the Gaussian distribution sub-models; represents probability; Indicates The index value of the Gaussian distribution sub-model; Indicates the number of Gaussian distribution sub-models; represents the exponential function;

[0205] S25, using the expectation maximization EM algorithm and the Bayesian information BIC criterion to determine the optimal parameter group of the GMM model, the expectation maximization EM algorithm includes an E step and an M step, and the E step and the M step are iterated until a preset optimization termination condition is reached; the optimization termination condition is expressed as a formula:

[0206] ;

[0207] ;

[0208] ;

[0209] In the formula, represents the log-likelihood function; Indicates The log-likelihood value after iterations; Indicates the maximum number of iterations; Indicates the total number of segments; Indicates The index value of the iteration; represents the logarithmic function;

[0210] S251, the E step is specifically to calculate the probability that each decoded variable value comes from the Gaussian distribution sub-model , expressed as:

[0211] , , ;

[0212] In the formula, Indicates The feature vector of the final decoded variable; Indicates The index value of the feature vector of the final decoded variable; The number of feature vectors representing the final decoded variables;

[0213] S252, the M step is specifically to calculate the parameters of the new iterative Gaussian distribution sub-model, which is expressed as:

[0214] ;

[0215] ;

[0216] ;

[0217] S253, using the BIC criterion to determine the optimal parameter group of the Gaussian distribution model, and obtain the optimal Gaussian distribution model, which is expressed as:

[0218] The BIC criterion is used to determine the optimal parameter group of the Gaussian distribution model, and the optimal Gaussian distribution model is obtained, which is expressed as follows:

[0219] ;

[0220] ;

[0221] In the formula, represents the maximum likelihood estimate of the Gaussian distribution model; Represents the number of parameters of the Gaussian distribution sub-model; Indicates the function for obtaining the maximum value of the log-likelihood function; represents the Bayesian information criterion value;

[0222] S3, using historical CVT data to train a Gaussian distribution model with an optimal parameter group to obtain a final CVT operating state evaluation model;

[0223] S4. Acquire current CVT data, input the CVT data into a final CVT operating state evaluation model, and obtain the operating state of the CVT.

[0224] Example 2

[0225] This embodiment provides a CVT operating state evaluation system based on an improved decoding layer model, the system includes a data acquisition module, a state evaluation module and a state output module, specifically:

[0226] The data acquisition module is used to acquire historical CVT data within a preset time period and perform data cleaning, wherein the historical CVT data includes secondary side voltage, secondary load and temperature; perform a hysteresis operation on the cleaned historical CVT data to obtain covariates corresponding to each historical CVT data, and splice the historical CVT data with the corresponding covariates to obtain connection variables; transmit the spliced ​​CVT data and its corresponding covariates to the state evaluation module;

[0227] The state assessment module has a built-in CVT operation state assessment model, including an embedding layer, Improved transformer decoding layer, CloFormer layer and distribution estimation layer, where:

[0228] The embedding layer is used to receive the spliced ​​CVT data and its corresponding covariates to generate embedded variables;

[0229] The improved transformer decoding layer includes a first RMSNorm normalization layer, an attention layer, a second RMSNorm normalization layer and a feedforward neural layer; the feedforward neural layer includes three linear layers and a SwiGLU activation function layer; the normalized embedded variables are input into the first improved transformer decoding layer, the output of the previous improved transformer decoding layer is used as the input of the next improved transformer decoding layer, and the iteration is repeated. The final decoding variable is obtained; wherein the attention layer obtains the normalized embedded variable to obtain the output variable of the attention layer; the output variable of the attention layer is Input the second RMSNorm normalization layer to get the normalized variable of the attention layer ;

[0230] The feedforward neural layer includes three linear layers and a SwiGLU activation function layer, wherein the linear layer includes a first linear layer, a second linear layer and a third linear layer; the normalized variables of the attention layer are input into the feedforward neural layer to obtain intermediate decoding variables, which are expressed as follows:

[0231] ;

[0232] ;

[0233] In the formula, represents the intermediate decoding variable; represents a feed-forward neural network; represents the first activation function; represents the third activation function; Indicates that the parameter is The second activation function of represents the parameters of the first linear layer; represents the parameters of the second linear layer; represents the parameters of the third linear layer; Indicates adjustable parameters;

[0234] The intermediate decoding variable and the output variable of the attention layer are residually fused to obtain the decoding variable , expressed as:

[0235] ;

[0236] in, represents residual fusion;

[0237] The decoded variable is used as the input of the next transformer decoding layer, iteratively times, generating the final decoded variable ;

[0238] The CloFormer layer is used to capture the contextual features of the final decoded variable and generate a feature vector of the final decoded variable;

[0239] The distribution estimation layer is a mixed Gaussian distribution GMM model, wherein the optimal parameter group of the mixed Gaussian distribution GMM model is determined by using the expectation maximization EM algorithm and the Bayesian information BIC criterion; the CVT operation status evaluation model is trained using historical CVT data to obtain a trained CVT operation status evaluation model; the current CVT data is obtained, and the current CVT data is input into the trained CVT operation status evaluation model to obtain the current CVT operation status;

[0240] The status output module is used to output the operating status of the CVT.

[0241] Example 3

[0242] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the CVT operating status assessment method based on the improved decoding layer model as described in any embodiment of the present invention is implemented.

[0243] Example 4

[0244] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the CVT operating state assessment method based on the improved decoding layer model as described in any embodiment of the present invention is implemented.

[0245] It is worth noting that the system, electronic device and computer-readable storage medium described in the present invention are based on the same principles as the method described in Example 1, and will not be described in detail here.

[0246] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A CVT operating status evaluation method based on an improved decoding layer model, characterized in that: The method comprises: Acquire historical CVT data within a preset time period and perform data cleaning, wherein the historical CVT data includes secondary side voltage, secondary load and temperature; perform a hysteresis operation on the cleaned historical CVT data to obtain covariates corresponding to each historical CVT data, and concatenate the historical CVT data with the corresponding covariates to obtain a connection variable; Construct a CVT operation status evaluation model, including the embedding layer, An improved transformer decoding layer, a CloFormer layer and a distribution estimation layer, wherein: the improved transformer decoding layer includes a first RMSNorm normalization layer, an attention layer, a second RMSNorm normalization layer and a feedforward neural layer; the feedforward neural layer includes three linear layers and a SwiGLU activation function layer; The connection variables are embedded by using the embedding layer to generate embedded variables; the normalized embedded variables are input into the first improved transformer decoding layer, and the output of the previous improved transformer decoding layer is used as the input of the next improved transformer decoding layer, and the The final decoding variable is obtained; wherein the attention layer obtains the normalized embedded variable to obtain the output variable of the attention layer; the output variable of the attention layer is Input the second RMSNorm normalization layer to get the normalized variable of the attention layer ; The feedforward neural layer includes three linear layers and a SwiGLU activation function layer, wherein the linear layer includes a first linear layer, a second linear layer and a third linear layer; the normalized variables of the attention layer are input into the feedforward neural layer to obtain intermediate decoding variables, which are expressed as follows: ； ； In the formula, represents the intermediate decoding variable; represents a feed-forward neural network; represents the first activation function; represents the third activation function; Indicates that the parameter is The second activation function of represents the parameters of the first linear layer; represents the parameters of the second linear layer; represents the parameters of the third linear layer; Indicates adjustable parameters; The intermediate decoding variable and the output variable of the attention layer are residually fused to obtain the decoding variable , expressed as: ； in, represents residual fusion; The decoded variable is used as the input of the next transformer decoding layer, iteratively times, generating the final decoded variable ; The CloFormer layer is used to extract the features of the final decoded variable and generate the feature vector of the final decoded variable; The characteristic vector of the final decoded variable is used as the input of the distribution estimation layer, and the distribution estimation layer is specifically a mixed Gaussian distribution GMM model, wherein the optimal parameter group of the mixed Gaussian distribution GMM model is determined by using the expectation maximization EM algorithm and the Bayesian information BIC criterion; the CVT operation status evaluation model is trained using historical CVT data to obtain a trained CVT operation status evaluation model; The current CVT data is obtained, and the current CVT data is input into a trained CVT operation status evaluation model to obtain the current CVT operation status.

2. The CVT operating state evaluation method based on the improved decoding layer model according to claim 1 is characterized in that: The steps of performing a hysteresis operation on the cleaned historical CVT data to obtain covariates corresponding to each historical CVT data, and splicing the historical CVT data with the corresponding covariates are specifically as follows: Get historical CVT data after cleaning, including voltage , Secondary load and temperature ,in Indicates the time, represents the matrix transpose operator, Indicates Historical CVT data, express Moment Historical CVT data; time-lag operation of voltage, secondary load and temperature; Preset an ordered set of hysteresis indices , Indicates the total number of time points, defined The hysteresis operation at time is expressed as: ； ； ； In the formula, express Historical CVT data at all times; express moment covariate data; Indicates covariates of historical CVT data; Represents a function mapping; Indicates the covariate type serial number; represents the set of real numbers; express Moment covariates of historical CVT data; Indicates Index value of historical CVT data; Get the voltage covariate , secondary load covariate and the temperature covariate ; The historical CVT data are spliced ​​with the corresponding covariates and expressed as follows: ； ； ； In the formula, represents the voltage connection variable; represents the secondary load connection variable; represents the temperature connection variable; express Voltage connection variable at the moment; express Secondary load connection variable at moment; express The temperature at the moment is connected to the variable; represents the connection function; Indicates a connection symbol.

3. The CVT operating state evaluation method based on the improved decoding layer model according to claim 2 is characterized in that: The embedding layer is used to embed the connection variables, generate embedded variables, and normalize the embedded variables, which can be expressed as follows: ； In the formula, represents a set of embedded variables; represents the embedded function; represents the voltage embedded variable; represents the secondary load embedded variable; represents the temperature embedded variable; The embedded variables are normalized using the first RMSNorm normalization layer. The first RMSNorm normalization layer specifically reconstructs the output normalization of the transformer decoding layer into the input normalization, and replaces the layerNorm normalization function in the transformer decoding layer with the RMSNorm normalization function, which is expressed as: ， ； ， ； ， ； In the formula, Indicates a preset positive number; represents the normalized voltage embedded variable; represents the normalized secondary loading embedded variable; represents the normalized temperature embedded variable; Indicates voltage embedded variables; Indicates A secondary load embedding variable; Indicates temperature embedded variables; Indicates the number of voltage covariate data; Indicates The index value of the voltage covariate data; Indicates the number of secondary load covariate data; Indicates The index value of the secondary loading covariate data; Indicates the number of temperature data covariate data; Indicates The index value of the temperature data.

4. The CVT operating state evaluation method based on the improved decoding layer model according to claim 3 is characterized in that: The attention layer obtains the normalized embedded variables, segments the normalized embedded variables and identifies outliers, and the steps of obtaining the output variables of the attention layer are specifically as follows: The attention layer reconstructs the attention mechanism of the transformer and uses the SwiGLU activation function to replace the ReLU activation function in the attention mechanism of the transformer; The normalized embedded variable is segmented according to the preset segment number and segment length to obtain the segmentation variable, which is expressed as: ； ； ； In the formula, Segmental variable representing the normalized voltage embedded variable; Segmental variable representing the normalized secondary loading embedded variable; Segmental variable representing the normalized embedded temperature variable; Indicates The segment length is The normalized voltage is embedded in the segmented variable; Indicates The segment length is The normalized quadratic loadings of the segmented variables are embedded in the variables; Indicates The segment length is The normalized quadratic loadings of the segmented variables are embedded in the variables; express Length; express Length; express Length; express The number of segments; express The number of segments; express The number of segments; express The segment length; express The segment length; express The segment length is ; Indicates The index value of the segmentation variable; Indicates The index value of the segmentation variable; Indicates The index value of the segmentation variable; The specific method of outlier identification is to calculate the quartiles and interquartile ranges of the segmented variables, which can be expressed as follows: ； ； ； ； In the formula, Indicates The interquartile range of the segmental variables; Indicates The segmentation variable value at one quarter of the segmentation variables; Indicates The segmentation variable values ​​that are at three-quarters of the segmentation variables; Indicates The number of segments for a segmentation variable; Indicates The index value of the segmentation variable; The segmentation variable is input into the linear layer built into the attention layer, which is expressed as: ， ； ； ； In the formula, Indicates The query value of a segmentation variable; Indicates The key of the segmentation variable; Indicates The value of the segmentation variable; Represents the query value linear transformation layer; represents the key linear transformation layer; Representation value linear transformation layer; Calculate the output of the attention layer, expressed as: ， ； ； ； ； ； In the formula, Represents the intermediate output variable of the attention layer; Represents the output variable of the attention layer; represents a set of embedded variables; Represents the query value of the segmentation variable; The key representing the segmentation variable; Represents the value of the segmentation variable; Indicates the number of segmentation variables; represents residual fusion; represents the built-in normalization function of the attention layer; Indicates the preset projection dimension; Represents the attention mechanism.

5. The CVT operating state evaluation method based on the improved decoding layer model according to claim 4 is characterized in that: The CloFormer layer is used to extract the feature vector of the final decoded variable, specifically: The CloFormer layer includes a convolution stem Convolution Stem, CloBlock and a convolution feedforward network ConvFFN. The CloFormer layer obtains the final decoding variable and performs a convolution operation using the Convolution Stem, which is expressed as follows: ； In the formula, Represents the output of Convolution Stem; Indicates the preset convolution kernel size; Indicates the preset convolution stride; Indicates the preset convolution padding; Indicates the preset number of output channels; Represents the convolution operation; The CloBlock includes a local branch and a global branch. The local branch is locally enhanced using the AttnConv module, which is expressed as: ； ； ⊙ ； In the formula, represents context-aware local enhancement; represents the context-aware weight; represents depthwise separable convolution; express The query vector of express Keys; express The value of represents a fully connected layer; represents the fourth activation function; represents the second activation function; Indicates the preset dimension; ⊙ indicates the XOR operation; The global branch uses the attention mechanism for global enhancement, which is expressed as: ； In the formula, Indicates global enhancement; represents the attention mechanism; represents the pooling layer; Combine local enhancement and global enhancement and input them into the convolutional feedforward network , expressed as: ； In the formula, represents the output of the CloFormer layer, which is the feature vector of the final decoded variable; represents the connection function; Represents a convolutional feed-forward network.

6. The CVT operating state evaluation method based on the improved decoding layer model according to claim 5 is characterized in that: The optimal parameter group of the GMM model is determined by using the expectation maximization EM algorithm and the BIC criterion; and the Gaussian distribution model with the optimal parameter group is trained using the historical CVT data set to obtain the final CVT operating status evaluation model, which is specifically: The probability distribution of the feature vector of the final decoded variable under the Gaussian distribution model is expressed as follows: ； ； ； In the formula, The average value of the feature vector representing the final decoded variable; represents the probability density function; The variance of the eigenvector representing the final decoded variable; represents pi; Indicates The parameters of the Gaussian distribution sub-model, where Indicates The expected value of the Gaussian distribution sub-model; Indicates The variance of the Gaussian distribution sub-model; Indicates The weight coefficients of the Gaussian distribution sub-models; represents probability; Indicates The index value of the Gaussian distribution sub-model; Indicates the number of Gaussian distribution sub-models; represents the exponential function; The parameters of the Gaussian distribution model are optimized using the expectation maximization EM algorithm and the BIC criterion. The expectation maximization EM algorithm includes an E step and an M step, and the E step and the M step are iterated until a preset optimization termination condition is reached; the optimization termination condition is expressed as follows: ； ； ； In the formula, represents the log-likelihood function; Indicates The log-likelihood value after iterations; Indicates the maximum number of iterations; Indicates the total number of segments; Indicates The index value of the iteration; represents the logarithmic function; The E step specifically calculates the probability that each decoded variable value comes from the Gaussian distribution submodel , expressed as: ， ， ； In the formula, Indicates The feature vector of the final decoded variable; Indicates The index value of the feature vector of the final decoded variable; The number of feature vectors representing the final decoded variables; The M step is specifically to calculate the parameters of the new iterative Gaussian distribution sub-model, which is expressed as follows: ； ； ； The BIC criterion is used to determine the optimal parameter group of the Gaussian distribution model, and the optimal Gaussian distribution model is obtained, which is expressed as follows: ； ； In the formula, represents the maximum likelihood estimate of the Gaussian distribution model; Represents the number of parameters of the Gaussian distribution sub-model; Indicates the function for obtaining the maximum value of the log-likelihood function; represents the Bayesian Information Criterion value.

7. A CVT operating status evaluation system based on an improved decoding layer model, characterized in that: The system includes a data acquisition module, a state evaluation module and a state output module, specifically: The data acquisition module is used to acquire historical CVT data within a preset time period and perform data cleaning, wherein the historical CVT data includes secondary side voltage, secondary load and temperature; perform a hysteresis operation on the cleaned historical CVT data to obtain covariates corresponding to each historical CVT data, and splice the historical CVT data with the corresponding covariates to obtain connection variables; transmit the spliced ​​CVT data and its corresponding covariates to the state evaluation module; The state assessment module has a built-in CVT operation state assessment model, including an embedding layer, Improved transformer decoding layer, CloFormer layer and distribution estimation layer, where: The embedding layer is used to receive the spliced ​​CVT data and its corresponding covariates to generate embedded variables; The improved transformer decoding layer includes a first RMSNorm normalization layer, an attention layer, a second RMSNorm normalization layer and a feedforward neural layer; the feedforward neural layer includes three linear layers and a SwiGLU activation function layer; the normalized embedded variables are input into the first improved transformer decoding layer, the output of the previous improved transformer decoding layer is used as the input of the next improved transformer decoding layer, and the iteration is repeated. The final decoding variable is obtained; wherein the attention layer obtains the normalized embedded variable to obtain the output variable of the attention layer; the output variable of the attention layer is Input the second RMSNorm normalization layer to get the normalized variable of the attention layer ; The feedforward neural layer includes three linear layers and a SwiGLU activation function layer, wherein the linear layer includes a first linear layer, a second linear layer and a third linear layer; the normalized variables of the attention layer are input into the feedforward neural layer to obtain intermediate decoding variables, which are expressed as follows: ； ； In the formula, represents the intermediate decoding variable; represents a feed-forward neural network; represents the first activation function; represents the third activation function; Indicates that the parameter is The second activation function of represents the parameters of the first linear layer; represents the parameters of the second linear layer; represents the parameters of the third linear layer; Indicates adjustable parameters; The intermediate decoding variable and the output variable of the attention layer are residually fused to obtain the decoding variable , expressed as: ； in, represents residual fusion; The decoded variable is used as the input of the next transformer decoding layer, iteratively times, generating the final decoded variable ; The CloFormer layer is used to capture the contextual features of the final decoded variable and generate a feature vector of the final decoded variable; The distribution estimation layer is a mixed Gaussian distribution GMM model, wherein the optimal parameter group of the mixed Gaussian distribution GMM model is determined by using the expectation maximization EM algorithm and the Bayesian information BIC criterion; the CVT operation status evaluation model is trained using historical CVT data to obtain a trained CVT operation status evaluation model; the current CVT data is obtained, and the current CVT data is input into the trained CVT operation status evaluation model to obtain the current CVT operation status; The status output module is used to output the operating status of the CVT.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the CVT operating status evaluation method based on the improved decoding layer model as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the CVT operating status evaluation method based on the improved decoding layer model as described in any one of claims 1 to 6 is implemented.

Images