Industrial equipment energy efficiency evaluation and maintenance decision method and system based on reinforcement learning

Abstract

The application provides an industrial equipment energy efficiency evaluation and maintenance decision method and system based on reinforcement learning, relates to the technical field of reinforcement learning, and comprises the following steps: collecting equipment operation data and extracting features, constructing a maintenance timing decision model based on deep reinforcement learning, combining transfer learning to realize model optimization, generating a maintenance decision scheme according to energy efficiency index analysis and timing causal relationship, executing maintenance and recording process data for model online learning update. The application improves the energy efficiency management level of industrial equipment, prolongs the service life of the equipment, and reduces operation and maintenance costs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to an industrial equipment energy efficiency evaluation and maintenance decision method and system based on reinforcement learning. BACKGROUND

[0002] Traditional industrial equipment maintenance mainly relies on periodic inspection and preventive maintenance strategies, usually performing equipment inspection and maintenance at fixed time intervals, which cannot dynamically adjust the maintenance strategy according to the actual running state of the equipment. At the same time, equipment energy efficiency evaluation is often independent of maintenance decision-making, lacking a whole life cycle integrated perspective, resulting in resource waste and low energy efficiency. In addition, existing maintenance decision systems are mostly based on simple rules and threshold judgments, which are difficult to handle the uncertainty and complexity in the process of equipment operation, and cannot effectively cope with the dynamic characteristics of equipment performance changing over time.

[0003] The current industrial equipment maintenance decision and energy efficiency management has the following main problems: the traditional maintenance decision method lacks the ability to deeply mine the whole life cycle data of the equipment, and cannot fully utilize the equipment performance degradation and energy efficiency change law implied in the massive operation data, resulting in insufficient pertinence of the maintenance decision. The existing maintenance strategy is usually static, cannot be adjusted in real time according to the environmental changes and equipment state, lacks self-adaptive learning ability, and cannot realize dynamic optimization of the maintenance decision. The knowledge sharing and experience transfer mechanism between different types of equipment is missing, and the maintenance model of each equipment needs to be trained from scratch, which is difficult to use the maintenance experience of existing equipment to guide the maintenance decision of new equipment, resulting in low learning efficiency and resource waste. SUMMARY

[0004] The embodiment of the present application provides an industrial equipment energy efficiency evaluation and maintenance decision method and system based on reinforcement learning, which can solve the problems in the prior art.

[0005] In a first aspect, the embodiment of the present application provides an industrial equipment energy efficiency evaluation and maintenance decision method based on reinforcement learning, comprising:

[0006] Collecting real-time running data of the industrial equipment as a first feature data set; performing feature extraction and principal component analysis on the first feature data set to obtain a second feature data set;

[0007] Constructing a maintenance time series decision model based on a deep reinforcement learning algorithm, extracting time series features from the first feature data set and the second feature data set and generating state representation, and determining a maintenance decision action according to a predetermined maintenance action space;

[0008] Online optimization and update of the maintenance time series decision model using a transfer learning method, transferring a multi-source domain knowledge feature matrix to a target equipment field through feature mapping, and updating network parameters of the maintenance time series decision model;

[0009] Based on the action decision result output by the maintenance timing decision model, the equipment energy efficiency index is analyzed, and a maintenance decision scheme is generated based on the timing causal relationship optimization solution;

[0010] The maintenance decision scheme is executed, the time, operation and resource data in the maintenance operation process are recorded to form a maintenance process record, the first feature data set and the second feature data set after the maintenance are collected, the maintenance process record and the feature data set after the maintenance are jointly used as training samples, and the maintenance timing decision model is input for online learning to update the model parameters.

[0011] In an optional implementation,

[0012] The first feature data set is subjected to feature extraction and principal component analysis to obtain a second feature data set, including:

[0013] The first feature data set is subjected to data preprocessing, and an abnormal score is calculated according to the distribution density of each data point in the first feature data set, and a data point with a distribution density less than a preset threshold is determined as an abnormal data point;

[0014] A plurality of data before and after the abnormal data point are selected to construct a sliding window, a median in the sliding window is calculated to replace the abnormal data, and preprocessed feature data are generated;

[0015] The preprocessed feature data are subjected to correlation analysis, a timing correlation coefficient matrix of the feature data in the time dimension is calculated, and a spatial correlation coefficient matrix of the feature data in the space dimension is calculated;

[0016] The timing correlation coefficient matrix and the spatial correlation coefficient matrix are superimposed according to a preset weight to generate a multi-dimensional feature correlation matrix;

[0017] Based on the multi-dimensional feature correlation matrix, a feature contribution weight is calculated, the preprocessed feature data are divided into a key feature group and a secondary feature group according to the feature contribution weight, a principal feature vector is extracted from the key feature group by using a principal component analysis method, and a local feature vector is extracted from the secondary feature group by using a local linear embedding method;

[0018] The principal feature vector and the local feature vector are subjected to adaptive weight fusion to obtain a fused feature vector, and the fused feature vector is used as a second feature data set.

[0019] In an optional implementation,

[0020] constructing a maintenance time sequence decision model based on a deep reinforcement learning algorithm, extracting time sequence features from the first feature data set and the second feature data set and generating state representation, determining a maintenance decision action according to a preset maintenance action space, comprising:

[0021] segmenting and aligning the first feature data set and the second feature data set based on timestamps and constructing an optimal time segmentation sequence, extracting a device state time sequence feature vector in each time sequence segment, and generating a time sequence feature sequence;

[0022] constructing a hierarchical maintenance action space based on the time sequence feature sequence, modeling the correspondence between the time sequence feature sequence and the maintenance action using a Markov decision process, and constructing an action transition constraint matrix;

[0023] extracting an initial time sequence state representation after encoding the time sequence feature sequence using a bidirectional long short-term memory network, inputting the initial time sequence state representation into a multi-head attention layer to fuse state information of different time scales, and obtaining a time sequence state representation;

[0024] constructing a maintenance time sequence decision model based on the time sequence state representation; inputting the time sequence state representation into a multi-layer time sequence convolutional network, extracting multi-scale features through dynamic dilation rate, and performing feature fusion using bidirectional residual connection to obtain enhanced time sequence features.

[0025] inputting the enhanced time sequence features into a value network and an advantage network to obtain a state value function and an action advantage function, combining the state value function and the action advantage function to obtain a time sequence value score of each maintenance action, and selecting a maintenance decision action in combination with the action transition constraint matrix.

[0026] In an optional implementation,

[0027] inputting the time sequence state representation into a multi-layer time sequence convolutional network, extracting multi-scale features through dynamic dilation rate, and performing feature fusion using bidirectional residual connection to obtain enhanced time sequence features, comprising:

[0028] performing multi-scale decomposition on the time sequence state representation and extracting features of different frequency bands, performing importance evaluation on each frequency band feature, dynamically adjusting feature weights based on the importance evaluation results, and reconstructing the weighted feature components through inverse transformation to obtain enhanced time sequence state representation;

[0029] inputting the enhanced time sequence state representation into a multi-layer time sequence convolutional network, setting an exponentially increasing dilation rate sequence in each network, extracting multi-receptive field local time sequence features using dilated convolution operation, and constructing a bidirectional residual connection path between adjacent convolutional layers, and adaptively selecting and fusing the local time sequence features extracted from the previous layer and the features of the current layer through a gating mechanism to obtain multi-scale local time sequence features.

[0030] constructing a query matrix, a key matrix and a value matrix from the multi-scale local time sequence features, performing a scaled dot product operation on the query matrix and the key matrix to calculate an attention score, and obtaining a feature weight after performing a mask processing on the attention score;

[0031] adding time sequence position information to the feature by using a sine position encoding method, and performing a tensor multiplication operation on the feature weight after adding the position information and the value matrix to obtain a weighted feature;

[0032] inputting the weighted feature into a feedforward neural network and constructing a skip connection with an original time sequence state representation to generate an enhanced time sequence feature.

[0033] In an optional implementation,

[0034] updating the maintenance time sequence decision model online by using a transfer learning method, migrating a multi-source domain knowledge feature matrix to a target device field through feature mapping, and updating network parameters of the maintenance time sequence decision model, including:

[0035] obtaining a maintenance time sequence decision model of a target device and historical maintenance data of other devices, constructing a device similarity matrix from the historical maintenance data by using a multi-view spectral clustering method, and calculating a maintenance mode similarity matrix in combination with a dynamic time warping algorithm;

[0036] performing adaptive fusion on the device similarity matrix and the maintenance mode similarity matrix to generate a multi-source domain knowledge feature matrix;

[0037] performing probabilistic encoding on the multi-source domain knowledge feature matrix by using a variational autoencoder, inputting a feature sequence after the encoding into a bidirectional gated recurrent unit network to extract long-range time sequence dependent features, capturing dynamic associations between features in combination with a multi-head self-attention mechanism, and generating a source domain conditional feature distribution through a residual connection and layer normalization;

[0038] calculating a target domain feature distribution according to the maintenance time sequence decision model of the target device, calculating a difference measure between the target domain feature distribution and the source domain conditional feature distribution based on a bulldozer distance, and generating a source domain weight distribution vector by weighting and combining the difference measure and the multi-source domain knowledge feature matrix;

[0039] inputting the source domain conditional feature distribution into a feature encoder, performing feature mapping on the source domain weight distribution vector in combination with a multi-scale convolution and a double attention mechanism, constructing a weighted optimization target to update parameters of the maintenance time sequence decision model.

[0040] In an optional implementation,

[0041] inputting the source domain conditional feature distribution into a feature encoder, performing feature mapping on the source domain weight distribution vector by combining multi-scale convolution and double attention mechanism, constructing a weighted optimization target to update parameters of the maintenance time sequence decision model, including:

[0042] inputting the source domain conditional feature distribution into a multi-branch feature encoder to extract multi-granularity feature representation, adaptively aggregating the multi-granularity feature representation to generate source domain feature representation;

[0043] calculating spatial attention weight and channel attention weight for the source domain feature representation respectively, performing tensor multiplication operation on the spatial attention weight and the channel attention weight to obtain joint attention map, and generating multi-scale attention weight according to long-range dependence relationship of the joint attention map;

[0044] concatenating the source domain feature representation and the multi-scale attention weight to input into a coarse-grained generation network to obtain initial target domain feature, inputting the initial target domain feature and the source domain weight distribution vector into a fine-grained generation network for feature refinement to generate final target domain feature;

[0045] calculating discriminant score on different feature levels based on the final target domain feature, constructing multi-scale adversarial loss based on the discriminant score, and adaptively adjusting the source domain feature representation on the corresponding feature level to obtain softened knowledge representation;

[0046] calculating distillation loss of the final target domain feature and the softened knowledge representation at each level, weighting and combining the multi-scale adversarial loss and the distillation loss at each level according to the source domain weight distribution vector, and updating parameters of the feature transfer network through an alternating optimization algorithm.

[0047] In an alternative embodiment,

[0048] based on the action decision result output by the maintenance time sequence decision model, analyzing the equipment energy efficiency index, and generating a maintenance decision scheme based on time sequence causal relationship optimization solution, including:

[0049] extracting equipment energy efficiency index from the second feature data set, performing multi-scale sampling on the equipment energy efficiency index to extract time-frequency feature matrix, and constructing multi-modal probability distribution of equipment energy efficiency index based on the time-frequency feature matrix;

[0050] inputting the action decision result and the multi-modal probability distribution into a multi-layer prediction unit to generate a probability prediction sequence of equipment energy efficiency change;

[0051] Importance sampling is performed on the probability prediction sequence to obtain multiple groups of energy efficiency prediction samples, and energy efficiency evaluation indexes under different confidence levels are estimated for the multiple groups of energy efficiency prediction samples; a multi-criteria comprehensive evaluation model is constructed for the energy efficiency evaluation indexes to generate a maintenance benefit evaluation vector;

[0052] A graph structure model is used to establish a time sequence causal relationship between the action decision result and the probability prediction sequence, the intervention effect of each maintenance action is calculated through a counterfactual framework, the direct influence and indirect influence corresponding to the intervention effect are quantified based on a structural equation, and the maintenance benefit evaluation vector is adaptively fused with the direct influence and indirect influence to obtain an action importance score;

[0053] According to the action importance score, a priority weight is assigned to the action decision result to obtain a weighted decision result, and an optimization objective considering time sequence constraints is constructed for the weighted decision result to solve an optimal maintenance time sequence and generate a maintenance decision scheme.

[0054] In a second aspect of the embodiment of the present application, an industrial equipment energy efficiency evaluation and maintenance decision system based on reinforcement learning is provided, comprising:

[0055] A first unit is configured to collect real-time running data of industrial equipment as a first feature data set, perform feature extraction and principal component analysis on the first feature data set to obtain a second feature data set;

[0056] A second unit is configured to construct a maintenance time sequence decision model based on a deep reinforcement learning algorithm, extract time sequence features from the first feature data set and the second feature data set and generate state representation, and determine a maintenance decision action according to a preset maintenance action space;

[0057] A third unit is configured to update the maintenance time sequence decision model online by using a transfer learning method, and transfer a multi-source domain knowledge feature matrix to a target equipment field through feature mapping to update network parameters of the maintenance time sequence decision model;

[0058] A fourth unit is configured to analyze equipment energy efficiency indexes based on an action decision result output by the maintenance time sequence decision model, and generate a maintenance decision scheme based on time sequence causal relationship optimization solution;

[0059] A fifth unit is configured to execute the maintenance decision scheme, record time, operation and resource data in a maintenance operation process to form a maintenance process record, collect the first feature data set and the second feature data set after the maintenance is completed, use the maintenance process record and the feature data set after the maintenance as training samples together, input the training samples into the maintenance time sequence decision model for online learning, and update model parameters.

[0060] In a third aspect, the present application provides an electronic device, comprising:

[0061] a processor;

[0062] a memory for storing processor-executable instructions;

[0063] wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0064] In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method described above.

[0065] The present application can effectively obtain key features of the equipment operation state, reduce the data dimension, improve the data processing efficiency and feature expression ability, and provide more accurate data support for subsequent decision-making, by collecting real-time operation data and combining feature extraction and principal component analysis technology. The maintenance timing decision-making model constructed based on deep reinforcement learning can capture the timing characteristics and state change rules from the historical operation data of the equipment, learn the optimal decision-making strategy through interaction with the environment, realize the intelligentization and automation of maintenance decision-making, and significantly improve the timeliness and pertinence of equipment maintenance. The decision-making model is updated online by using the transfer learning method, so that the system can transfer the existing equipment maintenance experience to new equipment, effectively solve the data sparsity and cold start problem, and continuously optimize the model through comprehensive analysis of the maintenance process record and feature data, realize the closed-loop optimization of the energy efficiency evaluation and maintenance decision-making of the equipment throughout its life cycle, and improve the equipment operation efficiency and service life. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 FIG. 1 is a flowchart of the industrial equipment energy efficiency evaluation and maintenance decision-making method based on reinforcement learning according to an embodiment of the present application;

[0067] Figure 2 FIG. 4 is an attention weight distribution thermodynamic diagram according to an embodiment of the present application;

[0068] Figure 3 FIG. 5 is a flowchart of a maintenance decision-making scheme generation process based on timing causality according to an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0070] The technical solutions of the present application will be described in detail below with specific examples. The following specific examples can be combined with each other, and some examples may not be described in detail for the same or similar concepts or processes.

[0071] Figure 1 The flowchart of the embodiment of the present application based on the industrial equipment energy efficiency evaluation and maintenance decision method of reinforcement learning is shown in FIG. 1. Figure 1 The method comprises the following steps:

[0072] Collecting real-time running data of the industrial equipment as a first feature data set; performing feature extraction and principal component analysis on the first feature data set to obtain a second feature data set;

[0073] Constructing a maintenance time sequence decision model based on a deep reinforcement learning algorithm, extracting time sequence features from the first feature data set and the second feature data set and generating state representation, and determining maintenance decision actions according to a pre-set maintenance action space;

[0074] Using a transfer learning method to perform online optimization and update of the maintenance time sequence decision model, transferring a multi-source domain knowledge feature matrix to a target equipment field through feature mapping, and updating network parameters of the maintenance time sequence decision model;

[0075] Based on the action decision results output by the maintenance time sequence decision model, analyzing the equipment energy efficiency index, and generating a maintenance decision scheme based on time sequence causal relationship optimization solution;

[0076] Executing the maintenance decision scheme, recording time, operation and resource data in the maintenance operation process to form a maintenance process record, collecting the first feature data set and the second feature data set after the maintenance is completed, and inputting the maintenance process record and the feature data set after the maintenance as training samples into the maintenance time sequence decision model for online learning to update the model parameters.

[0077] In an optional embodiment, the feature extraction and principal component analysis on the first feature data set to obtain a second feature data set comprises:

[0078] Performing data preprocessing on the first feature data set, calculating an abnormal score according to the distribution density of each data point in the first feature data set, and determining that a data point with a distribution density less than a pre-set threshold as an abnormal data point;

[0079] Selecting a plurality of data before and after the abnormal data point to construct a sliding window, calculating the median in the sliding window to replace the abnormal data, and generating pre-processed feature data;

[0080] performing correlation analysis on the preprocessed feature data, calculating a time correlation coefficient matrix of the feature data in a time dimension, and calculating a space correlation coefficient matrix of the feature data in a space dimension;

[0081] superimposing the time correlation coefficient matrix and the space correlation coefficient matrix according to preset weights to generate a multi-dimensional feature correlation matrix;

[0082] calculating a feature contribution weight based on the multi-dimensional feature correlation matrix, dividing the preprocessed feature data into a key feature group and a secondary feature group according to the feature contribution weight, extracting a principal feature vector from the key feature group by using a principal component analysis method, and extracting a local feature vector from the secondary feature group by using a local linear embedding method;

[0083] performing adaptive weight fusion on the principal feature vector and the local feature vector to obtain a fused feature vector, and taking the fused feature vector as a second feature data set.

[0084] For example, data preprocessing is performed on the first feature data set. The system receives a first feature data set containing multi-dimensional features, such as an industrial equipment operation parameter data set, which includes time series data of ten dimensions such as temperature, pressure, vibration, and current, each dimension containing 1000 sampling points. The system calculates an anomaly score according to the distribution density of each data point. The specific implementation is as follows: the average distance between each data point and its adjacent data points is calculated, and the distance is converted into a standardized score. In actual application, the K-Nearest Neighbor algorithm can be selected to calculate the distribution density, and the value of K can be set to 10. When the distribution density of a certain data point is less than a preset threshold (for example, 0.05), the system marks it as an abnormal data point.

[0085] For the detected abnormal data points, the system constructs a sliding window for correction. For example, the 150th data point in the temperature data is detected as abnormal (the value is 85°C, while the normal interval is 30-45°C), and the system selects 5 data points before and after the point to construct a sliding window with a size of 11, and calculates the median (such as 38°C) in the window to replace the abnormal value. After processing all abnormal points, a preprocessed feature data set is generated.

[0086] For the preprocessed feature data, correlation analysis is performed, the autocorrelation of each feature with time and the cross-correlation between different features in the time dimension are calculated, and a time correlation coefficient matrix is formed. For example, the time delay correlation between temperature and pressure is analyzed, and it is found that the pressure rises after the temperature rises for 10 minutes, and the correlation coefficient is 0.78. In the space dimension, the system analyzes the correlation of different features at the same time point to form a space correlation coefficient matrix. For example, the space correlation coefficient of temperature and current is 0.65, indicating that the trend of simultaneous change of the two parameters is relatively consistent.

[0087] The system superimposes the two correlation coefficient matrices according to preset weights to generate a multi-dimensional feature correlation matrix. In a specific implementation, the time series correlation coefficient weight can be set to 0.6, and the spatial correlation coefficient weight can be set to 0.4. For example, the time series correlation coefficient of temperature and pressure is 0.78, and the spatial correlation coefficient is 0.55. Therefore, the superimposed correlation is 0.78 x 0.6 + 0.55 x 0.4 = 0.688.

[0088] Based on the multi-dimensional feature correlation matrix, the system calculates the contribution weight of each feature. The specific method is to calculate the correlation sum of each feature with all other features and perform normalization processing. For example, the correlation sum of the temperature feature with other features is 6.5, ranking second among all features, and the normalized contribution weight is 0.23. The system divides the features into a key feature group and a secondary feature group according to the contribution weight, and the features with a contribution weight greater than 0.15 (such as temperature, pressure, and current) are classified into the key feature group, and the remaining features are classified into the secondary feature group.

[0089] For the key feature group, the system uses the principal component analysis method to extract the principal feature vector. The specific operation is to calculate the covariance matrix of the key features, solve the eigenvalues and eigenvectors, and select the largest several eigenvectors to form the principal feature vector. For example, from the three key features of temperature, pressure, and current, two principal components are extracted, which explain 75% and 20% of the variance of the original data, respectively.

[0090] For the secondary feature group, the system uses the locally linear embedding method to extract the local feature vector. First, determine the K nearest neighbors (K can be set to 12) of each data point, then construct a local reconstruction weight matrix, and finally solve the characteristic equation to obtain the local feature vector. For example, from the secondary features such as vibration and noise, one local feature vector is extracted, which explains 95% of the local structure information of the original secondary features.

[0091] According to the proportion of data variance explained by the principal feature vector and the local feature vector, respectively, the fusion weight is dynamically adjusted. In this example, the principal feature vector weight is set to 0.85, and the local feature vector weight is set to 0.15. The fusion process is realized through weighted connection, and a fusion feature vector with a length of 3 is obtained, of which the first two dimensions come from the principal feature vector, and the third dimension comes from the local feature vector. The values of each dimension have been adjusted according to the weight. This fused feature vector is used as the second feature data set for subsequent data analysis and processing.

[0092] The application detects abnormal values by calculating distribution density and corrects by using a sliding window, effectively improving data quality; constructs a multi-dimensional feature matrix combining time sequence and spatial correlation analysis to accurately identify the correlation between features; uses principal component analysis and local linear embedding method to process key features and secondary features respectively, and performs adaptive weight fusion, reducing data dimension while retaining key information and local structure features, improving the accuracy and computational efficiency of feature extraction, and laying a reliable foundation for subsequent data analysis.

[0093] In an optional embodiment, a maintenance time sequence decision model is constructed based on a deep reinforcement learning algorithm, time sequence features are extracted from the first feature data set and the second feature data set to generate state representation, and a maintenance decision action is determined according to a preset maintenance action space, including:

[0094] The first feature data set and the second feature data set are segmented and aligned based on timestamps to construct an optimal time segmentation sequence, and device state time sequence feature vectors are extracted in each time sequence segment to generate a time sequence feature sequence;

[0095] A hierarchical maintenance action space is constructed based on the time sequence feature sequence, and a Markov decision process is used to model the correspondence between the time sequence feature sequence and the maintenance action to construct an action transition constraint matrix;

[0096] After encoding the time sequence feature sequence using a bidirectional long short-term memory network, an initial time sequence state representation is extracted, the initial time sequence state representation is input into a multi-head attention layer to fuse state information of different time scales, and a time sequence state representation is obtained;

[0097] A maintenance time sequence decision model is constructed based on the time sequence state representation; the time sequence state representation is input into a multi-layer time sequence convolution network, multi-scale features are extracted through dynamic hole rate, feature fusion is performed using bidirectional residual connection, and enhanced time sequence features are obtained.

[0098] The enhanced time sequence features are input into a value network and an advantage network to obtain a state value function and an action advantage function, the state value function and the action advantage function are combined to obtain a time sequence value score of each maintenance action, and a maintenance decision action is selected in combination with the action transition constraint matrix.

[0099] For example, the first feature data set and the second feature data set are segmented and aligned based on timestamps to construct an optimal time segmentation sequence, the two data sets are sorted according to timestamps, and then a dynamic time warping algorithm is used to calculate the time alignment relationship between the two data sets. For example, for a certain device, the sensor data in the first feature data set is collected at a frequency of once per minute, and the maintenance records in the second feature data set are unevenly distributed. At this time, the dynamic time warping algorithm is used to divide the two data sets into several time segments, and the length of each time segment is 6 hours, ensuring that the data in each time segment is complete and has time continuity.

[0100] In each time sequence segment, the device state time sequence feature vector is extracted to generate a time sequence feature sequence. For sensor data, statistical features (mean, variance, peak, valley, and change rate) are calculated; for text-based maintenance records, the TF-IDF method is used to extract keywords and convert them into vector representations. For example, for a certain wind power device, in a 6-hour time segment, the statistical features of 10 kinds of sensor data such as wind speed, power generation, bearing temperature, etc. are extracted to form a 50-dimensional feature vector, and then combined with the 30-dimensional text feature vector extracted from the maintenance records, an 80-dimensional time sequence feature vector is finally obtained.

[0101] Based on the time sequence feature sequence, a hierarchical maintenance action space is constructed, and the maintenance action types are induced from the historical maintenance records to form a hierarchical action space containing specific maintenance actions. A Markov decision process is used to model the correspondence between the time sequence feature sequence and the maintenance action, and a action transition constraint matrix is constructed. According to the order of maintenance actions in historical data, the transition probabilities between different maintenance actions are calculated to form a 7x7 action transition constraint matrix.

[0102] After encoding the time sequence feature sequence using a bidirectional long short-term memory network, the initial time sequence state representation is extracted. A bidirectional LSTM network with 128 hidden units is constructed, which inputs an 80-dimensional time sequence feature vector sequence and outputs a 256-dimensional initial state representation vector. The initial time sequence state representation is input into a multi-head attention layer to fuse state information of different time scales, and a time sequence state representation is obtained. A multi-head attention layer with 8 attention heads is constructed, each attention head has an output dimension of 32, and the total output dimension is 256, which realizes the attention and fusion of features of different time scales (such as short-term, medium-term, and long-term).

[0103] Based on the time sequence state representation, a maintenance time sequence decision model is constructed. The time sequence state representation is input into a multi-layer time sequence convolutional network to extract multi-scale features through dynamic dilation rate. A bidirectional residual connection is used for feature fusion to obtain enhanced time sequence features.

[0104] The enhanced timing features are respectively input into a value network and an advantage network to obtain a state value function and an action advantage function.

[0105] The state value function and the action advantage function are combined to obtain a timing value score of each maintenance action, and a maintenance decision action is selected in combination with an action transition constraint matrix.

[0106] The application realizes accurate alignment and segmentation of data sets through a dynamic time warping algorithm, effectively extracts timing features, realizes multi-scale state representation through a bidirectional LSTM and a multi-head attention mechanism, enhances the model's ability to capture timing features, combines dynamic hole rate timing convolution and bidirectional residual connection to improve the receptive field of feature extraction, accurately evaluates the value of maintenance actions through a value network and an advantage network, and uses an action transition constraint matrix to ensure the rationality of the decision sequence, significantly improving the accuracy and reliability of maintenance decisions.

[0107] In an alternative embodiment, the timing state representation is input into a multi-layer timing convolution network, multi-scale features are extracted through dynamic hole rate, bidirectional residual connection is used for feature fusion, and enhanced timing features are obtained, including:

[0108] The timing state representation is multi-scale decomposed and different frequency band features are extracted, the importance of each frequency band feature is evaluated, the feature weight is dynamically adjusted based on the importance evaluation result, the weighted feature components are reconstructed through inverse transformation to obtain enhanced timing state representation;

[0109] The enhanced timing state representation is input into a multi-layer timing convolution network, an exponentially increasing hole rate sequence is set in each network, multi-receptive field local timing features are extracted through a hole convolution operation, and a bidirectional residual connection path is constructed between adjacent convolution layers, the local timing features extracted from the previous layer are adaptively selected and fused with the features of the current layer through a gating mechanism to obtain multi-scale local timing features;

[0110] A query matrix, a key matrix and a value matrix are constructed for the multi-scale local timing features, a scaled dot product operation is performed on the query matrix and the key matrix to calculate an attention score, and a feature weight is obtained after mask processing of the attention score;

[0111] The sine position coding mode is used to add time sequence position information to the features, and a weighted feature is obtained by performing tensor multiplication on the feature weight and value matrix after adding the position information.

[0112] The weighted feature is input into a feedforward neural network to construct a skip connection with the original time sequence state representation, thereby generating an enhanced time sequence feature.

[0113] For example, the time sequence state representation is decomposed into different frequency band features. Specifically, the wavelet transform is used to decompose the input time sequence state representation into different frequency band sub-signals. For example, for a time sequence state representation with a length of 1024, it can be decomposed into 4 frequency bands, corresponding to sub-signals with frequency ranges of 0-16 Hz, 16-32 Hz, 32-64 Hz and 64-128 Hz, respectively.

[0114] The importance of each frequency band feature is evaluated by calculating the energy distribution of each frequency band sub-signal to identify the frequency components that contribute more to the target task. For example, the variance or entropy value of each frequency band sub-signal is calculated as an importance indicator, and a larger variance or higher entropy value indicates that the frequency band contains more effective information. In actual application, it is assumed that the importance scores of the four frequency bands are 0.8, 0.5, 0.3 and 0.1, respectively. A soft attention mechanism is used to assign different weight coefficients to each frequency band feature, and the frequency band with high importance obtains a larger weight. For example, based on the above scores, the weights of the four frequency bands can be set to 0.47, 0.29, 0.18 and 0.06, respectively, ensuring that the weight sum is 1. The weighted feature components are reconstructed by inverse wavelet transform to obtain an enhanced time sequence state representation.

[0115] The enhanced time sequence state representation is input into a multi-layer time sequence convolutional network. In this embodiment, the network is composed of 8 layers of time sequence convolutional layers, each containing 64 convolutional kernels with a kernel size of 3.

[0116] An exponentially increasing dilation rate sequence is set in each layer of the network. The dilation rate refers to the sampling interval in the convolution operation, and the larger the dilation rate, the larger the receptive field. Specifically, from the first layer to the eighth layer, the dilation rates are set to 1, 2, 4, 8, 16, 32, 64 and 128, respectively, so that the network can capture multi-scale features from local details to global patterns. For example, for the third layer convolution, the dilation rate is 4, which means that during the convolution operation, every 3 elements are selected as sample points for convolution calculation, effectively expanding the receptive field range.

[0117] Dilated convolution is used to extract multi-receptive field local temporal features. For input feature X, dilated convolution kernel W and dilated rate d are used for convolution, and output feature Y can be calculated by the product of the input elements at the corresponding positions and the convolution kernel weights. For example, for the second layer of the convolution layer with a dilated rate of 2, when the convolution kernel size is 3, the actual receptive field covers 5 elements in the input sequence (the original element plus 2 interval elements on the left and right).

[0118] Bidirectional residual connection paths are constructed between adjacent convolution layers. The forward path delivers the output feature of the i-th layer to the i+1-th layer, and the backward path feeds back the i+1-th layer feature to the i-th layer after 1x1 convolution transformation. For example, the third layer convolution output feature F3 is directly delivered to the fourth layer through the forward path, and the fourth layer feature F4 is fused with F3 through the backward path after 1x1 convolution transformation.

[0119] The gating mechanism realizes adaptive feature selection and fusion. The sigmoid activation function is used to generate the gating signal G, which controls the fusion ratio of different level features. In practical applications, the value range of G is 0 to 1, when the quality of a certain position feature Fi is higher, the value of G at the corresponding position is close to 1, and vice versa.

[0120] Query matrix, key matrix and value matrix are constructed for multi-scale local temporal features. Assuming that the feature dimension is 64, the feature F is projected into query matrix Q, key matrix K and value matrix V through three different linear transformations, and their dimensions are all [sequence length, 64].

[0121] The scaling dot product operation is performed on the query matrix and the key matrix to calculate the attention score. Specifically, the transpose of QK is calculated and divided by the scaling factor 8 (the square root of 64) to obtain the attention score matrix A. For example, for an input with a sequence length of 100, the dimension of the attention matrix A is [100, 100], and A[i,j] represents the attention weight of position i to position j.

[0122] Feature weights are obtained after masking the attention score. In order to ensure the temporal causality, an upper triangular mask matrix is used to mask the future information after the current time, and the specific operation is to set the attention score at the corresponding position to negative infinity. Then the weight matrix W is obtained by applying the softmax function to the masked attention score. For example, for position i, only W[i,j] has non-zero value when j≤i, and the sum of all weights is 1.

[0123] The sinusoidal position encoding method is used to add time sequence position information to the features. For each position and feature dimension in the sequence, the position encoding value is calculated using the sine and cosine functions respectively, considering the proportional relationship between the position and the dimension. The calculated position encoding value is added to the original feature, so that the model can effectively identify and distinguish the feature information of different time steps. The feature weight after adding the position information is multiplied by the value matrix to obtain the weighted feature. Specifically, the matrix product of W and V is calculated to obtain the context feature C, with a dimension of [sequence length, 64]. For example, the context feature C[i] of position i is the weighted sum of the features V[j] of all positions j≤i with weights W[i,j].

[0124] The weighted feature is input into the feedforward neural network to construct a skip connection with the original time sequence state representation to generate enhanced time sequence features. The feedforward network includes two fully connected layers, which use the ReLU activation function in the middle, and the feature dimension is expanded from 64 to 128 and then mapped back to 64. Finally, the network output and the original feature are added through a residual connection to obtain the final enhanced time sequence feature. In specific implementation, if the original feature dimension is 64, the parameter matrices of the two fully connected layers of the feedforward network are [64, 128] and [128, 64] respectively, and the processing sequence length is 100. The dimension of the final output enhanced feature is [100, 64].

[0125] Figure 2 The attention weight distribution thermogram of the application shows the weight distribution characteristics of the attention mechanism of the application in time sequence data processing: the attention distribution is highly focused, and the weight value is mainly concentrated in a specific time step, and the attention weight of most key positions reaches a high value of 0.84-0.93; the weight on the diagonal line is generally high, indicating that the current time step feature has a strong attention to itself, such as the weight of t2 position is 0.93 and the weight of t3 position is 0.89. The attention distribution presents a block cluster structure rather than uniform dispersion, indicating that the model can adaptively identify the key time points in the time sequence data and their mutual relationship. The low value area (light color part) in the weight matrix indicates that the model has successfully filtered out the interference information of irrelevant time steps, improving the accuracy of feature extraction. Overall, the attention mechanism of the application realizes accurate capture of key dependent relationships in time sequence data, and has stronger feature selection ability and pattern recognition efficiency compared with traditional methods.

[0126] The application realizes multi-scale decomposition and frequency band importance evaluation through wavelet transform, effectively extracts key timing characteristics, adopts a multi-layer convolutional network with an exponentially increasing hole rate and a bidirectional residual connection to expand the receptive field range and enhance the feature extraction capability, combines a query-key-value attention mechanism and a sinusoidal position encoding to accurately capture long-range timing dependencies, realizes feature adaptive selection and fusion through a gating mechanism, significantly improves the expression ability and robustness of timing features, and provides high-quality feature representation for subsequent maintenance decisions.

[0127] In an optional embodiment, a migration learning method is used to update the maintenance timing decision model online, multi-source domain knowledge feature matrices are migrated to the target device field through feature mapping, network parameters of the maintenance timing decision model are updated, and the network parameters include:

[0128] Maintenance timing decision models of a target device and historical maintenance data of other devices are obtained, a multi-view spectral clustering method is used to construct a device similarity matrix from the historical maintenance data, and a dynamic time warping algorithm is used to calculate a maintenance mode similarity matrix;

[0129] The device similarity matrix and the maintenance mode similarity matrix are adaptively fused to generate a multi-source domain knowledge feature matrix;

[0130] A variational autoencoder is used to probabilistically encode the multi-source domain knowledge feature matrix, a feature sequence after encoding is input into a bidirectional gated recurrent unit network to extract long-range timing dependency features, a multi-head self-attention mechanism is combined to capture dynamic associations between features, a source domain conditional feature distribution is generated through residual connection and layer normalization;

[0131] A target domain feature distribution is calculated according to the maintenance timing decision model of the target device, a difference measure between the target domain feature distribution and the source domain conditional feature distribution is calculated based on the bulldozer distance, the difference measure and the multi-source domain knowledge feature matrix are weighted and combined to generate a source domain weight distribution vector;

[0132] The source domain conditional feature distribution is input into a feature encoder, a multi-scale convolution is combined with a double attention mechanism to perform feature mapping on the source domain weight distribution vector, a weighted optimization target is constructed to update parameters of the maintenance timing decision model.

[0133] Exemplarily, the maintenance timing decision model of the target device is acquired together with historical maintenance data of other devices. A multi-view spectral clustering method is used to construct a device similarity matrix based on the acquired historical maintenance data. Specifically, feature extraction is performed on the device data from three perspectives of device parameters, fault types, and maintenance periods, and a similarity sub-matrix is generated for each perspective. For the device parameter perspective, statistical features (mean, variance, peak value, etc.) of the device operating parameters are extracted to construct a parameter similarity sub-matrix; for the fault type perspective, the proportion of the same fault types among different devices is counted as a fault similarity sub-matrix; and for the maintenance period perspective, the distribution difference of the device maintenance interval time is calculated as a maintenance period similarity sub-matrix. The three sub-matrices are weighted and fused to obtain a 10x10 device similarity matrix S.

[0134] At the same time, a maintenance mode similarity matrix is calculated in combination with the dynamic time warping algorithm. The maintenance activity sequence of each device is regarded as a time series, and the dynamic time warping algorithm is applied to the maintenance sequences of any two devices to calculate the timing distance. Taking the first and second devices as examples, the maintenance activity time series M1 = [t 11 ,t 12 ,...,t1n] and M2 = [t 21 ,t 22 ,...,t2m] within one year of each are extracted, and the optimal alignment path is solved by dynamic programming to obtain the timing distance D(M1, M2) = 218.5 hours. After normalization of the timing distance calculation results of all device pairs, a 10x10 maintenance mode similarity matrix P is constructed.

[0135] The device similarity matrix S and the maintenance mode similarity matrix P are adaptively fused to generate a multi-source domain knowledge feature matrix K. The fusion process adopts a dynamic weight mechanism, and the weights of the two matrices are automatically adjusted according to the current state of the target device. For example, when the target device has just completed a maintenance activity, the weight of the maintenance mode similarity matrix is increased (α = 0.7); when the device is in a normal operating state, the weight of the device similarity matrix is increased (α = 0.6). The specific fusion formula is the weighted sum of the two matrices, such as when α = 0.6, K = 0.6xS + 0.4xP, a 10x10 multi-source domain knowledge feature matrix is obtained.

[0136] The variational autoencoder is used to probabilistically encode the multi-source domain knowledge feature matrix K. The variational autoencoder includes an encoder and a decoder, and the encoder is composed of three fully connected networks (input layer 10x10, hidden layer 128, latent space 64), which maps K to the mean vector μ and the variance vector σ of the latent space, generates the latent variable z through the reparameterization technique, and the dimension is 64. The decoder is also composed of three fully connected networks, which reconstructs z into the original feature matrix.

[0137] The encoded feature sequence z is input into a bidirectional gated recurrent unit network to extract long-range timing dependence features. The network includes two layers of bidirectional GRU structure, with 128 hidden units and an input of 64-dimensional latent variables z generated by the variational autoencoder. The timing information is captured through forward and backward propagation, and a 256-dimensional feature vector h is output.

[0138] The dynamic association between features is captured by combining the multi-head self-attention mechanism, and an 8-head attention mechanism is used to divide the feature vector h of the previous step into 8 subspaces, and the attention weight is calculated independently for each subspace. For example, the first attention head maps the input to query, key and value vectors, calculates the similarity through dot product, applies the softmax function to obtain the normalized weight, and weights the summary to obtain the enhanced feature. The output features of the 8 heads are connected and linearly transformed to obtain a 256-dimensional attention feature a. The source domain conditional feature distribution Fs is generated through residual connection and layer normalization. The h and a are added to form a residual connection, and the source domain conditional feature distribution Fs is obtained through layer normalization processing, with a dimension of 256. The target domain feature distribution Ft is calculated according to the maintenance timing decision model of the target device. The same feature extraction process as described above is applied to the historical data of the target device to obtain the target domain feature distribution Ft, which is also a 256-dimensional vector.

[0139] The difference measure d between the target domain feature distribution Ft and the source domain conditional feature distribution Fs is calculated based on the bulldozer distance. The bulldozer distance measures the distance between two distributions by solving the optimal transport problem, and the calculation result d = 0.385. The difference measure d is combined with the multi-source domain knowledge feature matrix K to generate a source domain weight distribution vector w = [0.12, 0.15, 0.06, 0.09, 0.18, 0.11, 0.08, 0.14, 0.07], representing the knowledge transfer weights of the 9 source domain devices.

[0140] The source domain conditional feature distribution Fs is input into the feature encoder, and the multi-scale convolution and double attention mechanism are combined with the source domain weight distribution vector w to perform feature mapping. A weighted optimization target is constructed to update the parameters of the maintenance timing decision model.

[0141] The present application accurately calculates the similarity between devices by multi-view spectral clustering and dynamic time warping algorithm, and constructs a multi-source domain knowledge feature matrix; uses a variational autoencoder to realize feature probability coding, and captures long-range dependence by combining bidirectional gated recurrent units and multi-head attention mechanism; based on the bulldozer distance to measure the difference between the domains and generate a weight distribution vector, and realize feature transfer mapping through multi-scale convolution and double attention mechanism, which significantly improves the generalization ability and maintenance decision accuracy of the model on the target device.

[0142] In an alternative embodiment, the source domain conditional feature distribution input feature encoder, using multi-scale convolution and double attention mechanism combined with the source domain weight distribution vector for feature mapping, constructing a weighted optimization target to update the parameters of the maintenance time sequence decision model, including:

[0143] The source domain conditional feature distribution is input into a multi-branch feature encoder to extract multi-granularity feature representation, and the multi-granularity feature representation is adaptively aggregated to generate source domain feature representation;

[0144] The spatial attention weight and the channel attention weight are calculated for the source domain feature representation, respectively, the spatial attention weight and the channel attention weight are subjected to tensor multiplication operation to obtain a joint attention map, and the multi-scale attention weight is generated according to the long-range dependence relationship of the joint attention map;

[0145] The source domain feature representation and the multi-scale attention weight are concatenated to input a coarse-grained generation network to obtain an initial target domain feature, and the initial target domain feature and the source domain weight distribution vector are input into a fine-grained generation network for feature refinement to generate a final target domain feature;

[0146] The final target domain feature is calculated for the discriminant score at different feature levels, and a multi-scale adversarial loss is constructed based on the discriminant score, and the source domain feature representation is adaptively temperature adjusted to obtain a softened knowledge representation at the corresponding feature level;

[0147] The distillation loss of the final target domain feature and the softened knowledge representation at each level is calculated, the multi-scale adversarial loss and the distillation loss at each level are combined according to the source domain weight distribution vector, and the parameters of the feature transfer network are updated by an alternating optimization algorithm.

[0148] Exemplarily, the source domain conditional feature distribution is input into a multi-branch feature encoder to extract a multi-granularity feature representation. In actual implementation, the multi-branch feature encoder contains three parallel branches: (1) the first branch adopts three layers of convolutional layers, with a convolution kernel size of 3x3 and a step size of 1, and each layer is followed by batch normalization and a ReLU activation function; (2) the second branch adopts two layers of convolutional layers, with a convolution kernel size of 5x5 and a step size of 1; and (3) the third branch adopts one layer of convolutional layers, with a convolution kernel size of 7x7 and a step size of 1. The three branches respectively extract feature information of different granularities from the source domain input features. For example, for an image feature with an input dimension of 64x64x3, the three branches respectively output feature maps of 32x32x64, 32x32x128, and 32x32x256. The multi-granularity feature representations are adaptively aggregated to generate a source domain feature representation. In the implementation process, a channel attention mechanism is used to weight the importance of each branch feature. First, the global average pooling is performed on each branch feature to obtain a feature vector in the channel dimension, then the weight coefficients of each channel are calculated through two fully connected layers, and finally the weighted features are connected and fused through a 1x1 convolution to generate the source domain feature representation. In the example, the fused source domain feature representation has a dimension of 32x32x256.

[0149] The spatial attention weight and the channel attention weight are calculated for the source domain feature representation. The spatial attention weight calculation process is as follows: the maximum value and the average value of the channel dimension of the source domain feature representation are calculated to obtain two spatial feature maps, the two feature maps are connected and then passed through a 7x7 convolutional layer followed by a Sigmoid function to obtain a spatial attention map with a size of 32x32x1. The channel attention weight calculation process is as follows: the global average pooling and the maximum pooling of the spatial dimension of the source domain feature representation are performed to obtain two channel feature vectors, the two vectors are passed through a fully connected layer with shared weights, and finally fused and passed through a Sigmoid function to obtain a channel attention vector with a size of 1x1x256. The spatial attention weight and the channel attention weight are subjected to tensor multiplication operation to obtain a joint attention map. In specific implementation, the channel attention vector is first expanded in the spatial dimension to have a size of 32x32x256, and then multiplied element by element with the spatial attention map (expanded to 32x32x256) to obtain an initial joint attention map. Subsequently, a non-local operation is performed to capture long-range dependencies to generate multi-scale attention weights. In the implementation process, the initial joint attention map is passed through convolutional layers with 1x1, 3x3, and 5x5 convolution kernels to obtain attention features of three scales, respectively. Then, these features are connected and passed through a 1x1 convolution to obtain the final multi-scale attention weights with a dimension of 32x32x256.

[0150] The source domain feature representation and the multi-scale attention weight are concatenated and input into the coarse-grained generation network. First, the source domain feature representation and the multi-scale attention weight are connected in the channel dimension to obtain a concatenated feature with a dimension of 32x32x512, and then an initial target domain feature with a dimension of 64x64x128 is generated by a coarse-grained generation network composed of a residual block and two transpose convolution layers. The initial target domain feature and the source domain weight distribution vector are input into the fine-grained generation network for feature refinement. In an embodiment, the source domain weight distribution vector has a dimension of 128, which is first converted into a spatial feature map through a fully connected layer and a reshaping operation, and then connected with the initial target domain feature. A final target domain feature with a dimension of 64x64x64 is generated by a fine-grained generation network composed of two residual blocks and a convolution layer.

[0151] The final target domain feature is calculated for the discriminant score at different feature levels, and a multi-scale discriminator network is used, which includes three sub-discriminators of different scales to discriminate the features of the original size, 1 / 2 size and 1 / 4 size, and output the corresponding discriminant score. For example, for a target domain feature of 64x64x64, three discriminators output discriminant score maps of 64x64, 32x32 and 16x16, respectively.

[0152] Based on the discriminant score, a multi-scale adversarial loss is constructed. For each scale of the discriminant score, the mean square error between it and the real label is calculated, and then the weighted adversarial loss is obtained according to the corresponding weight value in the source domain weight distribution vector. At the same time, the source domain feature representation is softened by adaptive temperature adjustment to obtain a softened knowledge representation. In a specific implementation, the statistics in the channel dimension are first calculated for each layer of features, and then the temperature parameter T (usually between 1-10) is determined according to these statistics. The original feature is divided by T and then passed through the Softmax function to obtain the softened knowledge representation.

[0153] The distillation loss of the final target domain feature and the softened knowledge representation at each level is calculated, and the multi-scale adversarial loss and the distillation loss at each level are combined by weighting according to the source domain weight distribution vector. In actual application, the weighting coefficients are usually set to 0.5 and 0.5, and the parameters of the feature transfer network are updated by an alternating optimization algorithm. The optimization process uses the Adam optimizer with a learning rate of 0.0002, a batch size of 16 and a training iteration number of 50000. In this way, the effective transfer of source domain knowledge to the target domain is realized, and the performance of the model in the target domain is enhanced.

[0154] The application extracts multi-granularity feature representations through a multi-branch feature encoder, generates a joint attention map in combination with spatial and channel attention mechanisms, realizes accurate feature migration by using a two-level generation network and source domain weight distribution, constructs a weighted optimization target based on a multi-scale adversarial loss and a hierarchical knowledge distillation loss, and realizes dynamic updating of the feature migration network through an alternating optimization algorithm, thereby significantly improving the feature expression capability and migration learning effect of the model in the target domain and providing effective protection for online optimization of the decision model.

[0155] In an optional embodiment, the equipment energy efficiency index is analyzed based on an action decision result output by the maintenance time sequence decision model, and a maintenance decision scheme is generated based on a time sequence causal relationship optimization solution, including:

[0156] An equipment energy efficiency index is extracted from the second feature data set, a time-frequency feature matrix is extracted by multi-scale sampling of the equipment energy efficiency index, and a multi-modal probability distribution of the equipment energy efficiency index is constructed based on the time-frequency feature matrix;

[0157] The action decision result and the multi-modal probability distribution are input into a multi-layer prediction unit to generate a probability prediction sequence of equipment energy efficiency change;

[0158] Importance sampling is performed on the probability prediction sequence to obtain a plurality of energy efficiency prediction samples, and an energy efficiency evaluation index under different confidence levels is estimated for the plurality of energy efficiency prediction samples; a multi-criteria comprehensive evaluation model is constructed for the energy efficiency evaluation index to generate a maintenance benefit evaluation vector;

[0159] A time sequence causal relationship is established between the action decision result and the probability prediction sequence by using a graph structure model, the intervention effect of each maintenance action is calculated through a counterfactual framework, the direct influence and indirect influence corresponding to the intervention effect are quantified based on a structural equation, and the maintenance benefit evaluation vector is adaptively fused with the direct influence and indirect influence to obtain an action importance score;

[0160] A priority weight is assigned to the action decision result according to the action importance score to obtain a weighted decision result, and an optimization target considering time sequence constraints is constructed for the weighted decision result based on the probability prediction sequence to solve an optimal maintenance time sequence and generate a maintenance decision scheme.

[0161] For example, in combination with Figure 3The maintenance decision scheme generation flowchart based on the time sequence causal relationship is described as follows: the equipment energy efficiency indicators are extracted from the second feature data set, the time-frequency feature matrix is extracted by multi-scale sampling on the equipment energy efficiency indicators, and the multi-modal probability distribution of the equipment energy efficiency indicators is constructed based on the time-frequency feature matrix. Specifically, the energy efficiency indicators including equipment energy consumption rate, conversion efficiency, power factor, etc. are extracted from the second feature data set. Each energy efficiency indicator is sampled by using different time windows (such as 1 hour, 12 hours, 24 hours, 7 days) to form a feature matrix, and the time-frequency features in each window are extracted by using wavelet transform, and the multi-modal probability distribution of the time-frequency feature matrix is constructed by using Gaussian mixture model.

[0162] The action decision result and the multi-modal probability distribution are input into a multi-layer prediction unit to generate a probability prediction sequence of the equipment energy efficiency change. Specifically, the maintenance action decision result (such as replacing the filter, cleaning the heat exchanger) output by the time sequence decision model is encoded into a vector, which is connected with the multi-modal probability distribution parameters of the energy efficiency indicators and input into a multi-layer prediction unit composed of long short-term memory network (LSTM). The prediction unit contains 3 layers of LSTM, each layer has 128 neurons, and outputs the probability distribution of the equipment energy efficiency change in the future 30 days.

[0163] The probability prediction sequence is importance sampled to obtain a plurality of energy efficiency prediction samples, and the energy efficiency evaluation indicators under different confidence levels are estimated for the plurality of energy efficiency prediction samples. 1000 groups of energy efficiency change samples are extracted from the probability prediction sequence by using the Monte Carlo importance sampling method. For each sample, the energy efficiency evaluation indicators under 90%, 95% and 99% confidence levels are estimated, including average energy saving rate, peak power reduction ratio and system stability indicators. A multi-criteria comprehensive evaluation model is constructed for the energy efficiency evaluation indicators to generate a maintenance benefit evaluation vector. The weights of each energy efficiency indicator are determined by using the analytic hierarchy process to construct a comprehensive evaluation model. For example, for a certain equipment, the weight of the energy saving rate is 0.5, the weight of the peak power reduction ratio is 0.3, and the weight of the system stability is 0.2. The maintenance benefit evaluation vector is calculated as [0.72, 0.65, 0.81], which represents the short-term, medium-term and long-term maintenance benefit scores, respectively.

[0164] The time sequence causal relationship is established between the action decision result and the probability prediction sequence by using a graph structure model, the intervention effect of each maintenance action is calculated through a counterfactual framework, the direct influence and the indirect influence corresponding to the intervention effect are quantified based on a structural equation, a directed acyclic graph is used to represent the causal relationship between the maintenance action and the energy efficiency change, a node represents the maintenance action or the energy efficiency indicator, and an edge represents the causal influence. Through the counterfactual framework, the comparative experiment of "doing vs not doing" is performed on each maintenance action, and the intervention effect is calculated. The maintenance benefit evaluation vector is adaptively fused with the direct influence and the indirect influence to obtain an action importance score. An adaptive weight fusion algorithm is used to dynamically adjust the weights of the benefit evaluation vector, the direct influence and the indirect influence according to the current state of the system, and the importance score of each maintenance action is calculated.

[0165] According to the action importance score, a priority weight is assigned to each maintenance action to form a weighted decision result. An optimization objective function is constructed, considering the maintenance resource constraints (personnel, spare parts, time window) and the time sequence dependence constraints (some maintenance must be performed in a specific order). The genetic algorithm is used to solve the optimization problem to obtain the optimal maintenance time sequence. For example, for an industrial refrigeration system, the generated maintenance decision scheme is: on the first day, perform the detection of the water chiller unit (importance 0.95), on the third day, perform the cleaning of the cooling tower (importance 0.87), on the fifth day, perform the maintenance of the water pump (importance 0.82), on the seventh day, perform the inspection of the pipeline system (importance 0.76), and on the tenth day, perform the debugging of the end device (importance 0.68).

[0166] The present application extracts a time-frequency feature matrix through multi-scale sampling, constructs a multi-modal probability distribution of the energy efficiency indicator, generates a probability prediction sequence by using a multi-layer prediction unit, realizes maintenance benefit quantification by combining importance sampling and multi-criteria evaluation, establishes a time sequence causal relationship by using a graph structure model, analyzes the intervention effect of the maintenance action through a counterfactual framework, and calculates the action importance based on a structural equation and adaptive fusion, thereby significantly improving the scientificity and reliability of the maintenance decision scheme.

[0167] In a second aspect of the embodiment of the present application, an industrial equipment energy efficiency evaluation and maintenance decision system based on reinforcement learning is provided, comprising:

[0168] A first unit is configured to collect real-time running data of the industrial equipment as a first feature data set, perform feature extraction and principal component analysis on the first feature data set to obtain a second feature data set;

[0169] A second unit is configured to construct a maintenance time sequence decision model based on a deep reinforcement learning algorithm, extract time sequence features from the first feature data set and the second feature data set, and generate a state representation, and determine a maintenance decision action according to a preset maintenance action space;

[0170] The third unit is configured to update the maintenance timing decision model online by using a transfer learning method, and transfer a multi-source domain knowledge feature matrix to a target device field through feature mapping to update network parameters of the maintenance timing decision model.

[0171] The fourth unit is configured to analyze a device energy efficiency index based on an action decision result output by the maintenance timing decision model, and generate a maintenance decision scheme based on a timing causal relationship optimization solution.

[0172] The fifth unit is configured to execute the maintenance decision scheme, record time, operation and resource data in a maintenance operation process to form a maintenance process record, collect the first feature data set and the second feature data set after the maintenance is completed, input the maintenance process record and the feature data set after the maintenance as training samples into the maintenance timing decision model for online learning, and update model parameters.

[0173] In a third aspect, an electronic device is provided, including:

[0174] a processor;

[0175] a memory for storing processor-executable instructions;

[0176] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0177] In a fourth aspect, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0178] The present application can be a method, device, system and / or computer program product. The computer program product can include a computer-readable storage medium having computer-readable program instructions loaded thereon, which are used to execute various aspects of the present application.

[0179] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for industrial equipment energy efficiency assessment and maintenance decision making based on reinforcement learning, characterized in that, The method comprises the following steps: Collecting real-time operation data of industrial equipment as a first feature data set; Performing feature extraction and principal component analysis on the first feature data set to obtain a second feature data set; Building a maintenance time series decision model based on a deep reinforcement learning algorithm, extracting time series features from the first feature data set and the second feature data set, and generating state representation, determining maintenance decision actions according to a pre-set maintenance action space; Using a transfer learning method to update the maintenance time series decision model online, migrating multi-source domain knowledge feature matrices to the target equipment field through feature mapping, and updating the network parameters of the maintenance time series decision model; Based on the action decision result output by the maintenance time series decision model, analyzing the equipment energy efficiency index, and generating a maintenance decision scheme based on the time series causal relationship optimization solution; Executing the maintenance decision scheme, recording the time, operation and resource data during the maintenance operation process to form a maintenance process record, collecting the first feature data set and the second feature data set after the maintenance is completed, and inputting the maintenance process record and the feature data set after the maintenance as training samples into the maintenance time series decision model for online learning to update the model parameters; The method for building a maintenance time series decision model based on a deep reinforcement learning algorithm, extracting time series features from the first feature data set and the second feature data set, and generating state representation, determining maintenance decision actions according to a pre-set maintenance action space, comprises: Segmenting and aligning the first feature data set and the second feature data set based on timestamps and constructing an optimal time segmentation sequence, extracting device state time series feature vectors in each time sequence segment to generate a time series feature sequence; Based on the time series feature sequence, a hierarchical maintenance action space is constructed, and a Markov decision process is used to model the correspondence between the time series feature sequence and the maintenance action, and an action transition constraint matrix is constructed; After encoding the time series feature sequence using a bidirectional long short-term memory network, an initial time series state representation is extracted, the initial time series state representation is input into a multi-head attention layer to fuse state information of different time scales, and a time series state representation is obtained; Based on the time series state representation, a maintenance time series decision model is constructed; The time series state representation is input into a multi-layer time series convolution network, multi-scale features are extracted through dynamic hole rate, feature fusion is performed using bidirectional residual connection, and enhanced time series features are obtained; The enhanced time series features are input into a value network and an advantage network respectively to obtain a state value function and an action advantage function, the state value function and the action advantage function are combined to obtain a time series value score of each maintenance action, and a maintenance decision action is selected in combination with the action transition constraint matrix.

2. The method of claim 1, wherein, The method for performing feature extraction and principal component analysis on the first feature data set to obtain a second feature data set comprises: Performing data preprocessing on the first feature data set, calculating an anomaly score according to the distribution density of each data point in the first feature data set, and determining that a data point with a distribution density less than a pre-set threshold as an abnormal data point; A plurality of data points before and after the abnormal data point are selected to construct a sliding window, the median in the sliding window is calculated to replace the abnormal data, and preprocessed feature data is generated; Correlation analysis is performed on the preprocessed feature data, a time sequence correlation coefficient matrix of the feature data in the time dimension is calculated, and a spatial correlation coefficient matrix of the feature data in the spatial dimension is calculated; The time sequence correlation coefficient matrix and the spatial correlation coefficient matrix are superimposed according to a preset weight to generate a multi-dimensional feature correlation matrix; Based on the multi-dimensional feature correlation matrix, a feature contribution weight is calculated, the preprocessed feature data is divided into a key feature group and a secondary feature group according to the feature contribution weight, a principal component analysis method is used to extract a principal feature vector for the key feature group, and a local linear embedding method is used to extract a local feature vector for the secondary feature group; The principal feature vector and the local feature vector are adaptively fused to obtain a fused feature vector, and the fused feature vector is taken as a second feature data set.

3. The method of claim 1, wherein, The time sequence state representation is input into a multi-layer time sequence convolutional network, multi-scale features are extracted through a dynamic hole rate, feature fusion is performed through a bidirectional residual connection, and enhanced time sequence features are obtained, including: The time sequence state representation is subjected to multi-scale decomposition and different frequency band features are extracted, the importance of each frequency band feature is evaluated, the feature weight is dynamically adjusted based on the importance evaluation result, the weighted feature components are reconstructed through inverse transformation to obtain enhanced time sequence state representation; The enhanced time sequence state representation is input into a multi-layer time sequence convolutional network, an exponentially increasing hole rate sequence is set in each network, multi-receptive field local time sequence features are extracted through a hole convolution operation, and a bidirectional residual connection path is constructed between adjacent convolution layers, the local time sequence features extracted from the previous layer and the features of the current layer are adaptively selected and fused through a gating mechanism to obtain multi-scale local time sequence features; A query matrix, a key matrix and a value matrix are constructed based on the multi-scale local time sequence features, a scaling dot product operation is performed on the query matrix and the key matrix to calculate an attention score, and a feature weight is obtained after the attention score is subjected to mask processing; The feature is added with time sequence position information in a sinusoidal position encoding manner, the feature weight added with the position information is subjected to tensor multiplication operation with the value matrix to obtain a weighted feature; The weighted feature is input into a feedforward neural network and a jump connection is constructed with the original time sequence state representation to generate enhanced time sequence features.

4. The method of claim 1, wherein, The maintenance time sequence decision model is updated online by using a transfer learning method, multi-source domain knowledge feature matrices are transferred to a target device field through feature mapping, and network parameters of the maintenance time sequence decision model are updated, including: A maintenance time sequence decision model of a target device and historical maintenance data of other devices are obtained, a device similarity matrix is constructed based on the historical maintenance data by using a multi-view spectral clustering method, and a maintenance mode similarity matrix is calculated by combining a dynamic time warping algorithm; The device similarity matrix and the maintenance mode similarity matrix are adaptively fused to generate a multi-source domain knowledge feature matrix; The multi-source domain knowledge feature matrix is probabilistically encoded by a variational autoencoder, and the encoded feature sequence is input into a bidirectional gated recurrent unit network to extract long-range temporal dependence features, combined with a multi-head self-attention mechanism to capture dynamic associations between features, and a residual connection and layer normalization are used to generate source domain conditional feature distribution; The target domain feature distribution is calculated based on the maintenance timing decision model of the target device, the difference measure between the target domain feature distribution and the source domain conditional feature distribution is calculated based on the bulldozer distance, the difference measure and the multi-source domain knowledge feature matrix are weighted and combined to generate a source domain weight distribution vector; The source domain conditional feature distribution is input into a feature encoder, and a multi-scale convolution combined with a double attention mechanism is used to map features with the source domain weight distribution vector, and a weighted optimization target is constructed to update the parameters of the maintenance timing decision model.

5. The method of claim 4, wherein, The source domain conditional feature distribution is input into a feature encoder, and a multi-scale convolution combined with a double attention mechanism is used to map features with the source domain weight distribution vector, and a weighted optimization target is constructed to update the parameters of the maintenance timing decision model, including: The source domain conditional feature distribution is input into a multi-branch feature encoder to extract multi-granularity feature representations, and the multi-granularity feature representations are adaptively aggregated to generate source domain feature representations; The spatial attention weight and the channel attention weight are calculated for the source domain feature representations, and the spatial attention weight and the channel attention weight are tensor multiplied to obtain a joint attention map, and the multi-scale attention weight is generated based on the long-range dependence relationship of the joint attention map; The source domain feature representations and the multi-scale attention weight are concatenated and input into a coarse-grained generation network to obtain initial target domain features, and the initial target domain features and the source domain weight distribution vector are input into a fine-grained generation network for feature refinement to generate final target domain features; The final target domain features are calculated for discriminant scores at different feature levels, a multi-scale adversarial loss is constructed based on the discriminant scores, and the source domain feature representations are adaptively temperature-adjusted to obtain softened knowledge representations at corresponding feature levels; The distillation loss of the final target domain features and the softened knowledge representations at each level is calculated, the multi-scale adversarial loss and the distillation loss at each level are weighted and combined based on the source domain weight distribution vector, and the parameters of the feature transfer network are updated by an alternating optimization algorithm.

6. The method of claim 1, wherein, Based on the action decision result output by the maintenance timing decision model, the device energy efficiency indicators are analyzed, and a maintenance decision scheme is generated based on the timing causal relationship optimization solution, including: The device energy efficiency indicators are extracted from the second feature data set, and the device energy efficiency indicators are multi-scale sampled to extract a time-frequency feature matrix, and a multi-modal probability distribution of the device energy efficiency indicators is constructed based on the time-frequency feature matrix; The action decision result and the multi-modal probability distribution are input into a multi-layer prediction unit to generate a probability prediction sequence of device energy efficiency changes; The probability prediction sequence is importance sampled to obtain multiple energy efficiency prediction samples, and the energy efficiency evaluation indicators under different confidence levels are estimated based on the multiple energy efficiency prediction samples; A multi-criteria comprehensive evaluation model is constructed for the energy efficiency evaluation index to generate a maintenance benefit evaluation vector; A graph structure model is used to establish a time sequence causal relationship between the action decision result and the probability prediction sequence, the intervention effect of each maintenance action is calculated through a counterfactual framework, the direct influence and indirect influence corresponding to the intervention effect are quantified based on a structural equation, and the maintenance benefit evaluation vector, the direct influence and the indirect influence are adaptively fused to obtain an action importance score; According to the action importance score, a priority weight is assigned to the action decision result to obtain a weighted decision result, and an optimization objective considering time sequence constraints is constructed based on the weighted decision result and the probability prediction sequence to solve an optimal maintenance time sequence and generate a maintenance decision scheme.

7. A system for industrial equipment energy efficiency assessment and maintenance decision making based on reinforcement learning for implementing the method of any of the preceding claims 1-6, characterized in that, It comprises: A first unit is configured to collect real-time running data of an industrial equipment as a first feature data set; Feature extraction and principal component analysis are performed on the first feature data set to obtain a second feature data set; A second unit is configured to construct a maintenance time sequence decision model based on a deep reinforcement learning algorithm, extract time sequence features from the first feature data set and the second feature data set, and generate state representation, and determine a maintenance decision action according to a preset maintenance action space; A third unit is configured to update the maintenance time sequence decision model online by using a transfer learning method, and transfer a multi-source domain knowledge feature matrix to a target equipment field through feature mapping to update network parameters of the maintenance time sequence decision model; A fourth unit is configured to analyze equipment energy efficiency indicators based on an action decision result output by the maintenance time sequence decision model, and generate a maintenance decision scheme based on time sequence causal relationship optimization solution; A fifth unit is configured to execute the maintenance decision scheme, record time, operation and resource data during maintenance operation to form a maintenance process record, collect the first feature data set and the second feature data set after maintenance is completed, use the maintenance process record and the feature data set after maintenance as training samples, input the training samples into the maintenance time sequence decision model for online learning, and update model parameters.

8. An electronic device, comprising: It comprises: A processor; A memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 6.

Images