Power transmission and transformation project multi-physical-quantity sensing method and system based on ISSA improved CNN-LSTM model

Abstract

The invention discloses a power transmission and transformation project multi-physical-quantity sensing method and system based on an ISSA improved CNN-LSTM model, and the method comprises the steps: firstly obtaining original multi-modal data, carrying out the enhanced fusion of the data of each modal, and obtaining a fused multi-modal feature; secondly, optimizing hyper-parameters of the CNN-LSTM model based on a sparrow search algorithm of an improved mixing mechanism, improving a single-head attention mechanism and performing multi-head parallel computing, and constructing a multi-physical-quantity perception prediction model; and finally, training the model to realize prediction and perception of multiple physical quantities of the power transmission and transformation project. According to the method, on the basis of multi-modal learning, an ISSA algorithm is introduced to optimize a CNN-LSTM model, the complex relation of multi-physical-quantity data is analyzed in an all-around mode, the spatio-temporal data processing efficiency is improved, a single-head attention mechanism is improved, relative position information of sequence elements is accurately captured, the weight is dynamically adjusted, the internal structure of the data is mined, then a multi-head attention mechanism is introduced, and the multi-physical-quantity data processing efficiency is improved. And when the long sequence is processed, key time steps are automatically focused, redundant interference is reduced, and the sensing precision of the system is remarkably improved.

Description

Technical Field

[0001] This invention relates to monitoring methods for power transmission and transformation projects, belonging to the field of power transmission and transformation project monitoring, and particularly to a multi-physical quantity sensing method and system for power transmission and transformation projects based on an improved CNN-LSTM model using ISSA. Background Technology

[0002] In power transmission and transformation engineering construction, the monitoring and analysis of multiple physical quantity data plays a crucial role in ensuring the safe and stable operation of the project and optimizing the construction process. However, traditional monitoring methods, which rely on single sensors or manual recording, suffer from many shortcomings, such as insufficient real-time performance, data fragmentation, and low analysis efficiency, making it difficult to meet the needs of modern complex engineering projects.

[0003] With the continuous development of artificial intelligence technology, data-driven intelligent analysis methods are gradually emerging in the field of engineering monitoring, demonstrating significant advantages. In particular, novel models that integrate spatiotemporal features, such as models combining convolutional neural networks (CNN), long short-term memory networks (LSTM), and attention mechanisms, can deeply explore the correlation patterns between multi-dimensional data, providing an efficient and intelligent solution for the sensing of multiple physical quantities in power transmission and transformation projects. In existing technologies, some have proposed a deep learning model based on a dual-branch convolutional neural network structure and incorporating a multi-attention mechanism. This model can efficiently sense vibration data of transmission lines under different operating conditions. However, this model can only sense a single physical quantity and cannot perform multi-physical quantity sensing functions. Other technologies train convolutional neural networks and long short-term memory networks by inputting different modal information of the same object separately to fully learn the interdependence of information between different modalities. However, this method is not good in terms of the sensing accuracy of each modal data. There are also methods that predict the current line state based on the multi-source data features obtained from training. However, using only LSTM networks is still insufficient in multi-modal data fusion and extraction of complex spatiotemporal features, and has limited attention focusing ability for long-term series. Summary of the Invention

[0004] The purpose of this invention is to overcome the aforementioned defects and problems in the prior art and to provide a multi-physical quantity sensing method and system for power transmission and transformation engineering based on an improved CNN-LSTM model with high sensing accuracy.

[0005] To achieve the above objectives, the technical solution of this invention is: a multi-physical quantity sensing method for power transmission and transformation engineering based on an improved CNN-LSTM model using ISSA, comprising:

[0006] Acquire raw multimodal data of power transmission and transformation equipment; the raw multimodal data includes temperature, strain, vibration, and video stream;

[0007] Each modal data point in the original multimodal data is modally enhanced and then stitched together to obtain fused multimodal features. Among them, time-series translation processing is used for weak modal time-series data including temperature, strain, and vibration; and random data pruning is used for single strong modal data including video streams.

[0008] The sparrow search algorithm based on the improved hybrid mechanism optimizes the hyperparameters of the CNN-LSTM model; the single-head attention mechanism of the CNN-LSTM model is improved and multi-head parallel computation is performed to construct a multi-physical quantity perception prediction model; wherein, the improved hybrid mechanism consists of Circle mapping, dimension-wise inverse learning, Cauchy mutation, sine and cosine strategies, and inertial weights.

[0009] Multimodal features are integrated and input into the multi-physical quantity sensing and prediction model for training, so as to achieve predictive sensing of multiple physical quantities in power transmission and transformation projects.

[0010] The acquisition of fused multimodal features specifically includes:

[0011] For raw multimodal data Each modal data point in the dataset undergoes modal augmentation processing; specifically: time-series data of temperature, strain, and vibration are processed by time-series translation to obtain a time-series augmented dataset; and random cropping is applied to the video stream to obtain a random cropped augmented dataset.

[0012] The expression for the time-shifting process is as follows:

[0013] ;

[0014] in: This is a time-series augmented dataset after time-series shifting. For the first One set of raw multimodal data, For data offset, To ensure uniform distribution, The maximum value of the time series data;

[0015] The expression for the random data pruning process is as follows:

[0016] ;

[0017] ;

[0018] in: This is a dataset augmented by random pruning after data pruning. for Initial values ​​for dimensional clipping. for Initial values ​​for dimensional clipping. , This refers to the dimensions after cropping in the corresponding dimension. , This represents the original size for the corresponding dimension;

[0019] Based on the dataset after all modal augmentation processing, an augmented positive sample group is generated; for each modality... A perturbation strategy is used to generate partial negative samples, thus generating a perturbation negative sample group;

[0020] The expression for the enhanced positive sample group is as follows:

[0021] ;

[0022] in: For the first The dataset after modal augmentation;

[0023] The expression for the perturbed negative sample group is as follows:

[0024] ;

[0025] in: The union of all modal perturbation samples. For the first One negative sample, The number of negative samples generated;

[0026] Based on the enhanced positive sample group and the perturbed negative sample group, a contrastive learning loss function is introduced to maximize the similarity of positive samples and minimize the similarity of negative samples to obtain distinguishing features;

[0027] The expression for the contrastive learning loss function is as follows:

[0028] ;

[0029] in: To compare the learning loss function, To calculate the exponent, For fusion encoders, For temperature parameters, This is the initial sample;

[0030] Based on the perturbed negative sample group, the contribution of each mode is evaluated by comparing the values ​​calculated by the learning loss function, and the perturbation ratio is dynamically adjusted. The expression is as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] in: For the first The accuracy of modality discrimination. This represents the perturbation ratio of the negative samples generated after dynamic adjustment. Before generating negative samples, the first The proportion of each mode This represents the proportion of remaining samples before negative samples were generated. For the first One negative sample, The total number of negative samples. This represents the perturbation ratio of the remaining samples generated after dynamic adjustment.

[0035] Using the dynamically adjusted perturbation ratio as the gating weight, all modal features are fused to obtain fused multimodal features, the expression of which is as follows:

[0036] ;

[0037] ;

[0038] in: To fuse multimodal features, For the first The gating weights of each modality For the first Prediction results for each modality.

[0039] The total loss function of the multi-physical quantity sensing prediction model is expressed as follows:

[0040] ;

[0041] in: For the total loss function, To compare the learning loss function, To compare the weight coefficients of the learning loss function, This is the cross-entropy loss function.

[0042] The sparrow search algorithm based on the improved hybrid mechanism optimizes the hyperparameters of the CNN-LSTM model, specifically including:

[0043] The initial population is generated based on Circle mapping initialization, and its expression is as follows:

[0044] ;

[0045] in: This represents the current position of an individual in the population. For modulo operation, For random perturbations;

[0046] Based on dimension-wise backward learning, we find the backward solution for each dimension of the feasible solution for each individual in the initial population, and obtain the position of the current best individual. The expression is as follows:

[0047] ;

[0048] in: To obtain the optimal individual that conforms to the dimensional-wise backward learning, in the first dimension... The position of the dimension For the current optimal individual in the th... The position of the dimension , This represents the upper and lower bounds of the search space;

[0049] The position of the current best individual is perturbed based on Cauchy mutation, and its expression is as follows:

[0050] ;

[0051] in: For the optimal individual to follow the Cauchy mutation, the random number is generated in the first... The position of the dimension For random numbers that conform to the Cauchy mutation;

[0052] Alternating between dimension-wise inverse learning and Cauchy mutation perturbation, the position of the optimal individual is dynamically updated, and the optimal solution of the optimal individual is output. The expression is as follows:

[0053] ;

[0054] in: For the optimal individual after the update, in the th... The position of the dimension It is a random number;

[0055] Based on inertia weights and the periodicity of sine and cosine functions, the search range of other individuals in the initial population is dynamically adjusted to determine the optimal position of other individuals in the initial population, and the optimal solution of other individuals is output. The expression is as follows:

[0056] ;

[0057] ;

[0058] ;

[0059] in: for During the nth iteration The individual in the first The position of the dimension For inertial weights, Step size factor , A randomly generated vector. It is a random number. , The maximum and initial values ​​of the inertia weight are given. This represents the current iteration number;

[0060] The optimal solution of the best individual in the initial population and the optimal solution of other individuals are used as the hyperparameters of the CNN-LSTM model.

[0061] The improved CNN-LSTM model's single-head attention mechanism and multi-head parallel computation specifically refer to:

[0062] Based on the positional relationship between the query matrix and the key matrix and the attention weights, a single-head adaptive weighted attention mechanism with relative positional encoding is constructed, and its expression is as follows:

[0063] ;

[0064] in: For querying the matrix, The key matrix, For value matrices, Let be the dimension of the key matrix vector. This is a relative position encoding matrix. This is a learnable weight matrix.

[0065] By employing a single-head adaptive weighted attention mechanism that computes multiple relative positional codes in parallel, the relationships between different subspaces are captured. Its expression is as follows:

[0066] ;

[0067] ;

[0068] in: To output weights, , , The first Size , , Linear transformation weights.

[0069] A multi-physical quantity sensing system for power transmission and transformation engineering based on an improved CNN-LSTM model using ISSA, comprising:

[0070] The data acquisition unit is used to acquire raw multimodal data of the power transmission and transformation equipment; the raw multimodal data includes temperature, strain, vibration, and video stream.

[0071] The data augmentation and fusion unit is used to perform modal augmentation on each modal data in the original multimodal data and then stitch and fuse them to obtain fused multimodal features. Among them, time-series translation processing is used for weak modal time-series data including temperature, strain, and vibration; and random data pruning processing is used for single strong modal data including video streams.

[0072] The prediction model building unit is used to optimize the hyperparameters of the CNN-LSTM model based on the sparrow search algorithm with an improved hybrid mechanism; improve the single-head attention mechanism of the CNN-LSTM model and perform multi-head parallel computation to build a multi-physical quantity perception prediction model; wherein, the improved hybrid mechanism consists of Circle mapping, dimension-wise inverse learning, Cauchy mutation, sine and cosine strategies, and inertial weights.

[0073] The predictive sensing unit is used to train the multi-physical quantity sensing prediction model by inputting fused multi-modal features into it, so as to achieve predictive sensing of multiple physical quantities in power transmission and transformation projects.

[0074] The data enhancement and fusion unit performs data enhancement and fusion according to the following steps:

[0075] For raw multimodal data Each modal data point in the dataset undergoes modal augmentation processing; specifically: time-series data of temperature, strain, and vibration are processed by time-series translation to obtain a time-series augmented dataset; and random cropping is applied to the video stream to obtain a random cropped augmented dataset.

[0076] The expression for the time-shifting process is as follows:

[0077] ;

[0078] in: This is a time-series augmented dataset after time-series shifting. For the first One set of raw multimodal data, For data offset, To ensure uniform distribution, The maximum value of the time series data;

[0079] The expression for the random data pruning process is as follows:

[0080] ;

[0081] ;

[0082] in: This is a dataset augmented by random pruning after data pruning. for Initial values ​​for dimensional clipping. for Initial values ​​for dimensional clipping. , This refers to the dimensions after cropping in the corresponding dimension. , This represents the original size for the corresponding dimension;

[0083] Based on the dataset after all modal augmentation processing, an augmented positive sample group is generated; for each modality... A perturbation strategy is used to generate partial negative samples, thus generating a perturbation negative sample group;

[0084] The expression for the enhanced positive sample group is as follows:

[0085] ;

[0086] in: For the first The dataset after modal augmentation;

[0087] The expression for the perturbed negative sample group is as follows:

[0088] ;

[0089] in: The union of all modal perturbation samples. For the first One negative sample, The number of negative samples generated;

[0090] Based on the enhanced positive sample group and the perturbed negative sample group, a contrastive learning loss function is introduced to maximize the similarity of positive samples and minimize the similarity of negative samples to obtain distinguishing features;

[0091] The expression for the contrastive learning loss function is as follows:

[0092] ;

[0093] in: To compare the learning loss function, To calculate the exponent, For fusion encoders, For temperature parameters, This is the initial sample;

[0094] Based on the perturbed negative sample group, the contribution of each mode is evaluated by comparing the values ​​calculated by the learning loss function, and the perturbation ratio is dynamically adjusted. The expression is as follows:

[0095] ;

[0096] ;

[0097] ;

[0098] in: For the first The accuracy of modality discrimination. This represents the perturbation ratio of the negative samples generated after dynamic adjustment. Before generating negative samples, the first The proportion of each mode This represents the proportion of remaining samples before negative samples were generated. For the first One negative sample, The total number of negative samples. This represents the perturbation ratio of the remaining samples generated after dynamic adjustment.

[0099] Using the dynamically adjusted perturbation ratio as the gating weight, all modal features are fused to obtain fused multimodal features, the expression of which is as follows:

[0100] ;

[0101] ;

[0102] in: To fuse multimodal features, For the first The gating weights of each modality For the first Prediction results for each modality.

[0103] The total loss function of the multi-physical quantity sensing prediction model constructed by the prediction model construction unit is expressed as follows:

[0104] ;

[0105] in: For the total loss function, To compare the learning loss function, To compare the weight coefficients of the learning loss function, This is the cross-entropy loss function.

[0106] The prediction model building unit optimizes the hyperparameters of the CNN-LSTM model according to the following steps:

[0107] The initial population is generated based on Circle mapping initialization, and its expression is as follows:

[0108] ;

[0109] in: This represents the current position of an individual in the population. For modulo operation, For random perturbations;

[0110] Based on dimension-wise backward learning, we find the backward solution for each dimension of the feasible solution for each individual in the initial population, and obtain the position of the current best individual. The expression is as follows:

[0111] ;

[0112] in: To obtain the optimal individual that conforms to the dimensional-wise backward learning, in the first dimension... The position of the dimension For the current optimal individual in the th... The position of the dimension , This represents the upper and lower bounds of the search space;

[0113] The position of the current best individual is perturbed based on Cauchy mutation, and its expression is as follows:

[0114] ;

[0115] in: For the optimal individual to follow the Cauchy mutation, the random number is generated in the first... The position of the dimension For random numbers that conform to the Cauchy mutation;

[0116] Alternating between dimension-wise inverse learning and Cauchy mutation perturbation, the position of the optimal individual is dynamically updated, and the optimal solution of the optimal individual is output. The expression is as follows:

[0117] ;

[0118] in: For the optimal individual after the update, in the th... The position of the dimension It is a random number;

[0119] Based on inertia weights and the periodicity of sine and cosine functions, the search range of other individuals in the initial population is dynamically adjusted to determine the optimal position of other individuals in the initial population, and the optimal solution of other individuals is output. The expression is as follows:

[0120] ;

[0121] ;

[0122] ;

[0123] in: for During the nth iteration The individual in the first The position of the dimension For inertial weights, Step size factor , A randomly generated vector. It is a random number. , The maximum and initial values ​​of the inertia weight are given. This represents the current iteration number;

[0124] The optimal solution of the best individual in the initial population and the optimal solution of other individuals are used as the hyperparameters of the CNN-LSTM model.

[0125] The prediction model building unit improves the single-head attention mechanism of the CNN-LSTM model and performs multi-head parallel computation according to the following steps:

[0126] Based on the positional relationship between the query matrix and the key matrix and the attention weights, a single-head adaptive weighted attention mechanism with relative positional encoding is constructed, and its expression is as follows:

[0127] ;

[0128] in: For querying the matrix, The key matrix, For value matrices, Let be the dimension of the key matrix vector. This is a relative position encoding matrix. This is a learnable weight matrix.

[0129] By employing a single-head adaptive weighted attention mechanism that computes multiple relative positional codes in parallel, the relationships between different subspaces are captured. Its expression is as follows:

[0130] ;

[0131] ;

[0132] in: To output weights, , , The first Size , , Linear transformation weights.

[0133] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0134] This invention discloses a method and system for multi-physical quantity sensing in power transmission and transformation projects based on an improved CNN-LSTM model using ISSA. The method first acquires raw multimodal data, then performs modal enhancement on the data of each modality and merges them to obtain fused multimodal features. Next, it optimizes the hyperparameters of the CNN-LSTM model using a sparrow search algorithm with an improved hybrid mechanism, improves the single-head attention mechanism, and performs multi-head parallel computation to construct a multi-physical quantity sensing and prediction model. Finally, it inputs the fused multimodal features into the model for training, achieving the prediction and sensing of multiple physical quantities in power transmission and transformation projects. In application, this design is based on multimodal learning and incorporates the ISSA algorithm to optimize the CNN-LSTM model. The improvements enable the model to comprehensively and deeply analyze the intricate relationships between multiple physical quantity data, while significantly enhancing its processing efficiency for spatiotemporal data. Furthermore, by improving the single-head attention mechanism, it can accurately capture the relative position information of each element in the sequence and dynamically adjust the attention weights according to the actual situation, thereby more efficiently mining the inherent structure of the data. On this basis, a multi-head attention mechanism is further introduced, enabling the model to automatically filter and focus on the most informative time step when processing long sequence data, effectively reducing the interference of redundant information, greatly enhancing the model's ability to perceive and capture key information, and thus significantly improving the perception accuracy of the entire system. Attached Figure Description

[0135] Figure 1 This is a flowchart of the method of the present invention.

[0136] Figure 2 This is a comparative schematic diagram of the convergence curves of various algorithms in Embodiment 1 of the present invention.

[0137] Figure 3 This is a schematic diagram comparing the error distribution of various algorithms in Embodiment 1 of the present invention.

[0138] Figure 4 This is a system structure diagram of the present invention.

[0139] Figure 5 This is a structural diagram of the device of the present invention.

[0140] In the diagram: Data acquisition unit 1, data augmentation and fusion unit 2, prediction model construction unit 3, prediction perception unit 4, processor 5, memory 6, computer program code 61. Detailed Implementation

[0141] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0142] Example 1:

[0143] See Figure 1 A method for sensing multiple physical quantities in power transmission and transformation engineering based on an improved CNN-LSTM model using ISSA, comprising:

[0144] Acquire raw multimodal data of power transmission and transformation equipment; the raw multimodal data includes temperature, strain, vibration, and video stream;

[0145] This solution uses a variety of sensors, such as temperature sensors and strain sensors, to collect raw data from power transmission and transformation equipment in order to obtain rich information covering all aspects of the environment. The information provided by different sensors is complementary and can comprehensively reflect the environmental state, providing basic data support for subsequent multi-physical quantity analysis and modeling.

[0146] Raw data collected from multiple sensors often exhibit significant heterogeneity, manifested in structural differences in data format and physical meaning. Therefore, to eliminate the impact of such differences on subsequent analysis, this approach employs data preprocessing and Z-score normalization techniques to integrate the raw multimodal data. The specific implementation steps are as follows:

[0147] First, noise removal and missing value repair operations are performed. By using signal processing techniques such as median filtering and moving average, outliers and non-stationary fluctuations in the data are filtered out. Then, data repair strategies such as interpolation, mean filling, and regression filling are used to complete the missing data records, thereby ensuring data integrity and avoiding interference with subsequent analysis.

[0148] The cleaned data is then normalized. This scheme uses Z-score normalization to normalize temperature, strain, and other data to ensure the stability of model training. The expression for Z-score normalization is as follows:

[0149] ;

[0150] in: For normalized data; This is the original data; , These represent the mean and standard deviation of the original data, respectively. Preprocessing the original data improves its quality and consistency, providing a reliable foundation for subsequent model training and analysis.

[0151] Each modal data point in the original multimodal data is modally enhanced and then stitched together to obtain fused multimodal features. Among them, time-series translation processing is used for weak modal time-series data including temperature, strain, and vibration; and random data pruning is used for single strong modal data including video streams.

[0152] Traditional CNN-LSTM models extract spatial features from input data using convolutional neural networks (CNNs). The basic operation of CNNs is convolution, and its expression is as follows:

[0153] ;

[0154] in: For the convolution result, For the input features, For convolution kernel, For the first The bias value of each convolutional kernel. CNN gradually extracts high-order spatial features from the input data by stacking multiple convolutional layers and pooling layers. Convolutional operations preserve local spatial correlations, while pooling operations (such as max pooling or average pooling) are used to reduce dimensionality and computation. The spatial feature sequence extracted by CNN is input into LSTM to capture the dynamic information of time series.

[0155] The Long Short-Term Memory (LSTM) network model is a variant of the Recurrent Neural Network (RNN) that solves the gradient vanishing problem in RNNs. LSTM introduces three gates: the input gate, the forget gate, and the output gate, as well as memory cells with the same dimension as the hidden states to record additional information. The expression for the LSTM network model is as follows:

[0156] ;

[0157] in: , , These are the outputs of the forget gate, input gate, and output gate, respectively. Use the Sigmoid activation function; for The hidden layer input at each time step; for The hidden layer output at each moment; , , , , , These are the weight matrices for each variable; , , These are the bias terms for each variable.

[0158] Current time step memory cells Combined with the previous time step memory cells and current time step Candidate memory cells The information is controlled by a forget gate and an input gate, and its expression is as follows:

[0159] ;

[0160] ;

[0161] in: for Candidate memory cells for a given moment; , This is the weight matrix; For bias terms; for Memory cells for specific moments; for Memory cells of moments; time steps The hidden state is:

[0162] ;

[0163] in: for Output from the hidden layer at all times.

[0164] Based on the principles of the CNN-LSTM network model above, it can be seen that traditional multimodal learning, addressing the heterogeneity of multi-physical quantity data (such as temperature, strain, video streams, etc.), employs gating units to control the degree of fusion of features from different modalities. However, the improved multimodal learning approach in this scheme forces the model to focus on the complementary information of each modality within a multimodal tuple through a contrastive learning objective (TupleInfoNCE), rather than relying solely on a single strong modality (such as video streams), thereby improving the utilization rate of weak modalities (such as temperature, strain). Challenging negative samples are generated through perturbation to dynamically adjust the weight allocation of each modality, avoiding the limitations of manually setting gating parameters. Furthermore, a feature-level fusion method is used to unify the data from each modality into a single feature space after feature extraction. The specific steps are as follows:

[0165] Let the original multimodal data be... ,in For the first Data for each modality; features for each modality are extracted using a corresponding feature extraction function, the expression of which is as follows:

[0166] ;

[0167] in: For the first Features of each modality No. Features of each modality The feature extraction network.

[0168] For raw multimodal data Each modal data point in the dataset undergoes modal augmentation processing; specifically: time-series data of temperature, strain, and vibration are processed by time-series translation to obtain a time-series augmented dataset; and random cropping is applied to the video stream to obtain a random cropped augmented dataset.

[0169] The expression for the time-shifting process is as follows:

[0170] ;

[0171] in: This is a time-series augmented dataset after time-series shifting. For the first One set of raw multimodal data, For data offset, To ensure uniform distribution, The maximum value of the time series data;

[0172] The expression for the random data pruning process is as follows:

[0173] ;

[0174] ;

[0175] in: This is a dataset augmented by random pruning after data pruning. for Initial values ​​for dimensional clipping. for Initial values ​​for dimensional clipping. , This refers to the dimensions after cropping in the corresponding dimension. , This represents the original size for the corresponding dimension;

[0176] Based on the dataset after all modal augmentation processing, an augmented positive sample group is generated; for each modality... A perturbation strategy is used to generate partial negative samples, thus generating a perturbation negative sample group;

[0177] The expression for the enhanced positive sample group is as follows:

[0178] ;

[0179] in: For the first The dataset after modal augmentation;

[0180] For each mode A perturbation strategy is used to generate partial negative samples (e.g., replacing a certain modality with data from other scenarios), resulting in a perturbation negative sample group, the expression of which is as follows:

[0181] ;

[0182] in: The union of all modal perturbation samples. For the first One negative sample, This represents the number of negative samples generated.

[0183] For the generated positive and negative sample groups, a contrastive learning objective (TupleInfoNCE) is introduced to maximize the similarity of positive samples and minimize the similarity of negative samples. Since the samples depend on the fused encoder, the backpropagation of the contrastive loss updates the sub-encoder, thereby generating more robust and discriminative features, as detailed below:

[0184] Based on the enhanced positive sample group and the perturbed negative sample group, a contrastive learning loss function is introduced to maximize the similarity of positive samples and minimize the similarity of negative samples to obtain distinguishing features;

[0185] The expression for the contrastive learning loss function is as follows:

[0186] ;

[0187] in: To compare the learning loss function, To calculate the exponent, For fusion encoders, For temperature parameters, This is the initial sample;

[0188] The generated negative samples are evaluated based on the magnitude of the cross-modal discrimination task (such as predicting strain waveforms based on temperature characteristics) and the loss function value, and the perturbation ratio is dynamically adjusted, that is, the mode with lower accuracy is assigned a higher ratio, as expressed below:

[0189] ;

[0190] ;

[0191] ;

[0192] in: For the first The accuracy of modality discrimination. This represents the perturbation ratio of the negative samples generated after dynamic adjustment. Before generating negative samples, the first The proportion of each mode This represents the proportion of remaining samples before negative samples were generated. For the first One negative sample, The total number of negative samples. This represents the perturbation ratio of the remaining samples generated after dynamic adjustment.

[0193] Using the dynamically adjusted perturbation ratio as the gating weight, all modal features are fused to obtain fused multimodal features, the expression of which is as follows:

[0194] ;

[0195] ;

[0196] in: To fuse multimodal features, For the first The gating weights of each modality For the first The prediction results for each mode. In this scheme... The proportions are calculated by using the adjusted proportion of negative samples and the proportion of undisturbed samples as the weights of each gate output to ensure the contribution of each mode.

[0197] Based on fusion features By outputting device state predictions through a fully connected layer and jointly optimizing the contrast loss and prediction loss, the system balances self-supervised learning with task-specific training to improve prediction accuracy. Multimodal learning fusion can adjust the impact of each modality on the final result based on its contribution level, thereby improving the overall prediction performance and perception results.

[0198] Furthermore, the total loss function of the multi-physical quantity sensing prediction model is expressed as follows:

[0199] ;

[0200] in: For the total loss function, To compare the learning loss function, To compare the weight coefficients of the learning loss function, The cross-entropy loss function;

[0201] The sparrow search algorithm based on the improved hybrid mechanism optimizes the hyperparameters of the CNN-LSTM model; the single-head attention mechanism of the CNN-LSTM model is improved and multi-head parallel computation is performed to construct a multi-physical quantity perception prediction model; among which, the improved hybrid mechanism includes Circle mapping, dimension-wise inverse learning, Cauchy mutation, sine and cosine strategies, and inertial weights.

[0202] Traditional SSA algorithms are highly sensitive to parameter selection; improper parameter settings can lead to performance degradation. To address the shortcomings of traditional SSA, the improved ISSA algorithm significantly enhances performance through the following methods: by combining Circle mapping, inverse learning, Cauchy mutation, sine / cosine strategies, and inertial weights, the ISSA algorithm achieves a good balance between global search and local exploitation, improving the robustness and accuracy of the CNN-LSTM model. The ISSA algorithm specifically includes the following:

[0203] An initial population is generated by initializing using Circle mapping. Leveraging the periodicity and ergodicity of Circle mapping, a more evenly distributed and diverse initial population is generated, improving population diversity and quality and preventing traditional random initialization algorithms from prematurely converging to local optima. A highly diverse initial population provides a richer perturbation basis for subsequent mixing mechanisms, preventing mutations from getting trapped in local optima. Its expression is as follows:

[0204] ;

[0205] in: This represents the current position of an individual in the population. For modulo operation, For random perturbations;

[0206] By employing dimension-wise inverse learning, reverse solutions are found for feasible solutions in each dimension, thereby reducing the risk of getting trapped in local optima. Dimension-wise inverse learning is an improved strategy designed to address the dimensional characteristics of the solution space in optimization algorithms. It expands the search scope and reduces the risk of getting trapped in local optima by independently generating reverse solutions for the current solution in each dimension. Its core idea is to leverage the independence of each dimension in the solution space to perform targeted optimization for each dimension, rather than performing a global inverse operation on the entire solution vector. This allows for a more accurate exploration of potential optimal solutions in multidimensional space. Its expression is as follows:

[0207] ;

[0208] in: To obtain the optimal individual that conforms to the dimensional-wise backward learning, in the first dimension... The position of the dimension For the current optimal individual in the th... The position of the dimension , The upper and lower bounds of the search space are defined by these bounds. However, when the optimal solution generated by reverse learning has already fallen into a local optimum, the reverse solution may still be in the same local neighborhood, causing bidirectional search to become a back-and-forth oscillation within the local neighborhood, making it impossible to achieve long-distance global exploration.

[0209] The Cauchy mutation perturbs the position of the current best individual to enhance global search capabilities. Its expression is as follows:

[0210] ;

[0211] in: For the optimal individual to follow the Cauchy mutation, the random number is generated in the first... The position of the dimension The random number is subject to Cauchy mutation; however, with only Cauchy mutation, the algorithm becomes blindly searching due to over-reliance on randomness, especially when the population distribution is scattered in the early stages of iteration. The perturbed solution may be far away from the high-potential region, resulting in a significant decrease in convergence speed.

[0212] To address the limitations of using only a single method, this scheme combines the two approaches to form a hybrid mechanism. It alternates between dimension-wise inverse learning and Cauchy mutation perturbation to dynamically update the position of the optimal individual and output the optimal solution for the optimal individual. The expression is as follows:

[0213] ;

[0214] in: For the optimal individual after the update, in the th... The position of the dimension The random number is used to determine whether to perform inverse learning or Cauchy mutation. When the random number is less than 0.5, inverse learning is performed dimension by dimension; when the random number is greater than 0.5, Cauchy mutation is performed to generate a perturbation solution. The random number is chosen to be 0.5 because it is the midpoint of the interval, ensuring that inverse learning and Cauchy mutation have equal trigger probabilities in a single iteration. This hybrid strategy avoids the over-dominance of one strategy and ensures that the two mechanisms dynamically complement each other during the search process.

[0215] Finally, to balance global search and local exploitation, and to improve convergence speed and accuracy, based on inertial weights and combined with the periodicity of sine and cosine functions, the search range of other individuals in the initial population is dynamically adjusted to determine the optimal position of other individuals in the initial population, and the optimal solution of other individuals is output. The expression is as follows:

[0216] ;

[0217] ;

[0218] ;

[0219] in: for During the nth iteration The individual in the first The position of the dimension; Inertial weight; Step size factor; , A vector randomly generated within the range [0,1], with the same dimension as the individual population. This is a random number used to determine whether the current individual is in the exploration or development phase; , These are the maximum and initial values ​​of the inertia weight; This represents the current iteration number;

[0220] It is important to note that the sine-cosine strategy and inertia weight work together with the aforementioned hybrid mechanism. The purpose is to dynamically adjust the search range through inertia weight, and combine the periodicity of the sine and cosine functions to achieve a smooth transition from global exploration to local development. This mechanism uses a threshold to determine whether the current stage is exploration or development. If the inertia weight is large, the optimal solution generated by the hybrid mechanism guides the population to gather in high-potential areas. When the inertia weight is small, the high-frequency oscillation of the sine and cosine functions performs a refined search of the neighborhood of the optimal solution, and accelerates convergence by utilizing the historical optimization results of the hybrid mechanism.

[0221] The optimal solution of the best individual in the initial population and the optimal solution of other individuals are used as the hyperparameters of the CNN-LSTM model.

[0222] The improved ISSA algorithm differs from the individual use of various strategies. Instead, it uses a hybrid mechanism for dynamic scheduling and multi-dimensional asynchronous optimization, forming a dynamic whole from Circle mapping, inverse learning, Cauchy mutation, inertia weights, and sine and cosine strategies. Circle mapping provides the search starting point for the hybrid mechanism. By combining inverse learning and Cauchy mutation, a hybrid mechanism is formed, enabling bidirectional search and covering a wider solution space. This allows the overall algorithm to satisfy both exploration and development needs. Furthermore, by combining inertia weights and sine and cosine strategies, the search strategy is adaptively adjusted based on the optimal solution state output by the hybrid mechanism. The hybrid mechanism as a whole achieves the above effects.

[0223] The hyperparameters of the CNN-LSTM model are optimized by combining the improved Sparrow Search Algorithm (ISSA), and the traditional single-head attention mechanism is improved by focusing on key time step information through a multi-head attention mechanism to reduce redundant interference. The specific steps are as follows:

[0224] S1. Initialize the population: Determine the range of hyperparameters such as learning rate, number of hidden layer nodes, regularization coefficient, and convolutional kernel length, and generate the initial population using Circle mapping;

[0225] S2. Calculate the fitness value: Calculate the fitness value of each individual in the population based on the prediction error of the CNN-LSTM model;

[0226] S3. Update population position: Update the position of the best individual through a mixing mechanism, and update the positions of other individuals through a sine-cosine strategy and inertia weight;

[0227] S4. Iterative optimization: Repeat the above steps until the maximum number of iterations is reached or the convergence condition is met.

[0228] S5. Output the optimal solution: Use the optimal solution of the best individual in the initial population and the optimal solutions of other individuals as hyperparameters of the CNN-LSTM model to construct a multi-physical quantity perception prediction model.

[0229] The CNN-LSTM model optimized by ISSA can more effectively capture the dependency characteristics of spatiotemporal data, improving prediction accuracy and robustness. The improved model exhibits stronger robustness and accuracy in multi-physical quantity sensing tasks, providing reliable technical support for the comprehensive analysis and sensing of multi-physical quantity data in power transmission and transformation engineering.

[0230] In this approach, to improve the model's ability to perceive key information data, a multi-head attention mechanism is used to improve CNN-LSTM.

[0231] Traditional single-head attention mechanisms focus on key information by weighted summation of the query, key, and value; the expression for the traditional single-head attention mechanism is as follows:

[0232] ;

[0233] in: For query matrix; The key matrix; It is a value matrix; This represents the dimension of the key matrix vector, used for scaling to avoid excessively large values.

[0234] Traditional single-head attention mechanisms only consider the similarity between the query and the key, neglecting their positional relationship and the dynamic adjustment of attention weights based on the input data. Therefore, this invention constructs a relative positional encoding single-head adaptive weighted attention mechanism based on the positional relationship and attention weights between the query and key matrices, as expressed below:

[0235] ;

[0236] in: For querying the matrix, The key matrix, For value matrices, Let be the dimension of the key matrix vector. This is a relative position encoding matrix. This is a learnable weight matrix.

[0237] Multi-head attention mechanisms play a crucial role in long sequence data processing, especially in addressing the information concentration in time-series data. Traditional LSTM may face the "long-term dependency problem," while multi-head attention mechanisms help the model focus on key information within long sequences. The multi-head attention mechanism in this scheme computes multiple single attention heads in parallel, allowing the model to capture different levels of dependencies in different subspaces. Its expression is as follows:

[0238] ;

[0239] ;

[0240] in: To output weights, , , The first Size , , The linear transformation weights. For spatiotemporal data (such as time series, video streams, etc.), the multi-head attention mechanism can focus between different time steps, automatically selecting and focusing on the most informative time step.

[0241] The multimodal features are fused and input into the multi-physical quantity sensing and prediction model for training, in order to predict and sense multiple physical quantities in power transmission and transformation projects. The steps are as follows:

[0242] A1. Collect environmental information based on sensors, and clean and normalize the raw data;

[0243] A2. Extract features from the raw data of different modalities by passing them through independent CNN modules to obtain the multimodal features of the raw data;

[0244] A3. Introduce a gating unit to fuse features from different modalities using a splicing operation to obtain fused features;

[0245] A4. Input the fused features into the ISSA-CNN-LSTM model to obtain the spatiotemporal features of the data;

[0246] A5. Based on the improved CNN-LSTM model's single-head attention mechanism and multi-head parallel computation multi-head attention mechanism, it further focuses on the key time steps of the data's spatiotemporal features to obtain key time-weighted features;

[0247] A6. The key time-weighted features are processed through the fully connected layer of the multi-physical quantity perception prediction model to output the multi-physical quantity perception results.

[0248] See Figure 2 In this embodiment, the convergence curves of various algorithms are compared. Figure 2 The convergence curves of three algorithms, CNN-LSTM, SSA-CNN-LSTM and ISSA-CNN-LSTM, are compared. The horizontal axis represents the number of iterations and the vertical axis represents the optimal fitness value.

[0249] Among them, CNN-LSTM has a slower convergence speed and larger fitness value fluctuations, indicating that it is not stable enough during the optimization process and is prone to getting trapped in local optima. Compared with CNN-LSTM, SSA-CNN-LSTM has a better convergence speed and smaller fitness value fluctuations, showing better stability, but some fluctuations still exist, indicating that there is still room for improvement in its balance between global search and local exploitation. ISSA-CNN-LSTM has the fastest convergence speed and the smallest fitness value fluctuations, almost staying near the minimum value, indicating that it performs well in the optimization process and can effectively balance global search and local exploitation, avoiding getting trapped in local optima.

[0250] See Figure 3 , Figure 3 This represents the error distribution of various algorithms across different physical quantities. Figure 3 As can be seen from the figure, ISSA-CNN-LSTM has the lowest MSE across all physical quantities, indicating that it has the highest prediction accuracy. SSA-CNN-LSTM outperforms the original CNN-LSTM but is slightly inferior to ISSA-CNN-LSTM. This figure visually demonstrates the advantages of the improved SSA algorithm, i.e., the ISSA algorithm, in signal processing and model prediction.

[0251] Example 2:

[0252] See Figure 4 A multi-physical quantity sensing system for power transmission and transformation engineering based on an improved CNN-LSTM model using ISSA, comprising:

[0253] Data acquisition unit 1 is used to acquire raw multimodal data of power transmission and transformation equipment; the raw multimodal data includes temperature, strain, vibration, and video stream;

[0254] Data augmentation and fusion unit 2 is used to perform modal augmentation on each modal data in the original multimodal data and then stitch and fuse them to obtain fused multimodal features; among them, time-series translation processing is used for weak modal time-series data including temperature, strain, and vibration; and random data pruning processing is used for single strong modal data including video streams.

[0255] The prediction model building unit 3 is used to optimize the hyperparameters of the CNN-LSTM model based on the sparrow search algorithm with an improved hybrid mechanism; improve the single-head attention mechanism of the CNN-LSTM model and perform multi-head parallel computation to build a multi-physical quantity perception prediction model; wherein, the improved hybrid mechanism consists of Circle mapping, dimension-wise inverse learning, Cauchy mutation, sine and cosine strategies, and inertial weights.

[0256] Predictive sensing unit 4 is used to train the multi-physical quantity sensing prediction model by inputting the fused multi-modal features into it, so as to achieve predictive sensing of multiple physical quantities in power transmission and transformation projects.

[0257] Furthermore, the steps for implementing specific functions of the data acquisition unit 1, data enhancement and fusion unit 2, prediction model construction unit 3, and prediction perception unit 4 are described in the corresponding sections of Embodiment 1.

[0258] Example 3:

[0259] See Figure 5 A multi-physical quantity sensing device for power transmission and transformation engineering based on an improved CNN-LSTM model using ISSA, the device comprising a processor 5 and a memory 6;

[0260] The memory 6 is used to store computer program code 61 and transmit the computer program code 61 to the processor 5;

[0261] The processor 5 is used to execute the multi-physical quantity sensing method for power transmission and transformation engineering based on the ISSA improved CNN-LSTM model described in Embodiment 1 according to the instructions in the computer program code 61.

[0262] This embodiment also includes a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed on a computer, the method for sensing multiple physical quantities in power transmission and transformation engineering based on the ISSA-improved CNN-LSTM model described in Embodiment 1 is implemented.

[0263] Generally, the computer instructions for implementing the method of the present invention can be carried on any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media can include any computer-readable medium except for the signal itself, which is temporarily propagating.

[0264] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EKROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0265] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smarttalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. In particular, Python, suitable for neural network computation, and platform frameworks such as TensorFlow and PyTorch can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer or to an external computer (e.g., via the Internet using an Internet service provider) through any type of network, including a local area network (LAN) or a wide area network (WAN).

[0266] For details regarding the aforementioned equipment and non-transitory computer-readable storage media, please refer to the specific description of a multi-physical quantity sensing method for power transmission and transformation engineering based on an improved CNN-LSTM model using ISSA and its beneficial effects; further details will not be provided here.

[0267] Although embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for sensing multiple physical quantities in power transmission and transformation engineering based on an improved CNN-LSTM model using ISSA, characterized in that... include: Acquire raw multimodal data of power transmission and transformation equipment; The raw multimodal data includes temperature, strain, vibration, and video stream; Each modal data point in the original multimodal data is modally enhanced and then stitched together to obtain fused multimodal features. Among them, time-series translation processing is used for weak modal time-series data including temperature, strain, and vibration; and random data pruning is used for single strong modal data including video streams. The sparrow search algorithm based on the improved hybrid mechanism optimizes the hyperparameters of the CNN-LSTM model; the single-head attention mechanism of the CNN-LSTM model is improved and multi-head parallel computation is performed to construct a multi-physical quantity perception prediction model; wherein, the improved hybrid mechanism consists of Circle mapping, dimension-wise inverse learning, Cauchy mutation, sine and cosine strategies, and inertial weights. Multimodal features are integrated and input into the multi-physical quantity sensing and prediction model for training, so as to achieve predictive sensing of multiple physical quantities in power transmission and transformation projects.

2. The method for sensing multiple physical quantities in power transmission and transformation engineering based on the ISSA-improved CNN-LSTM model according to claim 1, characterized in that: The acquisition of fused multimodal features specifically includes: For raw multimodal data Each modal data point in the dataset undergoes modal augmentation processing; specifically: time-series data of temperature, strain, and vibration are processed by time-series translation to obtain a time-series augmented dataset; and random cropping is applied to the video stream to obtain a random cropped augmented dataset. The expression for the time-shifting process is as follows: ； in: This is a time-series augmented dataset after time-series shifting. For the first One set of raw multimodal data, This is the data offset. To ensure uniform distribution, The maximum value of the time series data; The expression for the random data pruning process is as follows: ； ； in: This is a dataset augmented by random pruning after data pruning. for The initial value for dimensional clipping. for The initial value for dimensional clipping. , This refers to the dimensions after cropping in the corresponding dimension. , This represents the original size for the corresponding dimension; Based on the dataset after all modal augmentation processing, an augmented positive sample group is generated; for each modality... A perturbation strategy is used to generate partial negative samples, thus generating a perturbation negative sample group; The expression for the enhanced positive sample group is as follows: ； in: For the first The dataset after modal augmentation; The expression for the perturbed negative sample group is as follows: ； in: The union of all modal perturbation samples. For the first One negative sample, The number of negative samples generated; Based on the enhanced positive sample group and the perturbed negative sample group, a contrastive learning loss function is introduced to maximize the similarity of positive samples and minimize the similarity of negative samples to obtain distinguishing features; The expression for the contrastive learning loss function is as follows: ； in: To compare the learning loss function, To calculate the exponent, For fusion encoders, For temperature parameters, This is the initial sample; Based on the perturbed negative sample group, the contribution of each mode is evaluated by comparing the values ​​calculated by the learning loss function, and the perturbation ratio is dynamically adjusted. The expression is as follows: ； ； ； in: For the first The accuracy of modality discrimination. This represents the perturbation ratio of the negative samples generated after dynamic adjustment. Before generating negative samples, the first The proportion of each mode This represents the proportion of remaining samples before negative samples were generated. For the first One negative sample, The total number of negative samples. This represents the perturbation ratio of the remaining samples generated after dynamic adjustment. Using the dynamically adjusted perturbation ratio as the gating weight, all modal features are fused to obtain fused multimodal features, the expression of which is as follows: ； ； in: To fuse multimodal features, For the first The gating weights of each modality For the first Prediction results for each modality.

3. The method for sensing multiple physical quantities in power transmission and transformation engineering based on the ISSA-improved CNN-LSTM model according to claim 2, characterized in that: The total loss function of the multi-physical quantity sensing prediction model is expressed as follows: ； in: For the total loss function, To compare the learning loss function, To compare the weight coefficients of the learning loss function, This is the cross-entropy loss function.

4. The method for sensing multiple physical quantities in power transmission and transformation engineering based on the ISSA-improved CNN-LSTM model according to claim 1, characterized in that: The sparrow search algorithm based on the improved hybrid mechanism optimizes the hyperparameters of the CNN-LSTM model, specifically including: The initial population is generated based on Circle mapping initialization, and its expression is as follows: ； in: This represents the current position of an individual in the population. For modulo operation, For random perturbations; Based on dimension-wise backward learning, we find the backward solution for each dimension of the feasible solution for each individual in the initial population, and obtain the position of the current best individual. The expression is as follows: ； in: To obtain the optimal individual that conforms to the dimensional-wise backward learning, in the first dimension... The position of the dimension For the current optimal individual in the th... The position of the dimension , This represents the upper and lower bounds of the search space; The position of the current best individual is perturbed based on Cauchy mutation, and its expression is as follows: ； in: For the optimal individual to follow the Cauchy mutation, the random number is generated in the first... The position of the dimension For random numbers that conform to the Cauchy mutation; Alternating between dimension-wise inverse learning and Cauchy mutation perturbation, the position of the optimal individual is dynamically updated, and the optimal solution of the optimal individual is output. The expression is as follows: ； in: For the optimal individual after the update, in the th... The position of the dimension It is a random number; Based on inertia weights and the periodicity of sine and cosine functions, the search range of other individuals in the initial population is dynamically adjusted to determine the optimal position of other individuals in the initial population, and the optimal solution of other individuals is output. The expression is as follows: ； ； ； in: for During the nth iteration The individual in the first The position of the dimension For inertial weights, Step size factor , A randomly generated vector. It is a random number. , The maximum and initial values ​​of the inertia weight are given. This represents the current iteration number; The optimal solution of the best individual in the initial population and the optimal solution of other individuals are used as the hyperparameters of the CNN-LSTM model.

5. The method for sensing multiple physical quantities in power transmission and transformation engineering based on the ISSA-improved CNN-LSTM model according to claim 1, characterized in that: The improved CNN-LSTM model's single-head attention mechanism and multi-head parallel computation specifically refer to: Based on the positional relationship between the query matrix and the key matrix and the attention weights, a single-head adaptive weighted attention mechanism with relative positional encoding is constructed, and its expression is as follows: ； in: For querying the matrix, The key matrix, For value matrices, Let be the dimension of the key matrix vector. This is a relative position encoding matrix. The weight matrix is ​​a learnable matrix; By employing a single-head adaptive weighted attention mechanism that computes multiple relative positional codes in parallel, the relationships between different subspaces are captured. Its expression is as follows: ； ； in: To output weights, , , The first Size , , Linear transformation weights.

6. A multi-physical quantity sensing system for power transmission and transformation engineering based on an improved CNN-LSTM model using ISSA, characterized in that, include: The data acquisition unit (1) is used to acquire the raw multimodal data of the power transmission and transformation equipment; The raw multimodal data includes temperature, strain, vibration, and video stream; The data augmentation and fusion unit (2) is used to perform modal augmentation on each modal data in the original multimodal data and splice and fuse them to obtain fused multimodal features; among them, the weak modal time series data including temperature, strain and vibration are processed by time series translation; and the single strong modal data including video streams are processed by random data cropping. The prediction model building unit (3) is used to optimize the hyperparameters of the CNN-LSTM model based on the sparrow search algorithm with the improved hybrid mechanism; improve the single-head attention mechanism of the CNN-LSTM model and perform multi-head parallel computation to build a multi-physical quantity perception prediction model; wherein, the improved hybrid mechanism consists of Circle mapping, dimension-wise inverse learning, Cauchy mutation, sine and cosine strategies, and inertial weights; The prediction sensing unit (4) is used to train the multi-physical quantity sensing prediction model by inputting the fused multi-modal features into the multi-physical quantity sensing prediction model, so as to realize the prediction sensing of multi-physical quantities in the power transmission and transformation project.

7. The multi-physical quantity sensing system for power transmission and transformation engineering based on the ISSA improved CNN-LSTM model according to claim 6, characterized in that: The data enhancement and fusion unit (2) performs data enhancement and fusion according to the following steps: For raw multimodal data Each modal data point in the dataset undergoes modal augmentation processing; specifically: time-series data of temperature, strain, and vibration are processed by time-series translation to obtain a time-series augmented dataset; and random cropping is applied to the video stream to obtain a random cropped augmented dataset. The expression for the time-shifting process is as follows: ； in: This is a time-series augmented dataset after time-series shifting. For the first One set of raw multimodal data, For data offset, To ensure uniform distribution, The maximum value of the time series data; The expression for the random data pruning process is as follows: ； ； in: This is a dataset augmented by random pruning after data pruning. for The initial value for dimensional clipping. for The initial value for dimensional clipping. , This refers to the dimensions after cropping in the corresponding dimension. , This represents the original size for the corresponding dimension; Based on the dataset after all modal augmentation processing, an augmented positive sample group is generated; for each modality... A perturbation strategy is used to generate partial negative samples, thus generating a perturbation negative sample group; The expression for the enhanced positive sample group is as follows: ； in: For the first The dataset after modal augmentation; The expression for the perturbed negative sample group is as follows: ； in: The union of all modal perturbation samples. For the first One negative sample, The number of negative samples generated; Based on the enhanced positive sample group and the perturbed negative sample group, a contrastive learning loss function is introduced to maximize the similarity of positive samples and minimize the similarity of negative samples to obtain distinguishing features; The expression for the contrastive learning loss function is as follows: ； in: To compare the learning loss function, To calculate the exponent, For fusion encoders, For temperature parameters, This is the initial sample; Based on the perturbed negative sample group, the contribution of each mode is evaluated by comparing the values ​​calculated by the learning loss function, and the perturbation ratio is dynamically adjusted. The expression is as follows: ； ； ； in: For the first The accuracy of modality discrimination. This represents the perturbation ratio of the negative samples generated after dynamic adjustment. Before generating negative samples, the first The proportion of each mode This represents the proportion of remaining samples before negative samples were generated. For the first One negative sample, The total number of negative samples. This represents the perturbation ratio of the remaining samples generated after dynamic adjustment. Using the dynamically adjusted perturbation ratio as the gating weight, all modal features are fused to obtain fused multimodal features, the expression of which is as follows: ； ； in: To fuse multimodal features, For the first The gating weights of each modality For the first Prediction results for each modality.

8. The multi-physical quantity sensing system for power transmission and transformation engineering based on the ISSA improved CNN-LSTM model according to claim 7, characterized in that: The total loss function of the multi-physical quantity sensing prediction model constructed by the prediction model construction unit (3) is expressed as follows: ； in: For the total loss function, To compare the learning loss function, To compare the weight coefficients of the learning loss function, This is the cross-entropy loss function.

9. The multi-physical quantity sensing system for power transmission and transformation engineering based on the ISSA-improved CNN-LSTM model according to claim 6, characterized in that: The prediction model building unit (3) optimizes the hyperparameters of the CNN-LSTM model according to the following steps: The initial population is generated based on Circle mapping initialization, and its expression is as follows: ； in: This represents the current position of an individual in the population. For modulo operation, For random perturbations; Based on dimension-wise backward learning, we find the backward solution for each dimension of the feasible solution for each individual in the initial population, and obtain the position of the current best individual. The expression is as follows: ； in: To obtain the optimal individual that conforms to the dimensional-wise backward learning, in the first dimension... The position of the dimension For the current optimal individual in the th... The position of the dimension , This represents the upper and lower bounds of the search space; The position of the current best individual is perturbed based on Cauchy mutation, and its expression is as follows: ； in: For the optimal individual to follow the Cauchy mutation, the random number is generated in the first... The position of the dimension For random numbers that conform to the Cauchy mutation; Alternating between dimension-wise inverse learning and Cauchy mutation perturbation, the position of the optimal individual is dynamically updated, and the optimal solution of the optimal individual is output. The expression is as follows: ； in: For the optimal individual after the update, in the th... The position of the dimension It is a random number; Based on inertia weights and the periodicity of sine and cosine functions, the search range of other individuals in the initial population is dynamically adjusted to determine the optimal position of other individuals in the initial population, and the optimal solution of other individuals is output. The expression is as follows: ； ； ； in: for During the nth iteration The individual in the first The position of the dimension For inertial weights, Step size factor , A randomly generated vector. It is a random number. , The maximum and initial values ​​of the inertia weight are given. This represents the current iteration number; The optimal solution of the best individual in the initial population and the optimal solution of other individuals are used as the hyperparameters of the CNN-LSTM model.

10. The multi-physical quantity sensing system for power transmission and transformation engineering based on the ISSA improved CNN-LSTM model according to claim 6, characterized in that: The prediction model building unit (3) improves the single-head attention mechanism of the CNN-LSTM model and performs multi-head parallel computation according to the following steps: Based on the positional relationship between the query matrix and the key matrix and the attention weights, a single-head adaptive weighted attention mechanism with relative positional encoding is constructed, and its expression is as follows: ； in: For querying the matrix, The key matrix, For value matrices, Let be the dimension of the key matrix vector. This is a relative position encoding matrix. The weight matrix is ​​a learnable matrix; By employing a single-head adaptive weighted attention mechanism that computes multiple relative positional codes in parallel, the relationships between different subspaces are captured. Its expression is as follows: ； ； in: To output weights, , , The first Size , , Linear transformation weights.

Images