A multi-source perception abnormal state discrimination method and system for a hydropower station sending line

Abstract

The application discloses a multi-source perception abnormal state discrimination method and system for a hydropower station sending line, and the content is as follows: a modal relationship graph is constructed by taking calibrated image features, calibrated electrical features and weather working condition codes as nodes, and inter-modal relationship weights are dynamically generated according to the weather working condition codes; information propagation and dynamic fusion are carried out based on the modal relationship graph, and a hydropower station electrical equipment abnormal state category, a category probability and a modal contribution weight are output. Through cross-modal relationship reconstruction and dynamic fusion, the application avoids the defects of traditional static fusion in which a fixed modal contribution structure is adopted under different working conditions, and improves the discrimination stability under complex weather conditions.

Description

Technical Field

[0001] This invention relates to the fields of intelligent inspection of electrical equipment in hydropower stations, multimodal fusion, and abnormal state identification under complex meteorological conditions. Specifically, it is a multi-source sensing abnormal state identification method and system for transmission lines of hydropower stations. Background Technology

[0002] Hydropower stations and their electrical equipment operate long-term in complex natural and hydraulic engineering environments. The equipment area encompasses powerhouses, dam areas, switchyards, corridors, and outdoor ancillary facilities. Anomaly detection is influenced not only by the equipment's physical condition but also by meteorological conditions such as rain, fog, snow, low temperatures, high humidity, strong winds, and low light levels. Complex meteorological conditions can weaken edge, texture, and icing details in inspection images and cause fluctuations in electrical response amplitude, stage drift, and modal relationships. With the development of drone inspections, online monitoring, micro-meteorological monitoring, and meteorological disaster early warning systems, hydropower station equipment operation and maintenance scenarios now possess the data foundation to jointly use imagery, electrical, and meteorological information for anomaly detection.

[0003] Existing image discrimination methods typically rely on color, texture, shape, and spatial location in inspection images to identify abnormal conditions such as conductive connection anomalies, foreign objects in equipment, equipment icing, and icing of insulators or bushings. These methods are effective under clear, high-visibility conditions and with good image quality. However, in rain, fog, snow, low-light conditions, or under conditions of camera shake caused by strong winds, image edges are weakened, icing textures are interfered with by background brightness, and the outlines of foreign objects may be confused with the edges of hydraulic structures, plant frameworks, or equipment shadows. Relying solely on images can easily lead to misjudgments.

[0004] Existing electrical discrimination methods can use voltage, current, zero-sequence quantities, negative-sequence quantities, frequency, power, or protection recordings to reflect the dynamic response before and after an abnormal state occurs. However, meteorological conditions can also alter electrical response characteristics. Low temperatures and icing conditions may change the operating state of conductive connections, bushings, and outdoor primary equipment; high humidity conditions may affect insulation conditions; and strong winds may change the contact state between foreign objects and live parts. If electrical characteristics do not consider these meteorological conditions, the model may misinterpret weather-induced response drift as differences in anomaly categories.

[0005] Existing image-electrical joint discrimination methods often statically stitch image features and electrical features together, fuse them with fixed weights, or fuse them with shared attention, typically assuming that the contribution relationships between different modalities are relatively stable. However, image reliability and electrical response strength change dynamically under different meteorological conditions. For example, image texture weakens under foggy and rainy conditions, reducing the reliability of the visual branch; under low temperature and rain / snow conditions, the changes in icing-related electrical responses are more pronounced, increasing the importance of the electrical branch and meteorological constraints; under clear conditions, image evidence is more direct, and meteorological constraints should not excessively interfere with visual discrimination.

[0006] If meteorological information is only used as an external additional feature input to the final classifier, it is difficult to participate in image representation calibration, electrical representation calibration, and cross-modal relationship reconstruction. Static stitching, fixed gating, or shared attention methods cannot explicitly respond to differences in meteorological conditions between samples and are prone to discrimination fluctuations on complex weather subsets. Therefore, a method for identifying abnormal states of electrical equipment in hydropower stations based on the fusion of inspection images, electrical time series, and meteorological conditions is needed. This method transforms meteorological information from a common additional variable into a condition constraint variable, enabling it to participate in both single-modal representation calibration and dynamic reconstruction of cross-modal relationships. Summary of the Invention

[0007] The purpose of this invention is to provide a method for identifying multi-source sensing anomalies in the transmission lines of hydropower stations, the steps of which include:

[0008] Obtain inspection images, electrical response time series and meteorological status data corresponding to abnormal events of electrical equipment in the same hydropower station, establish event-level multi-source sample alignment relationship, and thus generate multimodal joint samples;

[0009] Meteorological condition data is encoded to obtain meteorological operating condition codes;

[0010] Feature extraction was performed on the inspection images and electrical response time series to obtain the image anomaly characteristics and electrical anomaly evolution characteristics;

[0011] Image scale calibration parameters and image offset calibration parameters are generated based on meteorological condition coding. The abnormal image features are then calibrated according to meteorological conditions to obtain image calibration features based on meteorological conditions.

[0012] Electrical scale calibration parameters and electrical offset calibration parameters are generated based on meteorological condition codes. The electrical anomaly evolution characteristics are then calibrated according to meteorological conditions to obtain electrical calibration characteristics based on meteorological conditions.

[0013] A modal relationship graph is constructed using calibrated image features, calibrated electrical features, and meteorological condition codes as nodes, and inter-modal relationship weights are dynamically generated based on the meteorological condition codes.

[0014] Information is disseminated and dynamically fused based on modal relationship graphs, outputting the abnormal state categories, category probabilities, and modal contribution weights of hydropower station electrical equipment.

[0015] Furthermore, the multimodal joint sample is represented as:

[0016]

[0017] in, Let i represent the i-th joint sample. Represents inspection images. Indicates the electrical response timing. Represents meteorological status data. Indicates an abnormal status label;

[0018] The meteorological data includes one or more of the following: weather type, ambient temperature, relative humidity, wind speed, wind direction, precipitation intensity, visibility, air pressure, dew point temperature, or icing risk level.

[0019] Furthermore, the image anomaly features are extracted using an image discrimination model, an object detection model, a visual encoder, or a visual basic model;

[0020] The image anomaly features include at least one or more of the following: busbar or cable continuity changes, foreign object suspension patterns, icing textures, objects attached to icing, insulator or bushing areas, plant equipment background, and anomaly area responses.

[0021] The electrical anomaly evolution features are extracted by an electrical timing encoder; the electrical timing encoder includes TSMixer, temporal convolutional network, recurrent neural network, gated recurrent unit network, long short-term memory network, and Transformer timing network.

[0022] Furthermore, the meteorological condition code is obtained through numerical normalization, category embedding, and multilayer perceptron mapping;

[0023] Among them, continuous meteorological variables are obtained by minimax normalization, standardization or quantile normalization to obtain numerical meteorological vectors;

[0024] Discrete weather categories are obtained as category meteorological vectors through unique thermal coding or embedding coding;

[0025] The meteorological condition codes are as follows:

[0026]

[0027] in, For discrete weather category variables, As a continuous meteorological variable, For category embedding functions, For numerical normalization functions, For meteorological coding network, This is a code for meteorological conditions.

[0028] Furthermore, the image calibration features based on meteorological conditions are as follows:

[0029]

[0030] in, and These are image scale calibration parameters and image offset calibration parameters, respectively. Network for generating image calibration parameters For image abnormality features, For the calibrated image features, This represents element-wise multiplication;

[0031] The electrical calibration characteristics based on meteorological conditions are as follows:

[0032]

[0033] in, and These are the electrical dimension calibration parameters and the electrical offset calibration parameters, respectively. For generating electrical calibration parameters network, Characteristics of electrical anomaly evolution. For calibrated electrical characteristics.

[0034] Furthermore, the image scale calibration parameters and image offset calibration parameters Meteorological condition coding The calibration parameters of the input image are generated by the network.

[0035] The image calibration parameter generation network includes an input projection layer, a nonlinear mapping layer, and scale parameter output branches and offset parameter output branches, which are encoded using meteorological conditions. As input, output respectively and The and It is learned along with the network parameters during the joint training process of classification loss, modality consistency constraint, relation sparsity constraint and meteorological calibration stability constraint;

[0036] The electrical dimension calibration parameters and electrical offset calibration parameters Meteorological condition coding The electrical calibration parameters are input into the network to generate the desired result.

[0037] The electrical calibration parameter generation network includes an input projection layer, a nonlinear mapping layer, and scale parameter output branches and offset parameter output branches, which are encoded using meteorological conditions F. m As input, output respectively and The and It is obtained through joint learning by the electrical timing encoder, dynamic fusion discrimination module and classifier during the training process of historical samples.

[0038] Furthermore, the modal relationship diagram is shown below:

[0039]

[0040] in, This is a modal relationship diagram. For a set of nodes, It is a relation matrix. A relation generation network; the relation matrix Used to describe the dynamic co-strength among image modes, electrical modes, and meteorological modes.

[0041] Furthermore, the steps for outputting the abnormal state categories, category probabilities, and modal contribution weights of the hydropower station's electrical equipment include:

[0042] Based on the modal relationship graph, graph propagation, interactive attention, or gating weighting are performed to obtain multimodal fusion features H. H is then input into a classifier, which outputs the probability of abnormal state categories. ;

[0043] The classifier includes a feature normalization layer, a fully connected mapping layer, a non-linear activation layer, and a softmax output layer, used to map the multimodal fusion feature H to the probabilities of each abnormal state category; the modality contribution weights The gating weights and / or normalized relation matrix A of the dynamic fusion discrimination module are obtained, representing the contributions of image mode, electrical mode, and meteorological mode to the final discrimination result, respectively. .

[0044] Abnormal state category probability As shown below:

[0045]

[0046] Where H represents the multimodal fusion feature, Interaction functions include graph propagation, interactive attention, gated weighting, and feature product. This represents the probability of an abnormal state category.

[0047] Furthermore, the classifier is trained using historical data; the training objective is the classification loss. Modal consistency constraints sparse relation constraint and meteorological calibration stability constraints Minimum;

[0048] The total losses are shown below:

[0049]

[0050] in, , and These are the weighting coefficients.

[0051] An abnormal state discrimination system for electrical equipment in a hydropower station based on the method described above, which integrates inspection images, electrical time series and meteorological conditions, includes a multi-source information acquisition module, an event-level alignment module, a meteorological condition coding module, an image feature extraction module, an electrical feature extraction module, an image sign calibration module, an electrical evolution calibration module, a cross-modal relationship reconstruction module, a dynamic fusion discrimination module and an abnormal state output module.

[0052] The multi-source information acquisition module is used to acquire inspection images, electrical response timing, meteorological status data, and abnormal status labels;

[0053] The event-level alignment module is used to establish event-level correspondences between the three types of data;

[0054] The meteorological condition coding module is used to output meteorological condition codes;

[0055] The image feature extraction module is used to output the features of abnormal signs in the image;

[0056] The electrical feature extraction module is used to output electrical anomaly evolution features;

[0057] The image feature calibration module and the electrical evolution calibration module are used to adjust image features and electrical features respectively according to meteorological condition codes;

[0058] The cross-modal relationship reconstruction module is used to construct modal relationship graphs and generate relationship matrices;

[0059] The dynamic fusion discrimination module is used to output multimodal fusion features;

[0060] The abnormal status output module is used to output the abnormal status category, category probability, modal contribution weight, and meteorological risk warning.

[0061] This invention transforms meteorological information from external auxiliary variables into operational constraints that participate in the discrimination process, enabling the model to dynamically adjust image representation, electrical representation, and modal synergy relationships according to different weather conditions. Through image anomaly calibration based on meteorological conditions, it alleviates representation mismatch problems caused by image edge weakening, texture degradation, and enhanced background interference under conditions such as rain, fog, snow, and low illumination. Through electrical anomaly evolution calibration based on meteorological conditions, it alleviates discrimination instability caused by fluctuations in electrical response amplitude and stage drift under conditions such as high humidity, low temperature, and strong wind. Through cross-modal relationship reconstruction and dynamic fusion, it avoids the shortcomings of traditional static fusion which uses a fixed modal contribution structure under different operational conditions, thus improving discrimination stability under complex meteorological conditions. Attached Figure Description

[0062] Figure 1 This is a schematic diagram illustrating the influence relationship between image, electrical, and meteorological multi-source information under complex meteorological conditions;

[0063] Figure 2 This is a flowchart of meteorological-guided image anomaly calibration and electrical anomaly evolution calibration.

[0064] Figure 3 It is a meteorological-guided cross-modal relationship reconstruction and dynamic fusion discrimination structure diagram. Detailed Implementation

[0065] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0066] Example 1:

[0067] See Figures 1 to 3 A method for identifying multi-source sensing anomalies in hydropower station transmission lines, comprising the following steps:

[0068] Obtain inspection images, electrical response time series and meteorological status data corresponding to abnormal events of electrical equipment in the same hydropower station, establish event-level multi-source sample alignment relationship, and thus generate multimodal joint samples;

[0069] Meteorological condition data is encoded to obtain meteorological operating condition codes;

[0070] Feature extraction was performed on the inspection images and electrical response time series to obtain the image anomaly characteristics and electrical anomaly evolution characteristics;

[0071] Image scale calibration parameters and image offset calibration parameters are generated based on meteorological condition coding. The abnormal image features are then calibrated according to meteorological conditions to obtain image calibration features based on meteorological conditions.

[0072] Electrical scale calibration parameters and electrical offset calibration parameters are generated based on meteorological condition codes. The electrical anomaly evolution characteristics are then calibrated according to meteorological conditions to obtain electrical calibration characteristics based on meteorological conditions.

[0073] A modal relationship graph is constructed using calibrated image features, calibrated electrical features, and meteorological condition codes as nodes, and inter-modal relationship weights are dynamically generated based on the meteorological condition codes.

[0074] Information is disseminated and dynamically fused based on modal relationship graphs, outputting the abnormal state categories, category probabilities, and modal contribution weights of hydropower station electrical equipment.

[0075] Example 2:

[0076] A method for identifying multi-source sensing anomalies in hydropower station transmission lines, with the same technical content as in Embodiment 1, further wherein the multimodal joint sample is represented as follows:

[0077]

[0078] in, Let i represent the i-th joint sample. Represents inspection images. Indicates the electrical response timing. Represents meteorological status data. Indicates an abnormal status label;

[0079] The meteorological data includes one or more of the following: weather type, ambient temperature, relative humidity, wind speed, wind direction, precipitation intensity, visibility, air pressure, dew point temperature, or icing risk level.

[0080] Example 3:

[0081] A method for identifying multi-source sensing anomalies in the transmission lines of hydropower stations, with the same technical content as any one of embodiments 1-2, further wherein the image anomaly features are extracted by an image discrimination model, a target detection model, a visual encoder, or a visual basic model;

[0082] The image anomaly features include at least one or more of the following: busbar or cable continuity changes, foreign object suspension patterns, icing textures, objects attached to icing, insulator or bushing areas, plant equipment background, and anomaly area responses.

[0083] The electrical anomaly evolution features are extracted by an electrical timing encoder; the electrical timing encoder includes TSMixer, temporal convolutional network, recurrent neural network, gated recurrent unit network, long short-term memory network, and Transformer timing network.

[0084] TSMixer is a temporal modeling architecture based on multilayer perceptrons. Its core approach is to alternate between temporal mixing and feature mixing: temporal mixing applies an MLP along the time axis to each feature channel individually to capture temporal dependencies; feature mixing applies an MLP along the feature axis to each time step individually to learn the interactions between channels.

[0085] TSMixer consists of N Mixer blocks stacked together, each block containing:

[0086] Temporal hybrid MLP: The transposed feature sequence is processed using a shared two-layer fully connected network and residual connections.

[0087] Feature Hybrid MLP: Uses a shared two-layer fully connected network and residual connections for the original feature dimensions.

[0088] Each MLP is preceded by LayerNorm and followed by Dropout and residual connections.

[0089] Finally, a fixed-length encoding is obtained through global average pooling (along the time dimension), or the sequence output is preserved.

[0090] Temporal convolutional networks consist of stacked residual blocks, each containing:

[0091] Two layers of causal dilated convolution (to ensure temporal causality and expand the receptive field), with a kernel size of k and an inflation factor that grows exponentially with each layer.

[0092] Recurrent neural networks (RNNs) use a hidden state `ht` to update cyclically, with each time step sharing the same weights. The encoder can output the hidden state of the last time step as a global encoding, or it can use a bidirectional RNN to concatenate the last state.

[0093] Gated recurrent unit networks introduce update gates and reset gates, mitigating gradient vanishing with fewer gating structures.

[0094] The Long Short-Term Memory (LSTM) network includes forgetting gates, input gates, output gates, and cell states, which can effectively maintain long-term memory.

[0095] Example 4:

[0096] A method for identifying multi-source sensing anomalies in hydropower station transmission lines, with the same technical content as any one of embodiments 1-3, further wherein the meteorological condition code is obtained through numerical normalization, category embedding and multilayer perceptron mapping;

[0097] Among them, continuous meteorological variables are obtained by minimax normalization, standardization or quantile normalization to obtain numerical meteorological vectors;

[0098] Discrete weather categories are obtained as category meteorological vectors through unique thermal coding or embedding coding;

[0099] The meteorological condition codes are as follows:

[0100]

[0101] in, For discrete weather category variables, As a continuous meteorological variable, For category embedding functions, For numerical normalization functions, For meteorological coding network, This is a code for meteorological conditions.

[0102] Example 5:

[0103] A method for identifying multi-source sensing anomalies in hydropower station transmission lines, with technical content identical to any one of embodiments 1-4, further comprising the following image calibration features based on meteorological conditions:

[0104]

[0105] in, and These are image scale calibration parameters and image offset calibration parameters, respectively. Network for generating image calibration parameters For image abnormality features, For the calibrated image features, This represents element-wise multiplication;

[0106] The electrical calibration characteristics based on meteorological conditions are as follows:

[0107]

[0108] in, and These are the electrical dimension calibration parameters and the electrical offset calibration parameters, respectively. For generating electrical calibration parameters network, Characteristics of electrical anomaly evolution. For calibrated electrical characteristics.

[0109] Example 6:

[0110] A method for identifying multi-source sensing anomalies in hydropower station transmission lines, with technical content identical to any one of embodiments 1-5, further comprising the following: the image scale calibration parameters... and image offset calibration parameters Meteorological condition coding The calibration parameters of the input image are generated by the network.

[0111] The image calibration parameter generation network includes an input projection layer, a nonlinear mapping layer, and scale parameter output branches and offset parameter output branches, which are encoded using meteorological conditions. As input, output respectively and The and It is learned along with the network parameters during the joint training process of classification loss, modality consistency constraint, relation sparsity constraint and meteorological calibration stability constraint;

[0112] The electrical dimension calibration parameters and electrical offset calibration parameters Meteorological condition coding The electrical calibration parameters are input into the network to generate the desired result.

[0113] The electrical calibration parameter generation network includes an input projection layer, a nonlinear mapping layer, and scale parameter output branches and offset parameter output branches, which are encoded using meteorological conditions F. m As input, output respectively and The and It is obtained through joint learning by the electrical timing encoder, dynamic fusion discrimination module and classifier during the training process of historical samples.

[0114] Example 7:

[0115] A method for identifying multi-source sensing anomalies in hydropower station transmission lines, with technical content identical to any one of embodiments 1-6, further wherein the modal relationship diagram is shown below:

[0116]

[0117] in, This is a modal relationship diagram. For a set of nodes, It is a relation matrix. A relation generation network; the relation matrix Used to describe the dynamic co-strength among image modes, electrical modes, and meteorological modes.

[0118] Example 8:

[0119] A method for identifying multi-source sensing anomaly states of hydropower station transmission lines, with technical content identical to any one of embodiments 1-7, further comprising the steps of outputting the anomaly state category, category probability, and modal contribution weight of the hydropower station's electrical equipment, including:

[0120] Based on the modal relationship graph, graph propagation, interactive attention, or gating weighting are performed to obtain multimodal fusion features H. H is then input into a classifier, which outputs the probability of abnormal state categories. The classifier includes a feature normalization layer, a fully connected mapping layer, a non-linear activation layer, and a softmax output layer, used to map the multimodal fusion feature H into the probabilities of each abnormal state category; the modality contribution weights The gating weights and / or normalized relation matrix A of the dynamic fusion discrimination module are obtained, representing the contributions of image mode, electrical mode, and meteorological mode to the final discrimination result, respectively. .

[0121] Abnormal state category probability As shown below:

[0122]

[0123] Where H represents the multimodal fusion feature, Interaction functions include graph propagation, interactive attention, gated weighting, and feature product. This represents the probability of an abnormal state category.

[0124] Example 9:

[0125] A method for identifying multi-source sensing anomalies in hydropower station transmission lines, with technical content identical to any one of embodiments 1-8, further wherein the classifier is trained using historical data; the training objective is the classification loss. Modal consistency constraints sparse relation constraint and meteorological calibration stability constraints Minimum;

[0126] The total losses are shown below:

[0127]

[0128] in, , and These are the weighting coefficients.

[0129] Example 10:

[0130] A method for identifying multi-source sensing anomalies in hydropower station transmission lines includes the following:

[0131] The system acquires inspection images, electrical response timing sequences, and meteorological status data corresponding to abnormal events of electrical equipment in the same hydropower station. The inspection images are used to characterize the external visible signs of abnormal states of electrical equipment in the hydropower station. The electrical response timing sequences are used to characterize one or more dynamic responses of voltage, current, zero-sequence quantity, negative-sequence quantity, frequency, power, or protection recording quantity before and after the occurrence of abnormal states of electrical equipment in the hydropower station. The meteorological status data are used to characterize the environmental conditions such as weather type, temperature, humidity, wind, rain, fog, snow, and icing risk when the abnormal event occurs.

[0132] Based on the abnormal event number, power plant name, equipment name, equipment area, data collection timestamp, fault trigger time, inspection task number or maintenance record number, establish an event-level alignment relationship between inspection images, electrical response timing and meteorological status data, and combine the three types of data and abnormal status labels corresponding to the same abnormal event or the same equipment area in the same time window to form a multimodal joint sample;

[0133] Meteorological state data is encoded to obtain meteorological condition codes. These codes are obtained by fusing normalized representations of continuous meteorological variables with embedded representations of discrete weather categories, and are used to describe the meteorological conditions of the current sample.

[0134] Image features are extracted from the inspection images to obtain image anomaly features. These image anomaly features are used to characterize busbars or cables, insulators, bushings, foreign objects in equipment, icing patterns, abnormal area locations, equipment attachment objects, and the spatial relationship with the inspection background.

[0135] Electrical response timing is resampled, missing values ​​are repaired, outliers are handled, amplitude is normalized, local response is fused, and key intervals are enhanced to obtain electrical anomaly evolution characteristics. These electrical anomaly evolution characteristics are used to characterize the dynamic response law of the abnormal state in the fault triggering, development, and stabilization stages.

[0136] Image scale calibration parameters and image offset calibration parameters are generated based on meteorological condition codes. Based on the image scale calibration parameters and image offset calibration parameters, the abnormal image features are calibrated according to meteorological conditions to obtain the calibrated image features.

[0137] Electrical scale calibration parameters and electrical offset calibration parameters are generated based on meteorological condition codes. The electrical anomaly evolution characteristics are then calibrated according to meteorological conditions based on the electrical scale calibration parameters and electrical offset calibration parameters to obtain the calibrated electrical characteristics.

[0138] A modal relationship graph is constructed using calibrated image features, calibrated electrical features, and meteorological condition codes as nodes. Image-electrical relationship weights, image-meteorological relationship weights, and electrical-meteorological relationship weights are dynamically generated based on the meteorological condition codes.

[0139] Based on the modal relationship graph, graph propagation, interactive attention, gated weighting, or feature product interaction are performed to obtain multimodal fusion features. Based on the multimodal fusion features, the abnormal state categories, category probabilities, and contribution weights of image modality, electrical modality, and meteorological modality of hydropower station electrical equipment are output.

[0140] The meteorological status data shall include at least one or more of the following: weather category, ambient temperature, relative humidity, wind speed, wind direction, precipitation intensity, visibility, air pressure, dew point temperature, icing risk level, estimated ice thickness, or historical meteorological risk level; the weather category shall include at least sunny, rain, fog, snow, low temperature, high humidity, strong wind, low illumination, or mixed weather conditions; the abnormal status category shall include at least abnormal conductive connection parts, foreign objects in equipment, equipment icing, and icing of insulators or bushings.

[0141] The multimodal joint sample is represented as: ,in This represents the inspection image corresponding to the i-th abnormal event. This represents the electrical response timing corresponding to the i-th abnormal event. This represents the meteorological status data corresponding to the i-th abnormal event. The abnormal status label is used to indicate the event-level alignment relationship. The establishment of the event-level alignment relationship includes: using the abnormal event number, equipment area, equipment number, collection timestamp, and fault trigger time as indexes to determine whether the inspection image, electrical response sequence, and meteorological status data correspond to the same abnormal event; when the collection times of the three types of data are inconsistent, a time tolerance is set with the abnormal event occurrence window as the center, and the data within the time tolerance and with consistent equipment areas are determined as candidate joint samples. A matching score is generated based on time distance, regional consistency, and consistency of operation and maintenance records, and the data group with the highest matching score is selected as the multimodal joint sample.

[0142] The meteorological condition code is obtained through numerical normalization, category embedding, and multilayer perceptron mapping. Continuous meteorological variables are processed using min-max normalization, standardization, or quantile normalization to obtain numerical meteorological vectors. Discrete weather categories are processed using one-thermal coding or embedding coding to obtain category meteorological vectors. The numerical meteorological vectors and category meteorological vectors are concatenated and input into the meteorological coding network to obtain the meteorological condition code. The meteorological condition code It serves not only as input to the final classifier, but also as a condition for generating image calibration parameters, electrical calibration parameters, and modal relationship weights.

[0143] The image anomaly features are extracted by an image discrimination model, a target detection model, a visual encoder, or a visual basic model. Before extracting the image anomaly features, the inspection image is processed by one or more of the following: equipment area focusing and cropping, scale normalization, low visibility enhancement, background response suppression, or anomaly area response extraction. The image anomaly features include at least one or more of the following: busbar or cable continuity changes, foreign object hanging morphology, icing texture, icing attachment objects, insulator or bushing area, plant equipment background, and anomaly area response.

[0144] The electrical anomaly evolution features are extracted by an electrical timing encoder; the electrical timing encoder is a TSMixer, a temporal convolutional network, a recurrent neural network, a gated recurrent unit network, a long short-term memory network, a Transformer timing network, or other deep timing encoders; preferably, the electrical timing encoder simultaneously models the coupling relationship between voltage, current, zero-sequence quantity, negative-sequence quantity, frequency, and power in the time dimension and variable channel dimension, and outputs the electrical anomaly evolution features. .

[0145] The image scale calibration parameters and image offset calibration parameters Meteorological condition coding The calibration parameters of the input image are generated by the network to obtain the calibrated image features. satisfy ,in For image abnormality features, This indicates element-wise multiplication; the image scale calibration parameter is used to adjust the contribution of different dimensions of image features according to working conditions such as rain, fog, snow, low light and strong wind; the image offset calibration parameter is used to compensate for feature offset caused by image edge weakening, texture degradation, background brightness changes and shooting shake caused by meteorological conditions.

[0146] Furthermore, the image calibration parameter generation network It consists of a linear projection layer, a ReLU or GELU nonlinear activation layer, a normalization layer, and two parallel output branches, which generate image scale calibration parameters respectively. and image offset calibration parameters ; and It is driven by meteorological condition coding and automatically learned through backpropagation under the constraint of total loss L.

[0147] The electrical dimension calibration parameters and electrical offset calibration parameters Meteorological condition coding The electrical characteristics are obtained by generating a network from the input electrical calibration parameters. satisfy ,in The electrical anomaly evolution characteristics are defined by the electrical scale calibration parameters, which are used to adjust the contributions of different dimensions of electrical characteristics according to operating conditions such as low temperature, high humidity, strong wind, rain, snow and icing risk. The electrical offset calibration parameters are used to compensate for the response amplitude drift, stage position drift and channel contribution changes caused by meteorological conditions.

[0148] Furthermore, the electrical calibration parameter generation network It consists of a linear projection layer, a nonlinear activation layer, a normalization layer, and two parallel output branches. The two output branches generate electrical scale calibration parameters, respectively. and electrical offset calibration parameters ; and It is obtained by back-optimization of the loss function during the joint training of the electrical timing encoder, the cross-modal relationship reconstruction module, and the classifier.

[0149] The modality diagram uses calibrated image features calibrated electrical characteristics and meteorological condition codes As nodes, the meteorological conditions are coded. The input relation generation network yields relation matrix A, which describes the coordination strength between image mode, electrical mode, and meteorological mode under different meteorological conditions. Specifically, under clear and high visibility conditions, the image-meteorological relation weight decreases while the image-electrical relation weight remains stable. Under rain, fog, or low illumination conditions, the image mode reliability weight decreases while the electrical-meteorological relation weight increases. Under low temperature, rain, and snow conditions, the relation weight of the meteorological mode to icing-related anomalies increases.

[0150] The dynamic fusion discrimination includes: performing graph propagation, interactive attention, or gated weighting based on the modality graph to obtain a multimodal fusion feature H, and inputting H into a classifier to output the probability of abnormal state categories. Training objectives include classification loss. Modal consistency constraints sparse relation constraint and meteorological calibration stability constraints The total loss satisfies ,in , and The weighting coefficients are used to ensure that different modal features of the same abnormal event have the same orientation in the discrimination space. The relation sparsity constraint is used to prevent the modal relation graph from degenerating into an indifferent fully connected graph. The meteorological calibration stability constraint is used to prevent drastic fluctuations in calibration parameters under similar meteorological conditions.

[0151] Furthermore, the classifier can be represented as ,in and These are the learnable parameters for the classifier; the classifier sequentially includes a feature normalization layer, a fully connected layer, an activation layer, an optional Dropout layer, and a Softmax layer. The modal contribution weights W are obtained from the normalized result of the fusion gating network or the relation matrix A, and include image modal contribution weights. Electrical mode contribution weight and meteorological mode contribution weight , is used to characterize the relative contribution of each mode to the probability of the abnormal state category.

[0152] Example 11:

[0153] like Figure 1 As shown, this invention provides a method for identifying abnormal states of electrical equipment in hydropower stations based on the fusion of inspection images, electrical timing sequences, and meteorological conditions. Specifically, it involves: using an abnormal event of electrical equipment in the same hydropower station as the basic organizational unit, acquiring inspection images, electrical response timing sequences, and meteorological condition data; encoding the meteorological condition data to obtain meteorological condition codes; extracting abnormal image features and electrical abnormal evolution features respectively; conditionally calibrating the image features and electrical features according to the meteorological condition codes; constructing a modal relationship diagram using the calibrated image features, calibrated electrical features, and meteorological condition codes; and performing dynamic fusion discrimination based on the modal relationship diagram to output the abnormal state category and modal contribution weight.

[0154] It should be noted that the meteorological condition-based adjustment in this invention does not simply concatenate meteorological variables to the final classification layer. Instead, it involves meteorological information in image representation calibration, electrical representation calibration, and cross-modal relationship reconstruction. In this way, the model can change its dependence on image, electrical, and meteorological information according to different meteorological conditions, thereby improving its stability under complex weather conditions.

[0155] Using abnormal events of electrical equipment in the same hydropower station as the basic sample unit, inspection images are acquired. Electrical response timing Meteorological status data and abnormal status labels This constitutes a three-modal joint sample. Inspection images are used to characterize external signs of abnormal conditions, electrical response timing is used to characterize the evolution process of abnormal conditions, and meteorological status data is used to describe the operating environment constraints when abnormal events occur.

[0156] Meteorological data can come from weather stations, micro-meteorological monitoring devices in the factory area, sensors mounted on drones, or external meteorological services. The meteorological data must include at least one or more of the following: weather category, ambient temperature, relative humidity, wind speed, wind direction, precipitation intensity, and visibility. For icing-related anomalies, icing risk level, dew point temperature, estimated ice thickness, or historical ice storm risk level can also be included.

[0157] When constructing samples, it is necessary to ensure that the three types of data correspond to the same anomalous event or the same time window. If the meteorological data sampling interval is long, linear interpolation or nearest neighbor matching can be used to align it to the time of the anomalous event; if there is a time difference between electrical response and image acquisition, a time tolerance window can be set centered on the anomalous event record; if multiple candidate samples exist within the same segment, they can be filtered by combining equipment area, equipment number, acquisition time, and maintenance records. Multimodal joint samples can be represented as:

[0158]

[0159] in, Let i represent the i-th joint sample. Represents inspection images. Indicates electrical response timing, Represents meteorological status data. Indicates an abnormal status label;

[0160] This organizational approach allows the three modalities to be modeled uniformly at the same level of abnormal events, avoiding incorrect data pairing at different times and in different device regions.

[0161] Table 1. Composition of Multimodal Joint Sample Data

[0162]

[0163] Meteorological state vector This includes both continuous and discrete variables. Continuous variables, such as ambient temperature, relative humidity, wind speed, precipitation intensity, visibility, air pressure, and dew point temperature, can be normalized. Discrete variables, such as weather categories like sunny, rainy, foggy, snowy, strong winds, and low light levels, can be encoded using one-heat coding or embedded coding. The two are then concatenated and input into a multilayer perceptron or embedded network to obtain the meteorological condition code. .

[0164] Meteorological condition coding is not only used for final classification, but also participates in subsequent image feature calibration, electrical feature calibration, and modal relationship weight generation. This design transforms meteorological information from a mere supplementary input into a conditional variable that modulates the discrimination process. For example, under low visibility conditions, meteorological coding can guide the model to reduce the absolute dependence on image texture features; under low temperature and rain / snow conditions, meteorological coding can guide the model to increase the weight of icing-related features; under strong wind conditions, meteorological coding can indicate changes in electrical response caused by the movement or intermittent contact of foreign objects.

[0165] Meteorological condition codes can be represented as:

[0166]

[0167] in, For discrete weather category variables, As a continuous meteorological variable, For category embedding functions, For numerical normalization functions, For meteorological coding network, This is a code for meteorological conditions.

[0168] It should be noted that the dimensions of meteorological coding can be adjusted according to the data scale and application scenario. If there are few meteorological variables, low-dimensional embedding can be used; if weather category, temperature, humidity, wind speed, visibility, and icing risk level are included simultaneously, higher-dimensional coding can be used. Changes in coding dimensions do not affect the basic process of meteorological condition calibration and relationship reconstruction.

[0169] Table 2 Meteorological variables and their discriminative functions

[0170]

[0171] Complex weather conditions directly impact the quality of inspection images. Rain and fog reduce image contrast, snow alters background brightness and icing texture, low light reduces the sharpness of busbar or cable edges, and strong winds can cause camera platform vibration and changes in the posture of foreign objects. If image features do not consider weather conditions, the model may easily misinterpret weather-induced imaging changes as anomaly category differences.

[0172] This embodiment is based on meteorological condition coding. Generate image scale calibration parameters and image offset calibration parameters Let the basic features of the image be... The features of the calibrated image are: :

[0173]

[0174] in, Network for generating image calibration parameters Used for scaling image features of different dimensions. This is used to compensate for feature shifts caused by weather conditions. For example, in foggy conditions, the model can reduce the weights of feature dimensions that depend on edge sharpness while enhancing features related to the structure of the device object; in snowy conditions, the model can calibrate the difference between a white background and icing textures to avoid misclassifying snowy backgrounds as icing anomalies; and in strong wind conditions, the model can reduce unstable texture responses caused by image blurring. Through this calibration process, the image branch can form an adaptive representation for different weather conditions.

[0175] The image calibration parameter generation network can employ a two- or multi-layer perceptron structure, with meteorological condition codes as its input. The output is the image's basic features. Dimensionally consistent or broadcastable matching and ; and The final classification loss and meteorological calibration stability constraints are used for reverse learning.

[0176] Table 3 Key Points for Image Calibration under Different Meteorological Conditions

[0177]

[0178] Meteorological conditions also affect electrical response characteristics. Low temperatures and icing conditions can alter the mechanical state and electrical parameters of equipment, high humidity can affect insulation conditions, strong winds can change the contact state between foreign objects and live parts, and rainfall can alter some leakage paths and transient noise levels. If electrical characteristics do not consider these meteorological conditions, the model may misinterpret weather-induced response drift as class differences.

[0179] This embodiment is based on meteorological condition coding. Generate electrical scale calibration parameters and electrical offset calibration parameters Let the characteristics of the electrical foundation be... The calibrated electrical characteristics are then... :

[0180]

[0181] in, A network for generating electrical calibration parameters is provided. This calibration process can dynamically adjust the response amplitude, stage location, and channel contribution under different meteorological conditions. For example, under low-temperature rain and snow conditions, slowly changing features related to icing in the electrical branch can be given higher weight; under strong wind conditions, fluctuating features related to foreign object swaying or intermittent contact can be given more attention; under clear conditions, the electrical calibration intensity can be reduced, making the judgment more dependent on clear image evidence.

[0182] The electrical calibration parameter generation network can adopt a two-layer or multi-layer perceptron structure, and its input is the meteorological condition code. The output is related to the electrical basic characteristics. Dimensionally consistent or broadcastable matching and , and We learn by jointly using anomalous state labels from historical samples, classification loss, and multimodal constraints.

[0183] Table 4 Key Points of Electrical Calibration under Different Meteorological Conditions

[0184]

[0185] Traditional multimodal fusion typically assumes fixed modal relationships, such as always fusing images and electrical features with the same weights. However, in the identification of abnormal states of electrical equipment in hydropower stations, different meteorological conditions can alter the reliability of each mode and the correspondence between modes. Therefore, this embodiment constructs a modal relationship graph G based on meteorological conditions.

[0186] Modality diagrams based on calibrated image features calibrated electrical characteristics and meteorological condition codes For nodes. The input relation generation network yields a relation matrix A. Relation matrix A includes image-electrical relation weights. Image-meteorological relationship weights Weighting of electrical-meteorological relationship The relationship matrices for different samples can be different, thus reflecting the dynamic collaborative relationships under different working conditions. The modal relationship diagram satisfies:

[0187]

[0188] Where V represents the set of modal nodes, and A represents the relation matrix generated by the meteorological condition coding. Under clear and high visibility conditions, image modalities are generally more reliable, and the model can make more use of external anomaly signs; under rain, fog, or low illumination conditions, the reliability of image modalities decreases, and the model should refer more to electrical response and meteorological constraints; under low temperature rain and snow conditions, meteorological information plays an important role in identifying icing-related anomalies.

[0189] Table 5 Meaning of Modal Relationship Matrix

[0190]

[0191] On modal graphs, information fusion can be achieved using graph propagation, interactive attention, or gated weighting. For example, for each modal node, information from other modal nodes is aggregated to obtain an updated node representation; then, the updated node representations are concatenated or weighted summed to obtain the multimodal fusion feature H. Dynamic fusion discrimination can be represented as:

[0192]

[0193] in, For information propagation functions based on modal graphs, This represents the probability of the abnormal state category output by the classifier. The goal of dynamic fusion discrimination is to automatically adjust the contributions of image, electrical, and meteorological information based on the current meteorological conditions, rather than using a fixed modality contribution structure across all samples.

[0194] Training objectives include classification loss. Modal consistency constraints sparse relation constraint and meteorological calibration stability constraints The classification loss ensures the correct classification of the final abnormal state; the modal consistency constraint ensures that the different modal features of the same abnormal event have consistent orientation in the discrimination space; the relation sparsity constraint avoids the modal relation graph from degenerating into an indifferent fully connected state; and the meteorological calibration stability constraint avoids drastic fluctuations in calibration parameters under similar meteorological conditions. The total loss satisfies:

[0195]

[0196] in, , and These are the weighting coefficients.

[0197] In actual training, the basic feature extractor can be pre-trained using image branch and electrical branch, and then meteorological calibration and relation reconstruction modules can be added for joint training; or end-to-end training can be used when the sample size is sufficient.

[0198] Table 6. Composition and Function of Loss Function

[0199]

[0200] During the inference phase, the system receives new abnormal event data of electrical equipment in the hydropower station, including inspection images, electrical response timing data, and meteorological status data. First, the meteorological coding module generates meteorological condition codes; second, the image branch and electrical branch generate basic features; then, the meteorological calibration module performs conditional calibration on the image features and electrical features respectively; finally, the cross-modal relationship reconstruction module generates a relationship matrix and completes dynamic fusion discrimination.

[0201] The system output includes the anomaly category and category probability. Furthermore, it can also output image mode contribution weights, electrical mode contribution weights, and meteorological mode contribution weights. These contribution weights can be used to interpret what evidence the model primarily relies on under current weather conditions. For example, when foggy images are unclear, the system may output a higher electrical contribution weight; when clear images are present and the electrical response is weak, the system may output a higher image contribution weight; and when an anomaly is identified as icing under low-temperature rain and snow conditions, the system may output a higher meteorological contribution weight.

[0202] In this invention, the modal contribution weight can be denoted as... ,in , and These correspond to the image mode, electrical mode, and meteorological mode, respectively; W can be directly output by the dynamic fusion gating layer, or it can be obtained by normalizing the weights related to each mode in the relation matrix A.

[0203] When a certain mode is missing, this invention can employ a missing mode masking mechanism. If meteorological data is missing, it can be replaced with historical average meteorological codes or codes from nearby meteorological stations; if electrical data is missing, only image-meteorological discrimination can be performed and the confidence level reduced; if image data is missing, electrical-meteorological discrimination can be performed and manual supplementation of inspection images can be prompted. This processing method enables the system to have a certain degree of robustness in engineering deployment.

[0204] In a set of implementation data, this invention was used to identify four types of abnormal conditions: abnormal conductive connections, foreign objects in equipment, equipment icing, and icing of insulators or bushings. Experimental results show that the overall test set accuracy of this invention can reach approximately 97.2%, which is about 1.4 percentage points higher than the image-electrical joint discrimination method and about 0.8 percentage points higher than the static three-modal fusion method. On complex weather subsets, this invention maintains good discrimination stability. For conditions such as rain, fog, and snow, meteorological condition calibration can mitigate the impact of image quality degradation and electrical response drift; cross-modal relationship reconstruction can adjust the contribution of each mode according to the sample conditions, avoiding misjudgments caused by fixed fusion weights.

[0205] This invention can be deployed in intelligent inspection platforms for hydropower station equipment, online monitoring systems, meteorological disaster early warning systems, and dispatching auxiliary analysis systems. For equipment areas with real-time meteorological monitoring devices, on-site meteorological data can be used directly; for sections without on-site meteorological monitoring devices, data from nearby meteorological stations or meteorological service data can be used as substitutes. For low-temperature rain, snow, and strong wind weather, the system can increase the monitoring frequency under corresponding meteorological conditions and simultaneously output meteorological risk warnings in the abnormal state identification results, thereby providing a basis for hydropower station equipment operation and maintenance plans.

[0206] Table 7 Examples of Discrimination Effect Comparison

[0207]

[0208] Example 12:

[0209] An abnormal state discrimination system for electrical equipment in hydropower stations based on the above method, which integrates inspection images, electrical time series and meteorological conditions, includes a multi-source information acquisition module, an event-level alignment module, a meteorological condition coding module, an image feature extraction module, an electrical feature extraction module, an image sign calibration module, an electrical evolution calibration module, a cross-modal relationship reconstruction module, a dynamic fusion discrimination module and an abnormal state output module.

[0210] The multi-source information acquisition module acquires inspection images, electrical response time series, meteorological status data, and abnormal status labels corresponding to abnormal events of electrical equipment in the same hydropower station. The event-level alignment module establishes event-level alignment relationships between three types of data based on the abnormal event number, power station name, equipment name, equipment area, acquisition timestamp, fault trigger time, or maintenance record number, obtaining multimodal joint samples. The meteorological condition coding module normalizes continuous variables in the meteorological status data, embeds and codes discrete weather categories, and outputs meteorological condition codes. The image feature extraction module extracts image anomaly features from inspection images. The electrical feature extraction module extracts electrical anomaly evolution features from electrical response time series. The image feature calibration module generates image scale calibration parameters and image offset calibration parameters based on the meteorological condition codes, obtaining calibrated image features. The electrical evolution calibration module generates electrical scale calibration parameters and electrical offset calibration parameters based on the meteorological condition codes, obtaining calibrated electrical features. The cross-modal relationship reconstruction module constructs a modal relationship graph using calibrated image features, calibrated electrical features, and meteorological condition codes as nodes, and dynamically generates inter-modal relationship weights based on the meteorological condition codes. The dynamic fusion and discrimination module performs information propagation and dynamic fusion based on the modal relationship graph to obtain multi-modal fusion features. The abnormal state output module outputs the abnormal state categories, category probabilities, modal contribution weights, and meteorological risk warnings for the hydropower station's electrical equipment, and sends the output results to the hydropower station's intelligent equipment inspection platform, online monitoring system, meteorological disaster early warning system, or dispatch auxiliary analysis system.

[0211] The image feature calibration module and the electrical evolution calibration module both include a calibration parameter generation network, which adopts an input projection layer, a nonlinear mapping layer and a dual-branch output layer structure. The classifier in the abnormal state output module adopts a fully connected classification structure and a Softmax output layer, and can simultaneously output the modal contribution weight W.

[0212] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific embodiments of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for identifying abnormal states of multi-source sensing in hydropower station transmission lines, characterized by the following steps: include: Obtain inspection images, electrical response time series and meteorological status data corresponding to abnormal events of electrical equipment in the same hydropower station, establish event-level multi-source sample alignment relationship, and thus generate multimodal joint samples; Meteorological condition data is encoded to obtain meteorological operating condition codes; Feature extraction was performed on the inspection images and electrical response time series to obtain the image anomaly characteristics and electrical anomaly evolution characteristics; Image scale calibration parameters and image offset calibration parameters are generated based on meteorological condition coding. The abnormal image features are then calibrated according to meteorological conditions to obtain image calibration features based on meteorological conditions. Electrical scale calibration parameters and electrical offset calibration parameters are generated based on meteorological condition codes. The electrical anomaly evolution characteristics are then calibrated according to meteorological conditions to obtain electrical calibration characteristics based on meteorological conditions. A modal relationship graph is constructed using calibrated image features, calibrated electrical features, and meteorological condition codes as nodes, and inter-modal relationship weights are dynamically generated based on the meteorological condition codes. Information is disseminated and dynamically fused based on modal relationship graphs, outputting the abnormal state categories, category probabilities, and modal contribution weights of hydropower station electrical equipment.

2. The method for determining abnormal states of multi-source sensing for transmission lines of hydropower stations according to claim 1, characterized in that, The multimodal joint sample is represented as: in, Let i represent the i-th joint sample. Represents inspection images. Indicates the electrical response timing. Represents meteorological status data. Indicates an abnormal status label; The meteorological data includes one or more of the following: weather type, ambient temperature, relative humidity, wind speed, wind direction, precipitation intensity, visibility, air pressure, dew point temperature, or icing risk level.

3. The method for determining abnormal states of multi-source sensing for transmission lines of hydropower stations according to claim 1, characterized in that, The abnormal image features are extracted through an image discrimination model, an object detection model, a visual encoder, or a visual basic model. The image anomaly features include at least one or more of the following: busbar or cable continuity changes, foreign object suspension patterns, icing textures, objects attached to icing, insulator or bushing areas, plant equipment background, and anomaly area responses. The electrical anomaly evolution features are extracted by an electrical timing encoder; the electrical timing encoder includes TSMixer, temporal convolutional network, recurrent neural network, gated recurrent unit network, long short-term memory network, and Transformer timing network.

4. The method for determining abnormal states of multi-source sensing for transmission lines of hydropower stations according to claim 1, characterized in that, The meteorological condition code is obtained through numerical normalization, category embedding, and multilayer perceptron mapping. Among them, continuous meteorological variables are obtained by minimax normalization, standardization or quantile normalization to obtain numerical meteorological vectors; Discrete weather categories are obtained as category meteorological vectors through unique thermal coding or embedding coding; The meteorological condition codes are as follows: in, For discrete weather category variables, As a continuous meteorological variable, For category embedding functions, For numerical normalization functions, For meteorological coding network, This is a code for meteorological conditions.

5. The method for determining abnormal states of multi-source sensing for transmission lines of hydropower stations according to claim 1, characterized in that, The image calibration features based on meteorological conditions are shown below: in, and These are image scale calibration parameters and image offset calibration parameters, respectively. Network for generating image calibration parameters For image abnormality features, For the calibrated image features, This represents element-wise multiplication; The electrical calibration characteristics based on meteorological conditions are as follows: in, and These are the electrical dimension calibration parameters and the electrical offset calibration parameters, respectively. For generating electrical calibration parameters network, Characteristics of electrical anomaly evolution. For calibrated electrical characteristics.

6. The method for determining abnormal states of multi-source sensing for transmission lines of hydropower stations according to claim 5, characterized in that, The image scale calibration parameters and image offset calibration parameters Meteorological condition coding The calibration parameters of the input image are generated by the network. The image calibration parameter generation network includes an input projection layer, a nonlinear mapping layer, and scale parameter output branches and offset parameter output branches. The electrical dimension calibration parameters and electrical offset calibration parameters Meteorological condition coding The electrical calibration parameters are input into the network to generate the desired result. The electrical calibration parameter generation network includes an input projection layer, a nonlinear mapping layer, and scale parameter output branches and offset parameter output branches.

7. The method for determining abnormal states of multi-source sensing for transmission lines of hydropower stations according to claim 1, characterized in that, The modal relationship diagram is shown below: in, This is a modal relationship diagram. For a set of nodes, It is a relation matrix. A relation generation network; the relation matrix Used to describe the dynamic co-strength among image modes, electrical modes, and meteorological modes.

8. The method for determining abnormal states of multi-source sensing for transmission lines of hydropower stations according to claim 1, characterized in that, The steps for outputting the abnormal state categories, category probabilities, and modal contribution weights of electrical equipment in hydropower stations include: Based on the modal relationship graph, graph propagation, interactive attention, or gating weighting are performed to obtain multimodal fusion features H. H is then input into a classifier, which outputs the probability of abnormal state categories. ; The classifier includes a feature normalization layer, a fully connected mapping layer, a nonlinear activation layer, and a Softmax output layer, which are used to map the multimodal fusion feature H into the probabilities of each abnormal state category. Abnormal state category probability As shown below: Where H represents the multimodal fusion feature, Interaction functions include graph propagation, interactive attention, gated weighting, and feature product. This represents the probability of an abnormal state category.

9. The method for determining abnormal states of multi-source sensing for transmission lines of hydropower stations according to claim 1, characterized in that, The classifier is trained using historical data; The training objective is classification loss. Modal consistency constraints sparse relation constraint and meteorological calibration stability constraints Minimum; The total losses are shown below: in, , and These are the weighting coefficients.

10. A system for identifying abnormal states of electrical equipment in a hydropower station based on the method described in any one of claims 1-9, characterized in that, It includes a multi-source information acquisition module, an event-level alignment module, a meteorological condition coding module, an image feature extraction module, an electrical feature extraction module, an image sign calibration module, an electrical evolution calibration module, a cross-modal relationship reconstruction module, a dynamic fusion discrimination module, and an abnormal state output module; The multi-source information acquisition module is used to acquire inspection images, electrical response timing, meteorological status data, and abnormal status labels; The event-level alignment module is used to establish event-level correspondences between the three types of data; The meteorological condition coding module is used to output meteorological condition codes; The image feature extraction module is used to output the features of abnormal signs in the image; The electrical feature extraction module is used to output electrical anomaly evolution features; The image feature calibration module and the electrical evolution calibration module are used to adjust image features and electrical features respectively according to meteorological condition codes; The cross-modal relationship reconstruction module is used to construct modal relationship graphs and generate relationship matrices; The dynamic fusion discrimination module is used to output multimodal fusion features; The abnormal status output module is used to output the abnormal status category, category probability, modal contribution weight, and meteorological risk warning.

Images