Multi-mode self-learning diagnosis and early warning method and system for power distribution system

Abstract

The invention discloses a multi-mode self-learning diagnosis and early warning method and system for a power distribution system, and relates to the technical field of intelligent operation and maintenance of power systems. According to the method, a visible light image, an infrared thermal image, a sound signal, an environmental parameter and an electrical parameter of the power distribution equipment are collected through a multi-mode sensor, and time synchronization and feature extraction are carried out; multi-modal features are constructed into a hypergraph structure, and cross-modal feature fusion is realized by using a graph attention mechanism; adopting a strategy of combining self-supervised pre-training and supervised fine tuning to carry out multi-objective joint optimization on the fused features to obtain a self-learning diagnosis model; the system inputs multi-modal data in real time during operation, outputs an equipment operation state and a health degree index, and realizes early warning and fault alarm through threshold comparison. According to the method, deep fusion and adaptive learning of multi-source information can be realized, high-precision diagnosis, real-time early warning and online updating capabilities are realized, and the intelligence and safety of a power distribution system are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology for power systems, specifically to a multimodal self-learning diagnosis and early warning method and system for power distribution systems. Background Technology

[0002] Current power distribution system operation monitoring primarily relies on single-type sensor data such as electrical quantities, temperature, or images, using traditional threshold judgments or experience-based feature analysis for fault detection. However, the operating status of power distribution equipment is influenced by a combination of factors, especially under complex operating conditions. Single-mode monitoring information is insufficient to comprehensively and accurately reflect the overall health status of the equipment, making diagnostic results susceptible to interference from environmental noise, local anomalies, and other factors. As power distribution systems evolve towards intelligence and digitalization, operational data exhibits characteristics of multi-source heterogeneity, nonlinearity, and dynamic changes, placing higher demands on traditional diagnostic algorithms.

[0003] Current diagnostic algorithms generally rely on preset rules or fixed model structures, making it difficult to effectively integrate temporal relationships and semantic information from different modalities. Therefore, they cannot fully utilize the complementary information between multimodal data. Traditional methods lack the ability to update models when faced with continuous equipment evolution and changing failure modes, leading to a decline in diagnostic accuracy. Furthermore, existing technologies lack effective compensation mechanisms and adaptive learning capabilities when dealing with missing modal data or anomalous sampling, failing to guarantee the stability and robustness of diagnostic results.

[0004] Therefore, the core problem facing existing technologies is: how to effectively integrate high-dimensional information from multimodal sensing data in power distribution systems, achieve adaptive fault diagnosis and health assessment in complex environments and with incomplete data, and possess dynamic learning and early warning capabilities during system operation to ensure the accuracy, real-time performance, and stability of diagnostic results. Summary of the Invention

[0005] The purpose of this invention is to provide a multimodal self-learning diagnostic and early warning method and system for power distribution systems, which solves the limitations of existing power distribution system monitoring, such as the inability of a single data source to comprehensively reflect the health status of equipment, insufficient accuracy in fault mode identification, and poor adaptability to new fault modes. Through multimodal data fusion, self-learning optimization, and real-time diagnostic mechanisms, this invention can effectively assess the health status of power distribution equipment and diagnose faults in complex environments and under conditions of incomplete data, improving the adaptability, accuracy, and stability of power distribution equipment, and ensuring that the system can operate efficiently and stably under new operating conditions.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] On the one hand, the present invention provides a multimodal self-learning diagnosis and early warning method for power distribution systems, comprising the following steps:

[0008] Step 1: Collect multimodal sensing data of the power distribution equipment operation. The multimodal sensing data includes visible light images, infrared thermal imaging, sound signals, environmental parameters and electrical parameters. Perform time synchronization, preprocessing and feature extraction on the multimodal sensing data to obtain a multimodal feature set corresponding to the multimodal sensing data.

[0009] Step 2: Construct a hypergraph structure from the multimodal feature set. The nodes in the hypergraph structure represent different modal features in the multimodal feature set. Hyperedges connect multiple modal feature nodes to model the correlation between modalities. Use graph attention mechanism to fuse features in the hypergraph structure to generate a fused feature vector.

[0010] Step 3: Input the fused feature vector into the diagnostic model training process, and train the diagnostic model using a strategy that combines self-supervised pre-training and supervised fine-tuning. Perform multi-objective optimization through a joint loss function that includes feature consistency constraints, fault discrimination constraints, and modal reconstruction constraints to obtain a self-learning diagnostic model.

[0011] Step 4: Input real-time multimodal data into the self-learning diagnostic model, output the operating status information and health indicators of the power distribution equipment, compare the health indicators with preset thresholds, and trigger early warning signals or fault alarm signals based on the comparison results.

[0012] A further improvement of the present invention is that the process of collecting multimodal sensing data of power distribution equipment operation includes the following steps:

[0013] Establish a unified time-domain coordinate system for multimodal sensing data; perform time synchronization and delay compensation based on a dynamic sliding window to align different modal sensing signals within the same time window; extract key features of each modality from the synchronized multimodal sensing data; and calculate the corresponding modal quality coefficient based on the signal-to-noise ratio, data integrity, and stability of each modality.

[0014] Adaptive weighting of key multimodal features is performed using modal quality coefficients to generate a weighted feature set;

[0015] The weighted feature set is mapped to a unified semantic space through a semantic embedding mapping function; the output is a multimodal feature set with temporal consistency, quality adaptability and semantic alignment, and the output multimodal feature set is used as input data for the fusion model to construct the hypergraph structure.

[0016] A further improvement of the present invention is that the process of feature fusion of the hypergraph structure using the graph attention mechanism includes the following steps:

[0017] A dynamic hypergraph model containing multimodal feature nodes and high-order associated hyperedges is established. Within each time window, hyperedge connection relationships are generated based on the semantic similarity, temporal correlation and spatial proximity between multimodal features.

[0018] In the dynamic hypergraph model, feature updates based on a multi-head attention mechanism are performed on the features of each node; the formula for updating node features is as follows:

[0019] ;

[0020] Among them, attention weight The calculation method is as follows:

[0021] ;

[0022] in: Indicates the first The first in the layer Feature vectors of each node; This represents the updated features of a node after feature aggregation and nonlinear mapping; Indicates containing nodes The set of superedges; Represents a node Belongs to superedge The set of nodes connected; Indicates the superedge The total number of connected nodes; Indicates the first The learnable weight matrix of the layer; and The first The query mapping matrix and key mapping matrix of the layer; This represents the similarity calculation function; The normalization range is limited to the hyperedge. Internal node index This is used to normalize the weights inside the hyperedge; This represents a nonlinear activation function; a high-order weighted aggregation of cross-modal features is achieved in the hypergraph structure through a multi-head attention mechanism, and a fused feature vector containing semantic, temporal and spatial correlation information is output. The output fused feature vector is used as input data for the diagnostic model training process.

[0023] A further improvement of this invention is that, during the feature fusion of the hypergraph structure using the graph attention mechanism, dynamic feedback correction and adaptive parameter update are performed on the attention weight coefficients between nodes. The formula for correcting and updating the weight coefficients is as follows: ;

[0024] ;

[0025] in: This represents the attention weights after dynamic feedback correction; , These are the query and key mapping matrices, respectively. Represents a node In the The feature vector of the layer; Indicates the first Dynamic feedback correction terms for the layer; This indicates the dynamic feedback correction item from the previous layer; Indicates the smoothing coefficient; This represents the feedback correction function; Indicates a pair of nodes The residual term is obtained by calculating the feature propagation error; through the dynamic feedback correction mechanism, the model automatically adjusts the correlation strength between nodes according to the feature propagation error during the iteration process, realizing self-learning feature aggregation optimization.

[0026] A further improvement of the present invention is that the construction process of the self-learning diagnostic model is as follows:

[0027] Using the fused feature vector as input, the diagnostic model undergoes a learning process combining self-supervised pre-training and supervised fine-tuning. During pre-training, positive and negative sample pairs are constructed to constrain feature consistency. During fine-tuning, the fused feature vectors of labeled samples are used to perform joint training for fault classification and modality reconstruction. The objective function for model training is defined as the multi-objective joint loss function:

[0028] ;

[0029] in: The sum of the joint loss functions for multiple objectives; The cross-entropy-based loss is used for fault classification. For the consistency loss of fusion features based on contrastive learning; The mode reconstruction loss is based on mean square error; For regularization constraints; The weight coefficients are the weights corresponding to each loss term. During model training, the weight coefficients are dynamically adjusted based on the relative magnitude of the gradient change rate of each loss term. Through the multi-objective joint optimization and weight adaptive adjustment mechanism, the fault classification accuracy, feature representation consistency, and modality completion capabilities of the diagnostic model are improved simultaneously, and the self-learning diagnostic model updated after training is output.

[0030] A further improvement of the present invention is that the self-supervised pre-training and supervised fine-tuning process includes: a self-supervised pre-training process, in which the model learns a feature representation robust to modality loss based on the fused feature vector through a contrastive learning strategy; and a supervised fine-tuning process, in which the model's fault classification ability is optimized based on the fused feature vector and its fault category label.

[0031] A further improvement of the present invention is that the calculation steps for the operating status information and health indicators of the power distribution equipment include:

[0032] Real-time multimodal data is input into the self-learning diagnostic model to obtain the operating status assessment results and health indicators of the power distribution equipment; based on the comparison results of the health indicators and preset thresholds, a warning signal or fault alarm signal is triggered; the health indicators are obtained by calculating the deviation between the current operating status assessment results and the representation vector of the historical normal operating status.

[0033] A further improvement of the present invention is that the method further includes an online model update step during operation: collecting confirmed new fault sample data and storing it in the sample cache during system operation; and performing incremental learning on the self-learning diagnostic model and updating the model parameters based on the new fault samples when the preset update cycle or triggering conditions are met.

[0034] A further improvement of the present invention is that, during the training process of the self-learning diagnostic model, when some modal data is missing, the self-learning diagnostic model calls the pre-learned cross-modal reconstruction mapping, generates feature representations of the missing modal based on the existing modal data, and completes diagnostic reasoning based on the completed feature representations.

[0035] On the other hand, the present invention also provides a multimodal self-learning diagnosis and early warning system for power distribution systems, which applies the multimodal self-learning diagnosis and early warning method for power distribution systems as described above, including:

[0036] The multimodal data acquisition module is used to acquire multimodal sensing data of the operation of power distribution equipment. The multimodal sensing data includes visible light images, infrared thermal imaging, sound signals, environmental parameters and electrical parameters. The module performs time synchronization, preprocessing and feature extraction on the multimodal sensing data to obtain a multimodal feature set corresponding to the multimodal sensing data.

[0037] The hypergraph fusion module is used to construct the multimodal feature set into a hypergraph structure. The nodes of the hypergraph structure are used to represent different modal features. Hyperedges connect multiple modal feature nodes to model the correlation between modalities. The module also uses a graph attention mechanism to perform feature fusion on the hypergraph structure to generate a fused feature vector.

[0038] The self-learning diagnostic model training module is used to receive the fused feature vector and perform model training. The model training adopts a strategy that combines self-supervised pre-training and supervised fine-tuning. It performs multi-objective optimization through a joint loss function that includes feature consistency constraints, fault discrimination constraints and modality reconstruction constraints to obtain a self-learning diagnostic model.

[0039] The real-time diagnosis and early warning module is used to input the real-time collected multimodal data into the self-learning diagnosis model, output the operating status information and health index of the power distribution equipment, compare the health index with the preset threshold, trigger an early warning signal when the health index is lower than the early warning threshold, and trigger a fault alarm signal when the health index is lower than the alarm threshold.

[0040] The beneficial effects of this invention are as follows: By constructing an intelligent analysis framework integrating multimodal perception and self-learning diagnosis, this invention achieves comprehensive perception, deep feature fusion, adaptive learning, and dynamic early warning of the operating status of power distribution systems. Compared with existing diagnostic methods that rely solely on a single signal source or fixed rules, this invention can simultaneously acquire multi-source data such as visible light images, infrared thermal imaging, sound signals, environmental parameters, and electrical parameters, fusing them to form a high-dimensional feature set with temporal consistency and semantic alignment, fundamentally solving the problems of single information and one-sided features in traditional monitoring methods. By mapping multimodal features to a hypergraph structure and introducing a graph attention mechanism, this invention can establish high-order correlations between modes in the feature space, realizing weighted aggregation and dynamic optimization of cross-modal features, enabling the system to accurately identify potential abnormal modes of equipment even when facing complex operating environments and nonlinear coupling states. This invention employs a learning strategy combining self-supervised pre-training and supervised fine-tuning. This enables the model to possess adaptive learning capabilities while simultaneously achieving synergistic optimization of feature consistency constraints, fault discrimination constraints, and modal reconstruction constraints through a multi-objective joint loss function. This improves the model's generalization ability and diagnostic accuracy for different operating conditions and novel faults. During operation, the method can input multimodal data in real time and output the health index of power distribution equipment. Through dynamic comparison of the health index with preset thresholds, a hierarchical response mechanism from early hazard detection to fault alarm is implemented, significantly enhancing the safety and reliability of the power distribution system. This invention achieves breakthroughs in the depth of data fusion, the adaptability of model learning, and the intelligence of early warning response. It effectively improves the accuracy, real-time performance, and self-learning capability of power distribution equipment operation status diagnosis, possessing high engineering application value and promotional significance. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a system modular structure diagram of an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0043] like Figure 1 As shown, this is an embodiment of the present invention, which provides a multimodal self-learning diagnosis and early warning method for power distribution systems, including the following steps:

[0044] Step 1: Collect multimodal sensing data of the power distribution equipment operation. The multimodal sensing data includes visible light images, infrared thermal images, sound signals, environmental parameters, and electrical parameters. Perform time synchronization, preprocessing, and feature extraction on the multimodal sensing data to obtain a multimodal feature set corresponding to the multimodal sensing data.

[0045] To ensure the integrity and timing consistency of data acquisition, the process of acquiring multimodal sensor data of power distribution equipment operation includes the following steps:

[0046] The process of collecting multimodal sensor data on the operation of power distribution equipment includes the following steps:

[0047] A unified time-domain coordinate system is established for multimodal sensor data. In this embodiment, considering the differences in sampling frequency and response delay among different types of sensors in the power distribution system, a unified timestamp is first assigned to various types of sensor data. Preferably, a unified time-domain coordinate system can be established through clock synchronization technology (e.g., network time synchronization based on GPS or NTP protocol) to ensure consistent time reference for data collected by different sensors, providing a unified benchmark for subsequent data processing and fusion.

[0048] This embodiment utilizes a dynamic sliding window algorithm to perform time synchronization and delay compensation, aligning different modal sensing signals within the same time window. Considering the time delay and asynchronous response issues present in the sampling process of different modal sensors, a dynamic sliding window algorithm is employed to synchronize and compensate for the delays of each modal signal. Specifically, by setting the window length and correcting based on the signal time offset, alignment of the modal data is achieved within the same time window. Preferably, the window length can be dynamically adjusted according to the data acquisition frequency of the power distribution equipment, thereby achieving precise matching of multimodal data in the time domain.

[0049] Extract key features of each modality from synchronized multimodal sensing data;

[0050] After timing synchronization is completed, feature extraction is performed on each modal signal. For image modalities, surface feature information of the equipment can be extracted through edge detection, texture analysis, or feature point recognition; for sound signals, fault feature frequency bands can be extracted using spectrum analysis or wavelet transform; for electrical signals, the amplitude, phase, and waveform stability features of parameters such as current and voltage are extracted. Key feature parameters representing the operating status of the equipment are obtained through feature extraction algorithms, forming a multimodal feature set.

[0051] The modal quality coefficient is calculated based on the signal-to-noise ratio, data integrity, and stability of each mode.

[0052] In this embodiment, the quality of each extracted modal data is assessed. Signal clarity is evaluated by calculating the signal-to-noise ratio (SNR), the validity of the sampled data is measured by data integrity, and the stability of the signal is assessed by indicators such as volatility or variance. Preferably, the above three types of indicators can be weighted and combined to obtain a modal quality coefficient for each modality, which reflects the reliability of the modal data.

[0053] Adaptive weighting of key multimodal features is performed using modal quality coefficients to generate a weighted feature set;

[0054] After quality assessment, key features of each mode are weighted based on modal quality coefficients. Preferably, modes with higher quality coefficients are assigned greater weights, while modes with lower quality coefficients are assigned less weight, thus forming a weighted feature set. This weighting mechanism ensures that high-quality modes have a greater influence in subsequent fusion processes, avoiding interference from low-quality signals in the fusion results.

[0055] The weighted feature set is mapped to a unified semantic space through a semantic embedding mapping function;

[0056] In this embodiment, to achieve a unified expression of features from different modalities, a semantic embedding mapping is performed on the weighted feature set. Preferably, the semantic embedding mapping function can employ dimensionality reduction algorithms such as Principal Component Analysis (PCA) or t-SNE, or an embedding function based on feature space alignment, to map features from different modalities to a unified semantic space. This mapping process ensures the semantic alignment of feature representations while maintaining the differences in information across modalities.

[0057] The output is a multimodal feature set with temporal consistency, quality adaptability, and semantic alignment. The output multimodal feature set is used as input data for the fusion model to construct the hypergraph structure.

[0058] After the above processing steps, the output multimodal feature set simultaneously possesses temporal consistency, quality adaptability, and semantic alignment. Preferably, this feature set is input into the subsequent hypergraph fusion model to establish the hypergraph structure and perform multimodal feature fusion, providing unified and high-quality input data for power distribution system operation status identification, fault diagnosis, and health assessment.

[0059] This embodiment enables time uniformity, quality adaptation, and semantic alignment during the acquisition, synchronization, and feature extraction of multimodal sensor data, ensuring the integrity and accuracy of the input data. This implementation provides fundamental support for subsequent feature fusion, self-learning modeling, and real-time diagnosis, thereby improving the accuracy and robustness of power distribution system operating status identification.

[0060] In this embodiment, step 2: construct a hypergraph structure for the multimodal feature set. The nodes in the hypergraph structure represent different modal features in the multimodal feature set. Hyperedges connect multiple modal feature nodes to model the correlation between modalities. The graph attention mechanism is used to perform feature fusion on the hypergraph structure to generate a fused feature vector.

[0061] In this embodiment, the hypergraph structure is used to characterize the high-order relationships between multimodal features, and the dynamic fusion of cross-modal features is achieved through a graph attention mechanism. Preferably, the nodes of the hypergraph represent feature vectors of different types of modalities, and hyperedges connect nodes with semantic or temporal relationships to describe the mutual influence between different modal features.

[0062] The process of feature fusion of hypergraph structures using graph attention mechanism includes the following steps:

[0063] A dynamic hypergraph model containing multimodal feature nodes and high-order associated hyperedges is established. Within each time window, hyperedge connection relationships are generated based on the semantic similarity, temporal correlation and spatial proximity between multimodal features.

[0064] In this process, each modal feature node forms a hyperedge connection based on its proximity in the semantic space and the relative delay of the sampling time, thereby constructing a hypergraph model that can be updated over time within the dynamic window.

[0065] In the dynamic hypergraph model, feature updates based on a multi-head attention mechanism are performed on the features of each node; the formula for updating node features is as follows:

[0066] ;

[0067] Among them, attention weight The calculation method is as follows:

[0068] ;

[0069] in: Indicates the first The first in the layer Feature vectors of each node; This represents the updated features of a node after feature aggregation and nonlinear mapping; Indicates containing nodes The set of superedges; Represents a node Belongs to superedge The set of nodes connected; Indicates the superedge The total number of connected nodes; Indicates the first The learnable weight matrix of the layer; and The first The query mapping matrix and key mapping matrix of the layer; This represents the similarity calculation function; The normalization range is limited to the hyperedge. Internal node index This is used to normalize the weights inside the hyperedge; This represents a nonlinear activation function; a high-order weighted aggregation of cross-modal features is achieved in the hypergraph structure through a multi-head attention mechanism, and a fused feature vector containing semantic, temporal and spatial correlation information is output. The output fused feature vector is used as input data for the diagnostic model training process.

[0070] In this embodiment, the multi-head attention mechanism dynamically calculates the correlation between nodes in the hypergraph structure to achieve high-dimensional weighted fusion of modal features, thereby generating a fused feature vector containing semantic, temporal, and spatial correlation information. This fused feature vector serves as input data for subsequent self-learning model training, providing a unified representation for fault diagnosis and health assessment.

[0071] Specifically, in the process of feature fusion of the hypergraph structure using the graph attention mechanism, dynamic feedback correction and adaptive parameter update are performed on the attention weight coefficients between nodes. The formula for correcting and updating the weight coefficients is as follows:

[0072] ;

[0073] ;

[0074] in: This represents the attention weights after dynamic feedback correction; , These are the query and key mapping matrices, respectively. Represents a node In the The feature vector of the layer; Indicates the first Dynamic feedback correction terms for the layer; This indicates the dynamic feedback correction item from the previous layer; Indicates the smoothing coefficient; This represents the feedback correction function; Indicates a pair of nodes The residual term is obtained by calculating the feature propagation error; through the dynamic feedback correction mechanism, the model automatically adjusts the correlation strength between nodes according to the feature propagation error during the iteration process, realizing self-learning feature aggregation optimization.

[0075] Through the aforementioned dynamic feedback correction mechanism, the model adaptively adjusts the correlation strength between nodes based on residual information during feature propagation, thereby achieving dynamic optimization and weight balancing of feature aggregation. This process enhances the robustness and semantic consistency of fused features, enabling the model to maintain high feature fusion accuracy even when multimodal data contains noise or missing data.

[0076] In this embodiment, the multimodal feature set of the power distribution system is mapped into a dynamic hypergraph structure. Under the action of the multi-head attention mechanism and the dynamic feedback correction mechanism, the high-order semantic association between different modal feature nodes is modeled and optimized, thereby realizing the deep fusion of multimodal features, providing high-quality input features for the self-learning diagnostic model, and improving the accuracy and stability of the subsequent diagnostic stage.

[0077] In this embodiment, step 3: input the fused feature vector into the diagnostic model training process, and train the diagnostic model using a strategy that combines self-supervised pre-training and supervised fine-tuning. Multi-objective optimization is performed through a joint loss function that includes feature consistency constraints, fault discrimination constraints and modal reconstruction constraints to obtain a self-learning diagnostic model.

[0078] In this embodiment, the construction process of the self-learning diagnostic model is as follows:

[0079] Using the fused feature vector as input, the diagnostic model undergoes a learning process combining self-supervised pre-training and supervised fine-tuning. Preferably, the model training phase is divided into two stages: a self-supervised pre-training stage and a supervised fine-tuning stage. The pre-training stage utilizes unlabeled historical data and employs feature consistency learning to enable the model to recognize and complete cross-modal features. The supervised fine-tuning stage optimizes the model's classification performance based on fault-labeled sample data. In the pre-training stage, positive and negative sample pairs are constructed to enforce feature consistency constraints. In the fine-tuning stage, the fused feature vectors of labeled samples are used to perform joint training for fault classification and modality reconstruction. The objective function for model training is defined as a multi-objective joint loss function:

[0080] ;

[0081] in: The sum of the joint loss functions for multiple objectives; The cross-entropy-based loss is used for fault classification. For the consistency loss of fusion features based on contrastive learning; The mode reconstruction loss is based on mean square error; For regularization constraints; The weight coefficients are the weights corresponding to each loss term. During model training, the weight coefficients are dynamically adjusted based on the relative magnitude of the gradient change rate of each loss term. Through the multi-objective joint optimization and weight adaptive adjustment mechanism, the fault classification accuracy, feature representation consistency, and modality completion capabilities of the diagnostic model are improved simultaneously, and the self-learning diagnostic model updated after training is output.

[0082] During model training, the weight coefficients are dynamically adjusted based on the relative magnitude of the gradient changes of each loss term. Preferably, through multi-objective joint optimization and adaptive weight adjustment mechanisms, the model's fault classification accuracy, feature representation consistency, and modality completion capability can be improved simultaneously. After training is completed, a self-learning diagnostic model updated by training is output.

[0083] The process of self-supervised pre-training and supervised fine-tuning includes:

[0084] Self-supervised pre-training process: Based on the fused feature vector, a contrastive learning strategy is used to enable the model to learn feature representations under modality missing conditions. Specifically, positive and negative sample pairs from different perspectives are constructed by randomly occluding some modal data, minimizing the feature differences of the same sample under different perspectives to enhance the model's modality consistency and adaptability.

[0085] Supervised fine-tuning process: Based on the fused feature vector and the corresponding fault category labels, the model's classification output is optimized. Model parameters are adjusted using the cross-entropy loss function to further improve the model's discrimination ability in multi-fault scenarios. Preferably, the fine-tuning stage can also utilize modal reconstruction loss to constrain the feature space, thereby maintaining the stability of the feature distribution.

[0086] In this embodiment, the fused feature vector is input into a self-learning diagnostic model. The model employs a training strategy combining self-supervised pre-training and supervised fine-tuning to achieve multi-objective optimization of consistent learning of multimodal features, modality completion, and fault classification. This self-learning diagnostic model possesses adaptive feature learning and dynamic update capabilities, providing accurate and stable model support for subsequent real-time diagnosis and early warning stages.

[0087] In this embodiment, step 4: input real-time multimodal data into the self-learning diagnostic model, output the operating status information and health index of the power distribution equipment, compare the health index with a preset threshold, and trigger a warning signal or fault alarm signal based on the comparison result.

[0088] During the operation of the power distribution system, real-time multimodal data is collected as input. This data, processed by the aforementioned self-learning diagnostic model, performs inference calculations and outputs the equipment's operating status and health assessment results. This process belongs to the system's online diagnostic phase. Combined with the fusion feature space formed during the model's pre-training and fine-tuning phases, dynamic identification and risk assessment of equipment operating status can be achieved.

[0089] The calculation steps for the operating status information and health indicators of the power distribution equipment include:

[0090] Real-time multimodal data is input into the self-learning diagnostic model to obtain the operating status assessment results and health indicators of the power distribution equipment; based on the comparison results of the health indicators and preset thresholds, a warning signal or fault alarm signal is triggered; the health indicators are obtained by calculating the deviation between the current operating status assessment results and the representation vector of the historical normal operating status.

[0091] In this embodiment, the health index is used to quantify the overall operating level of the equipment, representing the degree of difference between the current characteristic state and the baseline normal state. Preferably, the deviation of the representation vector can be calculated using metrics such as cosine similarity, Euclidean distance, or Mahalanobis distance, and the health value is output in a normalized form. When the health level is higher than the preset safety threshold, the system determines that the device is in normal operating condition; when the health level drops below the warning threshold, the system triggers a risk warning signal; if the health level further falls below the alarm threshold, a fault alarm is immediately triggered.

[0092] To ensure the model maintains diagnostic accuracy when the equipment operating environment changes or new faults occur,

[0093] The method also includes an online model update step during operation: collecting confirmed new fault sample data and storing it in the sample cache during system operation; and performing incremental learning on the self-learning diagnostic model and updating the model parameters based on the new fault samples when the preset update cycle or triggering conditions are met.

[0094] Preferably, the triggering conditions may include any one or more of the following: (1) the number of newly collected fault samples reaches a set threshold; (2) the classification confidence of the model on the new data drops below a set range; (3) the operational health index fluctuates significantly. When the triggering conditions are met, the system automatically enters the model update state and performs incremental training on the new data stored in the sample cache. The updated model parameters replace the original diagnostic model parameters, enabling the model to have continuous learning and adaptive capabilities, thereby ensuring that it can maintain a high level of diagnostic performance when the equipment operating conditions change.

[0095] Furthermore, during real-time diagnosis, considering that some of the multimodal data collected by on-site sensors may be missing, during the training process of the self-learning diagnostic model, when some modal data is missing, the self-learning diagnostic model calls the pre-learned cross-modal reconstruction mapping, generates feature representations of the missing modal based on the existing modal data, and completes diagnostic reasoning based on the completed feature representations.

[0096] In this embodiment, the cross-modal reconstruction mapping can be obtained through the modal reconstruction loss term during the model training phase. Its function is to learn the mapping relationship between different modalities in the feature space. During runtime, when data for a certain modality is missing, the system uses the remaining modal data to predict the feature vector of the missing modality through the mapping function, forming a complete feature input, thereby ensuring the continuity and robustness of the diagnostic process. This mechanism ensures that the model still has stable diagnostic capabilities even when modal data is incomplete or sensor malfunctions.

[0097] In this embodiment, the self-learning diagnostic model can receive multimodal input data in real time during the operation of the power distribution system and output operating status information and health indicators, enabling continuous assessment and dynamic early warning of equipment health status. Combined with the model's online incremental update mechanism and modal reconstruction compensation strategy, the system can maintain high-reliability diagnostic performance under environmental changes, sample distribution shifts, and data gaps, ensuring the safety and stability of the power distribution system operation.

[0098] like Figure 2 As shown, another embodiment of the present invention provides a multimodal self-learning diagnosis and early warning system for power distribution systems, applying the multimodal self-learning diagnosis and early warning method for power distribution systems as described above, including:

[0099] The multimodal data acquisition module is used to acquire multimodal sensing data of the operation of power distribution equipment. The multimodal sensing data includes visible light images, infrared thermal imaging, sound signals, environmental parameters and electrical parameters. The module performs time synchronization, preprocessing and feature extraction on the multimodal sensing data to obtain a multimodal feature set corresponding to the multimodal sensing data.

[0100] The hypergraph fusion module is used to construct the multimodal feature set into a hypergraph structure. The nodes of the hypergraph structure are used to represent different modal features. Hyperedges connect multiple modal feature nodes to model the correlation between modalities. The module also uses a graph attention mechanism to perform feature fusion on the hypergraph structure to generate a fused feature vector.

[0101] The self-learning diagnostic model training module is used to receive the fused feature vector and perform model training. The model training adopts a strategy that combines self-supervised pre-training and supervised fine-tuning. It performs multi-objective optimization through a joint loss function that includes feature consistency constraints, fault discrimination constraints and modality reconstruction constraints to obtain a self-learning diagnostic model.

[0102] The real-time diagnosis and early warning module is used to input the real-time collected multimodal data into the self-learning diagnosis model, output the operating status information and health index of the power distribution equipment, compare the health index with the preset threshold, trigger an early warning signal when the health index is lower than the early warning threshold, and trigger a fault alarm signal when the health index is lower than the alarm threshold.

[0103] The entire system can be deployed in a power distribution monitoring host or an industrial control server, with modules communicating via a data bus or system bus. Preferably, each module can be implemented using program code executed by a computer processor, or it can be implemented using a combination of hardware and software to meet the real-time and reliability requirements of different application environments.

[0104] Through the system architecture described in this embodiment, the multimodal sensing data of the power distribution system can be collected, fused, trained, and diagnosed within a unified framework, forming an intelligent closed-loop system integrating data acquisition, feature fusion, model training, real-time diagnosis, and dynamic early warning. This system possesses data self-learning, online updating, and modal compensation functions, continuously improving diagnostic accuracy and robustness, and effectively ensuring the safe and stable operation of power distribution equipment in complex environments.

[0105] In summary, this invention constructs a complete technical system integrating multimodal perception, self-learning diagnosis, and dynamic early warning in the field of intelligent operation and maintenance of power distribution systems, realizing the coordinated operation of data acquisition, feature fusion, model optimization, and real-time response. This invention not only achieves efficient fusion of multi-source information and adaptive feature learning at the algorithm level, but also forms a continuously updated and closed-loop optimized intelligent diagnostic architecture at the system level, enabling power distribution systems to possess long-term self-evolution capabilities for complex operating conditions. The method and system exhibit good engineering adaptability and scalability, and can be deeply integrated with existing power monitoring platforms or industrial automation systems, meeting the high reliability and high intelligence requirements of future smart grids and the energy internet for equipment health management. It possesses significant technological innovation value and broad industrial application prospects.

[0106] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0107] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0108] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multimodal self-learning diagnosis and early warning method for power distribution systems, characterized in that, Includes the following steps: Multimodal sensing data of power distribution equipment operation is collected, including visible light images, infrared thermal imaging, sound signals, environmental parameters and electrical parameters; the multimodal sensing data is time-synchronized, preprocessed and feature extracted to obtain a multimodal feature set corresponding to the multimodal sensing data; The multimodal feature set is constructed as a hypergraph structure. The nodes in the hypergraph structure represent different modal features in the multimodal feature set. Hyperedges connect multiple modal feature nodes to model the correlation between modalities. The graph attention mechanism is used to fuse features in the hypergraph structure to generate a fused feature vector. The fused feature vectors are input into the diagnostic model training process. A strategy combining self-supervised pre-training and supervised fine-tuning is used to train the diagnostic model. Multi-objective optimization is performed through a joint loss function that includes feature consistency constraints, fault discrimination constraints, and modal reconstruction constraints to obtain a self-learning diagnostic model. Real-time multimodal data is input into the self-learning diagnostic model, which outputs the operating status information and health indicators of the power distribution equipment. The health indicators are compared with preset thresholds, and early warning signals or fault alarm signals are triggered based on the comparison results.

2. The method for multimodal self-learning diagnosis and early warning of a power distribution system according to claim 1, characterized in that, The process of collecting multimodal sensor data on the operation of power distribution equipment includes the following steps: A unified temporal coordinate system is established for multimodal sensing data; time synchronization and delay compensation are performed based on a dynamic sliding window to align different modal sensing signals within the same time window; key features of each modality are extracted from the synchronized multimodal sensing data; modal quality coefficients are calculated based on the signal-to-noise ratio, data integrity, and stability of each modality; the key features of the multimodal sensing data are adaptively weighted using the modal quality coefficients to generate a weighted feature set; the weighted feature set is mapped to a unified semantic space through a semantic embedding mapping function; and a multimodal feature set with temporal consistency, quality adaptability, and semantic alignment is output, which serves as the input data for the fusion model to construct the hypergraph structure.

3. The method for multimodal self-learning diagnosis and early warning of a power distribution system according to claim 1, characterized in that, The process of feature fusion of hypergraph structures using graph attention mechanism includes the following steps: A dynamic hypergraph model containing multimodal feature nodes and high-order associated hyperedges is established. Within each time window, hyperedge connection relationships are generated based on the semantic similarity, temporal correlation and spatial proximity between multimodal features. In the dynamic hypergraph model, feature updates based on a multi-head attention mechanism are performed on the features of each node; the formula for updating node features is as follows: ； Among them, attention weight The calculation method is as follows: ； in: Indicates the first The first in the layer Feature vectors of each node; This represents the updated features of a node after feature aggregation and nonlinear mapping; Indicates containing nodes The set of superedges; Represents a node Belongs to superedge The set of nodes connected; Indicates the superedge The total number of connected nodes; Indicates the first The learnable weight matrix of the layer; and The first The query mapping matrix and key mapping matrix of the layer; This represents the similarity calculation function; The normalization range is limited to the hyperedge. Internal node index This is used to normalize the weights inside the hyperedge; This represents a nonlinear activation function; a high-order weighted aggregation of cross-modal features is achieved in the hypergraph structure through a multi-head attention mechanism, and a fused feature vector containing semantic, temporal and spatial correlation information is output. The output fused feature vector is used as input data for the diagnostic model training process.

4. The method for multimodal self-learning diagnosis and early warning of a power distribution system according to claim 3, characterized in that, In the process of feature fusion of the hypergraph structure using the graph attention mechanism, dynamic feedback correction and adaptive parameter update are performed on the attention weight coefficients between nodes. The formula for the correction and update of the weight coefficients is as follows: ； ； in: This represents the attention weights after dynamic feedback correction; , These are the query and key mapping matrices, respectively. Represents a node In the The feature vector of the layer; Indicates the first Dynamic feedback correction terms for the layer; This indicates the dynamic feedback correction item from the previous layer; Indicates the smoothing coefficient; This represents the feedback correction function; Indicates a pair of nodes The residual term is obtained by calculating the feature propagation error; through the dynamic feedback correction mechanism, the model automatically adjusts the correlation strength between nodes according to the feature propagation error during the iteration process, realizing self-learning feature aggregation optimization.

5. The method for multimodal self-learning diagnosis and early warning of a power distribution system according to claim 1, characterized in that, The construction process of the self-learning diagnostic model is as follows: Using the fused feature vector as input, the diagnostic model undergoes a learning process combining self-supervised pre-training and supervised fine-tuning. During pre-training, positive and negative sample pairs are constructed to constrain feature consistency. During fine-tuning, the fused feature vectors of labeled samples are used to perform joint training for fault classification and modality reconstruction. The objective function for model training is defined as the multi-objective joint loss function: ； in: The sum of the joint loss functions for multiple objectives; The cross-entropy-based loss is used for fault classification. For the consistency loss of fusion features based on contrastive learning; The mode reconstruction loss is based on mean square error; For regularization constraints; The weight coefficients are the weights corresponding to each loss term. During model training, the weight coefficients are dynamically adjusted based on the relative magnitude of the gradient change rate of each loss term. Through the multi-objective joint optimization and weight adaptive adjustment mechanism, the fault classification accuracy, feature representation consistency, and modality completion capabilities of the diagnostic model are improved simultaneously, and the self-learning diagnostic model updated after training is output.

6. The method for multimodal self-learning diagnosis and early warning of a power distribution system according to claim 5, characterized in that, The self-supervised pre-training and supervised fine-tuning process includes: a self-supervised pre-training process, in which the model learns a feature representation robust to modality loss based on the fused feature vector through a contrastive learning strategy; and a supervised fine-tuning process, in which the model's fault classification ability is optimized based on the fused feature vector and its fault category label.

7. The method for multimodal self-learning diagnosis and early warning of a power distribution system according to claim 1, characterized in that, The calculation steps for the operating status information and health indicators of the power distribution equipment include: Real-time multimodal data is input into the self-learning diagnostic model to obtain the operating status assessment results and health indicators of the power distribution equipment; based on the comparison results of the health indicators and preset thresholds, a warning signal or fault alarm signal is triggered; the health indicators are obtained by calculating the deviation between the current operating status assessment results and the representation vector of the historical normal operating status.

8. The method for multimodal self-learning diagnosis and early warning of a power distribution system according to claim 7, characterized in that, The method also includes an online model update step during operation: collecting confirmed new fault sample data and storing it in the sample cache during system operation; When the preset update cycle or triggering conditions are met, the self-learning diagnostic model is incrementally learned and the model parameters are updated based on the newly added fault samples.

9. The method for multimodal self-learning diagnosis and early warning of a power distribution system according to claim 1, characterized in that, During the training process of the self-learning diagnostic model, when some modal data is missing, the self-learning diagnostic model calls the pre-learned cross-modal reconstruction mapping, generates feature representations of the missing modal based on the existing modal data, and completes diagnostic reasoning based on the completed feature representations.

10. A multimodal self-learning diagnosis and early warning system for a power distribution system, employing the multimodal self-learning diagnosis and early warning method for a power distribution system as described in any one of claims 1-9, characterized in that, include: The multimodal data acquisition module is used to acquire multimodal sensing data of the operation of power distribution equipment. The multimodal sensing data includes visible light images, infrared thermal imaging, sound signals, environmental parameters and electrical parameters. The module performs time synchronization, preprocessing and feature extraction on the multimodal sensing data to obtain a multimodal feature set corresponding to the multimodal sensing data. The hypergraph fusion module is used to construct the multimodal feature set into a hypergraph structure. The nodes of the hypergraph structure are used to represent different modal features. Hyperedges connect multiple modal feature nodes to model the correlation between modalities. The module also uses a graph attention mechanism to perform feature fusion on the hypergraph structure to generate a fused feature vector. The self-learning diagnostic model training module is used to receive the fused feature vector and perform model training. The model training adopts a strategy that combines self-supervised pre-training and supervised fine-tuning. It performs multi-objective optimization through a joint loss function that includes feature consistency constraints, fault discrimination constraints and modality reconstruction constraints to obtain a self-learning diagnostic model. The real-time diagnosis and early warning module is used to input the real-time collected multimodal data into the self-learning diagnosis model, output the operating status information and health index of the power distribution equipment, compare the health index with the preset threshold, trigger an early warning signal when the health index is lower than the early warning threshold, and trigger a fault alarm signal when the health index is lower than the alarm threshold.

Images