A hidden danger early warning method, device, equipment and storage medium

Abstract

This invention discloses a method, apparatus, device, and storage medium for hazard early warning, relating to the field of computer technology. The method includes: acquiring multimodal monitoring data and preprocessing the multimodal monitoring data to obtain preprocessed data, wherein the processing parameters involved in the preprocessing are determined based on historical hazard identification results; extracting features from the preprocessed data to obtain multimodal features, and fusing the multimodal features to obtain comprehensive features characterizing hazard risk, wherein the fusion parameters involved in the feature fusion are determined based on the historical contribution of each modality of monitoring data in historical hazard identification; using a preset hazard identification model to identify hazard based on the comprehensive features to obtain hazard identification results, and issuing an early warning based on the hazard identification results. The technical solution of this invention constructs an intelligent hazard early warning system with multi-source perception, closed-loop feedback, and performance self-enhancement.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for early warning of potential hazards. Background Technology

[0002] As a critical infrastructure of the power system, the safe operation of transmission lines is directly related to the stability of the power grid and the reliability of power supply. With the expansion of the scale of transmission lines and the increasing complexity of the operating environment, traditional manual inspections and single monitoring methods are no longer sufficient to meet the needs of accurate identification of multi-source hidden dangers in high-voltage transmission lines.

[0003] Currently, intelligent monitoring and early warning technology based on multi-source data fusion has gradually become a research hotspot. Its core objective is to integrate multiple types of data to achieve real-time monitoring, intelligent identification, and risk assessment of potential hazards in power transmission lines, thereby improving the overall operational safety level and maintenance efficiency of the system.

[0004] However, intelligent monitoring and early warning methods based on multi-source data fusion are usually open-loop static processes, lacking self-optimization capabilities and difficult to adapt to the complex and ever-changing environment along transmission lines. Their long-term operational performance may stagnate or even decline. Summary of the Invention

[0005] This invention provides a method, device, equipment, and storage medium for hazard early warning, in order to solve the problem that hazard identification and early warning cannot adapt to complex and ever-changing environments.

[0006] In a first aspect, the present invention provides a method for early warning of potential hazards, comprising: Acquire multimodal monitoring data and preprocess the multimodal monitoring data to obtain preprocessed data, wherein the processing parameters involved in the preprocessing are determined based on the historical hazard identification results; Feature extraction is performed on the preprocessed data to obtain multimodal features, and feature fusion is performed on the multimodal features to obtain comprehensive features characterizing hidden danger risks. The fusion parameters involved in feature fusion are determined based on the historical contribution of each modality monitoring data in historical hidden danger identification. A hazard identification model is used to identify hazards based on the comprehensive features to obtain hazard identification results, and an early warning is issued based on the hazard identification results.

[0007] Secondly, the present invention provides a hazard warning device, comprising: The preprocessing module is used to acquire multimodal monitoring data and preprocess the multimodal monitoring data to obtain preprocessed data. The processing parameters involved in the preprocessing are determined based on the historical hazard identification results. The comprehensive feature extraction module is used to extract features from the preprocessed data to obtain multimodal features, and to fuse the multimodal features to obtain comprehensive features that characterize the risk of hidden dangers. The fusion parameters involved in the feature fusion are determined based on the historical contribution of each modal monitoring data in historical hidden danger identification. The hazard warning module is used to identify hazards based on the comprehensive features using a preset hazard identification model, so as to obtain the hazard identification result and issue a warning based on the hazard identification result.

[0008] Thirdly, the present invention provides an electronic device comprising: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores a computer program that can be executed by at least one processor, which is executed by at least one processor to enable the at least one processor to perform the aforementioned hazard warning method in the first aspect.

[0009] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the aforementioned risk warning method of the first aspect.

[0010] The hazard identification scheme provided by this invention integrates the features of multimodal monitoring data and updates the preprocessing parameters and feature fusion parameters based on historical data. It constructs an intelligent hazard early warning system with multi-source perception, closed-loop feedback, and performance self-enhancement. It has the ability to self-optimize, can not only cope with complex scenarios, but also become more and more intelligent in long-term operation, and ultimately achieve continuous autonomous improvement in early warning accuracy and operation and maintenance efficiency.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a hidden danger early warning method provided in Embodiment 1 of the present invention; Figure 2This is a flowchart of a hidden danger early warning method provided according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of a hidden danger early warning device according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided according to Embodiment 4 of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0016] Example 1 Figure 1 The flowchart of a hazard warning method is provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of hazard warning for power transmission lines. The method can be executed by a hazard warning device, which can be implemented in hardware and / or software. The hazard warning device can be configured in an electronic device, which can be composed of two or more physical entities or a single physical entity.

[0017] like Figure 1As shown, the hazard warning method provided in Embodiment 1 of the present invention specifically includes the following steps: S101. Acquire multimodal monitoring data and preprocess the multimodal monitoring data to obtain preprocessed data, wherein the processing parameters involved in the preprocessing are determined based on the historical hazard identification results.

[0018] In this embodiment, multimodal monitoring data of the transmission line can be acquired first, and then preprocessed. The processing parameters involved in the preprocessing, such as various thresholds, are updated based on historical hazard identification results. For example, if the accuracy of historical hazard identification results decreases, certain thresholds involved in the preprocessing can be appropriately reduced.

[0019] S102. Feature extraction is performed on the preprocessed data to obtain multimodal features, and feature fusion is performed on the multimodal features to obtain comprehensive features that characterize the hidden danger risk. The fusion parameters involved in feature fusion are determined based on the historical contribution of each modal monitoring data in historical hidden danger identification.

[0020] In this embodiment, features can be extracted from each modality of the preprocessed data to obtain multimodal features. These modal features are then fused to obtain comprehensive features that characterize potential risks. Specifically, the fusion parameters involved in feature fusion, such as fusion weights, can be determined based on the historical contribution of each modality of monitoring data in historical hazard identification. For example, if the historical contribution of image features in the multimodal features increases, the fusion weight of the image features can be increased.

[0021] S103. Using a preset hazard identification model, hazard identification is performed based on the comprehensive characteristics to obtain hazard identification results, and warnings are issued based on the hazard identification results.

[0022] In this embodiment, comprehensive features can be input into a preset hazard identification model, which can output hazard identification results, such as insulator damage, conductor galloping, tower tilting, icing overload, and external environmental interference (such as wildfires and foreign object intrusion). Warnings can then be issued based on the hazard identification results.

[0023] For example, a preliminary early warning signal can be generated based on the hazard identification results, and then input into a multi-level early warning decision module. This module, combining preset early warning thresholds, hazard occurrence probability models, and power grid operation safety standards, can dynamically generate tiered early warning information. The early warning information is then pushed to the mobile terminals of maintenance personnel or the monitoring center via a wireless communication network, while simultaneously recording the time, location, hazard type, and handling status of the early warning event in the system log for subsequent model training and early warning strategy optimization. Furthermore, this module can construct a strategy optimization loop. By statistically analyzing the response efficiency, handling results, and hazard development trends of previous early warnings, it automatically optimizes early warning thresholds, risk level classifications, and strategy rules, allowing the early warning strategy to continuously evolve as the system operates, thus forming an early warning decision system that runs throughout the entire process.

[0024] The technical solution of this invention integrates the features of multimodal monitoring data and updates the preprocessing parameters and feature fusion parameters based on historical data to construct an intelligent hidden danger early warning system with multi-source perception, closed-loop feedback and performance self-enhancement. It has the ability to self-optimize, can not only cope with complex scenarios, but also become more and more intelligent in long-term operation, and ultimately achieve continuous autonomous improvement in early warning accuracy and operation and maintenance efficiency.

[0025] Optionally, the multimodal monitoring data includes image data, infrared thermal imaging data, vibration and tension data, and environmental and power grid operation data.

[0026] Specifically, by deploying various types of sensors along the transmission line and external data interfaces, multi-source heterogeneous data (i.e., multi-modal monitoring data) of the transmission line can be collected in real time. This includes, but is not limited to, image data of the line itself, infrared thermal imaging data, vibration monitoring data, conductor tension data, insulator pollution data, environmental data (such as wind speed, temperature, humidity and icing thickness), and power grid operation status data (such as load current and voltage fluctuations).

[0027] Optionally, the step of performing feature fusion on the multimodal features to obtain comprehensive features characterizing the hidden danger risk includes: updating the historical weight coefficients by utilizing the historical contribution of each modal monitoring data in historical hidden danger identification in historical multimodal monitoring data to obtain the current weight coefficients; and using the current weight coefficients to perform weighted fusion on the multimodal features to obtain comprehensive features characterizing the hidden danger risk.

[0028] For example, methods for determining the comprehensive characteristic F characterizing potential risks include: in, The current (attention) weight coefficient for the i-th modality monitoring data in the historical multimodal monitoring data. The historical weighting coefficients for the i-th modal monitoring data in the historical multimodal monitoring data are given. For the features of the i-th modality monitoring data in the multimodal features, The historical contribution of the i-th type of modal monitoring data in historical multimodal monitoring data to historical hazard identification. This represents the attention learning rate.

[0029] Optionally, the step of using a preset hazard identification model to identify hazards based on the comprehensive features to obtain hazard identification results includes: determining the time-frequency window of the vibration and tension data signals in the preprocessed data, and determining the power spectral density for the signals within each time-frequency window; determining a time-frequency anomaly evaluation index to characterize the proportion of abnormal energy based on the distribution characteristics of the power spectral density, and generating extended comprehensive features using the time-frequency anomaly evaluation index and the comprehensive features; and outputting hazard identification results using the preset hazard identification model based on the extended comprehensive features and the time-frequency anomaly evaluation index.

[0030] For example, the vibration and tension data signals in the preprocessed data are segmented by time, and a specific frequency range is focused for each time segment. Each such combination of time segment and frequency segment constitutes a time-frequency window. This time-frequency anomaly evaluation index is used to characterize the proportion of anomalous energy. The method for determining it is as follows: in, For the first Power spectral density of a time window Based on Q and Determined window weights. The upper and lower limits of integration are determined by the channel bandwidth parameter. The aim is to quantify the proportion of abnormal energy of vibration and tension on a comparable scale and incorporate it into the comprehensive feature F to form an extended comprehensive feature, so as to highlight the dynamic abnormal signal.

[0031] Extended comprehensive features can be generated by incorporating time-frequency anomaly evaluation indicators into the comprehensive features.

[0032] For example, if image, infrared, meteorological, and power grid operation data are fused using an attention mechanism, the resulting original comprehensive feature vector is F = [0.32, 0.18, 0.45, 0.27, 0.51]. This represents a time-frequency anomaly evaluation index characterizing the proportion of anomalous energy. (High scores indicate the presence of significant dynamic anomalies, such as time-frequency signals related to conductor galloping). Combining these two elements, the final extended comprehensive feature vector is: =[0.32,0.18,0.45,0.27,0.51,0.89].

[0033] Specifically, the confirmation result of each early warning event (true, false, or missed) can be used as reverse feedback to automatically input into the model update module, so that the model can continuously correct the classification boundary and update the risk assessment parameters during continuous operation, thereby forming a model reinforcement whose recognition ability continuously improves with the running time.

[0034] Optionally, you can also... A direct mapping is established between the engineering semantics of specific hazard types, thereby enhancing the interpretability and physical consistency of features. Anomaly patterns of each hazard type in the frequency and time domains are statistically analyzed using years of operational data. A category correlation matrix M is constructed to describe the statistical coupling strength between energy distribution in different frequency bands and hazard categories. Category-level time-frequency correlation scores are constructed using normalized tensor products. : in, Its N-dimensional time-frequency window energy vector For the first Normalized energy fraction for each time-frequency window. This is an N×K dimensional category correlation matrix, where k is the total number of hazard categories. For the first The characteristic frequency band and the first The statistical coupling strength of a category of hidden dangers refers to the degree of statistical correlation between the energy distribution of the i-th characteristic frequency band and the occurrence of the j-th type of hidden danger. It primarily reflects the indicative ability of the abnormal signal in that frequency band to indicate the corresponding hidden danger type and is a key parameter for constructing the category correlation matrix. Its calculation is based on historical fault data and multi-source operational data accumulated by the system. It involves statistically analyzing the actual occurrence probability of the j-th type of hidden danger during the period when the normalized energy anomaly occurs in the i-th characteristic frequency band, the co-occurrence frequency of the anomaly in that frequency band when the hidden danger occurs, and combining mutual information gain to quantify the statistical dependence between the two. Finally, after normalization, a quantized value in the range of 0 to 1 is obtained. A higher value indicates a stronger correlation between the characteristic frequency band and the corresponding hidden danger. This means performing a tensor product operation between the time-frequency window energy vector and the category correlation matrix to obtain a tensor of dimension N×N×K. Indicates extraction and the first The dimensions corresponding to each type of hidden danger are summed and compressed into scalars. As a global normalization factor, it can ensure The quantized score is in the range of 0 to 1, matching the engineering application scenario. (The above...) The determination method was achieved by projecting the original time-frequency features without category attributes onto the hazard type space, thus... It is no longer merely a mathematical quantity at the signal level, but a risk interpretation quantity with a clear direction. It can be... Incorporating the extended comprehensive feature vector F enhances the stability of classification boundary updates and risk assessment parameter corrections in subsequent steps.

[0035] Example 2 Figure 2 This is a flowchart of a hidden danger early warning method provided in Embodiment 2 of the present invention. The technical solution of the present invention is further optimized based on the above optional technical solutions, and provides a specific method for early warning of hidden dangers in transmission lines.

[0036] Optionally, the preprocessing of the multimodal monitoring data to obtain preprocessed data includes: determining a first median of the multimodal monitoring data and a second median of the absolute deviation of the multimodal monitoring data; determining the quotient of the difference between the multimodal monitoring data and the first median and the second median, and using the quotient to determine the robustness score of each data point in the multimodal monitoring data; removing multimodal monitoring data with robustness scores greater than a preset threshold to obtain first preprocessed data, wherein the preset threshold is positively correlated with the false alarm rate of historical hazard identification results and negatively correlated with the false negative rate of historical hazard identification results; wherein, the feature extraction of the preprocessed data to obtain multimodal features includes: performing feature extraction on the first preprocessed data to obtain multimodal features.

[0037] Optionally, after obtaining the first preprocessed data, the method further includes: standardizing the dimensions of the first preprocessed data to obtain the second preprocessed data; determining the cross-correlation peak ratio of the second preprocessed data, wherein the cross-correlation peak ratio is the ratio of the peak value of the cross-correlation function of the signals of different modalities in the second preprocessed data to the cross-correlation function values ​​other than the peak value; determining a time-series alignment window based on the product of a preset coefficient and the cross-correlation peak ratio and a historical time-series alignment window, and performing time-series alignment on the second preprocessed data based on the time-series alignment window to obtain the third preprocessed data, wherein the preset coefficient is determined based on the accuracy of the historical hazard identification results; wherein, the step of extracting features from the first preprocessed data to obtain multimodal features includes: extracting features from the third preprocessed data to obtain multimodal features.

[0038] like Figure 2 As shown, the hazard warning method provided in Embodiment 2 of the present invention specifically includes the following steps: S201. Acquire multimodal monitoring data, which includes image data, infrared thermal imaging data, vibration and tension data, and environmental and power grid operation data.

[0039] S202. Determine the first median of the multimodal monitoring data and the second median of the absolute deviation of the multimodal monitoring data; determine the quotient of the difference between the multimodal monitoring data and the first median and the second median, and use the quotient to determine the robustness score of each data in the multimodal monitoring data; remove multimodal monitoring data with robustness scores greater than a preset threshold to obtain the first preprocessed data.

[0040] The preset threshold is positively correlated with the false alarm rate of historical hazard identification results and negatively correlated with the false negative rate of historical hazard identification results.

[0041] Specifically, robust anomaly detection based on median absolute deviation can be used to identify outliers in the time series of each sensor data point in multimodal monitoring data. Specifically, the median *m* and median absolute deviation *MAD* in the sequence are first calculated, and then the robust normalization factor *r* (i.e., the score of each modality in the multimodal monitoring data) is calculated. Where 'a' is an empirical constant selected based on the statistical characteristics of the normal distribution, which can be taken as 0.6745, and 'x' represents multimodal monitoring data. When 'r' is greater than the current preset threshold T, the data corresponding to 'r' is marked as an anomaly and processed according to priority for retention or deletion. The data priority is determined comprehensively based on the safety priority of the transmission line operation scenario, the reliability weight of the data channel, and the correlation with potential hazards. Anomaly data from high-reliability channels directly related to core safety, such as insulator contamination and conductor tension, can be set as high priority and retained to avoid missing key hazard signals. If the time-frequency characteristics and infrared temperature rise of the anomaly data have a high mutual information gain with historical hazard data, their priority is increased for retention. The processing strategy for anomaly data includes interpolation reconstruction or retention as candidate anomalies for subsequent manual confirmation or model re-judgment.

[0042] For example, the preset threshold can be updated in the following way: in, Indicates the historical preset threshold. The current preset thresholds are: FP represents the false alarm rate of historical hazard identification results, and FN represents the false negative rate of historical hazard identification results. The learning rate is represented by the above update method, which means that the cleaning intensity is relaxed when false alarms increase to reduce feature loss caused by over-cleaning, and the cleaning intensity is tightened when false negatives increase to improve the anomaly retention capacity, so that the preprocessing results and the actual hazard identification performance form a stable positive feedback. The system automatically updates based on the exponential decay rule of historical recognition accuracy. Through stable and convergent parameter adjustments, it ensures that the window width update always matches the temporal correlation characteristics of the data.

[0043] S203. Standardize the dimensions of the first preprocessed data to obtain the second preprocessed data; determine the cross-correlation peak ratio of the second preprocessed data; determine the time alignment window based on the product of the preset coefficient and the cross-correlation peak ratio and the historical time alignment window, and perform time alignment on the second preprocessed data based on the time alignment window to obtain the third preprocessed data.

[0044] The preset coefficient is determined based on the accuracy of the historical hazard identification results, and the cross-correlation peak ratio is the ratio of the peak value of the cross-correlation function of the signals of different modalities in the second preprocessed data to the cross-correlation function values ​​other than the peak value.

[0045] Specifically, dynamic standardization can be implemented for the dimensions of different sensors. A weighted Z-score method within a sliding window is used to compensate for the offset and scale drift of the first preprocessed data, with the weights determined by the data integrity and signal-to-noise ratio of the nearest time intervals. Then, an adaptive window alignment algorithm based on cross-correlation is used to perform temporal alignment on the second preprocessed data. The peak cross-correlation ratio of two or more sensor signals in the second preprocessed data is then used. As an alignment confidence metric, the time-series alignment window width W is adjusted according to this ratio to ensure consistency across different data sources on the time-series axis.

[0046] The timing alignment window width W can be adjusted in the following ways: in, Indicates the current time-aligned window width. Indicates the width of the historical time-aligned window. This is the window adjustment factor. The above formula means that when the alignment confidence index increases, the width of the time-series alignment window should be appropriately increased to improve its capacity to accommodate complex time differences; conversely, when the confidence index decreases, the width of the time-series alignment window should be decreased to reduce the risk of alignment shift. When the alignment confidence index increases, By appropriately widening the window in conjunction with the alignment confidence index, the ability to accommodate complex time differences is enhanced. When the alignment confidence index decreases, This helps to shrink the window and reduce the risk of alignment misalignment. Among them, complex time difference refers to the non-fixed and fluctuating time asynchrony phenomenon caused by the differences in the acquisition principles, sampling frequencies, and transmission delays of different data sources and the influence of environmental interference during the acquisition and transmission of multimodal monitoring data of transmission lines.

[0047] S204. Perform feature extraction on the third preprocessed data to obtain multimodal features.

[0048] For example, convolutional neural networks can be used to extract geometric deformation and surface defect features of conductors, insulators, and towers from image data in the third preprocessed data. Thermal imaging analysis algorithms can be used to extract hotspot distribution and temperature rise anomaly features from infrared thermal imaging data in the third preprocessed data. Time-frequency analysis can be used to extract abnormal vibration frequencies and tension fluctuation thresholds from vibration and tension data in the third preprocessed data. Statistical models can be used to extract dynamic correlation features between environmental parameters and operating status from environmental and power grid operation data in the third preprocessed data.

[0049] S205. Utilize the historical contribution of each modal monitoring data in historical hazard identification to update the historical weight coefficients to obtain the current weight coefficients; use the current weight coefficients to perform weighted fusion of the multimodal features to obtain comprehensive features characterizing hazard risk.

[0050] Optionally, the method for determining the historical contribution includes: performing a quality assessment on the data of each modality in the preprocessed data to obtain a corresponding quality assessment index, and using the quality assessment index to determine the reliability index of each modality monitoring data in the multimodal monitoring data; determining the mutual information between each modality feature and the hazard label in the historical multimodal features, and determining the label entropy of the hazard label; determining the quotient of the mutual information and the label entropy, and determining the product of the quotient and the reliability index as the historical contribution.

[0051] For example, historical contribution The determination method can be: in, As a reliability indicator, For the quality assessment metrics of the i-th modality data, it can include the proportion of outlier samples determined after preprocessing and the effectiveness of the processing, the signal-to-noise ratio after sliding window weighted Z-score standardization, and The quality quantification value of a single dimension, such as processing effectiveness, can be understood as the degree of consistency between the processing result and the subsequent feedback on hidden dangers and the actual hidden danger situation after performing operations such as interpolation reconstruction or marking as candidate anomalies on the identified abnormal samples. Here, b is the reliability sensitivity coefficient, and b is the bias term. The purpose of determining the method is to map data quality to a reliability scale of 0 to 1, so as to directly affect the initial value of the weight coefficient. For the i-th modal feature The mutual information between the hazard label Y and the hazard tag Y This represents the label entropy. Wherein, These are historical values, calculated based on historical data from the most recent rounds of identification. Specifically, they are obtained by statistically analyzing the features of the i-th modality in historical iterations. The cumulative correlation with the corresponding hazard label Y is a historical statistic that reflects the long-term discriminative value of the feature. The determination method and the attention weights described above The determination method is characterized by incorporating both data quality and statistical contribution into the driving factors of attention updates. This allows features with high contribution indicators to gradually increase their corresponding attention weights during iterations, thereby obtaining a higher weight ratio in the weighted fusion of multimodal features and participating more fully in the construction of comprehensive features. Meanwhile, the weights of features with low contribution are gradually weakened, ultimately achieving the effect of strengthening effective features and suppressing ineffective features, thus improving the discriminative power of comprehensive features.

[0052] S206. Determine the time-frequency window of the vibration and tension data signals in the preprocessed data, and determine the power spectral density for the signals within each time-frequency window; determine the time-frequency anomaly evaluation index for characterizing the proportion of abnormal energy based on the distribution characteristics of the power spectral density, and generate extended comprehensive features using the time-frequency anomaly evaluation index and the comprehensive features; output the hazard identification result using a preset hazard identification model based on the extended comprehensive features and the time-frequency anomaly evaluation index.

[0053] Specifically, extended comprehensive features can be implemented. Evaluation indicators of time and frequency anomalies As input to the preset hazard identification model, the preset hazard identification model can output the hazard category probability (vector) P, and the hazard identification result can be determined based on this probability.

[0054] Optionally, to overcome probability bias and sample distribution drift during long-term online operation, a feedback calibration mechanism can be used to define a calibration gain term. The normalized difference of the true confirmation statistics in the most recent sliding window: in, and The first The count of true positive cases and the count of false alarms for each type of potential hazard within the sliding window. This is a preset value to prevent the denominator from being zero. The calibration gain is used to adjust the logarithmic odds of the original probability to obtain the calibration probability. The calibration probability is determined as follows: in, As the feedback sensitivity coefficient, the above formula can suppress the category probability of hazard categories with historically high false alarm rates, while enhancing the category probability of hazard categories with historically high hit rates, thereby improving the reliability of the model's immediate decision-making. A risk assessment function can be constructed based on the calibration probability to simultaneously reflect identification confidence, inherent hazard hazard severity, and multimodal contribution. The risk score R can be defined as: in, The hazard weight corresponding to the j-th type of hazard is defined based on expert analysis and historical loss statistics. Weighting coefficient. The method for determining it is as follows: Attention weight Reliability indicators The coupling mapping is represented and normalized: in, The above formula represents the contribution mapping coefficient from the learned channel to the hazard type. Its purpose is to incorporate modal-level quality information and historical contributions into the decision-making process, giving preference to high-quality, high-contribution channels in risk calculation. To achieve online self-learning model reinforcement, incremental training can be performed using parameter update rules with feedback regularization terms. The update method is as follows: in These are the current model parameters. Based on the loss of the current batch of labeled data, The basic learning rate, As a feedback-driven weight, the feedback term is adjusted by gain. Amplifying or suppressing gradient contributions of various categories can quickly correct long-term biases. The purpose of the above equation is to make the model parameters driven by confirmation feedback on the basis of traditional gradient descent, thereby gradually converging to a solution that is more sensitive to the actual distribution of hidden dangers during continuous operation. A calibration probability vector is output in each processing cycle. Risk score R, and updated model parameters These outputs, along with the statistics of the most recent window, are then passed to the multi-level early warning decision module in step five to support dynamic grading and strategy optimization.

[0055] S207. Issue an early warning based on the identified hazard results.

[0056] The hazard early warning method provided in this invention integrates heterogeneous data from multiple sources, including images, infrared, vibration, tension, environmental data, and power grid operating status. This achieves full-time and full-weather coverage and multi-dimensional information complementarity at the perception level, significantly improving perception reliability and hazard identification range in extreme scenarios such as severe weather, nighttime, or visual obstruction (e.g., identifying hazards that cannot be directly detected visually, such as icing overload, conductor galloping, and insulator contamination). Simultaneously, this method dynamically updates the preset threshold and alignment window width of the preprocessing through feedback, and iteratively updates the attention weights based on contribution during the feature fusion stage, forming a positive cycle of continuous optimization across three key aspects: data quality, feature extraction effectiveness, and model judgment accuracy.

[0057] Example 3 Figure 3 This is a schematic diagram of a potential hazard warning device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a preprocessing module 301, a comprehensive feature extraction module 302, and a hazard warning module 303, wherein: The preprocessing module is used to acquire multimodal monitoring data and preprocess the multimodal monitoring data to obtain preprocessed data. The processing parameters involved in the preprocessing are determined based on the historical hazard identification results. The comprehensive feature extraction module is used to extract features from the preprocessed data to obtain multimodal features, and to fuse the multimodal features to obtain comprehensive features that characterize the risk of hidden dangers. The fusion parameters involved in the feature fusion are determined based on the historical contribution of each modal monitoring data in historical hidden danger identification. The hazard warning module is used to identify hazards based on the comprehensive features using a preset hazard identification model, so as to obtain the hazard identification result and issue a warning based on the hazard identification result.

[0058] The hidden danger early warning device provided in this embodiment of the invention integrates the features of multimodal monitoring data and updates the preprocessing parameters and feature fusion parameters according to historical data, thus constructing an intelligent hidden danger early warning system with multi-source perception, closed-loop feedback and performance self-enhancement. It has the ability to self-optimize, can not only cope with complex scenarios, but also become more and more intelligent in long-term operation, and finally achieve continuous autonomous improvement in early warning accuracy and operation and maintenance efficiency.

[0059] Optional, the preprocessing module includes: The median determination unit is used to determine the first median of the multimodal monitoring data and the second median of the absolute deviation of the multimodal monitoring data. A score determination unit is used to determine the quotient of the difference between the multimodal monitoring data and the first median and the second median, and to use the quotient to determine the robust score of each data in the multimodal monitoring data; A data elimination unit is used to eliminate multimodal monitoring data with robustness scores greater than a preset threshold to obtain first preprocessed data, wherein the preset threshold is positively correlated with the false alarm rate of historical hazard identification results and negatively correlated with the false alarm rate of historical hazard identification results. The comprehensive feature extraction module includes: The first feature extraction unit is used to extract features from the first preprocessed data to obtain multimodal features.

[0060] Optionally, the preprocessing module may also include: The standardization unit is used to standardize the dimensions of the first preprocessed data after obtaining the first preprocessed data, so as to obtain the second preprocessed data. A ratio determination unit is used to determine the cross-correlation peak ratio of the second preprocessed data, wherein the cross-correlation peak ratio is the ratio of the peak value of the cross-correlation function of the signals of different modes in the second preprocessed data to the cross-correlation function values ​​other than the peak value. The time-series alignment unit is used to determine a time-series alignment window based on the product of a preset coefficient and the cross-correlation peak ratio and a historical time-series alignment window, and to perform time-series alignment on the second preprocessed data based on the time-series alignment window to obtain third preprocessed data, wherein the preset coefficient is determined based on the accuracy of the historical hidden danger identification results. The comprehensive feature extraction module includes: The second feature extraction unit is used to extract features from the third preprocessed data to obtain multimodal features.

[0061] Optionally, the multimodal monitoring data includes image data, infrared thermal imaging data, vibration and tension data, and environmental and power grid operation data.

[0062] Optionally, the comprehensive feature extraction module includes: The weight coefficient determination unit is used to update the historical weight coefficients by utilizing the historical contribution of each modal monitoring data in the historical hazard identification in the historical multimodal monitoring data, so as to obtain the current weight coefficients; The weighted fusion unit is used to perform weighted fusion of the multimodal features using the current weight coefficients to obtain comprehensive features that characterize potential risks.

[0063] Furthermore, the method for determining the historical contribution includes: performing quality assessment on the data of each modality in the preprocessed data to obtain the corresponding quality assessment index, and using the quality assessment index to determine the reliability index of each modality monitoring data in the multimodal monitoring data; determining the mutual information between each modality feature and the hazard label in the historical multimodal features, and determining the label entropy of the hazard label; determining the quotient of the mutual information and the label entropy, and determining the product of the quotient and the reliability index as the historical contribution.

[0064] Optional, the hazard warning module includes: A power spectral density determination unit is used to determine the time-frequency window of the vibration and tension data signals in the preprocessed data, and to determine the power spectral density for the signals within each time-frequency window. An extended comprehensive feature determination unit is used to determine a time-frequency anomaly evaluation index for characterizing the proportion of anomalous energy based on the distribution characteristics of the power spectral density, and to generate extended comprehensive features using the time-frequency anomaly evaluation index and the comprehensive features. The hazard identification unit is used to output hazard identification results based on the extended comprehensive features and the time-frequency anomaly evaluation index using a preset hazard identification model.

[0065] The hazard warning device provided in the embodiments of the present invention can execute the hazard warning method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0066] Example 4 Figure 4 A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0067] like Figure 4As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from storage unit 48 into the RAM 43. The RAM 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0068] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0069] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as hazard warning methods.

[0070] In some embodiments, the hazard warning method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the hazard warning method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the hazard warning method by any other suitable means (e.g., by means of firmware).

[0071] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoC) systems, complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0072] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0073] The computer equipment provided above can be used to execute the hidden danger warning method provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0074] Example 5 In the context of this invention, the computer-readable storage medium may be a tangible medium, and the computer-executable instructions, when executed by a computer processor, are used to perform a hazard warning method, the method comprising: Acquire multimodal monitoring data and preprocess the multimodal monitoring data to obtain preprocessed data, wherein the processing parameters involved in the preprocessing are determined based on the historical hazard identification results; Feature extraction is performed on the preprocessed data to obtain multimodal features, and feature fusion is performed on the multimodal features to obtain comprehensive features characterizing hidden danger risks. The fusion parameters involved in feature fusion are determined based on the historical contribution of each modality monitoring data in historical hidden danger identification. A hazard identification model is used to identify hazards based on the comprehensive features to obtain hazard identification results, and an early warning is issued based on the hazard identification results.

[0075] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by, or in conjunction with, an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0076] The computer equipment provided above can be used to execute the hidden danger warning method provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0077] It is worth noting that in the embodiments of the above-mentioned hidden danger warning device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0078] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for early warning of potential hazards, characterized in that, include: Acquire multimodal monitoring data and preprocess the multimodal monitoring data to obtain preprocessed data, wherein the processing parameters involved in the preprocessing are determined based on the historical hazard identification results; Feature extraction is performed on the preprocessed data to obtain multimodal features, and feature fusion is performed on the multimodal features to obtain comprehensive features characterizing hidden danger risks. The fusion parameters involved in feature fusion are determined based on the historical contribution of each modality monitoring data in historical hidden danger identification. A hazard identification model is used to identify hazards based on the comprehensive features to obtain hazard identification results, and an early warning is issued based on the hazard identification results.

2. The method according to claim 1, characterized in that, The preprocessing of the multimodal monitoring data to obtain preprocessed data includes: Determine the first median of the multimodal monitoring data, and determine the second median of the absolute deviation of the multimodal monitoring data; Determine the quotient between the difference between the multimodal monitoring data and the first median and the second median, and use the quotient to determine the robustness score of each data point in the multimodal monitoring data; Multimodal monitoring data with robustness scores greater than a preset threshold are removed to obtain the first preprocessed data. The preset threshold is positively correlated with the false alarm rate of historical hazard identification results and negatively correlated with the false negative rate of historical hazard identification results. The step of extracting features from the preprocessed data to obtain multimodal features includes: Feature extraction is performed on the first preprocessed data to obtain multimodal features.

3. The method according to claim 2, characterized in that, After obtaining the first preprocessed data, the process also includes: The dimensions of the first preprocessed data are standardized to obtain the second preprocessed data; Determine the peak cross-correlation ratio of the second preprocessed data, wherein the peak cross-correlation ratio is the ratio of the peak value of the cross-correlation function of the signals of different modes in the second preprocessed data to the cross-correlation function values ​​other than the peak value; Based on the product of the preset coefficient and the cross-correlation peak ratio and the historical time-series alignment window, a time-series alignment window is determined, and based on the time-series alignment window, the second preprocessed data is time-series aligned to obtain the third preprocessed data, wherein the preset coefficient is determined according to the accuracy of the historical hidden danger identification results; The step of extracting features from the first preprocessed data to obtain multimodal features includes: Feature extraction is performed on the third preprocessed data to obtain multimodal features.

4. The method according to claim 1, characterized in that, The multimodal monitoring data includes image data, infrared thermal imaging data, vibration and tension data, and environmental and power grid operation data.

5. The method according to claim 1 or 4, characterized in that, The feature fusion of the multimodal features to obtain comprehensive features characterizing potential risks includes: By utilizing the historical contribution of each modal monitoring data in the historical hazard identification in the historical multimodal monitoring data, the historical weight coefficients are updated to obtain the current weight coefficients; The multimodal features are weighted and fused using the current weighting coefficients to obtain comprehensive features that characterize potential risks.

6. The method according to claim 1, characterized in that, The methods for determining the historical contribution include: The quality of each modality in the preprocessed data is evaluated to obtain the corresponding quality evaluation index, and the reliability index of each modality monitoring data in the multimodal monitoring data is determined using the quality evaluation index. Determine the mutual information between each modal feature and the hazard label in the historical multimodal features, and determine the label entropy of the hazard label; The quotient of the mutual information and the tag entropy is determined, and the product of the quotient and the reliability index is determined as the historical contribution.

7. The method according to claim 1 or 4, characterized in that, The step of using a preset hazard identification model to identify hazards based on the comprehensive features, in order to obtain hazard identification results, includes: Determine the time-frequency window of the vibration and tension data signals in the preprocessed data, and determine the power spectral density for the signal within each time-frequency window; Based on the distribution characteristics of the power spectral density, a time-frequency anomaly evaluation index for characterizing the proportion of anomalous energy is determined, and an extended comprehensive feature is generated using the time-frequency anomaly evaluation index and the comprehensive feature. Using a preset hazard identification model, based on the extended comprehensive features and the time-frequency anomaly evaluation index, the hazard identification result is output.

8. A hazard warning device, characterized in that, include: The preprocessing module is used to acquire multimodal monitoring data and preprocess the multimodal monitoring data to obtain preprocessed data. The processing parameters involved in the preprocessing are determined based on the historical hazard identification results. The comprehensive feature extraction module is used to extract features from the preprocessed data to obtain multimodal features, and to fuse the multimodal features to obtain comprehensive features that characterize the risk of hidden dangers. The fusion parameters involved in the feature fusion are determined based on the historical contribution of each modal monitoring data in historical hidden danger identification. The hazard warning module is used to identify hazards based on the comprehensive features using a preset hazard identification model, so as to obtain the hazard identification result and issue a warning based on the hazard identification result.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the hazard warning method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the hazard warning method according to any one of claims 1-7.

Images