A Method and System for Holographic Anomaly Detection of Circuit Lines Based on Cross-Modal Intelligent Collaboration

Abstract

This invention relates to the field of power line inspection technology, providing a method and system for holographic anomaly detection of power lines based on cross-modal intelligent collaboration. The method includes: multimodal data acquisition; cross-modal fusion to generate correlation tensors; anomaly joint inference using temporal graph neural networks and reinforcement learning to output anomaly confidence scores; dynamic knowledge-driven decision-making through Bayesian computation and transfer learning to adaptively adjust detection thresholds; hierarchical edge computing to achieve local real-time response; and multi-objective collaborative optimization feedback to improve detection accuracy. The system consists of a multimodal perception fusion layer, an intelligent analysis layer, an edge execution layer, and an optimization control layer. This invention solves the problems of isolated multimodal information, response delay, and environmental adaptability, significantly improving defect detection rate and system robustness.

Description

Technical Field

[0001] This invention belongs to the field of power line inspection technology, and relates to a method and system for detecting line holographic anomalies based on cross-modal intelligent collaboration. Background Technology

[0002] With the rapid development of smart grid construction, the demand for intelligent inspection of 10kV distribution lines is becoming increasingly urgent. Traditional line inspection mainly relies on a combination of regular manual inspections and fixed monitoring devices, which suffers from problems such as low efficiency, numerous blind spots, and poor real-time performance. For example, traditional manual inspection requires inspectors to walk along the line, relying on experience for visual inspection or using equipment such as infrared thermometers and telescopes to determine defects. This method is not only inefficient but also has unstable inspection quality, especially at night or when inspecting equipment at height. In addition, manual inspection also faces significant safety hazards, such as close contact with live equipment, high risks in mountainous areas, and inability to carry out work normally under extreme weather conditions.

[0003] In recent years, robot inspection systems based on single sensors have gradually become a research hotspot, but existing technologies still have many shortcomings. For example, visual inspection systems typically identify defects such as broken conductor strands and damaged insulators based on traditional image processing algorithms. However, because they rely on only a single visual modality, their performance degrades significantly under complex lighting conditions, and they struggle to detect non-visible defects such as overheating and partial discharge, resulting in decreased accuracy at night. Furthermore, these systems suffer from poor navigation flexibility, relying on pre-installed RFID tags for track navigation, leading to high deployment costs and an inability to adapt to temporary line changes. In addition, data processing is entirely cloud-based, with an average delay of up to 3 seconds from detection to alarm, failing to meet real-time requirements. False alarm rates increase in rainy conditions, and positioning accuracy significantly decreases in environments with strong electromagnetic interference, further limiting their application scope.

[0004] Currently, while modern intelligent inspection systems are beginning to integrate various sensing devices such as infrared thermal imagers, visible light cameras, lidar, and ultrasonic sensors to achieve multi-dimensional detection of line equipment, multi-sensor fusion technology remains at a simple data or feature layer fusion stage, struggling to achieve deep cross-modal information correlation. For example, the lack of joint analysis between thermal imaging data and equipment vibration characteristics makes it impossible to accurately identify complex defects caused by mechanical deformation. Furthermore, traditional algorithms use fixed thresholds to determine abnormal states, failing to adapt to environmental influences such as seasonal changes, weather conditions, and load fluctuations, easily leading to misjudgments or missed detections. The centralized processing of massive amounts of data in the cloud also results in significant delays in the identification and response to critical defects, a problem further exacerbated by the rigid allocation of computing power at the edge. Summary of the Invention

[0005] This invention addresses the problems of isolated multimodal sensing information, lack of environmental adaptation mechanism, contradiction between response delay and computing power, insufficient cross-scenario generalization ability, and imbalance of multi-objective optimization in existing intelligent inspection technology for 10kV distribution networks. This invention provides a method and system for holographic anomaly detection of lines based on cross-modal intelligent collaboration.

[0006] This invention adopts the following technical solution: a method for detecting line holographic anomalies based on cross-modal intelligent collaboration, comprising the following steps:

[0007] Step 1. Multimodal data acquisition: Simultaneously acquire thermal infrared imaging data, lidar point cloud data, and acoustic signature data. Establish a temperature field distribution model based on the thermal infrared imaging data to obtain temperature field parameters. Fit the geometry of the pole using lidar point cloud data and model the direction of the pole's axis in three-dimensional space to calculate the pole's tilt angle. Perform wavelet packet decomposition on the acoustic signature data to obtain acoustic signature features.

[0008] Step 2. Cross-modal fusion: By jointly calculating temperature field parameters, pole deformation data, and acoustic signature features, the cross-modal correlation tensor is obtained;

[0009] Step 3. Anomaly Joint Reasoning: Align the dynamic electrical topology graph with the cross-modal correlation tensor, construct the initial node feature matrix and input it into the spatiotemporal correlation graph neural network, analyze the pole topology relationship based on the spatiotemporal correlation graph neural network, combine physical constraint reinforcement learning to dynamically optimize the detection strategy, and output anomaly confidence parameters containing various defects;

[0010] Step 4. Dynamic knowledge-driven decision-making: Convert the anomaly confidence parameter into likelihood probability and prior probability, and realize rapid generalization reasoning of cross-scene defect features and adaptive adjustment of detection threshold through Bayesian anomaly confidence calculation and online transfer learning;

[0011] Step 5. Layered edge computing and scheduling: Based on the adaptively adjusted detection threshold and defects in the range above the anomaly confidence threshold, a lightweight inference engine is deployed on the inspection robot. The localized real-time identification and decision-making of key defects are realized through the lightweight edge inference engine and the adaptive computing power allocation mechanism.

[0012] Step 6. Multi-objective collaborative optimization: Multi-objective collaborative optimization is performed using the overall detection rate and false alarm attenuation ratio as evaluation indicators, and the optimization results are fed back to the physical constraint reinforcement learning stage in Step 3.

[0013] Specifically, the cross-modal correlation tensor is represented by the following formula:

[0014] ;

[0015] in, For the cross-modal correlation tensor, i is the index of the thermal infrared imaging data; j is the index of the lidar point cloud data; and k is the index of the acoustic text data. This indicates the value of the i-th thermal infrared imaging data. Thermal infrared imaging features obtained by performing Gaussian kernel transformation for thermal imaging; For point cloud spatial encoding functions, This represents the point cloud data of the j-th lidar sensor. Spatial information of lidar point cloud obtained through processing and encoding; For the k-th voiceprint data, This indicates the use of acoustic wavelet packet decomposition operators to decompose pairs of... Voiceprint features obtained by wavelet packet decomposition.

[0016] Specifically, in step 3, pole coordinates and line connection relationships are extracted from the power grid company's GIS database to construct a spatial topology network, integrate circuit breaker and disconnector status data, generate a dynamic electrical topology map, align the dynamic electrical topology map with the cross-modal correlation tensor, and construct an initial node feature matrix.

[0017] Specifically, the spatiotemporal correlation graph neural network iteratively calculates the input initial node feature matrix, and the final layer outputs the anomaly confidence parameters of each node.

[0018] Specifically, the Physically Constrained Reinforcement Learning (PCRL) constructs the reward function as follows:

[0019] ;

[0020] in, The cumulative reward value is denoted by t, where t is the time and T is the total training time. To improve detection accuracy Weighting coefficients; For false alarm rate The penalty coefficient; For energy costs The penalty coefficient; physical constraint reinforcement learning optimizes the reward function and dynamically adjusts the weights of the spatiotemporal correlation graph neural network, providing real-time optimized values ​​for the likelihood probability correction term for Bayesian computation, and incorporating spatiotemporal correlation constraints of multimodal features.

[0021] Specifically, in step 4, the posterior probability of various defects is calculated using Bayesian anomaly confidence.

[0022] Specifically, in step 5, the adaptive computing power allocation mechanism adopts a sliding window, defined as:

[0023] ;

[0024] ;

[0025] in, Forgetting factor; Let be the dynamic mean at time t; The dynamic mean at time t-1; Let be the dynamic variance at time t. Let be the dynamic variance at time t-1; Let be the observed value at time t.

[0026] The present invention also provides a line holographic anomaly detection system based on cross-modal intelligent collaboration, including a multimodal perception fusion layer, an intelligent analysis layer, an edge execution layer and an optimization control layer;

[0027] The multimodal perception fusion layer consists of a thermal infrared imaging infrared sensor, a lidar sensor, an acoustic signature acquisition sensor, and a multimodal fusion model.

[0028] A multimodal fusion model is used to receive temperature field parameters, pole tilt angle and acoustic signature features, construct cross-modal correlation tensors, and transmit them to the joint inference engine for abnormal states;

[0029] The intelligent analysis layer includes an abnormal state joint reasoning engine and a dynamic knowledge-driven decision framework;

[0030] The joint reasoning engine for abnormal states includes a spatiotemporal correlation graph neural network and a physical constraint reinforcement learning module. It aligns the dynamic electrical topology graph with the cross-modal correlation tensor, constructs an initial node feature matrix and inputs it into the spatiotemporal correlation graph neural network, analyzes the pole topology relationship based on the spatiotemporal correlation graph neural network, dynamically optimizes the detection strategy by combining physical constraint reinforcement learning, and outputs anomaly confidence parameters that include various defects.

[0031] The dynamic knowledge-driven decision framework converts the anomaly confidence parameter into likelihood probability and prior probability, and achieves rapid generalization reasoning of cross-scenario defect features and adaptive adjustment of detection threshold through Bayesian anomaly confidence calculation and online transfer learning.

[0032] The edge execution layer includes a layered edge computing and resource scheduling module and an inspection robot;

[0033] The hierarchical edge computing and resource scheduling module deploys a lightweight inference engine on the inspection robot to provide millisecond-level local responses to high-risk defects; it dynamically allocates computing power through a sliding window mean drift algorithm: it elastically schedules CPU / GPU resources according to the urgency of the task; and it drives the inspection robot to perform actions after receiving decision instructions.

[0034] Inspection robots precisely execute inspection tasks: based on hierarchical edge computing and resource scheduling modules, they navigate to defect points, perform emergency handling, and continuously track status changes. Execution performance data is transmitted back to the optimization control layer in real time.

[0035] The optimization control layer includes a multi-objective collaborative optimization mechanism and a cloud-based knowledge base.

[0036] Furthermore, the multi-objective collaborative optimization mechanism globally balances detection accuracy, false alarm rate, and energy consumption cost. After analyzing the feedback data from the inspection robot, it dynamically adjusts the detection strategy and updates the cloud knowledge base. The cloud knowledge base stores historical fault case databases, equipment topology maps, and environmental adaptation parameters. It provides prior knowledge of physical laws to the dynamic knowledge-driven decision-making framework and receives model parameters updated by the multi-objective collaborative optimization mechanism, forming a closed-loop evolutionary system.

[0037] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the computer program, when run by the processor, executes the steps of the above-described method for detecting line holographic anomalies based on cross-modal intelligent collaboration.

[0038] Beneficial effects of this invention:

[0039] By constructing a cross-modal correlation tensor, multi-source data such as thermal infrared imaging, lidar point clouds, and acoustic signature features are deeply fused. Combined with a spatiotemporal graph neural network (ST-GNN), spatiotemporal correlation analysis and collaborative reasoning of multimodal data are realized, effectively solving the problem of isolated multimodal perception information in traditional technologies and significantly improving the comprehensive detection capability of complex defects.

[0040] By leveraging adaptive adjustment of detection thresholds and physical constraint reinforcement learning, the system can optimize detection strategies in real time based on environmental factors such as seasonal changes, weather conditions, and equipment load fluctuations. This enables adaptive adjustment of detection thresholds, effectively addressing detection needs under different environments and enhancing the system's adaptability to complex environments.

[0041] By employing a layered edge computing and resource scheduling module, a lightweight inference engine is deployed on the inspection robot to make local real-time decisions on high-risk defects. At the same time, computing resources are dynamically allocated according to task priority, which significantly shortens the detection response latency, balances the computing power allocation between the edge and the cloud, and improves the system's real-time performance and resource utilization efficiency.

[0042] By leveraging transfer learning, prior knowledge such as historical failure cases and equipment topology relationships can be transformed into reusable knowledge units, enabling rapid adaptation and generalization of the model in different scenarios, reducing dependence on data from new scenarios, and lowering the cost and time overhead of model reconstruction for cross-scenario deployment.

[0043] By designing a reward function that integrates detection accuracy, false alarm rate, and energy consumption cost, and combining the comprehensive detection rate formula with the false alarm attenuation ratio mechanism, differentiated weights are designed for the severity of different types of defects. This achieves multi-objective collaborative optimization of detection accuracy, system robustness, and energy consumption efficiency, avoiding system performance imbalance caused by optimizing a single indicator. Attached Figure Description

[0044] Figure 1 This is a flowchart of a line holographic anomaly detection method based on cross-modal intelligent collaboration.

[0045] Figure 2 This is a schematic diagram of the overall architecture of a line holographic anomaly detection system based on cross-modal intelligent collaboration. Detailed Implementation

[0046] The invention will now be explained in further detail with reference to the accompanying drawings.

[0047] Example 1

[0048] like Figure 1 As shown, a method for detecting line holographic anomalies based on cross-modal intelligent collaboration includes the following steps:

[0049] Step 1. Multimodal Data Acquisition: Simultaneously acquire thermal infrared imaging data, lidar point cloud data, and acoustic signature data. Establish a temperature field distribution model based on the thermal infrared imaging data to obtain temperature field parameters. Fit the geometry of the utility pole using lidar point cloud data and model the direction of the pole's axis in three-dimensional space to calculate the pole's tilt angle. Perform wavelet packet decomposition on the acoustic signature data to obtain acoustic signature features. This step outputs temperature field parameters, pole tilt angle, and acoustic signature features, providing a foundation for subsequent fusion analysis.

[0050] Step 2. Cross-modal fusion: By jointly calculating temperature field parameters, pole deformation data, and acoustic signature features, a cross-modal correlation tensor is obtained. For example, by fusing the coordinates of temperature anomalies with the corresponding acoustic signature pulses, the composite defects of insulator flashover accompanied by mechanical deformation can be accurately identified, overcoming the detection blind spots of single sensors at night or in environments with strong interference. This fusion mechanism effectively solves the single-mode detection blind spots under nighttime or strong electromagnetic interference conditions and outputs a cross-modal correlation tensor.

[0051] Step 3. Joint Anomaly Reasoning: Align the dynamic electrical topology graph with the cross-modal correlation tensor to construct an initial node feature matrix and input it into the spatiotemporal correlation graph neural network. Analyze the pole topology relationships based on the spatiotemporal correlation graph neural network, dynamically optimize the detection strategy using physical constraint reinforcement learning, and output anomaly confidence parameters encompassing various types of defects. Specifically, parameters such as temperature gradients and acoustic signatures of adjacent poles are used as node features to capture defect propagation paths (e.g., a pole tilting causing sag of adjacent conductors). Combined with physical constraint reinforcement learning, prior knowledge such as heat conduction is introduced to dynamically optimize the detection strategy, balancing high accuracy with low false alarm requirements. Through joint reasoning using the spatiotemporal graph neural network and physical constraint reinforcement learning, anomaly confidence parameters encompassing various types of defects are output.

[0052] Step 4. Dynamic Knowledge-Driven Decision Making: Anomaly confidence parameters are converted into likelihood and prior probabilities. Bayesian anomaly confidence calculation and online transfer learning enable rapid generalization inference of defect features across scenarios and adaptive adjustment of detection thresholds. Dynamic knowledge-driven decision making adjusts sensitivity in real-time based on historical fault probabilities. Online transfer learning automatically aligns with data distribution differences when the system is deployed to a new environment, completing adaptation within 7 days and significantly reducing cross-scenario reconstruction costs.

[0053] Step 5. Layered Edge Computing and Scheduling: Based on the adaptively adjusted detection threshold and defects exceeding the anomaly confidence threshold, a lightweight inference engine is deployed on the inspection robot. This lightweight edge inference engine, combined with an adaptive computing power allocation mechanism, enables localized real-time identification and decision-making for critical defects. Local real-time decisions are made for high-risk defects such as broken wire strands (defects exceeding the anomaly confidence threshold). Computing power is dynamically allocated using a sliding window mean-shift algorithm: resources are flexibly scheduled based on task priority, addressing the issues of high latency in traditional cloud processing and rigid computing power at the edge.

[0054] Step 6. Multi-objective collaborative optimization: Multi-objective collaborative optimization is performed using the overall detection rate and false alarm attenuation ratio as evaluation indicators, and the optimization results are fed back to the physical constraint reinforcement learning stage in Step 3.

[0055] In step 1 of this embodiment, the method for establishing the temperature field distribution model based on thermal infrared imaging data is as follows:

[0056] ;

[0057] in, For any point temperature, y is the x-coordinate of the pixel, and y is the y-coordinate of the pixel. To represent the number of heat spots in thermal infrared imaging, Let h be the x-coordinate of the h-th heating point. Let h be the ordinate of the h-th heat source. The heat source intensity of the h-th heating point, Let be the thermal diffusivity of the h-th heating point, and e be the natural constant.

[0058] In step 1 of this embodiment, the geometry of the utility pole is fitted using lidar point cloud data, and the direction of the pole's axis in three-dimensional space is modeled:

[0059] ;

[0060] in, These are the X, Y, and Z coordinates (in meters) of the center point of the pole's bottom in a Cartesian coordinate system, typically calculated from the centroid of lidar point cloud data. , , respectively, are the X, Y, and Z coordinates of any point on the pole in a spatial rectangular coordinate system; a, b, and c are the components of the pole's directional vectors along the X, Y, and Z axes, used to describe the pole's extension direction in space.

[0061] pole tilt angle calculate:

[0062] ;

[0063] like The pole was determined to be tilted abnormally.

[0064] In this embodiment, the formula for detecting abnormal voiceprint features is:

[0065] ;

[0066] Where t is time, The acoustic signature characteristics pre-recorded during normal operation of the transformer are called the reference MFCC coefficient; The transformer acoustic signature characteristics detected in real time are called the measured MFCC coefficients; The greater the difference between the transformer's acoustic signature and its normal state, the more abnormal the acoustic signature becomes. The higher the value, the better; the higher the detection threshold, when calculated in real time. More than 2.3 times When this happens, it is judged as abnormal. In historical normal data The average value.

[0067] In step 2 of this embodiment, the cross-modal correlation tensor is expressed by the following formula:

[0068] ;

[0069] in, For the cross-modal correlation tensor, i is the index of thermal infrared imaging data, such as different thermal infrared imaging image numbers; j is the index of lidar point cloud data, such as different point cloud set numbers or different spatial partitions of point clouds; k is the index of acoustic data, such as acoustic signals collected at different frequency bands or at different times. This indicates the value of the i-th thermal infrared imaging data. Thermal infrared imaging features obtained by performing Gaussian kernel transformation for thermal imaging; For point cloud spatial encoding functions, This represents the point cloud data of the j-th lidar sensor. Spatial information of lidar point cloud obtained through processing and encoding; For the k-th voiceprint data, This indicates the use of acoustic wavelet packet decomposition operators to decompose pairs of... Voiceprint features obtained by wavelet packet decomposition.

[0070] In step 3, pole coordinates and line connection relationships are extracted from the power grid company's GIS database to construct a spatial topology network. Circuit breaker and disconnector status data are integrated to generate a dynamic electrical topology map. The dynamic electrical topology map is then aligned with the cross-modal correlation tensor to construct an initial node feature matrix. :

[0071] ;

[0072] in, N is the number of pole nodes, and D is the node degree matrix; These are operation functions that change the dimensions and shape of the data. Indicates the number of dimensions of the output features.

[0073] The anomaly state joint inference engine combines a spatiotemporal correlation graph neural network (ST-GNN) with physical constraint reinforcement learning (PCRL) to achieve spatiotemporal correlation analysis of multi-dimensional parameters and dynamic optimization of detection strategies. Specifically, the ST-GNN defines a node feature matrix as follows:

[0074] ;

[0075] Where A is the pole connection adjacency matrix constructed based on the pole topology; It is the square root of the inverse of the degree matrix D; For graph neural networks The node feature matrix of the layer; For the first The learnable weight matrix of the layer, The activation function is calculated iteratively through a multi-layer spatiotemporal graph neural network (ST-GNN), and the final layer outputs the anomaly confidence parameters of each node.

[0076] The elements of the pole connection adjacency matrix A are:

[0077] ;

[0078] in, Let be the element in the nth row and mth column of the adjacency matrix A, used to represent the connection strength between the nth pole node and the mth pole node. and The three-dimensional coordinates of the nth pole and the mth pole (from...) ), This is the distance decay coefficient, used to control the decay rate of adjacent matrix elements as distance changes.

[0079] The Physically Constrained Reinforcement Learning (PCRL) constructs the reward function as follows:

[0080] ;

[0081] in, The cumulative reward value is denoted by t, where t is the time and T is the total training time. To improve detection accuracy The weighting coefficient represents the system's emphasis on correctly detecting abnormal events. In this embodiment... ; For false alarm rate The penalty coefficient represents the tolerance for false alarms. In this embodiment... ; For energy costs The penalty coefficient represents the constraint on system energy consumption in this embodiment. Physically constrained reinforcement learning optimizes the reward function and dynamically adjusts the weights of the spatiotemporal correlation graph neural network, providing real-time optimized values ​​for the likelihood probability correction term for Bayesian computation, and incorporating spatiotemporal correlation constraints of multimodal features.

[0082] The anomaly confidence parameters of each node in the final layer output of the spatiotemporal correlation graph neural network are as follows:

[0083] ;

[0084] in, Let the anomaly confidence score vector of the nth pole node be denoted as , and assuming a U-type defect, then , This represents the unnormalized confidence score for the existence of the u-th type of defect at the n-th pole node, where u = {1, 2, ..., U}, and U is the total number of defect categories.

[0085] In step 4, the anomaly confidence parameter is converted into likelihood probability and prior probability, as shown below:

[0086] ;

[0087] Historical statistical probability of type u defect in cloud-based knowledge base;

[0088] in, This represents the unnormalized confidence score indicating that the nth pole node has a type v defect. Let u represent the likelihood probability of the u-th type defect. Let u represent the prior probability of the u-th type defect. Let U represent the u-th type of defect, and E represent the observed abnormal feature of the n-th pole node; softmax is the activation function; the prior probability is dynamically adjusted through online transfer learning.

[0089] In step 4, the Bayesian anomaly confidence formula is defined as follows:

[0090] ;

[0091] in, Let v represent the likelihood probability of the v-th defect. This represents the prior probability of the v-th type of defect. Let be the posterior probability of the u-th type of defect.

[0092] The domain adaptation loss function used in the online transfer learning is:

[0093] ;

[0094] in, To adapt the loss function to the domain, This is the task loss function, which measures the performance error of the model on the target task (such as defect detection). To maximize the mean difference, the difference in data distribution between the source domain (training data) and the target domain (new route environment) is quantified. The domain adaptation strength coefficient controls the balance between distribution alignment and task performance.

[0095] In step 5, localized real-time identification and decision-making for key defects are achieved through a lightweight edge inference engine and an adaptive computing power allocation mechanism. The lightweight edge inference engine deploys a low-complexity model on the inspection robot, supporting real-time processing of high-priority tasks. The adaptive computing power allocation mechanism employs a sliding window mean-shift algorithm, defined as:

[0096] ;

[0097] ;

[0098] in, The forgetting factor controls the weight of historical data in the current statistic; the larger the value, the longer the influence of historical data lasts. The dynamic mean at time t reflects the baseline level of environmental parameters. The dynamic mean at time t-1; Let be the dynamic variance at time t, representing the range of fluctuation of the environmental parameters. Let be the dynamic variance at time t-1; The observed values ​​at time t (such as conductor temperature, vibration amplitude, etc.).

[0099] In step 6, the entire process data is integrated, and a comprehensive detection rate-weighted statistical defect hazard assessment is designed: broken wires are given a higher priority than surface corrosion, improving the detection rate of high-risk defects; a false alarm attenuation mechanism is used: false alarms caused by environmental interference are automatically reduced through physical constraint reinforcement learning, and the optimization results are fed back to the physical constraint reinforcement learning stage in step 3. A reward function that integrates detection accuracy, false alarm rate, and energy consumption cost is designed to achieve a multi-dimensional balance between detection accuracy, system robustness, and energy efficiency. The comprehensive detection rate formula is defined as follows:

[0100] ;

[0101] in, To achieve the overall detection rate, This represents the number of type u defects (such as broken wires, overheating, pole tilting, etc.) that were correctly detected. This represents the number of type u defects that actually exist but have not been detected (missed detections). is the weighting coefficient for the u-th type of defect, reflecting the degree of impact of different defects on power grid security.

[0102] The false alarm attenuation ratio formula is defined as follows:

[0103] ;

[0104] in, The false alarm attenuation ratio. This represents the number of false alarms at the current moment, i.e., the number of times the system has mistakenly identified normal events as defects. This represents the number of false alarms at the previous moment, used to compare the decay trend of false alarms. This is the learning rate decay coefficient, which controls the rate at which the number of false alarms decreases over time. It is a natural constant. The number of false alarms is attenuated and corrected over time, reflecting the effect of the system gradually reducing false alarms through continuous learning.

[0105] Example 2

[0106] like Figure 2 As shown in Example 1, this example provides a line holographic anomaly detection system based on cross-modal intelligent collaboration, including a multimodal perception fusion layer, an intelligent analysis layer, an edge execution layer, and an optimization control layer.

[0107] The multimodal perception fusion layer consists of a thermal infrared imaging sensor, a lidar sensor, an acoustic signature sensor, and a multimodal fusion model. The thermal infrared imaging sensor collects thermal infrared imaging data of the line equipment to identify local hot spots. Based on the thermal infrared imaging data, a temperature field distribution model is established to obtain temperature field parameters. The lidar sensor collects geometric shapes to obtain equipment deformation vectors, such as modeling the direction of the pole axis in three-dimensional space, thereby calculating the pole tilt angle. The acoustic signature sensor captures abnormal noises from the transformer to obtain acoustic signature data.

[0108] A multimodal fusion model is used to receive temperature field parameters, pole tilt angles, and acoustic signature features, construct cross-modal correlation tensors, and transmit them to the joint inference engine for abnormal states.

[0109] The intelligent analysis layer includes an abnormal state joint reasoning engine and a dynamic knowledge-driven decision-making framework.

[0110] The joint inference engine for abnormal states includes a spatiotemporal correlation graph neural network and a physical constraint reinforcement learning module. It aligns the dynamic electrical topology graph with the cross-modal correlation tensor, constructs an initial node feature matrix and inputs it into the spatiotemporal correlation graph neural network. Based on the spatiotemporal correlation graph neural network, it analyzes the pole topology relationship, combines physical constraint reinforcement learning to dynamically optimize the detection strategy, and outputs anomaly confidence parameters containing various defects. The anomaly confidence parameters can be regarded as risk scores.

[0111] The dynamic knowledge-driven decision-making framework converts anomaly confidence parameters into likelihood and prior probabilities. Through Bayesian anomaly confidence calculation and online transfer learning, it enables rapid generalization inference of defect features across scenarios and adaptive adjustment of detection thresholds. Dynamic knowledge-driven decision-making adjusts sensitivity in real time based on historical failure probabilities. Online transfer learning automatically aligns with data distribution differences when the system is deployed to a new environment, completing adaptation within 7 days and significantly reducing cross-scenario reconstruction costs.

[0112] The edge execution layer includes a layered edge computing and resource scheduling module and an inspection robot.

[0113] A hierarchical edge computing and resource scheduling module deploys a lightweight inference engine on the inspection robot, enabling millisecond-level local responses to high-risk defects. It dynamically allocates computing power using a sliding window mean-shift algorithm: elastically scheduling CPU / GPU resources based on task urgency. Upon receiving decision commands, it drives the inspection robot to execute actions.

[0114] The inspection robot precisely executes inspection tasks: based on instructions from the layered edge computing and resource scheduling module, it navigates to the defect point, performs emergency treatments (such as spraying and cooling at partial discharge points), and continuously tracks status changes. It then transmits execution effect data (such as the temperature curve after cooling) back to the optimization control layer in real time.

[0115] The optimization control layer includes a multi-objective collaborative optimization mechanism and a cloud-based knowledge base.

[0116] A multi-objective collaborative optimization mechanism is employed to globally balance detection accuracy, false alarm rate, and energy consumption cost: high-risk defect detection rate is weighted statistically, and environmental interference false alarms are automatically attenuated through physical constraint reinforcement learning. After analyzing feedback data from the inspection robot, the detection strategy is dynamically adjusted and the cloud-based knowledge base is updated.

[0117] The cloud-based knowledge base stores historical fault case studies, equipment topology maps, and environmental adaptation parameters. It provides prior knowledge of physical laws to the dynamic knowledge-driven decision-making framework, receives updated model parameters from a multi-objective collaborative optimization mechanism, and forms a closed-loop evolutionary system.

[0118] This invention utilizes a multimodal perception fusion model to deeply integrate multi-source data such as infrared thermal imaging, laser point clouds, and acoustic signature features, overcoming the limitations of traditional single-modal detection and significantly improving the detection capability of concealed defects. Based on a sliding window mean-shifting algorithm and adaptive adjustment of the detection threshold, the system can perceive changes in environmental parameters in real time and autonomously adjust its detection strategy, effectively addressing the impact of seasonal changes, diurnal temperature variations, and extreme weather on detection accuracy. Employing a hierarchical edge computing architecture, a lightweight inference engine is deployed on the inspection robot, enabling localized real-time identification and decision-making for key defects, significantly reducing the latency from detection to response. Combined with transfer learning, the system can transform historical fault characteristics, equipment topology relationships, and physical laws into reusable knowledge units, significantly reducing model reconstruction costs. Through a physical constraint reinforcement learning strategy and a multi-objective reward function design, the algorithm simultaneously optimizes three core indicators during training: detection accuracy, false alarm suppression, and energy efficiency, achieving an organic balance between operational efficiency, economy, and safety.

[0119] In summary, this invention, through innovative technology integration and systematic optimization design, achieves holographic anomaly detection of 10kV distribution network lines, solving problems such as isolated multimodal sensing information, lack of environmental adaptation mechanism, contradiction between response delay and computing power, insufficient cross-scenario generalization ability, and imbalance of multi-objective optimization in traditional inspection technology, thus providing important technical support for the construction of smart grids.

[0120] Another embodiment of the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the computer program, when run by the processor, executes the steps of the above-described method for detecting line holographic anomalies based on cross-modal intelligent collaboration.

[0121] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting line holographic anomalies based on cross-modal intelligent collaboration, characterized by the following steps: as follows: Step 1. Multimodal data acquisition: Simultaneously acquire thermal infrared imaging data, lidar point cloud data, and acoustic signature data. Establish a temperature field distribution model based on the thermal infrared imaging data to obtain temperature field parameters. Fit the geometry of the pole using lidar point cloud data and model the direction of the pole axis in three-dimensional space to calculate the pole tilt angle. Wavelet packet decomposition is performed on the voiceprint data to obtain voiceprint features; Step 2. Cross-modal fusion: Through joint calculation of temperature field parameters, pole deformation data, and acoustic signature features, a cross-modal correlation tensor is obtained; the cross-modal correlation tensor is expressed by the following formula: ； in, For the cross-modal correlation tensor, i is the index of the thermal infrared imaging data; j is the index of the lidar point cloud data; and k is the index of the acoustic text data. This indicates the value of the i-th thermal infrared imaging data. Thermal infrared imaging features obtained by performing Gaussian kernel transformation for thermal imaging; For point cloud spatial encoding functions, This represents the point cloud data of the j-th lidar sensor. Spatial information of lidar point cloud obtained through processing and encoding; For the k-th voiceprint data, This indicates the use of acoustic wavelet packet decomposition operators to decompose pairs of... Voiceprint features obtained by wavelet packet decomposition; Step 3. Anomaly Joint Inference: Align the dynamic electrical topology graph with the cross-modal correlation tensor, construct an initial node feature matrix and input it into the spatiotemporal correlation graph neural network. Analyze the pole topology relationships based on the spatiotemporal correlation graph neural network, dynamically optimize the detection strategy by combining physical constraint reinforcement learning, and output anomaly confidence parameters including various defects; the reward function constructed by the physical constraint reinforcement learning is: ； in, The cumulative reward value is denoted by t, where t is the time and T is the total training time. To improve detection accuracy Weighting coefficients; For false alarm rate The penalty coefficient; For energy costs The penalty coefficient; Physical constraint reinforcement learning optimizes the reward function and dynamically adjusts the weights of the spatiotemporal correlation graph neural network, providing real-time optimized values ​​for the likelihood probability correction term for Bayesian computation, and incorporating spatiotemporal correlation constraints of multimodal features; Step 4. Dynamic knowledge-driven decision-making: Convert the anomaly confidence parameter into likelihood probability and prior probability, and realize rapid generalization reasoning of cross-scene defect features and adaptive adjustment of detection threshold through Bayesian anomaly confidence calculation and online transfer learning; Step 5. Layered edge computing and scheduling: Based on the adaptively adjusted detection threshold and defects in the range above the anomaly confidence threshold, a lightweight inference engine is deployed on the inspection robot. The localized real-time identification and decision-making of key defects are realized through the lightweight edge inference engine and the adaptive computing power allocation mechanism. Step 6. Multi-objective collaborative optimization: Multi-objective collaborative optimization is performed using the overall detection rate and false alarm attenuation ratio as evaluation indicators, and the optimization results are fed back to the physical constraint reinforcement learning stage in Step 3.

2. The method for detecting line holographic anomalies based on cross-modal intelligent collaboration according to claim 1, characterized in that, it also... include: The coordinates of power poles and the connection relationships of lines are extracted from the GIS database of the power grid company to construct a spatial topology network. The status data of circuit breakers and disconnectors are integrated to generate a dynamic electrical topology map. The dynamic electrical topology map is aligned with the cross-modal correlation tensor to construct an initial node feature matrix.

3. The method for detecting line holographic anomalies based on cross-modal intelligent collaboration according to claim 2, characterized in that, The spatiotemporal correlation graph neural network iteratively calculates the initial node feature matrix input, and finally outputs the anomaly confidence parameters of each node.

4. The method for detecting line holographic anomalies based on cross-modal intelligent collaboration according to claim 1, characterized in that, In step 4, the posterior probability of various defects is calculated using Bayesian anomaly confidence.

5. The method for detecting line holographic anomalies based on cross-modal intelligent collaboration according to claim 1, characterized in that, The adaptive computing power allocation mechanism adopts a sliding window, defined as follows: ； ； in, Forgetting factor; Let be the dynamic mean at time t; The dynamic mean at time t-1; Let be the dynamic variance at time t. Let be the dynamic variance at time t-1; Let be the observed value at time t.

6. A system for implementing the line holographic anomaly detection method based on cross-modal intelligent collaboration as described in any one of claims 1-5, characterized in that, It includes a multimodal perception fusion layer, an intelligent analysis layer, an edge execution layer, and an optimization control layer; The multimodal perception fusion layer consists of a thermal infrared imaging infrared sensor, a lidar sensor, an acoustic signature acquisition sensor, and a multimodal fusion model. A multimodal fusion model is used to receive temperature field parameters, pole tilt angle and acoustic signature features, construct cross-modal correlation tensors, and transmit them to the joint inference engine for abnormal states; The intelligent analysis layer includes an abnormal state joint reasoning engine and a dynamic knowledge-driven decision framework; The joint reasoning engine for abnormal states includes a spatiotemporal correlation graph neural network and a physical constraint reinforcement learning module. It aligns the dynamic electrical topology graph with the cross-modal correlation tensor, constructs an initial node feature matrix and inputs it into the spatiotemporal correlation graph neural network, analyzes the pole topology relationship based on the spatiotemporal correlation graph neural network, dynamically optimizes the detection strategy by combining physical constraint reinforcement learning, and outputs anomaly confidence parameters that include various defects. The dynamic knowledge-driven decision framework converts the anomaly confidence parameter into likelihood probability and prior probability, and achieves rapid generalization reasoning of cross-scenario defect features and adaptive adjustment of detection threshold through Bayesian anomaly confidence calculation and online transfer learning. The edge execution layer includes a layered edge computing and resource scheduling module and an inspection robot; The hierarchical edge computing and resource scheduling module deploys a lightweight inference engine on the inspection robot to provide millisecond-level local responses to high-risk defects; it dynamically allocates computing power through a sliding window mean drift algorithm: it elastically schedules CPU / GPU resources according to the urgency of the task; and it drives the inspection robot to perform actions after receiving decision instructions. Inspection robots accurately execute inspection tasks: based on hierarchical edge computing and resource scheduling modules, they navigate to defect points, perform emergency handling, continuously track status changes, and transmit execution effect data back to the optimization control layer in real time; The optimization control layer includes a multi-objective collaborative optimization mechanism and a cloud-based knowledge base.

7. The system according to claim 6, characterized in that, The multi-objective collaborative optimization mechanism globally balances detection accuracy, false alarm rate, and energy consumption cost. After analyzing the feedback data from the inspection robot, it dynamically adjusts the detection strategy and updates the cloud knowledge base. The cloud knowledge base stores historical fault case databases, equipment topology maps, and environmental adaptation parameters. It provides prior knowledge of physical laws to the dynamic knowledge-driven decision-making framework, receives model parameters updated by the multi-objective collaborative optimization mechanism, and forms a closed-loop evolutionary system.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is run by the processor, it executes the steps of the line holographic anomaly detection method based on cross-modal intelligent collaboration as described in any one of claims 1-5.

Images