Substation inspection multi-modal data cross-station correlation analysis and defect tracing method

Abstract

The application discloses a substation inspection multi-modal data cross-station correlation analysis and defect tracing method, belonging to the technical field of intelligent inspection of power systems, including collecting geographic coordinates and time stamps and generating a space-time reference track, synchronously collecting micro-meteorological data, voiceprint data and visible light images, and encoding the multi-modal data volume according to the space-time reference track; taking the substation building layout as a reference to establish a local coordinate system in the substation, mapping the multi-modal data volume to the local coordinate system in the substation for coordinate conversion and sorting integration to obtain a registered multi-modal observation sequence and the like. The application solves the technical problems of logical discontinuity between defect symptom identification and defect tracing analysis and the lack of cross-station comparison of a historical defect database in the prior art, realizes a complete tracing closed loop from suspected defect symptom judgment to defect propagation path tracing, and makes defect online detection results based on cross-station defect correlation mapping and defect tracing conclusions.

Description

Technical Field

[0001] This application belongs to the field of intelligent inspection technology of power systems, and specifically relates to a method for cross-station correlation analysis and defect tracing of multimodal data in substation inspection. Background Technology

[0002] Substation inspection is a crucial technical means to ensure the safe operation of power systems. By monitoring and evaluating the operating status of equipment, it can promptly identify defects and trace their causes, playing a vital role in the stable operation of the power grid. With the widespread application of intelligent inspection equipment, inspection technology based on multimodal data fusion has become a core means of improving efficiency.

[0003] During substation inspections, spatiotemporal alignment and deep correlation analysis of multi-source heterogeneous data are fundamental to identifying equipment defects and tracing their root causes. The inspection system needs to achieve data fusion and state evolution analysis within a unified spatiotemporal framework to support complete source tracing from defect symptoms to root cause inference. This places high demands on the system's cross-site collaborative capabilities, multimodal fusion depth, and root cause reasoning capabilities.

[0004] However, existing technologies have the following shortcomings: The spatiotemporal references for cross-station data are fragmented; different substations use independent coordinate systems and time bases, making it difficult to achieve cross-station data alignment and correlation comparison; multimodal data fusion remains at the superficial level, with multi-source data only superficially stitched together, failing to explicitly model the physical causal relationships between the data; there is a logical disconnect between defect identification and root cause inference; existing systems can identify defect manifestations but cannot trace the root causes, let alone use cross-station historical data to verify root cause inferences, resulting in a lack of targeted operational decisions. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method for cross-site correlation analysis and defect tracing of multimodal data in substation inspections. This method resolves the technical problems in existing technologies, such as the logical disconnect between defect symptom identification and defect tracing analysis, and the lack of cross-site comparison with historical defect databases. It achieves a complete closed loop for tracing defects from suspected defect symptom determination to defect propagation path tracing, enabling online defect detection results to be based on cross-site defect correlation mapping and defect tracing conclusions.

[0006] This application provides a method for cross-site correlation analysis and defect tracing of multimodal data from substation inspections, including the following steps: Geographic coordinates and timestamps are collected and spatiotemporal reference trajectories are generated. Micro-meteorological data, acoustic data and visible light images are collected simultaneously and encoded into multimodal data volumes based on the spatiotemporal reference trajectories. A local coordinate system within the substation is established with reference to the substation building layout. The multimodal data volume is mapped to the local coordinate system for coordinate transformation and sorting integration to obtain the registered multimodal observation sequence. Cross-temporal correlation analysis was performed on the registered multimodal observation sequences to extract temporal evolution features and generate equipment state evolution maps; Defect signs are identified based on equipment state evolution maps, suspected defect signs are determined and defect coordinates are extracted; Based on the defect coordinates, an air-to-ground stereoscopic observation pair is triggered to collect local data and obtain detailed defect observation data; Cross-station correlation comparison was performed on the detailed defect observation data, and a cross-station defect correlation mapping was established to obtain the cross-station defect correlation results; Based on the cross-site defect correlation results, defect source tracing analysis is performed to trace the defect propagation path and generate defect source tracing conclusions. Based on the defect source tracing conclusions, online defect detection results are output.

[0007] In one embodiment, geographic coordinates and timestamps are collected to generate a spatiotemporal reference trajectory. Simultaneously, micro-meteorological data, acoustic signature data, and visible light images are collected and encoded into a multimodal data volume based on the spatiotemporal reference trajectory, including: Obtain the real-time geographic coordinates of the inspection vehicle and simultaneously record the corresponding timestamp information to obtain the original spatiotemporal data pair; The original spatiotemporal data pairs are time-aligned and coordinate-smoothed to eliminate positioning noise and obtain smoothed spatiotemporal data pairs. Smoothed spatiotemporal data pairs are sorted and stitched together in chronological order to generate a spatiotemporal baseline trajectory; Micro-meteorological data is obtained by collecting ambient temperature, humidity and air pressure data through micro-meteorological sensors; acoustic data is obtained by collecting acoustic fingerprint signals of the device operation through acoustic sensors; and visible light images are obtained by collecting surface images of the device through a visible light camera. Micrometeorological data, acoustic data, and visible light images are aligned with timestamps and spatiotemporal reference trajectories to obtain aligned multimodal data; The aligned multimodal data is encapsulated in a unified format and encoded into a multimodal data body.

[0008] In one embodiment, a local coordinate system is established within the substation, using the substation building layout as a reference. The multimodal data volume is mapped to the local coordinate system for coordinate transformation and sorting integration to obtain the registered multimodal observation sequence, including: Using the substation building layout drawings as a reference, a local coordinate system within the station is established, and the origin and direction of the coordinate axes are determined. Extract the geographic coordinates from the spatiotemporal reference trajectory, calculate the relative coordinates of the geographic coordinates in the local coordinate system within the station, and obtain the local coordinate mapping relationship; Based on the local coordinate mapping relationship, the micro-meteorological data, acoustic data and visible light images in the multimodal data volume are transformed to the local coordinate system within the station to obtain the multimodal data after coordinate transformation; The multimodal data after coordinate transformation are sorted and integrated according to timestamp and spatial location to obtain the registered multimodal observation sequence.

[0009] In one embodiment, cross-temporal correlation analysis is performed on the registered multimodal observation sequences to extract temporal evolution features and generate a device state evolution map, including: The registered multimodal observation sequences were grouped according to the equipment number to obtain the single-equipment observation sequences for each equipment. The time window is divided into several continuous time segments by dividing the single-device observation sequence of each device; Within each consecutive time segment, the temporal change rate of micro-meteorological data, acoustic data, and visible light images is calculated to obtain the temporal change rate of each mode. By fusing and correlating the temporal change rates of each mode, cross-modal temporal evolution features are extracted to obtain temporal evolution features; Device state nodes are constructed based on temporal evolution characteristics, and each device state node is connected in chronological order to generate a device state evolution map.

[0010] In one embodiment, defect symptom identification is performed based on the device state evolution map, and suspected defect symptom is determined and defect coordinates are extracted, including: The feature differences between the current device state node and the historical normal state node are extracted from the device state evolution graph to obtain the state deviation features; Input the state deviation features into the preset defect symptom recognition model, calculate the defect probability value, and obtain the defect probability distribution; Filter out device status nodes that exceed a preset threshold in the defect probability distribution to identify suspected defect signs; Extract the spatial coordinates of the equipment status nodes identified as suspected defect signs in the local coordinate system within the station to obtain the defect coordinates.

[0011] In one embodiment, a three-dimensional observation pair is triggered based on the defect coordinates to collect local data and obtain detailed defect observation data, including: The defect coordinates are sent to the air-to-ground stereo observation pair to obtain the observation task instructions; the air-to-ground stereo observation pair is composed of a drone and a robot dog working together. Control the drone to fly to the position above the defect coordinates, collect the top view image and infrared thermal image data of the defect area, and obtain aerial observation data; The robot dog is controlled to move to the near-ground position corresponding to the defect coordinates, and detailed images and local acoustic data of the defect surface are collected to obtain ground observation data; By aligning and modally fusing aerial and ground-based observation data in time and space, detailed defect observation data can be obtained.

[0012] In one embodiment, cross-site correlation comparison is performed on the detailed defect observation data, and a cross-site defect correlation mapping is established to obtain the cross-site defect correlation results, including: Defect type features and defect severity features are extracted from the detailed defect observation data to obtain the defect feature vector; The similarity between the defect feature vector and the historical defect features in the historical defect database is calculated to obtain the defect similarity matrix; Based on the defect similarity matrix, relevant historical defect records are filtered to establish a cross-site defect association mapping between the current defect and historical defects; Based on the cross-site defect association mapping, a defect propagation association chain is generated to obtain the cross-site defect association results.

[0013] In one embodiment, defect source tracing analysis is performed based on cross-site defect correlation results to trace the defect propagation path and generate defect source tracing conclusions, including: The topological structure of the defect propagation association chain is extracted from the cross-site defect association results to obtain the defect association topology; Root node analysis is performed on the defect association topology to determine the defect starting node and obtain the defect starting position. Tracing the direction and timing of defect propagation in reverse along the defect correlation topology, and reconstructing the defect propagation path; Based on the defect's starting location and propagation path, a defect tracing conclusion is generated, which includes the defect's cause and scope of impact.

[0014] In one embodiment, the online defect detection result is output based on the defect tracing conclusion, including: From the defect tracing conclusions, the defect type, defect location, defect cause, and defect impact range are extracted to obtain key defect information; The key information about defects is arranged in a structured manner according to a preset report template to obtain a structured defect report; The structured defect report is pushed to the operation and maintenance management platform, and the online defect detection results are output.

[0015] In one embodiment, the preset defect symptom recognition model is trained in the following manner: Multimodal observation sequences corresponding to historical normal and historical defective states are collected as training samples. The temporal evolution features of each training sample are extracted and labeled with state labels to obtain a labeled training set. The labeled training set is input into the initial recognition model for iterative training. The model parameters are optimized until convergence, and the preset defect symptom recognition model is obtained.

[0016] Compared with the prior art, this application has the following advantages: 1. This invention obtains detailed defect observation data by triggering three-dimensional observation of air and ground based on defect coordinates to collect high-resolution local data. It then performs cross-station correlation comparison on the detailed defect observation data and establishes a cross-station defect correlation mapping to obtain cross-station defect correlation results. Based on the cross-station defect correlation results, it performs defect source tracing analysis to trace the defect propagation path and generates defect source tracing conclusions. This solves the technical problems in the prior art where there is a logical break between defect symptom identification and defect source tracing analysis and a lack of cross-station comparison with historical defect databases. It realizes a complete source tracing closed loop from the determination of suspected defect symptoms to the tracing of defect propagation paths, and enables online defect detection results to be based on cross-station defect correlation mapping and defect source tracing conclusions. 2. This invention extracts temporal evolution features from the registered multimodal observation sequences through cross-temporal correlation analysis and generates an equipment state evolution map. Based on the equipment state evolution map, it identifies and determines suspected defect signs and extracts defect coordinates. This solves the technical problem in the prior art that multimodal data fusion does not fully utilize cross-temporal correlation analysis and temporal evolution features for equipment state monitoring. It realizes the explicit representation of the temporal evolution law of equipment state by the equipment state evolution map, and makes defect sign identification based on temporal evolution features and equipment state evolution map. 3. This invention collects geographic coordinates and timestamps to generate a spatiotemporal reference trajectory, simultaneously collects micro-meteorological data, acoustic data, and visible light images, and encodes them into multimodal data volumes based on the spatiotemporal reference trajectory. Using the substation building layout as a reference, a local coordinate system within the station is established, and the multimodal data volumes are mapped to the local coordinate system within the station to obtain a registered multimodal observation sequence. This solves the technical problem in the prior art where the use of independent coordinate systems and time references for inspection data from different substations leads to the inability to directly correlate and compare cross-station data. It realizes the coordinate transformation and sorting integration of multimodal data volumes under a unified local coordinate system within the station, enabling the registered multimodal observation sequence to have a basis for cross-station spatiotemporal alignment.

[0017] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1A schematic diagram of the process provided by the present invention is shown; Figure 2 This invention illustrates the flowchart of the construction of the mobile spatiotemporal base for unmanned vehicles and the spatiotemporal alignment of air-ground multimodal data. Figure 3 This invention provides a flowchart of the complementary evidence collection and multimodal physical causal graph root cause reasoning process for air-to-ground stereoscopic observation. Figure 4 This invention illustrates the closed-loop logic diagram of cross-site evolutionary knowledge base matching verification and unmanned vehicle mobile verification provided by the present invention. Figure 5 A schematic diagram of the multimodal physical causality graph structure provided by the present invention is shown. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Existing technologies for substation operation and maintenance inspection mainly encompass three levels. The first is autonomous in-station inspection based on air-to-ground intelligent equipment such as drones and robotic sensors, using multiple sensors including visible light, infrared thermal imaging, and lidar to collect equipment status data. The second is equipment defect identification based on edge computing or cloud-based multimodal large-scale models, utilizing visual large-scale models to fuse and analyze data such as images, thermal images, and acoustic signatures, outputting defect categories and confidence levels. The third is in-station 3D visualization based on digital twins or BIM models, mapping inspection data onto a 3D scene for spatial positioning. Furthermore, some advanced systems have introduced unmanned vehicles as cross-site transportation carriers for robotic sensors, enabling equipment scheduling across multiple sites.

[0022] However, existing integrated air-ground substation inspection technologies suffer from the following fundamental flaws. First, the spatiotemporal reference for cross-station data is fragmented. Robotic robots and drones at different substations are deployed and operate independently, with each station using its local coordinate system and timestamp, resulting in a lack of a unified spatiotemporal reference for cross-station correlation analysis. Second, the unmanned vehicles have limited functionality and their data value is not fully explored. The unmanned vehicles only perform the single function of transporting robotic robots across stations, completely discarding the environmental parameters and high-precision spatiotemporal reference information continuously collected along the route. Third, multimodal data fusion remains at the superficial level. Only shallow stitching of multimodal data from robotic robots and drones at the feature or decision levels is performed, without explicitly modeling the physical causal relationships between the multimodal data. Fourth, there is a logical disconnect between defect identification and root cause inference. Existing systems can identify defects such as thermal anomalies, corrosion, and deformation, but cannot distinguish whether the thermal anomaly is caused by contamination of the insulator itself, internal winding faults, or sudden changes in regional temperature and humidity.

[0023] To address this, this application provides a method for cross-station correlation analysis and defect tracing of multimodal data in substation inspections. By constructing a global spatiotemporal reference trajectory and inter-station environmental gradient curves using unmanned vehicles, it achieves spatiotemporal alignment and physical causal reasoning of cross-station multimodal data, solving the technical problems of fragmented cross-station data, lack of value mining of unmanned vehicle data, and inability to trace the root causes of defects in existing technologies.

[0024] The following is a detailed description of the cross-station correlation analysis and defect tracing method for multimodal data of substation inspection provided in this application.

[0025] like Figures 1 to 4 As shown, the method includes the following steps S1 to S7.

[0026] S1. Collect geographic coordinates and timestamps and generate spatiotemporal reference trajectory. Simultaneously collect micro-meteorological data, voiceprint data and visible light images, and encode them into multimodal data volumes based on the spatiotemporal reference trajectory.

[0027] In one embodiment, step S1 includes the following sub-steps S11 to S16.

[0028] S11. Obtain the real-time geographic coordinates of the inspection vehicle and simultaneously record the corresponding timestamp information to obtain the original spatiotemporal data pair.

[0029] When the unmanned vehicle performs cross-station commuting tasks, it is equipped with a combined positioning module consisting of a GNSS receiver and an INS inertial measurement unit, achieving positioning accuracy better than centimeter level. It also carries a high-precision time synchronization module, outputting a unified time reference via BeiDou time synchronization. During cross-station travel, it collects global geographic coordinates and absolute timestamps at a frequency of 1Hz to generate a continuous global spatiotemporal reference trajectory. This 1Hz frequency is determined based on the common sampling standard for GNSS dynamic positioning. This frequency ensures trajectory continuity while controlling data storage, avoiding redundant data accumulation caused by excessively high frequencies.

[0030] S12. Perform time alignment and coordinate smoothing on the original spatiotemporal data pairs to eliminate positioning noise and obtain smooth spatiotemporal data pairs.

[0031] Kalman filtering was used to smooth the coordinates of the original spatiotemporal data pairs. The filtered state vector included longitude, latitude, elevation, and velocity components. The observation noise covariance was set according to the nominal accuracy of the GNSS receiver, and the process noise covariance was set according to the kinematic model of the unmanned vehicle.

[0032] S13. Sort and stitch the smoothed spatiotemporal data in chronological order to generate a spatiotemporal reference trajectory.

[0033] The spatiotemporal reference trajectory is denoted as a sequence. ,in For the first The three-dimensional geographic coordinate vector of each sampling point To correspond to the absolute timestamp, This represents the total number of trajectory sampling points.

[0034] S14. Collect ambient temperature, humidity and air pressure data through a micro-meteorological sensor to obtain micro-meteorological data; collect the acoustic fingerprint signal of the device operation through an acoustic sensor to obtain acoustic fingerprint data; collect the surface image of the device through a visible light camera to obtain a visible light image.

[0035] S15. Align the micro-meteorological data, acoustic data, and visible light images with the timestamp and spatiotemporal reference trajectory to obtain aligned multimodal data.

[0036] S16. Encapsulate the aligned multimodal data in a unified format and encode it into a multimodal data body.

[0037] Multimodal data volume is denoted as ,in For the first The modal feature vectors at each sampling time point are concatenated from micro-meteorological data, acoustic data, and visible light images after feature extraction. For the corresponding geographic coordinates, For the corresponding timestamp, This is a modal type identifier. This represents the total number of multimodal sampling points.

[0038] The unmanned vehicle is equipped with a mobile environmental monitoring module, which includes at least temperature and humidity sensors, a laser dust concentration sensor, a salt spray deposition sampling plate, and an electromagnetic field intensity probe. During inter-station travel, the environmental monitoring module synchronously and continuously collects micro-environmental parameters along the route at a frequency of 0.1Hz, including at least ambient temperature, ambient humidity, dust concentration, salt spray deposition rate, and electromagnetic field intensity. The 0.1Hz frequency is chosen based on the slow-changing characteristics of environmental parameters. The change period of ambient temperature, humidity, and dust concentration on the inter-station commuting scale is typically on the order of minutes or even hours. 0.1Hz, or sampling once every 10 seconds, is sufficient to capture environmental gradients and significantly reduce sensor power consumption and data volume.

[0039] Based on the spatial coordinates in the global spatiotemporal reference trajectory, the aforementioned microenvironmental parameters are mapped to inter-station environmental gradient curves distributed along the driving path. For any point on the trajectory to be interpolated... Its environmental parameter values The calculation is performed using inverse distance weighted interpolation, with the coordinates of the point to be interpolated as the input. and Coordinates of environmental monitoring sampling points and corresponding parameter values The output is the interpolated environmental parameter value. The calculation formula is: ; in, This represents the total number of environmental monitoring sampling points. For the first Microenvironmental parameter values ​​at each sampling point For the first The weights of the sampling points to be interpolated are calculated according to the following formula: ; in, Points to be interpolated With the Environmental monitoring sampling points The Euclidean distance between them This is the distance power exponent, with a value of 2. The power exponent is determined based on the first law of geography, which states that spatial correlation decreases with increasing distance. A power exponent of 2 is a commonly used value in the field of spatial interpolation, achieving a balance between local smoothing and capturing global trends.

[0040] After the unmanned vehicle arrives at the target substation and parks in the designated docking area, it broadcasts differential positioning and time synchronization signals to the substation via its onboard broadcast unit. The signal coverage radius is no less than 500m, and the update frequency is no less than 10Hz. The 500m coverage radius is based on the typical substation area; generally, the radius of the main equipment area within a 220kV or lower substation does not exceed 500m. This coverage range ensures that the robot dog and drone receive differential signals at any operating point within the substation. The 10Hz update frequency is based on the dynamic operating requirements of the robot dog and drone. The robot dog's walking speed is approximately 1m / s to 2m / s, and the drone's flight speed is approximately 5m / s to 10m / s. The 10Hz differential signal update frequency can control the dynamic positioning delay to within 100ms, corresponding to a displacement error of 10cm to 20cm for the robot dog and 50cm to 100cm for the drone, meeting the real-time requirements of centimeter-level positioning.

[0041] S2. Establish a local coordinate system within the substation with reference to the substation building layout, map the multimodal data volume to the local coordinate system within the substation, perform coordinate transformation and sorting integration, and obtain the registered multimodal observation sequence.

[0042] In one embodiment, step S2 includes the following sub-steps S21 to S24.

[0043] S21. Using the substation building layout drawings as a reference, establish a local coordinate system within the station and determine the origin and direction of the coordinate axes.

[0044] The local coordinate system within the station is denoted as The origin of the coordinate system is set at the corner of the main control building of the substation. The X-axis is arranged longitudinally along the main equipment area, the Y-axis is arranged laterally, and the Z-axis is vertically upward.

[0045] S22. Extract the geographic coordinates from the spatiotemporal reference trajectory, calculate the relative coordinates of the geographic coordinates in the local coordinate system within the station, and obtain the local coordinate mapping relationship.

[0046] The local coordinate mapping relationship is established using a seven-parameter Bursa model. The input is a three-dimensional coordinate vector in the geographic coordinate system, and the output is a three-dimensional coordinate vector in the local coordinate system within the station. The conversion formula is: ; in, A three-dimensional coordinate vector in a geographic coordinate system. This is a three-dimensional coordinate vector in the local coordinate system within the station. For rotation matrix, As a scale factor, This is the translation vector. This seven-parameter model is a standard method for coordinate transformation in the surveying field, suitable for establishing local coordinate systems with centimeter-level accuracy.

[0047] S23. Based on the local coordinate mapping relationship, the micro-meteorological data, acoustic data and visible light images in the multimodal data volume are transformed to the local coordinate system within the station to obtain the multimodal data after coordinate transformation.

[0048] S24. Sort and integrate the multimodal data after coordinate transformation according to timestamp and spatial location to obtain the registered multimodal observation sequence.

[0049] The registered multimodal observation sequence is denoted as ,in , For the first Modal feature vectors of each observation point For the first The three-dimensional coordinates of each observation point in the local coordinate system within the station. For the corresponding timestamp, This represents the total number of observation points.

[0050] S3. Perform cross-temporal correlation analysis on the registered multimodal observation sequences, extract temporal evolution features, and generate equipment state evolution maps.

[0051] In one embodiment, step S3 includes the following sub-steps S31 to S35.

[0052] S31. Group the registered multimodal observation sequences according to the equipment number to obtain the single-equipment observation sequences of each equipment.

[0053] S32. Divide the single-device observation sequence of each device into time windows to obtain several continuous time segments.

[0054] The time window length is denoted as The sliding step size is denoted as Then the first a time sequence segment From index Start continuous It consists of several observation points. The value is based on the time scale of the equipment state evolution, with a value of 24 hours, that is, a time segment of each day. This value can capture the daily operating cycle characteristics of the equipment. The value is set to 12 hours, meaning that adjacent segments overlap by 50%. This value ensures temporal continuity and increases the number of training samples.

[0055] S33. Within each consecutive time segment, calculate the temporal change rate of micro-meteorological data, acoustic data, and visible light images respectively to obtain the temporal change rate of each mode.

[0056] For any modal feature sequence Its time-series rate of change Based on differential calculation, the input is the modal characteristic values ​​at adjacent sampling times, and the output is the temporal rate of change. The calculation formula is: ; in, For the first Modal feature values ​​at each sampling time, The modal feature value at the previous sampling time. This represents the time interval between adjacent sampling times.

[0057] S34. The temporal change rates of each mode are fused and correlated to extract cross-modal temporal evolution features, thus obtaining temporal evolution features.

[0058] Cross-modal temporal evolution characteristics are calculated using a multimodal fusion matrix. Let the micrometeorological temporal change rate vector be... The voiceprint temporal rate of change vector is The visible light temporal rate of change vector is The fused temporal evolution feature vector for: ; in, This represents a vector concatenation operation. The input dimension is a trainable multimodal fusion weight matrix. , and The total dimension after concatenating the three elements is the output dimension. , The fused feature dimension is set to 128. This dimension is chosen based on the common intermediate layer dimension of deep learning feature encoding, which can achieve a balance between representational ability and computational efficiency.

[0059] S35. Construct device state nodes based on temporal evolution characteristics, and connect each device state node in chronological order to generate a device state evolution map.

[0060] The equipment state evolution diagram is denoted as ,in A set of device status nodes, each node Carrying temporal evolution feature vector and corresponding timestamp ; Let be the set of temporal edges, where Represents a node The temporal evolution relationship between the next time step and the next time step.

[0061] S4. Based on the equipment state evolution map, identify defect signs, determine suspected defect signs, and extract defect coordinates.

[0062] In one embodiment, step S4 includes the following sub-steps S41 to S44.

[0063] S41. Extract the feature differences between the current device state node and the historical normal state node from the device state evolution map to obtain the state deviation features.

[0064] State deviation characteristics Based on the characteristics of the current node Compared with the historical normal node characteristic mean The difference calculation takes the current node's features and the set of historical normal node features as input, and outputs a state deviation feature vector. The calculation formula is: ; in, , For the first The temporal evolution feature vector of each historical normal state node This represents the total number of nodes in the historical normal state.

[0065] S42. Input the state deviation features into the preset defect symptom recognition model, calculate the defect probability value, and obtain the defect probability distribution.

[0066] The defect symptom identification model employs a temporal classification network based on an attention mechanism. Its structure includes the following levels and connections: Input layer: Receives the feature vector of the state deviation , dimension This layer directly passes the input vector to the LSTM coding layer; The LSTM encoding layer consists of two bidirectional LSTM sublayers, each composed of a forward LSTM unit and a backward LSTM unit connected in parallel. The first bidirectional LSTM sublayer has a hidden layer dimension of 128 and is used to extract primary temporal dependency features; its output is passed to the second bidirectional LSTM sublayer. The second bidirectional LSTM sublayer has a hidden layer dimension of 64 and is used to extract deep abstract features; its output is passed to the attention layer. Each LSTM unit contains an input gate, a forget gate, an output gate, and candidate memory units. Each gating mechanism uses a trainable weight matrix and bias vector to filter information and update the state. Attention layer: Employs a scaled dot product attention mechanism. First, the query vector is calculated. With key vector sequence similarity score ,in The key vector dimension is set to 64. This scaling factor is based on the original design of scaling dot product attention and is used to prevent the dot product value from becoming too large, which could cause the softmax gradient to vanish. The attention weights are then obtained by normalization using the softmax function. Finally, the value vector sequence Weighted aggregation is performed to obtain the context vector. ; Fully connected layer: Contains two linear transformation sub-layers. The first linear transformation sub-layer has an output dimension of 32, and the second linear transformation sub-layer has an output dimension of 1. A ReLU activation function and a Dropout regularization layer are inserted between the two linear transformation sub-layers, with a Dropout ratio of 0.3. This 0.3 ratio is based on general regularization experience in deep learning; in temporal classification tasks, this ratio can effectively suppress the risk of overfitting while retaining sufficient model expressive power. Output layer: Employs the Sigmoid activation function to output defect probability values. The value ranges from 0 to 1.

[0067] The training steps and parameter settings for the defect symptom recognition model are as follows: Step 1: Collect multimodal observation sequences corresponding to historical normal and historical defective states as training samples, extract the temporal evolution features of each training sample, and label them with state labels to obtain a labeled training set. Normal state samples are labeled with 0, and defective state samples are labeled with 1.

[0068] Step two involves dividing the labeled training set into a training subset and a validation subset in an 8:2 ratio. This ratio is based on a common partitioning standard in the field of machine learning, ensuring sufficient training data while providing reliable validation and evaluation.

[0069] Step 3: Input the training subset into the initial recognition model for iterative training, using the binary cross-entropy loss function. The inputs are the true labels and the model output, and the output is the loss value. The calculation formula is: ; in, The batch size is set to 64. This value is based on the current mainstream GPU memory capacity and the number of model parameters. 64 can achieve a balance between memory utilization and gradient estimation stability. For the first The true label of each sample This represents the defect probability value output by the model.

[0070] Step four: Update the model parameters using the Adam optimizer, with an initial learning rate of 0.001. This learning rate is based on the default recommended value of the Adam optimizer, which is suitable for training small to medium-sized time series data and can achieve a balance between convergence speed and stability.

[0071] Step 5: After each iteration, calculate the accuracy and recall on the validation subset. Stop training when the validation loss does not decrease for 5 consecutive rounds to obtain the preset defect symptom recognition model.

[0072] S43. Filter out device status nodes that exceed the preset threshold in the defect probability distribution and determine suspected defect signs.

[0073] The preset threshold is denoted as ,when It was identified as a suspected defect node. The value is set to 0.7, which is based on the preference for accuracy in defect detection tasks. The threshold of 0.7 can effectively filter out low-confidence false alarms and ensure the high value of subsequent air-to-ground stereo observations for triggering.

[0074] S44. Extract the spatial coordinates of the equipment status nodes identified as suspected defect symptoms in the local coordinate system within the station to obtain the defect coordinates. The defect coordinates are marked as follows: Directly from suspected defective nodes Extracted from spatial attributes.

[0075] S5. Based on the defect coordinates, trigger the air-ground three-dimensional observation pair, collect local data, and obtain detailed defect observation data.

[0076] In one embodiment, step S5 includes the following sub-steps S51 to S55.

[0077] S51. Send the defect coordinates to the air-to-ground stereo observation pair to obtain the observation task instruction. The air-to-ground stereo observation pair is composed of a UAV and a robot dog working together.

[0078] S52. Control the UAV to fly to the position above the defect coordinates, collect the top view image and infrared thermal image data of the defect area, and obtain aerial observation data.

[0079] The hovering altitude of the drone is recorded as The value is determined by a comprehensive consideration of the optimal imaging distance of the infrared thermal imager and the safe clearance height of the substation, and is set at 15m. At this height, the infrared thermal imager can obtain thermal field images with centimeter-level resolution, while also meeting the safe distance requirements for energized equipment in the substation.

[0080] S53. Control the robot dog to move to the near-ground position corresponding to the defect coordinates, collect detailed images and local acoustic data of the defect surface, and obtain ground observation data.

[0081] S54. Simultaneously start the unmanned vehicle's onboard environmental monitoring module to collect baseline data of the micro-environment around the device, and obtain environmental baseline data.

[0082] S55. Spatiotemporal alignment and modal fusion of aerial observation data, ground observation data and environmental baseline data are performed to obtain detailed defect observation data.

[0083] Spatiotemporal alignment maps the three types of data to a unified spatiotemporal framework based on the absolute timestamps and geographic coordinates in the global spatiotemporal reference trajectory. Modality fusion employs feature-level stitching, with fine-grained defect observation data denoted as... ,in For aerial observation data feature vectors, For ground observation data feature vectors, This is the feature vector of the environmental baseline data.

[0084] S6. Perform cross-station correlation comparison on the detailed observation data of defects and establish cross-station defect correlation mapping to obtain cross-station defect correlation results.

[0085] In one embodiment, step S6 includes the following sub-steps S61 to S64.

[0086] S61. Extract defect type features and defect severity features from the detailed defect observation data to obtain the defect feature vector.

[0087] Defect type features are extracted using a pre-trained multi-label classification network, and defect severity features are calculated using an anomaly quantification function. The inputs are the maximum temperature rise in the defect area and the equipment's normal operating baseline temperature; the output is the defect severity value. The calculation formula is as follows: ; in, This represents the severity value of the defect. This represents the maximum temperature rise in the defect area. This is the reference temperature for normal equipment operation. The defect feature vector is denoted as... ,in This is the feature vector for the defect type.

[0088] S62. Calculate the similarity between the defect feature vector and the historical defect features in the historical defect database to obtain the defect similarity matrix.

[0089] Similarity calculation uses Euclidean distance as the metric. The input is two defect feature vectors, and the output is the distance value. The calculation formula is: ; in, The current defect feature vector With the Historical defect feature vectors The Euclidean distance between them For the defect feature vector dimension, The current defect feature vector is the first... One portion, For the first The first historical defect feature vector Each component.

[0090] Defect similarity Based on the exponential decay mapping of Euclidean distance, the similarity is calculated as follows: The input is Euclidean distance, and the output is the similarity value. ; This mapping compresses the distance space to the range of 0 to 1. When the distance is 0, the similarity is 1, and when the distance approaches infinity, the similarity approaches 0, thus avoiding the problem of zero denominator that may occur in linear mapping.

[0091] S63. Based on the defect similarity matrix, filter related historical defect records and establish a cross-site defect association mapping between the current defect and historical defects.

[0092] The screening criteria are: if Then establish the current defect and the first The association of historical defect records, among which The similarity threshold is set to 0.85. This value is based on the high requirements for the reliability of cross-site association analysis. The high threshold of 0.85 ensures that an association is established only when the defect features are highly similar, avoiding low-quality associations from interfering with subsequent root cause inference.

[0093] S64. Generate a defect propagation association chain based on the cross-site defect association mapping to obtain the cross-site defect association result.

[0094] Defect propagation association chain denoted as ,in This is the current defect node. to These are the related historical defect nodes, sorted in descending order of similarity. This represents the number of associated nodes.

[0095] S7. Based on the cross-site defect correlation results, perform defect source tracing analysis, trace the defect propagation path and generate defect source tracing conclusions, and output online defect detection results based on the defect source tracing conclusions.

[0096] In one embodiment, step S7 includes the following sub-steps S71 to S77.

[0097] S71. Extract the topological structure of the defect propagation association chain from the cross-site defect association results to obtain the defect association topology.

[0098] The defect-related topology is denoted as a graph. ,in Includes the current defect node and its associated historical defect nodes. It includes similarity-weighted edges between nodes.

[0099] S72. Perform root node analysis on the defect association topology to determine the defect starting node and obtain the defect starting position.

[0100] Root node analysis employs an iterative importance algorithm based on similarity propagation. Node importance values. The following iterative formula is used to calculate the importance value of each node, with the input being the adjacency relationship and similarity weight of the defect association topology: ; in, For nodes In the Importance value of the next iteration For nodes The set of neighboring nodes, For nodes and Similarity weights between them. Initial values ​​for iteration. Set as unified , This represents the total number of nodes in the defect association topology. When the maximum difference in importance values ​​between two consecutive iterations is less than... The iteration stops when the node with the highest importance value is identified as the defect initiation node. This convergence threshold is determined based on commonly used precision standards in numerical computation, achieving a balance between computational efficiency and result stability.

[0101] S73. Tracing the direction and timing of defect propagation in reverse along the defect association topology to reconstruct the defect propagation path.

[0102] Defect propagation path reconstruction employs the minimum cumulative cost path algorithm. Starting with the defect's originating node as the source and the current defect node as the destination, the algorithm searches for the minimum cumulative cost path within the defect association topology. The cumulative cost of the path is then calculated. The calculation is as follows: ; in, For the path from the source to the destination, ( u , v () represents an edge on the path. For the edge ( u , v The similarity weights are calculated using a cost function that maps high-similarity edges to low-cost edges, ensuring that historically flawed nodes with high relevance are preferentially included in the propagation path.

[0103] S74. Based on the defect's starting location and propagation path, generate a defect tracing conclusion that includes the defect's cause and scope of impact.

[0104] S75. Extract the defect type, defect location, defect cause, and defect impact range from the defect tracing conclusion to obtain key defect information.

[0105] S76. Arrange the key defect information in a structured manner according to the preset report template to obtain a structured defect report.

[0106] S77. Push the structured defect report to the operation and maintenance management platform and output the online defect detection results.

[0107] In one embodiment, the cross-site evolutionary knowledge base is constructed and maintained in the following manner: Based on historical cross-site inspection data, a standardized multimodal feature space is established for key equipment of the same model, batch, and service life under a unified global spatiotemporal reference. Standardization is achieved using Z-score normalization; the input is the original multimodal feature values, and the output is the standardized feature values. The calculation formula is as follows: ; in, These are the original multimodal feature values. This is the mean of the feature in historical data. Standard deviation, These are the standardized feature values. This standardization method brings modal features of different dimensions to the same numerical scale, eliminating the impact of dimensional differences on cross-site comparisons.

[0108] Historical inspection data of the device at different sites are mapped to the same feature space, storing full-cycle multimodal time-series data from normal state to early signs to obvious defects, and labeling them with real root cause tags confirmed by humans, forming a cross-site evolutionary knowledge base.

[0109] In one embodiment, the temporal graph neural network achieves cross-site evolutionary knowledge base matching in the following manner: The temporal graph neural network takes the current defect multimodal causal graph as input. causal graphs verified by history The network outputs the similarity value Sim between the two images. The network includes the following modules, layers, and connections: Graph Attention Encoding Layer: Contains two graph attention sub-layers, each employing a multi-head attention mechanism. The first graph attention sub-layer's input dimension is the node feature dimension. The first graph attention sublayer has an output dimension of 64; the second graph attention sublayer has an input dimension of 64 and an output dimension of 32. The number of attention heads is 4, which is based on the original design of the graph attention network. The multi-head mechanism can extract node relationships from different subspaces, and the four heads strike a balance between representational diversity and computational cost. (Single-head attention coefficient) The computational input is the node With nodes The feature vectors are used as the feature vectors, and the output is the attention weights. The calculation formula is: ; in, For nodes The input feature vector, For a trainable linear transformation weight matrix, For trainable attention weight vectors, For nodes The set of neighboring nodes, This represents vector concatenation. The activation function is a linear unit with leakage correction, and the negative slope is 0.2. Features after node update. for: ; in, This is the ELU activation function.

[0110] Temporal Encoding Layer: The node features output from the graph attention encoding layer are flattened over time steps and input into a unidirectional LSTM sublayer with a hidden layer dimension of 32 to extract the temporal features of the causal graph evolution. The LSTM sublayer outputs the hidden state sequence for each time step. .

[0111] Graph pooling layer: Global average pooling is used to aggregate the node features at each time step into a graph-level feature vector. For the th... Each time step, graph-level feature vector The calculation is as follows: ; in, For the first The number of nodes at each time step. For the first The first time step Temporal coding features of each node.

[0112] Similarity measurement layer: Calculates the current causal graph-level feature vector sequence. With historical causal graph-level feature vector sequence The cosine similarity between two graph-level feature sequences is calculated using the following formula: (Input: Two graph-level feature sequences; Output: Similarity value) ; in, For the current causal graph, the first... Graph-level feature vectors at each time step For the historical causal diagram The graph-level feature vectors at each time step have the numerator being the sum of the inner products of the graph-level features after time step alignment, and the denominator being the product of the magnitudes of the two sets of graph-level feature sequences.

[0113] The training steps and parameter settings for a time-series graphical neural network are as follows: Step 1: Extract validated causal graph pairs from the cross-site evolutionary knowledge base. Causal graph pairs with the same root cause are labeled as positive samples, and causal graph pairs with different root causes are labeled as negative samples, thus constructing a training graph pair set.

[0114] Step two involves iteratively training the initial temporal graph neural network with the training graph pair set as input, using a contrastive loss function. The input consists of positive and negative sample graph pairs, and the output is both the similarity score and the loss value. The calculation formula is as follows: ; in, The set of all training image pairs, including a subset of positive samples. With negative sample subset ; The value is 1 for positive samples and 1 for negative samples. For a single sample loss, the value for a positive sample is [value to be filled in]. For negative samples, the value is... ; For the first The first causal diagram and the first The similarity output of each causal graph. The threshold value is 0.5. This value of 0.5 is based on a common setting in the field of metric learning, which can effectively distinguish the similarity distribution of positive and negative samples, making the similarity of positive samples approach 1 and the similarity of negative samples lower than 0.5.

[0115] Step 3: Update the network parameters using the Adam optimizer, with an initial learning rate of 0.001 and a batch size of 32. This value of 32 is based on the memory limitations and gradient stability in graph neural network training; 32 graph pairs per batch can complete training on mainstream GPUs while maintaining controllable gradient estimation variance.

[0116] Step four: When the training loss decreases by less than 0.001 over 10 consecutive iterations, it is considered to have reached convergence and stability, training is stopped, and the trained temporal graph neural network is obtained. This 0.001 convergence threshold is based on the commonly used early stopping criterion in deep learning training, which can achieve a balance between sufficient training and preventing overfitting.

[0117] In one embodiment, when the discrimination result is a regional common defect cluster, an unmanned vehicle carrying a mobile environmental sampling device performs a cross-site verification task, while a drone simultaneously expands the monitoring range to scan the thermal field distribution of similar equipment in the area, and a robot dog delves into suspected affected equipment to perform detailed multimodal verification. The verification data is then aligned with a global spatiotemporal benchmark and compared with the original root cause inference results for consistency.

[0118] Consistency comparison uses consistency coefficients The input consists of the original root cause probability vector and the validated root cause probability vector, and the output is the consistency coefficient value. The calculation formula is as follows: ; in, This is the root cause probability vector in the original root cause inference results. This is the root cause probability vector after cross-site validation. This represents the maximum possible difference between the probability vectors, and is set to 1. This value is based on the normalization property of the probability space; the maximum difference between two probability vectors is 1. When the verification result supports the original inference, the complete data is stored in the cross-site evolutionary knowledge base; when If the verification result negates the original inference, the confidence weight of the corresponding causal edge in the defective multimodal causal graph is corrected by backtracking, and the root cause probability inference is performed again. The value of 0.8 is based on the high confidence standard in engineering verification, ensuring that the knowledge base only includes high-quality samples that have undergone rigorous verification.

[0119] like Figure 5 As shown, in one embodiment, the construction of the multimodal physical causal graph and the root cause probabilistic inference are achieved in the following manner: Using key components of suspected defective equipment as nodes and physical causal relationships between modes as edges, a multimodal causal graph of defects is constructed. The types of edges include at least: dirt visual feature nodes pointing to leakage current nodes, leakage current nodes pointing to thermal anomaly nodes; vibration transmission nodes pointing to structural deformation nodes, structural deformation nodes pointing to laser point cloud deviation nodes, laser point cloud deviation nodes pointing to contact resistance nodes, contact resistance nodes pointing to local heating nodes; environmental temperature and humidity nodes pointing to thermal field distribution nodes, and thermal field distribution nodes pointing to thermal anomaly nodes.

[0120] The baseline data of the microenvironment collected by the unmanned vehicle and the environmental parameters at the corresponding location of the station in the inter-station environmental gradient curve are used as environmental factor nodes to inject into the defect multimodal causal graph, and an explicit causal connection between environmental factors and equipment status is established.

[0121] The modal features collected by the air-to-ground stereoscopic observation are used as evidence input to the defective multimodal causal graph. A Bayesian network confidence propagation algorithm is employed to calculate the posterior probability of each potential root cause. The input is the evidence set and the causal graph structure; the output is the posterior probability value of each root cause. The joint probability distribution calculation formula is: ; in, to Let each node in the defect multimodal causal graph be a random variable. For nodes The set of parent nodes, These are the parameters of the conditional probability table, estimated using historical statistical data.

[0122] The formula for calculating the posterior probability of each potential root cause is as follows: ; in, For the first One potential root cause, For the collection of evidence, root cause The prior probability, root cause Provide evidence The likelihood probability, The marginal probability of the evidence is calculated using the law of total probability. Potential root causes include at least four categories: degradation of the main material, loosening of connectors, environmental pollution, and external thermal interference.

[0123] The entire closed-loop cycle consists of second-level suspected defect judgment, minute-level air-ground three-dimensional observation for evidence collection, second-level root cause probability reasoning, and hour-level cross-station verification and knowledge base update, forming a complete closed loop for multi-modal data cross-station correlation analysis and defect tracing of substation inspections.

[0124] This application also provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, it implements the above-mentioned method for cross-station correlation analysis and defect tracing of multimodal data in substation inspection.

[0125] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for cross-station correlation analysis and defect tracing of multimodal data during substation inspection.

[0126] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0127] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for cross-site correlation analysis and defect tracing of multimodal data from substation inspections, characterized in that, Includes the following steps: Geographic coordinates and timestamps are collected and spatiotemporal reference trajectories are generated. Micro-meteorological data, acoustic data and visible light images are collected simultaneously and encoded into multimodal data volumes based on the spatiotemporal reference trajectories. A local coordinate system within the substation is established with reference to the substation building layout. The multimodal data volume is mapped to the local coordinate system for coordinate transformation and sorting integration to obtain the registered multimodal observation sequence. Cross-temporal correlation analysis was performed on the registered multimodal observation sequences to extract temporal evolution features and generate equipment state evolution maps; Defect signs are identified based on equipment state evolution maps, suspected defect signs are determined and defect coordinates are extracted; Based on the defect coordinates, an air-to-ground stereoscopic observation pair is triggered to collect local data and obtain detailed defect observation data; Cross-station correlation comparison was performed on the detailed defect observation data, and a cross-station defect correlation mapping was established to obtain the cross-station defect correlation results; Based on the cross-site defect correlation results, defect source tracing analysis is performed to trace the defect propagation path and generate defect source tracing conclusions. Based on the defect source tracing conclusions, online defect detection results are output.

2. The method for cross-site correlation analysis and defect tracing of multimodal data in substation inspection according to claim 1, characterized in that, Geographic coordinates and timestamps are collected to generate a spatiotemporal reference trajectory. Simultaneously, micro-meteorological data, acoustic signature data, and visible light images are collected and encoded into a multimodal data volume based on the spatiotemporal reference trajectory, including: Obtain the real-time geographic coordinates of the inspection vehicle and simultaneously record the corresponding timestamp information to obtain the original spatiotemporal data pair; The original spatiotemporal data pairs are time-aligned and coordinate-smoothed to eliminate positioning noise and obtain smoothed spatiotemporal data pairs. Smoothed spatiotemporal data pairs are sorted and stitched together in chronological order to generate a spatiotemporal baseline trajectory; Micro-meteorological data is obtained by collecting ambient temperature, humidity and air pressure data through micro-meteorological sensors; acoustic data is obtained by collecting acoustic fingerprint signals of the device operation through acoustic sensors; and visible light images are obtained by collecting surface images of the device through a visible light camera. Micrometeorological data, acoustic data, and visible light images are aligned with timestamps and spatiotemporal reference trajectories to obtain aligned multimodal data; The aligned multimodal data is encapsulated in a unified format and encoded into a multimodal data body.

3. The method for cross-station correlation analysis and defect tracing of multimodal data in substation inspection according to claim 2, characterized in that, A local coordinate system was established within the substation, using the substation building layout as a reference. The multimodal data volume was then mapped to this local coordinate system for coordinate transformation and sorting, resulting in the registered multimodal observation sequence, including: Using the substation building layout drawings as a reference, a local coordinate system within the station is established, and the origin and direction of the coordinate axes are determined. Extract the geographic coordinates from the spatiotemporal reference trajectory, calculate the relative coordinates of the geographic coordinates in the local coordinate system within the station, and obtain the local coordinate mapping relationship; Based on the local coordinate mapping relationship, the micro-meteorological data, acoustic data and visible light images in the multimodal data volume are transformed to the local coordinate system within the station to obtain the multimodal data after coordinate transformation; The multimodal data after coordinate transformation are sorted and integrated according to timestamp and spatial location to obtain the registered multimodal observation sequence.

4. The method for cross-site correlation analysis and defect tracing of multimodal data in substation inspection according to claim 3, characterized in that, Cross-temporal correlation analysis was performed on the registered multimodal observation sequences to extract temporal evolution features and generate equipment state evolution maps, including: The registered multimodal observation sequences were grouped according to the equipment number to obtain the single-equipment observation sequences for each equipment. The time window is divided into several continuous time segments by dividing the single-device observation sequence of each device; Within each consecutive time segment, the temporal change rate of micro-meteorological data, acoustic data, and visible light images is calculated to obtain the temporal change rate of each mode. By fusing and correlating the temporal change rates of each mode, cross-modal temporal evolution features are extracted to obtain temporal evolution features; Device state nodes are constructed based on temporal evolution characteristics, and each device state node is connected in chronological order to generate a device state evolution map.

5. The method for cross-site correlation analysis and defect tracing of multimodal data in substation inspection according to claim 4, characterized in that, Based on the equipment state evolution map, defect symptoms are identified, suspected defect symptoms are determined, and defect coordinates are extracted, including: The feature differences between the current device state node and the historical normal state node are extracted from the device state evolution graph to obtain the state deviation features; Input the state deviation features into the preset defect symptom recognition model, calculate the defect probability value, and obtain the defect probability distribution; Filter out device status nodes that exceed a preset threshold in the defect probability distribution to identify suspected defect signs; Extract the spatial coordinates of the equipment status nodes identified as suspected defect signs in the local coordinate system within the station to obtain the defect coordinates.

6. The method for cross-site correlation analysis and defect tracing of multimodal data in substation inspection according to claim 1, characterized in that, Based on the defect coordinates, an air-to-ground stereoscopic observation pair is triggered to collect local data, obtaining detailed defect observation data, including: The defect coordinates are sent to the air-to-ground stereo observation pair to obtain the observation task instructions; the air-to-ground stereo observation pair is composed of a drone and a robot dog working together. Control the drone to fly to the position above the defect coordinates, collect the top view image and infrared thermal image data of the defect area, and obtain aerial observation data; The robot dog is controlled to move to the near-ground position corresponding to the defect coordinates, and detailed images and local acoustic data of the defect surface are collected to obtain ground observation data; By aligning and modally fusing aerial and ground-based observation data in time and space, detailed defect observation data can be obtained.

7. The method for cross-site correlation analysis and defect tracing of multimodal data in substation inspection according to claim 1, characterized in that, Cross-site correlation comparisons were performed on the detailed defect observation data, and cross-site defect correlation mappings were established to obtain cross-site defect correlation results, including: Defect type features and defect severity features are extracted from the detailed defect observation data to obtain the defect feature vector; The similarity between the defect feature vector and the historical defect features in the historical defect database is calculated to obtain the defect similarity matrix; Based on the defect similarity matrix, relevant historical defect records are filtered to establish a cross-site defect association mapping between the current defect and historical defects; Based on the cross-site defect association mapping, a defect propagation association chain is generated to obtain the cross-site defect association results.

8. The method for cross-site correlation analysis and defect tracing of multimodal data in substation inspection according to claim 7, characterized in that, Based on cross-site defect correlation results, defect source tracing analysis is performed to trace the defect propagation path and generate defect source tracing conclusions, including: The topological structure of the defect propagation association chain is extracted from the cross-site defect association results to obtain the defect association topology; Root node analysis is performed on the defect association topology to determine the defect starting node and obtain the defect starting position. Tracing the direction and timing of defect propagation in reverse along the defect correlation topology, and reconstructing the defect propagation path; Based on the defect's starting location and propagation path, a defect tracing conclusion is generated, which includes the defect's cause and scope of impact.

9. The method for cross-station correlation analysis and defect tracing of multimodal data in substation inspection according to claim 1, characterized in that, Based on the defect tracing conclusions, the online defect detection results are output, including: From the defect tracing conclusions, the defect type, defect location, defect cause, and defect impact range are extracted to obtain key defect information; The key information about defects is arranged in a structured manner according to a preset report template to obtain a structured defect report; The structured defect report is pushed to the operation and maintenance management platform, and the online defect detection results are output.

10. The method for cross-station correlation analysis and defect tracing of multimodal data in substation inspection according to claim 5, characterized in that, The pre-defined defect symptom recognition model is trained in the following way: Multimodal observation sequences corresponding to historical normal states and historical defect states are collected as training samples. The temporal evolution features of each training sample are extracted and labeled with state labels to obtain a labeled training set. The labeled training set is input into the initial recognition model for iterative training. The model parameters are optimized until convergence, and the preset defect symptom recognition model is obtained.

Images