Electric power material intelligent detection method based on multi-modal data fusion

Abstract

The invention relates to an electric power material intelligent detection method based on multi-modal data fusion, and aims to improve the accuracy and automation level of material state recognition. According to the method, in the electric power material operation or circulation process, multi-modal data such as images, infrared thermal imaging, radio frequency identification, vibration response and environmental parameters are acquired through a unified time index, and a structured time sequence data set is constructed. And after normalization and exception elimination processing, multi-dimensional feature vectors including structural strength, temperature distribution, label continuity and dynamic stability are extracted, and weighted statistics and correlation calculation are executed to generate a comprehensive state index. And further through comparison with a historical reference, identifying an abnormal state according to a deviation threshold value, outputting corresponding labels and feature information, and obtaining material state evaluation and disposal suggestions based on rule reasoning. According to the method, accurate monitoring and abnormal early warning of the electric power materials under the driving of the multi-source data are realized, and the method has good practicability and expansibility.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology, and in particular to an intelligent detection method for electric power materials based on multimodal data fusion. Background Art

[0002] In existing technologies, the detection and status assessment of power supplies often rely on single-mode data collection and manual judgment. For example, these methods use manual inspection records, static image recognition, or infrared temperature measurement to determine whether a material is abnormal. These detection methods are generally low-cost and easy to use, but they rely on limited data dimensions, making it difficult to fully reflect the complex state of power supplies during dynamic operation, and to achieve the simultaneous integration and analysis of multi-dimensional information.

[0003] However, with the increasing complexity of power system equipment and the diversification of operating conditions, existing detection methods based on a single data source suffer from inaccuracy, poor timeliness, and slow response to abnormal conditions, making them unable to meet the demand for high-reliability monitoring and predictive maintenance of power supplies. Furthermore, some technologies that employ fusion solutions fail to achieve temporal alignment and structural consistency between multimodal data, resulting in noisy and poorly interpretable fusion results, limiting the widespread application of intelligent detection.

[0004] In view of the above situation, it is urgent to propose a new detection method to achieve a more accurate and comprehensive evaluation of the operating status of power materials in complex scenarios. Summary of the Invention

[0005] The present application provides an intelligent detection method for electric power materials based on multimodal data fusion to improve the accuracy and automation level of electric power material status identification.

[0006] This application provides a method for intelligent detection of power materials based on multimodal data fusion, including:

[0007] During the collection period set during the operation or circulation of power materials, image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data of the target materials are acquired, and aligned based on a unified time index to construct a time-ordered structured multimodal dataset.

[0008] Normalizing and removing outliers on the structured multimodal data set, retaining image edge features, temperature distribution information, radio frequency unique identifiers, vibration spectrum characteristics, and environmental fluctuation trends, and generating standardized fusion data;

[0009] Based on a preset multimodal feature mapping rule, a multidimensional feature vector representing the operating status of the target material is extracted from the standardized fusion data, the multidimensional feature vector including structural strength change, thermal imbalance degree, identification tag continuity, and dynamic stability factor;

[0010] Performing weighted statistics and correlation calculation operations on the multidimensional feature vector to generate a comprehensive status index for measuring the current operational integrity and environmental adaptability of the target material;

[0011] Compare the comprehensive status indicator with the historical performance benchmark, identify abnormal status through the set deviation threshold, and output the abnormal label and corresponding key feature components;

[0012] Based on the abnormal labels and comprehensive status indicators, rule reasoning and level mapping are performed to output the status assessment results and processing suggestions of the target materials.

[0013] The beneficial effects of the technical solution provided by this application include:

[0014] (1) It realizes the unified fusion and time-series alignment of multimodal heterogeneous data, and can synchronously collect and integrate multi-dimensional information such as images, thermal imaging, radio frequency, vibration and environment during the operation or circulation of power materials. Compared with traditional single-modal detection methods, it improves the data coverage breadth and state description capabilities. (2) Through normalization processing and feature retention mechanism, it ensures the quality consistency of each modal data before fusion, effectively eliminates outliers and redundant interference, and enhances the stability and accuracy of subsequent feature extraction and analysis. (3) It constructs a multi-dimensional feature vector system based on physical characteristics and environmental factors, and generates comprehensive status indicators through weighted and correlation analysis, making the detection results more physically meaningful and with real-time reference value, significantly improving the assessment accuracy of the operating status of power materials. (4) It combines historical performance benchmarks with rule reasoning mechanisms to realize anomaly identification and level judgment, and can automatically output disposal suggestions. It has strong intelligent early warning capabilities and decision-making assistance value, which helps to improve the operation and maintenance efficiency and safety assurance capabilities of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flowchart of an intelligent detection method for electric power materials based on multimodal data fusion provided in the first embodiment of the present application. DETAILED DESCRIPTION

[0016] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0017] The first embodiment of the present application provides an intelligent detection method for electric power materials based on multimodal data fusion. Figure 1 , which is a schematic diagram of the first embodiment of this application. Figure 1 The first embodiment of the present application provides a detailed description of an intelligent detection method for electric power materials based on multimodal data fusion.

[0018] Step S101: During a set collection period during the operation or circulation of electric power materials, image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data of target materials are acquired, and aligned based on a unified time index to construct a structured multimodal dataset sorted by time.

[0019] In step S101, a stable and representative collection cycle must be set according to the actual application scenario during the operation or circulation of electric power materials, such as every 30 seconds, 1 minute, or other suitable durations. This collection cycle should cover the key state change points during the operation of the materials to ensure that the dynamic performance characteristics of the materials can be reflected. In each collection cycle, it is necessary to obtain image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data of the target materials through a variety of sensor devices or collection equipment deployed at key nodes. The five types of data sources must be clearly configured and the implementation methods must be given in the instructions.

[0020] Image data can be acquired through high-resolution industrial cameras, using continuous frame shooting to capture surface images of power materials, and used to capture visual anomalies such as appearance defects, surface cracks, and wear. Infrared thermal imaging data can be acquired by an infrared thermal imager. The acquired data should be a temperature distribution diagram with a two-dimensional matrix structure, in degrees Celsius, reflecting the thermal distribution of the material surface or interface area. Radio frequency identification data is acquired through an RFID reader / writer, which reads the RFID tag information attached to the surface of the power material and records its unique identification, batch number, production information, and circulation history. Vibration response data must be acquired through an accelerometer or MEMS vibration monitor, recording the vibration frequency, amplitude, and its changing trend in a specific direction, with particular attention paid to high-frequency vibration anomalies. Environmental parameter data includes but is not limited to temperature, humidity, atmospheric pressure, wind speed, electromagnetic interference level, etc., and is acquired in real time through the environmental monitoring unit to analyze the impact of external environmental changes on the test results.

[0021] The timestamps for all of the aforementioned data should be accurate to the millisecond level, and the acquisition system should use a unified time synchronization protocol (such as NTP or PTP) for data time alignment to ensure comparability and temporal consistency across all modal data within the same acquisition cycle. All acquired data should be organized in a structured format, using a unified data structure format such as JSON or a multidimensional array. Each record should contain the corresponding image frame, heat map matrix, RFID string, vibration time series data, and environmental parameter array, along with a standard timestamp.

[0022] After the data structure is constructed, data samples from each acquisition cycle must be arranged chronologically, ultimately forming a structured, time-ordered multimodal dataset. This dataset forms the foundation for subsequent processing operations such as normalization, feature extraction, and state analysis. Data integrity, modal consistency, and temporal continuity must be ensured to ensure the correct execution of subsequent steps. The dataset can be stored in a local cache, database, or real-time processing platform. A time-series database system that supports multimodal indexing and batch processing, such as InfluxDB or time-series-scalable PostgreSQL, is recommended to meet the high-frequency acquisition and multimodal processing requirements of industrial sites.

[0023] In summary, the execution of step S101 not only requires the accurate collection of the five types of data, but also ensures their time alignment and structural organization, forming a structured multimodal dataset with clear logic, unified format, and time order, laying the foundation for subsequent multimodal fusion and intelligent analysis.

[0024] Step S102: normalizing and removing outliers on the structured multimodal data set, retaining image edge features, temperature distribution information, radio frequency unique identifier, vibration spectrum characteristics and environmental fluctuation trends, and generating standardized fusion data.

[0025] In step S102, normalization and outlier removal operations need to be performed on the time-ordered structured multimodal dataset constructed in step S101. The purpose is to improve the consistency of the data and the robustness of subsequent processing, while ensuring that the features of each modality are relatively comparable and retaining key features that are discriminative of the target material status.

[0026] Normalization targets include image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data. For image data, local contrast enhancement or linear grayscale stretching can be used to normalize pixel values ​​to the range [0, 1] or [-1, 1] while preserving the edge structure of the original image. Preprocessing with an edge-preserving filter (such as a Gaussian bilateral filter) is recommended to prevent loss of image edge information during normalization. Images can be normalized by channel, such as independently normalizing the R, G, and B channels in an RGB image.

[0027] For infrared thermal imaging data, each pixel value in the two-dimensional temperature matrix should be subtracted from the average temperature of the current frame and divided by the standard deviation to ensure that the thermal distribution characteristics remain relatively stable under different environmental backgrounds. If environmental interference causes abnormal offset of the thermal image, the sudden change should be detected through inter-frame differencing and invalidated.

[0028] Unique identifiers in RFID data typically exist as strings and are not subject to numerical normalization. However, format standardization is required, such as unifying character encoding formats and removing whitespace and unusual characters. Tag status data (such as read intensity or latency) should be processed according to minimum and maximum normalization rules to facilitate subsequent integration with other modal data.

[0029] Vibration response data is typically a one-dimensional time series signal. Features such as spectral energy or acceleration peaks can be extracted using a sliding window. Normalization uses a zero-mean unit variance transformation to ensure consistent response dimensions across different frequency bands or axes. A reasonable physical threshold should also be set (e.g., high-frequency vibrations unlikely to occur in mechanical structures), and data points exceeding the physical upper limit should be eliminated.

[0030] Normalization of environmental parameter data requires processing based on historical distribution ranges. For example, data such as temperature, humidity, and air pressure should be mapped to the [0, 1] range. Collection anomalies (such as constant values ​​or missing data caused by sensor detachment) should be detected and abnormal values ​​marked as missing. In fault-tolerant systems, some missing data can be interpolated, such as using linear or spline interpolation. If the system requires data rigor, missing samples can be directly eliminated.

[0031] The logic for outlier removal should be integrated throughout the normalization process described above. This involves detecting outliers by setting the dynamic range of each modal feature or based on statistical characteristics (e.g., exceeding ±3 standard deviations from the mean), and verifying these based on historical sampling frequency and time series context. Data segments that are short-lived but physically feasible can be retained to represent sudden material changes. Data that clearly violates physical laws or system accuracy limits should be removed.

[0032] After normalization and outlier removal, the modal data must be realigned by timestamp and standardized fused data constructed in a unified format. Each time point corresponds to a set of processed image matrices, thermal imaging matrices, RFID tags, vibration vectors, and environmental parameter arrays. This fused data set serves as the input for step S103 and must be structurally complete, scaled, and clean, with key feature information retained for each modality.

[0033] In summary, the key to step S102 is to ensure the uniformity of the numerical range, the integrity of the feature information, and the reliability in a physical sense of the structured multimodal dataset, thereby improving the data quality and analysis accuracy of the entire intelligent detection process of power materials.

[0034] Furthermore, the normalization and outlier removal of the structured multimodal dataset includes:

[0035] In the structured multimodal dataset, a consistency check of cross-abnormal regions between modalities is performed on the image data and the infrared thermal imaging data. By constructing a pixel-level coordinate mapping relationship, it is determined whether the visible damage area appearing in the image and the local temperature rise area existing in the infrared thermal imaging spatially overlap. If the overlap is lower than a preset threshold, the data point is marked as an inconsistent outlier and is removed.

[0036] For the vibration response data, the boundary drift rate is calculated based on a statistical model of the rate of change of the main peak position of the spectrum in continuous time segments, and a direction-sensitive filtering strategy is used to identify the mutation points. If the mutation point shows a unilateral deviation trend in the adjacent time window, it is marked as a dynamic outlier and removed;

[0037] Based on the preset physical distribution prior information of the target material, a modal normalization scale adjustment factor is constructed for the image data, infrared thermal imaging data, and vibration response data, respectively, so that the unit change values ​​under different modes maintain scale consistency under the same physical meaning. The adjustment factor is used to correct the initial normalization results and generate standardized fusion data under a unified physical scale;

[0038] The standardized fusion data after removing outliers and completing scale correction is provided as input to the step of extracting a multidimensional feature vector for representing the operating status of the target material from the standardized fusion data based on the preset multimodal feature mapping rule.

[0039] The overall goal of this step is to process the structured multimodal dataset constructed in the previous phase. This dataset includes data from multiple different modalities: image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data. Given the significant differences in data format, sampling frequency, physical meaning, and unit dimensions, normalization and outlier removal must be performed before further feature extraction to ensure uniformity and comparability across all data points. This also eliminates anomalous data points that could impact the overall judgment quality.

[0040] First, performing inter-modal cross-abnormal area consistency check on image data and infrared thermal imaging data is a key step in the present invention for improving the accuracy of image-type modal anomaly removal. Image data is usually acquired by a high-resolution industrial camera and saved as a two-dimensional pixel matrix in three-channel (RGB) or grayscale format; infrared thermal imaging data is provided by an infrared thermal imager and represented in the form of a temperature matrix. In order to achieve inter-modal consistency check, the two modalities must first be aligned in coordinate space. The specific method is to set a consistent imaging perspective in the acquisition device and configure a calibration plate or common viewing area in the camera and thermal imager to achieve pixel-level coordinate mapping relationship construction. For each frame of the image, image edge detection or segmentation algorithm (such as gradient-based Canny operator or color and shape-based region segmentation algorithm) is used to extract the suspicious damage area; for the infrared thermal image at the corresponding time point, temperature threshold segmentation or region growing algorithm is used to extract the local temperature rise area.

[0041] After establishing the mapping relationship, the spatial overlap between each image damage area and its corresponding thermal imaging temperature rise area needs to be calculated. The IoU (Intersection over Union) indicator can be used as the overlap metric. If a suspected damage area cannot find a corresponding temperature rise area in the infrared thermal image (that is, the IoU is much smaller than the set threshold, for example, IoU < 0.1), it means that the changes detected in the image may be artifacts, surface stains or light interference, rather than real structural degradation; vice versa, if local overheating occurs in the thermal imaging but there is no structural abnormality in the image, it can be considered that the internal hidden danger has not yet manifested externally. Therefore, when the overlap is lower than the preset threshold, the data point is marked as an inconsistent outlier. Such outliers will be eliminated from subsequent analysis to avoid misleading judgments caused by contradictory signals between modalities.

[0042] Secondly, regarding the processing of vibration response data, in structured multimodal datasets, vibration response data exists in the form of time-series acceleration or velocity values, and multidimensional sensors are often used to collect lateral, longitudinal, and axial vibration quantities. In order to identify dynamic outliers, the concept of the rate of change of the main peak position of the spectrum needs to be introduced. The specific method is to perform a Fourier transform on the vibration signal within each time window to obtain the spectral energy distribution and locate the main peak frequency. The change trajectory of the main peak frequency over time is recorded in multiple consecutive time windows, and its first-order difference sequence is constructed, namely the main peak position change rate curve. The boundary drift rate statistical model is used to identify the mutation points that appear in this curve. Typical algorithms include the CUSUM algorithm based on mean shift detection, sliding variance mutation analysis, or convolution mutation detector.

[0043] To enhance robustness, a direction-sensitive filtering strategy is further introduced. This strategy not only assesses the strength of the mutation but also examines its temporal unilaterality. For example, a continuous rise or fall in the dominant frequency without rebound suggests possible physical changes, such as structural loosening or a change in the vibration source. If the duration of a mutation exceeds a set threshold (e.g., more than three sampling windows) and the direction of the shift is consistent, the point is identified as a dynamic outlier and removed from the standardized fusion data to prevent unstable states from affecting the accuracy of subsequent state assessments.

[0044] Furthermore, to address the issue of physical unit differences between image data, infrared thermal imaging data, and vibration response data, the present invention constructs a normalized scale adjustment factor within the modality based on the preset physical distribution prior information of the target material. This prior information includes the structural dimensions of the target material (e.g., busbar width, casing length), material properties (e.g., thermal conductivity, elastic modulus), and environmental usage conditions (e.g., ambient temperature fluctuation range). Before normalization, the measurement range of each modality is initially standardized, such as linear scaling to [0,1] or standard deviation normalization.

[0045] Adjustment factors are then introduced to modify the standardized results, ensuring that different modes maintain scale consistency under the same physical meaning. For example, for the thermal map response caused by a 1°C temperature increase, if the corresponding material is a high heat capacity material, the physical significance of this change is far lower than the same amplitude on a low heat capacity material. In this case, the thermal map adjustment factor can be set based on the material's thermal diffusivity; the spatial noise tolerance factor for image edge changes can be set based on the target structure's thickness; and the response gain factor for vibration signals can be set based on the natural frequency of the mounting structure. These adjustment factors can be set in a tabular format or derived through material modeling and simulation.

[0046] After the above corrections, the obtained standardized fusion data not only achieves intra-modal normalization, but also completes the consistent mapping of cross-modal physical scales, providing a solid data foundation for subsequent feature vector extraction.

[0047] Finally, it is important to specify that the results of this normalization and outlier removal serve as cleaned, valid input data and are directly fed into the step of "extracting a multidimensional feature vector representing the target material's operating status from the standardized fused data based on pre-defined multimodal feature mapping rules." To facilitate this data transfer, fields such as the timestamp, modality identifier, location index, and modality fusion flag should be retained in the data structure to ensure that subsequent steps can correctly associate multimodal features and perform efficient feature fusion.

[0048] This step can be implemented on an edge computing terminal, a local data acquisition platform, or a centralized server. It is recommended to use a processing platform that supports vectorized operations and asynchronous cleaning tasks.

[0049] Step S103: Based on a preset multimodal feature mapping rule, a multidimensional feature vector is extracted from the standardized fusion data to represent the operating status of the target material. The multidimensional feature vector includes structural strength change, thermal imbalance degree, identification tag continuity and dynamic stability factor.

[0050] In step S103, based on pre-defined multimodal feature mapping rules, a multidimensional feature vector representing the target material's operating status is extracted from the standardized fused data obtained in step S102. This process requires a clear understanding of the physical characteristics and temporal distribution of each modal data. Using these mapping rules, the observed values ​​from different modalities are converted into a unified numerical representation for subsequent computational analysis.

[0051] First, for image data, it is recommended to extract contour regions using edge detection algorithms (such as the Canny or Sobel operators) and further calculate the structural continuity change value. For example, the overlap rate change of edge pixels within consecutive sampling cycles can be defined as a quantitative indicator of structural strength change to reflect whether the target material surface has crack propagation, mechanical looseness, or wear deterioration.

[0052] For infrared thermal imaging data, a thermal imbalance index should be calculated based on the spatial gradient of the temperature distribution in the two-dimensional thermal map. This index can be expressed as the pixel-level temperature standard deviation, the mean of the temperature gradient vector modulus, or the maximum temperature difference coefficient, reflecting conditions such as abnormal thermal diffusion, local overheating, or thermal mismatch. To ensure stability, temperature difference normalization should be used to reduce the interference of absolute temperature changes on the index.

[0053] Radio frequency identification data is mainly used to extract the continuity of identification tags. The tag reading stability can be judged based on whether the RFID tags of the same material are continuously received within the same time window. A tag continuity score is generated by calculating the number of tag recognition losses, response delays, or the fluctuation range of tag signal strength, thereby reflecting the tracking stability or tag integrity of the materials during the circulation process.

[0054] Vibration response data requires frequency-domain processing methods such as Fourier transform or wavelet decomposition of the acceleration time series to extract features such as the dominant frequency, resonance peak, and energy distribution. The dynamic stability factor can be defined as the amplitude of the dominant frequency position within consecutive cycles, the volatility of the vibration energy distribution, or the trend of the vibration response envelope curve. These characteristic indicators are used to reflect the mechanical stability and stress anomalies of materials during transportation, use, or storage.

[0055] All of the above features need to be organized in a unified feature vector format. That is, for each acquisition time point, a numerical vector containing structural strength changes, thermal imbalance, identification label continuity, and dynamic stability factors is constructed, and its temporal structure is maintained to form a complete multidimensional feature vector sequence.

[0056] Furthermore, feature mapping rules must be configurable and scalable, allowing for the adjustment of feature extraction methods and parameter settings for different types of power supplies. For example, for transformers requiring high thermal stability, the weight and resolution of the thermal imbalance feature should be increased; while for high-value consumables requiring tag traceability, the accuracy of tag continuity detection should be improved.

[0057] In summary, the core of step S103 is to extract a unified feature vector under multimodal fusion from the standardized fusion data in a physically interpretable, mathematically executable, and semantically mappable manner, so as to comprehensively reflect the state evolution of the target material during operation and provide a solid data foundation for the subsequent generation of comprehensive status indicators and anomaly identification.

[0058] Furthermore, the extracting of a multidimensional feature vector for representing the operating status of the target material from the standardized fusion data based on a preset multimodal feature mapping rule includes:

[0059] Based on the historical collaborative response relationship between the modal data in the standardized fusion data, a weighted directed graph model is constructed to characterize the inter-modal causal structure, wherein each node of the weighted directed graph corresponds to a feature channel of the image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data, and the edge weight represents the degree of causal coupling of inter-modal response changes in historical operating scenarios;

[0060] Calculating the modal influence distribution based on the total weight of the incoming edges of each node in the weighted directed graph, and combining the environmental parameter data of the current collection period of the electric power material with a context-aware mechanism to dynamically select a modal subset with effective information contribution in the current state as a feature extraction input;

[0061] Redundancy evaluation is performed on the original features in the selected modality subset. If there are features with similar physical expression dimensions between the modalities, such as temperature distribution features in thermal images and regional gradient features in image data, a feature folding path is constructed based on the physical dimension similarity, and weighted fusion coding is used to merge them into a unified sub-feature coding vector;

[0062] The sub-feature encoding vectors, together with the modal identifiers and time indexes on which they depend, are used to construct a cross-modal fusion feature set, and feature dimensions with weak correlation or high volatility are dynamically reduced in dimension and orthogonality enhanced to form a multi-dimensional feature candidate set;

[0063] From the multidimensional feature candidate set, an optimal subset is selected according to the matching degree between the structural category of the target material and the historical operation status label as the multidimensional feature vector for representing the operation status of the target material.

[0064] Before entering the feature extraction phase, the system completes the acquisition, time alignment, modal unification, and normalization of raw image data, infrared thermal imaging data, RFID data, vibration response data, and environmental parameter data. At this point, the input data forms a standardized, time-indexed fusion dataset with a stable structure, consistent modal data, and no outliers.

[0065] In order to fully explore the implicit correlation rules between modal data and improve the quality of cross-modal information fusion, a multimodal feature mapping framework based on a weighted directed graph structure is proposed. Specifically, the system first needs to build a directed graph structure to characterize the causal dependency between modalities in historical operation data. Each node in the directed graph represents a feature channel of a modality, such as the edge gradient of image data, the temperature distribution of infrared thermal imaging, the receiving intensity of the radio frequency identification tag, the spectral characteristics of the vibration response, and the temperature and humidity values ​​of the environmental parameters. The directed edge in the graph points from one modal feature node to another modal feature node, indicating that in the historical operation scenario, the state change of the former will produce an incremental information contribution to the latter.

[0066] The process of constructing this graph structure requires statistical modeling of massive amounts of historical data. For example, using image and thermal imaging features, if statistics show that the probability of a local temperature rise in the thermal image increases significantly with each edge anomaly, and this increase in probability is significantly higher than the baseline probability when the image anomaly is absent, then the "image → thermal image" path can be considered to have a positive causal contribution. An edge can then be established from the image feature channel to the thermal imaging channel, with the edge weight defined as this probability difference multiplied by the sample support. A similar approach can be used to construct causal edges such as vibration → image, thermal imaging → RF loss, and so on. The entire graph structure is a weighted directed graph, where the edge weight reflects the degree of causal coupling between modalities, with larger values ​​indicating stronger influence.

[0067] After the causal graph is constructed, when entering the actual feature extraction process in each acquisition cycle, the "modal influence distribution" of each modal node at the current moment needs to be calculated based on the graph structure. The method is to calculate the sum of the edge weights of each node's input to obtain the degree to which the node is dominated by other modes in the causal structure. This influence distribution is then combined with the actual environmental parameter data in the current acquisition cycle as input to trigger the context-aware mechanism. Under this mechanism, the system determines which modes have higher information value in the current state based on the current environmental characteristics (such as whether the temperature fluctuates violently or whether the humidity is close to the limit value). For example, in a high-temperature environment, the importance of the thermal imaging mode increases; in the event of severe mechanical shock, the vibration response mode takes precedence.

[0068] The above processing will dynamically select a set of modal subsets that have effective information contribution in the current cycle as input for subsequent feature extraction. When performing feature evaluation on this subset, it is necessary to further solve the problem of information redundancy between modalities. Since there is an overlap in the representation of physical quantities in multimodal data, for example, the high temperature area in the infrared thermal image may correspond to the area with significant grayscale changes in the image, it is necessary to perform redundancy evaluation and feature folding strategies. The system is based on the similarity of physical expression dimensions, that is, it determines whether two features describe the same state attributes in the physical space, but are measured through different modalities. If such highly redundant features exist, the system will establish a feature folding path and use weighted fusion coding to encode them into a set of unified sub-feature encoding vectors. The weighting method can be set based on the modal historical contribution, the current feature confidence or the context matching degree.

[0069] Subsequently, the system combines these sub-feature encoding vectors with the modal identifiers and time indexes on which they depend to construct a cross-modal fusion feature set. After the set is formed, it is not directly used as the final feature vector, but requires further feature selection optimization. First, the system calculates the correlation between feature dimensions and eliminates weakly correlated dimensions, that is, there is no significant statistical dependency between this dimension and other key features. Secondly, a volatility index is introduced to determine whether the amplitude of the feature change within the sliding window is too high, thereby eliminating unstable features. Thirdly, orthogonality enhancement methods such as PCA (principal component analysis) or ICA (independent component analysis) are used for dimensionality reduction to ensure that the final feature set has the triple characteristics of low redundancy, high discrimination, and high stability.

[0070] The resulting set of multidimensional feature candidates enters the final screening phase. The system retrieves structural category information for the target material, such as transformers, switchgear, conductors, and circuit breakers, and matches it against a database of historical operating status labels to identify the subset of feature combinations that most closely matches the current structural type and historical state evolution pattern. The screening criteria can be determined based on a combination of metrics such as cluster distance, discriminative power, and minimum intra-class variance. The resulting subset of features becomes the multidimensional feature vector representing the target material's operating status.

[0071] This multidimensional feature vector not only includes standardized values ​​but also includes confidence levels, reflecting the stability and explanatory power of each dimension within the given context. This final vector serves as direct input for the subsequent step of performing weighted statistics and correlation calculations on the multidimensional feature vector to generate a comprehensive status indicator, ensuring a closed-loop data chain throughout the entire inspection process.

[0072] The following is the reference implementation code for the above content, which covers the construction of a weighted directed graph between modalities using historical collaborative response relationships; dynamic selection of modal subsets based on environmental parameters; performing physical dimension redundancy judgment and feature folding and fusion; constructing a cross-modal fusion feature set and performing orthogonal dimensionality reduction; and selecting the optimal feature subset based on the target structure and state labels:

[0073]

[0074]

[0075]

[0076]

[0077]

[0078] Step S104: performing weighted statistics and correlation calculation operations on the multi-dimensional feature vector to generate a comprehensive status index for measuring the current operational integrity and environmental adaptability of the target material.

[0079] In step S104, weighted statistics and correlation calculations are performed on the multidimensional feature vectors representing the target material's operational status, extracted in step S103. This ultimately generates a comprehensive status indicator, used to measure the target material's current operational integrity and environmental adaptability. This step plays a crucial role in connecting the entire inspection process. Its output serves as both a quantitative representation of the fused multimodal features and a direct basis for subsequent anomaly identification and status assessment.

[0080] The multidimensional feature vector includes four types of indicators: structural strength change, thermal imbalance, identification tag continuity, and dynamic stability factor. Each type of indicator exists in the form of a time series, representing the evolution trend of the operating status of the target material within a continuous sampling period. On this basis, through weighted statistical operations, these feature vectors of different dimensions are fused under a unified dimension. Specifically, a pre-set weight coefficient can be assigned to each type of feature. The weight can be adjusted based on the actual application scenario, equipment type, or historical experience. For example, the weight of the thermal imbalance degree can be increased for equipment with higher thermal stability requirements.

[0081] Weighted statistical operations should utilize a sliding window strategy. Within each window period, statistics such as the weighted mean, variance, maximum, and minimum values ​​for each feature type within the current window are calculated to construct stable state characterization parameters. The weighted mean reflects the representativeness of the current feature, the variance reflects the degree of state fluctuation, and the maximum and minimum values ​​capture sudden anomalies. The combination of these multiple statistics effectively enhances the ability to quantitatively describe the operational status of materials.

[0082] After completing the weighted statistics, the correlation between different features needs to be analyzed. The goal of the correlation calculation operation is to evaluate the linear or nonlinear association between the features in the multidimensional feature vector, thereby revealing whether there are linkage anomalies, covariation patterns, or mutual interference between the features. For example, when the structural strength changes significantly, if it is accompanied by a sharp fluctuation in the degree of thermal imbalance, it may indicate a coupling failure trend between the mechanical structure and the heat dissipation performance. Correlation can be measured using mathematical indicators such as the Pearson correlation coefficient, Spearman rank correlation, and mutual information entropy.

[0083] In actual calculations, a feature covariance matrix or correlation coefficient matrix can be constructed to record the correlation indicators between any two features within multiple time windows. Principal component analysis or clustering can then be performed on this matrix to extract key joint feature dimensions. Furthermore, to ensure that abnormal states are not averaged out, an abnormality amplification mechanism should be introduced within the window, assigning higher weights to windows where the variance or correlation coefficient exceeds a threshold, ensuring that the comprehensive status indicator remains sensitive to potential faults.

[0084] Ultimately, the comprehensive status indicator is output as a multidimensional scalar vector, where each dimension represents the performance of the target material under specific state characteristics. This indicator should have a clear physical interpretation, such as "structural stability score," "thermal distribution balance," "label tracking integrity," and "dynamic response consistency." Each score can be normalized to the [0, 1] range and accompanied by a confidence level or error estimate.

[0085] In terms of weight distribution strategy, the four characteristics of structural strength change, thermal imbalance, identification tag continuity and dynamic stability factor should be differentiated according to the operating environment and key performance requirements of the power materials. Taking the transformer as an example, its thermal runaway risk is relatively high, so the weight of thermal imbalance can be set to 0.4, while the weights of structural strength change, identification tag continuity and dynamic stability factor are set to 0.2, 0.2 and 0.2 respectively. For transmission line materials, since mechanical fatigue is the dominant factor, the weights of structural strength change and dynamic stability factor should be increased. The weight value should meet the normalization condition, that is, the sum of all weights is 1, and dynamic adjustment based on empirical data is allowed between different material types.

[0086] The calculation of typical statistics uses a sliding window mechanism. In each statistical period, the representative value of each type of feature is calculated by weighted mean, and the formula is:

[0087]

[0088] Among them, μ w is the weighted mean, which represents the average level of the target feature in the current sliding window, taking into account the importance of samples at each moment. i represents the characteristic value at the i-th time point, w i is the corresponding sampling weight. n is the total number of samples in the current sliding window.

[0089] For indicators that reflect volatility, weighted variance can be used to calculate

[0090]

[0091] in, is the weighted variance, which is used to measure the degree of fluctuation of the eigenvalue around the weighted mean. The larger the value, the more unstable the feature state. i is the eigenvalue of the i-th sample point; μ w is the weighted mean calculated above. i Consistent with the mean formula, it represents the weight of the point. n is the total number of samples in the current sliding window.

[0092] The maximum, minimum, and range values ​​also need to be recorded synchronously to identify sudden abnormal behaviors. In addition, in dynamic monitoring scenarios, the sliding variance change rate can also be introduced as an early sign of instability.

[0093] The correlation index is used to determine the linkage relationship between different state features. It is recommended to use the Pearson correlation coefficient, which is calculated as follows:

[0094]

[0095] Where r is the Pearson correlation coefficient; x i ,y i They respectively represent the values ​​of two different characteristics at the time point, such as structural strength change and dynamic stability factor. is the sample mean of feature x; is the sample mean of feature y. If the relationship between features is nonlinear, methods such as the Spearman rank correlation coefficient or mutual information entropy can be used. In practical applications, |r| > 0.8 is considered a strong correlation, 0.5-0.8 is a moderate correlation, and less than 0.5 is considered a weak correlation or no correlation. Correlation results can be used for subsequent joint anomaly pattern recognition and redundant feature compression.

[0096] The representation format of the comprehensive status index adopts a fixed-length vector structure. It is recommended to use a four-dimensional vector to represent various scoring results. The format is

[0097] S=[s1,s2,s3,s4]

[0098] Here, S represents the comprehensive status indicator vector generated at a specific detection time point or window period, which quantitatively expresses the operating status of the target material. s1 represents the structural strength change score, s2 represents the thermal imbalance score, s3 represents the identification label continuity score, and s4 represents the dynamic stability factor score. Each score ranges from [0 to 1], with values ​​closer to 1 indicating a more abnormal state corresponding to that feature.

[0099] For the structural strength change score s1, the difference of the edge area in continuous image frames can be compared, the image structure change can be calculated using the absolute difference integral method, and the result can be normalized, that is, the edge pixel difference of each frame image is summed and divided by the preset maximum edge change reference value.

[0100] For the thermal imbalance score s2, the absolute value of the difference between all pixel points and the average temperature is calculated based on the temperature gradient map in the same frame of the infrared thermal imaging data, and the sum is used as the integral value. The integral value is then normalized to the interval [0, 1] to reflect the overall deviation of the temperature distribution.

[0101] For the label continuity score s3, we set an upper limit on the theoretical number of labels that should be received within a detection cycle, and use the difference in the number of labels actually received as the integral input, normalized by the proportion of missing labels. For example, if only 7 labels are received when 10 labels should be received, the integral result is 3, and the score is 3 / 10 = 0.3, indicating a high continuity anomaly.

[0102] For the dynamic stability factor score s4, the energy envelope of the vibration signal is extracted within a sliding time window. The integral of the change in the dominant frequency position and the total energy fluctuation is calculated and normalized into a score. Alternatively, the absolute value of the dominant frequency change between each time period can be directly accumulated as the integral result, with a larger value representing greater dynamic instability.

[0103] In addition, to enhance credibility, each score can be added with a confidence factor c i , for example, by adjusting based on score volatility, sample completeness, or correlation stability, to form a structured representation:

[0104]

[0105] To support explainable anomaly analysis and risk level mapping. This is a comprehensive status indicator with a confidence level annotation, used to further provide information on the credibility of the score. c1, c2, c3, and c4: represent the confidence factors for the corresponding scores s1, s2, s3, and s4, respectively. These values ​​range from [0 to 1] and indicate the credibility of the score. For example, when the sampling of a modal data is incomplete, its confidence level decreases, alerting the system or personnel that the score for that dimension may deviate from the actual status.

[0106] The entire processing flow should support three execution modes: batch processing, real-time stream processing, and periodic evaluation. In the batch processing scenario, historical data can be loaded in segments and calculated uniformly; in the real-time scenario, the collected data can be processed while being input based on sliding windows and streaming computing engines (such as Kafka+Spark or Flink); in the periodic evaluation scenario, a timed trigger mechanism can be set (for example, timed analysis every 30 minutes or daily) to automatically complete feature updates and status refreshes. The input format must be consistent in different modes, and structured data packets must be uniformly used to input standardized fusion data, and a fixed-format comprehensive state vector and state label set must be output to ensure cross-platform consistency and seamless integration with subsequent analysis modules.

[0107] Therefore, step S104 transforms complex multimodal features into compact, stable and interpretable comprehensive status indicators through weighted statistics and correlation calculation operations, which not only improves the responsiveness and accuracy of the detection system, but also provides a solid data foundation for subsequent anomaly identification and intelligent decision-making.

[0108] Furthermore, the comprehensive status indicator is compared with the historical performance benchmark, abnormal status is identified through a set deviation threshold, and abnormal labels and corresponding key feature components are output, including:

[0109] Based on the sliding window mechanism, scoring curves for comprehensive status indicators within the continuous acquisition cycle are constructed, generating scoring curves for structural strength change, thermal imbalance, identification label continuity, and dynamic stability factor, forming a multidimensional scoring curve group for trend analysis.

[0110] Perform a trend synchronization analysis on the multidimensional scoring curve group to calculate the consistency of the change direction and the overlap of the fluctuation time series between the scoring curves. When any two or more scoring curves show trend deviation at the same time in the same time period, and their synchronization index exceeds the set criterion, they are determined to be in a linkage anomaly candidate state, and the corresponding time period is marked as a key judgment window;

[0111] Within the key discrimination window, the change rate, stability coefficient, and difference from the historical performance benchmark of the comprehensive state indicators of each dimension are input into the fuzzy logic rule system for reasoning, and the deviation threshold of each dimensional state indicator is dynamically generated. The fuzzy rule system is constructed based on the preset "change rate-credible interval-tolerance limit" mapping relationship to ensure that the deviation threshold has contextual adaptability;

[0112] According to the dynamically generated deviation threshold, the state indicator dimension that deviates within the key discrimination window is identified, and the abnormal label and the corresponding key feature component are output.

[0113] This embodiment first requires the system to continuously collect comprehensive status indicator data over multiple cycles. These indicators are derived from the multi-dimensional feature vector constructed in the previous step to represent the operating status of the target material. After weighted statistics and correlation calculations, a comprehensive status indicator containing scores from several dimensions is generated. For example, the structural strength change score reflects image edge changes and vibration anomalies; the thermal imbalance score reflects the degree of temperature skewness in thermal imaging; the identification tag continuity score is derived from the signal reading frequency and interference intensity in the RFID data; and the dynamic stability factor score is derived from the stability of the position changes of the main peak of the vibration frequency.

[0114] Once the comprehensive status indicators have a stable output mechanism, the system must perform time series modeling based on a sliding window mechanism to construct a multidimensional scoring curve. The sliding window is typically set at a time granularity that matches the operating characteristics of the target material. For example, a 10-second window can be used for conductor devices subject to frequent vibration, while a 5-minute window can be used for equipment significantly affected by slowly changing ambient temperatures. Each scoring dimension forms a time series curve across multiple consecutive windows, such as a structural strength change scoring curve or a thermal imbalance scoring curve. Ultimately, a multidimensional scoring curve set is formed for trend analysis.

[0115] The purpose of performing trend synchronization analysis on the multi-dimensional scoring curve group is to identify abnormal patterns in which multiple indicators deviate simultaneously and have linkage characteristics within the same time period. The analysis process includes two levels. One is the consistency measurement of the direction of change, that is, whether the scoring curves show an upward or downward trend at the same time in the same time window; the other is the judgment of the coincidence of fluctuation time series, that is, to what extent the scoring curves synchronously appear peaks or troughs, or synchronously enter the abnormal fluctuation range. The specific calculation can adopt methods such as Pearson correlation coefficient, DTW (Dynamic Time Warping) distance, mutual information, trend slope matching, etc., among which trend slope matching and cross-correlation calculation within the time window are preferred.

[0116] When at least two scoring dimensions exhibit consistent deviations within the same time period, and synchronization indicators (e.g., average trend slope difference less than a set threshold, mutual information greater than a threshold) exceed the system's defined criteria, that time period is marked as a "linked anomaly candidate state," effectively becoming a key judgment window. Within this window, the system further performs fine-grained analysis of the scoring data for each dimension to identify any deviations beyond the normal range.

[0117] In the key discrimination window, the system uses the rate of change (i.e., score differential) and stability coefficient (such as local variance or coefficient of variation) of each dimension's comprehensive status indicator, along with its corresponding historical performance benchmark (including long-term mean, distribution pattern, maximum tolerance boundary, etc.) as input and feeds it into a fuzzy logic rule system. The system performs reasoning based on the mapping relationship between "rate of change-credible interval-tolerance limit." For example, if the rate of change in a dimension is higher than three standard deviations of the historical average, the score stability drops below the past 5% percentile, and the current score value approaches the upper tolerance limit, the system should derive a stricter deviation threshold, indicating a tightening of the risk critical boundary. Conversely, if the score changes significantly but remains highly stable, and historical performance also fluctuates significantly, the threshold can be relaxed.

[0118] Fuzzy logic rule systems can be constructed based on membership functions and rule bases. Membership functions describe the degree to which input variables fall within fuzzy concepts such as "fast-changing," "stable," "high-risk," and "tolerable." The rule base defines which deviation threshold (tight, loose, standard, etc.) should be output when the inputs are in a certain combination. This mechanism ensures that the system's anomaly recognition threshold is contextually adaptable under different environments, target materials, and load conditions, preventing both false alarms and missed detections.

[0119] Based on the above-mentioned dynamically generated deviation threshold, the system compares the current value of the comprehensive state indicator with it. If the degree of deviation exceeds the generated threshold, the scoring dimension can be judged to be abnormal. It should be noted that this deviation judgment is based on a threshold derived from fuzzy logic rather than a fixed value, and has greater flexibility and explanatory power. At the same time, after identifying the abnormal dimension, the system needs to extract the key feature components that contribute most to the abnormal score value from the multi-dimensional feature vector saved in the previous step. For example, if the structural strength change score is abnormal, it can be traced back to the position with the largest edge change in the original image and its corresponding vibration feature distribution to determine whether the score deviation is caused by edge fracture, crack growth or resonant structural disturbance.

[0120] To improve system explainability and traceability, the system further integrates these key feature components into anomaly labels and their associated attributes. These labels include the anomaly type identifier (e.g., "thermal imbalance," "structural mutation," "label disconnection," etc.), the associated modal channel (e.g., "infrared thermal imaging data," "image data," etc.), the involved feature field index, and the deviation direction (increase or decrease). The anomaly label and its corresponding key feature components serve as the final output of this step, providing a visual diagnostic basis for operations and maintenance personnel and can also be used by subsequent status assessment modules.

[0121] In summary, this embodiment incorporates multiple technical elements, including the construction of a sliding window multidimensional scoring curve, trend synchronization analysis, a fuzzy logic system to adjust deviation thresholds, and anomaly label output structures, to form a complete, dynamic, and context-aware anomaly identification system. Compared to traditional static threshold determination techniques, this system offers greater adaptability, lower false alarm rates, and greater interpretability. It also achieves an inverse chain from "anomaly phenomena" to "anomaly sources" through key feature component mapping.

[0122] Furthermore, performing weighted statistics and correlation calculation operations on the multidimensional feature vector to generate a comprehensive status indicator includes:

[0123] Extracting single physical interpretation features corresponding to structural strength change, thermal imbalance, identification label continuity, and dynamic stability factor from the multidimensional feature vector, respectively, and constructing a modal physical dependency graph, wherein the modal physical dependency graph is a directed graph structure, wherein nodes correspond to characteristic channels of different modes, and edge weights represent the causal contribution of the modal feature to the structural strength change in historical data, and the edge weights are calculated based on the co-shift frequency and time-delay coupling probability within a sliding window;

[0124] The modal participation factor is calculated based on the sum of the input edge weights of each node in the modal physical dependency graph and the absolute deviation amplitude of each physical interpretation feature in the current cycle. The modal participation factor is used to reflect the degree of participation of each mode in the overall structural state representation in the current task cycle and serves as the basis for adjusting the statistical weight of each dimensional feature.

[0125] When performing weighted statistical operations, the modal participation factor is introduced into the weight configuration of structural strength change, thermal imbalance, identification tag continuity, and dynamic stability factor. The contribution ratio of each dimensional feature in the weighted average and variance estimation is adjusted according to the distribution of the modal participation factor. The weighted mean, weighted standard deviation, and local fluctuation index of each feature dimension are calculated as the statistical expression of this period.

[0126] In the case where the dynamic stability factor and the degree of thermal imbalance have a nonlinear offsetting effect under specified operating conditions, the mutual information residual value between the two is calculated and used to correct the weight factor of the thermal imbalance dimension, forming a mutual information residual calibration mechanism to compensate for the fusion deviation caused by the mismatch between vibration and thermal information.

[0127] The weighted statistical indicators of each dimension after weight correction are linearly combined to form a set of normalized comprehensive state indicator vectors. The structural risk distribution density is calculated based on the second-order adjacency spectrum analysis of the modal physical dependency graph, and a set of structural integrity feature labels are generated. The structural integrity feature labels reflect whether the current state is in a high-coupling aggregation area in the modal network structure, which is used to indicate the evolution trend of potential fragile structures.

[0128] The purpose of this embodiment is to convert the multidimensional feature vectors extracted in the previous step to represent the operating status of the target material into a comprehensive status indicator that can be used for subsequent status identification and assessment. The multidimensional feature vectors include four characteristic dimensions: structural strength change, thermal imbalance, identification tag continuity, and dynamic stability factor. Each dimension is derived from a different subset of modal fusion data and has clear physical semantics and time indexing. Due to the coupling, redundancy, offset, and nonlinear correlation between these features, they cannot be directly averaged. Instead, weighted statistics and correlation modeling must be used to extract their state expression capabilities within the current cycle.

[0129] The first step is to extract single physical interpretation features from the multidimensional feature vectors and construct a modal physical dependency graph based on them. A single physical interpretation feature is a feature dimension that can stably represent a specific physical phenomenon after the raw data is normalized and fused. For example, edge gradient fluctuations in image data can reflect changes in structural strength, temperature distribution offsets in infrared thermal imaging can reflect the degree of thermal imbalance, continuity breaks in RFID readings can indicate tag continuity, and the stability of the main vibration frequency reflects the dynamic stability factor. Each extracted feature type is represented as a node in the graph, and all nodes together constitute the modal physical dependency graph.

[0130] The modal physical dependency graph adopts a directed graph structure, whose nodes are the above-mentioned physical interpretation features and edges are causal contribution paths. Specifically, each directed edge indicates that in the historical operation cycle, a certain modal feature has the ability to predict the state change of another modal feature or a physical driving effect. For example, if it is observed in multiple sets of historical data that the rise of thermal imbalance often precedes the fluctuation of structural strength change, a directed edge pointing from "thermal imbalance degree" to "structural strength change" can be set. The calculation of the edge weight is based on the frequency of cooperative offsets and the time-lag coupling probability in the sliding window, that is, in what proportion of the historical time window, the offset directions of the two feature dimensions are consistent, and the offset order appears within the specified time lag range. These statistics can be performed based on the sliding window length, sampling frequency and maximum time lag parameter settings.

[0131] After constructing the modal physical dependency graph, the second step is to calculate the modal participation factor based on the sum of the incoming edge weights of each node in the graph and the absolute deviation amplitude of the corresponding physical interpretation feature of the node in the current cycle. The sum of the incoming edge weights represents the response sensitivity of the feature in the modal coupling network, while the absolute deviation amplitude represents the activity of the feature in the current state. The combination of the two constitutes the "modal participation factor", which is used to reflect the actual contribution of each mode to the overall state expression in the current cycle. It can be represented by a normalized product model, that is, the participation factor of each mode is the total incoming edge weight multiplied by its current deviation value and then normalized to ensure that the sum of all modal factors is 1.

[0132] Next, in the weighted statistics stage, the above-mentioned modal participation factors need to be introduced into the weight configuration of the four feature dimensions to make the weighted statistical operation "task cycle adaptable". For each dimension of the feature, the system adjusts its contribution in the calculation of the weighted mean, weighted standard deviation and local fluctuation index according to the participation factor corresponding to its source modality. For example, the weighted mean of the structural strength change is composed of multiple image channels, and the distribution of each channel can be reweighted according to the participation factor. The local fluctuation index reflects the amplitude of the change of the dimension feature within the sliding window, which can be calculated through indicators such as local variance and peak interval. Finally, each dimension output includes the weighted mean, weighted standard deviation and local fluctuation, forming the basic statistical expression within the current cycle.

[0133] It is worth noting that there is a nonlinear offsetting effect between the dynamic stability factor and the degree of thermal imbalance in certain operating conditions of power materials. For example, when the transformer is subjected to high load, the temperature rise increases, causing the vibration amplitude to increase, which may cause the two to have opposite statistical directions but actually belong to the same state evolution. In order to avoid such fusion deviations, the mutual information residual calibration mechanism is introduced in the present invention. Specifically, the mutual information between the dynamic stability factor and the degree of thermal imbalance is calculated, and the mutual information is subtracted from the historical average mutual information to obtain the residual signal. If the residual is greater than the upper limit of a certain confidence interval, it means that a new strong coupling has appeared between the two, and the weight of the thermal imbalance needs to be increased; conversely, when the residual is lower than the lower limit, its influence is reduced. This correction step makes the weighted statistical results stable even when there is a nonlinear relationship between thermal-vibration information.

[0134] After weighted statistics are completed for all feature dimensions, they are linearly combined into a set of comprehensive state indicator vectors. The linear combination coefficients are weighted based on the modal participation factor and the mutual information adjustment factor, ensuring that the combined indicators are both statistically representative and maintain a balanced fusion of all modalities. The resulting comprehensive state indicator vector forms the core of the multidimensional state representation for this cycle and serves as direct input for subsequent anomaly identification.

[0135] Furthermore, to provide a more structurally meaningful interpretation of risk, a second-order adjacency spectrum analysis is performed on the modal physics dependency graph to generate a set of structural integrity signatures. This spectrum analysis employs the second-order eigenvalue decomposition of the graph's Laplace matrix to identify substructure units with high coupling and weak redundancy, thereby determining whether the current modal structure has entered a brittle coupling regime. The label output is presented as a Boolean vector or fuzzy value to assist in assessing the structural integrity level.

[0136] Step S105: Compare the comprehensive status indicator with the historical performance benchmark, identify abnormal status through the set deviation threshold, and output abnormal labels and corresponding key feature components.

[0137] In step S105, the comprehensive status indicator generated in step S104 is compared with historical performance benchmarks. Abnormal conditions are identified using a set deviation threshold, and an abnormality label and corresponding key characteristic components are output. The core of this step is to establish a quantifiable, reproducible, and applicable comparison and judgment mechanism for different types of power materials. This allows the identification of abnormal status characteristics in the current operating environment and provides clearly targeted abnormality diagnostic information.

[0138] Comprehensive status indicators typically encompass multiple dimensions, each representing a specific aspect of the material's operational status, such as structural stability, thermal distribution balance, RFID tag response stability, and dynamic response consistency. Before conducting a comparison, a historical performance benchmark must be established. This benchmark should be statistically processed from historical data of the target material under normal operating conditions. This benchmark can take the form of a mean, distribution interval, confidence interval, or empirical model. For different types or batches of materials, corresponding historical performance benchmarks should be established to account for manufacturing variations and differences in usage environments.

[0139] The comparison operation should calculate the deviation between the current value and the historical performance benchmark for each dimension of the comprehensive status indicator. The deviation can be expressed as a simple difference, a multiple of the standard deviation, or a relative rate of change. For example, if the current value of the thermal imbalance is 0.72, and its historical benchmark mean is 0.48 with a standard deviation of 0.06, then the value has deviated from the mean by more than four standard deviations and an abnormal flag should be triggered immediately. The setting of the deviation threshold can be implemented using a fixed threshold, a dynamic threshold, or a multi-level threshold based on risk level, depending on the sensitivity and fault tolerance required by the system.

[0140] Once the deviation of one or more status indicators exceeds the corresponding threshold, the target material is determined to be in an abnormal state. At this point, the system generates an anomaly label to identify the dimension and nature of the anomaly, such as "excessive thermal imbalance," "abnormal structural fluctuation," or "abnormal vibration mode change." Each anomaly label should be accompanied by an anomaly score to reflect the severity of the anomaly, facilitating subsequent sorting and response priority determination.

[0141] In addition to outputting anomaly labels, the corresponding key feature components must also be output. This involves extracting the primary original features from the multidimensional feature vector that cause deviations in the comprehensive status indicators, providing an explanatory basis. For example, a high structural strength change score can be traced back to a sudden change in edge continuity in the image data, an increase in crack growth rate, or the appearance of new high-energy resonant components in the vibration frequency. The output of these key feature components facilitates traceable, localizable, and interpretable anomaly identification, providing direct support for subsequent manual review, automated diagnosis, or maintenance decisions.

[0142] In practical implementation, the comparison algorithm should be capable of both online and batch processing, adapting to different scenarios such as high-frequency monitoring and periodic analysis. The system should also support an adaptive historical benchmark update mechanism, automatically optimizing the benchmark model during the long-term stable operation of the detection equipment to improve recognition accuracy and robustness.

[0143] In summary, step S105 identifies and marks anomalies in the current operating status of power materials by comparing the comprehensive status indicators with historical performance benchmarks and combining them with the set deviation threshold, while accurately locating the key characteristic components that cause the anomalies.

[0144] Step S106: Perform rule reasoning and level mapping based on the abnormal labels and comprehensive status indicators, and output the status assessment results and processing suggestions of the target materials.

[0145] In step S106, rule reasoning and level mapping are performed based on the anomaly labels and comprehensive status indicators identified in the previous step, ultimately outputting the target material's status assessment results and treatment recommendations. This step is the final decision-making link in the entire intelligent detection process for power materials. It transforms complex calculation results into practical diagnostic information and operational response recommendations. Therefore, it must have a clear logical path, a reproducible reasoning mechanism, and a set of response rules covering different status levels.

[0146] First, the anomaly label is the identification information used to characterize the anomaly type and location, identified in step S105 based on the deviation threshold. Each anomaly label should correspond to an abnormal deviation result from at least one comprehensive status indicator and be associated with its key characteristic components. The core of rule-based reasoning lies in determining the target material's status level based on the type, quantity, combination, and weight of these anomaly labels, combined with the various scoring dimensions of the current comprehensive status indicator, and judged according to pre-defined logical rules.

[0147] Inference rules should be defined in an enumerable form, perhaps using an "if-then" structure, a threshold tree structure, or a decision table based on expert experience. For example, if the "structural strength change score is > 0.8" and the "dynamic stability factor score is < 0.4," the system is judged to be "high risk for the mechanical structure." Alternatively, if the "thermal imbalance score is between 0.6 and 0.75" and the "historical trend is stable," the system is judged to be "moderately thermally unbalanced, monitoring recommended." To ensure scalability and configurability, it is recommended that rule sets be stored as configuration files or nested logic modules to facilitate adjustment and optimization for different types of power resources during system deployment.

[0148] Level mapping is the process of classifying and assigning grades to inference results. The status of target materials is typically divided into multiple levels, such as "normal," "mildly abnormal," "moderately abnormal," "severely abnormal," or "level one," "level two," and "level three." Each level should have clearly defined boundaries, a quantifiable indicator range, and a clear risk profile. For example, a "normal" state should correspond to all comprehensive status indicators being within a safe range and no abnormal labels being triggered; a "severely abnormal" state should include significant deviations in scores across multiple dimensions, the appearance of high-risk labels, and the presence of failure precursor characteristics in historical data.

[0149] After determining the status level, it is necessary to further output corresponding treatment suggestions. Treatment suggestions should be based on the status level determination results, and combined with the type of material, operating environment and historical behavior data, to provide specific and actionable measures. These suggestions may include "continue monitoring", "arrange manual re-inspection", "planned maintenance", "immediate decommissioning and replacement", etc. Different levels of suggestions should have obvious differences in timeliness, execution cost and risk control. In addition, for scenarios that can be adjusted through the system itself, such as adjustable loads, adjusting the cooling system or updating radio frequency tags, corresponding control instructions can be automatically generated and executed as part of the feedback loop.

[0150] Furthermore, the rule reasoning and level mapping are performed based on the abnormal labels and comprehensive status indicators to output the status assessment results and processing suggestions of the target materials, including:

[0151] Mapping the abnormal labels and corresponding key feature components back to the corresponding image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data in the standardized fusion data, marking the abnormal activation areas under the corresponding time index and spatial area, and constructing an abnormal structure tensor with modality, spatial position, and acquisition period as index dimensions. The abnormal structure tensor is used to represent the propagation path and concentrated distribution area of ​​the abnormal labels in the multimodal space-time domain;

[0152] Based on the abnormal structure tensor, the local change coordination degree of each modal data in the same acquisition cycle is statistically analyzed, the dominant modal channel corresponding to the abnormal label is identified, and the dominant modal channel is used as a benchmark to calculate the contribution of each modal channel to the comprehensive state index in the current acquisition cycle, and generate an abnormal impact factor vector. The abnormal impact factor vector is used to guide the priority weight configuration of rule reasoning;

[0153] The abnormal label, the comprehensive status index, and the abnormal impact factor vector are input into the rule reasoning module. The rule reasoning module activates the reasoning path that matches the current abnormal label structure based on the preset inference rule set using a recursive graph matching method, and calculates the status grade score of the current target material by combining the change direction and magnitude of each scoring dimension in the comprehensive status index.

[0154] Output a status assessment result including a status grade label, a dominant modal identifier, an impact area code, and a response time limit recommendation based on the status grade score and the spatial index position corresponding to the dominant modal channel in the abnormal impact factor vector. The status assessment result is used as a quantitative output of the operating status of the target material.

[0155] Based on the combination of the dominant modality identifier and the abnormality label in the state assessment result, natural language diagnostic suggestions are generated to guide manual decision-making. The diagnostic suggestions include: priority observation modality sequence, recommended re-inspection parameters for abnormal areas, recommended adjustment strategies for sampling periods, and state change trend prediction instructions, which are used to assist in the subsequent generation of processing suggestions.

[0156] In this embodiment, the focus is on the final rule reasoning and level mapping process, that is, how to further form status assessment results and processing suggestions with reasoning logic, diagnostic value and operational guidance significance based on identifying abnormal labels and obtaining comprehensive status indicators.

[0157] Before entering this step, the system has already extracted and weighted the multidimensional feature vectors, generating comprehensive status indicators based on them. The anomaly identification module has also generated anomaly labels and key feature components. The first operation described here is to map the anomaly labels and their corresponding key feature components back to the original standardized fusion data. Specifically, this involves reading the associated feature dimensions and their anomaly activation time index from the anomaly labels. Based on the spatial distribution information recorded in the key feature components, the system then reverse-localizes the data to the corresponding raw data segments in the image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data. This mapping process not only preserves the modal origin and temporal order, but also ensures the continuity, interpretability, and localizability of the anomaly events in the multimodal space.

[0158] After the mapping is completed, the system constructs an anomaly structure tensor with the modal type, spatial position coordinates, and acquisition cycle index as the three-dimensional coordinate axes. This tensor is constructed as follows: Under each modal channel, according to the spatial range covered by the abnormal label, the two-dimensional or one-dimensional raw data (such as image frames, heat map frames, label reading intensity or vibration waveform fragments) of the corresponding time point are extracted, stored uniformly in the spatial and temporal dimensions, and a complete "anomaly propagation landscape" is formed in the tensor structure. This tensor can adopt a sparse tensor storage mode to compress the storage load while retaining the complete information of the abnormal high-density area. Its function is to accurately characterize the concentration, diffusion, and coupling pattern of abnormal labels in space, time, and mode.

[0159] After the abnormal structure tensor is constructed, the system will perform modal channel co-variation analysis on it, that is, analyze whether there is synchronization between the abnormal activity levels of each modality within the same acquisition cycle. For example, if the image data and infrared thermal imaging data show significant changes in a certain area at the same time, while the radio frequency identification data does not change significantly, it can be inferred that the former two are dominant modes and the latter is a weak channel. The specific analysis method can be calculated using quantitative indicators such as local variance, principal component contribution, and mutual information value under time alignment, and finally generate an abnormal impact factor vector. This vector contains the abnormal impact coefficients of all modal channels in the current cycle. Each dimension represents the causal strength of a modality in the formation of the current anomaly, which can be used to configure the priority activation weights of each rule branch in the subsequent reasoning module.

[0160] Entering the rule reasoning stage, the system inputs the anomaly label, comprehensive status indicator, and anomaly impact factor vector into the preset rule reasoning module. This module is structured as a multi-rule chain graph, the core of which is a number of inference paths in the rule set. Each path is activated based on three conditions: "anomaly label combination" + "state indicator deviation direction" + "influence factor distribution pattern." For example, if the system identifies that "thermal imbalance anomaly" and "dynamic stability score drop sharply" occur simultaneously, and the infrared thermal imaging channel value in the anomaly impact factor vector is higher than 0.6, path 1 can be activated; if "structural strength change abnormality" and the radio frequency tag are intermittent, path 2 will be activated. This module uses a recursive graph matching method to search for the structural subgraph closest to the current combination in the internal inference graph, and calculates the status level score based on parameters such as its historical trigger frequency, average evolution time, and post-trigger risk level distribution.

[0161] The status rating is calculated using a multi-dimensional weighted structure, including a base rating value, a confidence coefficient for the inference path, a cluster density index for the abnormal structure tensor, and a trend distribution factor for the comprehensive status index. The final output is a rating label within a predefined rating domain. The rating label not only indicates the severity of the current status but also serves as an indicator of the evolutionary stage of the device status lifecycle.

[0162] Subsequently, a structured state assessment result is generated based on the state grade score and the spatial position index corresponding to the dominant modal channel in the anomaly impact factor vector. This assessment result includes at least the following: first, a state grade label (such as "medium risk" or "warning required"); second, a dominant modal identifier (such as "thermal imbalance dominant" or "structural change dominant"); third, an impact area code, which is a unique code for the spatial range of the anomaly, facilitating map positioning or inspection system calls; and fourth, a response time limit recommendation, which indicates the latest time window for taking response action under the state grade, such as "re-inspection recommended within 24 hours" or "manual inspection should be arranged immediately."

[0163] To further support human-assisted decision-making, the system generates diagnostic recommendations in natural language based on the combination of dominant modal identifiers and anomaly labels in the state assessment results. These recommendations are populated with embedded natural language templates. The templates combine comprehensive information such as the activation intensity distribution of the anomaly structural tensor, the dimensional features contained in the anomaly labels, and the behavioral patterns of the dominant modal in similar historical states to form semantically complete and actionable text. For example, a diagnostic recommendation might include the following: "A thermal imbalance anomaly, primarily caused by infrared thermal imaging, has been detected at the busbar insulation connection. It is recommended to increase the thermal imaging resolution to 0.2°C / pixel, adjust the re-inspection frequency to every two hours, and simultaneously retrieve the synchronous vibration response signal to rule out structural linkage anomalies." This text includes the order of priority observed modal observations, recommended re-inspection parameters for the anomaly area, sampling cycle adjustment strategies, and a description of the predicted state change trend, truly building a "language bridge" between algorithmic identification and engineering action.

[0164] A second embodiment of the application provides an electronic device, comprising:

[0165] processor;

[0166] The memory is used to store a program. When the program is read and executed by the processor, it executes an intelligent detection method for electric power materials based on multimodal data fusion provided in the first embodiment of the present application.

[0167] The third embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the program executes an intelligent detection method for electric power materials based on multimodal data fusion provided in the first embodiment of the present application.

[0168] Although the present application is disclosed as above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. An intelligent detection method for power materials based on multimodal data fusion, characterized in that: include: During the collection period set during the operation or circulation of power materials, image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data of the target materials are acquired, and aligned based on a unified time index to construct a time-ordered structured multimodal dataset. Normalizing and removing outliers on the structured multimodal data set, retaining image edge features, temperature distribution information, radio frequency unique identifiers, vibration spectrum characteristics, and environmental fluctuation trends, and generating standardized fusion data; Based on a preset multimodal feature mapping rule, a multidimensional feature vector representing the operating status of the target material is extracted from the standardized fusion data, the multidimensional feature vector including structural strength change, thermal imbalance degree, identification tag continuity, and dynamic stability factor; Performing weighted statistics and correlation calculation operations on the multidimensional feature vector to generate a comprehensive status index for measuring the current operational integrity and environmental adaptability of the target material; Compare the comprehensive status indicator with the historical performance benchmark, identify abnormal status through the set deviation threshold, and output the abnormal label and corresponding key feature components; Based on the abnormal labels and comprehensive status indicators, rule reasoning and level mapping are performed to output the status assessment results and processing suggestions of the target materials.

2. The intelligent detection method for electric power materials based on multimodal data fusion according to claim 1 is characterized in that: The normalizing and outlier removal of the structured multimodal dataset includes: In the structured multimodal dataset, a consistency check of cross-abnormal regions between modalities is performed on the image data and the infrared thermal imaging data. By constructing a pixel-level coordinate mapping relationship, it is determined whether the visible damage area appearing in the image and the local temperature rise area existing in the infrared thermal imaging spatially overlap. If the overlap is lower than a preset threshold, the data point is marked as an inconsistent outlier and is removed. For the vibration response data, the boundary drift rate is calculated based on a statistical model of the rate of change of the main peak position of the spectrum in continuous time segments, and a direction-sensitive filtering strategy is used to identify the mutation points. If the mutation point shows a unilateral deviation trend in the adjacent time window, it is marked as a dynamic outlier and removed; Based on the preset physical distribution prior information of the target material, a modal normalization scale adjustment factor is constructed for the image data, infrared thermal imaging data, and vibration response data, respectively, so that the unit change values ​​under different modes maintain scale consistency under the same physical meaning. The adjustment factor is used to correct the initial normalization results and generate standardized fusion data under a unified physical scale; The standardized fusion data after removing outliers and completing scale correction is provided as input to the step of extracting a multidimensional feature vector for representing the operating status of the target material from the standardized fusion data based on the preset multimodal feature mapping rule.

3. The intelligent detection method for electric power materials based on multimodal data fusion according to claim 1 is characterized in that: The step of extracting a multidimensional feature vector for representing the operating status of the target material from the standardized fusion data based on a preset multimodal feature mapping rule includes: Based on the historical collaborative response relationship between the modal data in the standardized fusion data, a weighted directed graph model is constructed to characterize the inter-modal causal structure, wherein each node of the weighted directed graph corresponds to a feature channel of the image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data, and the edge weight represents the degree of causal coupling of inter-modal response changes in historical operating scenarios; Calculating the modal influence distribution based on the total weight of the incoming edges of each node in the weighted directed graph, and combining the environmental parameter data of the current collection period of the electric power material with a context-aware mechanism to dynamically select a modal subset with effective information contribution in the current state as a feature extraction input; Redundancy evaluation is performed on the original features in the selected modality subset. If there are features with similar physical expression dimensions between the modalities, such as temperature distribution features in thermal images and regional gradient features in image data, a feature folding path is constructed based on the physical dimension similarity, and weighted fusion coding is used to merge them into a unified sub-feature coding vector; The sub-feature encoding vectors, together with the modal identifiers and time indexes on which they depend, are used to construct a cross-modal fusion feature set, and feature dimensions with weak correlation or high volatility are dynamically reduced in dimension and orthogonality enhanced to form a multi-dimensional feature candidate set; From the multidimensional feature candidate set, an optimal subset is selected according to the matching degree between the structural category of the target material and the historical operation status label as the multidimensional feature vector for representing the operation status of the target material.

4. The intelligent detection method for electric power materials based on multimodal data fusion according to claim 1 is characterized in that: The method of comparing the comprehensive status indicator with the historical performance benchmark, identifying abnormal status through a set deviation threshold, and outputting abnormal labels and corresponding key feature components includes: Based on the sliding window mechanism, scoring curves for comprehensive status indicators within the continuous acquisition cycle are constructed, generating scoring curves for structural strength change, thermal imbalance, identification label continuity, and dynamic stability factor, forming a multidimensional scoring curve group for trend analysis. Perform a trend synchronization analysis on the multidimensional scoring curve group to calculate the consistency of the change direction and the overlap of the fluctuation time series between the scoring curves. When any two or more scoring curves show trend deviation at the same time in the same time period, and their synchronization index exceeds the set criterion, they are determined to be in a linkage anomaly candidate state, and the corresponding time period is marked as a key judgment window; Within the key discrimination window, the change rate, stability coefficient, and difference from the historical performance benchmark of each dimensional comprehensive status indicator are input into the fuzzy logic rule system for inference, dynamically generating a deviation threshold for each dimensional status indicator. The fuzzy rule system is constructed based on a preset "change rate-credible interval-tolerance limit" mapping relationship to ensure that the deviation threshold is context-adaptive. According to the dynamically generated deviation threshold, the state indicator dimension that deviates within the key discrimination window is identified, and the abnormal label and the corresponding key feature component are output.

5. The intelligent detection method for electric power materials based on multimodal data fusion according to claim 1 is characterized in that: The performing weighted statistics and correlation calculation operations on the multidimensional feature vector to generate a comprehensive status indicator includes: Extracting single physical interpretation features corresponding to structural strength change, thermal imbalance, identification label continuity, and dynamic stability factor from the multidimensional feature vector, respectively, and constructing a modal physical dependency graph, wherein the modal physical dependency graph is a directed graph structure, wherein nodes correspond to characteristic channels of different modes, and edge weights represent the causal contribution of the modal feature to the structural strength change in historical data, and the edge weights are calculated based on the co-shift frequency and time-delay coupling probability within a sliding window; The modal participation factor is calculated based on the sum of the input edge weights of each node in the modal physical dependency graph and the absolute deviation amplitude of each physical interpretation feature in the current cycle. The modal participation factor is used to reflect the degree of participation of each mode in the overall structural state representation in the current task cycle and serves as the basis for adjusting the statistical weight of each dimensional feature. When performing weighted statistical operations, the modal participation factor is introduced into the weight configuration of structural strength change, thermal imbalance, identification tag continuity, and dynamic stability factor. The contribution ratio of each dimensional feature in the weighted average and variance estimation is adjusted according to the distribution of the modal participation factor. The weighted mean, weighted standard deviation, and local fluctuation index of each feature dimension are calculated as the statistical expression of this period. In the case where the dynamic stability factor and the degree of thermal imbalance have a nonlinear offsetting effect under specified operating conditions, the mutual information residual value between the two is calculated and used to correct the weight factor of the thermal imbalance dimension, forming a mutual information residual calibration mechanism to compensate for the fusion deviation caused by the mismatch between vibration and thermal information. The weighted statistical indicators of each dimension after weight correction are linearly combined to form a set of normalized comprehensive state indicator vectors. The structural risk distribution density is calculated based on the second-order adjacency spectrum analysis of the modal physical dependency graph, and a set of structural integrity feature labels are generated. The structural integrity feature labels reflect whether the current state is in a high-coupling aggregation area in the modal network structure, which is used to indicate the evolution trend of potential fragile structures.

6. The intelligent detection method for electric power materials based on multimodal data fusion according to claim 1 is characterized in that: The rule reasoning and level mapping are performed based on the abnormal labels and comprehensive status indicators to output the status assessment results and processing suggestions of the target materials, including: Mapping the abnormal labels and corresponding key feature components back to the corresponding image data, infrared thermal imaging data, radio frequency identification data, vibration response data, and environmental parameter data in the standardized fusion data, marking the abnormal activation areas under the corresponding time index and spatial area, and constructing an abnormal structure tensor with modality, spatial position, and acquisition period as index dimensions. The abnormal structure tensor is used to represent the propagation path and concentrated distribution area of ​​the abnormal labels in the multimodal space-time domain; Based on the abnormal structure tensor, the local change coordination degree of each modal data in the same acquisition cycle is statistically analyzed, the dominant modal channel corresponding to the abnormal label is identified, and the dominant modal channel is used as a benchmark to calculate the contribution of each modal channel to the comprehensive state index in the current acquisition cycle, and generate an abnormal impact factor vector. The abnormal impact factor vector is used to guide the priority weight configuration of rule reasoning; The abnormal label, the comprehensive status index, and the abnormal impact factor vector are input into the rule reasoning module. The rule reasoning module activates the reasoning path that matches the current abnormal label structure based on the preset inference rule set using a recursive graph matching method, and calculates the status grade score of the current target material by combining the change direction and magnitude of each scoring dimension in the comprehensive status index. Output a status assessment result including a status grade label, a dominant modal identifier, an impact area code, and a response time limit recommendation based on the status grade score and the spatial index position corresponding to the dominant modal channel in the abnormal impact factor vector. The status assessment result is used as a quantitative output of the operating status of the target material. Based on the combination of the dominant modality identifier and the abnormality label in the state assessment result, natural language diagnostic suggestions are generated to guide manual decision-making. The diagnostic suggestions include: priority observation modality sequence, recommended re-inspection parameters for abnormal areas, recommended adjustment strategies for sampling periods, and state change trend prediction instructions, which are used to assist in the subsequent generation of processing suggestions.

Images