Power equipment fault detection method and system based on equipment multi-modal model confidence coefficient and multi-scale cooperation

Abstract

The invention relates to the technical field of power operation and maintenance, in particular to a power equipment fault detection method and system based on equipment multi-modal model confidence and multi-scale collaboration.The power equipment fault detection method based on equipment multi-modal model confidence and multi-scale collaboration.The power equipment fault detection method is characterized by comprising the steps that operation index data of equipment to be detected are acquired; performing index side primary detection based on the operation index of the to-be-detected equipment, and calculating the index side fault probability of the to-be-detected equipment; according to the method, the appearance image of the to-be-detected equipment is obtained, primary detection of an index side and an image side is processed in parallel, and a detection channel is dynamically switched on the basis of confidence threshold evaluation, so that the limitation of traditional single-mode processing is solved, and cooperative processing and confidence dynamic management of multi-mode data are realized; and the accuracy and the real-time performance of fault detection of the power equipment are remarkably improved.

Description

Technical Field

[0001] This application relates to the technical field of power operation and maintenance, and in particular to a method and system for power equipment fault detection based on equipment multimodal model confidence and multi-scale collaboration. Background Technology

[0002] The current field of power equipment fault detection faces multiple technical bottlenecks. Traditional detection systems generally adopt a single-modal data processing architecture, relying solely on dissolved gas analysis data or equipment appearance images for isolated judgments, resulting in a lack of effective interaction and collaboration mechanisms between different modal information. When equipment is under complex operating conditions or encounters boundary samples, the detection results of a single modality often have significant limitations, failing to eliminate noise interference through cross-modal information complementarity, and are prone to missed detections and false detections. More significantly, existing systems lack the ability to dynamically manage the uncertainty of model predictions. When the primary detection model outputs low-confidence results, the system still mechanically executes subsequent processes, failing to automatically switch to a deep inference path based on the confidence threshold, causing unreliable predictions to directly enter the decision-making stage, greatly increasing the risk of misdiagnosis. At the same time, industry knowledge and engineering rules are not deeply integrated into the detection process, making it difficult for data-driven models to call upon professional semantic information such as equipment fault knowledge graphs and operation and maintenance standards, resulting in a lack of interpretability in diagnostic conclusions; while large language models have the advantage of semantic reasoning, they cannot directly parse sensor values ​​and engineering images, causing their output results to deviate from engineering reality without combining physical rules and historical cases. Furthermore, the multi-source data is fragmented in its application, with key information such as operational indicators, inspection images, and historical maintenance records being processed separately. The lack of a unified framework for multi-scale feature fusion and cross-modal consistency verification makes it difficult for the system to form stable and reliable comprehensive judgments under complex and abnormal scenarios. These deficiencies severely restrict the accuracy and real-time performance of power equipment fault early warning. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this application provides a power equipment fault detection method and system based on equipment multimodal model confidence and multi-scale collaboration. By parallel processing of primary detection on the indicator side and the image side, preliminary fusion of multimodal information is achieved. In the example, this parallel processing mechanism is manifested in the simultaneous analysis of oil temperature data and transformer appearance images, rather than relying on only one information source, thus overcoming the problems of missed detections and false detections that traditional methods are prone to under complex operating conditions.

[0004] The above-mentioned objective of this application is achieved through the following technical solution: A power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration is characterized by acquiring the operating index data of the equipment under inspection; Based on the operating indicators of the equipment under test, perform preliminary testing on the indicator side and calculate the failure probability of the indicator side of the equipment under test. Acquire an image of the appearance of the device to be inspected; Based on the appearance image of the device under inspection, perform preliminary image-side detection and calculate the overall confidence level of the device under inspection based on the image side. Based on a preset confidence threshold, the failure probability on the indicator side and the overall confidence level on the image side are evaluated. When the failure probability on the indicator side and the comprehensive confidence level on the image side of the device under inspection are both higher than the threshold, the initial detection result is output to achieve efficient and rapid discrimination. If the confidence level on either side is lower than the threshold, the sample is marked as an uncertain sample and switched to the deep inference channel for deep detection. The primary detection involves parallel processing of both the indicator side and the image side.

[0005] This application also provides a transformer detection method, based on the aforementioned fault detection method, including: The absolute and relative gas production rates of the critical fault gas are calculated to characterize the magnitude and relative trend of gas generation variation, providing another important time-series feature besides trend residual characteristics: R_abs(t)=[C(t)−C(t−Δt)] / Δt R_rel(t)=[C(t)−C(t−Δt)] / [C(t−Δt)·Δt]×100% When C(t−Δt) is less than a preset threshold, the relative gas production rate can be smoothed or a preset substitution strategy can be adopted to avoid numerical instability under low concentration conditions. An ARIMA time series prediction model is constructed based on historical normal operation data to generate a baseline prediction curve for the corresponding gas concentration. The deviation between the actual monitored value and the predicted value is calculated, and the deviation is defined as the trend residual feature using the L2 norm of the residual vector. TRF=||C_real(t)−C_pred(t)||2 The trend residual feature is used to quantify the degree of deviation of the current operating state from historical statistical patterns, serving as one of the statistical features reflecting the intensity of potential abnormal changes, and is used in conjunction with subsequent model inference results; The preprocessed original gas concentration value, IEC three ratio encoding result, absolute and relative gas production rate characteristics, and trend residual characteristics (TRF) are spliced ​​together in a predetermined order to construct a structured high-dimensional feature vector, which serves as the input to the primary fault identification model. An improved CatBoost model is used as the primary detector on the indicator side. During the model training phase, a cost-sensitive loss function is introduced to give higher weights to samples with larger trend residual features (TRF) to enhance the model's sensitivity to abnormally changing samples. During the model inference phase, the probability distribution vector P_ind of the candidate fault type is output, and the relative contribution of each input feature is calculated to characterize the importance of different features in the current discrimination result, thereby providing a reference for subsequent comprehensive inference.

[0006] This application also provides a system for the fault detection method as described above, including: The indicator-side data acquisition module is used to collect the operating indicator data of the equipment under inspection; The indicator-side primary detection module is used to perform primary detection on the equipment under test from the indicator side based on the operational indicator data collected by the indicator-side data acquisition module. Image-side data acquisition module, used to acquire appearance image data of the device under inspection; The image-side primary inspection module is used to perform primary inspection of the device under inspection from the image side based on the appearance image data acquired by the image-side data acquisition module. The confidence assessment module is used to evaluate the failure probability on the indicator side and the overall confidence level on the image side. The large model inference module is used for deep inference to perform in-depth testing of the device under test.

[0007] In summary, this application has the following beneficial technical effects: By parallel processing of primary detection on the indicator side and image side, and dynamically switching detection channels based on confidence threshold evaluation, the limitations of traditional single-modal processing are overcome. This enables collaborative processing of multimodal data and dynamic management of confidence, significantly improving the accuracy and real-time performance of power equipment fault detection. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the fault detection method and system of this application. Detailed Implementation

[0009] The present application will be further described in detail below with reference to the accompanying drawings.

[0010] In power equipment fault detection systems, single-modal analysis and independent model operation result in a lack of information interaction and collaborative reasoning capabilities between different information sources. This prevents the system from achieving cross-modal information complementarity under complex operating conditions, boundary samples, and noisy data, increasing the possibility of missed and false detections. Furthermore, the lack of a predictive uncertainty management mechanism leads the system to rely on unreliable results for subsequent judgments even when model prediction confidence is low. It cannot dynamically adjust the inference path or invoke higher-level knowledge based on confidence levels, increasing the risk of misdiagnosis. In addition, insufficient knowledge utilization prevents the system from effectively integrating professional semantic information such as power equipment industry knowledge, operation and maintenance rules, and expert experience, resulting in a lack of interpretability and consistency in diagnostic results. Fragmented system architecture leads to the dispersed use of multi-source data, making it difficult to form stable and reliable comprehensive judgments.

[0011] For example, during routine inspections of high-voltage transformers, when dissolved gas analysis data shows abnormal gas concentrations but the clarity of the external image is reduced due to rain or fog, traditional systems may only output fault warnings based on gas data, ignoring the uncertainties in image data. Furthermore, the image-side model operates independently and lacks a confidence assessment mechanism, making it impossible for the system to recognize the impact of low-quality images, leading to false alarms. Moreover, because it cannot effectively utilize equipment fault knowledge graphs, the system struggles to verify whether gas anomalies conform to typical fault modes, making the diagnostic results difficult for maintenance personnel to understand and trust.

[0012] If the above problems are not addressed, the system will frequently generate incorrect diagnoses under noisy data and boundary conditions, leading to operational and maintenance errors, increasing the risk of equipment damage and maintenance costs. The uninterpretability of diagnostic results will reduce maintenance personnel's trust in the system, affect decision-making efficiency, and may trigger safety hazards in critical failure scenarios. Therefore, these problems severely restrict the practicality and reliability of the fault detection system.

[0013] In response, this application proposes a power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration. For ease of understanding, some key terms in this embodiment are explained below: Equipment multimodal model confidence refers to the quantitative assessment of the reliability of model outputs from different data modalities (such as operational index data and appearance image data) when detecting faults in power equipment. This confidence level reflects the model's confidence in the current detection results and is an important reference for subsequent decision-making.

[0014] Multi-scale collaboration refers to the collaborative work of models and inference mechanisms of different granularities or complexities during fault detection. For example, an efficient primary detection model can be used for rapid judgment first. For samples that are difficult to determine by the primary model, a deeper and more complex inference channel can be used for detailed analysis to achieve a balance between detection efficiency and accuracy.

[0015] Operational performance data typically refers to various sensor data generated by power equipment during operation, such as transformer oil temperature, winding temperature, load current, voltage, and dissolved gas content. These data reflect the internal operating status and health condition of the equipment.

[0016] Preliminary fault detection on the indicator side refers to the initial fault assessment based on the equipment's operational indicator data, using relatively lightweight models or algorithms. This detection aims to quickly identify obvious anomalies and calculate the probability of faults on the indicator side.

[0017] The probability of failure on the indicator side (P_ind) refers to the likelihood that the model will determine the existence of a certain type of failure in the equipment based on operational indicator data during the initial indicator-side detection. This probability value quantifies the failure risk revealed by the indicator data.

[0018] External images refer to images of the exterior of electrical equipment acquired through visual sensors (such as cameras). These images can capture visual features such as physical damage, abnormal discharge traces, component deformation, and oil leaks on the outside of the equipment, providing intuitive evidence for fault diagnosis.

[0019] Image-based preliminary detection refers to the initial fault assessment based on an image of the device's appearance, using image processing and analysis techniques. This detection aims to quickly identify abnormal visual features in the image and calculate the overall confidence level on the image side.

[0020] Image-side comprehensive confidence (Conf_img) refers to the degree of confidence with which the model judges a device to have a certain fault based on the appearance image during primary image detection. This comprehensive confidence takes into account factors such as image quality, target recognition results, and relevance to professional knowledge.

[0021] The confidence threshold is a pre-set value used to compare the failure probability on the indicator side and the overall confidence level on the image side. When the confidence level of the detection result is higher than the threshold, the result is considered reliable; otherwise, the result is considered to have uncertainty.

[0022] The initial detection result refers to the fault judgment conclusion directly output by the system when both the indicator-side and image-side initial detections meet the preset reliability requirements. This result is typically used for rapid response and handling of obvious faults.

[0023] Uncertain samples refer to samples in the initial detection stage where the failure probability on the indicator side or the overall confidence level on the image side fails to reach the preset threshold simultaneously. For these samples, the system cannot provide a clear initial judgment and requires deeper analysis.

[0024] The deep inference channel refers to a more advanced and complex fault diagnosis process activated by the system when a sample is labeled as an uncertain sample. This channel typically introduces more powerful models (such as large language models) and richer knowledge bases to perform deep fusion and inference of multimodal information.

[0025] Deep detection refers to the detailed analysis of uncertain samples through complex models and algorithms in the deep inference channel in order to obtain more accurate and reliable fault diagnosis results.

[0026] Parallel processing means that the primary detection on the indicator side and the primary detection on the image side can be performed simultaneously without interference, thereby improving the overall detection efficiency.

[0027] This embodiment provides a power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration. This method aims to address the limitations of single-modal analysis, the lack of predictive uncertainty management mechanisms, and insufficient knowledge utilization in existing technologies.

[0028] Specifically, this method first acquires the operational data of the equipment under test. For example, the values ​​of temperature, pressure, and current displayed by the equipment's sensors can be recorded periodically by manual recording. Alternatively, a simple sensor connection method can be used to directly convert the analog signals output by the sensors into digital signals and store them in a local database.

[0029] Subsequently, based on the acquired operating indicators of the equipment under test, preliminary indicator-side testing is performed, and the failure probability (P_ind) of the equipment under test is calculated. For example, a fixed threshold can be set; when an indicator (such as temperature) exceeds this threshold, it is considered an anomaly and assigned a preset failure probability value. Alternatively, the historical average and standard deviation of the indicator can be calculated; when the current value deviates from the average by more than a certain multiple of the standard deviation, it is considered an anomaly, and a failure probability is simply mapped to the degree of deviation.

[0030] Simultaneously, this method acquires images of the device under inspection. For example, it can be done through manual inspection, with maintenance personnel using a handheld camera to take pictures of the device. Alternatively, a fixed camera can be installed near the device to periodically capture images of its appearance and store them on local storage.

[0031] Next, based on the acquired image of the device under inspection, a preliminary image-side detection is performed, and the overall image-side confidence score (Conf_img) of the device under inspection is calculated. For example, simple image processing algorithms, such as edge detection or color histogram analysis, can be used to identify whether there are obvious abnormal regions in the image, and an overall confidence score can be assigned manually or through simple rules based on the size or degree of color change of the abnormal region.

[0032] Furthermore, based on a preset confidence threshold, the fault probability on the indicator side (P_ind) and the overall confidence level on the image side (Conf_img) are evaluated. For example, a fixed value, such as 0.7, can be set as a threshold for judging whether the primary detection result is reliable. When both the fault probability on the indicator side and the overall confidence level on the image side are higher than this fixed threshold, the primary detection result is considered reliable.

[0033] When both the fault probability on the indicator side and the overall confidence level on the image side of the device under test are higher than the threshold, the system outputs a preliminary detection result, achieving efficient and rapid discrimination. For example, if both confidence levels are higher than 0.7, the system directly outputs a simple judgment of "equipment is normal" or "a certain fault exists".

[0034] If the confidence level on either side is below the threshold, the sample is marked as an uncertain sample and switched to the deep inference channel for deep detection. For example, if the failure probability on the indicator side is 0.8 but the overall confidence level on the image side is 0.6, the sample will be marked as uncertain because the confidence level on the image side is below the threshold of 0.7, triggering a more complex analysis process.

[0035] It is worth noting that the indicator side and image side of the primary detection are processed in parallel. This means that after acquiring the running indicator data and appearance image, the primary detection on the indicator side and the primary detection on the image side can be started simultaneously and run independently without waiting for the results of the other side, thereby shortening the overall detection time.

[0036] The following example will provide a more detailed explanation of the above technical solution: Suppose a critical transformer is operating in a power substation at location A. The system needs to detect faults in it. First, the system will acquire the transformer's operating parameters, such as real-time data collected by sensors, including oil temperature, winding temperature, and load current. Simultaneously, the system will also acquire images of the transformer's exterior, such as images of the transformer body, bushings, and radiators taken by an inspection robot.

[0037] After acquiring this multimodal data, primary detection on the indicator side and primary detection on the image side are initiated in parallel. On the indicator side, the system performs preliminary analysis based on operational indicator data. For example, if the oil temperature remains above the historical average, or if abnormal fluctuations occur in the load current, the indicator-side model will calculate an indicator-side fault probability (P_ind). Simultaneously, on the image side, the system performs preliminary analysis based on appearance images. For example, if slight oil contamination is detected on the casing surface or slight deformation of the radiator is detected in the image, the image-side model will calculate an image-side comprehensive confidence score (Conf_img).

[0038] Subsequently, the system evaluates P_ind and Conf_img based on a preset confidence threshold (e.g., 0.7). If both P_ind and Conf_img are higher than 0.7, for example, P_ind is 0.85 and Conf_img is 0.78, the system will directly output a preliminary detection result, such as "The transformer is operating normally, with no obvious faults." In this case, the system achieves efficient and rapid identification of the equipment status, avoiding unnecessary in-depth analysis.

[0039] However, if P_ind is 0.8 but Conf_img is only 0.65, or P_ind is 0.6 and Conf_img is 0.8, meaning the confidence level on either side is below the threshold, the sample will be marked as an uncertain sample. In this case, the system will not directly output the primary detection result, but will automatically switch to the deep inference channel for deep detection. In the deep inference channel, the system will activate more powerful analysis tools, such as large language models, and combine them with a richer professional knowledge base to perform more detailed analysis on uncertain samples. For example, the large language model may comprehensively consider trend residual features in the indicator data, semantic information in the image, and historical fault cases to perform cross-modal consistency analysis and conflict resolution inference, ultimately outputting a corrected fault probability distribution result. Through this multi-scale collaborative approach, the system can effectively handle complex situations that are difficult to determine with primary detection, ensuring the accuracy and reliability of the diagnostic results.

[0040] Based on the above examples, the technical solution of this embodiment differs from the limitations of single-modal analysis and independent model operation in existing technologies. This embodiment achieves preliminary fusion of multimodal information by parallel processing of the initial detection on the indicator side and the image side. In the example, this parallel processing mechanism is manifested in the simultaneous analysis of oil temperature data and transformer appearance images, rather than relying on only one information source, thereby overcoming the problem of missed detections and false detections that are prone to occur in traditional methods under complex operating conditions.

[0041] Furthermore, addressing the lack of a predictive uncertainty management mechanism in existing technologies, this embodiment introduces an evaluation mechanism based on confidence thresholds and a dynamic switching strategy for deep inference channels. In the example, when the confidence level on either side falls below the threshold, the system can mark the sample as uncertain and automatically switch to the deep inference channel, rather than blindly relying on unreliable preliminary results. This mechanism enables the system to dynamically adjust the inference path based on confidence levels, calling upon higher-level knowledge for in-depth analysis, significantly reducing the risk of misdiagnosis and improving the overall reliability of the system.

[0042] This embodiment effectively integrates efficient primary detection with sophisticated deep inference through a multi-scale collaborative detection strategy. In the example, for samples with high confidence, the system can quickly make a judgment, ensuring efficiency; for samples with low confidence, deep inference can be initiated, ensuring accuracy. This hierarchical processing approach enables the system to handle complex, boundary, or noisy data while maintaining detection efficiency, thereby forming a stable and reliable comprehensive judgment and solving the fragmentation problem of existing system architectures.

[0043] In some embodiments of this application, the initial detection of power equipment faults mainly relies on whether the fault probability on the indicator side and the overall confidence level on the image side are lower than a preset threshold to determine whether to enter the deep inference channel. However, in actual operation, there may be some early or slowly changing faults whose trend residual characteristics have shown obvious anomalies, but the confidence level of the primary model has not yet fallen below the threshold for triggering deep detection, which may cause the system to fail to detect these potential anomalies that require further analysis in a timely manner.

[0044] In response, this application further proposes that when the Trend Residual Feature (TRF) shows obvious anomalies but the results of the primary model are unclear, the system should mark the sample as an uncertain sample and switch to the deep inference channel.

[0045] Among them, the Trend Residual Feature (TRF) indicating a significant anomaly refers to the deviation between the actual and predicted values ​​of equipment operating indicators exceeding the normal range. TRF is an indicator that quantifies the degree of deviation of the current operating state from historical statistical patterns, and its value directly reflects the intensity of the anomaly. When TRF shows a significant anomaly, it indicates that the equipment's operating mode has changed, potentially foreshadowing a potential fault. This "significant anomaly" judgment can be achieved in various ways. For example, by setting dynamic or static statistical thresholds, when the TRF continuously or instantaneously exceeds these thresholds, it is judged as an anomaly; or, machine learning models can be used to detect anomalies in TRF, identifying TRF values ​​that deviate from the normal distribution. The ambiguous results of the primary model mean that, in the primary detection stage, the fault probability on the indicator side (P_ind) and the comprehensive confidence score on the image side (Conf_img) are neither high enough for efficient and rapid discrimination nor low enough to directly trigger the deep inference channel threshold. This indicates that the primary model's judgment of the current sample is in an intermediate state, unable to provide a clear conclusion of normal or fault. This "ambiguous" state can be achieved by defining a confidence interval. For example, when P_ind or Conf_img falls within a preset ambiguity interval, it is considered an uncertain result. The purpose of marking this sample as uncertain is to separate samples with unclear initial detection results but other anomalies from the efficient and rapid discrimination process for more in-depth analysis. This marking can be an internal status flag or an attribute stored in the database for subsequent process control. Switching to the deep inference channel means that the system will no longer rely on the simple threshold judgment of the initial detection, but will instead initiate a more complex, time-consuming, but more accurate analysis process. This channel typically activates advanced inference tools such as Large Language Models (LLM), combining more contextual information and expertise for comprehensive judgment.

[0046] In power equipment fault detection, this application enhances the system's sensitivity to potential anomalies by introducing additional consideration of Trend Residual Features (TRF). In the initial detection stage, the system processes indicator-side and image-side data in parallel and performs preliminary evaluations based on their respective fault probabilities (P_ind) and overall confidence levels (Conf_img). Typically, if both are above a preset threshold, the detection result is quickly output; if the confidence level on either side is below the threshold, the sample is marked as uncertain and switched to the deep inference channel. Building upon this, this application further refines the identification mechanism for uncertain samples. Even if the fault probability on the indicator side and the overall confidence level on the image side of the initial model have not yet fallen below the threshold for triggering deep inference, if the Trend Residual Features (TRF) show obvious anomalies and the initial model results themselves are in an ambiguous state, the system will proactively mark the sample as uncertain and direct it to the deep inference channel. This mechanism enables the system to capture early or slowly changing faults that the initial model may have failed to clearly identify due to its design limitations. These faults often manifest as subtle deviations in operating trends rather than instantaneous large fluctuations or significant defects in the image. In this way, while maintaining the high efficiency of primary detection, the system significantly improves its ability to detect complex and hidden faults, ensuring comprehensive coverage and in-depth analysis of all potential anomalies, thereby avoiding the risk of missed detections due to unclear judgments in the primary model.

[0047] The following is a concrete example to illustrate this. The anomaly judgment rule for the Trend Residual Feature (TRF) can be set as follows: when the TRF value exceeds the dynamic upper limit of its historical mean plus two standard deviations for three consecutive sampling periods, or when the instantaneous TRF value exceeds the static upper limit of its historical mean plus three standard deviations, it is considered that the TRF shows a significant anomaly. Meanwhile, the judgment of unclear results from the primary model can be defined as: when the fault probability (P_ind) on the indicator side or the overall confidence level (Conf_img) on ​​the image side falls between 0.4 and 0.6. For example, during the operation of a power transformer, the Trend Residual Feature (TRF) of its winding temperature shows a slow upward trend for several consecutive days and has exceeded the dynamic upper limit of the historical mean plus two standard deviations, indicating a potential overheating risk. However, at this time, the fault probability P_ind output by the indicator-side primary detection model is 0.55, and the overall confidence level Conf_img output by the image-side primary detection model is 0.58. Based on the logic of the initial detection described above, since neither P_ind nor Conf_img is below the preset threshold of 0.4, the system may not immediately switch to the deep inference channel. However, according to the solution in this application, since the TRF has shown obvious anomalies, and both P_ind and Conf_img are within the ambiguous range of 0.4 to 0.6, the system will immediately mark the transformer sample as an uncertain sample and switch it to the deep inference channel. In the deep inference channel, the large language model will combine the abnormal trend of the TRF, the fuzzy judgment of the initial model, and other relevant data (such as historical operation records, maintenance logs, etc.) for comprehensive analysis, thereby identifying early overheating faults in the transformer windings earlier and more accurately, and proposing corresponding maintenance suggestions.

[0048] Through the above technical solution, this application effectively solves the technical problem that in power equipment fault detection, the primary model may fail to detect potential anomalies in a timely manner due to the ambiguity of its judgment results. By introducing Trend Residual Feature (TRF) as an additional judgment criterion, and combining it with the ambiguity of the primary model results, the system can more sensitively capture samples that have not yet reached a clear fault threshold but have already shown abnormal trends. This allows the system to send these potentially risky samples into the deep inference channel for detailed analysis earlier, thereby significantly improving the ability to identify early, slowly changing, or hidden faults, reducing the risk of missed detection, and thus improving the reliability and timeliness of the entire fault detection system.

[0049] In some embodiments described above in this application, a method is proposed for performing primary detection based on the operating indicators of the equipment under test and calculating the fault probability (P_ind) of the equipment under test based on the operating indicators. However, in practical applications, operating indicator data often have complex time series characteristics, and directly calculating the fault probability may not accurately capture subtle changes in the operating status of the equipment and potential early faults, resulting in insufficient accuracy and sensitivity of the primary detection.

[0050] In response, this application further proposes to calculate the failure probability of the equipment under inspection by including: preprocessing the operating indicator data and extracting time-series features; predicting and calculating trend residual features through the ARIMA model to quantify the degree of abnormal deviation; inputting the trend residual features into the CatBoost model for primary fault classification and outputting the failure probability of the indicator (P_ind).

[0051] Preprocessing of operational indicator data aims to eliminate noise, missing values, outliers, and other contaminants from the original data, transforming it into a format suitable for model analysis. Extracting time-series features involves mining the regularities reflecting the changes in equipment operating status over time from the preprocessed data. These features are crucial for capturing the dynamic behavior and potential failure modes of the equipment. Preprocessing can include data cleaning, such as interpolation to fill in missing values ​​and smoothing filtering to remove noise; and data normalization or standardization, such as Min-Max normalization or Z-score standardization, to eliminate the influence of different dimensions between indicators. Time-series feature extraction can include calculating statistical features, such as mean, variance, maximum, minimum, kurtosis, and skewness; it can also extract frequency domain features, such as spectral energy and dominant frequency obtained through Fourier transform; or features based on sliding windows, such as moving average and moving standard deviation. Furthermore, preprocessing can use wavelet transform for multi-scale decomposition to separate signals of different frequency components. Time-series feature extraction can utilize autocorrelation functions and partial autocorrelation functions to analyze the serial correlation of the data, or use differencing operations to stabilize non-stationary time series for better processing by subsequent models.

[0052] The trend residual feature is quantified by predicting and calculating trend residual characteristics using the ARIMA model. The ARIMA (Autoregressive Integrated Moving Average) model is a classic statistical time series forecasting model that effectively captures the autoregressive, differencing, and moving average characteristics of time series data. Predicting operational indicator data using the ARIMA model yields the expected values ​​of equipment under normal operating conditions. Comparing the actual monitored values ​​with the predicted values ​​from the ARIMA model, the calculated residual is the trend residual feature. This feature quantifies the degree of deviation of the current equipment operating state from its historical normal trend, thus effectively identifying potential anomalies or signs of failure. The construction of an ARIMA model typically includes model identification, such as determining the p, d, and q parameters through analysis of ACF and PACF plots; parameter estimation; and model testing. During forecasting, the model generates predicted values ​​for future times based on historical data. The trend residual feature can be obtained by calculating the difference between the actual observed values ​​and the predicted values ​​from the ARIMA model; for example, it can be directly represented using the absolute difference or the squared difference. More complex residual calculation methods can also be used, such as standardizing the residuals to make them follow a specific distribution for better anomaly detection. In addition to the ARIMA model, other time series forecasting models, such as the exponential smoothing model (ETS), the Prophet model, or deep learning-based recurrent neural networks (RNN) and long short-term memory networks (LSTM), can be used to generate predicted values ​​and then calculate trend residual features.

[0053] The trend residual features are input into the CatBoost model for primary fault classification, outputting the indicator-side fault probability (P_ind). The CatBoost model is a gradient-boosting decision tree-based machine learning model, renowned for its excellent handling of categorical features and high accuracy. Using trend residual features as input, the CatBoost model learns the complex nonlinear relationship between residual features and different fault types, thus performing primary fault classification of equipment operating status. The indicator-side fault probability (P_ind) output by the model represents the probability that the current equipment belongs to a specific fault type, providing a basis for subsequent confidence assessment and deep detection. Before training, the trend residual features require appropriate feature engineering; for example, binning or combining the residual features can be performed to enhance the model's learning ability. During model training, historical fault data can be used as labels, and model parameters are optimized by minimizing the loss function. During the inference phase, the model outputs a probability vector, where each element corresponds to a fault type and represents the probability of that fault type occurring. Besides the CatBoost model, other ensemble learning models, such as XGBoost, LightGBM, or classification models like Support Vector Machines (SVM) and neural networks, can also be used for primary fault classification. These models can also accept trend residual features as input and output the corresponding fault probabilities.

[0054] This application's solution improves the accuracy and sensitivity of primary indicator detection by refining and analyzing power equipment operation indicator data through multi-stage analysis. First, the raw operation indicator data is preprocessed to eliminate interference and standardize the data format, ensuring the quality of subsequent analysis. Based on this, time-series features reflecting the dynamic changes of the equipment are extracted from the preprocessed data; these features are crucial for capturing the evolution of the equipment's operating state. Subsequently, the ARIMA model is used to model and predict these time-series features, generating a baseline of expected equipment behavior under normal operating conditions. By comparing actual observations with the ARIMA model's predictions, trend residual features are calculated. These features directly quantify the degree to which the current equipment state deviates from the normal trend, effectively identifying potential anomalies or early signs of failure. Finally, these trend residual features are input into the CatBoost model. Leveraging its powerful learning capabilities, the CatBoost model can identify complex patterns associated with different failure modes from these residual features and output the indicator-side failure probability (P_ind) of the inspected equipment belonging to various failure types. This layered and progressive analysis process enables primary detection to more accurately capture subtle anomalies in equipment operation, providing more reliable input for subsequent confidence assessment. This effectively solves the problem of insufficient accuracy of primary detection in complex time series data using traditional methods.

[0055] The following is a concrete example to illustrate this. The probability of failure on the indicator side can be calculated using dissolved gas analysis (DGA) data from power transformer oil. First, the concentration data of key gases such as acetylene, methane, ethylene, ethane, and hydrogen in the transformer oil are preprocessed. Preprocessing steps may include: filling missing data with linear or spline interpolation; smoothing outliers (such as spikes caused by momentary sensor failures) using median filtering; and performing Min-Max normalization on all gas concentration data, scaling them to the range of 0 to 1. Next, time-series features are extracted from the normalized gas concentration time series. For example, the average concentration, maximum concentration, minimum concentration, standard deviation, and rate of change between adjacent time points for each gas over the past 24 hours can be calculated. Subsequently, an ARIMA(p,d,q) model is constructed for each gas concentration. For example, for acetylene concentration, appropriate p, d, and q parameters can be determined by analyzing the autocorrelation function (ACF) and partial autocorrelation function (PACF) of its historical data, such as ARIMA(1,1,1). The ARIMA model predicts future acetylene concentrations based on historical acetylene concentration data. Then, the actual monitored acetylene concentration is compared with the ARIMA model's prediction, and the absolute difference between the two is calculated as the trend residual feature for acetylene. Similar processing is performed on other gases to obtain their respective trend residual features. Finally, the trend residual features of all gases (e.g., acetylene residual, methane residual, etc.) and some preprocessed original time-series features (e.g., gas change rate) are combined into a feature vector, which is then input into a pre-trained CatBoost model. Based on these features, the CatBoost model outputs the probability distribution of possible transformer fault types (e.g., partial discharge, overheating fault, arc discharge, etc.), i.e., the index-side fault probability (P_ind). For example, the model might output a probability of 0.75 for partial discharge, 0.20 for overheating fault, and 0.05 for normal operation.

[0056] Through the above technical solutions, this application enables in-depth mining and analysis of power equipment operation index data, significantly improving the accuracy and reliability of primary detection on the index side. By preprocessing and extracting time-series features from the operation index data, noise can be effectively removed and dynamic information captured from the data. The method of predicting and calculating trend residual features using an ARIMA model allows the system to accurately quantify the degree of deviation of the equipment's operating state from the normal trend, thereby more sensitively detecting early or slowly changing faults. Inputting these highly indicative trend residual features into a CatBoost model for fault classification fully leverages the advantages of the CatBoost model in handling complex feature relationships, outputting more accurate fault probabilities on the index side. This allows the primary detection stage to more effectively identify potential faults, reducing the risk of false alarms and false negatives, and providing a solid foundation for subsequent confidence assessment and rapid discrimination, thereby improving the efficiency and accuracy of the entire fault detection method.

[0057] In other embodiments, this application proposes a power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration. This method acquires an image of the equipment under inspection, performs initial image-side detection based on the image, and calculates the comprehensive image-side confidence of the equipment under inspection. However, in practical applications, relying solely on the raw classification probability output by the target detection model to evaluate the image-side detection results may have limitations. For example, when the image quality is poor (e.g., blurry, insufficient lighting), the model may give a high classification probability, but the actual detection result is unreliable; or, a certain "abnormal" phenomenon detected by the model may contradict known engineering common sense or physical laws, which, if not identified, may lead to false alarms or incorrect judgments of the severity of the fault.

[0058] To address this, this application further proposes a method for calculating the comprehensive confidence score of the image of the device under inspection. This method includes: performing initial detection using a lightweight target detection model (such as the YOLO series) to output the bounding boxes of candidate targets and the original classification probability P_model; and comprehensively forming the image-side comprehensive confidence score Conf_img by calculating the image quality score Q_data and the knowledge association degree R_know based on the knowledge graph (measured by the similarity S_sem between visual embedding and text semantic embedding). The image-side comprehensive confidence score is a weighted sum of the image quality score, the knowledge association degree based on the knowledge graph, and the original classification probability. Q_data is the sharpness score calculated based on the Laplacian variance of the image. R_know is the entity association degree calculated based on the device fault knowledge graph. The association degree can be modeled using an exponential decay function based on the shortest path length of the entity in the knowledge graph to suppress detection results that clearly violate common sense in engineering.

[0059] Lightweight object detection models aim to quickly identify target objects in images and determine their locations. Their "lightweight" nature means a simplified model structure and lower computational resource requirements, enabling efficient real-time or near real-time processing. Besides the YOLO series, SSD (SingleShot MultiBoxDetector) or EfficientDet models can also be used. These models are known for their good balance between speed and accuracy, making them suitable for edge devices or scenarios with strict response time requirements. The bounding box of a candidate target is a rectangular region surrounding the detected target object in the image, indicating the target's location and size. The original classification probability P_model is the confidence score of the object detection model for each detected target belonging to a specific category, reflecting the model's initial judgment of the classification result. For example, the model might output a bounding box with a probability value indicating the likelihood that the object within the box is a "broken insulator." The image quality score Q_data is used to quantify the visual quality of the input image to assess its reliability for subsequent analysis. Low-quality images (such as blurred, overexposed, or underexposed images) may lead to inaccurate model detection results. In addition to sharpness scoring based on Laplacian variance, Q_data can also assess image noise levels by calculating signal-to-noise ratio (SNR) or peak signal-to-noise ratio (PSNR), or comprehensively evaluate the overall visual effect of an image by analyzing indicators such as contrast and brightness distribution. Knowledge graph-based knowledge association R_know aims to incorporate domain expertise to assess the degree to which image detection results conform to known engineering common sense or fault modes. This helps filter out "illusion" or low-confidence results that the model may produce but do not match reality. Besides modeling association based on the shortest path length of entities in the knowledge graph using an exponential decay function, R_know can also measure it by calculating the semantic similarity between detected fault types and related entities such as equipment components and operating environments in the knowledge graph, or by logically verifying the detection results through an expert system rule base to determine whether they conform to preset engineering constraints. Image-side comprehensive confidence Conf_img is a comprehensive quantification of the reliability of the initial image detection results. It provides a more comprehensive and reliable confidence assessment by weighting and fusing the model's classification probability P_model, the image quality Q_data, and the degree of conformity with domain knowledge R_know. This weighted sum calculation method allows for adjustments based on the importance of different factors. For example, detection results with poor image quality or low correlation with the knowledge graph can be assigned a lower overall confidence score, thereby reducing their impact on the final decision.

[0060] This application's scheme first utilizes a lightweight object detection model to rapidly analyze the acquired images of the equipment under inspection during the initial image-side detection process. This identifies potential abnormal targets within the images and outputs the bounding boxes of these targets, along with the model's initial classification probabilities P_model for each target category. To ensure the reliability of these preliminary detection results, this method further introduces an image quality score Q_data and a knowledge graph-based knowledge association score R_know. The image quality score Q_data quantifies visual attributes such as image sharpness, assessing the reliability of the image data itself and avoiding misjudgments due to poor image quality. Simultaneously, the knowledge graph-based knowledge association score R_know compares the detection results with professional knowledge in the field of power equipment faults. By evaluating the correlation strength between detected entities and known fault modes and equipment components, it effectively suppresses detection results that clearly violate engineering common sense or physical laws. Finally, the initial classification probabilities P_model, image quality score Q_data, and knowledge association score R_know are weighted and fused to form the image-side comprehensive confidence score Conf_img. This comprehensive confidence level not only reflects the model's confidence in the detection results but also incorporates considerations of data quality and domain knowledge, thus providing a more comprehensive and reliable basis for subsequent evaluation. In this way, even when the model's initial probability is high, if the image quality is poor or the detection results contradict professional knowledge, the comprehensive confidence level will decrease accordingly. This effectively improves the accuracy and reliability of the initial detection results, avoiding misjudgments or missed detections due to insufficient single indicators in the initial stage, thereby optimizing the efficiency and accuracy of the entire fault detection process.

[0061] One specific implementation example is as follows: When performing fault detection on a transformer, the system first acquires images of the transformer's appearance using a camera mounted on a drone. These images are input into a pre-trained YOLOv8 model, which can quickly identify components such as insulators, bushings, and radiators in the images, and detect anomalies such as oil stains, cracks, and discharge traces. It also outputs the bounding box of each detected anomaly region and its corresponding original classification probability P_model. For example, the model might detect a suspected crack on the surface of an insulator and give a P_model of 0.85. Next, the system calculates the Laplacian variance of the image to obtain an image quality score Q_data, for example, 0.7 (out of 1.0, indicating good image clarity). Simultaneously, the system queries a power equipment fault knowledge graph to evaluate the correlation R_know between the detection result of "insulator crack" and "transformer insulation fault." If the knowledge graph shows that insulator cracks are a typical manifestation of transformer insulation faults and the path is short, then R_know might be 0.9. Finally, using preset weights (e.g., P_model weight 0.5, Q_data weight 0.2, R_know weight 0.3), the overall confidence score on the image side is calculated as Conf_img = 0.85*0.5 + 0.7*0.2 + 0.9*0.3 = 0.425 + 0.14 + 0.27 = 0.835. This overall confidence score will be compared with a preset threshold to determine whether to directly output the initial detection result or switch to the deep inference channel.

[0062] The above technical solution considers not only the classification probability of the target detection model itself during initial image detection, but also incorporates image quality scoring and knowledge graph-based knowledge association. This multi-dimensional comprehensive evaluation mechanism effectively avoids misjudgments caused by poor image quality or detection results contradicting common sense, significantly improving the reliability and accuracy of initial image detection results. When image quality is low or detection results are inconsistent with professional knowledge, the overall confidence level decreases accordingly, enabling more timely and accurate identification of uncertain samples requiring further in-depth analysis. This avoids blind decision-making in the initial detection stage, provides more accurate input for subsequent deep reasoning, and ultimately improves the robustness and diagnostic accuracy of the entire power equipment fault detection method.

[0063] In other embodiments, this application proposes a power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration. In some of the aforementioned embodiments, a lightweight target detection model is used to perform primary detection on the acquired appearance image of the equipment to be inspected, in order to calculate the overall image-side confidence. However, in practical applications, due to the influence of factors such as the acquisition environment, equipment status, and lighting conditions, the original image may have problems such as inconsistent size, uneven brightness, insufficient contrast, or limited sample size. These problems may lead to a decrease in the recognition accuracy and generalization ability of the lightweight target detection model, thereby affecting the accuracy and reliability of the primary detection.

[0064] To address this, this application further proposes processing the images by size normalization, brightness and contrast adjustment, and data augmentation before inputting them into a lightweight target detection model for primary detection. The appearance images of the device under inspection and its auxiliary components refer to visual data used for fault detection, covering the main structure of the power equipment and the visual representation of its functionally related auxiliary components. These images form the basis for image-based primary detection. Size normalization involves adjusting images of different sizes to a preset standard size. Its purpose is to eliminate the impact of image size differences on model input and feature extraction, ensuring the model can process data in a uniform format and improving training efficiency and inference stability. Implementation methods may include: scaling the image to a fixed width and height, for example, through bilinear interpolation or nearest-neighbor interpolation algorithms; or, after adjusting the image to a fixed size, padding any insufficient parts, such as zero-padding or edge pixel duplication. Brightness and contrast adjustment optimizes the overall brightness and darkness of the image and the grayscale differences in different areas of the image. Its purpose is to compensate for image quality degradation caused by changes in lighting conditions, enhance the distinction between the target and the background in the image, make target features more prominent, and thus improve the target detection model's ability to perceive features. Implementation methods can include: adjusting image brightness through linear transformation or gamma correction; or enhancing image contrast through histogram equalization or contrast-limited adaptive histogram equalization (CLAHE). Data augmentation refers to generating new training samples by performing a series of transformations on existing image data to expand the dataset. Its purpose is to increase the diversity of training data, simulate various changes that may occur in real-world applications, thereby improving the model's generalization ability and robustness, and reducing the risk of overfitting. Implementation methods can include: random cropping, random flipping (horizontal or vertical), random rotation, adding noise (such as Gaussian noise), color dithering (such as randomly changing brightness, saturation, and hue), etc. Inputting these processed images into a lightweight object detection model for primary detection refers to using pre-processed and enhanced image data as input, feeding it into a pre-trained lightweight object detection model for inference, to quickly identify target objects in the image and output their location information and preliminary classification probabilities.

[0065] The proposed solution optimizes image quality and expands the dataset by performing a series of preprocessing operations on the acquired images of the device under inspection and its accessories, including size normalization, brightness and contrast adjustment, and data augmentation. Specifically, size normalization ensures that all images are input into the lightweight object detection model at a uniform size, eliminating interference caused by size differences; brightness and contrast adjustment compensates for the impact of uneven lighting and other environmental factors on image quality, making key features in the images clearer and more identifiable; and data augmentation effectively improves the model's generalization ability and adaptability to complex environments by generating diverse training samples. These optimized images are then input into the lightweight object detection model for primary detection, enabling the model to more accurately and stably identify target objects and their potential anomalies in the images. In this way, the solution provides high-quality and robust input data for primary image detection, significantly improving the recognition accuracy and reliability of the lightweight target detection model. This lays a solid foundation for the accurate calculation of the subsequent comprehensive confidence score (Conf_img) on ​​the image side, enabling the entire fault detection system to make preliminary judgments more efficiently and accurately when facing complex and ever-changing real-world conditions.

[0066] As a specific implementation method, after the system acquires the appearance images of the device body and its accessories under inspection, these images are first preprocessed. For example, for size normalization, all images can be uniformly scaled to 640x640 pixels while maintaining the aspect ratio, and any insufficient areas are filled with gray pixels. Next, brightness and contrast are adjusted, which can be achieved by applying the Adaptive Histogram Equalization (CLAHE) algorithm to enhance local contrast and suppress noise. In addition, to improve the generalization ability of the model, data augmentation processing can be performed on the training dataset, such as randomly flipping the images horizontally, randomly rotating them between -15 degrees and +15 degrees, and randomly adjusting the saturation, brightness, and hue of the images to simulate image changes under different lighting and shooting angles. The images after these processing steps can then be used as input to the pre-trained YOLOv8s model for primary detection to quickly identify device components and output their bounding boxes and classification probabilities.

[0067] By employing the aforementioned technical solutions, the original images undergo size normalization, brightness and contrast adjustment, and data augmentation. This effectively addresses the challenges of poor image quality and insufficient model generalization ability caused by complex acquisition environments, variable lighting conditions, and limited data volume. The preprocessed and enhanced images provide a more stable and representative input for the lightweight target detection model, significantly improving the model's accuracy and robustness in identifying power equipment fault features. This not only greatly enhances the accuracy of primary image-based detection, reducing the risk of false alarms and false negatives, but also strengthens the adaptability and reliability of the entire fault detection system under complex and variable operating conditions. This ensures more accurate calculation of the overall confidence score (Conf_img) on ​​the image side, providing a more solid and reliable basis for subsequent fault assessment and deep inference.

[0068] In some embodiments described above in this application, when the confidence level on either side is lower than a threshold, the sample is marked as an uncertain sample and switched to the deep inference channel for deep detection. However, in the deep inference channel, if the original indicator data is directly input for deep detection, the complexity or potential ambiguity of the data itself may cause the deep inference model to have difficulty ensuring the accuracy and physical rationality of the inference results in the absence of clear domain knowledge constraints, thereby affecting the reliability of the final fault determination.

[0069] In response, this application further proposes that, in the deep inference channel, before the large language model is activated, the index data be processed according to physical and engineering rules (such as the Duval triangle decision of DGA and the DL / T722 standard) to form hard constraints.

[0070] The deep inference channel refers to a module or process used by the system to perform deeper and more refined analysis when the initial detection results are unclear or lack confidence. It typically involves more complex models, richer knowledge bases, or more advanced inference mechanisms, aiming to address complex or ambiguous fault situations that the initial detection failed to effectively identify. Its implementation methods may include, but are not limited to: activating a large language model for semantic reasoning, combining expert systems for rule judgment, or utilizing multimodal fusion models for deep feature learning. "Before the large language model is activated" refers to a preprocessing stage before the system calls or starts the large language model for inference tasks. This stage aims to provide the large language model with more standardized and domain-specific input to improve its inference efficiency and accuracy. Its implementation methods may include, but are not limited to: data cleaning, feature engineering, rule-based processing, or converting raw data into structured prompts that the large language model can easily understand. Rule-based processing of indicator data based on physical and engineering rules refers to using recognized physical laws, industry standards, expert experience, and other rigid regulations within the power equipment field to perform pre-logical judgments and transformations on the original operating indicator data. This processing aims to map raw data to discrete states or judgment results with clear physical or engineering meaning. For example, based on the Duval triangle method of dissolved gas analysis (DGA) in transformer oil, a specific gas concentration ratio can be directly classified into a certain fault type (such as partial discharge, overheating, etc.); or, according to industry standards such as DL / T722, the threshold values ​​of certain indicators can be judged to determine whether they exceed the normal range. Another approach is to establish a series of logical judgment expressions based on equipment design parameters, operating condition limitations, etc., to filter, verify, or classify indicator data. Forming hard constraints refers to transforming the judgment results of physical and engineering rules into mandatory or highly guiding inputs for subsequent deep reasoning models (such as large language models) through rule-based processing. These constraints can be clear fault type judgments, data validity verification results, or reasonable range limits for specific parameters. For example, if the DGA analysis clearly indicates a certain fault type, this result will serve as inviolable prior knowledge that must be considered during large language model reasoning. Another approach is to encode the results of the rule-based processing as Boolean variables or discrete labels as part of the input prompts for the large language model, thereby forcing the model to follow these pre-defined domain knowledge during the inference process.

[0071] In the deep inference channel, the proposed solution first performs rule-based processing on the indicator data based on physical and engineering rules before activating the large language model. Specifically, when the initial detection results are unclear, and the sample is marked as uncertain and enters the deep inference channel, the system does not immediately input the original indicator data or its preliminary features directly into the large language model. Instead, it utilizes mature and proven physical laws and engineering standards in the power equipment field, such as the Duval triangle determination method of DGA or the DL / T722 standard, to perform rigorous logical judgments and classifications on these indicator data. The results of these rule-based processing, such as clear fault type determinations or data validity verifications, are transformed into hard constraints that have a mandatory guiding effect on subsequent large language model inference. In this way, when the large language model performs complex cross-modal consistency analysis and conflict resolution inference, it can obtain input that has been pre-verified and normalized by domain knowledge, thereby avoiding inference biases caused by the complexity or ambiguity of the original data and ensuring that its output fault probability distribution results are physically reasonable and highly consistent with professional knowledge. This preprocessing mechanism effectively addresses the accuracy and reliability issues that may arise in deep reasoning models when there is a lack of explicit domain knowledge constraints, and significantly improves the diagnostic quality of uncertain samples.

[0072] As a specific implementation method, when the operating index data of the transformer under test (such as the concentration of dissolved gases in the oil, including acetylene, methane, ethylene, ethane, hydrogen, carbon monoxide, and carbon dioxide) is marked as an uncertain sample by the primary detection and enters the deep inference channel, the system can first activate a Duval triangle judgment module based on DGA (Dissolved Gas Analysis). This module calculates the corresponding ratio according to the gas concentration ratios specified in the IEC60599 standard or DL / T722 standard (such as C2H2 / C2H4, CH4 / H2, etc.) and projects them onto the Duval triangle area, thereby directly determining the potential fault type, such as partial discharge, overheating fault, or arc discharge. These explicit fault judgment results, such as "high-energy discharge fault exists," serve as hard constraints and, when the large language model is activated subsequently, are used as structured prompt inputs along with the original index features. Furthermore, the system can also perform threshold judgments on the absolute values ​​or rates of change of certain key indicators (such as oil temperature, winding temperature, and partial discharge) based on industry standards such as DL / T722. If the values ​​exceed preset thresholds, hard constraint information of "indicator exceeding limits" is directly generated. These highly reliable judgment results, pre-determined through engineering rules, can effectively guide the large language model to reason within a smaller and more reasonable fault space, avoiding blind inferences based on highly uncertain raw data.

[0073] Through the above technical solution, in the deep inference channel, the large language model receives hard constraints formed by index data preprocessed with physical and engineering rules before being activated. This ensures that the inference process and results of the large language model during deep detection strictly adhere to the professional knowledge and physical laws of the power equipment field, significantly improving the accuracy and reliability of fault determination for uncertain samples. This solution effectively avoids inference results from the large language model that do not conform to engineering reality due to the complexity or ambiguity of the original data, ensuring the physical rationality and professional consistency of the final fault diagnosis, thereby improving the diagnostic quality and credibility of the entire fault detection system.

[0074] In some embodiments described above in this application, the system performs preliminary detection on the indicator side and the image side, and combines physical and engineering rules to process the indicator data in a rule-based manner to preliminarily determine the fault status of power equipment. However, when faced with complex and ever-changing fault modes or unclear preliminary model results, relying solely on rule-based judgment or simple model output may be insufficient for comprehensive, accurate, and physically plausible fault diagnosis. This is especially true in scenarios requiring the integration of multi-source information for deep reasoning, where effectively utilizing this information for decision-making remains a challenge.

[0075] In response, this application further proposes a step for switching to the deep inference channel for deep detection, including: activating the Large Language Model (LLM), and then using the initial detection results, rule determination information, and image semantic labels as query conditions, retrieving the most similar historical fault cases and corresponding mechanism descriptions from the vectorized fault knowledge base to form a professional context; the Large Language Model receives structured prompts including indicator features (including TRF), Duval projection results, image semantic consistency scores, rule determination results, and retrieved knowledge context, performs cross-modal consistency analysis and conflict resolution inference, and outputs the corrected fault probability distribution results to ensure the physical rationality and professional consistency of the inference results.

[0076] Activating the Large Language Model (LLM) refers to launching a deep learning model with powerful language understanding, generation, and reasoning capabilities. This large language model can be a pre-trained general-purpose model, such as the GPT, BERT, or Llama series, or a specialized model fine-tuned for the power industry. Its role is to leverage its advanced reasoning capabilities to perform in-depth analysis and comprehensive judgment of multi-source heterogeneous information, thereby overcoming the limitations of primary detection and rule-based judgment. Using primary detection results, rule-based judgment information, and image semantic tags as query conditions, the system retrieves the most similar historical fault cases and corresponding mechanism descriptions from a vectorized fault knowledge base, forming a professional context. This aims to provide rich background knowledge and experience references for the large language model's reasoning. Primary detection results include the fault probability P_ind on the indicator side and the comprehensive confidence score Conf_img on the image side. The rule-based judgment information consists of hard constraints formed after regularizing the indicator data based on physical and engineering rules (such as the Duval triangle judgment of DGA and the DL / T722 standard). The image semantic tags originate from the analysis of appearance images by the primary detection on the image side. The vectorized fault knowledge base is a knowledge collection storing a large amount of historical fault data, expert experience, fault mechanisms, and industry standards. This knowledge is transformed into high-dimensional vector representations through vector embedding technology. The retrieval process can employ metrics such as cosine similarity and Euclidean distance to quickly find the historical cases and mechanism descriptions that best match the current query conditions, thus providing targeted professional knowledge support for the large language model. The large language model receives structured input prompts including indicator features (including TRF), Duval projection results, image semantic consistency scores, rule determination results, and retrieved knowledge context. This is to provide multimodal and multi-scale information to the large language model for processing in a unified format. Indicator features include raw operating indicator data and trend residual features (TRF), reflecting quantitative information about the equipment's operating status. Duval projection results are based on dissolved gas analysis (DGA) data to determine the fault type, providing a chemical-level diagnostic basis. Image semantic consistency scores assess the degree of matching between image detection results and text descriptions or known fault modes. Rule determination results are based on preset engineering rules. The retrieved knowledge context provides historical experience and professional mechanisms. This information is structured, for example, through JSON format or specific text templates, ensuring that the large language model can accurately understand and utilize this information for reasoning. Performing cross-modal consistency analysis and conflict resolution reasoning is one of the core functions of the large language model. It aims to integrate information from different modalities (indicators, images, rules, knowledge), identify consistency and potential conflicts, and perform logical reasoning to arrive at more reliable conclusions. Cross-modal consistency analysis refers to the large language model evaluating whether there are mutually corroborating or contradictory situations between information from different modalities.Conflict resolution reasoning refers to the process where, when contradictions exist between different information sources, the large language model utilizes its built-in knowledge and reasoning capabilities, combined with factors such as confidence level, rule priority, and historical experience, to determine which information is more reliable and provide reasonable interpretations or corrections, thereby eliminating uncertainty and ensuring the accuracy of the final diagnostic results. Outputting the corrected fault probability distribution results, ensuring the physical rationality and professional consistency of the reasoning results, is the goal of the deep detection phase. The corrected fault probability distribution results are no longer preliminary judgments from a single modality, but rather a more comprehensive and reliable representation of fault types and their probabilities obtained after the large language model integrates multi-source information, performs cross-modal analysis, and resolves conflicts. Physical rationality means that the diagnostic results conform to physical laws and equipment operating mechanisms. Professional consistency means that the diagnostic results are consistent with the professional knowledge, standards, and expert experience of the power industry.

[0077] In the deep inference channel, when there is uncertainty in the initial detection results or rule determinations, the system activates the large language model. This large language model first uses the initial detection results, rule determination information, and image semantic tags as query conditions to retrieve historical fault cases and corresponding mechanism descriptions most similar to the current state of the device under inspection from a pre-built vectorized fault knowledge base, thus providing rich professional contextual information for subsequent inference. Subsequently, the large language model receives a structured prompt input that integrates multimodal information, including detailed indicator features (e.g., Trend Residual Feature (TRF), Duval projection results based on physical rules, image semantic consistency scores evaluating image detection quality, rule determination results, and previously retrieved professional knowledge context. Based on this comprehensive input, the large language model performs complex cross-modal consistency analysis and conflict resolution inference. This means that it not only identifies mutually corroborating parts between different information sources but also proactively discovers and resolves potential contradictions or inconsistencies. For example, when indicator data and image information give conflicting fault indications, the large language model combines its built-in knowledge and retrieved professional context to perform logical judgments and weight allocation to arrive at the most reasonable explanation. Finally, the large language model outputs the corrected fault probability distribution, which has undergone in-depth integration and logical reasoning of multi-source information, thereby ensuring the physical rationality of the diagnostic results and consistency with the professional knowledge of the power industry, effectively solving the limitations of primary detection and rule judgment in complex scenarios.

[0078] As a specific implementation, suppose that during fault detection of a transformer, the primary detection on the indicator side shows a high trend residual feature (TRF) of a certain winding temperature indicator, suggesting a possible overheating fault, but the fault probability (P_ind) is only slightly above the threshold, making the result unclear. Simultaneously, the primary detection on the image side, using a lightweight object detection model, identifies slight oil contamination on the transformer bushing surface, and the overall confidence level (Conf_img) on ​​the image side is also at a moderate level. Due to the unclear result, the system marks this sample as an uncertain sample and switches to the deep inference channel. In this channel, a pre-trained large language model for the power domain is first activated. The system uses the primary detection results (such as "moderate probability of winding overheating" and "bushing oil contamination"), the Duval triangle judgment results based on DGA data (such as "possibility of partial discharge"), and image semantic tags (such as "abnormal bushing surface") as query conditions to retrieve data from a vectorized fault knowledge base. The knowledge base returns several historical fault cases related to "winding overheating," "partial discharge," and "bushing oil contamination," along with detailed mechanistic descriptions of these faults. For example, partial discharge may cause winding overheating, and oil contamination may be related to bushing insulation degradation. Subsequently, the large language model receives a structured input containing detailed indicator features (including specific TRF values), Duval projection results (e.g., "T2 region"), image semantic consistency scores (e.g., the correlation score between oil contamination and partial discharge), rule-based judgment results (e.g., "DGA results match partial discharge characteristics"), and retrieved knowledge context. Based on this information, the large language model performs cross-modal consistency analysis. It may find that although the initial detection results are unclear, the increase in TRF, the Duval projection results pointing to partial discharge, and the retrieved knowledge indicating that partial discharge is a common cause of winding overheating and bushing oil contamination corroborate each other. Through conflict resolution reasoning, the large language model ultimately outputs a corrected fault probability distribution. For example, it significantly increases the probability of "winding overheating caused by partial discharge" and reduces the probability of "simple bushing oil contamination," thus providing a more physically reasonable and professionally consistent diagnostic conclusion.

[0079] Through the above technical solution, this application effectively addresses the limitations of primary detection and rule-based judgment in complex fault scenarios, as well as the difficulty in integrating multi-source information. The introduction of a large language model enables the system to deeply integrate multimodal information from indicators, images, physical rules, and historical knowledge, performing refined cross-modal consistency analysis and conflict resolution reasoning. This significantly improves the accuracy and reliability of fault diagnosis, especially when facing ambiguous or contradictory initial judgments, enabling the output of more physically reasonable and professionally consistent corrected fault probability distribution results. This solution not only enhances the intelligence level of fault detection but also provides a more comprehensive and reliable decision-making basis for power equipment operation and maintenance.

[0080] In some of the embodiments described above in this application, a large language model receives structured prompts in a deep inference channel, performs cross-modal consistency analysis and conflict resolution inference, and outputs a corrected fault probability distribution. However, in practical applications, how to effectively integrate corrected fault probabilities from different modalities (such as the indicator side and the image side), especially when there are complex cooperative or mutually exclusive relationships between different fault types, to form a final fault determination that is physically reasonable and professionally consistent, remains a problem that needs to be solved.

[0081] To address this, this application further proposes a system for constructing a fault coupling matrix M, where the matrix element M_ij represents the correlation strength between fault type i on the indicator side and fault type j on the image side, used to characterize their cooperative or inhibitory relationship. This matrix quantifies the correlation strength (cooperative or mutually exclusive relationship) between different fault types on the indicator side and fault types on the image side. The coupling matrix can be calibrated based on historical fault statistics and expert experience, and supports dynamic updates. Based on the fault probability results corrected by the Large Language Model (LLM) and the fault coupling matrix, the system calculates the joint risk value and performs a final fusion judgment according to the following strategies: if the fault types are strongly coupled, it is judged as a composite severe fault; if they are physically mutually exclusive, the confidence level is compared and low-confidence results are suppressed; if neutral, a multi-source independent fault warning is output. Through the above mechanism, this application can dynamically utilize the efficient feature extraction capability of the small model and the semantic reasoning capability of the large model, combined with professional knowledge, to achieve a comprehensive, reliable, and interpretable judgment on the status of power equipment.

[0082] The fault coupling matrix M is a key technical feature that explicitly quantifies the correlation strength between different fault types. The elements M_ij of this matrix can represent the synergistic or inhibitory relationship between fault type i on the indicator side and fault type j on the image side. One way to construct this matrix is ​​based on historical fault statistics analysis; for example, by statistically analyzing the frequency or conditional probability of different fault types occurring simultaneously in historical records to determine their correlation strength. Another approach is to combine expert experience and domain knowledge for calibration, with senior engineers setting the correlation degree between different fault types based on their professional judgment. This matrix also supports dynamic updates to adapt to new fault modes or changes in the equipment operating environment. After obtaining the fault probability results after the large language model correction, the system uses the fault coupling matrix M to calculate the joint risk value. Calculating the joint risk value can be achieved by multiplying or weighting the fault probabilities after the large language model correction with the corresponding elements of the coupling matrix to reflect the risk of different fault type combinations, or by designing a multivariate function that takes the corrected probabilities and the coupling matrix as input and outputs a single risk index. The final fusion judgment strategy is then based on the calculated joint risk value and the indication of the fault coupling matrix M. Specifically, if there is a strong coupling relationship between fault types (i.e., the value of the coupling matrix M_ij is higher than the preset "strong coupling" threshold), and both the fault probability on the indicator side and the overall confidence level on the image side after correction by the large language model are higher than their respective fault thresholds), the system will determine it as a complex severe fault. This helps to identify complex and severe fault situations pointed to by multiple modal evidence. If the fault types are physically mutually exclusive (i.e., the value of the coupling matrix M_ij is lower than the preset "mutually exclusive" threshold), and both the fault probability on the indicator side and the overall confidence level on the image side after correction by the large language model exist, the system will compare the confidence levels of different modal results and suppress results with lower confidence levels, thereby resolving contradictory information between modalities and improving diagnostic accuracy. If the relationship between fault types is neutral (i.e., the value of the coupling matrix M_ij is in the "neutral" range), and both the fault probability on the indicator side and the overall confidence level on the image side after correction by the large language model exist, the system will output multi-source independent fault warnings to handle situations where there is no clear correlation between modalities and provide comprehensive warnings.

[0083] This application's solution introduces a fault coupling matrix M, further refining the fusion and judgment of multimodal fault information based on the corrected fault probability distribution output by the large language model in the deep inference channel. In previous solutions, while the large language model could perform cross-modal consistency analysis and conflict resolution inference and output corrected fault probabilities, a further mechanism was needed to transform these corrected probabilities into a unified, physically plausible final diagnostic conclusion, especially when complex correlations exist between fault indications from different modalities. This application constructs a fault coupling matrix M, explicitly quantifying the synergistic or inhibitory relationship between fault types on the indicator side and those on the image side. This matrix, as a priori knowledge or empirical summary, provides a structured basis for subsequent decision-making. After the large language model outputs the corrected fault probability results, the system no longer simply superimposes or maximizes these probabilities, but instead calculates a comprehensive joint risk value based on the fault coupling matrix M. This joint risk value not only considers the fault probabilities of each modality, but more importantly, it incorporates the inherent physical correlations between different fault types. For example, if the indicator detects a certain fault, and the image also detects another fault strongly coupled to it, the joint risk value will increase significantly, prompting the system to classify it as a complex severe fault. Conversely, if the two detected faults are physically mutually exclusive, the system will selectively suppress them based on their confidence levels, avoiding contradictory diagnoses. This mechanism enables the system to dynamically utilize the efficient feature extraction capabilities of the small model and the semantic reasoning capabilities of the large model, combined with professional knowledge, to achieve a comprehensive, reliable, and interpretable judgment on the state of power equipment. This compensates for the shortcomings of relying solely on the probability distribution output by the large model in the final decision fusion, ensuring the physical rationality and professional consistency of the reasoning results.

[0084] The following is a concrete example. Taking transformer fault detection as an example, the system can construct a fault coupling matrix M for transformer faults. This matrix can be calibrated by analyzing a large number of historical transformer fault reports and expert interviews. For example, when the acetylene content in the transformer oil is abnormally high (fault type i on the indicator side), it is often accompanied by partial discharge of the transformer body (fault type j on the image side), then the matrix element M_ij will be calibrated as a strong coupling relationship. Conversely, if the excessive water content in the oil (fault type k on the indicator side) and the transformer bushing rupture (fault type l on the image side) are physically mutually exclusive, then the matrix element M_kl will be calibrated as a mutually exclusive relationship. This fault coupling matrix M can be stored in a dedicated knowledge base and supports dynamic updates and adjustments through the user interface to adapt to new equipment models or operating environments. Suppose that in a single detection, after deep inference, the large language model outputs a probability of 0.85 for "acetylene exceeding the standard in the oil" on the indicator side and a probability of 0.70 for "partial discharge traces" on the image side. The system calculates a weighted combination of the probabilities of "excessive acetylene in oil" and "partial discharge traces" based on the strong coupling relationship between them in the fault coupling matrix M. For example, using a preset fusion function (such as weighted average or product), a higher joint risk value is calculated, indicating a risk of a complex severe fault. Ultimately, because "excessive acetylene in oil" and "partial discharge traces" are defined as strongly coupled in the fault coupling matrix M, the system will determine it as a "complex severe fault inside the transformer" and issue a corresponding warning. As another example, if the probability of "excessive moisture in oil" on the output indicator side of the large language model is 0.90, while the probability of "bushing rupture" on the image side is 0.60, since "excessive moisture in oil" and "bushing rupture" are defined as physically mutually exclusive in the fault coupling matrix M, the system will compare the confidence levels of these two probabilities. Assuming that "excessive moisture in oil" has a higher confidence level, the system will suppress the low-confidence result of "bushing rupture" and ultimately determine it as a "transformer oil moisture excess fault".

[0085] Through the above technical solution, this application can clearly quantify the cooperative or mutually exclusive relationships between different modalities (indicator-side and image-side) of fault types, thereby enabling more refined joint risk value calculation and final fusion judgment based on the fault probability corrected by LLM. This mechanism effectively solves the complex fault association problem that is difficult to handle with single-modality or simple probability fusion, avoiding misjudgment or omission. Especially when facing multi-source information conflicts or cooperative indications, it can dynamically adjust the diagnostic strategy based on physical rationality and professional experience. Through the above mechanism, the system can achieve a comprehensive, reliable, and interpretable judgment of the state of power equipment, significantly improving the accuracy of fault diagnosis and the scientific nature of decision-making.

[0086] The revised probability results refer to the more accurate and physically plausible probability distributions for various potential fault types, obtained by the large language model in the deep inference channel, combining indicator features, rule-based judgment information, image semantic labels, and retrieved professional knowledge context. These probability results reflect the large language model's comprehensive judgment of multimodal information. The fault coupling matrix is ​​a matrix that quantifies the correlation strength between different indicator-side fault types and image-side fault types. The matrix element M_ij represents the cooperative or inhibitory relationship between indicator-side fault type i and image-side fault type j. This matrix can be calibrated based on historical fault statistics and expert experience and supports dynamic updates. This matrix provides prior knowledge of the mutual influence between cross-modal fault types. Calculating the joint risk value refers to combining the revised probabilities output by the large language model and the fault coupling matrix, and using specific algorithms (e.g., weighted summation, Bayesian network inference, fuzzy logic inference, etc.) to comprehensively assess the overall fault risk currently faced by the equipment. This is not simply a matter of summing probabilities, but rather considers the interaction between different fault types.

[0087] In some implementations, when the confidence level on either side falls below a threshold, the system marks the sample as uncertain and switches to the deep inference channel for deep detection. Here, the large language model is activated and receives information including indicator features, Duval projection results, image semantic consistency scores, rule determination results, and retrieved knowledge context. However, in practical applications, effectively organizing and presenting this heterogeneous information to ensure the large language model can accurately and efficiently perform cross-modal consistency analysis and conflict resolution inference, and output physically reasonable and professionally consistent fault probability distribution results, is a problem that needs to be solved.

[0088] In response, this application further proposes to construct a structured prompt input that includes indicator feature information (including trend residual features), Duval projection results, image semantic consistency score, rule judgment results, and retrieval case information, to guide the large language model to perform cross-modal consistency analysis and conflict resolution reasoning tasks, and output the corrected fault probability distribution results.

[0089] Structured prompt input is a predefined input data structure with predefined format and content, designed to integrate information from different sources and modalities (such as numerical values, text, and image semantics) so that large language models can efficiently parse and utilize it. Its role is to improve the efficiency and accuracy of large language models in understanding input information, reduce ambiguity, and guide their reasoning process. This structured prompt input can use standard data exchange formats such as JSON and XML to encapsulate various information items, with each field corresponding to a specific input element, such as "indicator features" or "Duval projection results." Alternatively, specific text templates can be used to fill in various information items according to preset placeholders, forming prompt text that mixes natural language and structured data, such as "Equipment operation indicator features are [indicator features], Duval projection results are displayed as [Duval projection results], image semantic consistency score is [image semantic consistency score], rule determination result is [rule determination result], historical case information is [retrieval case information]." Indicator feature information refers to various sensor data and status parameters collected during the operation of power equipment, as well as the trend residual feature (TRF) obtained through time series analysis. This information provides a quantitative description of the equipment's operating status and is the basic data for determining whether the equipment is abnormal. The trend residual feature (TRF) is particularly effective in reflecting the degree of deviation of the equipment's operating status from the normal mode. When constructing structured prompt input, the raw indicator data can be normalized and directly used as a numerical vector input, while the calculated trend residual feature is appended as a separate numerical value or vector. Alternatively, the indicator features and trend residual features can be converted into descriptive text, such as "The current equipment temperature is [temperature value] degrees Celsius, the pressure is [pressure value] MPa, and the trend residual feature (TRF) value is XX, indicating a slight deviation." Duval projection results are derived from dissolved gas analysis (DGA) data, using Duval triangle plots to determine the type of internal faults in transformers. It provides a fault diagnosis basis based on physicochemical principles and is a classic method in the field of power equipment fault diagnosis, offering a preliminary judgment of fault types under strong rule constraints for large language models. The results can be input as text labels, such as specific fault types like "partial discharge," "arc discharge," or "overheating." Alternatively, the coordinates or regions of the Duval projection in the triangle plot can be input as numerical values ​​or codes, along with their corresponding fault type descriptions.

[0090] Image semantic consistency scoring is a quantitative indicator that measures the degree of consistency between the semantic information extracted from an image of the equipment under inspection and a description of known fault modes or normal states. Its purpose is to assess the credibility of image information and its relevance to fault diagnosis, helping large language models determine the validity of image evidence. This scoring can be achieved by extracting image features using an image recognition model and calculating similarity with image semantic embeddings in a fault knowledge base, resulting in a score between 0 and 1. Alternatively, it can be combined with expert experience or pre-defined rules to associate detected anomalies (such as oil stains, rust, or component deformation) in the image with fault types and provide a consistency score.

[0091] The rule-based judgment result refers to the judgment conclusion obtained after processing the indicator data according to preset physical and engineering rules (such as DL / T722 standards, equipment operation procedures, etc.). It provides the large language model with rigid, industry-standard-based constraints and preliminary judgments, avoiding inference results that contradict common engineering sense. This result can be directly input as a Boolean value (compliant / non-compliant) or a classification label (normal / abnormal / warning). Alternatively, a detailed description of the rule judgment, such as "According to DL / T722 standards, the gas concentration exceeds the standard and is judged as abnormal," can also be input as text.

[0092] The case retrieval process involves extracting historical fault cases similar to the current state of the equipment under inspection from a vectorized fault knowledge base, along with their corresponding mechanism descriptions. This information provides the large language model with rich professional context and experiential knowledge, helping it understand the possible causes and development trends of the current fault, and improving the accuracy and interpretability of its reasoning. When constructing structured input prompts, the ID, fault type, occurrence time, handling measures, and a summary of the mechanism description of the retrieved historical cases can be used as text input. Alternatively, the vector representation of the retrieved cases can be directly concatenated with the text content of the relevant mechanism description as input.

[0093] Guiding a large language model to perform cross-modal consistency analysis and conflict resolution reasoning tasks involves using carefully designed prompting engineering to guide the large language model in integrating and comparing information from different modalities (indicators, images, rules, knowledge), identifying consistency or conflict, and performing logical reasoning to arrive at a final conclusion. Its purpose is to ensure that the large language model can fully utilize all input information, perform deep semantic understanding and logical judgment, resolve potential contradictions in multi-source information, and improve the accuracy and reliability of diagnosis. This can be achieved by explicitly instructing the large language model in prompts such as, "Please analyze the following indicator data, image semantics, rule judgments, and historical cases to identify whether conflicts exist, and provide the final fault diagnosis result and confidence level." Alternatively, multi-turn dialogues or chain-like reasoning prompts can be used to gradually guide the large language model to first analyze information from each modality, then perform conflict detection, and finally make a comprehensive judgment.

[0094] The corrected fault probability distribution output is a probability prediction of various possible fault types given by the large language model after comprehensively analyzing all input information and performing inference. It provides a quantitative fault diagnosis result, corrected for multimodal information and enhanced with expertise, which is more accurate and reliable than the initial detection result of a single modality. This result can be a list or dictionary containing fault types and their corresponding probabilities, such as "{'Partial Discharge': 0.7, 'Overheating Fault': 0.2, 'Normal': 0.1}". Alternatively, it can be a natural language description, such as "Based on comprehensive analysis, the probability of partial discharge in the equipment is 70%, and the probability of overheating fault is 20%". In power equipment fault detection, this application's solution first rapidly identifies clear faults or normal states through preliminary detection on both the indicator and image sides. When the confidence level on either side falls below a preset threshold, the system marks the sample as an uncertain sample and switches to the deep inference channel for deep detection. In the deep inference channel, before the large language model is activated, the indicator data is processed according to physical and engineering rules to form hard constraints. Based on this, this application further constructs a structured prompt input that includes indicator feature information (including trend residual features), Duval projection results, image semantic consistency scores, rule determination results, and retrieval case information. This structured prompt input serves as a unified interface for the large language model to receive information, preprocessing and formatting heterogeneous information from different modalities and sources to ensure that the large language model can efficiently and accurately parse and understand all key evidence. Through this carefully organized input, the large language model is guided to perform cross-modal consistency analysis and conflict resolution inference tasks. This means that the large language model can not only independently analyze each input but also systematically compare the consistency or potential conflicts between information from different modalities. For example, when indicator features and Duval projection results point to a certain fault type, but the image semantic consistency score is low or the rule judgment results are contradictory, the large language model can leverage its powerful semantic understanding and logical reasoning capabilities, combined with retrieved historical case information, to weigh these pieces of information, resolve conflicts, and conduct in-depth causal chain analysis. This mechanism allows the large language model to fully utilize all available multimodal information and expertise, avoiding the limitations of single-modal judgment and overcoming reasoning biases that may result from simple information piling. Ultimately, the large language model outputs a corrected fault probability distribution result, which not only reflects the comprehensive judgment of multimodal information but also ensures the physical rationality and professional consistency of the reasoning results, thereby significantly improving the accuracy and reliability of fault diagnosis.

[0095] The following is a concrete example to illustrate this. When the system detects an uncertain sample in a transformer, it constructs a structured prompt input. This prompt input can be a JSON-formatted text file containing: a field named `indicator_features`, recording the current temperature, pressure, and calculated Trend Residual Feature (TRF) value of the device, for example, `{"temperature":85,"pressure":0.5,"TRF":0.85,"gas_concentrations":{"C2H2":50,"CH4":100}}`; a field named `duval_result`, explicitly indicating the Duval triangle result determined based on DGA data, such as "partial discharge"; a field named `image_semantic_consistency_score`, quantifying the degree of matching between image semantics and fault mode, for example, 0.92; and a field named `...` The "rule_judgment" field describes the judgment result based on rules such as the DL / T722 standard, for example, `{"DL / T722_compliant":false,"reason":"C2H2 exceeds the standard"}`; and a field named "retrieved_cases" lists summary information of similar historical fault cases retrieved from the knowledge base, including fault type and mechanism description, for example, `[{"case_id":"2023001","fault_type":"partial discharge","mechanism":"insulation aging leads to partial discharge, accompanied by C2H2 increase"},{"case_id":"2022005","fault_type":"overheating fault","mechanism":"winding overheating, accompanied by CH4 increase"}]`. After receiving such a structured prompt, the large language model will first parse each field, and then, according to the preset instructions in the prompt, such as "Please comprehensively analyze all information, identify potential conflicts, and give the most likely fault type and its probability", it will perform cross-modal consistency analysis and conflict resolution reasoning, and finally output a corrected fault probability distribution result, such as `{"partial discharge":0.75,"overheating fault":0.20,"normal":0.05}`.

[0096] Through the above technical solution, this application effectively solves the problem of how to efficiently and accurately guide a large language model to process multi-source heterogeneous information in a deep inference channel, while ensuring the physical rationality and professional consistency of the inference results. By constructing structured input prompts, key evidence such as indicator feature information, Duval projection results, image semantic consistency scores, rule judgment results, and retrieval case information are presented to the large language model in a unified and clear format, greatly improving the efficiency of the large language model in understanding complex information and the accuracy of inference. This structured input method enables the large language model to more effectively perform cross-modal consistency analysis and conflict resolution inference, avoiding misjudgments or low-confidence results caused by chaotic information organization or missing key context. The final output of the corrected fault probability distribution not only integrates the advantages of multimodal data and professional knowledge, but also undergoes rigorous logical verification, thereby significantly improving the reliability, accuracy, and interpretability of power equipment fault diagnosis.

[0097] In other embodiments, this application proposes a power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration. In some of the embodiments described above, when the initial detection results are unclear, a switch to a deep inference channel is proposed for deep detection, and a Large Language Model (LLM) is activated to perform cross-modal consistency analysis and conflict resolution inference. However, in practical applications, the accuracy and professionalism of the LLM's deep inference are highly dependent on the acquired contextual information. Without systematic and structured professional knowledge support, the LLM may struggle to effectively handle complex fault modes, leading to insufficient reliability of the inference results and even judgments inconsistent with engineering common sense, thus affecting the overall accuracy and reliability of fault detection.

[0098] To address this, this application further proposes a structured or vectorized knowledge set for storing power equipment fault mechanisms, industry standards, typical cases, and expert experience, which is used to provide professional context support for large-scale model reasoning. Using primary detection results, rule judgment information, and image semantic tags as query conditions, the most similar historical fault cases and corresponding mechanism descriptions are retrieved from the vectorized fault knowledge base to form a professional context for large-scale models used for reasoning.

[0099] The "structured or vectorized knowledge set storing power equipment fault mechanisms, industry standards, typical cases, and expert experience" refers to the systematic organization, encoding, and storage of various professional knowledge related to power equipment fault diagnosis, including the principles of fault occurrence, industry-recognized judgment standards, historical fault instances and their handling experience, as well as the tacit knowledge of domain experts. This knowledge set can be organized in a structured form, such as through relational databases, knowledge graphs, or ontology, where fault mechanisms are described as text fields, industry standards are encoded as rule sets, typical cases are stored as records with labels and attributes, and expert experience is stored in the form of rules or decision trees. Alternatively, this knowledge set can be stored in a vectorized form, for example, using Natural Language Processing (NLP) technology to convert the aforementioned textual knowledge (such as fault reports, standard documents, and expert notes) into high-dimensional vector representations. These vectors can capture the semantic information of the text, making similar knowledge points closer in the vector space, facilitating subsequent similarity retrieval. For example, models such as Word2Vec, BERT, and Sentence-BERT can be used to embed the text.

[0100] The phrase "providing professional context support for large-scale model reasoning" refers to using the aforementioned stored knowledge set to provide the large language model with relevant background information, explanations of professional terminology, historical experience, and judgment criteria when performing fault diagnosis reasoning tasks. This can be achieved by retrieving highly relevant knowledge fragments from the knowledge set after a query and submitting these fragments as additional input (i.e., context) along with the original query to the large language model. Alternatively, the knowledge set can also serve as a dataset for fine-tuning the large language model, enabling it to learn professional knowledge in the field of power equipment faults during the training phase, thus allowing it to directly utilize this internalized knowledge for judgment during reasoning.

[0101] The phrase "using primary detection results, rule-based judgment information, and image semantic tags as query conditions" defines the input criteria for retrieving relevant knowledge from the knowledge base. These query conditions are a comprehensive product of multimodal primary detection and rule-based judgment, capable of more comprehensively reflecting the abnormal state of the equipment under inspection. Specifically, the fault probability output by primary detection, the conclusion of rule-based judgment (e.g., "Duval triangle indicates a T2 type fault"), and the semantic information identified by image recognition (e.g., "oil contamination found on the casing surface") can be combined into a structured query string or vector. This information can be used as different dimensions of the query. For example, primary detection results can be used as the main query keywords, rule-based judgment information as filtering conditions, and image semantic tags as auxiliary keywords for multi-dimensional retrieval.

[0102] The phrase "retrieve the most similar historical fault cases and corresponding mechanism descriptions in a vectorized fault knowledge base" describes the process of obtaining specific professional context from the knowledge base. Specifically, it involves finding historical fault instances and their underlying principles that are most similar to the current equipment under inspection, based on query conditions. If the knowledge base is vectorized, the query conditions can also be converted into vectors. Then, the cosine similarity or Euclidean distance between the query vector and all knowledge vectors in the knowledge base is calculated, and the most similar cases and mechanism descriptions are selected. If the knowledge base is structured, techniques such as keyword matching, semantic search, and graph traversal can be used to search the knowledge graph or database based on the query conditions to find cases and mechanism descriptions that meet the criteria.

[0103] The aforementioned "forming a large model-specific context for reasoning" refers to integrating and formatting retrieved historical failure cases and mechanism descriptions into input information that a large language model can understand and effectively utilize. This can be achieved by directly concatenating retrieved text fragments (such as case descriptions and failure mechanism explanations) into a long text as part of the large language model's prompt. Alternatively, the retrieved information can be further structured, for example, by encapsulating it in JSON format or a specific markup language (such as XML), explicitly indicating which are cases and which are mechanisms, so that the large language model can better parse and utilize it.

[0104] This application's solution constructs and maintains a comprehensive power equipment fault knowledge base, which stores fault mechanisms, industry standards, typical cases, and expert experience in a structured or vectorized form. When the initial inspection results of power equipment are unclear, and the system switches to the deep inference channel, it utilizes the initial inspection results, rule-based judgment information, and image semantic tags as multimodal query conditions. These query conditions are used to efficiently retrieve historical fault cases and corresponding mechanism descriptions most similar to the current state of the equipment under inspection from the vectorized fault knowledge base. This retrieved highly relevant professional knowledge is then integrated and formatted to form the professional context required for deep inference by the large language model. Upon receiving this rich professional context, the large language model can combine structured input prompts such as indicator features, Duval projection results, image semantic consistency scores, and rule-based judgment results to perform more accurate cross-modal consistency analysis and conflict resolution inference. This mechanism ensures that the large language model can fully utilize authoritative domain knowledge when performing complex fault diagnosis, thereby outputting corrected fault probability distribution results and guaranteeing the physical rationality and professional consistency of the inference results.

[0105] This application also provides a transformer detection method. In some of the embodiments described above, a power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration is proposed. In this method, the primary detection on the indicator side involves preprocessing the operating indicator data, extracting time-series features, predicting and calculating trend residual features using an ARIMA model, and then inputting the trend residual features into a CatBoost model for primary fault classification. This includes: calculating the absolute gas production rate R_abs(t) and relative gas production rate R_rel(t) of the critical fault gas, where R_abs(t) = [C(t) − C(t − Δt)] / Δt, and R_rel(t) = [C(t) − C(t − Δt)] / [C(t − Δt) · Δt] × 100%. When C(t − Δt) is less than a preset threshold, the relative gas production rate can be smoothed or a preset substitution strategy can be adopted to avoid numerical instability under low concentration conditions; constructing an ARIMA time series prediction model based on historical normal operation data, generating a baseline prediction curve for the corresponding gas concentration, and calculating the degree of deviation between the actual monitored value and the predicted value. The deviation is defined as the trend residual feature (TRF) in the form of the L2 norm of the residual vector, TRF = ||C_real(t) − C_pred(t)||2. The trend residual feature is used for... The deviation of the current operating state from historical statistical patterns is quantified as one of the statistical features reflecting the intensity of potential abnormal changes, and is used in conjunction with the subsequent model inference results. The preprocessed original gas concentration values, IEC three-ratio encoding results, the absolute and relative gas production rate features, and the trend residual feature (TRF) are concatenated in a predetermined order to construct a structured high-dimensional feature vector, which serves as the input to the primary fault identification model. An improved CatBoost model is used as the primary detector on the indicator side. A cost-sensitive loss function is introduced during the model training phase, and samples with larger trend residual features (TRF) are given higher weights to enhance the model's sensitivity to abnormal change samples. During the model inference phase, the probability distribution vector P_ind of the candidate fault type is output, and the relative contribution of each input feature is calculated to characterize the importance of different features in the current discrimination result, thereby providing a reference for subsequent comprehensive inference.

[0106] The calculation of the absolute and relative gas generation rates of critical fault gases aims to quantify the dynamic indicators of dissolved gas concentration changes in transformer oil, reflecting the speed and intensity of fault development. The absolute gas generation rate directly reflects the absolute increase in gas concentration per unit time, suitable for assessing the severity of the fault; while the relative gas generation rate reflects the proportion of gas concentration change relative to the current level, making it more sensitive for early fault detection at low concentrations. This can be achieved by continuously monitoring the concentration C(t) of dissolved gases (such as H2, CH4, C2H6, C2H4, C2H2, CO, CO2, etc.) in transformer oil and recording the sampling time interval Δt, then directly applying the formula for calculation. Alternatively, techniques such as sliding window averaging or exponential smoothing can be used to preprocess the original concentration data to reduce the impact of instantaneous noise on the rate calculation before proceeding. When C(t−Δt) is less than a preset threshold, the relative gas generation rate is smoothed or a preset substitution strategy is adopted to avoid numerical instability or abnormal increases in the relative gas generation rate due to the denominator approaching zero at extremely low gas concentrations, ensuring the robustness of the calculation results. Smoothing can be achieved by using methods such as moving average or exponential smoothing to make the relative changes at low concentrations more stable. The preset substitution strategy can be to set R_rel(t) directly to 0 or a preset minimum value when C(t-Δt) is below the threshold, or to use the absolute gas production rate as the substitution index.

[0107] An ARIMA time series forecasting model is constructed based on historical normal operation data to generate a baseline forecast curve for the corresponding gas concentration. The deviation between the actual and predicted values ​​is calculated, and this deviation is defined as the trend residual feature (TRF) in the form of the L2 norm of the residual vector. Its function is to quantify the degree of deviation of the current operating state from historical statistical patterns, providing a standardized measure of anomaly as one of the statistical features reflecting the intensity of potential abnormal changes. The ARIMA (Autoregressive Integrated Moving Average) model predicts future gas concentrations by analyzing the autocorrelation, trend, and seasonality in historical data. Appropriate p, d, and q parameters need to be selected during its construction. Besides ARIMA, other time series forecasting models, such as exponential smoothing, the Prophet model, or deep learning-based LSTM / GRU models, can also be used to generate the baseline forecast curve. The L2 norm (Euclidean norm) of the residual vector calculates the overall magnitude of the difference between the actual and predicted values.

[0108] The preprocessed raw gas concentration values, IEC three-ratio encoding results, absolute and relative gas production rate features, and trend residual features (TRF) are concatenated in a predetermined order to construct a structured high-dimensional feature vector. The purpose is to provide more comprehensive and richer input information to the primary fault identification model by integrating multi-source and multi-dimensional features, thereby improving the model's recognition capability and accuracy. The preprocessed raw gas concentration values ​​may include normalization and standardization operations. IEC three-ratio encoding is a classic method for transformer fault diagnosis, encoding gas concentration ratios as specific fault types. During concatenation, the order of each feature is ensured to form a fixed-length vector. In addition to the above features, other features related to the transformer's operating state, such as oil temperature, load current, and voltage, can be added to further enrich the feature vector. The concatenated vector serves as the input layer of the model.

[0109] An improved CatBoost model is used as the primary detector on the indicator side. A cost-sensitive loss function is introduced during the model training phase, assigning higher weights to samples with larger Trend Residual Functions (TRFs) to enhance the model's sensitivity to anomalous samples. This cost-sensitive loss function makes the model pay more attention to samples that may represent early or slowly changing faults during training, thereby reducing the risk of false negatives and improving the sensitivity to anomalous changes and the reliability of overall diagnosis. The cost-sensitive loss function can be based on standard loss functions (such as cross-entropy loss), assigning different weights to samples of different categories or different feature values. For samples with larger TRFs, a higher penalty for misclassification can be set. Besides CatBoost, similar cost-sensitive mechanisms can also be introduced into other ensemble learning models (such as XGBoost, LightGBM) or neural network models to achieve weighted learning of specific samples. During the model inference phase, the probability distribution vector P_ind of candidate fault types is output, and the relative contribution of each input feature is calculated. The output probability distribution vector P_ind provides a probability assessment of the fault type, rather than a simple binary judgment, providing more refined information for subsequent deep inference. Calculating feature contributions enhances model interpretability and helps understand the basis of model decisions. The CatBoost model itself supports outputting multi-class probability distributions. Feature contributions can be calculated using SHAP (SHapley Additive ex Planations) values, Permutation Importance, or built-in feature importance evaluation methods (such as those based on Giniimpurity or Gain). These contributions can be presented numerically or in the form of visual charts, helping experts quickly identify key indicators leading to fault determination.

[0110] This application's solution significantly improves transformer fault detection capabilities by introducing more refined gas generation rate analysis and more targeted trend residual feature quantification, combined with multi-source features to construct a more comprehensive input vector. Specifically, the method first calculates the absolute and relative gas generation rates of the key fault gas. These dynamic indicators can more sensitively capture the magnitude and trend of gas concentration changes. In particular, by smoothing or replacing the relative gas generation rate under low concentration conditions, the numerical instability problem is effectively solved, ensuring the reliable extraction of early weak fault signals. Simultaneously, an ARIMA model is constructed based on historical normal operation data, and the trend residual feature (TRF) is calculated, further quantifying the deviation of the current operating state from the normal pattern. Subsequently, these multi-dimensional, high-value features (including pre-processed raw gas concentration, IEC three-ratio coding, absolute and relative gas generation rates, and TRF) are concatenated to form a structured high-dimensional feature vector, providing extremely rich and comprehensive information for the primary fault identification model. Furthermore, an improved CatBoost model is adopted as the primary detector on the indicator side. A cost-sensitive loss function is introduced during the model training phase, assigning higher weights to samples with larger trend residual features (TRF). This strategy allows the model to focus more on key samples that may represent early or gradually changing faults during the learning process, significantly enhancing the model's sensitivity to abnormal changes and effectively reducing the risk of missed detections. During the model inference phase, not only is the probability distribution vector P_ind of candidate fault types output, but the relative contribution of each input feature is also calculated. This provides a quantitative basis for fault determination and enhances the model's interpretability, offering strong support for subsequent deep inference and expert analysis. Through the above mechanisms, this application introduces more refined gas production rate analysis and more targeted trend residual feature quantification on the basis of the original primary detector on the indicator side, and constructs a more comprehensive input vector by combining multi-source features. Simultaneously, by improving the training strategy of the CatBoost model, its sensitivity to early or gradually changing faults is significantly enhanced, effectively solving the problem of missed detection that may exist in the early diagnosis of transformer faults using traditional methods. This approach, which combines multi-dimensional feature fusion with intelligent model optimization, enables primary detection on the indicator side to identify potential transformer faults more accurately and reliably, providing a more solid foundation for subsequent deep reasoning.

[0111] The following is a concrete example to illustrate this. Assume an online monitoring system for a power transformer continuously collects concentration data of dissolved gases (such as acetylene C2H2, methane CH4, hydrogen H2, etc.) in the oil. At a certain time t, the system obtains the current gas concentration C(t) and the gas concentration C(t-Δt) at the previous time t-Δt. First, the system calculates the absolute and relative gas production rates for each gas using the formulas R_abs(t)=[C(t)−C(t−Δt)] / Δt and R_rel(t)=[C(t)−C(t−Δt)] / [C(t−Δt)·Δt]×100%. For example, if C(t-Δt) is lower than a preset threshold (e.g., 0.5ppm), R_rel(t) is smoothed, for example, by using the exponential moving average of the past 5 time steps. Simultaneously, the system utilizes an ARIMA(p,d,q) model trained based on the transformer's historical normal operation data to predict the current baseline gas concentration C_pred(t), and calculates the L2 norm of the residual vector between the actual monitored value C_real(t) and the predicted value C_pred(t), obtaining the trend residual feature (TRF). Subsequently, the preprocessed original gas concentration values ​​(e.g., after Min-Max normalization), IEC three-ratio encoding results (e.g., 0-1-2 representing partial discharge), the calculated absolute gas generation rate, relative gas generation rate, and the trend residual feature TRF are concatenated into a high-dimensional feature vector in a preset order. This feature vector is input into a pre-trained improved CatBoost model. During training, the CatBoost model assigns higher weights to historical fault samples with larger TRF values ​​to ensure that the model can more sensitively capture these early abnormal signals. After model inference, it outputs a probability distribution vector P_ind containing multiple candidate fault types such as partial discharge, overheating, and arcing. For example, P_ind=[0.05,0.85,0.10] indicates that the probability of an overheating fault is 85%. Simultaneously, the model also outputs the relative contribution of each input feature (such as the relative acetylene gas production rate and TRF) to the overheating fault determination. For example, the relative acetylene gas production rate contributes 30%, and the TRF contributes 25%. This helps maintenance personnel quickly understand the key criteria for fault determination.

[0112] Through the above technical solutions, this application can capture the dynamic changes in the operating status of power equipment more comprehensively and precisely in fault detection, especially transformer fault diagnosis. The introduction of absolute and relative gas generation rates, combined with special processing for low-concentration conditions, significantly enhances the sensitivity to early, weak fault signals. Simultaneously, by fusing multi-source features (including original gas concentration, IEC code, gas generation rate, and trend residual features) to construct a high-dimensional feature vector, richer information is provided to the primary detection model. Crucially, a cost-sensitive loss function is introduced during the CatBoost model training phase, and samples with larger trend residual features are given higher weights. This allows the model to focus more on and effectively identify samples that may represent early or slowly changing faults, thereby significantly reducing the risk of missed detections. This improvement not only enhances the ability and accuracy of primary detection on the indicator side to identify potential transformer faults but also provides more reliable and insightful input for subsequent deep inference, thus improving the overall reliability and timeliness of power equipment fault diagnosis.

[0113] This application also proposes a system for implementing the above method, the system comprising an indicator-side data acquisition module, an indicator-side primary detection module, an image-side data acquisition module, an image-side primary detection module, a confidence assessment module, and a large model inference module.

[0114] The system comprises three modules: The indicator-side data acquisition module collects operational indicator data of the equipment under inspection. This module can be a sensor network interface to receive real-time data streams from SCADA systems, DCS systems, or independent sensors (such as online oil chromatographs or partial discharge monitors); or it can be a data interface to read equipment operation logs from a historical database. The indicator-side primary detection module performs preliminary inspections of the equipment under inspection based on the operational indicator data collected by the indicator-side data acquisition module. This module can be an embedded processor or server cluster running a pre-trained machine learning model (such as a CatBoost model) to analyze time-series features and output fault probabilities; or it can be a rule-based engine system that makes preliminary judgments based on preset indicator thresholds and change rates. The image-side data acquisition module collects external image data of the equipment under inspection. This module can be a high-definition camera array deployed around the equipment to capture real-time images of its appearance; or it can be a drone inspection system that periodically captures images of the equipment and transmits them to the processing center. The image-side primary detection module performs preliminary inspections of the equipment under inspection based on the external image data collected by the image-side data acquisition module. This module can be a GPU-accelerated image processing unit running a lightweight object detection model (such as the YOLO series) to identify defects in images (such as oil stains, corrosion, and loose parts); or it can be a system based on image feature extraction and matching, discovering anomalies by comparing them with normal image templates. The confidence assessment module is used to evaluate the probability of failure on the indicator side and the overall confidence level on the image side. This module can be a central processing unit integrating algorithms to calculate the probability of failure on the indicator side and the overall confidence level on the image side, and compare them with preset thresholds; or it can be a decision support system that outputs a comprehensive evaluation result based on weighted fusion of multi-source information. The large model inference module is used for deep inference to perform deep inspection of the equipment under inspection. This module can be a high-performance computing cluster deploying a large language model (LLM) or other deep learning models for cross-modal data fusion, knowledge inference, and conflict resolution; or it can be a cloud-based AI service interface that calls remote large model services via API for complex inference.

[0115] This application's system, through a modular design, achieves effective separation and coordination of data acquisition, preliminary judgment, and deep inference. Specifically, the indicator-side data acquisition module continuously acquires the equipment's operational indicator data and transmits it to the indicator-side preliminary detection module. Simultaneously, the image-side data acquisition module captures the equipment's appearance image and sends it to the image-side preliminary detection module. These two preliminary detection modules work in parallel. The indicator-side preliminary detection module calculates the indicator-side fault probability P_ind based on the operational indicator data, while the image-side preliminary detection module calculates the image-side comprehensive confidence score Conf_img based on the appearance image. Subsequently, the confidence score evaluation module receives these two preliminary detection results and performs a rapid evaluation based on a preset confidence score threshold. If both P_ind and Conf_img are higher than the threshold, the system directly outputs the preliminary detection result, achieving efficient and rapid discrimination. However, if the confidence score on either side is lower than the threshold, it indicates uncertainty in the preliminary detection result. The confidence score evaluation module marks this sample as an uncertain sample and triggers the large model inference module. Once activated, the large-scale model inference module integrates structured input prompts such as primary detection results, rule-based decision information, image semantic labels, and professional context retrieved from a vectorized fault knowledge base. It then performs cross-modal consistency analysis and conflict resolution inference, outputting a revised fault probability distribution. This system architecture ensures reliable fault diagnosis at different confidence levels and efficiently coordinates the processing and decision-making processes of multimodal data.

[0116] As a specific implementation, the indicator-side data acquisition module can use an industrial Ethernet interface to receive data in real time from the transformer oil chromatography online monitoring device, winding temperature sensor, and load current transformer. The indicator-side primary detection module can be an edge computing device equipped with an ARM processor and pre-installed with a CatBoost model to analyze time-series data such as dissolved gas concentration, temperature, and current in the oil, and calculate the fault probability. The image-side data acquisition module can consist of multiple fixed-installation industrial-grade network cameras, transmitting video streams via the RTSP protocol. The image-side primary detection module can be a device equipped with an NVIDIA Jetson series embedded AI computing platform, running an optimized YOLOv5 model to detect visual anomalies such as oil leakage, corrosion, and insulator damage in transformer components such as bushings and radiators in real time. The confidence assessment module can be integrated into the main control PLC or SCADA server, using software logic to determine whether the probabilities and confidence levels output by the two primary detection modules meet preset conditions. When the confidence level of any primary detection result is insufficient, the sample data (including the original indicators, image frames, and primary judgment results) will be sent to the large model inference module. The large model inference module can be a cluster of GPU servers deployed in the cloud, running, for example, a large language model based on the Transformer architecture. This large language model receives structured data through an API interface and combines it with a pre-loaded knowledge graph of power equipment faults and a historical case library to perform in-depth analysis, ultimately outputting a more accurate and interpretable fault diagnosis report.

[0117] Through the aforementioned system architecture, this application effectively implements complex power equipment fault detection methods. The indicator-side data acquisition module and the image-side data acquisition module ensure comprehensive acquisition of multi-source heterogeneous data, providing a foundation for subsequent analysis. The parallel processing of the indicator-side primary detection module and the image-side primary detection module enables the system to quickly and efficiently distinguish most normal or typical abnormal situations, significantly improving detection efficiency. The confidence assessment module, as the hub of intelligent decision-making, can intelligently determine whether to initiate resource-intensive deep inference processes based on the reliability of the primary detection results, avoiding unnecessary computational overhead. The introduction of the large-model inference module provides powerful cross-modal fusion and knowledge reasoning capabilities when the primary detection results are unclear, enabling it to handle complex and ambiguous fault scenarios and output physically reasonable and professionally consistent diagnostic results. Overall, through a modular and hierarchical processing mechanism, this system achieves comprehensive, reliable, and interpretable judgment of the power equipment status, effectively reducing the risk of missed and false alarms and improving the safety and stability of power system operation.

[0118] The following is a concrete example to illustrate this. Suppose a fault detection is performed on a transformer. The initial detection results show a slight increase in the content of acetylene (C2H2) and ethylene (C2H4) in its dissolved gas analysis (DGA) data, but it has not yet reached a clear fault threshold. This leads to ambiguity in the judgment of the specific fault type (such as partial discharge or overheating) by the fault probability P_ind on the indicator side. At the same time, the analysis of the appearance image of the transformer bushing reveals a slight local discoloration, but the overall confidence level Conf_img on the image side is low, making it impossible to clearly diagnose it as bushing flashover or oil contamination. At this point, the system marks this sample as an uncertain sample and switches to the deep inference channel. Before activating the large language model, the system uses physical and engineering rules, such as performing Duval triangle judgment on the DGA data, to determine if it falls within the boundary region of T1 (low-temperature overheating) and D1 (low-energy discharge), forming rule judgment information. Simultaneously, the image analysis module extracts image semantic tags such as "bushing," "discoloration," and "oil contamination." To provide professional contextual support for the large language model, the system first constructs a vectorized knowledge base containing power equipment fault mechanisms, the DL / T722 standard, historical transformer fault cases (including DGA data, visual inspection reports, and expert diagnostic conclusions), and expert experience. Subsequently, the system searches the vectorized fault knowledge base using primary detection results (DGA data ambiguity, image discoloration), rule-based judgment information (Duval triangle T1 / D1 boundary), and image semantic labels (bushing, discoloration, oil stains) as query conditions. The system retrieves historical fault cases highly similar to the current situation, such as "a transformer's DGA data matches the Duval T1 / D1 boundary, and the bushing shows slight discoloration, ultimately diagnosed as insulation aging caused by early partial discharge," along with the corresponding description of the partial discharge fault mechanism. These retrieved historical cases and mechanism descriptions are integrated to form professional context, which is provided to the large language model as part of the structured prompt input. After receiving information including index features, Duval projection results, image semantic consistency scores, rule determination results, and retrieved knowledge context, the large language model performs cross-modal consistency analysis and conflict resolution inference, and finally outputs the corrected fault probability distribution results, such as significantly increasing the probability of early partial discharge and suppressing other fault types with lower probability.

[0119] Through the aforementioned technical solution, this application provides high-quality, highly relevant professional context support for large language models by systematically storing and efficiently retrieving professional knowledge in the field of power equipment faults. This significantly enhances the ability of large language models to perform cross-modal consistency analysis and conflict resolution reasoning in deep inference channels, enabling them to handle complex fault scenarios with unclear initial detection results more accurately and reliably. This solution effectively compensates for the shortcomings of general-purpose large language models in terms of knowledge depth in specific professional fields, ensuring the physical rationality and professional consistency of fault diagnosis results, thereby improving the overall accuracy and reliability of power equipment fault detection.

[0120] The embodiments described in this specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration, characterized in that, Parallel acquisition of operational index data and appearance images of the equipment under inspection; Perform parallel primary inspections on both the indicator side and the image side of the equipment under inspection, and calculate the failure probability on the indicator side and the comprehensive confidence level on the image side of the equipment under inspection. Based on a preset confidence threshold, the failure probability on the indicator side and the overall confidence level on the image side are evaluated. When the failure probability on the indicator side and the comprehensive confidence level on the image side of the device under inspection are both higher than the threshold, the initial detection result is output to achieve efficient and rapid discrimination. If the confidence level on either side is lower than the threshold, the sample is marked as an uncertain sample and switched to the deep inference channel for deep detection. The primary detection involves parallel processing of both the indicator side and the image side.

2. The power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration according to claim 1, characterized in that, It also includes a feature where, when the trend residual features show obvious anomalies but the results of the primary model are unclear, the system marks the sample as an uncertain sample and switches it to the deep inference channel.

3. The power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration according to claim 1, characterized in that, Calculating the failure probability of the equipment under inspection includes: Preprocess the operational indicator data to extract time-series features; The trend residual characteristics are predicted and calculated using the ARIMA model to quantify the degree of abnormal deviation. The trend residual features are input into the CatBoost model for primary fault classification, and the output index is the fault probability.

4. The power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration according to claim 1, characterized in that, The calculation of the overall confidence level of the image side of the device under inspection includes: A lightweight object detection model is used for primary detection, outputting the bounding boxes of candidate objects and the original classification probabilities. An image-side comprehensive confidence score is formed by calculating the image quality score and the knowledge association degree based on the knowledge graph. The image-side comprehensive confidence score is a weighted sum of the image quality score, the knowledge association degree based on the knowledge graph, and the original classification probability. Among them, the knowledge correlation degree is the entity correlation degree calculated based on the equipment fault knowledge graph. It can be modeled by an exponential decay function based on the shortest path length of the entity in the knowledge graph, so as to suppress detection results that obviously violate common sense in engineering.

5. The power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration according to claim 1, characterized in that, In the deep inference channel, before the large language model is activated, the indicator data is processed according to physical and engineering rules to form hard constraints.

6. The power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration according to claim 5, characterized in that, Switching to the deep inference channel for depth detection includes: Activate the large language model; Using preliminary detection results, rule-based judgment information, and image semantic tags as query conditions, the system retrieves the most similar historical fault cases and corresponding mechanism descriptions from the vectorized fault knowledge base to form a professional context. The large language model receives structured input including indicator features, Duval projection results, image semantic consistency scores, rule determination results, and retrieved knowledge context. It performs cross-modal consistency analysis and conflict resolution reasoning, and outputs a corrected fault probability distribution result, maintaining the physical rationality and professional consistency of the reasoning result.

7. The power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration according to claim 5, characterized in that, It also includes constructing a fault coupling matrix M, where the matrix element M_ij represents the correlation strength between fault type i on the index side and fault type j on the image side; Calculate the joint risk value. If the fault types are strongly coupled, they are classified as a compound severe fault. If they are physically mutually exclusive, the confidence levels are compared and low-confidence results are suppressed. If neutral, output multi-source independent fault warning.

8. The power equipment fault detection method based on equipment multimodal model confidence and multi-scale collaboration according to claim 6, characterized in that, It also includes: a structured or vectorized knowledge set that stores power equipment failure mechanisms, industry standards, typical cases and expert experience, which is used to provide professional context support for large model reasoning; using primary detection results, rule judgment information and image semantic tags as query conditions, it retrieves the most similar historical failure cases and corresponding mechanism descriptions in the vectorized failure knowledge base to form a professional context for large model reasoning.

9. A transformer testing method, based on the method described in any one of claims 1-8, comprising: Calculate the absolute and relative gas production rates of the critical fault gas; An ARIMA time series prediction model is constructed based on historical normal operation data to generate a baseline prediction curve for the corresponding gas concentration. The deviation between the actual monitored value and the predicted value is calculated. The deviation is defined as the trend residual feature in the form of the L2 norm of the residual vector. The preprocessed original gas concentration values, IEC three-ratio encoding results, absolute and relative gas production rate characteristics, and trend residual characteristics are concatenated in a predetermined order to construct a structured high-dimensional feature vector, which serves as the input to the primary fault identification model.

10. A system for using the method of any one of claims 1-8, comprising: The indicator-side data acquisition module is used to collect the operating indicator data of the equipment under inspection; The indicator-side primary detection module is used to perform primary detection on the equipment under test from the indicator side based on the operational indicator data collected by the indicator-side data acquisition module. Image-side data acquisition module, used to acquire appearance image data of the device under inspection; The image-side primary inspection module is used to perform primary inspection of the device under inspection from the image side based on the appearance image data acquired by the image-side data acquisition module. The confidence assessment module is used to evaluate the failure probability on the indicator side and the overall confidence level on the image side. The large model inference module is used for deep inference to perform in-depth testing of the device under test.

Images