A cross-modal substation self-evolution fusion detection method and system

Abstract

The application discloses a kind of across modal substation self-evolution fusion detection method and system, method includes: infrared image and visible light image are registered and aligned to obtain image, and it is input to the detection fusion model with infrared and visible light detection branch;And based on the detection result of model, cross-modal causal consistency residual calculation and introduction role switching self-distillation mechanism are carried out, and detection fusion model is optimized;Pseudo hot spot physical model library is constructed, and shielding loss function is set;In the process of optimizing training detection fusion model, shielding loss function shields the suspected pseudo hot spot area identified by pseudo hot spot physical model library;Self-evolution index is set, when self-evolution index exceeds preset threshold, threshold trigger mechanism is triggered, and is updated;And after updating, self-evolution index is recalculated, if self-evolution index does not drop but rise, start rollback mechanism.The application can realize the continuous self-adaptive optimization of multi-modal fusion detection.

Description

Technical Field

[0001] This invention relates to the field of power line inspection image processing technology, specifically to a cross-modal substation equipment self-evolutionary fusion detection method and system. Background Technology

[0002] The reliable operation of power equipment is crucial to the power supply system, thus requiring efficient and accurate monitoring of equipment in substations and other similar scenarios. Traditional manual inspections suffer from low efficiency and strong subjectivity, making machine vision inspection increasingly mainstream. However, each single-modal image monitoring has its limitations: infrared thermal imaging can operate at night and in inclement weather, providing information on equipment temperature distribution, detecting hotspots invisible to the naked eye, and possessing anti-interference and smoke-penetration capabilities. However, infrared images are susceptible to environmental noise, exhibiting low contrast, blurred edges, and low signal-to-noise ratios, and when used alone, they struggle to present details as clearly as visible light images. Visible light images offer high clarity and rich detail, performing well in target recognition under normal conditions, but become almost ineffective in the presence of smoke, rain, snow, or complete darkness. Therefore, fusing visible light and infrared sensor information can leverage their respective strengths to achieve all-weather, multi-scenario equipment status awareness.

[0003] Several methods for monitoring power equipment using infrared and visible light fusion have been developed. For example, multi-scale attention fusion methods can generate fused images to enhance target details. However, these methods primarily focus on image-level fusion display and lack in-depth consideration of the heterogeneity of information between different modalities. Simply multiplying or weighting features without fully utilizing modal complementarity and causal relationships may reduce the effectiveness of the fused features. Furthermore, many existing methods lack intelligent analysis of the results after fusion, failing to effectively identify the true causes of abnormal hotspots, resulting in low diagnostic reliability. Especially in practical applications, infrared-visible light fusion detection still faces the following prominent problems: False hotspots are a common problem in infrared images, where high-temperature areas not caused by equipment malfunctions are mistakenly identified as faults. These false hotspots can be caused by environmental factors (such as direct sunlight reflection or a high-temperature background) or high-temperature components of the equipment during normal operation. They present a high-temperature signal similar to a fault in the infrared image but lack corresponding abnormal behavior in visible light, easily leading to false alarms. Research shows that in long-distance thermal imaging monitoring, the infrared radiation or reflection from high-voltage equipment such as conductors and towers can produce bright areas similar to early fires, causing false alarms. Early dual-modal monitoring systems attempted to utilize the redundancy of infrared-visible light sensing information and combine it with a fuzzy rule base to filter these false alarms. However, traditional methods are cumbersome to formulate rules and have limited generalization capabilities; therefore, false hotspots remain a major challenge in the infrared detection of power equipment.

[0004] Model degradation and lack of continuous updates: Staticly trained fault detection models may experience a gradual performance decline during long-term deployment. This is due to the constantly changing data distribution in real-world scenarios, influenced by seasons, environment, and equipment conditions, resulting in "concept drift." For example, rising ambient temperatures in summer can increase the overall temperature of equipment, causing discrepancies between sensor image features and the original training data; similarly, aging camera equipment can alter image quality. These factors all contribute to a decrease in model detection accuracy. However, most existing fusion models lack online self-updating mechanisms. Once deployed, these models struggle to adapt to new situations, requiring manual collection and offline labeling of new data before retraining—a time-consuming and service-interrupting process. Furthermore, frequent full-scale training faces the challenges of forgetting old knowledge and high computational costs. Models lacking continuous learning capabilities cannot maintain optimal performance in the field for extended periods, failing to meet the requirements of continuous high accuracy and low false alarms in power line inspection.

[0005] In summary, existing technologies still have shortcomings in terms of robustness and continuous evolution of infrared and visible light fusion detection. It is necessary to propose a new method that can distinguish between true and false fault hotspots based on cross-modal causal consistency, and combine this with model self-distillation to achieve intermodal knowledge complementarity. During field operation, small-scale model updates are triggered based on self-evolution indicators, thereby reducing false alarms, slowing model degradation, and ultimately improving the reliability and adaptability of power equipment fault detection. Summary of the Invention

[0006] The technical problem to be solved by this invention is to propose a method that can identify true and false fault hotspots based on cross-modal causal consistency, and combine model self-distillation to achieve intermodal knowledge complementarity. In field operation, the method can trigger small updates of the model based on self-evolution indicators, thereby reducing false alarms, slowing down model degradation, and ultimately improving the reliability and adaptability of power equipment fault detection.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A cross-modal self-evolving fusion detection method for substation equipment, comprising: The infrared image and the visible light image are registered and aligned to obtain image pairs; Image pairs are input into a detection fusion model with infrared and visible light detection branches to obtain the detection results of the infrared and visible light detection branches; cross-modal causal consistency residual calculation is performed based on the comparison of detection results, and a role switching self-distillation mechanism is introduced to optimize the detection fusion model; A pseudo-hotspot physical model library is constructed, and a masking loss function is set. Based on the pseudo-hotspot physical model library, during the process of optimizing the training and detection fusion model, the masking loss function masks the suspected pseudo-hotspot areas identified by the pseudo-hotspot physical model library. Set a self-evolution index as an indicator to describe the online performance and environmental adaptation of the optimized detection fusion model; when the self-evolution index exceeds a preset threshold, trigger the threshold triggering mechanism to update the detection fusion model; When updating the detection fusion model, a restricted parameter update strategy is adopted. After the update is completed, the self-evolution index is recalculated. If the self-evolution index increases instead of decreasing, the rollback mechanism is initiated. Once the update passes the self-evolution index verification, the new detection fusion model is put into online use and serves as the benchmark for the next evolution.

[0008] In this embodiment, cross-modal causal consistency residuals are used as the basis for judging pseudo-faults; a pseudo-fault threshold is set, and if the cross-modal causal consistency residuals exceed the pseudo-fault threshold, an anomaly of inconsistent causal relationships is judged, and pseudo-hot spots may exist.

[0009] In this embodiment, cross-modal causal consistency residual calculation is performed based on the comparison of detection results, including: When the detection result is a scalar, the cross-modal causal consistency residual is obtained using the following formula: ; When the detection results are distributed, the cross-modal causal consistency residuals are calculated using the norm or KL divergence. If multiple target detection results exist, the detection boxes of the infrared detection branch and the visible light detection branch can be matched, and the overall consistency confidence score can be calculated. Based on the confidence score, the cross-modal causal consistency residual of each pair of infrared and visible light images can be calculated and expressed by the following formula: ; In the formula, For cross-modal causal consistency residuals, The detection results are output by the infrared detection branch. The detection results are from the visible light detection branch. To map the expected outcome of visible light based on normal causality, To match the target total number, , The first matching Confidence scores for the infrared detection branch and visible light detection branch of each target.

[0010] In this embodiment, the role-switching self-distillation mechanism is as follows: Based on the features of the input image and the cross-modal causal consistency residual, a reliability score is assigned to both the infrared and visible light modalities. A role-switching self-distillation mechanism is introduced based on the reliability scores of the two modalities, temporarily treating the branch detection of the modality with the high reliability score as the "teacher" and the branch detection of the modality with the low reliability score as the "student". Information is transferred between the teacher and student through knowledge distillation, enabling the student modality to learn the discrimination ability of the teacher modality. Furthermore, the distillation loss of the role switching self-distillation mechanism adopts the KL divergence or mean square error between the teacher output and the student output; by minimizing the distillation loss, the output of the student branch gradually approaches the teacher branch; when the scene changes and the reliability score changes, the roles of teacher and student are dynamically switched.

[0011] In this embodiment, during the optimization of the training and detection fusion model, the masking loss function masks suspected pseudo-hotspot regions identified by the pseudo-hotspot physics model library, including: Based on the thermodynamic characteristics of power equipment and environmental conditions, a pseudo-hotspot physical model library is constructed. This library is used to simulate and generate pseudo-hotspot samples or to identify suspected pseudo-hotspot regions from measured data. During the optimization of the training and detection fusion model, a masking loss function is applied to regions or samples marked as suspected pseudo-hotspots. A masking weight matrix is ​​defined. For each pixel location in the image If it is determined to be a suspected fake hot topic, then... ;otherwise, The masking loss function is then expressed by the following formula: ; In the formula, To shield the loss, For routine detection loss in The value at the pixel.

[0012] In this embodiment, the infrared and visible light detection results output by the optimized detection fusion model are fused. The fusion method adopts decision-level fusion with reliability scores as weights: for each potentially faulty target, the confidence scores of infrared and visible light are combined to calculate the final confidence score, expressed as: ; In the formula, For cross-modal fusion detection results, , These are the reliability scores for the infrared and visible light modes, respectively. , These represent the confidence levels of the infrared and visible light detection branches, respectively.

[0013] In this embodiment, if the cross-modal fusion detection result If the alarm threshold is exceeded, a fault alarm will be output, including the location of the target in the image.

[0014] In this embodiment, the self-evolution index is represented by the following formula: ; In the formula, for The self-evolution index at time t. This is the historical attenuation coefficient. for Cross-modal causal consistency residuals at time step 1 for Features of the detection results output from both infrared and visible light detection branches are constantly fused; For comprehensive Moment and The performance deviation evaluation function.

[0015] In this embodiment, a constrained parameter update strategy is adopted when updating the detection fusion model, including: Only some parameters of the detection fusion model are allowed to be adjusted within a small range, while the remaining parameters are frozen to prevent significant changes from causing the model to forget existing parameters. Simultaneously, regularization constraints on parameter changes are added to the loss function. ; In the formula, To test the total loss function of the fusion model during the self-evolutionary update phase, The loss calculated for routine detection on newly collected samples. For adjustment coefficients, , These are the updated parameter set and the parameters before the update, respectively. It is a norm.

[0016] The present invention also provides a system for applying the above-described cross-modal substation self-evolutionary fusion detection method, comprising: The alignment module is used to register and align infrared images with visible light images to obtain image pairs; The fusion detection module is used to input image pairs into a detection fusion model with infrared and visible light detection branches to obtain the detection results of the infrared and visible light detection branches; and to perform cross-modal causal consistency residual calculation based on the comparison of detection results, and to introduce a role switching self-distillation mechanism to optimize the detection fusion model. The optimization module is used to build a pseudo-hotspot physical model library and set a masking loss function. Based on the pseudo-hotspot physical model library, during the optimization of the training and detection fusion model, the masking loss function masks the suspected pseudo-hotspot areas identified by the pseudo-hotspot physical model library. The self-evolution module is used to set the self-evolution index, which serves as an indicator to describe the online performance and environmental adaptability of the optimized detection fusion model. When the self-evolution index exceeds a preset threshold, a threshold triggering mechanism is activated to update the detection fusion model. The update strategy module is used to update the detection fusion model using a restricted parameter update strategy. After the update is completed, the self-evolution index is recalculated. If the self-evolution index increases instead of decreasing, the rollback mechanism is initiated. Once the update passes the self-evolution index verification, the new detection fusion model is put into online use and serves as the benchmark for the next evolution.

[0017] Compared with the prior art, the beneficial effects of the present invention are: This invention relates to a method for fault detection of power equipment by fusing infrared and visible light images, and introduces a model self-evolution mechanism to achieve continuous adaptive optimization of multimodal image fusion detection in power equipment inspection.

[0018] This invention distinguishes between real faults and false hotspots by identifying the consistency of residuals between visible light and infrared detection outputs. It combines a physical model with a shielding loss function to form an independent discrimination mechanism, proactively identifying false hotspot patterns and shielding their learning. Based on modal reliability, this invention dynamically switches between teacher and student roles in the infrared and visible light branches, achieving cross-modal self-distillation optimization.

[0019] The data flow of this invention is as follows: infrared / visible light image acquisition → image alignment and preprocessing → modal branch detection → consistency residual calculation → reliability evaluation & distillation optimization → decision fusion output; in parallel, the performance monitoring module updates according to the output. This determines whether to enter the model evolution and update process. Through system-level module division and interaction design, this invention achieves deep fusion of infrared and visible light information and self-evolution and upgrading of the model, which can automatically ensure detection accuracy over a long period of time. Attached Figure Description

[0020] Figure 1 This is a flowchart of a cross-modal substation self-evolutionary fusion detection method according to an embodiment of the present invention.

[0021] Figure 2 This is a block diagram of the overall structure of the self-evolutionary fusion detection system for power equipment according to an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of cross-modal causality in an embodiment of the present invention.

[0023] Figure 4 This is a flowchart of the detection method according to an embodiment of the present invention.

[0024] Figure 5 This is a state machine diagram of the self-evolution process in an embodiment of the present invention. Detailed Implementation

[0025] To facilitate understanding of the technical solution of the present invention by those skilled in the art, the technical solution of the present invention will now be further described in conjunction with the accompanying drawings.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0027] Please see Figure 1 As shown, this invention provides a cross-modal self-evolving fusion detection method for substation equipment, comprising: S10, register and align the infrared image with the visible light image.

[0028] In one embodiment of the present invention, the substation inspection robot simultaneously acquires visible light images of the equipment. and infrared images After preprocessing for denoising and scaling, the infrared image is then registered using image registration algorithms, such as feature point-based or template-matching methods. Geometric transformation to visible light image Aligned images Resulting in pixel-level registered image pairs This step ensures that the same device components are aligned in both images, providing a basis for subsequent feature comparison.

[0029] S20: The registered and aligned infrared and visible light images are input into the detection fusion model with infrared and visible light detection branches respectively to obtain the detection results of the infrared and visible light detection branches; and cross-modal causal consistency residual calculation is performed based on the comparison of the detection results, and a role switching self-distillation mechanism is introduced to optimize the detection fusion model.

[0030] In one embodiment of the present invention, dual-branch fault feature extraction: aligning images... Input infrared detection branch network Visible light images Input visible light detection branch The two branches share some convolutional parameters in the backbone layer to extract common structural features, and extract infrared thermal distribution and visible light texture features separately in higher layers, which can be achieved through parameter sharing or feature alignment modules. Each branch has a single detection head: the classification task outputs fault confidence. , The localization task outputs the bounding box and confidence score. , The network architecture of the detection fusion model can adopt an improved dual-branch convolutional structure, such as the ResNet, EfficientNet, or Swin Transformer backbone, and achieve cross-modal alignment through a feature fusion layer.

[0031] In one embodiment of the present invention, cross-modal causal consistency residual calculation: consistency assessment is performed by comparing detection results. and Calculate cross-modal causal consistency residuals As mentioned earlier, when the detection result is a scalar, the cross-modal causal consistency residual is obtained using the following formula: When the detection results are distributed, the cross-modal causal consistency residuals are calculated using the norm or KL divergence. If multiple target detection results exist, infrared and visible light bounding boxes can be matched, the difference for each pair of matches can be calculated, and then the overall consistency score can be obtained. For example, it can be defined as follows: ; In the formula, For cross-modal causal consistency residuals, The detection results are output by the infrared detection branch. The detection results are from the visible light detection branch. To map the expected outcome of visible light based on normal causality, To match the target total number, , The first matching Confidence scores for the infrared detection branch and visible light detection branch of each target.

[0032] Please see Figure 3 As shown in this embodiment, when the device does malfunction, infrared thermal anomalies and visible light anomalies are often causally related. and The residuals should tend to be consistent. The difference is relatively small. Conversely, if infrared detection reveals high temperatures while visible light shows no abnormalities, it may be a false hotspot, and the cross-modal residual will be significantly larger. Therefore, this invention establishes a causal consistency evaluation module in the fusion model, extracting the correlation difference between infrared and visible light features as the consistency residual. This residual quantifies the degree of causal inconsistency between the two modal information, serving as an important basis for identifying false faults. If... If the set false fault threshold is exceeded, it is determined that the current infrared high temperature lacks visible light support, which is an anomaly of inconsistent cause and effect, i.e., a possible false hot spot. The target or the frame is marked as needing special processing.

[0033] Please see Figure 4As shown, in one embodiment of the present invention, a dynamic modal reliability scoring and role-switching self-distillation mechanism are employed: Addressing the differences in reliability of infrared and visible light modal information at different times and in different scenarios, the present invention introduces a role-switching self-distillation mechanism, proposing a dynamic modal reliability scoring method. Based on the features of the current input image and the magnitude of the cross-modal causal consistency residual, a reliability score is assigned to both the infrared and visible light modalities. and (Value range 0 to 1). For example, when the ambient light is good and the device appearance is clear, the visible light modal information is rich and reliable, which can be assigned... Higher values; infrared radiation shows a clear advantage at night or when there is smoke or dust interference, and can provide... Higher values; if inconsistent results between the two modes are detected and A larger value results in a lower score for the abnormal mode. This is based on real-time computation. Modal reliability scoring: This invention introduces a role-switching self-distillation mechanism during model training. Modal branch detections with high reliability scores are temporarily considered "teachers," while those with low reliability scores are temporarily considered "students." Knowledge distillation transfers information between teachers and students, enabling student modalities to learn the discriminative ability of teacher modalities. Distillation loss can be the Kullback-Leibler divergence or mean squared error between teacher and student outputs, for example: ; in and These are represented as branch detections for teachers and students, respectively, for the current input image. The predicted distribution. For Kullback-Leibler divergence, For distillation loss, The KL divergence between the two modalities is calculated. By minimizing the distillation loss, the student's branch detection output gradually approximates the teacher's branch, thus compensating for its shortcomings. When changes in the scenario cause the reliability scores to shift, the system dynamically switches the roles of teacher and student, achieving alternating optimization of dual-branch detection. This role-switching self-distillation learning mechanism allows the two modal models to act as teachers and students to each other, iteratively improving their understanding of the other modality's information, ultimately enhancing the fusion effect.

[0034] More specifically, in this embodiment, the reliability score is assigned based on the historical performance and current consistency of each branch. The specific calculation process includes three steps: Let the reliability score at the previous moment be... First, adjust based on the current residual: If The risk of misjudgment stemming from the infrared detection branch (e.g., a malfunction in infrared detection leading to failure to detect visible light) reduces [the effectiveness of the detection]. improve Conversely, the same applies. Next, a baseline score is determined based on the current image quality metrics. The visible light mode is comprehensively evaluated using image brightness, contrast, and noise level, while the infrared mode is calculated based on temperature range and signal-to-noise ratio. Finally, the residual correction result is weighted and fused with the baseline score and normalized to obtain the comprehensive reliability score for this moment. And it can be standardized to make For example, in clear daytime scenes, the basic reliability of visible light accounts for a weight of 0.7 or more, and if the infrared and visible results are consistent, the scores for both are slightly increased; at night, the basic infrared weight is as high as 0.8 or more.

[0035] More specifically, in this embodiment, the calculated... and This invention performs knowledge self-distillation of role switching in the background detection fusion model training thread. Specifically, it compares the sizes of the two: like If the infrared detection branch is considered the teacher and the visible light detection branch the student, then the infrared detection branch is considered the teacher and the visible light detection branch the student. Taking several recent frames of infrared / visible light features and outputs, a fitting term for the infrared detection branch output is added to the loss function of the visible light detection branch. For example, for classification outputs, the logarithmic probability distribution of the visible light student can approximate the output probability distribution of the infrared teacher, with the loss being: ; In the formula, Distillation loss when the infrared detection branch is considered as the teacher. For infrared detection branch output, pair category The predicted probability, For visible light detection branch output, pair categories The predicted probability.

[0036] like If the visible light detection branch represents the teacher and the infrared detection branch represents the student, then the knowledge distillation training of the infrared detection branch results in the following loss: ; In the formula, Distillation loss when the visible light detection branch is the teacher.

[0037] This setup allows modes with low reliability scores to gradually learn the judgment features of modes with high reliability scores. It's important to note that in real-time systems, the detection fusion model parameters are not drastically adjusted every frame. Instead, distillation updates are accumulated to a certain extent or performed offline to avoid impacting real-time detection speed. After a period of training, the performance of the detection branches corresponding to the students improves. When environmental changes cause a reversal in the relative strength of reliability scores, the original students can then act as teachers, thus achieving cyclical improvement.

[0038] S30. Construct a pseudo-hotspot physical model library and set a masking loss function. Based on the pseudo-hotspot physical model library, during the process of optimizing the training and detection fusion model, the masking loss function masks the suspected pseudo-hotspot areas identified by the pseudo-hotspot physical model library.

[0039] In one embodiment of the present invention, to address false alarms caused by infrared false hotspots, the present invention models the characteristics of false hotspots by combining prior physical knowledge, constructs a physical model library of false hotspots, and introduces a shielding mechanism to reduce interference during the training of the physical model library. First, based on the thermodynamic characteristics of power equipment and environmental conditions, a physical model library of false hotspots is established: for example, uniform temperature rise caused by direct sunlight, localized high temperatures caused by background high-temperature radiation reflection, and common temperature rise patterns of certain components (such as bushing joints) during normal equipment operation. Using these physical models, false hotspot samples can be simulated or suspected false hotspot areas can be identified from measured data. Then, during the training process, a shielding loss function is applied to the areas or samples marked as false hotspots. Specifically, when calculating the detection loss, pixel areas belonging to false hotspots are given lower weights or their error propagation is ignored. For example, a shielding weight matrix is ​​defined. For each pixel position If it is determined to be a pseudo-hot topic, then ,otherwise The improved loss function can then be expressed as: ; In the formula, To shield the loss, For routine detection loss in The value at the pixel. Conventional detection loss includes cross-entropy or IoU loss. Through this masking process, the detection fusion model automatically ignores the contribution of pseudo-hotspots to the loss during training, avoiding the learning of pseudo-hotspot erroneous features as fault modes. Simultaneously, this invention can also add samples generated from a pseudo-hotspot physical model library to the training set as negative samples for reinforcement learning, causing the detection fusion model to tend to output a normal judgment when there is high temperature in the infrared but no visible light anomaly, thereby significantly reducing the false alarm rate caused by pseudo-hotspots.

[0040] In one embodiment of the present invention, the fusion decision output consists of the prediction results updated by each of the distillation-optimized detection fusion models for the current frame. The system then performs a fusion decision on the infrared and visible light detection results. The fusion method can be rule-based or learning-based, for example, using reliability scores as weights in the decision-level fusion: for each potentially faulty target, the confidence scores of both infrared and visible light are combined, and a weighted final confidence score is calculated.

[0041] In the formula, For cross-modal fusion detection results, , These are the reliability scores for the infrared and visible light modes, respectively. , These are the confidence levels of the infrared and visible light detection branches, respectively. If... If the alarm threshold is exceeded, a fault alarm is output, including the target's location in the image (which can be the intersection of two modal detection boxes or the more reliable one), and a corresponding description of the temperature / visible anomaly. In the case of a false hotspot, since no anomaly is detected in the visible light branch and Higher, after fusion This will be lowered, which helps suppress false alarms. Furthermore, a pseudo-hotspot masking strategy is applied during the fusion process: for areas identified as pseudo-hotspots, even if... Slightly high levels may not need to be reported to the alarm to avoid interference.

[0042] S40 sets the self-evolution index as an indicator to describe the online performance and environmental adaptation of the optimized detection fusion model; when the self-evolution index exceeds a preset threshold, the threshold triggering mechanism is triggered to update the detection fusion model.

[0043] In one embodiment of the present invention, an index describing the online performance and environmental adaptability of the detection fusion model is introduced, namely the self-evolution index. This self-evolutionary index is used to quantify the degree to which a fusion model urgently needs evolutionary updates. It can be dynamically calculated by integrating multiple factors, such as the recent false alarm rate, the average value of cross-modal consistency residuals, changes in the model output confidence distribution, and the degree of deviation from the initial training data distribution. For example, the number of false alarms and the residual constant can be used as incremental signals, and the current value can be calculated using an exponential moving average. value: ; In the formula, for The self-evolution index at time t. This is the historical attenuation coefficient. for Cross-modal causal consistency residuals at time step 1 for Features of the detection results output from both infrared and visible light detection branches are constantly fused; For comprehensive Moment and The performance deviation evaluation function.

[0044] If large residuals or false positives persist, the self-evolutionary index... It will gradually increase. When the self-evolutionary index... Exceeding the preset threshold When the threshold trigger mechanism is activated, it indicates that the current detection fusion model is no longer adequately adapted to the current data and should be updated through self-evolution. At this point, the system enters a small-step update process, selecting representative samples from the most recent inspection data (which may include samples manually reviewed in the short term) to update the detection fusion model.

[0045] In this embodiment, the system updates the self-evolutionary index after each fusion decision output. As defined above, the self-evolution index This can be obtained through statistical analysis of observations over a time window. For example, the percentage of false alarms, the average residual level, and changes in model output entropy over the past 100 frames can all be used as calculation components. Assuming that... The output of the constant monitoring is an error alarm (e.g., later manually confirmed as a false alarm) and If it is very large, then the self-evolution index will be large. It increases by a certain amount immediately. Normally, its self-evolutionary index increases periodically. The self-evolutionary index can also decay at a certain rate to prevent the continued impact of outdated errors. When the system operates stably for a long time without false alarms, the self-evolutionary index... It will approach a low value. In this embodiment, the threshold trigger judgment is as follows: when the self-evolution index... It continues to accumulate and eventually exceeds a pre-set threshold. When the threshold is reached, it indicates that the detection fusion model has deteriorated to an unacceptable level, requiring a self-evolutionary update. The threshold can be determined based on the maximum tolerable false alarm rate or the minimum detection rate required on site. For example, if the false alarm rate is required to be less than 1% or the false alarm rate is not to exceed once per day, then the threshold... This corresponds to the exponential critical value exceeding this error level. Once... The system will send a signal to pause the regular detection process and enter the model small-step update state.

[0046] In this embodiment, to ensure the stability and reliability of the update process, the present invention adopts a restricted parameter update strategy: only some parameters of the detection fusion model (such as the last few layers or certain key convolutional kernels) are allowed to be adjusted within a small range, while the remaining parameters are frozen to prevent significant changes from causing the existing ability to be forgotten. Simultaneously, regularization constraints on parameter changes are added to the loss function, for example: ; In the formula, To test the total loss function of the fusion model during the self-evolutionary update phase, The loss calculated for routine detection on newly collected samples. For adjustment coefficients, , These are the updated parameter set and the parameters before the update, respectively. It is a norm.

[0047] In this embodiment, data from a period prior to the trigger point, along with typical error cases appearing near the trigger point (such as repeatedly occurring pseudo-hotspot images), are locked. This data, along with new situations that the original model has recently struggled to correctly identify, forms the fine-tuning dataset. Subsequently, without interrupting monitoring for an extended period, the system performs a rapid model update: loading the parameters of the previously stable model. Using the initial values, a finite number of gradient descent training steps are performed on the selected dataset. Training only exposes local parameters of the model (e.g., the last fully connected layer and the penultimate convolutional layer) for updates, while the remaining layers remain frozen. Furthermore, a small learning rate and the aforementioned parameter regularization terms are used during training to constrain the update magnitude. After training for approximately 100 mini-batch iterations, the updated model parameters are obtained. .

[0048] S50: When updating the detection fusion model, a restricted parameter update strategy is adopted. After the update is completed, the self-evolution index is recalculated. If the self-evolution index increases instead of decreasing, the rollback mechanism is initiated. Once the update passes the self-evolution index verification, the new detection fusion model is put into online use and serves as the benchmark for the next evolution.

[0049] Please see Figure 5 As shown, in one embodiment of the present invention, this constraint ensures that the differences between the old and new detection fusion models are controlled within a certain range. Through a small number of gradient iterations, the detection fusion model completes functional adaptive evolution, obtaining updated parameters. After the update, the system recalculates the validation set or recent data. Values ​​and key performance indicators. If the update fails to achieve the expected results, a rollback condition will be triggered—for example, the false positive rate of the new model increases instead of decreasing during validation, or... If the index rises instead of falling, or even exceeds the second threshold, the rollback mechanism will be immediately activated to restore the detection fusion model parameters to their pre-update state. The update failure is recorded to adjust the next evolution strategy. By setting rollback conditions and mechanisms, it is ensured that self-evolution only takes effect when it is beneficial to performance, avoiding negative impacts of bad updates on the system. Once the update is verified (e.g., a decrease in false alarm rate), Once the error rate is reduced to a safe level, the new model is put into online use and serves as a benchmark for the next evolution.

[0050] In this embodiment, the update effect is verified by using the updated detection fusion model for re-inference on recent data, focusing on verifying whether performance has improved under the problem scenarios that triggered the update. For example, the model is tested on previously frequently falsely reported hotspot images to see if it no longer reports errors; images from normal devices are checked for new false alarms; and the detection rate of existing faulty samples is tested to see if it remains unchanged. If the verification results show that the new detection fusion model improves or remains unchanged in key indicators (e.g., false hotspots are no longer reported, no new false alarms are added under normal circumstances, and the fault detection rate does not decrease), the update is considered successful, and the process proceeds to the next step. If unexpected problems are found after the update (e.g., the number of false alarms for normally normal images increases, indicating a catastrophic oversight), the rollback conditions are met, and the update process is immediately stopped.

[0051] In this embodiment, model deployment or rollback: For updates that pass verification, the system will update the parameters. The old detection fusion model will be officially deployed to replace the old one in subsequent online detection. At the same time, this version of the detection fusion model will be archived and its self-evolution index will be reset. (Or reduce it to reflect that the problem has been alleviated). If the verification fails, the rollback mechanism is initiated: discard. Restore use The detection fusion model continues to run, and logs the update failure (including time, reason, and data sample) in the log. The threshold can also be... Make appropriate adjustments (e.g., slightly increase the level to avoid frequent triggering again in a short period of time) or strengthen the pseudo-hotspot blocking strategy, and try to evolve it again after accumulating more new data.

[0052] In this embodiment, normal inspection is restored: regardless of whether the update is successful or a rollback occurs, the system returns to the normal monitoring state and continues the cycle of the above steps. If the detection fusion model successfully evolves, the new detection fusion model, having learned the latest environmental features, is expected to produce lower false alarms and more accurate detections, thereby... The index will remain at a low level; as time goes on, if performance declines again, the above evolutionary process will repeat.

[0053] Please see Figure 2 As shown, the present invention also provides a system for self-evolving fusion detection of substations using the above-described cross-modal method, comprising: The alignment module is used to register and align infrared images with visible light images to obtain image pairs; The fusion detection module is used to input image pairs into a detection fusion model with infrared and visible light detection branches to obtain the detection results of the infrared and visible light detection branches; and to perform cross-modal causal consistency residual calculation based on the comparison of detection results, and to introduce a role switching self-distillation mechanism to optimize the detection fusion model. The optimization module is used to build a pseudo-hotspot physical model library and set a masking loss function. Based on the pseudo-hotspot physical model library, during the optimization of the training and detection fusion model, the masking loss function masks the suspected pseudo-hotspot areas identified by the pseudo-hotspot physical model library. The self-evolution module is used to set the self-evolution index, which serves as an indicator to describe the online performance and environmental adaptability of the optimized detection fusion model. When the self-evolution index exceeds a preset threshold, a threshold triggering mechanism is activated to update the detection fusion model. The update strategy module is used to update the detection fusion model using a restricted parameter update strategy. After the update is completed, the self-evolution index is recalculated. If the self-evolution index increases instead of decreasing, the rollback mechanism is initiated. Once the update passes the self-evolution index verification, the new detection fusion model is put into online use and serves as the benchmark for the next evolution.

[0054] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0055] The above-described embodiments are merely examples of implementation methods of the invention. The scope of protection of the present invention is not limited to the above-described embodiments. For those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A self-evolving fusion detection method for cross-modal power equipment, characterized in that, include: The infrared image and the visible light image are registered and aligned to obtain image pairs; Image pairs are input into a detection fusion model with infrared and visible light detection branches to obtain the detection results of the infrared and visible light detection branches; cross-modal causal consistency residuals are calculated based on the comparison of detection results, and a role-switching self-distillation mechanism is introduced to optimize the detection fusion model; A pseudo-hotspot physical model library is constructed based on the thermodynamic characteristics and environmental conditions of power equipment. The pseudo-hotspot physical model library includes at least a uniform temperature rise model caused by direct sunlight, a local high temperature model caused by background high temperature radiation reflection, and a temperature rise model of normally operating components of the equipment. Pseudo-hotspot samples are generated or suspected pseudo-hotspot areas are identified using the pseudo-hotspot physical model library. A shielding loss function is set. Based on the pseudo-hotspot physical model library, during the optimization of the training and detection fusion model, the shielding loss function shields the suspected pseudo-hotspot areas identified by the pseudo-hotspot physical model library to suppress false alarms. Set a self-evolution index as an indicator to describe the online performance and environmental adaptation of the optimized detection fusion model; when the self-evolution index exceeds a preset threshold, trigger the threshold triggering mechanism to update the detection fusion model; When updating the detection fusion model, a restricted parameter update strategy is adopted. After the update is completed, the self-evolution index is recalculated. If the self-evolution index increases instead of decreasing, the rollback mechanism is initiated. Once the update passes the self-evolution index verification, the new detection fusion model is put into online use and serves as the benchmark for the next evolution. In addition, the self-evolution index is calculated based on the false alarm count and residual difference constant as incremental signals. Among them, for the phenomenon of false hotspots in infrared images: high-temperature areas that are not caused by equipment failure are mistakenly identified as faults, the true and false fault hotspots are distinguished based on cross-modal causal consistency. Cross-modal causal consistency residuals are used as the basis for judging false faults; a false fault threshold is set. If the cross-modal causal consistency residual exceeds the false fault threshold, an anomaly of inconsistent causal relationship is judged, and there may be a false hotspot. The character switching self-distillation mechanism is as follows: Based on the features of the input image and the cross-modal causal consistency residuals, a reliability score is assigned to both the infrared and visible light modalities. A role-switching self-distillation mechanism is introduced based on the reliability scores of the two modalities, temporarily treating the branch detection of the modality with the high reliability score as the "teacher" and the branch detection of the modality with the low reliability score as the "student". Information is transferred between the teacher and student through knowledge distillation, enabling the student modality to learn the discriminative ability of the teacher modality. The infrared and visible light detection results output by the optimized detection fusion model are fused. The fusion method adopts decision-level fusion with reliability scores as weights: for each potential faulty target, the confidence scores of infrared and visible light are combined and weighted to calculate the final confidence score, and the cross-modal fusion detection result is obtained.

2. The cross-modal substation self-evolutionary fusion detection method according to claim 1, characterized in that, And based on the comparison of detection results, cross-modal causal consistency residuals are calculated, including: When the detection result is a scalar, the cross-modal causal consistency residual is obtained using the following formula: ； When the detection results are distributed, the cross-modal causal consistency residuals are calculated using the norm or KL divergence. If multiple target detection results exist, the detection boxes of the matching infrared detection branch and visible light detection branch can be used to calculate the overall consistency confidence score. Based on the confidence score, the cross-modal causal consistency residual of each pair of infrared and visible light images can be calculated and expressed by the following formula: ； In the formula, For cross-modal causal consistency residuals, The detection results are output by the infrared detection branch. The detection results are from the visible light detection branch. To map the expected outcome of visible light based on normal causality, To match the target total number, , The first matching Confidence scores for the infrared detection branch and visible light detection branch of each target.

3. The cross-modal substation self-evolutionary fusion detection method according to claim 1, characterized in that, The distillation loss of the role-switching self-distillation mechanism is the KL divergence or mean square error between the teacher's output and the student's output. By minimizing the distillation loss, the output of the student branch gradually approaches that of the teacher's branch. When the scenario changes and the reliability score changes, the roles of the teacher and the student are dynamically switched.

4. The cross-modal substation self-evolutionary fusion detection method according to claim 1, characterized in that, During the optimization of the training and detection fusion model, the masking loss function filters out suspected pseudo-hotspot regions identified by the pseudo-hotspot physics model library, including: Based on the thermodynamic characteristics of power equipment and environmental conditions, a pseudo-hotspot physical model library is constructed. This library is used to simulate and generate pseudo-hotspot samples or to identify suspected pseudo-hotspot regions from measured data. During the optimization of the training and detection fusion model, a masking loss function is applied to regions or samples marked as suspected pseudo-hotspots. A masking weight matrix is ​​defined. For each pixel location in the image If it is determined to be a suspected fake hot topic, then... ;otherwise, The masking loss function is then expressed by the following formula: ； In the formula, To shield the loss, For routine detection loss in The value at the pixel.

5. The cross-modal substation self-evolutionary fusion detection method according to claim 3, characterized in that, The final confidence score is expressed as: ； In the formula, For cross-modal fusion detection results, , These are the reliability scores for the infrared and visible light modes, respectively. , These represent the confidence levels of the infrared and visible light detection branches, respectively.

6. The cross-modal substation self-evolutionary fusion detection method according to claim 5, characterized in that, If the cross-modal fusion detection result If the alarm threshold is exceeded, a fault alarm will be output, including the location of the faulty target in the image.

7. The cross-modal substation self-evolutionary fusion detection method according to claim 1, characterized in that, The self-evolutionary index is expressed by the following formula: ； In the formula, for The self-evolution index at time t. This is the historical attenuation coefficient. for Cross-modal causal consistency residuals at time step 1 for Features of the detection results output from both infrared and visible light detection branches are constantly fused; For comprehensive Moment and The performance deviation evaluation function.

8. The cross-modal substation self-evolutionary fusion detection method according to claim 1, characterized in that, When updating the detection fusion model, a constrained parameter update strategy is adopted, including: Only some parameters of the detection fusion model are allowed to be adjusted within a small range, while the remaining parameters are frozen to prevent significant changes from causing the model to forget existing parameters. Simultaneously, regularization constraints on parameter changes are added to the loss function. ； In the formula, To test the total loss function of the fusion model during the self-evolutionary update phase, The loss calculated for routine detection on newly collected samples. For adjustment coefficients, , These are the updated parameter set and the parameters before the update, respectively. It is a norm.

9. A system for applying the cross-modal self-evolutionary fusion detection method for substations according to any one of claims 1-8, characterized in that, include: The alignment module is used to register and align infrared images with visible light images to obtain image pairs; The fusion detection module is used to input image pairs into a detection fusion model with infrared and visible light detection branches to obtain the detection results of the infrared and visible light detection branches; and to perform cross-modal causal consistency residual calculation based on the comparison of detection results, and to introduce a role switching self-distillation mechanism to optimize the detection fusion model. The optimization module is used to build a pseudo-hotspot physical model library and set a masking loss function. Based on the pseudo-hotspot physical model library, during the optimization of the training and detection fusion model, the masking loss function masks the suspected pseudo-hotspot areas identified by the pseudo-hotspot physical model library. The self-evolution module is used to set the self-evolution index, which serves as an indicator to describe the online performance and environmental adaptability of the optimized detection fusion model. When the self-evolution index exceeds a preset threshold, a threshold triggering mechanism is activated to update the detection fusion model. The update strategy module is used to update the detection fusion model using a restricted parameter update strategy. After the update is completed, the self-evolution index is recalculated. If the self-evolution index increases instead of decreasing, the rollback mechanism is initiated. Once the update passes the self-evolution index verification, the new detection fusion model is put into online use and serves as the benchmark for the next evolution.

Images