A real-time visual detection method, system, and storage medium for substation faults

Abstract

This invention discloses a real-time visual detection method, system, and storage medium for substation faults, innovatively constructing a hierarchical early exit mechanism. Through ultra-lightweight early exit branches and dynamic lightweight detection branches, efficient dynamic allocation of computing power is achieved. Deformable convolution and multiple CBAM attention mechanisms are introduced into the complete backbone network, enhancing the network's adaptive feature focusing ability on irregular fault edges. Motion-guided spatiotemporal regularization and temporal deformation alignment modules are introduced. This design overcomes the limitation of traditional algorithms relying solely on single-frame static images by caching the core features of the previous frame and utilizing optical flow motion fields for spatial deformation and alignment, injecting spatiotemporal constraints consistent with the physical evolution of faults into the feature fusion stage. Regarding the improved loss function, a Gaussian penalty term and a smoothing focusing factor are superimposed. Furthermore, a spatiotemporal consistency regularization loss is introduced to construct a novel joint loss function.

Description

Technical Field

[0001] This invention belongs to the field of computer vision detection technology. It relates to a real-time substation fault visual detection method, system and storage medium. Background Technology

[0002] Object detection and tracking technologies have made significant progress in recent years. Among them, the YOLO series of algorithms, with their excellent real-time performance and good detection accuracy, have been widely used in various scenarios, such as autonomous driving, video surveillance, and robotics. The YOLO algorithm transforms the object detection task into a single regression problem, directly predicting bounding boxes and class probabilities from image pixels through a neural network, thus achieving efficient object detection. Its core idea is to use a single neural network structure to extract features and predict objects from the entire image, avoiding the complex process of generating candidate regions and then classifying them in traditional two-stage object detection methods (such as Faster R-CNN), significantly improving detection speed and enabling it to meet the needs of applications with high real-time requirements.

[0003] However, when these existing object detection algorithms are applied to real-time substation fault detection tasks, some problems and shortcomings arise. Most existing object detection algorithms are designed for general object detection tasks and are insufficiently adaptable to the specific needs of substation fault detection scenarios. For example, substation equipment fault modes are complex and diverse, including partial discharge, insulation damage, and equipment overheating. These faults may manifest visually as subtle texture changes, inconspicuous color differences, or specific morphological patterns, and existing object detection algorithms have limited ability to capture these subtle features. In real-time substation fault detection, the operating status of substation equipment needs to be monitored for extended periods. Existing algorithms lack the ability to correlate features in long-term data sequences and struggle to fully utilize the temporal redundancy in video data, thus affecting the accurate identification of fault modes.

[0004] Furthermore, driven by deep learning, Joint Detection and Embedding (JDE) models have emerged. These models integrate object detection and Re-ID feature extraction into a unified network framework. Through feature sharing and joint optimization, they significantly reduce computational overhead and improve model efficiency and practicality. Nevertheless, existing JDE models still have some limitations in substation fault detection tasks. Their network structure has limited ability to represent substation equipment fault features and cannot fully exploit the differences between fault features and normal states. Simultaneously, in real-time substation fault detection tasks, it is necessary to maximize detection speed while ensuring detection accuracy to meet real-time monitoring requirements, and existing JDE models still have room for improvement in this regard.

[0005] In existing technologies, real-time target detection and multi-target tracking solutions, represented by YOLO11-JDE, face three major technical bottlenecks when applied to real-world substation fault visual detection tasks: (1) Severe computational redundancy and lack of dynamic computing power allocation capability: The existing model adopts a fixed resolution and single-depth inference structure. In the 24 / 7 long-term video monitoring of substations, the vast majority (more than 85%) of the images are in a static and normal state without faults. Performing high-resolution, full-depth forward inference on all consecutive frames indiscriminately will lead to extremely serious waste of computing resources and make it difficult to meet the industrial-grade high real-time performance (FPS) requirements under the limited computing power at the edge.

[0006] (2) Lack of temporal context makes it easy for minor faults to flicker and go undetected: Substation faults (such as minor arcs, partial discharges, and insulator damage) are mostly non-rigid and small targets, and are easily affected by the complex on-site environment. Traditional single-frame detection algorithms lack physical continuity modeling across frames; while existing data association algorithms are mainly for macroscopic rigid body motion, which is difficult to deal with the spatiotemporal association of irregular minor faults, resulting in flickering detection boxes and broken trajectories in the video stream.

[0007] (3) Traditional loss functions are insufficient for the localization of small targets in substations and for constraining difficult-to-distinguish samples: In substation scenarios, the categories are extremely imbalanced and the features of small targets are ambiguous. Traditional loss functions (such as standard CIoU and triplet loss) are less sensitive to the subtle center point offset of small fault bounding boxes and fail to effectively focus on fault features that are difficult to distinguish, resulting in insufficient discriminative power in the feature space. Summary of the Invention

[0008] The purpose of this invention is to provide a real-time substation fault visual detection method, system, and storage medium. In the YOLO11-JDE network structure, a hierarchical early exit mechanism is innovatively constructed. Through ultra-lightweight early exit branches and dynamic lightweight detection branches, a large number of fault-free simple background frames can be classified and exited with high confidence in shallow networks. This effectively avoids the waste of computing resources caused by the indiscriminate execution of complete forward inference in traditional fixed-depth networks, thereby significantly improving the real-time monitoring frame rate of the system under limited computing power.

[0009] The technical solution to achieve the purpose of this invention is as follows: A real-time visual detection method for substation faults includes the following steps: Acquire images of the substation; A real-time substation fault visual detection network is constructed, which introduces an early exit branch and a dynamic lightweight detection branch based on YOLO11-JDE. The input image is first fed into the early exit branch for rapid classification. If the confidence level of the early exit branch is insufficient, the dynamic lightweight detection branch is activated for multi-scale rapid detection. If the confidence level of the dynamic lightweight detection branch is insufficient, the backbone network is activated. Deformable convolution and multiple CBAM attention mechanisms are introduced into the backbone network to enhance the network's adaptive feature focusing ability on irregular fault edges. The backbone network extracts and retains shallow and mid-level features that have not triggered the early exit mechanism and inputs them into the PANet module for multi-scale path aggregation. The features are spatially deformed and aligned, and then input into the PANet module to fuse with multi-scale features to obtain a composite feature map. The composite feature map is then split and input into the detection head and re-identification head to obtain the detection result. During the training phase, the loss function is calculated, and the model parameters are fine-tuned to obtain the trained real-time substation fault visual detection network. A trained real-time substation fault visual detection network is used to detect fault images in substations and obtain fault detection results.

[0010] In the preferred technical solution, the early branch exit processing includes: Image data is sequentially processed through a Focus module and convolutional operations for preliminary feature extraction. The extracted preliminary feature maps are then continuously fed into three cascaded Ghost bottleneck layers to extract lightweight features. The lightweight feature maps are compressed into vectors using global average pooling and fed into an optimized fully connected layer. The output is the confidence score for fault and non-fault predictions for the current frame. The confidence scores are then evaluated, and if a set threshold is reached, early exit is triggered directly, and a binary fault label is output.

[0011] In the preferred technical solution, the dynamic lightweight detection branch performs multi-scale rapid detection, including: The input feature map is sequentially passed through convolution operations, pointwise convolution operations, and the CBAM attention mechanism module to enhance and focus shallow channels and spatial details. The enhanced feature map enters a series combination module that repeats the loop three times. Each combination module consists of a Ghost bottleneck layer and a CBAM attention mechanism module connected in series. The feature map output from the combination module is connected to the detection head and the confidence score is output. If the confidence score reaches the set threshold, the dynamic lightweight detection result is output and the process exits.

[0012] In the preferred technical solution, the backbone network processing includes: The input feature map passes through the Focus module and convolution operation, CBAM attention mechanism, Ghost bottleneck layer and CBAM attention mechanism in sequence. Then, the geometric features of irregular faults are captured by the deformable convolution module. Subsequently, the deep features are passed through multiple sets of Ghost C3 modules and CBAM attention mechanism in alternating combinations, and the receptive field is expanded by the spatial pyramid pooling module to obtain deep features. In the final feature fusion and output stage, the multi-scale features extracted by the backbone network at different stages are extracted and fed into the PANet module for spatial scale stitching. Simultaneously, the network writes the optimal features of the current frame into the temporal and optical flow cache module of the previous frame through skip connections. For inference of the current frame, the temporal and optical flow information of the previous frame stored in the cache module, together with the deep features generated by the current backbone network, enter the motion-guided spatiotemporal regularization module. The motion-guided spatiotemporal regularization module calculates and outputs the motion-guided field, and controls the temporal deformation alignment module to perform physical-level spatial deformation and alignment of the temporal and optical flow information of the previous frame. The aligned historical enhancement features are fed into the PANet module. Finally, the PANet module fuses the multi-scale features and the aligned historical enhancement features to obtain a composite feature map, which is then split and input to the detection head and the re-identification head.

[0013] In the preferred technical solution, the calculation process of the motion-guided spatiotemporal regularization module includes: During each forward inference step, the original substation fault image at the current moment is first obtained. Image from the previous time step in the cache Low-resolution image pairs are generated through downsampling operations. and The image pairs are fed into a lightweight optical flow network for optical flow estimation, and a coarse optical flow field reflecting the pixel-level relative motion trend between the two frames is calculated and output. The coarse optical flow field is further interpolated using bilinear interpolation to obtain the standard motion field. Input the standard sports field into the sports field size adjustment module, and output the characteristic-level sports field. ; After completing the construction of the sports field, the network enters the feature space alignment stage: extracting the feature map of the current frame generated by the backbone network. And retrieve the cached feature map of the previous frame. , the feature map of the previous frame With characteristic level sports field Both inputs are fed into the feature alignment module, which uses the pixel-level offset vector (u, v) provided by the motion field, where u is the horizontal offset component and v is the vertical offset component. Perform spatial deformation and resampling operations to force the features from the previous time step to align with the current time step in the spatial coordinate system, and output the aligned feature map of the previous frame. In the c Feature intensity response values ​​for each channel at coordinates (x, y):

[0014] in, c For the channel index of the feature map; and These represent the horizontal and vertical spatial offset components, respectively, predicted by the feature-level motion field and located at the current coordinates (x, y). Current frame feature map The subsequent flow is controlled by the logic decision nodes activated on demand. If the model is in the inference phase and the preceding early exit branch has made a high-confidence non-fault determination for the current frame, the feature map of the current frame... This will directly trigger the bypass mechanism, skipping subsequent feature comparison and regularization calculations, and sending the data to the detection head to complete the prediction; conversely, if it is in the model training phase or encounters a difficult frame, the current frame feature map... The aligned feature map from the previous frame A common input mean squared error calculation node is used to calculate the representation difference between the two at the same spatial location, generating a spatiotemporal consistency regularization loss. .

[0015] In the preferred technical solution, the spatiotemporal consistency regularization loss for:

[0016] Where C represents the total number of channels in the feature map, and H' and W' represent the spatial height and spatial width of the deep feature map, respectively; Represents the feature map of the current frame The characteristic intensity response value at the c-th channel with coordinates (x, y); This represents the feature map of the previous frame after optical flow alignment. The characteristic intensity response value at the same channel and coordinates.

[0017] In a preferred embodiment, the loss function incorporates a Gaussian penalty term and a smoothing focusing factor, and includes a spatiotemporal consistency regularization loss. The total loss function is:

[0018] in, Represents the improved prediction box regression loss; For classification loss and; For confidence loss; For spatiotemporal consistency regularization loss, , , and These represent the weight hyperparameters of the corresponding loss terms; The improved prediction box regression loss is defined as:

[0019] Where, λ Gau For superparameters, For Gaussian penalty terms, For the regression loss of the predicted bounding box;

[0020] in, ) represents the prediction box With real frame The Euclidean distance between the center points; σ is the scaling parameter of the Gaussian function;

[0021] in, This represents the intersection-over-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. This is the distance penalty term; ρ is the Euclidean distance from the center point; c It is the diagonal distance of the smallest bounding rectangle that simultaneously contains both the predicted bounding box and the ground truth bounding box; Used to constrain the aspect ratio of the predicted bounding box and the ground truth bounding box. , w and h are the width and height of the prediction box, respectively. and These represent the width and height of the actual bounding box, respectively.

[0022] This invention also discloses a real-time substation fault visual detection system for implementing the aforementioned real-time substation fault visual detection method, comprising: The image acquisition module acquires images of the substation. A module for constructing a real-time substation fault visual detection network is provided. This network, based on YOLO11-JDE, introduces an early exit branch and a dynamic lightweight detection branch. The input image is first fed into the early exit branch for rapid classification. If the confidence level of the early exit branch is insufficient, the dynamic lightweight detection branch is activated for multi-scale rapid detection. If the confidence level of the dynamic lightweight detection branch is insufficient, the backbone network is activated. Deformable convolution and multiple CBAM attention mechanisms are introduced into the backbone network to enhance its adaptive feature focusing ability on irregular fault edges. The backbone network extracts and retains shallow and mid-level features that have not triggered the early exit mechanism and inputs them into the PANet module for multi-scale path aggregation. The features are spatially deformed and aligned, and then input into the PANet module to fuse with multi-scale features to obtain a composite feature map. This composite feature map is then split and input into the detection head and re-identification head to obtain the detection results. During the training phase, the loss function is calculated, and the model parameters are fine-tuned to obtain the trained real-time substation fault visual detection network. The fault detection module uses a trained real-time substation fault visual detection network to detect fault images in the substation and obtain fault detection results.

[0023] The present invention also discloses a computer storage medium storing a computer program thereon, wherein when the computer executes the computer program, it implements the real-time substation fault visual detection method described in any of the above claims.

[0024] The present invention also discloses an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor runs the computer program stored in the memory, wherein the computer program, when executed, implements the real-time substation fault visual detection method described in any of the above claims.

[0025] Compared with the prior art, the significant advantages of this invention are: 1. This invention, through systematic innovation in network structure, loss function, and spatiotemporal modeling, achieves improvements in efficiency, accuracy, and robustness in real-time substation fault detection compared to existing technologies represented by YOLO11-JDE. At the network architecture level, an early exit branch and a dynamic resolution-aware inference mechanism are introduced. The early exit branch performs fast classification of simple samples (such as normal equipment frames) through a 320×320 lightweight subnet. Actual testing shows that 85% of normal frames do not require activation of the complete backbone network, reducing computation by more than 35% and increasing inference speed to 70-80 FPS. Simultaneously, through cross-frame feature reuse (Temporal Warp) and deformable convolution, the recall rate for the approximately 20% of minor faults in substation scenarios is improved by 28%, and the false alarm rate is reduced by 41%. The collaborative mechanism between the dynamic resolution branch and the complete backbone network enables the model to adaptively switch computational complexity. In complex occlusion or multi-scale fault scenarios, the detection accuracy for medium and large faults above 40×40 pixels is improved by 19.2% compared to YOLO11-JDE.

[0026] 2. Regarding the improvement of the loss function, by superimposing a Gaussian penalty term and a smoothing focusing factor, the problems of class imbalance and blurred features of small targets in substation faults are addressed. The Gaussian penalty term reduces the mean square error of the predicted box center point offset, especially for small targets with a diagonal length of pixels, reducing the center localization error from an average of 18 pixels to 11 pixels. The smoothing focusing factor, by dynamically adjusting the weights of difficult samples, reduces the classification loss of medium-difficulty fault samples by 34%, significantly improving the ability to distinguish similar fault modes such as insulation damage and partial discharge.

[0027] 3. The spatiotemporal consistency regularization module uses optical flow-guided feature alignment and temporal constraints to force the feature difference between adjacent frames to be less than a threshold during training, effectively filtering out occasional noise such as pole reflections and raindrops, reducing the false detection rate by 38%. At the same time, for real faults that last for more than 2 frames, such as arcs, the feature consistency loss is maintained, increasing the trajectory integrity rate from 56.37% to 71.84%. Attached Figure Description

[0028] Figure 1 This is a flowchart of the real-time substation fault visual detection method in this embodiment; Figure 2 This is a diagram of the YOLO-DynLite-EE architecture for real-time substation fault visual detection network in this embodiment. Figure 3 This is a structural diagram of the motion-guided spatiotemporal regularization module in this embodiment; Figure 4 This is a graph showing the detection results of this embodiment. Detailed Implementation

[0029] The principle of this invention is as follows: The YOLO-DynLite-EE network structure innovatively constructs a hierarchical early exit mechanism. Through ultra-lightweight early exit branches and dynamic lightweight detection branches, a large number of fault-free simple background frames can be classified and exited with high confidence in a shallow network, effectively avoiding the waste of computational resources caused by the indiscriminate execution of complete forward inference in traditional fixed-depth networks. Deformable convolution and multiple CBAM attention mechanisms are introduced into the complete backbone network to enhance the network's adaptive feature focusing ability on irregular fault edges. To address the problem of false alarms easily caused by complex substation environments, motion-guided spatiotemporal regularization (MG-STR) and a temporal deformation alignment module are introduced. This design overcomes the limitation of traditional algorithms relying solely on single-frame static images by caching the core features of the previous frame and using optical flow motion fields to spatially deform and align them, injecting spatiotemporal constraints that conform to the physical evolution of faults into the feature fusion stage. Regarding the improvement of the loss function, a Gaussian penalty term and a smoothing focusing factor are superimposed. Furthermore, the spatiotemporal consistency regularization loss of the MG-STR module is introduced to construct a novel joint loss function. These improvements enable the model to more accurately identify and locate faults in real-time substation fault visual detection tasks, while maintaining a high inference speed.

[0030] Example: like Figure 1 As shown, a real-time visual detection method for substation faults includes the following steps: Acquire images of the substation; A real-time substation fault visual detection network is constructed, which introduces an early exit branch and a dynamic lightweight detection branch based on YOLO11-JDE. The input image is first fed into the early exit branch for rapid classification. If the confidence level of the early exit branch is insufficient, the dynamic lightweight detection branch is activated for multi-scale rapid detection. If the confidence level of the dynamic lightweight detection branch is insufficient, the backbone network is activated. Deformable convolution and multiple CBAM attention mechanisms are introduced into the backbone network to enhance the network's adaptive feature focusing ability on irregular fault edges. The backbone network extracts and retains shallow and mid-level features that have not triggered the early exit mechanism and inputs them into the PANet module for multi-scale path aggregation. The features are spatially deformed and aligned, and then input into the PANet module to fuse with multi-scale features to obtain a composite feature map. The composite feature map is then split and input into the detection head and re-identification head to obtain the detection result. During the training phase, the loss function is calculated, and the model parameters are fine-tuned to obtain the trained real-time substation fault visual detection network. A trained real-time substation fault visual detection network is used to detect fault images in substations and obtain fault detection results.

[0031] Specifically, this embodiment designs a real-time substation fault visual detection network YOLO-DynLite-EE (Dynamic Lightweight YOLO with Early-Exit).

[0032] The YOLO-DynLite-EE designed in this invention, compared to YOLO11-JDE, only adds a Re-ID branch to a fixed structure and performs joint detection and tracking. This scheme significantly reduces the reliance on large-scale network forward propagation through Early-Exit extreme pruning, Dynamic Resolution perceptual inference, and cross-resolution feature reuse. This ensures that most frames can be detected using only lightweight subnets, and the full backbone network is only activated when truly challenging frames are encountered. Furthermore, combined with deep temporal feature alignment and deformable convolution, it provides adaptive compensation for rapid movement and occlusion in substation scenarios.

[0033] like Figure 2 As shown, the YOLO-DynLite-EE network structure proposed in this invention innovatively constructs a hierarchical early exit mechanism to address the characteristic that the vast majority of images in long-term substation monitoring videos are in normal operating conditions. Through ultra-lightweight early exit branches and dynamic lightweight detection branches, a large number of fault-free, simple background frames can be classified and exited with high confidence in a shallow network layer. This effectively avoids the waste of computational resources caused by the indiscriminate execution of complete forward inference in traditional fixed-depth networks, thereby significantly improving the real-time monitoring frame rate of the system under limited computing power. Secondly, facing the technical challenges of complex and varied substation fault morphologies, many of which are small targets (such as minute arcs, partial discharges, and insulator damage), this invention introduces deformable convolution and multiple CBAM attention mechanisms into the complete backbone network, enhancing the network's adaptive feature focusing ability on irregular fault edges. At the same time, the network extracts and retains shallow and mid-level features that have not triggered the early exit mechanism, and inputs them into the PANet module for multi-scale path aggregation, achieving complementary advantages between shallow high-resolution spatial details and deep semantic information, and significantly improving the detection recall rate of weak substation faults.

[0034] In a preferred embodiment, to address the issue of false alarms easily caused by the complex environment of substation sites, a motion-guided spatiotemporal regularization (MG-STR) and temporal deformation alignment module is introduced. This design overcomes the limitation of traditional algorithms that rely solely on single-frame static images. By caching the core features of the previous frame and utilizing optical flow motion fields to spatially deform and align them, spatiotemporal constraints conforming to the physical evolution of faults are injected during the feature fusion stage. This mechanism not only effectively filters out sporadic visual noise during the inference stage but also enhances the continuity and consistency of real fault features over time, ultimately achieving high real-time performance, high accuracy, and strong robustness in end-to-end fault detection under complex and ever-changing substation scenarios.

[0035] Specifically, such as Figure 2 As shown, when a substation fault image is fed into the network, it first enters the ultra-lightweight Early Exit branch. The image data undergoes initial feature extraction via a Focus module and convolutional operations, followed by three consecutive Ghost bottleneck layers to continuously extract lightweight features with extremely low computational cost. The feature maps are then compressed into vectors using global average pooling and fed into an optimized fully connected layer, outputting the "fault" and "non-fault" prediction confidence scores for the current frame. The system judges these confidence scores; if they reach a set threshold, it directly triggers early exit, outputting a binary fault label, thus skipping all subsequent complex detection processes and greatly shortening the computation path for most normal frames.

[0036] If the confidence score of an early exit branch is insufficient, the network will activate further and enter the Dynamic Small Detection branch. In this branch, features are sequentially processed through convolutions, pointwise convolutions, and the CBAM attention mechanism to enhance and focus on shallow channels and spatial details. Subsequently, the features enter a three-cycle Ghost bottleneck layer combined with the CBAM attention mechanism to further extract richer local perceptual information. This branch terminates directly with a lightweight detection head and outputs the confidence score. If the confidence score meets the requirements, the network outputs the Dynamic Small Detection result and exits. During this process, the features extracted in previous branches are not discarded but are passed to deeper layers via cross-layer data flow for potential feature reuse later.

[0037] When the confidence level of the dynamic lightweight detection branch still does not meet the requirements, the network will fully activate the bottommost complete backbone network and multi-scale fusion system for high-precision inference. In the complete backbone network, features sequentially pass through the Focus module + convolution, the CBAM attention mechanism, the Ghost bottleneck layer, and another CBAM attention mechanism. To cope with the variability of substation fault morphologies, the network then introduces a deformable convolution module to adaptively capture the geometric features of irregular faults (such as arc deformation). Subsequently, deep features pass through multiple alternating combinations of Ghost C3 modules and the CBAM attention mechanism, and then through the Spatial Pyramid Pooling (SPP) module to significantly expand the receptive field, obtaining highly discriminative deep semantic features.

[0038] In the final feature fusion and output stage, the network is designed with multi-scale and spatiotemporal closed-loop logic. Multi-scale features (shallow / medium-level features) extracted by the backbone network at different stages are extracted and directly fed into PANet (Path Aggregation Network) for spatial scale concatenation. Simultaneously, the network writes the optimal features of the current frame into the "previous frame's temporal and optical flow" cache module via skip connections. For inference of the current frame, the temporal and optical flow information of the previous frame stored in the cache, along with the deep features generated by the current backbone network, enters the Motion Guided Spatiotemporal Regularization (MG-STR) module. The specific structure of the MG-STR module is as follows... Figure 3 As shown, the MG-STR module calculates and outputs the motion guidance field, controlling the temporal deformation alignment module on its left to perform physical-level spatial deformation and alignment of the cached features from the previous frame. The aligned historical enhanced features are then fed into PANet. Finally, PANet fully fuses the multi-scale backbone features with the temporal deformation alignment features, and then splits the composite feature map into the detection head and the re-identification head (Re-ID head), jointly outputting high-precision substation fault detection boxes, category labels, and spatiotemporally consistent target re-identification results, completing the closed loop from feature extraction, hierarchical exit to spatiotemporal joint decision-making.

[0039] Specifically, in real-time substation fault visual detection tasks, minor faults such as insulation damage, partial discharge, and arcing often exhibit specific physical spatiotemporal continuity, meaning that the fault source point typically remains stable in position or experiences only minor spatial shifts within consecutive video frames. However, traditional single-frame target detection algorithms lack the ability to correlate features in long-term data sequences and are highly susceptible to interference from the complex environment of substation sites (such as metal reflections from poles, raindrop splashes, and image noise), leading to frequent false alarms and insufficient ability to identify weak faults. To address the bottlenecks in temporal feature correlation and consistency maintenance in existing technologies, this network architecture innovatively introduces a spatiotemporal consistency regularization module based on optical flow after the feature extraction network and before the detection head. The core function of this module is to introduce kinematic constraints that conform to the physical laws of real faults at the feature representation level. By applying spatiotemporal consistency loss during the training phase, the network is forced to learn and maintain the physical continuity of real fault features over time, thereby effectively filtering out occasional noise, reinforcing real minor fault signals, and enhancing the model's global understanding of fault modes without significantly increasing inference time.

[0040] The data flow of this module consists of three collaborative stages: motion field estimation, feature space alignment, and dynamic routing and regularization. For example... Figure 3 As shown, the processing procedure of the motion-guided spatiotemporal regularization module is as follows: During each forward inference, the network first acquires the original substation fault image at the current moment. Image from the previous time step in the cache To ensure the inference speed requirements of real-time monitoring of substations, these two high-resolution 640×640 images were pre-downsampled to generate low-resolution image pairs. and Subsequently, the downsampled images are fed into a Lightweight Optical Flow (LF) network for optical flow estimation, which calculates and outputs a coarse optical flow field reflecting the pixel-level relative motion trend between the two frames. To eliminate the spatial quantization error caused by reduced resolution, the coarse optical flow field is further recovered to a finer scale using bilinear interpolation to obtain the standard motion field. To ensure spatial dimension matching, the motion field then enters the motion field size adjustment module, where necessary upsampling and downsampling operations are performed to output a feature-level motion field that strictly matches the spatial resolution of the deep feature map. After completing the sports field construction, the network enters the feature space alignment stage. The system extracts the feature map of the current frame generated by the backbone network. And retrieve the cached feature map of the previous frame. To establish a physical mapping across frames, the feature map from the previous frame... With characteristic level sports field The input is fed into the feature alignment (Warp) module. This module uses the pixel-level offset vector (u,v) provided by the motion field, where the horizontal offset component u represents the spatial displacement of the pixel along the horizontal axis between the current frame and the previous frame, and the vertical offset component v represents the spatial displacement of the pixel along the vertical axis. Together, they constitute the motion vector of the pixel in the temporal grid.

[0041] right Perform spatial deformation and resampling operations to force the features from the previous time step to align with the current time step in the spatial coordinate system, and output the aligned feature map of the previous frame. In the c Feature intensity response values ​​for each channel at coordinates (x, y):

[0042] in, Indicates the network in the previous frame The extracted feature map; c is the channel index of the feature map; and These represent the horizontal and vertical spatial offset components, respectively, predicted by the feature-level motion field and located at the current coordinates (x, y).

[0043] Finally, the feature map of the current frame The subsequent flow is strictly controlled by the "on-demand activation" logical decision node. If the model is in the inference phase and the preceding early-exit branch has made a high-confidence "non-fault" determination for the current frame, the current frame feature map... This will directly trigger a bypass mechanism, skipping subsequent feature comparison and regularization calculations, and sending the data to the detection head with zero latency to complete the prediction, thus saving computational power. Conversely, if the model is in the training phase or encounters a difficult frame, the current frame feature map... The aligned feature map from the previous frame A common input mean squared error (MSE) calculation node. This node calculates the representation difference between the two at the same spatial location, generating a spatiotemporal consistency regularization loss. .

[0044]

[0045] Where C represents the total number of channels in the feature map; H' and W' represent the spatial height and spatial width of the deep feature map, respectively; This represents the feature intensity response value of the current frame feature map at the c-th channel with coordinates (x, y). This represents the feature intensity response value of the previous frame's feature map at the same channel and coordinates after optical flow alignment.

[0046] This loss term participates in backpropagation as part of the total loss function, penalizing noise responses that cannot remain consistent over time, and enabling the network to accurately capture real fault characteristics with physical continuity in complex substation scenarios.

[0047] It should be noted that this embodiment uses optical flow + feature warp + mean square error constraint for the spatiotemporal consistency regularization module. Alternatively, a "temporal transformer" can be used, adding a lightweight multi-head between the backbone and the detection head. Neither of these alternatives requires explicit calculation of the optical flow field, yet they can still capture the temporal consistency of fault features to a certain extent, although they introduce different computational overhead and implementation complexity.

[0048] A preferred implementation method is to improve the loss function as follows: While retaining the advantages of the original YOLO11-JDE loss function, a Gaussian penalty term and a smoothing focusing factor are introduced. These improvements enhance the model's ability to learn and distinguish substation fault features. The Gaussian penalty term helps the model generate a more concentrated distribution in the feature space, improving the ability to distinguish different fault modes. The smoothing focusing factor makes the model pay more attention to fault features that are difficult to distinguish, enhancing the model's adaptability and robustness to complex scenarios. Furthermore, a novel joint loss function is constructed by introducing the spatiotemporal consistency regularization loss from the MG-STR module. These improvements enable the model to more accurately identify and locate faults in real-time substation fault visual detection tasks while maintaining high inference speed.

[0049] Specifically, in YOLO11-JDE, the bounding box regression loss adopts the CIoU (Complete-IoU) form, which simultaneously penalizes IoU, center point distance, and aspect ratio differences, while classification and confidence scores use binary cross-entropy. Building upon this, by improving the regression and classification losses respectively, the sensitivity to small targets or subtle displacements is enhanced, and the weights of easily classified and difficult-to-classify samples are balanced. This maintains a high frame rate while significantly improving the accuracy of capturing details such as minute sparks and arcs in substation fault visual detection. Specifically, let the predicted bounding box and the ground truth bounding box be... and The CIoU loss is:

[0050] in, These represent the predicted bounding box and the ground truth bounding box, respectively. This represents the intersection-over-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. ρ is the distance penalty term; ρ is the Euclidean distance of the center point; c is the diagonal distance of the smallest bounding rectangle that simultaneously contains the predicted box and the ground truth box. Used to constrain the aspect ratio of the predicted bounding box and the ground truth bounding box. , w and h are the width and height of the prediction box, respectively. and These represent the width and height of the actual bounding box, respectively.

[0051] A Gaussian penalty term is added to this loss, and the predicted boxes with large center point offsets are exponentially weighted:

[0052] in, ) represents the prediction box With real frame The Euclidean distance to the center point; σ is the scaling parameter of the Gaussian function, used to control the decay range of the penalty term.

[0053] The improved box regression loss is defined as:

[0054] Where, λ Gau This is a hyperparameter used to control the weight of the Gaussian penalty.

[0055] In terms of classification and object confidence, YOLO11-JDE uses BCE, which has the same effect on both positive and negative samples.

[0056] in, This represents the true label of the i-th grid point; This represents the probability that the network predicts the i-th grid point as positive. The original loss function has the same weight for both positive and negative samples, as well as easy and difficult samples, making it prone to gradient dominance by a large number of simple negative samples in complex scenarios.

[0057] This invention introduces a smoothing focus factor, and the smoothing focus cross-entropy is defined as:

[0058] in, This represents the network's prediction probability for the i-th grid point; Indicates the true label; >0 is a smoothing factor used to ensure numerical stability; α is the weight of positive samples; γ is the focusing exponent. Compared with the original Focal Loss, this invention adds to the logarithmic term... Achieve smoothness, avoid p i →The gradient is too large when the value is 0. At the same time, for By focusing on the index, more gradients are allocated to samples of medium difficulty.

[0059] The smoothing-focused cross-entropy formula is uniformly applied to confidence loss. With classification loss In the calculation. Specifically, when calculating the confidence loss, This represents the confidence probability that the target exists at the i-th grid point in the network prediction. The binary true labels represent the foreground target and the background; when calculating the classification loss, This indicates the predicted probability that the target within that grid point belongs to a specific category. This refers to the actual label for the corresponding specific category.

[0060] It should be noted that the loss function design incorporates a Gaussian penalty term and a smoothing focus factor to improve bounding box regression and classification errors. To replace this design, an "adaptive IoU loss" approach could be considered: this approach integrates traditional IoU loss with distance and aspect ratio penalties into a learnable weighted combination, dynamically balancing regression weights between small and large targets. This enhances the response to small offsets and difficult examples without increasing the complexity of the Gaussian term.

[0061] To ensure the physical continuity of substation faults in long-term video monitoring, this invention regularizes the spatiotemporal consistency loss generated by the MG-STR module. A global optimization objective is introduced. Based on the motion field alignment principle, this loss term calculates the features of the current frame. Features aligned with the previous frame after spatial deformation Mean square error between:

[0062] Where C represents the total number of channels in the feature map; H' and W' represent the spatial height and spatial width of the deep feature map, respectively; This represents the feature intensity response value of the current frame feature map at the c-th channel with coordinates (x, y). This represents the feature intensity response value of the previous frame's feature map at the same channel and coordinates after optical flow alignment.

[0063] The overall loss after the improvement is:

[0064] in, Represents the bounding box regression loss; For classification loss; For confidence loss; For spatiotemporal consistency regularization loss; , , and These represent the weight hyperparameters of the corresponding loss terms.

[0065] like Figure 4 As shown, this is a partial detection result of the model. Figure (a) shows the substation under mostly normal conditions with no faults, and the model has no false detections. Figure (b) shows the abnormal condition, and the model accurately detected all faults without any omissions.

[0066] Real-world testing shows that 85% of normal frames do not require activation of the full backbone network, reducing computational load by over 35% and increasing inference speed to 70-80 FPS. Simultaneously, through cross-frame feature reuse (Temporal Warp) and deformable convolution, it achieves a 28% improvement in recall and a 41% reduction in false alarm rate for the approximately 20% of minor faults in substation scenarios. The collaborative mechanism between the dynamic resolution branch and the full backbone network allows the model to adaptively switch computational complexity, resulting in a 19.2% improvement in detection accuracy for medium to large faults (40×40 pixels and above) compared to YOLO11-JDE in complex occlusion or multi-scale fault scenarios.

[0067] The spatiotemporal consistency regularization module uses optical flow-guided feature alignment and temporal constraints to force the feature difference between adjacent frames to be less than a threshold during training, effectively filtering out occasional noise such as pole reflections and raindrops, reducing the false detection rate by 38%. At the same time, for real faults that last for more than 2 frames, such as arcs, the feature consistency loss is maintained, increasing the trajectory integrity rate from 56.37% to 71.84%.

[0068] In another embodiment, a real-time substation fault visual detection system is used to implement the aforementioned real-time substation fault visual detection method, comprising: The image acquisition module acquires images of the substation. A module for constructing a real-time substation fault visual detection network is provided. This network, based on YOLO11-JDE, introduces an early exit branch and a dynamic lightweight detection branch. The input image is first fed into the early exit branch for rapid classification. If the confidence level of the early exit branch is insufficient, the dynamic lightweight detection branch is activated for multi-scale rapid detection. If the confidence level of the dynamic lightweight detection branch is insufficient, the backbone network is activated. Deformable convolution and multiple CBAM attention mechanisms are introduced into the backbone network to enhance its adaptive feature focusing ability on irregular fault edges. The backbone network extracts and retains shallow and mid-level features that have not triggered the early exit mechanism and inputs them into the PANet module for multi-scale path aggregation. The features are spatially deformed and aligned, and then input into the PANet module to fuse with multi-scale features to obtain a composite feature map. This composite feature map is then split and input into the detection head and re-identification head to obtain the detection results. During the training phase, the loss function is calculated, and the model parameters are fine-tuned to obtain the trained real-time substation fault visual detection network. The fault detection module uses a trained real-time substation fault visual detection network to detect fault images in the substation and obtain fault detection results.

[0069] The specific implementation method is the one described above, and will not be repeated here.

[0070] In another embodiment, a computer storage medium stores a computer program thereon, wherein when a computer executes the computer program, it implements the real-time substation fault visual detection method described in any of the above embodiments.

[0071] The specific implementation method is the one described above, and will not be repeated here.

[0072] In another embodiment, an electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor runs the computer program stored in the memory, wherein the computer program, when executed, implements the real-time substation fault visual detection method described in any of the preceding embodiments.

[0073] The specific implementation method is the one described above, and will not be repeated here.

[0074] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0075] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0076] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0078] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0079] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0080] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A real-time visual detection method for substation faults, characterized in that, Includes the following steps: Acquire images of the substation; A real-time substation fault visual detection network is constructed, which introduces an early exit branch and a dynamic lightweight detection branch based on YOLO11-JDE. The input image is first fed into the early exit branch for fast classification. If the confidence of the early exit branch is not up to standard, the dynamic lightweight detection branch is activated for multi-scale fast detection. If the confidence of the dynamic lightweight detection branch is not up to standard, the backbone network is activated. Deformable convolution and multiple CBAM attention mechanisms are introduced into the backbone network to enhance the network's ability to adaptively focus on irregular fault edges. The backbone network extracts and retains the shallow and mid-level features that have not triggered the early exit mechanism and inputs them into the PANet module for multi-scale path aggregation. The features are spatially deformed and aligned, and then input into the PANet module to fuse with multi-scale features to obtain a composite feature map. The composite feature map is then split and input into the detection head and the re-identification head to obtain the detection result. During the training phase, the loss function is calculated, and the model parameters are fine-tuned to obtain a trained real-time substation fault visual detection network. The processing for early branch exits includes: Image data is sequentially processed through a Focus module and convolutional operations for preliminary feature extraction. The extracted preliminary feature maps are then continuously fed into three cascaded Ghost bottleneck layers to extract lightweight features. These lightweight feature maps are then compressed into vectors using global average pooling and fed into an optimized fully connected layer. The output layer determines the confidence scores for fault and non-fault predictions for the current frame. The confidence scores are then evaluated; if a set threshold is reached, early exit is triggered, and a binary fault label is output. A dynamic lightweight detection branch performs multi-scale fast detection, including: The input feature map is sequentially passed through convolution operations, pointwise convolution operations, and the CBAM attention mechanism module to enhance and focus shallow channels and spatial details. The enhanced feature map enters a series combination module that repeats the loop three times. Each combination module consists of a Ghost bottleneck layer and a CBAM attention mechanism module connected in series. The feature map output from the combination module is connected to the detection head and the confidence score is output. If the confidence score reaches the set threshold, the dynamic lightweight detection result is output and the process exits. A trained real-time substation fault visual detection network is used to detect fault images in substations and obtain fault detection results.

2. The real-time substation fault visual detection method according to claim 1, characterized in that, The backbone network processing includes: The input feature map passes through the Focus module and convolution operation, CBAM attention mechanism, Ghost bottleneck layer and CBAM attention mechanism in sequence. Then, the geometric features of irregular faults are captured by the deformable convolution module. Subsequently, the deep features are passed through multiple sets of Ghost C3 modules and CBAM attention mechanism in alternating combinations, and the receptive field is expanded by the spatial pyramid pooling module to obtain deep features. In the final feature fusion and output stage, the multi-scale features extracted by the backbone network at different stages are extracted and fed into the PANet module for spatial scale stitching. Simultaneously, the network writes the optimal features of the current frame into the temporal and optical flow cache module of the previous frame through skip connections. For inference of the current frame, the temporal and optical flow information of the previous frame stored in the cache module, together with the deep features generated by the current backbone network, enter the motion-guided spatiotemporal regularization module. The motion-guided spatiotemporal regularization module calculates and outputs the motion-guided field, and controls the temporal deformation alignment module to perform physical-level spatial deformation and alignment of the temporal and optical flow information of the previous frame. The aligned historical enhancement features are fed into the PANet module. Finally, the PANet module fuses the multi-scale features and the aligned historical enhancement features to obtain a composite feature map, which is then split and input to the detection head and the re-identification head.

3. The real-time substation fault visual detection method according to claim 2, characterized in that, The calculation process of the motion-guided spatiotemporal regularization module includes: During each forward inference step, the original substation fault image at the current moment is first obtained. Image from the previous time step in the cache Low-resolution image pairs are generated through downsampling operations. and The image pairs are fed into a lightweight optical flow network for optical flow estimation, and a coarse optical flow field reflecting the pixel-level relative motion trend between the two frames is calculated and output. The coarse optical flow field is further interpolated using bilinear interpolation to obtain the standard motion field. Input the standard sports field into the sports field size adjustment module, and output the characteristic-level sports field. ; After completing the construction of the sports field, the network enters the feature space alignment stage: extracting the feature map of the current frame generated by the backbone network. And retrieve the cached feature map of the previous frame. , the feature map of the previous frame With characteristic level sports field Both inputs are fed into the feature alignment module, which uses the pixel-level offset vector (u, v) provided by the motion field, where u is the horizontal offset component and v is the vertical offset component. Perform spatial deformation and resampling operations to force the features from the previous time step to align with the current time step in the spatial coordinate system, and output the aligned feature map of the previous frame. In the c Feature intensity response values ​​for each channel at coordinates (x, y): in, c For the channel index of the feature map; and These represent the horizontal and vertical spatial offset components, respectively, predicted by the feature-level motion field and located at the current coordinates (x, y). Current frame feature map The subsequent flow is controlled by the logic decision nodes activated on demand. If the model is in the inference phase and the preceding early exit branch has made a high-confidence non-fault determination for the current frame, the feature map of the current frame... This will directly trigger the bypass mechanism, skipping subsequent feature comparison and regularization calculations, and sending the data to the detection head to complete the prediction; conversely, if it is in the model training phase or encounters a difficult frame, the current frame feature map... The aligned feature map from the previous frame A common input mean squared error calculation node is used to calculate the representation difference between the two at the same spatial location, generating a spatiotemporal consistency regularization loss. .

4. The real-time substation fault visual detection method according to claim 3, characterized in that, Spatiotemporal consistency regularization loss for: Where C represents the total number of channels in the feature map, and H' and W' represent the spatial height and spatial width of the deep feature map, respectively; Represents the feature map of the current frame In the c The characteristic intensity response values ​​of each channel at coordinates (x, y); This represents the feature map of the previous frame after optical flow alignment. The characteristic intensity response value at the same channel and coordinates.

5. The real-time substation fault visual detection method according to claim 1, characterized in that, The loss function incorporates a Gaussian penalty term and a smoothing focusing factor, and also includes a spatiotemporal consistency regularization loss. The total loss function is: in, Represents the improved prediction box regression loss; For classification loss and; For confidence loss; For spatiotemporal consistency regularization loss, , , and These represent the weight hyperparameters of the corresponding loss terms; The improved prediction box regression loss is defined as: Where, λ Gau For superparameters, For Gaussian penalty terms, For the regression loss of the predicted bounding box; in, ) represents the prediction box With real frame The Euclidean distance between the center points; σ is the scaling parameter of the Gaussian function; in, This represents the intersection-over-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. This is the distance penalty term; ρ is the Euclidean distance from the center point; c It is the diagonal distance of the smallest bounding rectangle that simultaneously contains both the predicted bounding box and the ground truth bounding box; Used to constrain the aspect ratio of the predicted bounding box and the ground truth bounding box. , w and h are the width and height of the prediction box, respectively. and These represent the width and height of the actual bounding box, respectively.

6. A real-time substation fault visual detection system, characterized in that, The method for implementing the real-time substation fault visual detection method as described in any one of claims 1-5 includes: The image acquisition module acquires images of the substation. A module for constructing a real-time substation fault visual detection network is provided. This network, based on YOLO11-JDE, introduces an early exit branch and a dynamic lightweight detection branch. The input image is first fed into the early exit branch for rapid classification. If the confidence level of the early exit branch is insufficient, the dynamic lightweight detection branch is activated for multi-scale rapid detection. If the confidence level of the dynamic lightweight detection branch is insufficient, the backbone network is activated. Deformable convolution and multiple CBAM attention mechanisms are introduced into the backbone network to enhance its adaptive feature focusing ability on irregular fault edges. The backbone network extracts and retains shallow and mid-level features that have not triggered the early exit mechanism and inputs them into the PANet module for multi-scale path aggregation. The features are spatially deformed and aligned, and then input into the PANet module to fuse with multi-scale features to obtain a composite feature map. This composite feature map is then split and input into the detection head and re-identification head to obtain the detection results. During the training phase, the loss function is calculated, and the model parameters are fine-tuned to obtain the trained real-time substation fault visual detection network. The fault detection module uses a trained real-time substation fault visual detection network to detect fault images in the substation and obtain fault detection results.

7. A computer storage medium having a computer program stored thereon, characterized in that, When the computer executes the computer program, it implements the real-time substation fault visual detection method according to any one of claims 1-5.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor runs the computer program stored in the memory. When the computer program is executed, it implements the real-time substation fault visual detection method according to any one of claims 1-5.

Images