A method for monitoring the position status of circuit breaker switches based on video recognition technology

Abstract

This invention relates to the field of circuit breaker switch monitoring technology, and provides a method for monitoring the position and status of a circuit breaker switch based on video recognition technology. The method includes acquiring real-time images of the circuit breaker using a camera device, determining the physical state of the circuit breaker, and establishing a first determination result; acquiring real-time data from a relay protection device and recording the operating status of the circuit breaker, and establishing a second determination result; inputting the first and second determination results into a preset signal response device to generate corresponding response actions; wherein the response actions include: opening action, fault action, and closing action.

Description

Technical Field

[0001] This invention relates to the field of circuit breaker switch monitoring, and in particular to a method for monitoring the position status of circuit breaker switches based on video recognition technology. Background Technology

[0002] With the rapid development of smart grids, circuit breakers, as key protection and control devices in power systems, directly impact the safety and reliability of the power grid through real-time monitoring and rapid response capabilities of their operating status. Traditional circuit breaker status monitoring mainly relies on mechanical position sensors (such as auxiliary contacts and microswitches) or electrical quantity measurements (such as opening and closing coil current), but these methods have the following technical shortcomings:

[0003] Sensor limitations: Mechanical sensors are susceptible to environmental corrosion, vibration and aging, which can lead to contact oxidation or poor contact, resulting in false signals; while pure electrical quantity monitoring cannot directly reflect the physical position of the switch (such as hidden faults such as loose contact or mechanical jamming).

[0004] Poor environmental adaptability: Existing optical detection methods (such as laser ranging and infrared thermometry) have significantly reduced recognition accuracy under complex lighting conditions, rainy or foggy weather, or metal reflection interference, making it difficult to meet the all-weather monitoring needs of outdoor substations.

[0005] Information silo problem: Circuit breaker status data and relay protection system lack deep coordination. Protection action decisions rely only on electrical parameters (such as overcurrent and differential protection), and cannot be combined with the physical state of the switch (such as incomplete contact separation) to make complex fault judgments, resulting in protection maloperation or failure to operate.

[0006] Insufficient intelligence: Traditional image processing algorithms (such as threshold segmentation and edge detection) rely on manual feature design, which makes it difficult to adapt to dynamic scenes such as dirty equipment and uneven lighting, and they lack self-optimization capabilities, requiring frequent manual calibration.

[0007] In recent years, attempts to improve related technologies have included:

[0008] Patent CN202010730692.0 proposes a circuit breaker status monitoring method based on infrared thermal imaging, which can determine poor contact of contacts by temperature anomalies, but it does not solve the problem of image noise caused by reflection from the metal casing.

[0009] The paper "Switchgear Status Recognition Based on Deep Learning" (Automation of Electric Power Systems, 2021) uses a convolutional neural network (CNN) to identify the opening and closing status of circuit breakers, but its model does not integrate multispectral data and has limited generalization ability in low-light environments.

[0010] In summary, current technologies have not yet achieved a deep integration of robust visual monitoring and intelligent decision-making in relay protection, particularly exhibiting significant technical bottlenecks in areas such as adaptability to complex environments, multimodal data collaboration, and automated closed-loop control. Therefore, an innovative technical solution is urgently needed to improve the accuracy, real-time performance, and reliability of circuit breaker status assessment through deep integration of video recognition and protection systems, thereby providing a higher level of proactive defense capabilities for smart grids. Summary of the Invention

[0011] This invention proposes a method for monitoring the position status of circuit breakers based on video recognition technology, which solves the problem that traditional circuit breaker status monitoring mainly relies on mechanical position sensors, such as auxiliary contacts and micro-switches, to measure electrical quantities, such as the opening and closing coil current, resulting in poor monitoring results.

[0012] This invention proposes a method for monitoring the position status of circuit breaker switches based on video recognition technology, comprising:

[0013] Real-time images of the circuit breaker are captured by camera equipment, and the physical state of the circuit breaker is determined to establish the first judgment result.

[0014] Acquire real-time data from the relay protection device and record the operating status of the circuit breaker to determine the second judgment result;

[0015] The first and second judgment results are input into a preset signal response device to generate corresponding response actions; the response actions include: opening action, fault action and closing action.

[0016] Furthermore, the real-time images include visible light images, near-infrared images, depth thermal maps, and images after supplemental lighting;

[0017] Furthermore, the method of acquiring real-time images of the circuit breaker via camera equipment also includes:

[0018] A multi-resolution scene database is pre-built, and adaptive optics analysis is performed to determine the resolution status of the current scene;

[0019] Adaptive optics analysis includes multispectral imaging analysis, dynamic polarization analysis, and interference light source array analysis.

[0020] Based on multispectral imaging analysis, the inter-frame error of the visible light image and near-infrared feature image under alternating acquisition is calculated.

[0021] Based on dynamic polarization analysis, the real-time polarization angles of the rotatable linear polarizer array and the annular phase delay plate of the camera device are calculated.

[0022] Based on the analysis of the interference light source array, a structured light distribution array for the current scene is generated to determine the areas that do not meet the preset clarity and require supplemental lighting.

[0023] Based on inter-frame error, polarization angle, and the area to be illuminated, an image enhancement network is constructed for real-time images to enhance them.

[0024] Furthermore, the image enhancement network includes an input layer, a data alignment sub-network, a physical feature encoder, a conditional modulation enhancement unit, and a cascaded enhancement strategy unit; wherein,

[0025] The input layer includes image data for receiving real-time images through signal channels; wherein, the signal channels include a visible light image channel, a near-infrared image channel, a polarization angle tensor channel, and a structured photomask channel;

[0026] The data alignment sub-network is used to perform the first enhancement process on the image data through the spatiotemporal alignment network, thereby eliminating inter-frame errors;

[0027] The physical feature encoder is used to extract different modal features from image data through a multi-branch parallel structure and generate a multimodal feature map;

[0028] The conditional modulation enhancement unit is used to determine the image region that needs to be enhanced by using the multimodal feature map and inter-frame error as a conditional vector;

[0029] The cascaded enhancement strategy unit is used to stack multiple dynamic enhancement blocks to enhance image regions of different scales.

[0030] Furthermore, the multi-branch parallel structure includes: a spectral branch, a polarization branch, and a supplementary light branch;

[0031] Among them, the spectral branch is used to extract reflective features and texture features separated by channel attention extraction;

[0032] Polarization branching is used to extract rotationally weighted features guided by the polarization angle of the camera device and suppress metallic reflection features through directionally separable convolution;

[0033] The supplementary lighting branch is used to determine the grayscale deviation characteristics of the image through mask-guided local contrast enhancement.

[0034] Furthermore, determining the physical state of the circuit breaker and establishing the first determination result includes:

[0035] A multimodal feature fusion network is constructed. Visible light images, near-infrared images, depth thermal maps, and images after supplemental lighting are input into the spatiotemporal alignment module. Texture features, thermodynamic features, and polarization suppression features are fused through a cross-modal attention mechanism to generate a high-confidence circuit breaker state feature map.

[0036] Based on a pre-defined physical constraint state decision tree, key regions in the state feature map of high-confidence circuit breakers are classified and judged hierarchically.

[0037] First-level judgment: Identify the contact closure / separation state based on the near-infrared reflectivity and temperature rise gradient of the contact area;

[0038] The second layer of judgment: Based on the integrity of the outline of the mechanical interlocking device in the image after polarization suppression, detect mechanical jamming or deformation;

[0039] The third layer of judgment: the level of surface contamination of the insulating component is determined by the ratio of the local contrast anomaly between the supplementary lighting area and the original image;

[0040] Output a circuit breaker physical status report that integrates multi-modal judgment results, determine the first judgment result, which includes a three-dimensional status code of switch position, mechanical health and insulation status.

[0041] Furthermore, the acquisition of real-time data from the relay protection device, recording of the circuit breaker's operating status, and determination of the second judgment result include:

[0042] Based on the relay protection device, a multi-source data acquisition device is configured to acquire the real-time electrical parameters of the relay protection device and the multi-dimensional operating status data of the circuit breaker.

[0043] A time-aligned multi-source data fusion model is constructed to perform dynamic collaborative analysis of the real-time electrical parameters and multi-dimensional operating status data, and a fault-action mapping matrix is ​​generated.

[0044] Multi-dimensional anomaly detection is performed on the fault-action mapping matrix using an autoregressive anomaly propagation model.

[0045] Generate a second determination result containing the root cause of the fault, and output it in a quadruple encoding format.

[0046] Furthermore, the real-time electrical parameters include the protection action signal waveform, fault current amplitude, trip circuit impedance, and harmonic distortion rate;

[0047] The multidimensional operating status data includes the mechanical vibration spectrum of the circuit breaker, the current curve of the opening and closing coils, the pressure value of the energy storage mechanism, and the wear index of the switch contacts.

[0048] Furthermore, the autoregressive anomaly propagation model includes a deviation alarm layer, a risk identification layer, and a fault location layer; wherein,

[0049] The deviation alarm layer is used to compare the deviation between the time the trip command is issued and the actual opening time of the circuit breaker. When the deviation exceeds a preset threshold, a mechanical response delay alarm is triggered.

[0050] The risk identification layer is used to analyze the nonlinear relationship between the number of protective device actions and the contact electrical life loss curve, and to identify the hidden fault risks caused by abnormally frequent actions.

[0051] The fault location layer is used for coupled analysis based on vibration spectrum entropy and energy storage pressure fluctuation to locate faults such as jamming of the mechanism or spring fatigue.

[0052] Furthermore, the step of inputting the first judgment result and the second judgment result into a preset signal response device to generate a corresponding response action includes:

[0053] Receive the three-dimensional state code of the first judgment result and the quadruple code of the second judgment result;

[0054] Construct a multimodal response decision model, dynamically fuse coded data, and generate a set of response decisions;

[0055] A strategy for generating composite responses based on quadruple encoding for fault root cause tracing.

[0056] Response actions are generated based on the composite response strategy and the response decisions triggered in the set of response actions.

[0057] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0058] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0060] Figure 1 This is a flowchart illustrating a method for monitoring the position status of a circuit breaker switch based on video recognition technology, as described in an embodiment of the present invention.

[0061] Figure 2 This is a flowchart illustrating the process of acquiring real-time images of a circuit breaker using a camera device in an embodiment of the present invention.

[0062] Figure 3 This is a block diagram illustrating the function of the image enhancement network in an embodiment of the present invention.

[0063] Figure 4 This is an association diagram of the multi-branch parallel structure in an embodiment of the present invention;

[0064] Figure 5 This is a flowchart illustrating the implementation of the first judgment result in an embodiment of the present invention;

[0065] Figure 6This is a flowchart illustrating the implementation of the second determination result in this embodiment of the invention.

[0066] Figure 7 This is a functional diagram of real-time electrical parameters and multi-dimensional operating status parameters in an embodiment of the present invention;

[0067] Figure 8 This is a functional diagram of the autoregressive anomaly propagation model in an embodiment of the present invention;

[0068] Figure 9 This is a flowchart for generating the corresponding response action in an embodiment of the present invention. Detailed Implementation

[0069] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0070] Example 1:

[0071] To address the shortcomings of traditional technical solutions, this application proposes a method for monitoring the position status of circuit breaker switches based on video recognition technology. (See reference...) Figure 1 :

[0072] Firstly, traditional methods relying on limit switches or current / voltage sensors are prone to misjudgment due to mechanical wear or electromagnetic interference. This application uses a camera device to acquire real-time images of the circuit breaker and determine its physical state to establish a first judgment result. For example, by acquiring real-time images of the circuit breaker through a camera, and using methods such as target detection, state classification, and image segmentation, the physical state of the circuit breaker, such as the open / close position, can be analyzed. This non-contact video monitoring method for determining the physical state of the circuit breaker avoids the problems of traditional mechanical / electrical sensors being susceptible to environmental interference and having complex installation requirements.

[0073] Traditional methods, relying on a single electrical data source, cannot handle extreme scenarios such as sensor failures or communication interruptions. This application, based on real-time data from relay protection devices and recording the circuit breaker's operating status, determines a second judgment result. It aims to combine electrical parameters during circuit breaker operation, such as opening and closing currents and protection signals, to provide dual verification and enhance the reliability of status judgment.

[0074] To address the shortcomings of traditional technical crime detection methods, which rely on independent judgment logic and lack multi-dimensional collaborative analysis capabilities, this application inputs the first and second judgment results into a preset signal response device to generate corresponding response actions. These response actions include: activation, fault detection, and deactivation. In other words, by fusing visual and electrical data, more accurate response actions (such as fault alarms and automatic interlocking) are generated, reducing the risk of false activation / failure to activate. The aim is to eliminate the unreliability of single data sources, thereby improving the false judgment rate and reducing misjudgments of mechanical jamming and electrical faults. Furthermore, both visual and electrical detection methods possess their own high efficiency.

[0075] The key feature of this application is the construction of a dual-redundant detection logic. The signal response device relies on two types of data inputs simultaneously and generates a response action through logical operations. There is a conflict between the real-time requirements of video recognition (needing high frame rate processing) and the high reliability requirements of relay protection data (needing strict synchronization timing). This application solves the data synchronization problem through a timestamp alignment algorithm. By combining video capture of stuttering frames with abnormal coil currents in the electrical data, fault types can be identified.

[0076] In practice:

[0077] The operating status of the circuit breaker is determined by real-time data from the relay protection device, such as current, voltage, and protection signals, combined with logic rules or timing analysis.

[0078] By combining visual judgments with sensor data through pre-defined logic, such as weighted voting fusion or Bayesian network fusion, the limitations of a single data source are overcome, and the robustness of state judgments is improved.

[0079] By cross-verifying the mechanical position of the circuit breaker through video identification, such as the opening and closing angle of the disconnect switch, with the electrical status recorded by the relay protection device, such as whether the current is zero, potential anomalies can be identified, such as mechanical jamming leading to misinterpretation of electrical signals.

[0080] In practice, decision trees can be designed based on dual-source data input, for example:

[0081] Both vision and sensors determine it as "shutdown" → generate a shutdown action;

[0082] The visual display shows "closed" but the sensor has no current → triggering a fault action;

[0083] Sensor alarm but visual status is normal → Initiate secondary verification process.

[0084] The beneficial effects of the above technical solution are as follows:

[0085] This application employs a dual verification mechanism to reduce the false alarm rate and achieves the fusion of the two methods. Traditional sensors are susceptible to electromagnetic interference, leading to false alarms, while video verification can eliminate false alarms. When the vision system is obstructed or damaged, sensor data can provide redundant backup. In the event of a fault, the video recording is aligned with the electrical data timestamps, allowing for the retrospective analysis of critical states prior to the fault and aiding in the analysis of the reasons for the circuit breaker's failure to operate or malfunction. Compared to manual inspection, this achieves 24 / 7 automatic monitoring; through early anomaly detection, such as action delays caused by contact oxidation, unplanned power outages can be avoided.

[0086] Example 2:

[0087] The real-time images include visible light images, near-infrared images, depth thermal maps, and images after supplemental lighting.

[0088] In practical implementation, this application can obtain different technical observation details based on different real-time images. Visible light images provide high-resolution visual information on the circuit breaker switch position, allowing direct observation of mechanical components such as the physical state of contacts and linkages. Traditional solutions rely solely on visible light images, which are easily affected by changes in ambient lighting (such as strong reflections and shadows), leading to misjudgments. Near-infrared images capture the circuit breaker status in low-light or nighttime environments, penetrating partial obstructions such as dust and fog, thus enhancing environmental adaptability. Deep thermal mapping detects abnormal heating at circuit breaker contact points through temperature distribution, providing early warning of faults. Traditional infrared temperature measurement is mainly based on single-point temperature, failing to perceive temperature globally and addressing localized overheating. Images with supplemental lighting, through controllable light sources, eliminate shadow or reflection interference, improving image consistency and ensuring algorithm stability.

[0089] In actual implementation, technical solutions can be integrated, with visible light images showing the contacts closed, but thermal images showing high temperatures at the contact points, triggering a "poor contact" alarm.

[0090] Example 3:

[0091] This application addresses the challenges of image clarity and interference resistance in complex environments encountered by traditional technical solutions by proposing a method for acquiring real-time images of circuit breakers using a camera device; see reference. Figure 2 :

[0092] To address the limitations of traditional methods that rely on fixed-parameter optical filtering and cannot adapt to dynamic and complex scenarios, such as oil spills, this application pre-constructs a multi-clarity scene database and performs adaptive optics analysis to determine the clarity state of the current scene. By pre-collecting circuit breaker images under different lighting, occlusion, and pollution scenarios, a training dataset is constructed, enabling the image enhancement network to have environmental adaptability and reducing the risk of model overfitting.

[0093] In this process, adaptive optics analysis methods include multispectral imaging analysis, dynamic polarization analysis, and interference light source array analysis;

[0094] To address the inability of traditional single-spectrum imaging to distinguish the superposition effect of natural light and equipment-assisted lighting, this application uses multispectral imaging analysis to calculate the inter-frame error of visible light images and near-infrared feature images under alternating acquisition in real time; by alternating acquisition of visible light and near-infrared images, the differences between the two are calculated, such as edge offset and contrast changes, to identify ambient light interference.

[0095] To address the issue that fixed-angle polarization filtering cannot adapt to multi-angle reflective scenarios, this application calculates the real-time polarization angle of the rotatable linear polarizer array and the annular phase delay plate of the camera device based on dynamic polarization analysis. That is, by dynamically adjusting the rotating polarizer array and the phase delay plate, reflections from metal surfaces, such as those from circuit breaker housings, are suppressed, thereby improving the clarity of image details.

[0096] To address the issue of light pollution or insufficient brightness in key areas caused by traditional global uniform illumination, this application analyzes the interfering light source array to generate a structured light distribution array for the current scene, identifies areas that do not meet the preset clarity and require additional lighting, thus generating a programmable lighting pattern (such as a grid-like light spot), and identifies areas that need enhancement (such as the location of a shadow-covered contact point) to achieve precise local lighting.

[0097] To address the limitations of traditional supplemental lighting, which uses a fixed architecture and cannot adapt to sudden changes in the photomask or localized occlusion, this application constructs a real-time image enhancement network based on inter-frame errors, polarization angles, and the area to be illuminated. This network enhances real-time images by combining multispectral (spectral dimension), dynamic polarization (polarization dimension), and structured supplemental lighting (spatial dimension) to form a multi-physics collaborative optimization mechanism. The image enhancement network is dynamically adjusted based on real-time analysis results, forming a closed-loop perception, analysis, and measurement system that resolves timing synchronization and computational delays. For interference resistance, dynamic polarization suppresses reflections, combined with structured supplemental lighting shadow elimination and key area identification, resulting in improved contrast. Multispectral inter-frame errors are used to locate interference areas and limit the computational range of the image enhancement network, reducing computational load. The optimized light distribution in this application can penetrate oil or dust, such as with near-infrared supplemental lighting, and combined with local repair by the enhancement network, significantly improving recognition accuracy.

[0098] In practical implementation, this application improves image analysis results, i.e., clarity, by pre-constructing a multi-clarity scene database in complex lighting or dynamic environments such as strong reflection, low light, and multiple light source interference.

[0099] The database needs to cover multiple scenarios, such as different times, weather conditions, and equipment aging states, to prevent overfitting. Inter-frame errors in visible and near-infrared images are used to capture dynamic changes, such as equipment heating and surface oxygen, while multispectral data enhances feature separation capabilities, such as distinguishing between metallic reflections and stains. Dynamic adjustment of rotating polarizers and phase delay plates can effectively suppress metallic surface reflections, commonly found on circuit breaker housings, improving the visibility of critical areas, such as switch contacts. Structured light distribution can actively compensate for areas with insufficient ambient light, such as shadows and backlighting, ensuring uniform illumination in the monitored area. Combining multimodal inputs such as inter-frame errors, polarization angles, and supplementary lighting areas, the network can specifically optimize image quality, such as denoising, deblurring, and enhancing contrast.

[0100] Example 4:

[0101] The image enhancement network of this application has multimodal input capability, see reference. Figure 3 While existing technologies do not encode optical parameters and input them into the network, this application proposes an image enhancement network comprising an input layer, a data alignment sub-network, a physical feature encoder, a conditional modulation enhancement unit, and a cascaded enhancement strategy unit; among which,

[0102] To address the limitation of traditional solutions that rely on a single image channel to fuse polarization and structured light information, this application's input layer includes image data for receiving real-time images via a signal channel. This signal channel comprises a visible light image channel, a near-infrared image channel, a polarization angle tensor channel, and a structured photomask channel. Through multi-channel input of visible light, near-infrared light, polarization tensor, and structured photomask, parallel fusion of multimodal data (spectral, polarization, and spatial supplementary lighting) is achieved, improving the completeness of feature representation.

[0103] To address the issue of traditional methods using fixed registration parameters, which prevents dynamic adaptation to camera and environmental jitter, this application employs a data alignment sub-network to perform a first enhancement process on image data via a spatiotemporal alignment network. This eliminates inter-frame errors. By eliminating spatiotemporal offsets in multimodal images, this application ensures the accuracy of feature acquisition, such as polarization angle reflection suppression features and near-infrared penetration features.

[0104] Traditional encoders, employing serial mechanisms, may suffer from multimodal interference. This application's physical feature encoder extracts different modal features from image data using a multi-branch parallel structure, generating a multimodal feature map. This solves the problem of extracting physical features of different modes through independent analysis, while preserving modality-specific information.

[0105] In view of the fact that traditional enhancement units use global parameters and cannot optimize for local interference, the conditional modulation enhancement unit of this application uses multimodal feature maps and inter-frame errors as conditional vectors to determine the image regions that need to be enhanced. That is, it dynamically adjusts the enhancement weights according to the inter-frame errors (reflecting the intensity of environmental interference) and prioritizes the enhancement of high error regions.

[0106] To address the issue that traditional networks with fixed layers cannot adapt to complex dynamic adjustments in enhancement depth, this application's cascaded enhancement strategy unit stacks multiple dynamic enhancement blocks. These blocks enhance image regions at different scales, thereby achieving hierarchical optimization through multi-scale enhancement blocks (such as capturing touch details at a small scale and identifying the overall position at a large scale), thus avoiding information loss at a single scale.

[0107] This application uses polarization tensors (physical optical parameters) and structured photomasks (spatial illumination requirements) as network input conditions to achieve synergistic optimization of physical rules and deep learning. Through the collaboration of conditional modulation units (local optimization) and cascaded strategy units (multi-scale optimization), it solves global and local problems in complex scenes. The dynamic enhancement block adaptively selects the enhancement level based on scene complexity, maintaining high accuracy while processing in real time.

[0108] In practical implementation, this application maps multispectral imaging visible light + near-infrared, dynamic polarization angle, and structured light supplementation area as multi-channel input, and does not rely solely on RGB images.

[0109] Based on the results of adaptive optics analysis, such as inter-frame error and dynamic adjustment of polarization angle network parameters, scene adaptation will be enhanced.

[0110] By introducing physical interpretability constraints, such as reflection suppression and supplemental lighting consistency, the resulting Generative Adversarial Network (GAN) can avoid overfitting caused by purely data-driven approaches.

[0111] Specifically, the input layer consists of 7 channels: visible light image (3 channels), near-infrared image (1 channel), polarization angle tensor (2 channels), and structured photomask (1 channel). The data alignment sub-network uses a lightweight spatiotemporal alignment network (such as deformable convolution based on optical flow) to eliminate inter-frame offsets in multispectral and polarization data. The conditional modulation enhancement unit inputs: multimodal feature map + real-time polarization angle / inter-frame error parameters (as conditional vectors). Multiple dynamic enhancement blocks are stacked, with each layer focusing on features at different scales (from global illumination compensation to local detail restoration).

[0112] In practical implementation, improvements based on U-Net are introduced, including:

[0113] The polarization physical loss constraint generates an image consistent with the reflected light distribution predicted by the polarization angle.

[0114] Multispectral consistency loss for cross-modal alignment of visible and near-infrared enhancement results. A multi-scale PatchGAN structure with an additional structured photomask as a condition to distinguish the plagiarism of illumination between real and generated images.

[0115] Example 5:

[0116] This application proposes a method for determining the characteristics affecting grayscale deviation, see reference. Figure 4 During implementation, the multi-branch parallel structure includes: spectral branch, polarization branch, and supplementary light branch;

[0117] Traditional technical solutions using fixed filters may fail to distinguish between reflective and real textures. This application uses spectral branching to extract reflective and texture features separated by channel attention. In implementation, reflective features and real texture features are separated by channel attention, thereby improving the recognition accuracy of key areas. Reflective features include highlights on metal surfaces, while real texture features include unearthed oxidation marks, etc.

[0118] Traditional convolutional kernels, with their fixed orientation, are only applicable to specific polarization angles and have poor generalization capabilities. This application's polarization branch extracts rotational weight features guided by the camera's polarization angle and suppresses metallic reflection features through directionally separable convolution. Furthermore, it dynamically adjusts the convolutional kernel orientation based on the polarization angle to suppress multi-angle metallic reflections while preserving edge details. The polarization branch embeds the polarization angle physical parameter into the directionally separable convolution, enabling the coordinated design of optical properties and network structure.

[0119] Traditional global contrast enhancement operations may lead to increased noise in non-visible areas. The supplementary lighting branch in this application is used for local contrast enhancement guided by a mask to determine the gray-scale deviation characteristics of the image. Then, a structured photomask is used to locate low-contrast areas, which can adjust the gray-scale distribution in a targeted manner to prevent overexposure caused by global enhancement.

[0120] The polarization branch suppresses reflections, and the spectral branch facilitates the separation of real textures. The supplementary light branch precisely enhances local contrast, amplifying errors at any stage to identify them and improve the recognition of erroneous areas.

[0121] In practice, the physical feature encoder uses a multi-branch parallel structure to extract features from different modalities:

[0122] For example: Spectral branch: 3×3 convolution + channel attention (ECA-Net) → separate reflections and textures.

[0123] Polarization branch: Directionally separable convolution + rotation weight guided by polarization angle → suppresses metallic reflection.

[0124] Fill light branch: Mask-guided local contrast enhancement → shadow area restoration.

[0125] Example 6:

[0126] This application will also determine the physical state of the circuit breaker and establish a first determination result. This process requires hierarchical determination; see [link / reference]. Figure 5 :

[0127] To address the limitations of traditional methods that rely on a single modality and cannot handle complex interference, such as misjudgments caused by reflections and high temperatures, this application constructs a multimodal feature fusion network. Visible light images, near-infrared images, depth thermal maps, and images after supplemental lighting are input into a spatiotemporal alignment module. A cross-modal attention mechanism fuses texture features, thermodynamic features, and polarization suppression features to generate a high-confidence circuit breaker status feature map. In this process, the cross-modal attention mechanism fuses visible light texture, near-infrared light penetrability, thermal map temperature, and non-visual image anti-interference capabilities to generate a high-confidence feature map covering mechanical, thermodynamic, and electrical multi-dimensional information, thus improving the comprehensiveness of status judgment. The spatiotemporal alignment module, which uses fixed registration parameters in traditional techniques and cannot dynamically adapt to equipment and environmental vibrations, eliminates spatiotemporal offsets caused by differences in acquisition time or viewing angles in multimodal images, ensuring the accuracy of feature fusion.

[0128] To address the issue that traditional solutions using a single threshold cannot distinguish between mechanical faults and electrical anomalies, a layered judgment is performed on key areas in the state characteristic map of high-confidence circuit breakers based on a pre-set physical constraint state decision tree.

[0129] First-level judgment: Based on the near-infrared reflectivity and temperature rise gradient of the contact area, the contact closure / separation state is identified. This application adopts a combination of infrared reflectivity, i.e., contact material characteristics and contact resistance anomalies of temperature rise gradient, which can accurately distinguish the closure / separation state.

[0130] The second layer of judgment: Based on the integrity of the outline of the mechanical interlocking device in the image after polarization suppression, mechanical jamming or deformation is detected. In the second layer of judgment, this application analyzes the outline of the interlocking device through the image after polarization suppression. For example, in the case of deformation and jamming, it avoids interference from metal reflection.

[0131] The third layer of judgment: by the ratio of the local contrast anomaly between the supplementary lighting area and the original image, the surface contamination level of the insulating component is determined. This application utilizes the local contrast anomaly between the supplementary lighting area and the original image to detect that dirt will cause a decrease in surface insulation performance.

[0132] Since traditional outputs only show an open or closed state, which cannot provide guidance, this application outputs a circuit breaker physical status report that integrates multi-modal judgment results to determine a first judgment result. The first judgment result includes a three-dimensional status code of switch position, mechanical health, and insulation status. Through this application, the codes of switch position, mechanical health, and insulation status can be unified as a decision vector to achieve multi-dimensional fault diagnosis.

[0133] In practical implementation, this application utilizes multispectral data from visible light, near-infrared (NIR), and depth thermal mapping to cover different physical properties, such as...

[0134] The visible light is used to capture the surface texture and oxidation marks of the circuit breaker. For example, oxidized areas may show color changes or spots under visible light.

[0135] By penetrating surface stains with near-infrared light, the contact status of the contacts can be detected. For example, the near-infrared reflectivity of metal oxides is significantly lower than that of normal metals.

[0136] Temperature rise at contact points is monitored using thermograms, such as Joule heating generated by contact resistance in a closed state.

[0137] By using an alternating exposure mechanism to dynamically adjust the exposure time, such as short exposure in visible light followed by long exposure in near-infrared light, motion blur is reduced. The inter-frame error matrix is ​​used to calculate the difference between adjacent frames, thereby extracting the oxidized region. The oxidized region has abnormal heat conduction due to oxidation, and the temperature rise gradient is significantly different from that of the normal region.

[0138] Reflective surfaces on the metal surfaces of circuit breakers can obscure critical details, such as contact wear and minor cracks. By coordinating the control of a rotatable linear polarizer array and a phase delay plate, the polarization direction is switched at a preset frequency to suppress specular reflections at different angles. This generates a polarization angle-reflectivity curve, making the reflectivity of oxidized or rough surfaces more sensitive to changes in polarization angle, thus distinguishing between normal and abnormal areas.

[0139] Benshenqing dynamically generates a structured light distribution based on the inter-frame error matrix, such as the oxidation region, and the polarization reflectivity mapping, such as the reflection suppression requirements. This provides supplementary lighting for shadow areas and supplementary lighting for the near-infrared band, thus more clearly displaying the details of the contact surface of the contactor.

[0140] To supplement illumination in oxidized areas, specific wavelengths of visible light are used to enhance the contrast of oxidized spots.

[0141] By comparing images before and after supplemental lighting, the ratio of local contrast anomalies can be calculated, such as the proportion of brightness increase in shadow areas after supplemental lighting, thereby quantifying the degree of dirt or insulation degradation.

[0142] This application employs timestamp synchronization and feature point matching, such as the SIFT algorithm, to ensure that multimodal data is consistent in time and space.

[0143] Design a multi-head attention layer and dynamically allocate weights, such as thermal features accounting for 70% of the weight in contact state judgment and polarization features accounting for 60% of the weight in mechanical interlock detection;

[0144] Furthermore, by fusing texture (visible light), thermodynamics (thermograph), and polarization suppression (reflectivity) features, a high-confidence feature map is generated. Based on the near-infrared reflectivity threshold (normal contact reflectivity >80%, after oxidation <50%), and the temperature rise gradient (temperature rise >3℃ / s in the closed state), logical rules are constructed.

[0145] In practical implementation, contour integrity analysis is performed on the image after polarization suppression:

[0146] Edge detection: The Canny algorithm is used to extract the contour of the interlocking device;

[0147] Deformation determination: Calculate the Hausdorff distance between the contour curvature and the standard template (if >2mm, it is determined to be deformed).

[0148] Insulation pollution level classification;

[0149] Based on the contrast anomaly ratio of the supplementary lighting area:

[0150] Dirt level 1 (clean): CR≤1.2;

[0151] Filth Level 2 (Mild): 1.2 <CR≤1.5;

[0152] Filth Level 3 (Severe): CR>1.5.

[0153] Example 7:

[0154] This application proposes a method for determining the source of a fault, see reference. Figure 6 The process of acquiring real-time data from the relay protection device, recording the operating status of the circuit breaker, and determining the second judgment result specifically includes:

[0155] This application addresses the problem that traditional single data sources cannot identify the correlation between mechanical and electrical faults. It employs relay protection devices and configures multi-source data acquisition devices to obtain real-time electrical parameters of the relay protection devices and multi-dimensional operating status data of the circuit breakers. By integrating the real-time electrical parameters of the relay protection devices, such as current, voltage, and trip signals, cross-domain data fusion of the circuit breaker's multi-dimensional motion status data can be achieved, such as the fusion of mechanical vibration, temperature, and number of operations, thereby improving the comprehensiveness of fault detection.

[0156] This application addresses the data misalignment problem caused by fixed-time serialization. It constructs a time-aligned multi-source data fusion model to dynamically and collaboratively analyze the real-time electrical parameters and multi-dimensional operating status data. A fault-action mapping matrix is ​​used, and a dynamic time warping algorithm is employed to align electrical and mechanical data with different sampling rates, thereby eliminating time-series offset errors and maintaining the accuracy of data collaborative analysis.

[0157] This application addresses the limitations of traditional solutions that rely on static rule matrices and cannot adapt to all fault types. It employs an autoregressive anomaly propagation model to perform multi-dimensional anomaly detection on the fault-action mapping matrix. Specifically, it uses time-series autoregression to trace the propagation path of abnormal signals, thereby generating quadruplet codes and producing traceable coded data. Moreover, the output is not a simple text warning, but a decision priority ranking based on structured coding of fault, location, time, and confidence. The autoregressive model introduces time-dimensional causal relationship analysis (such as anomaly propagation paths), whereas traditional solutions only statically associate faults with actions.

[0158] Generate a second determination result containing the root cause of the fault, and output it in a quadruple encoding format.

[0159] In practical implementation, the real-time electrical parameters described in this application include protection action signal waveforms, fault current amplitudes, trip circuit impedances, and harmonic distortion rates; the multi-dimensional operating status data includes the circuit breaker mechanical vibration spectrum, opening and closing coil current curves, energy storage mechanism pressure values, and switch contact wear indices; a sliding window mechanism is used to extract the gradient of circuit breaker mechanical characteristic changes within a ±100ms time window before and after the protection action event; based on a dynamic weight adjustment algorithm, the degree of synergistic correlation between electrical fault characteristics and mechanical status parameters is calculated to generate a spatiotemporally synchronized fault-action mapping matrix; after multi-dimensional anomaly detection, the encoding dimensions include electrical protection matching degree, mechanical action reliability, latent defect probability, and recommended maintenance priority; and an immutable time-series association hash value is generated by combining the judgment results with the original data through a blockchain evidence storage module.

[0160] Example 8:

[0161] This application addresses multi-micro cloud state data; see [link / reference]. Figure 7 Specific real-time electrical parameters include protection action signal waveform, fault current amplitude, trip circuit impedance, and harmonic distortion rate;

[0162] Multidimensional operating status data includes the mechanical vibration spectrum of the circuit breaker, the current curves of the opening and closing coils, the pressure value of the energy storage mechanism, and the wear index of the switch contacts.

[0163] Traditional solutions only record the operational status and cannot analyze latent faults caused by waveform distortion. This application captures the timing characteristics of protection actions (such as tripping delay and signal jitter) to identify malfunctions or failures to operate. Traditional peak detection ignores current change trends; this application quantifies the severity of short circuits / overloads and distinguishes between transient overcurrents and persistent faults by combining waveform analysis. Traditional solutions only manually measure impedance during periodic maintenance; this application detects abnormal circuit contact resistance to prevent tripping failures. Regarding harmonic distortion rate, it identifies abnormal heating and malfunctions of circuit breakers caused by grid harmonics. Multidimensional status data can detect latent faults such as mechanical jamming and linkage deformation through vibration frequency analysis, analyze coil current rise time / peak value to determine the health of the electromagnetic mechanism, monitor spring / hydraulic energy storage status in real time to prevent slow opening and closing due to insufficient pressure, and estimate the remaining contact life based on arc count and current integration to achieve predictive maintenance.

[0164] Example 9:

[0165] This application employs an autoregressive anomaly propagation model, including a deviation alarm layer, a risk identification layer, and a fault location layer, for deviation calculation, risk identification, and fault location. For specific implementation, please refer to... Figure 8 ;

[0166] Because traditional technical solutions only record the tripping time and cannot correlate the difference between the command and action timing, the deviation alarm layer of this application is used to compare the deviation between the tripping command issuance time and the actual tripping time of the circuit breaker. When the deviation exceeds the preset threshold, a mechanical response delay alarm is triggered. That is, by detecting the deviation between the tripping command and the actual tripping time (such as delay caused by mechanical jamming), a mechanical response delay alarm is triggered in real time to avoid protection failure.

[0167] In view of the fact that traditional solutions use linear life prediction and cannot identify nonlinear losses, the risk identification layer of this application is used to analyze the nonlinear relationship between the number of protective device operations and the contact electrical life loss curve, and to identify the hidden fault risk caused by abnormally frequent operations. In implementation, by establishing a nonlinear model (such as an exponential decay curve) of the number of operations and contact electrical life loss, the hidden damage to the contacts (such as metal fatigue microcracks) caused by abnormally frequent operations can be identified.

[0168] Traditional vibration analysis relies solely on amplitude thresholds, failing to distinguish between jamming and normal vibration. This application's fault location layer utilizes a coupled analysis based on vibration spectrum entropy and energy storage pressure fluctuation to pinpoint the location of mechanism jamming or spring fatigue. This application precisely locates fault locations (such as connecting rod jamming or spring fatigue) through coupled analysis of vibration spectrum entropy (reflecting mechanical looseness) and energy storage pressure fluctuation (reflecting spring stiffness).

[0169] Example 10:

[0170] This application addresses the identification of response actions; see [link / reference]. Figure 9 The first and second judgment results are input into a preset signal response device to generate a corresponding response action, including:

[0171] The system receives the three-dimensional state code of the first judgment result and the four-tuple code of the second judgment result, which integrates the three-dimensional code of the circuit breaker's physical state: switch position, mechanical health, insulation status and fault root cause, and the four-tuple code: fault type, location, time and confidence level, forming a full-dimensional decision basis to support complex fault handling.

[0172] A multimodal response decision model is constructed, and the encoded data is dynamically fused to generate a response decision set. That is, by dynamically assigning weights, such as prioritizing mechanical faults when the mechanical health weight is greater than the insulation state, the multimodal data is fused to generate a response decision set, such as "locking the switch + starting dehumidification".

[0173] Based on the root cause of the fault, a four-tuple coding strategy is generated to produce a composite response strategy. According to the root cause of the fault in the four-tuple coding, such as "contact wear + harmonic overload", a multi-action coordinated strategy is generated, such as "derating operation + harmonic filtering + maintenance early warning".

[0174] Based on the response decisions triggered in the composite response strategy and response action set, response actions are generated. That is, based on strategy priority, such as immediate blocking of high-confidence faults, and real-time equipment status, such as prohibiting tripping when energy storage pressure is insufficient, safe and feasible response actions are dynamically selected.

[0175] In practical implementation, the three-dimensional status coding described in this application includes a binary identifier of the switch position, a percentage value of mechanical health, and an insulation status level code; the four-tuple coding includes an electrical protection matching score, a mechanical action reliability index, a latent defect probability value, and a maintenance priority weight; a fuzzy logic controller is used to resolve the compatibility constraints between the switch position and the insulation status, and when the insulation status level code exceeds a preset risk threshold, a forced shutdown action priority overlay mechanism is triggered; the attenuation coupling coefficient between mechanical health and mechanical action reliability index is calculated through a spatiotemporal correlation matrix, and a preventive opening action command is generated when the coefficient is lower than a dynamic adjustment threshold; when the electrical protection matching score is lower than the first critical value and the latent defect probability is higher than the second critical value, a three-level fault action response is executed.

[0176] a) Level 1 response: Send an adaptive blocking signal to the relay protection device to suppress unnecessary tripping;

[0177] b) Secondary response: Activate the pre-pressurization module of the circuit breaker's arc-extinguishing chamber and start the vibration suppression device;

[0178] c) Level 3 response: Generate equipment isolation commands and topology reconfiguration schemes with maintenance priority weights;

[0179] When the machine's health percentage exceeds the safety corridor for N consecutive cycles, a progressive shutdown action sequence is triggered:

[0180] 1) First stage: Reduce the circuit breaker operating current to the maintenance mode threshold;

[0181] 2) Second stage: Activate the backup mechanical interlock device to take over the operation authority;

[0182] 3) Third stage: Upload the irreversible blocking request to the power grid dispatch master station.

[0183] Finally, the response action is encrypted and signed and stored on the blockchain through a Trusted Execution Environment (TEE) to generate a tamper-proof operation log containing a three-dimensional state snapshot, response logic chain, and timestamp.

[0184] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for monitoring the position status of a circuit breaker switch based on video recognition technology, characterized in that, include: Real-time images of the circuit breaker are captured by camera equipment, and the physical state of the circuit breaker is determined to establish the first judgment result. Acquire real-time data from the relay protection device and record the operating status of the circuit breaker to determine the second judgment result; The first and second judgment results are input into a preset signal response device to generate corresponding response actions; wherein, the response actions include: an activation action, a fault action, and a deactivation action; wherein, the step of inputting the first and second judgment results into the preset signal response device to generate corresponding response actions includes: Receive the three-dimensional state code of the first judgment result and the quadruple code of the second judgment result; Construct a multimodal response decision model, dynamically fuse coded data, and generate a set of response decisions; A strategy for generating composite responses based on quadruple encoding for fault root cause tracing. Generate response actions based on the composite response strategy and the response decisions triggered in the set of response actions; The method of acquiring real-time images of the circuit breaker via camera equipment also includes: A multi-resolution scene database is pre-built, and adaptive optics analysis is performed to determine the resolution status of the current scene; Adaptive optics analysis includes multispectral imaging analysis, dynamic polarization analysis, and interference light source array analysis. Based on multispectral imaging analysis, the inter-frame error of the visible light image and near-infrared feature image under alternating acquisition is calculated. Based on dynamic polarization analysis, the real-time polarization angles of the rotatable linear polarizer array and the annular phase delay plate of the camera device are calculated. Based on the analysis of the interference light source array, a structured light distribution array for the current scene is generated to determine the areas that do not meet the preset clarity and require supplemental lighting. Based on inter-frame error, polarization angle, and the area to be illuminated, an image enhancement network for real-time images is constructed to enhance the real-time images. The image enhancement network includes an input layer, a data alignment sub-network, a physical feature encoder, a conditional modulation enhancement unit, and a cascaded enhancement strategy unit; wherein... The input layer includes image data for receiving real-time images through signal channels; wherein, the signal channels include a visible light image channel, a near-infrared image channel, a polarization angle tensor channel, and a structured photomask channel; The data alignment sub-network is used to perform the first enhancement process on the image data through the spatiotemporal alignment network, thereby eliminating inter-frame errors; The physical feature encoder is used to extract different modal features from image data through a multi-branch parallel structure and generate a multimodal feature map; The conditional modulation enhancement unit is used to determine the image region that needs to be enhanced by using the multimodal feature map and inter-frame error as a conditional vector; The cascaded enhancement strategy unit is used to stack multiple dynamic enhancement blocks to enhance image regions of different scales.

2. The method for monitoring the position status of a circuit breaker switch based on video recognition technology as described in claim 1, characterized in that, The real-time images include visible light images, near-infrared images, depth thermal maps, and images after supplemental lighting.

3. The method for monitoring the position status of a circuit breaker switch based on video recognition technology as described in claim 1, characterized in that, The multi-branch parallel structure includes: a spectral branch, a polarization branch, and a supplementary light branch; Among them, the spectral branch is used to extract reflective features and texture features separated by channel attention extraction; Polarization branching is used to extract rotationally weighted features guided by the polarization angle of the camera device and suppress metallic reflection features through directionally separable convolution; The supplementary lighting branch is used to determine the grayscale deviation characteristics of the image through mask-guided local contrast enhancement.

4. The method for monitoring the position status of a circuit breaker switch based on video recognition technology as described in claim 1, characterized in that, The determination of the physical state of the circuit breaker and the determination of the first determination result include: A multimodal feature fusion network is constructed. Visible light images, near-infrared images, depth thermal maps, and images after supplemental lighting are input into the spatiotemporal alignment module. Texture features, thermodynamic features, and polarization suppression features are fused through a cross-modal attention mechanism to generate a high-confidence circuit breaker state feature map. Based on a pre-defined physical constraint state decision tree, key regions in the state feature map of high-confidence circuit breakers are classified and judged hierarchically. First-level judgment: Identify the contact closure / separation state based on the near-infrared reflectivity and temperature rise gradient of the contact area; The second layer of judgment: Based on the integrity of the outline of the mechanical interlocking device in the image after polarization suppression, detect mechanical jamming or deformation; The third layer of judgment: by the ratio of the abnormal local contrast of the supplementary lighting area to the original image, the level of dirt on the surface of the insulating component is determined to be abnormal. Output a circuit breaker physical status report that integrates multi-modal judgment results, determine the first judgment result, which includes a three-dimensional status code of switch position, mechanical health and insulation status.

5. The method for monitoring the position status of a circuit breaker switch based on video recognition technology as described in claim 1, characterized in that, The process of acquiring real-time data from the relay protection device, recording the operating status of the circuit breaker, and determining the second judgment result includes: Based on the relay protection device, a multi-source data acquisition device is configured to acquire the real-time electrical parameters of the relay protection device and the multi-dimensional operating status data of the circuit breaker. A time-aligned multi-source data fusion model is constructed to perform dynamic collaborative analysis of the real-time electrical parameters and multi-dimensional operating status data, and a fault-action mapping matrix is ​​generated. Multi-dimensional anomaly detection is performed on the fault-action mapping matrix using an autoregressive anomaly propagation model. Generate a second determination result containing the root cause of the fault, and output it in a quadruple encoding format.

6. The method for monitoring the position status of a circuit breaker switch based on video recognition technology as described in claim 5, characterized in that, The real-time electrical parameters include the protection action signal waveform, fault current amplitude, trip circuit impedance, and harmonic distortion rate. The multidimensional operating status data includes the mechanical vibration spectrum of the circuit breaker, the current curve of the opening and closing coils, the pressure value of the energy storage mechanism, and the wear index of the switch contacts.

7. The method for monitoring the position status of a circuit breaker switch based on video recognition technology as described in claim 5, characterized in that, The autoregressive anomaly propagation model includes a deviation alarm layer, a risk identification layer, and a fault location layer; wherein... The deviation alarm layer is used to compare the deviation between the time the trip command is issued and the actual opening time of the circuit breaker. When the deviation exceeds a preset threshold, a mechanical response delay alarm is triggered. The risk identification layer is used to analyze the nonlinear relationship between the number of protective device actions and the contact electrical life loss curve, and to identify the hidden fault risks caused by abnormally frequent actions. The fault location layer is used for coupled analysis based on vibration spectrum entropy and energy storage pressure fluctuation to locate faults such as jamming of the mechanism or spring fatigue.

Images