Information identification method and information identification system for substation protection device

Abstract

The application discloses a substation protection device information identification method and information identification system, and belongs to the technical field of computer vision. The method comprises the following steps: obtaining a short exposure image and a long exposure image of a substation protection device at the same time; obtaining a basic feature map based on the short exposure image and the long exposure image; performing hierarchical feature extraction on the basic feature map according to a global attention network, a local attention network and a super-local attention network, generating a normalized attention map, and generating a fusion feature map based on the short exposure image, the long exposure image and the normalized attention map; performing dynamic weight distribution fusion on the short exposure image and the long exposure image to obtain an enhanced image; performing stain area detection and stain repair processing on the enhanced image to output a stain-removed image; identifying a trip code and an indicator light state from the stain-removed image and outputting an information identification result. The embodiment of the application can effectively improve the accuracy of substation protection device trip information identification.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an information identification method and system for substation protection devices. Background Technology

[0002] Substation protection devices are crucial for the safe operation of power systems. Their tripping information (such as display screen characters and indicator light status) directly reflects the fault type and location, serving as vital evidence for maintenance personnel to quickly isolate faults and restore power. Existing technologies typically rely on fixed cameras to collect tripping information. However, fixed cameras are susceptible to environmental interference, leading to inaccurate identification of tripping information. Summary of the Invention

[0003] The purpose of this application is to provide an information identification method and system for substation protection devices, which can effectively improve the accuracy of tripping information identification for substation protection devices.

[0004] To achieve the above objectives, a first aspect of this application provides an information identification method for substation protection devices, comprising: Acquire short-exposure and long-exposure images of the substation protection device at the same time, wherein the exposure parameters of the short-exposure and long-exposure images are different; Based on the short-exposure image and the long-exposure image, a basic feature map is obtained. According to the global attention network, local attention network and hyperlocal attention network, the basic feature map is subjected to hierarchical feature extraction to generate a normalized attention map. A fused feature map is generated based on the short-exposure image, the long-exposure image and the normalized attention map. Based on the fused feature map and the normalized attention map, the short exposure image and the long exposure image are dynamically weighted and fused to obtain an enhanced image. The enhanced image is subjected to stain area detection and stain repair processing to output a stain-removed image; Identify the trip code and indicator light status from the stain removal image and output the identification result.

[0005] Compared with existing technologies, the information identification method for substation protection devices provided in this application has the following advantages: By simultaneously acquiring short-exposure and long-exposure images of the substation protection device, details in highlight areas and information in dark areas can be preserved respectively; relying on a three-layer attention network of global-local-superlocal, the basic feature map is extracted layer by layer to generate a normalized attention map, which can simultaneously take into account both global layout and local details; dynamic weight allocation fusion is achieved based on the fusion feature map and the normalized attention map, which can adaptively adjust the fusion weight of long and short exposure images for different regions of the image, overcoming the limitation of existing fixed weight fusion that is difficult to adapt to the local lighting differences in substations, and outputting enhanced images; targeted stain detection and repair can eliminate the obstruction and interference of stains such as dust and oil stains on the information area; finally, the trip code and indicator light status are identified, so that the information identification results can still maintain a high accuracy rate under complex environmental interference, effectively improving the accuracy of trip information identification of substation protection devices, and thus effectively improving the power grid operation and maintenance response efficiency and safe operation guarantee capability.

[0006] In some embodiments, the step of performing hierarchical feature extraction on the basic feature map based on a global attention network, a local attention network, and a hyperlocal attention network to generate a normalized attention map, and generating a fused feature map based on the short-exposure image, the long-exposure image, and the normalized attention map, includes: Based on the basic feature map, corresponding global attention map, local attention map and hyperlocal attention map are generated via the global attention network, the local attention network and the hyperlocal attention network; The global attention map, the local attention map, and the hyperlocal attention map are normalized to obtain a normalized attention map, which includes a normalized global attention map, a normalized local attention map, and a normalized hyperlocal attention map.

[0007] In some embodiments, the step of performing stain area detection and stain repair processing on the enhanced image to output a stain-removed image includes: The enhanced image is matched and fused pixel by pixel. The pixel values ​​of the fused feature map are determined to be less than a preset threshold for initial screening to obtain stain candidate areas. By calculating the difference between the pixel grayscale and the mean grayscale value within the candidate stain area, the suspected stain area is determined; Based on whether the suspected stain area overlaps with the character core area marked by the normalized local attention map or the edge area marked by the normalized hyperlocal attention map, the non-overlapping part is saved and the real stain area is obtained. The non-edge stain areas in the real stain area are repaired using neighborhood mean filtering, while the stain areas near the edges in the real stain area are repaired using bilateral filtering, and the stain-removed image is output.

[0008] In some embodiments, identifying the trip code and indicator light status from the stain removal image and outputting the identification result includes: The connected regions of the normalized local attention map are extracted from the stain removal image to obtain character candidate regions. After local adaptive binarization and median filtering preprocessing, the character candidate regions are used to identify trip codes and equipment number characters using a CRNN model. The closed connected regions of the normalized hyperlocal attention map are extracted from the stain removal image and filtered by region area to obtain the real indicator light area. After calculating the color and brightness features of the real indicator light area, the indicator light status is classified using an SVM model. Information recognition results are constructed based on the trip code and the indicator light status.

[0009] In some embodiments, the construction of information identification results based on the trip code and the indicator light status includes: Cross-validation was performed on the trip code and the indicator light status to obtain the verification results; If the verification result is successful, then an information recognition result is constructed based on the successful verification result.

[0010] In some embodiments, obtaining the basic feature map based on the short-exposure image and the long-exposure image includes: The basic features of the short-exposure image and the long-exposure image are extracted by the first three layers of the ResNet-18 network and then fused across exposures to obtain a basic feature map.

[0011] To achieve the above objectives, a second aspect of the present application provides an information identification system for substation protection devices, which applies the information identification method for substation protection devices as described in the first aspect above. The system includes a hardware layer and a software layer. The hardware layer includes the following modules connected in sequence: The image acquisition module is used to simultaneously acquire short-exposure and long-exposure images of substation protection devices; The processing module is used to run the software layer and obtain the information recognition result; A storage module is used to store the information recognition results; The communication module is used to interact with an external operation and maintenance platform and output the information recognition results; The software layer includes the following modules connected in sequence: The multi-exposure acquisition control module is used to generate an acquisition trigger signal to control the image acquisition module to acquire images, and to acquire short-exposure images and long-exposure images of the substation protection device at the same time, wherein the exposure parameters of the short-exposure images and the long-exposure images are different; The multi-scale attention feature extraction module is used to obtain a basic feature map based on the short exposure image and the long exposure image, perform hierarchical feature extraction on the basic feature map according to the global attention network, local attention network and hyperlocal attention network, generate a normalized attention map, and generate a fused feature map based on the short exposure image, the long exposure image and the normalized attention map. The dynamic fusion enhancement module is used to perform dynamic weight allocation fusion on the short exposure image and the long exposure image based on the fusion feature map and the normalized attention map to obtain an enhanced image. The stain detection and masking module is used to detect stain areas and repair stains in the enhanced image, and output a stain-removed image. The trip information recognition module is used to identify the trip code and indicator light status from the stain removal image and output the information recognition result.

[0012] In some embodiments, the image acquisition module is configured to simultaneously acquire a short-exposure image and a long-exposure image of the same scene of the substation protection device; the image acquisition module includes a supplementary lighting unit for providing illumination to the scene.

[0013] In some embodiments, the multi-exposure acquisition control module integrates a target detection algorithm. The multi-exposure acquisition control module is configured to generate an acquisition trigger signal based on the processing result of the target detection algorithm; and send the acquisition trigger signal to the image acquisition module to control it to simultaneously acquire short-exposure images and long-exposure images; and receive the short-exposure images and long-exposure images returned by the image acquisition module.

[0014] In some embodiments, a global attention network, which has 16×16 convolutional kernels, is configured to capture the overall layout features of the cabinet and output a global attention map; A local attention network, which has 8×8 convolutional kernels, is configured to focus on character semantic features and output a local attention map; The hyperlocal attention network, which has 4×4 convolutional kernels, is configured to capture character edge gradient features and output a hyperlocal attention map. Attached Figure Description

[0015] Figure 1 This is a flowchart of an information identification method for substation protection devices provided in an embodiment of this application; Figure 2 yes Figure 1 A flowchart of step S104 in the process; Figure 3 yes Figure 1 A flowchart of step S105 in the process; Figure 4 This is a schematic diagram of the information identification system of the substation protection device provided in the embodiments of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0017] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0018] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0019] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0020] Substation protection devices are core equipment for ensuring the safe and stable operation of power systems, and their tripping information (including characters on the display screen and indicator light status) is a key basis for power system fault diagnosis. Traditional tripping information collection relies on manual inspection or fixed cameras, which has three major technical bottlenecks: First, significant environmental interference, such as reflections on the glass surface of the protection cabinet (reflectivity exceeds 15% when the incident angle is >30°) and dirt (character recognition rate drops by 40% when dust coverage is >5%), easily leads to image quality degradation; second, limitations in feature extraction, such as traditional single-scale convolutional networks (e.g., ResNet-50) being unable to simultaneously capture global layout (e.g., cabinet structure) and local details (e.g., character edges), resulting in severe information loss in low-contrast areas; and third, the risk of semantic confusion, such as when indicator lights are similar in color to the background (e.g., red indicator lights against a dark cabinet), conventional threshold segmentation can easily misjudge their working status.

[0021] With the evolution of smart grids, automated data acquisition technology based on computer vision has become a research hotspot. However, the complex environment of substations places stringent demands on image acquisition quality. While existing technologies have attempted breakthroughs through hardware optimization (such as polarizing filters) and algorithm improvement (such as deep learning), both have significant limitations. 1. Hardware optimization scheme based on polarizing mirrors (such as Chinese patent CN201610962944): This scheme filters reflected light by adding polarizing mirrors (the polarizing mirror of the light source is parallel to the incident plane, and the polarizing mirror of the camera is perpendicular to it) in front of the camera and the light source. However, it is highly dependent on hardware (requires additional installation and precise angle adjustment, increasing equipment cost and maintenance complexity), has a narrow incident angle adaptation range (only effective around Brewster's angle ≈ 56°, and the reflection suppression effect drops sharply beyond this range), and does not involve stain detection and shielding, thus failing to solve the problem of dust and oil stain interference. 2. Deep learning-based object detection methods (such as improved YOLOv7, Chinese patent CN119782960A): These methods improve detection speed and accuracy through lightweight networks (such as introducing the Ghost module) and multi-scale fusion attention mechanisms. However, they are primarily designed for power equipment defect identification and lack scene adaptability (they are not optimized for reflections and stains on the display screen of the protection device, and the recognition accuracy drops below 70% in scenes with interference). Furthermore, feature extraction is incomplete (multi-scale attention only focuses on channel correlation, such as the SE module, and lacks global-local-superlocal spatial multi-scale fine extraction) and there is no multi-modal fusion design (they only process single visible light images and do not combine multi-exposure strategies to enhance details). 3. Multi-source data fusion method (such as Chinese patent CN117495825A): It fuses visible light and infrared images, and uses FPN pyramid structure and dynamic dimension convolution to realize tower pole foreign object detection. However, its application scenario is misaligned (it is mainly for tower pole foreign object detection and does not cover the close-range high-precision information collection needs of the protection device display screen), the fusion strategy is static (fixed weight fusion of data, which cannot dynamically adapt to the local features of reflective areas), and there is no special treatment mechanism for typical stains in substations.

[0022] Further analysis reveals three common shortcomings in existing technologies: First, environmental interference is not thoroughly addressed. Hardware solutions only address reflection issues, while algorithmic solutions rely on data augmentation for indirect optimization, neither of which specifically models interference features. Second, feature extraction granularity is limited. Traditional convolutional networks have fixed receptive fields, failing to consider both global and local information. Existing attention mechanisms only focus on channel importance, lacking spatial multi-scale analysis. Third, multimodal fusion efficiency is low. Multi-source data fusion does not consider multi-exposure enhancement strategies for the same modality (visible light), and the fusion weights cannot dynamically adapt to scene requirements.

[0023] Based on this, the purpose of this application embodiment is to provide an information identification method and system for substation protection devices, which can effectively improve the accuracy of tripping information identification for substation protection devices.

[0024] Please see Figure 1 , Figure 1 This is an optional flowchart of the information identification method for substation protection devices provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S105.

[0025] Step S101: Acquire short-exposure images and long-exposure images of the substation protection device at the same time. The exposure parameters of the short-exposure images and the long-exposure images are different. Step S102: Based on the short exposure image and the long exposure image, a basic feature map is obtained. According to the global attention network, the local attention network and the super local attention network, the basic feature map is subjected to hierarchical feature extraction to generate a normalized attention map. A fused feature map is generated based on the short exposure image, the long exposure image and the normalized attention map. Step S103: Based on the fused feature map and the normalized attention map, the short exposure image and the long exposure image are dynamically weighted and fused to obtain the enhanced image; Step S104: Perform stain area detection and stain repair processing on the enhanced image, and output a stain-removed image; Step S105: Identify the trip code and indicator light status from the stain removal image and output the identification results.

[0026] Steps S101 to S105 of this embodiment, by simultaneously acquiring short-exposure and long-exposure images of the substation protection device, can retain details in highlight areas and information in shadow areas respectively; relying on a three-layer attention network of global-local-superlocal, the basic feature map is extracted layer by layer to generate a normalized attention map, which can simultaneously take into account global layout and local details; dynamic weight allocation fusion is achieved based on the fused feature map and the normalized attention map, which can adaptively adjust the fusion weight of long and short exposure images for different regions of the image, overcoming the limitation of existing fixed weight fusion that is difficult to adapt to the local lighting differences of the substation, and outputting enhanced images; targeted stain detection and repair can eliminate the obstruction and interference of stains such as dust and oil stains on the information area; finally, the trip code and indicator light status are identified, so that the information identification results can still maintain a high accuracy rate under complex environmental interference, effectively improving the accuracy of trip information identification of substation protection devices, and thus effectively improving the power grid operation and maintenance response efficiency and safe operation guarantee capability.

[0027] In step S101 of some embodiments, the short exposure image can be an RGB image (2048×1536 resolution) captured using a 20-megapixel industrial camera (8-50mm adjustable focal length lens + 5500K ring light) at a shutter speed of 1 / 1000s and ISO 200, which can suppress reflections on the protective device's display screen glass. The long exposure image can be an RGB image of the same resolution captured using the same camera hardware at a shutter speed of 1 / 30s and ISO 800, which can enhance character details. Exposure parameters refer to parameters that control the amount of light entering the camera. In this step, precise control of the exposure parameters required for short and long exposure images is achieved by controlling the shutter speed (controlling the exposure time) and ISO (controlling the photosensitivity). The short exposure image uses a parameter combination of a shutter speed of 1 / 1000s and ISO 200. The extremely fast shutter speed can effectively freeze changes in light and significantly suppress specular reflections on the glass surface. Actual testing showed that under these parameters, the grayscale value of the reflective area in short-exposure images can be controlled to ≤50, minimizing the interference of reflected light spots on the image content and ensuring clear visibility of the display screen content. Long-exposure images are set with a shutter speed of 1 / 30s and ISO 800. The extended exposure time allows more light to enter the lens, while also increasing sensitivity and enhancing character details. Images acquired using these parameters achieve a character-to-background contrast ratio of ≥30, effectively capturing minute text, symbols, and status indicator light information on the display screen, providing rich image details for subsequent information recognition.

[0028] In one embodiment, multi-exposure image acquisition of the substation protection device is triggered by the YOLOv8 target detection algorithm, which achieves a detection accuracy of up to 99.2% for the protection device in a substation scenario. When the image acquisition module on the unmanned vehicle moves to an ideal range of 1.5m-2m from the substation protection device, short and long exposure acquisition are simultaneously initiated. The images are accompanied by millisecond-level timestamps for temporal alignment in the subsequent generation of basic feature maps. The aforementioned distance ensures that sufficiently clear details of the device's display screen are captured while avoiding image distortion and increased reflection caused by shooting too close.

[0029] In step S102 of some embodiments, the basic feature map (F) n This refers to extracting short-exposure image features (F) through the first three 3×3 convolutional layers of ResNet-18 (stride 1, padding=1). s ), long exposure image features (F l ), and then element-level addition (F n =F s +F l The resulting fusion feature carrier (size 512×384) was obtained.

[0030] Specifically, using the first three 3×3 convolutional layers of ResNet-18 (stride 1, padding=1), the short-exposure image I is processed respectively. s Long exposure image I l The process involves extracting low-level visual features such as edges and textures layer by layer, ultimately yielding the feature map F. s (Dimensions 512×384) and Feature Map F l (Dimensions 512×384); Next, element-wise addition (F n =F s +F l Generate the base feature map F after cross-exposure fusion. n This basic feature map effectively integrates I s The reflectivity suppression advantage (low reflectivity region characteristics) and I l The detail retention advantage (high-contrast character features) provides multi-dimensional visual information input for subsequent attention mechanisms.

[0031] It should be noted that the global attention network has 16×16 convolutional kernels and is configured to capture the overall layout features of the display cabinet and output a global attention map; the local attention network has 8×8 convolutional kernels and is configured to focus on the semantic features of characters and output a local attention map; the ultra-local attention network has 4×4 convolutional kernels and is configured to capture the gradient features of character edges and output an ultra-local attention map.

[0032] In some embodiments, based on the base feature map, corresponding global attention maps, local attention maps, and hyperlocal attention maps are generated via a global attention network, a local attention network, and a hyperlocal attention network; The global attention map, local attention map, and superlocal attention map are normalized to obtain a normalized attention map, which includes a normalized global attention map, a normalized local attention map, and a normalized superlocal attention map.

[0033] Specifically, based on the basic feature map F n Hierarchical feature extraction is achieved through differentiated convolutional kernel configuration, generating attention weight matrices at corresponding scales. A 16×16 large convolutional kernel (stride 8) is used to apply F... n Perform a global semantic scan to capture the overall layout features of the cabinet (such as the spatial positioning of the display screens and the arrangement of modules) and generate a global attention map. The output resolution is 256×192. The weight range is [0,1]. High-weight regions identify structural features that play a key role in display screen positioning (such as cabinet boundaries and module spacing). Furthermore, an 8×8 mesoscale convolution kernel (stride 4) is used to focus on the pre-positioned display screen ROI region, enhancing the extraction of semantic features such as numbers and symbols, and generating a local attention map. The output resolution is 512×384. A masking mechanism is used to force character region weights to be ≥0.8, ensuring that key information dominates feature fusion. A 4×4 small convolutional kernel (stride 2) is used to capture gradient change features of character edges (such as 0.5mm-level stroke contours), generating a super-local attention map. The output resolution is 1024×768. After gradient magnitude normalization, the weight of edge pixels is increased to ≥0.9, which enhances the feature expression ability of low-contrast edges.

[0034] First to , , Upsampling and downsampling are performed to a unified resolution of 512×384 (upsampling of the global attention map and downsampling of the hyperlocal attention map) to eliminate size differences. Cross-scale weight equalization is then used to eliminate numerical differences between branches, resulting in a normalized attention map, which includes the normalized global attention map. ), Normalized local attention map ( ) and normalized hyperlocal attention map ( ).

[0035] It should be noted that during the normalization process, it is necessary to ensure that the sum of the weights of the three types of attention is 1. This allows for the formation of a standardized weight matrix that can directly drive feature fusion, providing a unified metric for the dynamic weighting of subsequent double-exposure images.

[0036] In summary, by using differentiated convolution kernels to obtain F n The system extracts global layout, local characters, and hyperlocal edge features in a hierarchical manner to generate corresponding attention maps. After scale unification and normalization, a standardized matrix with a weight sum of 1 is obtained, providing a quantitative basis for dynamic fusion of double-exposure images. The generation mechanism of the three core attention maps breaks through the redundant mode of traditional independent processing of double-exposure images. Through the technical path of fusion feature input-scale hierarchical extraction-weight normalization, it achieves progressive feature enhancement from global layout to hyperlocal details, providing multi-dimensional attention guidance for information acquisition in complex scenes. The normalized weights are multiplied pixel-by-pixel with the basic features of the input images (short-exposure and long-exposure images) (extracted through the first 3 layers of ResNet-18) to obtain the fused feature map. In this formula, This represents image features extracted from a global perspective. This represents image features extracted from a local perspective. This represents image features extracted from a hyperlocal perspective. Through this fusion method, the system can not only comprehensively consider the overall layout information of the control cabinet, but also delve into the details of the character areas and their edges, thereby obtaining more comprehensive and representative image features, providing strong support for the accurate analysis of subsequent tripping information.

[0037] In step S103 of some embodiments, relying on the fused feature map (F) and the normalized attention map (F) , , By accurately identifying reflective areas, dynamically fusion weights, and layered fusion adaptation, short-exposure images can be achieved. (Advantages in suppressing reflection) and long exposure images ( Through deep optimization and fusion (preserving the advantages of detail), an enhanced image is obtained.

[0038] First, accurately identify reflective areas. For the fused feature map F, extract candidate highlight areas with grayscale values ​​> 200. The candidate highlight area encompasses both the reflective area of ​​the display cabinet to be judged and the high-brightness character area to be retained. Then, based on the candidate highlight area... and Pixel-by-pixel matching, preservation Area: After cross-scale normalization, the weight of areas only related to the overall structure of the cabinet (such as the glass surface of the display screen and the metal frame of the cabinet) is significantly higher than that of local character areas. This can directly eliminate localized false reflections such as "character highlighting"; and remove... Areas with high reflectivity: These areas mainly contain valid information such as characters and indicator lights, and are not considered reflective risk areas related to the overall structure. Through the aforementioned steps, the core structural areas of the cabinet (such as the display glass surface and cabinet frame) are retained, while valid information areas such as characters and indicator lights are removed. Finally, a gradient change is calculated using a 10×10 pixel sliding window. The true reflective area is confirmed through gradient verification (gradient value < 5): if the gradient value within the window is < 5 (the pixel grayscale of the reflective area is uniform, and the gradient change is gradual), it is ultimately determined to be a true reflective area. Conversely, the non-reflective area is the opposite. Using normalized It can eliminate the original numerical overlap (original) (Potentially confused with character area weights), thus accurately distinguishing between "global structural reflection" and "local character highlighting," reducing the false judgment rate of reflection from 3.2% to 1.5%, and effectively eliminating false reflection interference.

[0039] Furthermore, utilizing (Reflective area weight) (Character area weight) and (Fused feature maps) assign dynamic weights to different regions. First, because of the true reflective areas... The core requirement is to "maximize the short exposure image". The advantage of suppressing reflections, the weight of short-exposure images ( (the proportion in the integration) is determined by The weight of long exposure images is dynamically determined. Weights of short exposure images The sum is 1: The higher the value, the more likely the area is to be a "highly reflective zone of the entire display cabinet structure" (such as reflections from the center of the display screen glass), requiring a higher value. Suppress reflections. For example: (Strong reflection) ( (Dominant integration, powerful anti-reflective properties) (Weak reflectivity) (Balancing slight reflection suppression with preservation of basic details).

[0040] For non-reflective areas Because the non-reflective area can be further subdivided into the character core area and the background structure area, therefore, combined with (Focusing on character semantics) and fusion feature maps The low-frequency components (reflecting the global structure) are used to achieve differentiated weight allocation.

[0041] Character core area: when hour( After normalization, the character semantic region has a significantly higher weight than the background region, and is therefore identified as a character-dominated region that needs to be prioritized for preservation. The character details are given by the formula: For example, the core semantic area of ​​"0" in the character "T01" ,at this time ( Dominate and enhance character texture restoration.

[0042] Background structure area: when Time (such as the frame of the screen cabinet, blank areas), combined with low-frequency components ( (For global structures with low detail requirements), equal weight allocation is used: Balanced short exposure images Structural clarity and long exposure images The noise reduction effect.

[0043] For the edge transition region, because and In the boundary region (5-8 pixels wide), abrupt weight changes can easily produce fusion "ghosting," so a hyperlocal attention map is introduced. (Focusing on edge details), combined with standard deviation Gaussian smoothing, precisely controlling the smoothing range: when When (key edge areas such as character edges and indicator light outlines), reduce the smoothing radius to 2 pixels to avoid edge blurring; when During the background transition area (without key details), perform full-range smoothing to eliminate visual artifacts caused by weight abrupt changes.

[0044] As mentioned above, The main focus is on "adapting the intensity of reflective areas". Dominate "character area detail preservation", Dominate "edge region smoothing control", The system takes the lead in "background area weight balancing" and the four elements work together to achieve "on-demand allocation of fusion weights".

[0045] Furthermore, relying on the 6-layer Laplacian pyramid fusion technology, and based on the "global-local-hyperlocal" feature distribution of F, [the following is achieved / implemented]... , , The function is embedded in different pyramid layers: the low-frequency layer (layers 1-2, resolution ≤ 256×192) corresponds to the global structural features of F (cabinet layout), with The primary allocation of weights maintains the stability of the cabinet layout; the mid-frequency layer (layers 3-4, resolution 512×384~1024×768) focuses on the local character features (semantic information) of F, in order to To lead and enforce High weights enhance character texture details; high-frequency layers (5-6 layers, resolution ≥2048×1536) rely on the superlocal edge features (contours) of F to... To guide the improvement of edge gradient continuity (measured edge sharpness improved by 30%).

[0046] After decomposition of each pyramid level coefficient layer , coefficient layer ( (Based on the number of layers), weights are assigned and fusion is performed according to the following rules: Low-frequency layers (layers 1-2): with global structural features ( Guided by the principle of "primary", the reflective area adopts... Non-reflective areas are set To maintain the stability of the cabinet layout. Mid-frequency layer (layers 3-4): Focusing... Local character features ( Dominant), mandatory in all areas Specifically Enhance the texture details of characters such as "T01". High-frequency layer (layers 5-6): based on... Edge features ( (leading), set , The higher (e.g., the red border of the indicator light). The larger the value, the more edge gradient information is preserved (actual measurement shows an improvement of 30% in edge sharpness).

[0047] Obtain the enhanced image of the k-th layer The formula is: Finally, the enhanced image is reconstructed using the inverse Laplacian pyramid transform. (Resolution 2048×1536). This step eliminates redundant features, providing a high-quality image foundation for accurate acquisition of tripping information.

[0048] Please see Figure 2 In some embodiments, step S104 may include, but is not limited to, steps S201 to S204: Step S201: Match and fuse the feature map pixel by pixel in the enhanced image, and determine whether the pixel value of the fused feature map is less than a preset threshold for initial screening to obtain the stain candidate area. Step S202: By calculating the difference between the pixel grayscale and the mean grayscale value within the candidate stain area, the suspected stain area is determined; Step S203: Based on whether the suspected stain area overlaps with the character core area marked by the normalized local attention map or the edge area marked by the normalized hyperlocal attention map, save the non-overlapping part and obtain the real stain area. Step S204: Neighborhood mean filtering is used to repair non-edge stain areas in the real stain area, and bilateral filtering is used to repair stain areas near the edges in the real stain area, and the stain-removed image is output.

[0049] In step S201 of some embodiments, the enhanced image is... Pixel-wise matching and fusion feature maps Only retain The area is marked as a "stain candidate area". ".because high value range ( ) Corresponding valid information (characters) Edge of indicator light However, stains lack effective characteristics. Values ​​are generally lower than This step can reduce the detection range to less than 20% of the original image, reduce unnecessary computation by 60%, and improve detection efficiency.

[0050] In step S202 of some embodiments, the stain candidate area is... Perform Otsu adaptive threshold segmentation to filter grayscale abnormal regions: First, calculate candidate stain regions. mean gray level within (Based on the background characteristics of substation cabinets) Typically, it falls between 170 and 190; the exact range can be slightly adjusted depending on the material of the screen cabinet. When the grayscale of a certain pixel is... If the difference is greater than 20 (satisfying the "local grayscale change" characteristic), and all 3×3 pixel blocks meet this condition, it is marked as a "suspected stain area". "; Distinguish between "stains" and "normal background" by "difference in grayscale uniformity" - normal background has a smooth grayscale transition, while stains will cause sudden changes in local grayscale (such as "dark spots" caused by dust and "bright spots" caused by oil stains).

[0051] In step S203 of some embodiments, the suspected stain area is... Perform "Valid Information Removal" to avoid misidentifying characters and edges as stains. Character core area filtering: If Memory exists For regions (such as "0" in "T01"), directly remove that part to eliminate the risk of misjudging character blobs; edge segmentation: if and The edge areas (indicator light borders, character corners) overlap, only retaining... By avoiding misjudgment of edge shadows in the non-overlapping areas, the final output can accurately reflect the true stain area. No characters, edges, or other critical tripping information were mixed in.

[0052] In step S204 of some embodiments, considering the different characteristics of non-edge stain areas and stains near the edges, combined with A differentiated repair strategy is employed to mask stains while preserving the edge details of characters and indicator lights to the greatest extent possible. For non-edge stain areas, a 5×5 neighborhood mean filter is used for repair, while for areas with actual stains... middle For areas with visible dirt (such as dust in the blank background of the substation cabinet), a 5×5 neighborhood mean filter is used for repair. Specifically, focusing on the dirty pixel, non-dirty pixels within a 5×5 window are selected, their grayscale mean is calculated, and the original dirty pixel value is replaced. This balances repair efficiency (single window calculation time <0.1ms) with smooth transition, avoiding abrupt color blocks after repair and adapting to the single background characteristics of substation cabinets. For dirt areas near the edges, a bilateral filter is used for repair. For areas near the edges (such as oil stains next to the indicator light frame or dirt near the vertical line of the character "1"), bilateral filtering is used for repair. This approach considers both "spatial distance" and "grayscale similarity"—only non-stain pixels that are spatially close to the stain pixels and have a grayscale difference of <10 (based on the grayscale contrast characteristics of the display cabinet) are selected, with priority given to pixels on the same side of the edge. While smoothing the stains, the edge gradient differences are fully preserved (e.g., the grayscale difference between the "red" and "black" borders of the indicator lights remains >50), avoiding edge blurring and ensuring accurate contour capture in subsequent recognition. Finally, the destaining image is output. .

[0053] Please see Figure 3 In some embodiments, step S105 may include, but is not limited to, steps S301 to S303: Step S301: Extract the connected regions of the normalized local attention map from the stain removal image to obtain the character candidate region. After local adaptive binarization and median filtering preprocessing, the character candidate region is used to identify the trip code and equipment number characters using the CRNN model. Step S302: Extract the closed connected regions of the normalized hyperlocal attention map from the stain removal image and filter them by region area to obtain the real indicator light area. After calculating the color features and brightness features of the real indicator light area, use the SVM model to classify the indicator light status. Step S303: Construct information identification results based on trip codes and indicator light status.

[0054] In step S301 of some embodiments, considering the characteristics of inconsistent character font specifications in substation cabinets (significant differences between cabinets from different manufacturers) and susceptibility to interference from cabinet texture, combined with... To achieve fast character location and accurate recognition. First, use... Quickly locate the character area for stain removal images Direct extraction Connected regions are marked as character candidate regions. Traditional sliding window positioning methods require traversing the entire image, taking approximately 0.8-1.2 seconds per image, while this method only needs to extract... In the high-value area (time < 0.1s), positioning efficiency is improved by more than 87.5%; and Generated from character semantic features, the positioning accuracy reaches 99%, and it can completely eliminate misjudgments of background areas such as cabinet screws and textures.

[0055] Furthermore, on Binarization and denoising preprocessing are performed to enhance the distinction between characters and background (to address the issue of uneven grayscale values ​​of characters on the display cabinet). First, a local adaptive threshold is applied (based on the average grayscale value of the character area). The threshold is set to The characters are converted to black (grayscale 0) and the background is converted to white (grayscale 255). Based on 100 sets of screen cabinet character samples, this threshold can cover more than 95% of character scenes and effectively eliminate the slight reflection residue on the character surface. Secondly, a 3×3 median filter is used to remove the small noise residue after binarization (such as character edge burrs and cabinet texture residue) to avoid interfering with the subsequent model feature extraction.

[0056] Finally, a CRNN model is used to recognize characters. A convolutional recurrent neural network (CRNN) model is employed to recognize the preprocessed character area, adapting to the characteristics of substation trip codes. Because CRNN excels at handling character sequences of varying lengths (substation trip codes cover 2-4 digit formats such as "T01", "T102", and "T2005"), and has strong tolerance for font tilt (within ±8°, a common deviation range for substation cabinet installation) and slight blurring, no additional image correction is required. During character recognition, a convolutional layer (Conv2d) first extracts character texture features (such as the horizontal and vertical intersections of "T" and the closed arc of "0"), then a bidirectional recurrent layer (Bi-LSTM) captures the character sequence relationships (such as "T" always being followed by a number), and finally, a transcriptional layer (CTC) outputs the recognition result (such as "T01").

[0057] If the recognition result conforms to the substation trip code specification (the first character is "T", followed by numbers 0-9, with a length of 2-4 digits), it is marked as a "valid trip code"; if it does not conform, the character area is automatically reprocessed (e.g., the binarization threshold is adjusted by ±5). If two consecutive recognitions are inconsistent, a "manual review required" prompt is output to ensure the reliability of the recognition.

[0058] In step S302 of some embodiments, because the indicator light is small (actual diameter 5-10mm) and easily confused with the background color block, this step uses... Segment the indicator light area. Extract the stain removal image. middle The closed connected area (the indicator light has a clear metal frame, (This will form a closed edge outline), marked as "indicator candidate area". "; Then, filter by area (calculated based on an image resolution of 2048×1536, the corresponding pixel area of ​​an actual 5-10mm indicator light is 20-50 pixels). 2 After eliminating excessively small edge interference areas (such as cabinet screw shadows or scratches), the "real indicator light area" is finally obtained. ".

[0059] Then, extract the actual indicator light area. Color and brightness features. For color features, the smudge-removed image... The original RGB channel information is preserved, so the average red component of the indicator light area can be calculated directly. Green components —According to substation operation and maintenance specifications, a red light indicates equipment tripping / fault, a green light indicates normal operation, and a yellow light indicates an alarm. For brightness characteristics, the average grayscale value of the indicator light area is calculated. ,like If the brightness is extremely low, it is determined to be "light off", eliminating "misjudgment caused by bulb damage" (e.g., green light off ≠ tripping, it is necessary to make a comprehensive judgment based on the status of other indicator lights).

[0060] Finally, a Support Vector Machine (SVM) model was used to classify the indicator light states. The model training data came from 1000 sets of substation field samples (covering multiple manufacturers' cabinets, including scenarios such as strong daylight and weak nightlight, covering four states: red / green / yellow on and off). ratio, "As a 2D input feature (low computational cost, suitable for edge device computing power); if..." and The result is determined to be "red light on (power outage)"; if and If the light is green (normal), it is determined to be "green light on (normal)"; (Difference < 10) and If the light is yellow (alarm) on; The light was determined to be "off". Combined with... After precise segmentation, the SVM classification accuracy reached 98.2%, and it could still maintain an accuracy of over 96% under complex conditions such as strong light reflection and cabinet shadows, which is far higher than the traditional "fixed color threshold method" (85%).

[0061] In step S303 of some embodiments, this application further includes cross-validating the trip code and indicator light status to obtain a verification result; if the verification result is successful, an information identification result is constructed based on the successful verification result. Cross-validating the trip code identification result with the indicator light status result can ensure information consistency and ultimately output standardized trip information that can be used in engineering.

[0062] Specifically, if the red light is on (tripped) and a compliant trip code (such as "T01") is identified, it is determined as "valid trip information"; if the green light is on (normal) but a code is identified, the character area is automatically re-checked (it may misjudge the cabinet texture as characters); if the red light is on but there is no code, the stain removal image is called and partially magnified (2 times) to re-identify the character area (there may be slight occlusion).

[0063] Upon successful verification, the system outputs "trip code + indicator light status + identification confidence level + timestamp" and generates structured data in JSON format (example: {"trip_code":"T01","light_status":"red light on","confidence":0.985,"timestamp":"2025-08-30 14:25:30"}), facilitating direct access by the substation monitoring system. The identification confidence level can be obtained based on a probability-based model (such as Softmax regression), but is not limited to this. The timestamp is a millisecond-level timestamp accompanying the acquisition of short- or long-exposure images. This application can also provide a standardized data interface for substation SCADA (Supervisory Control and Data Acquisition) systems, supporting real-time alarm pop-ups (automatically pushed to the maintenance terminal upon tripping) and historical data tracing (storing tripping records for the past year), providing direct evidence for fault location (e.g., "T01" corresponds to feeder circuit 1).

[0064] Please see Figure 4 This application embodiment also provides an information identification system for substation protection devices, which applies the above-mentioned information identification method for substation protection devices. The system includes a hardware layer 410 and a software layer 420. Hardware layer 410 includes the following modules connected in sequence: Image acquisition module 411 is used to simultaneously acquire short-exposure images and long-exposure images of substation protection devices; Processing module 412 is used to run software layer 420 to obtain information recognition results; Storage module 413 is used to store the information recognition results; Communication module 414 is used to interact with external operation and maintenance platforms and output information recognition results; Software layer 420 includes the following modules connected in sequence: The multi-exposure acquisition control module 421 is used to generate an acquisition trigger signal to control the image acquisition module to acquire images, and to acquire short-exposure images and long-exposure images of the substation protection device at the same time. The exposure parameters of the short-exposure images and long-exposure images are different. The multi-scale attention feature extraction module 422 is used to obtain basic feature maps based on short-exposure images and long-exposure images, perform hierarchical feature extraction on the basic feature maps according to the global attention network, local attention network and hyperlocal attention network, generate normalized attention maps, and generate fused feature maps based on short-exposure images, long-exposure images and normalized attention maps. The dynamic fusion enhancement module 423 is used to perform dynamic weight allocation fusion on short-exposure images and long-exposure images based on the fusion feature map and the normalized attention map to obtain an enhanced image. The stain detection and masking module 424 is used to detect stain areas and repair stains in the enhanced image, and output a stain-removed image. The trip information recognition module 425 is used to identify the trip code and indicator light status from the stain removal image and output the information recognition result.

[0065] The hardware layer 410 constructs a complete data acquisition, processing, storage, and transmission system. The functions of each module are as follows: The image acquisition module 411 is configured to simultaneously acquire one short-exposure image and one long-exposure image of the same scene from the substation protection device. The image acquisition module 411 includes a supplementary lighting unit to provide illumination for the scene. The image acquisition module 411 uses a 20-megapixel industrial camera, equipped with an 8-50mm adjustable focal length lens and a 5500K ring light. It supports multiple exposure parameter adjustments, a shutter speed range of 1 / 1000s-1 / 30s, an ISO range of 200-1600, and can simultaneously acquire short-exposure and long-exposure RGB format images with a resolution of 2048×1536.

[0066] The processing module 412 is equipped with an NVIDIA Jetson AGX Orin edge computing unit (200 TOPS computing power) and a built-in neural network inference engine, supporting real-time operation of multi-scale attention algorithms, image fusion, and stain removal programs, with a single image processing time of ≤200ms. The multi-exposure acquisition control module 421 integrates a target detection algorithm. It is configured to generate an acquisition trigger signal based on the processing results of the target detection algorithm and send this signal to the image acquisition module 411 to control its simultaneous acquisition of short-exposure and long-exposure images; and to receive the short-exposure and long-exposure images returned by the image acquisition module 411. The preprocessing program in the multi-exposure acquisition control module 421 integrates the YOLOv8 target detection algorithm, which achieves a detection accuracy of up to 99.2% for protection devices in substation scenarios. By identifying the protection device area in real time, the image acquisition program is triggered when the image acquisition device on the unmanned vehicle moves to an ideal shooting range of 1.5-2m from the device. This distance ensures that the device's display screen details are captured clearly enough, while avoiding image distortion and increased reflection caused by shooting too close.

[0067] Storage module 413 is equipped with a 1TB SSD solid-state drive for storing original images, intermediate processing results, and a stain mask library (the stain mask library can assist the stain detection and masking module 424 in identifying typical stains in the substation), ensuring data integrity and traceability, and supporting historical data backtracking and algorithm iteration optimization.

[0068] The communication module 414 integrates 5G and Ethernet interfaces, complies with substation communication standards, and can push structured trip information to the operation and maintenance platform in real time.

[0069] The global attention network, with 16×16 convolutional kernels, is configured to capture the overall layout features of the display cabinet and output a global attention map; the local attention network, with 8×8 convolutional kernels, is configured to focus on the semantic features of characters and output a local attention map; and the ultra-local attention network, with 4×4 convolutional kernels, is configured to capture the gradient features of character edges and output an ultra-local attention map.

[0070] In the modular architecture of this system, the hardware layer 410 includes an image acquisition module 411, a processing module 412, a storage module 413, and a communication module 414, which interacts with the software layer 420 via USB 3.0, PCIe, and Ethernet interfaces. Specifically, the image acquisition module 411 transmits multi-exposure image data to the multi-exposure acquisition control module 421 via USB 3.0; the processing module 412 interacts with the software layer 420 via PCIe and provides an operating environment for the dynamic fusion enhancement module 423; and the storage module 413 interacts with the software layer 420 via an Ethernet interface. The five core modules of the software layer 420 operate in the processing module 412 in the order of acquisition → feature extraction → fusion → repair → recognition, and finally output the recognition results through the communication module 414.

[0071] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0072] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A method for identifying information of a substation protection device, characterized in that, include: Acquire short-exposure and long-exposure images of the substation protection device at the same time, wherein the exposure parameters of the short-exposure and long-exposure images are different; Based on the short-exposure image and the long-exposure image, a basic feature map is obtained. According to the global attention network, local attention network and hyperlocal attention network, the basic feature map is subjected to hierarchical feature extraction to generate a normalized attention map. A fused feature map is generated based on the short-exposure image, the long-exposure image and the normalized attention map. Based on the fused feature map and the normalized attention map, the short exposure image and the long exposure image are dynamically weighted and fused to obtain an enhanced image. The enhanced image is subjected to stain area detection and stain repair processing to output a stain-removed image; Identify trip codes and indicator light status from the stain-removed image and output the identification results; perform hierarchical feature extraction on the basic feature map based on global attention network, local attention network, and hyperlocal attention network to generate a normalized attention map, and generate a fused feature map based on the short exposure image, the long exposure image, and the normalized attention map, including: Based on the basic feature map, corresponding global attention map, local attention map and hyperlocal attention map are generated via the global attention network, the local attention network and the hyperlocal attention network; The global attention map, the local attention map, and the hyperlocal attention map are normalized to obtain a normalized attention map, which includes a normalized global attention map, a normalized local attention map, and a normalized hyperlocal attention map. The global attention network, with 16×16 convolutional kernels, is configured to capture the overall layout features of the display cabinet and output the global attention map. The local attention network, with 8×8 convolutional kernels, is configured to focus on character semantic features and output the local attention map. The hyperlocal attention network, with 4×4 convolutional kernels, is configured to capture character edge gradient features and output the hyperlocal attention map. The process of detecting and repairing stains in the enhanced image to output a stain-free image includes: The enhanced image is matched and fused pixel by pixel. The pixel values ​​of the fused feature map are determined to be less than a preset threshold for initial screening to obtain stain candidate areas. By calculating the difference between the pixel grayscale and the mean grayscale value within the candidate stain area, the suspected stain area is determined; Based on whether the suspected stain area overlaps with the character core area marked by the normalized local attention map or the edge area marked by the normalized hyperlocal attention map, the non-overlapping part is saved and the real stain area is obtained. The non-edge stain areas in the real stain area are repaired using neighborhood mean filtering, while the stain areas near the edges in the real stain area are repaired using bilateral filtering, and the stain-removed image is output.

2. The method as described in claim 1, characterized in that, The step of identifying the trip code and indicator light status from the stain removal image and outputting the identification result includes: The connected regions of the normalized local attention map are extracted from the stain removal image to obtain character candidate regions. After local adaptive binarization and median filtering preprocessing, the character candidate regions are used to identify trip codes and equipment number characters using a CRNN model. The closed connected regions of the normalized hyperlocal attention map are extracted from the stain removal image and filtered by region area to obtain the real indicator light area. After calculating the color and brightness features of the real indicator light area, the indicator light status is classified using an SVM model. Information recognition results are constructed based on the trip code and the indicator light status.

3. The method as described in claim 2, characterized in that, The information identification result constructed based on the trip code and the indicator light status includes: Cross-validation was performed on the trip code and the indicator light status to obtain the verification results; If the verification result is successful, then an information recognition result is constructed based on the successful verification result.

4. The method as described in claim 1, characterized in that, The process of obtaining the basic feature map based on the short-exposure image and the long-exposure image includes: The basic features of the short-exposure image and the long-exposure image are extracted by the first three layers of the ResNet-18 network and then fused across exposures to obtain a basic feature map.

5. An information identification system for substation protection devices, employing the information identification method for substation protection devices as described in claims 1 to 4, characterized in that, The system includes a hardware layer and a software layer; The hardware layer includes the following modules connected in sequence: The image acquisition module is used to simultaneously acquire short-exposure and long-exposure images of substation protection devices; The processing module is used to run the software layer and obtain the information recognition result; A storage module is used to store the information recognition results; The communication module is used to interact with an external operation and maintenance platform and output the information recognition results; The software layer includes the following modules connected in sequence: The multi-exposure acquisition control module is used to generate an acquisition trigger signal to control the image acquisition module to acquire images, and to acquire short-exposure images and long-exposure images of the substation protection device at the same time, wherein the exposure parameters of the short-exposure images and the long-exposure images are different; The multi-scale attention feature extraction module is used to obtain a basic feature map based on the short exposure image and the long exposure image, perform hierarchical feature extraction on the basic feature map according to the global attention network, local attention network and hyperlocal attention network, generate a normalized attention map, and generate a fused feature map based on the short exposure image, the long exposure image and the normalized attention map. The dynamic fusion enhancement module is used to perform dynamic weight allocation fusion on the short exposure image and the long exposure image based on the fusion feature map and the normalized attention map to obtain an enhanced image. The stain detection and masking module is used to detect stain areas and repair stains in the enhanced image, and output a stain-removed image. The trip information recognition module is used to identify the trip code and indicator light status from the stain removal image and output the information recognition result.

6. The system as described in claim 5, characterized in that, The image acquisition module is configured to simultaneously acquire a short-exposure image and a long-exposure image of the same scene of the substation protection device; the image acquisition module includes a supplementary lighting unit for providing illumination to the scene.

7. The system as described in claim 5, characterized in that, The multi-exposure acquisition control module integrates a target detection algorithm, and the multi-exposure acquisition control module is configured to generate an acquisition trigger signal based on the processing result of the target detection algorithm; The acquisition trigger signal is sent to the image acquisition module to control it to simultaneously acquire short-exposure and long-exposure images; and the short-exposure and long-exposure images returned by the image acquisition module are received.

Images