A deep learning-based power inspection image analysis algorithm

Abstract

The application discloses a kind of power inspection image analysis algorithm based on deep learning, belong to computer vision and artificial intelligence technical field.The method includes: constructing hierarchical annotation dataset;Design and train global-to-local double-flow feature fusion network, which dynamically weights and fuses global context and local detail features through adaptive feature fusion module;Multi-task collaborative learning is carried out, and the model is optimized in combination with self-supervised pre-training;Based on power equipment knowledge graph, the logical reasoning and verification of visual recognition result are carried out;A dynamic updating closed loop is established by fusing active learning and elastic weight consolidation technology, so that the system continuously evolves.The system should include data management, model training, image analysis, knowledge reasoning and active learning modules.The application combines the perception, reasoning and evolution ability in depth, significantly improves the accuracy, robustness and explainability of power equipment defect identification, and has the ability of continuous self-optimization from practical application data.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a deep learning-based image analysis algorithm for power line inspection. Background Technology

[0002] The safe and stable operation of power grids depends on regular inspections of facilities such as transmission lines, towers, and substations. Traditional inspection methods mainly rely on manual climbing or telescope observation, which suffers from low efficiency, high risk, limited coverage, and strong subjectivity in the results. With the widespread use of drones, robots, and fixed cameras, image-based automated inspection has become the mainstream trend.

[0003] In existing technologies, automatic analysis methods based on traditional image processing (such as edge detection, feature matching, and threshold segmentation) are greatly affected by factors such as changes in lighting, weather conditions, complex backgrounds, and diverse target scales. They have poor robustness, high false alarm rates and false negative rates, and are difficult to meet actual operation and maintenance needs.

[0004] In recent years, deep learning, especially convolutional neural networks (CNNs), has achieved significant success in object detection and image segmentation tasks. Some attempts have been made to apply general object detection models (such as Faster R-CNN and the YOLO series) or semantic segmentation models (such as U-Net and DeepLab) to power equipment defect identification. However, these direct applications have inherent drawbacks: 1. Insufficient model generalization ability: General models are not optimized for the small targets, long-tail distribution (rare defect samples are extremely scarce), and multi-scale characteristics of power equipment components; 2. Lack of contextual understanding and logical reasoning: Models only perform pixel-level or region-level classification and cannot understand the structural relationships between equipment components, potentially leading to misjudgments that violate physical common sense (e.g., reporting "pin detachment" when no insulator is detected); 3. Poor interpretability: The model output is a "black box," making it difficult for maintenance personnel to understand the basis for judgment, affecting their trust in the results and subsequent decisions; 4. Static models cannot adapt to changes: Deployed models cannot self-optimize using newly generated inspection data and are difficult to adapt to new defect types or equipment differences in different regions.

[0005] Therefore, there is an urgent need for a power line inspection image analysis solution that can deeply integrate visual perception and domain knowledge, and possess high precision, strong robustness, good interpretability, and continuous evolution capabilities. Summary of the Invention

[0006] This invention aims to overcome the shortcomings of the existing technologies and provide a deep learning-based image analysis algorithm for power line inspection. Through innovative network architecture, training paradigm, and post-processing logic, this invention constructs a complete technical solution integrating perception, analysis, reasoning, and evolution, achieving intelligent, accurate, and interpretable identification of power equipment and its defects.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a deep learning-based image analysis method for power line inspection, comprising the following steps: S1: Construct a hierarchical power equipment annotation dataset. Power inspection images are annotated in four layers: scene layer (e.g., images belonging to "transmission towers" or "substation outdoor fields"), equipment layer (e.g., "insulator strings" or "circuit breakers"), component layer (e.g., "single insulator pieces," "pins," or "equalizing rings"), and defect layer (e.g., pixel-level masks for "damage," "corrosion," or "missing"). This provides fine-grained supervision signals for the model.

[0008] S2: Design and train a "global-to-local" two-stream feature fusion network. This network includes: S2.1: Globally perceptual flow, employing a visual Transformer (e.g., SwinTransformer) based on a window self-attention mechanism as its backbone, is used to capture global contextual information and long-range dependencies of the image, and outputs feature maps. .

[0009] S2.2: Local Refinement Flow. This approach uses a network that preserves high-resolution feature map output (e.g., HRNet) as the backbone to extract high-precision local detail features from small parts and subtle defects of the device, outputting feature maps. .

[0010] S2.3: Adaptive feature fusion module, receiving feature maps from the global flow and local flow. and Through a location-guided gating mechanism, a spatial weight map is dynamically generated, and the two feature streams are adaptively weighted and fused. Specifically, this module will... and The layers are concatenated along the channel dimension and then fed into a lightweight quantum network consisting of two 3x3 convolutional layers and a sigmoid activation layer. This sub-network outputs a single-channel spatially adaptive weight map. , where each pixel value Final fusion features Calculate using the following formula: ,in This represents element-wise multiplication. Weighted graph The distribution of the network's dependencies on global and local information was visualized.

[0011] S3: Perform multi-task collaborative learning and optimization. Based on fusion features. The system executes two main supervised tasks in parallel: part-level target localization and classification based on the detection head (outputting bounding boxes and categories), and pixel-level defect segmentation based on the segmentation head (outputting defect category masks). In the early stages of training, a self-supervised auxiliary task is introduced for pre-training to learn general features. The auxiliary task specifically includes: S3.1: Masked Image Restoration Task: Randomly mask 40%–60% of a rectangular region in the input image. The network's goal is to reconstruct the pixel values ​​of the masked region based on the remaining portion, using the L1 loss function for optimization.

[0012] S3.2: Component Relationship Prediction Task: In data with component annotations, randomly select two component bounding boxes under the same equipment instance and calculate the normalized relative coordinate offset of their center points. Logarithmic ratio of width and height This forms a four-dimensional vector as the prediction target. The network performs regression prediction through fully connected layers based on the encoded features, using a smooth L1 loss function.

[0013] S4: Post-processing and logical reasoning based on the power equipment knowledge graph. A knowledge graph is constructed that includes equipment-component inclusion relationships, inter-component connection relationships, and normal state geometric and texture constraints. The knowledge graph is stored in the form of an attribute graph, where nodes represent equipment or component categories, edges represent relationships, and nodes are associated with attribute rules. For example, the "insulator string" node has the rule: its "HasPart" relationship points to the number of "single insulator" nodes. (N is a preset threshold, such as 5); the center points of these "single insulator" nodes should satisfy collinearity constraints (e.g., fitting a straight line using the RANSAC algorithm, and the distance from all center points to the straight line is less than the threshold D). During inference, the detection boxes and segmentation masks output by S3 are instantiated and associated with nodes in the knowledge graph, triggering relevant rules for verification one by one. If a "pin" is detected but its parent component "insulator string" is not detected, the detection result of the "pin" is marked as "suspicious"; if the number of components of the detected "insulator string" is less than N, a "component missing" alarm is generated. This process corrects the logical errors of the pure visual model and generates advanced semantic alarms.

[0014] S5: Establish a dynamic model update and active learning closed loop. After system deployment, collect data when the model prediction confidence is below a threshold. Samples with a value of 0.7 (e.g., 0.7) and manually reviewed and corrected samples are stored in a pool awaiting review. Samples with the highest information content are periodically selected from the pool using active learning strategies (e.g., marginal sampling based on uncertainty and core set sampling based on feature embedding diversity) and then manually labeled. Incremental learning is performed using elastic weight consolidation techniques: first, the network parameters are calculated on a subset of the dataset representing the old task. diagonal elements of the Fisher information matrix Then, use the newly labeled sample set. During training, the total loss function is defined as: ,in It is the cross-entropy loss of the new task. These are the parameters of the old model. It is a balancing hyperparameter (e.g., set to 0.8). This mechanism effectively prevents catastrophic forgetting of old knowledge while adapting to new knowledge.

[0015] To implement the above algorithm, the present invention also provides a power line inspection image analysis system, comprising: The data management module is used to store and manage the hierarchical labeled dataset and the unlabeled original images; The model training module is used to perform the construction, multi-task collaborative learning and optimization of the dual-stream network; The image analysis engine loads the trained model and performs the inference process S2 to S3 on the input inspection image; The knowledge reasoning module, which embeds the power equipment knowledge graph and reasoning rules, is used to perform post-processing and logical reasoning in S4. The active learning and management module is used to execute the sample collection, screening, annotation management and incremental learning process of S5; The human-computer interaction interface is used to display analysis results, alarm information, model confidence, and weight graphs. It also includes knowledge graph reasoning paths and receives human feedback.

[0016] The present invention has the following beneficial effects: 1. This invention takes into account both context and details through a global-to-local dual-stream design, and the adaptive fusion mechanism effectively improves the model's recognition accuracy and robustness for small targets, complex backgrounds and multi-scale targets.

[0017] 2. By introducing structured knowledge graphs and programmable reasoning rules, this invention combines data-driven visual perception with symbolic domain knowledge, which can identify and correct logical fallacies that may occur in pure visual models, significantly reduce the false alarm rate, and output diagnostic conclusions with semantic information.

[0018] 3. Spatial weight map generated by the adaptive fusion module of this invention. It visualizes the areas of interest of the model, provides a clear logical reasoning chain for knowledge graphs, makes the "black box" decision-making process partially transparent, helps to build user trust and assists in fault location.

[0019] 4. Through an active learning closed-loop and anti-forgetting incremental learning strategy, the system can optimize itself using new data constantly generated in the production environment, adapt to new scenarios, new equipment and new defects, extend the effective life cycle of the product and reduce long-term maintenance costs.

[0020] 5. This invention constitutes a closed-loop, implementable, and complete technical solution from data engineering, model design, training strategy, post-processing to system iteration, and has strong industrial application value and broad patent protection. Attached Figure Description

[0021] Figure 1 Here is a flowchart of the power line inspection image analysis algorithm; Figure 2 This is the flowchart for step S3. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0023] Example 1: Defect Detection and Analysis of Transmission Line Insulators This embodiment uses the inspection of power transmission line insulator strings by drone as an example to illustrate the implementation process of the present invention in detail.

[0024] Data Preparation: Collect 100,000 images of UAV inspections containing insulator strings. Following the four-layer system described in this invention, fine-grained annotation was performed using annotation tools. For example, an image might be annotated as follows: Scene Layer = "Transmission Tower", Equipment Layer = "Composite Insulator String", Component Layer = "Insulator Skirts 1 to 24" (24 bounding boxes in total), "High-Voltage Fittings", "Low-Voltage Fittings", Defect Layer = "A 'perforation' defect of approximately 50x30 pixels exists in the middle of the 5th skirt" (providing a pixel-level mask). The data was divided into training, validation, and test sets in an 8:1:1 ratio.

[0025] Model training: The network is constructed following steps S2 and S3. The global awareness stream uses Swin-Tiny, with the input resolution adjusted to 1024x1024. The local refinement stream uses HRNet-W32. The sub-network of the adaptive fusion module is Conv2d. Input Channel = , Output channel = 128, kernel_size = 3)->ReLU->Conv2d(128, 64, 3)->ReLU->Conv2d(64, 1, 3)->Sigmoid.

[0026] The training is divided into two phases: Pre-training phase: Self-supervised learning for masked image restoration (masking rate 50%) and component relationship prediction tasks is performed on 500,000 unlabeled power line inspection images. The learning rate is 1e-4, and a total of 100 epochs are trained.

[0027] Fine-tuning phase: Load pre-trained weights and jointly train the main detection and segmentation tasks on labeled data. The detection head uses a RetinaNet architecture, and the segmentation head uses an FPN architecture. The loss function is: ,in For FocalLoss, For DiceLoss. The AdamW optimizer was used with a learning rate of 3e-5, a batch size of 8, and a total training duration of 50 epochs.

[0028] Image analysis: An image of the transmission line to be inspected is input into the trained model. The model output includes: bounding boxes of 24 insulator skirts and their confidence scores (all > 0.95), a segmentation mask for the "perforation" defect of the 5th skirt (confidence score 0.88), and a spatial weight map. .from As can be observed in the figure, in the insulator string region, the network relies more on local detail features. The value is close to 0, while in areas with complex backgrounds, it relies more on the global context. (Value close to 1).

[0029] Knowledge Reasoning: The knowledge reasoning module loads the rule base related to "insulator strings". The rule base includes: Rule 1 (Quantity Rule): IF Device Type == "Insulator String" THENCOUNT Associated component type == "Monolithic insulator" >=5.

[0030] Rule 2 (Geometric Rule): IF Device Type == "Insulator String" THEN All associated "Mono-Insulator" center point fitted linear residual < 20 pixels.

[0031] Rule 3 (Existence Rule): If a component type == "pin" is detected, then its parent device type == "insulator string" must exist.

[0032] The inference engine performs the following operations: The 24 detected "single insulator" components were associated with the same "insulator string" device instance.

[0033] Both Rule1 and Rule2 were validated and passed.

[0034] Combining the visually detected "perforation" defect (confidence level 0.88) with the conclusion in the knowledge base that "perforation of a single insulator skirt is a serious defect", the final alarm is generated: "Serious defect: In the detection target #001 (insulator string), component #05 (single insulator) has a 'perforation' defect with a confidence level of 0.88. It is recommended to arrange replacement immediately." Results Presentation and Feedback: On the human-computer interaction interface, the 5th insulator is highlighted in red, and the defect segmentation area is overlaid. The alarm panel displays the above semantic alarm. After inspection, the maintenance personnel click the "Confirm" button, and the result is considered a positive sample.

[0035] Model Update: One week later, the active learning module selects 100 of the most informative samples from the pool of samples awaiting review (including the low-confidence defect edge samples in this example). After manual review and annotation, an incremental dataset is formed. Updates are performed using elastic weight consolidation: First, 10,000 images are randomly selected from the original training set to calculate parameter importance. Then During training, the total loss function is set as described in S5. The learning rate was reduced to 1e-5, and fine-tuned for 5 epochs. The updated model improved segmentation accuracy for similar edge defects by approximately 15%.

[0036] Example 2: Comprehensive Assessment of Substation Equipment Status This embodiment is applied to the scenario of fixed camera inspection in substations to illustrate the collaborative analysis capabilities of multiple devices.

[0037] The system extracts and analyzes frames from the real-time video stream every 10 seconds. The trained model can simultaneously identify various devices and their components (such as circuit breakers, voltage transformers (PTs), current transformers (CTs), etc.)

[0038] The knowledge graph defines more complex internal rules. For example: Rule 4 (State Consistency Rule): IF Device Type == "Circuit Breaker" AND Phase IN ["A", "B", "C"] THEN All phases' "Open / Close Indicator" states must be the same.

[0039] Rule 5 (Threshold Rule): IF Component Type == "Oil Level Gauge Pointer" THEN The percentage of oil level corresponding to its pointing angle must be in the range of [20%, 95%].

[0040] Rule 6 (Text Auxiliary Rule): IF if the “Device Nameplate” area THEN triggers the OCR submodule to recognize the text and compare it with the expected device number.

[0041] In one analysis, the image analysis engine output: Phase A of the circuit breaker is "open", while Phases B and C are "closed"; the oil level gauge pointer of PT01 points to an angle corresponding to an oil level of 15%.

[0042] The knowledge reasoning module performs reasoning: Rule4 was triggered, and an inconsistency in the three-phase status was detected, generating the following error: "Abnormality: The three-phase opening and closing status of the circuit breaker is inconsistent (phase A is open, phases B and C are closed). There may be a mechanical fault or identification error. On-site verification is recommended."

[0043] Rule 5 is triggered. If the oil level is found to be 15% lower than the lower threshold of 20%, an "Alert: PT01 oil level is low (15%). It is recommended to check for oil leaks and arrange for oil replenishment" is generated.

[0044] By integrating visual recognition results with knowledge rules, the system not only reports visible low oil levels but also infers potential state inconsistencies, achieving an upgrade from single defect detection to comprehensive assessment of system health status.

[0045] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based image analysis algorithm for power line inspection, characterized in that, Includes the following steps: S1: Construct a hierarchical annotation dataset to perform four-layer structured annotation on power inspection images: scene layer, equipment layer, component layer, and defect layer. S2: Design and train a global and local dual-stream feature fusion network, which includes a global awareness stream for capturing global context, a local refinement stream for extracting local details, and an adaptive feature fusion module for adaptively weighting and fusing the two streams of features. S3: Perform multi-task collaborative learning and optimization. Based on the fusion features, execute the main supervised tasks of component-level target detection and pixel-level defect segmentation in parallel, and introduce a self-supervised auxiliary task for pre-training in the early stage of training. S4: Based on the pre-built knowledge graph of power equipment, perform logical reasoning and rule verification on the detection and segmentation results output in step S3 to correct errors and generate semantic alarms; S5: Establish a dynamic model update and active learning closed loop. By collecting low-confidence samples and human feedback, active learning strategies are used to select samples for incremental learning, thereby achieving continuous evolution of model performance.

2. The deep learning-based power line inspection image analysis algorithm according to claim 1, characterized in that, In step S2, the specific operation of the adaptive feature fusion module includes: fusing the feature maps of the global perception stream... Feature map of local refinement flow The features are concatenated along the channel dimension; the concatenated features are then input into a subnetwork consisting of at least two convolutional layers and one sigmoid activation layer to generate a single-channel spatial adaptive weight map. According to the formula Calculation of fusion features ,in This indicates element-wise multiplication.

3. The deep learning-based power line inspection image analysis algorithm according to claim 1, characterized in that, In step S3, the self-supervised auxiliary task includes: S3.1: Masked image restoration task, randomly masking a portion of the input image, training the network to reconstruct the masked pixels; S3.2: Component Relationship Prediction Task. Based on component annotation data, train a network to predict the geometric relationship between any two component bounding boxes under the same device.

4. The deep learning-based power line inspection image analysis algorithm according to claim 1, characterized in that, In step S4, the power equipment knowledge graph is stored in the form of an attribute graph, where nodes represent equipment or component categories and are associated with attribute rules, and edges represent relationships between components; the logical reasoning includes: instantiating the visual detection results and associating them with knowledge graph nodes, and triggering the verification of associated attribute rules; the attribute rules include at least component quantity constraint rules, geometric relationship constraint rules between components, and component existence dependency rules.

5. The deep learning-based power line inspection image analysis algorithm according to claim 1, characterized in that, In step S5, the incremental learning employs an elastic weight consolidation technique, and its loss function is: ； in, For the losses of the new mission, For old model parameters, Parameters calculated on old task data The diagonal elements of the Fisher information matrix Hyperparameters to balance the importance of new and old tasks.

6. A power inspection image analysis system for implementing the deep learning-based power inspection image analysis algorithm as described in any one of claims 1-5, characterized in that, include: The data management module is used to store and manage hierarchical labeled datasets and original images; The model training module is used to perform the construction of two-stream networks, multi-task collaborative learning, and optimization. An image analysis engine is used to load the trained model and perform inferences on the input inspection images; The knowledge reasoning module, which embeds the power equipment knowledge graph and reasoning rules, is used to perform post-processing and logical reasoning. The Active Learning and Management module is used to execute sample collection, screening, annotation management, and incremental learning processes.

7. The system according to claim 6, characterized in that, The image analysis engine also includes a visualization unit for generating and displaying the spatial adaptive weight map. This allows us to visualize the network's dependence on global and local information.

8. The system according to claim 6, characterized in that, The human-computer interaction interface is used to display analysis results, semantic alerts, model confidence and reasoning path of knowledge graph, and provides a feedback interface for manual confirmation and correction of results.

Images