Electric power patrol full-scene anomaly detection method and related device

Abstract

The invention belongs to an electric power anomaly detection method, and provides an electric power patrol full-scene anomaly detection method and a related device for solving the technical problem that an existing electric power system anomaly detection method is low in recognition effect and recognition efficiency. The unified anomaly detection model adopts a ScaledYOLOv4 structure, corresponding detection heads are respectively connected behind a backbone network of the ScaledYOLOv4 structure according to different scenes, and when training is carried out, the detection heads are respectively connected with the backbone network of the ScaledYOLOv4 structure, and the detection heads are respectively connected with the backbone network of the ScaledYOLOv4 structure; and taking an intermediate detection result obtained by a single anomaly detection model as a calculation reference of a loss function in the training process of the unified anomaly detection model. According to the method, resource occupation is reduced, and the reasoning speed is increased. In addition, due to the fact that corresponding design is carried out for different scenes, the detection and recognition effect is improved.

Description

Technical Field

[0001] This application pertains to a power anomaly detection method, specifically involving a full-scenario anomaly detection method and related devices for power inspection. Background Technology

[0002] Equipment defects, personnel violations, and environmental hazards in the power transmission, distribution, and transformation processes can easily lead to safety accidents, necessitating the use of image acquisition and intelligent recognition technologies for anomaly detection.

[0003] Current technologies typically involve acquiring images via helicopters, drones, and cameras, then analyzing and recognizing them using models such as convolutional neural networks and Transformers, followed by manual review. However, this method clearly has the following shortcomings: (1) The labels of multi-region datasets vary greatly, and fusion training is prone to false positives and false negatives, while training separately wastes samples, resulting in weak generalization ability of the model; (2) Different models need to be trained separately for different scenarios, and full-scene detection requires multiple calls, which consumes a lot of resources and is slow inference; (3) Existing fusion methods are only suitable for a single power transmission scenario and cannot meet the needs of fusion in multiple scenarios; (4) Different scenarios have different optimal resolutions, and a uniform strategy is prone to false alarms; (5) The multi-head detector model lacks optimization for repeated categories and overlapping output boxes, resulting in large deviations in the results.

[0004] These problems collectively hinder the improvement of recognition effectiveness and efficiency, and urgently require optimized solutions. Summary of the Invention

[0005] This application addresses the technical problems of low identification effect and efficiency of existing power system anomaly detection methods by providing a power inspection full-scene anomaly detection method and related device.

[0006] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application proposes a method for anomaly detection across all scenarios during power line inspections, including: Acquire images of power grid inspection scenes; The power inspection scene image is input into a unified anomaly detection model to obtain the category and location information of the anomaly. The unified anomaly detection model adopts a ScaledYOLOv4 structure, and for different scenarios, corresponding detection heads are connected after the backbone network of the ScaledYOLOv4 structure. The method for calculating the loss function during the training of the unified anomaly detection model includes: Different scenario datasets with labeled defects are input into separate anomaly detection models to obtain intermediate detection results; The intermediate detection results are used as the benchmark for calculating the loss function during the training of the unified anomaly detection model.

[0007] Furthermore, the different scenario datasets of labeled defects input into the unique anomaly detection model also include the following processing methods: Images from different scenario datasets with labeled defects are scaled to different resolutions, and then sliced. The location and category information of the defect target boxes in the original images are mapped to the sliced ​​images.

[0008] Furthermore, the individual anomaly detection model adopts the ScaledYOLOv4 structure.

[0009] Furthermore, the backbone network correction method in the individual anomaly detection model includes: Replace the batch normalization layer with a network that does not contain a batch normalization layer; And / or, add a fast two-dimensional attention operator for the image; And / or, add a 2D multi-directional rotational position encoding mechanism.

[0010] Furthermore, the detection head correction method in the individual anomaly detection model includes: Based on the corresponding labeled defect datasets for different scenarios, we adopt the following methods: target existence probability correlation prediction, unified probability and location information prediction for fixed reference points of all image locations at multiple resolution scales, and output dimensions corresponding to different scenario datasets.

[0011] Furthermore, the loss function of the unified anomaly detection model includes:

[0012] Wherein, the total loss function is the loss function used by the unified anomaly detection model, α is the weighting coefficient of the model accuracy loss, and β is the weighting coefficient of the model distillation loss. The model distillation loss is calculated based on the intermediate detection results.

[0013] Furthermore, the process of obtaining the category and location information of the anomaly also includes: After the unified anomaly detection model outputs preliminary detection results, it filters and masks the preliminary detection results that are not below the preset resolution; it merges and deduplicates preliminary detection results with similar characteristics, and retains preliminary detection results with different characteristics according to different scenarios.

[0014] Secondly, this application proposes a full-scene anomaly detection system for power line inspection, including: The data module is used to acquire images of power inspection scenes; The detection module is used to input the power inspection scene image into the unified anomaly detection model to obtain the category information and location information of the anomaly. The unified anomaly detection model adopts the ScaledYOLOv4 structure, and for different scenarios, corresponding detection heads are connected after the backbone network of the ScaledYOLOv4 structure. The method for calculating the loss function during the training of the unified anomaly detection model includes: Different scenario datasets with labeled defects are input into separate anomaly detection models to obtain intermediate detection results; The intermediate detection results are used as the benchmark for calculating the loss function during the training of the unified anomaly detection model.

[0015] Thirdly, this application proposes an electronic device, including: a memory and one or more processors; the memory is coupled to the processors; wherein, the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the above-described power patrol full-scene anomaly detection method.

[0016] Fourthly, this application proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described power inspection full-scene anomaly detection method.

[0017] Compared with the prior art, this application has the following beneficial effects: This application proposes a method for anomaly detection across all scenarios during power line inspections. The method inputs images of the power line inspection scene into a unified anomaly detection model to obtain the category and location information of anomalies. The unified anomaly detection model employs a ScaledYOLOv4 architecture. For different scenarios, corresponding detection heads are connected after the backbone network of the ScaledYOLOv4 architecture. During training, intermediate detection results obtained from individual anomaly detection models are used as the benchmark for calculating the loss function of the unified anomaly detection model. This application integrates different inspection scenarios. The structure of the unified anomaly detection model includes a shared backbone and detection heads tailored to different scenarios. This reduces resource consumption and improves inference speed in the application of the unified anomaly detection model. Furthermore, the corresponding design for different scenarios also improves the detection and recognition performance.

[0018] This application also proposes a full-scene anomaly detection system for power inspection, an electronic device, and a computer-readable storage medium, which possesses all the advantages of the aforementioned full-scene anomaly detection methods for power inspection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating one aspect of the power patrol anomaly detection method for the entire scenario in this application. Figure 2 This is a schematic diagram illustrating the principle of one embodiment of the power patrol full-scene anomaly detection method of this application; Figure 3 This is a schematic diagram of the power patrol full-scene anomaly detection system of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0024] In the description of the embodiments of this application, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In addition, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0026] In the description of the embodiments of this application, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0027] In the field of power system operation, the stable operation of key links such as transmission, distribution, and substation is directly related to power grid safety. Equipment defects, personnel violations, and environmental hazards can all trigger power grid operation accidents, leading to serious consequences such as power outages, equipment damage, and even casualties. To avoid such situations, the industry commonly uses inspection methods to promptly detect and handle anomalies. Specifically, helicopters and drones are used to inspect and photograph transmission and distribution lines, while fixed cameras or robots are used to monitor the environment of the corridors and substations in real time. Then, intelligent image analysis models are used to analyze the collected images to identify potential defects and hazards. Finally, after manual review and confirmation of the problems, the issues are handed over to the maintenance department for anomaly elimination. Currently, image recognition technology is developing rapidly, with various model structures such as Convolutional Neural Networks (CNN), Transformer models, and Swin models constantly being updated. The recognition results have reached a certain level of practicality, providing technical support for image analysis in power system inspections.

[0028] Image acquisition and intelligent recognition solutions relied upon in power system inspections still face many unresolved issues in practical applications. Different provinces have varying equipment inspection needs in the same scenarios. Taking transmission equipment defect identification as an example, the appearance of equipment differs across provinces, and the key defect categories of concern also vary. This leads to differences in the defect types included in the labeled datasets of each province. Defect types not labeled in some provinces may be clearly labeled in the datasets of other provinces, directly affecting the accuracy of cross-regional model identification. Furthermore, for defects in transmission, distribution, and substation equipment, as well as personnel violations and environmental hazards, separate datasets need to be constructed and corresponding models trained. When performing full-scenario anomaly detection, multiple models need to be called sequentially and the results merged, which not only consumes significant runtime resources but also prolongs inference time and reduces inspection efficiency.

[0029] To address the aforementioned issues, some research has attempted to find solutions. Regarding model training, for defect identification in power transmission, distribution, and substation equipment, the proportion of defects in images varies greatly, ranging from almost half the image to just a few pixels. A multi-resolution strategy is typically employed, scaling the image to different resolutions for defect detection and then merging the results. For images of passageways and stations, where environmental hazards and personnel violations constitute a large proportion and the image resolution is low, a single-resolution anomaly detection model suffices. Furthermore, relevant patents have proposed multi-model training methods, such as the technical solution disclosed in Chinese invention patent application number 202211440558.2, which can fuse different datasets to train a single model, attempting to solve the problems of low sample utilization and a large number of models.

[0030] However, existing solutions still have significant shortcomings and fail to completely address the pain points in practical applications. Existing multi-dataset fusion training methods are only applicable to single power transmission scenarios and cannot achieve fusion training of datasets from multiple scenarios such as power equipment, personnel, and environment, making it difficult to adapt to the needs of full-scenario inspection and detection. The optimal resolution for model training varies in different scenarios. Power transmission equipment defects commonly use 4608 / 2048 / 1280 resolutions (the numbers represent the pixel values ​​scaled to the long side), power distribution equipment defects commonly use 4096 / 2048 / 1024 resolutions, and channel hazards commonly use 3072 / 1024 resolutions. Substation equipment defects and personnel violations usually do not require image scaling. When using a unified multi-resolution strategy, it is impossible to determine the corresponding scenario from the image. If a high-scale multi-resolution is used to adapt to high-resolution requirements, it will lead to a large number of false alarms in the identification of channel environment, personnel violations, and other scenarios. In the model structure of shared backbone network and multiple detector heads, each detector head corresponds to a dataset and outputs the anomaly types it contains. When different datasets have duplicate categories and overlapping output boxes, there is a lack of effective optimization mechanisms, resulting in a deviation between the model's merged output and the actual requirements. These combined issues mean that existing technologies still have significant room for improvement in terms of sample utilization, multi-scenario adaptation, resource consumption, and recognition accuracy, and cannot fully meet the actual needs of efficient and accurate power system inspections.

[0031] Based on the above, this application proposes a method and related device for anomaly detection in all scenarios of power inspection. The following is a detailed description of this application in conjunction with embodiments and accompanying drawings.

[0032] like Figure 1 The diagram shown is a flowchart of one aspect of the power patrol anomaly detection method of this application, which may include: S101, acquire images of the power inspection scene.

[0033] In practical applications, power inspection scene images include image data collected by equipment such as drones, fixed cameras, and robots during the inspection of power facilities such as transmission lines, power distribution equipment, and substations.

[0034] S102, the power inspection scene image is input into the unified anomaly detection model to obtain the category and location information of the anomaly. The unified anomaly detection model adopts a ScaledYOLOv4 structure, and for different scenarios, corresponding detection heads are connected after the backbone network of the ScaledYOLOv4 structure.

[0035] The unified anomaly detection model in this application, based on the multi-scale detection and efficient inference features of ScaledYOLOv4, reduces parameter redundancy and improves feature extraction efficiency through a shared backbone network. Simultaneously, it configures dedicated detection heads for different scenarios to ensure accurate capture of anomaly features in each scenario. After receiving an image, the unified anomaly detection model extracts common features through the backbone network, and then the corresponding detection head completes anomaly classification and location for that scenario, ultimately outputting structured detection results. This application, through the design of a shared backbone network and dedicated detection heads, achieves one-stop anomaly detection across all scenarios without switching between multiple models, thus improving detection efficiency. It also leverages the high efficiency of ScaledYOLOv4 to balance detection accuracy and running speed, adapting to the real-time detection needs of power line inspections. Furthermore, the dedicated detection heads ensure the specificity of anomaly identification for each scenario, reducing false alarms caused by cross-scenario interference.

[0036] A unified method for calculating the loss function during the training of anomaly detection models, including: (1) Input the different scenario datasets of labeled defects into separate anomaly detection models to obtain intermediate detection results; (2) The intermediate detection results are used as the benchmark for calculating the loss function during the training of the unified anomaly detection model.

[0037] The training method described in this application employs a model distillation mechanism, transferring the detection experience of individual anomaly detection models (as teacher models) to a unified anomaly detection model (as student models). Individual anomaly detection models are trained for single scenes, possessing high-precision detection capabilities within that scene. Their intermediate detection results serve as a reference benchmark, optimizing the loss calculation of the unified anomaly detection model and helping it quickly learn anomaly recognition patterns across various scenes. This application provides high-quality knowledge references through the intermediate detection results of individual anomaly detection models, reducing training bias caused by differences in data distribution across multiple scenes. It also reduces the unified anomaly detection model's dependence on large-scale labeled data, improving its training efficiency and final detection accuracy. Using intermediate detection results as a benchmark for loss calculation allows the unified anomaly detection model to leverage the advantages of individual anomaly detection models during training, improving its anomaly recognition accuracy across various scenes, especially for niche scenes or categories with small sample sizes. This balances the need for full-scene fusion with single-scene accuracy.

[0038] This application employs a unified ScaledYOLOv4 model with a shared backbone network and a dedicated detection head to achieve one-stop classification and localization of anomalies across all scenarios. During the model training phase, a separate anomaly detection model is innovatively introduced, using its intermediate detection results as the benchmark for calculating the loss function. Through joint loss function optimization, the unified anomaly detection model absorbs the experience gained from accurate single-scene identification. This application not only solves the problem of low detection efficiency in traditional multi-model switching but also compensates for the insufficient accuracy of the unified anomaly detection model in single scenarios through distillation-style training.

[0039] like Figure 2 The diagram shown is a schematic representation of an embodiment of the power inspection full-scene anomaly detection method of this application.

[0040] S201, Data Preprocessing.

[0041] (1) Data preparation.

[0042] Prepare pre-labeled datasets related to defects in power transmission equipment, power distribution equipment, substation equipment, personnel violations, and environmental hazards. If different regions have different datasets, prepare them separately. This will result in a total of n datasets, named DS1, DS2, ..., DS... n .

[0043] Let the number of anomaly types in each dataset be N1, N2, ..., N. n The annotation information includes the location and category information of various defects in each image. The location information includes: xc, yc, w, and h, where xc is the relative x-coordinate of the center of the target box, yc is the relative y-coordinate of the center of the target box, w is the relative width of the center of the target box, and h is the relative height of the center of the target box. The dataset is then divided into training and validation sets proportionally.

[0044] (2) Training set data slices.

[0045] Scaling the image to different resolutions means scaling the longer side of the image to C1 / C2 / ... / C... m Pixels. Then, image slicing is performed, and the position and category information of the target bounding boxes in the original image are mapped to the sliced ​​images. By mapping the position and category information to the sliced ​​images, it is ensured that the sliced ​​data retains complete defect annotation information. Different defect categories have corresponding reasonable resolutions. Only slices at reasonable resolutions are retained, and slices at unreasonable resolutions are discarded, which can improve the targeting and effectiveness of subsequent model training.

[0046] S202, for each dataset DS i Model training was performed separately.

[0047] (1) In this embodiment, the structure of the individual anomaly detection model is implemented based on ScaledYOLOv4, using the same multi-resolution scale C1 / C2 / ... / C m .

[0048] It's worth noting that ScaledYOLOv4 is a scaled-down version of the YOLOv4 object detection model. Through architectural scaling and parameter adjustments, it balances detection accuracy with model speed and hardware adaptability, making it suitable for object detection needs in scenarios with varying computing power. ScaledYOLOv4 eliminates the need for prior candidate region extraction, directly outputting the object's category and location information from the image, resulting in outstanding detection efficiency. Its core structure mainly includes: The backbone network enhances feature extraction capabilities and reduces computational redundancy through cross-stage local connectivity and residual learning. The neck network enables multi-scale feature fusion, improving adaptability to targets of different sizes; The detection head employs a multi-scale detection branch, enabling target prediction on feature maps of different resolutions, thus achieving good detection results for both small and large targets.

[0049] In power system image inspection scenarios, ScaledYOLOv4's multi-scale detection capabilities and flexible scaling features are particularly suitable. It can capture small defects in transmission lines through high-resolution input and deep networks, and can also adapt to the computing power limitations of inspection equipment through lightweight configuration. It is one of the mainstream target detection frameworks that balances detection accuracy and engineering practicality.

[0050] (2) Modification of the backbone network in the standalone anomaly detection model.

[0051] In this embodiment, the following modifications can be made: Replacing the batch normalization layers in the backbone network with other networks that do not include batch normalization, such as ViT, Swin, or LN (Layer-Norm) layers, can improve the stability of the standalone anomaly detection model. The effectiveness of batch normalization layers is easily affected by batch size, and their stability may be insufficient when training with mixed data from multiple power inspection scenarios or when sample sizes are imbalanced in some scenarios. Replacing them reduces batch dependency and improves the training and inference stability of the standalone anomaly detection model under complex data distributions.

[0052] A two-dimensional fast attention operator can also be added to improve model speed. Adding a two-dimensional fast attention operator to the backbone network optimizes attention computation, storage, and computation logic, reducing redundant operations and memory usage. This significantly improves computational efficiency during feature extraction, thereby accelerating the overall model speed and meeting the engineering requirements of real-time detection of power line inspection images.

[0053] A 2D multi-directional rotational position encoding mechanism can also be added to improve model performance. By enhancing the encoding of the spatial position information and rotation angle features of targets in the image, the model can more accurately capture the pose differences and spatial distribution features of targets such as power equipment defects and personnel violations. It is especially suitable for the recognition needs of complex targets such as small defects in transmission lines and tilted equipment components, thereby further improving the model's detection effect and recognition accuracy.

[0054] (3) Detection head modification. For each anchor (fixed reference point) at each location in the feature map at different resolutions, predict the classification probability of each class of object in the dataset, and obtain: 1-P obj (Background probability), P obj P1, P obj P2, ..., P obj P Ni The target bounding box coordinates are xc, yc, w, and h. The output dimension of the last layer of the model is K. (N i +5), where K is the number of anchors at each resolution.

[0055] (4) Merge the output results at each resolution and deduplicate the boxes of the same category.

[0056] S203, train the unified anomaly detection model M.

[0057] (1) The unified anomaly detection model M can adopt a shared backbone + multiple detection heads model structure, using the same multi-resolution scale C1 / C2 / ... / Cm as the teacher model, and the same backbone, followed by n detection heads H1, H2, ..., Hm. n Detection head H i For dataset DS respectively i The recognition category, with TM i They have the same structure.

[0058] It should be noted that the teacher model is the standalone anomaly detection model trained in step S202 for a single dataset, denoted as TM.i In this embodiment, all recognition tasks corresponding to the datasets share the same backbone network. The backbone network in the unified anomaly detection model is the same as the backbone network of the individual anomaly detection model trained in step S202. Then, n independent detection heads (H1 to H2) are connected after the shared backbone network. n Each detection head is specifically responsible for the recognition task of one dataset.

[0059] (2) Strategies for using training samples First, preset the weights PD1, PD2, ..., PD for each dataset. n Based on factors such as the number of categories and the number of samples in each dataset, the importance is determined, and corresponding weights W1, W2, ..., W3 are assigned. n .

[0060] In dataset DS i Preset weights PC for each category ij j=0,1,…,n i During initialization, the weights can be set to the average weights of each category, and then dynamically adjusted later.

[0061] When selecting each sample, first randomly select a dataset based on probability, then select a category within that dataset based on probability, and finally randomly select a slice of samples. Repeat this process until a batch (training batch) of images is selected for the next training iteration of the model.

[0062] After each epoch of training, the model calculates the recognition performance of each category on the validation set and adjusts the weights of each category based on their recognition accuracy. Generally, the weights of categories with high recognition accuracy are reduced, while the weights of categories with low recognition accuracy are increased.

[0063] (3) Modify the loss function.

[0064] In this embodiment, the loss function used is:

[0065] Here, model accuracy loss refers to the loss on the dataset DS. i The training samples are used to extract the feature dimensions corresponding to the last layer of the model output, and the model is then used to determine the classification probability 1-P for each target class predicted by the model. obj P obj P1, P obj P2, ..., P obj P NiThe target bounding box coordinates (xc, yc, w, h) are matched with the information of the sample labeled boxes to calculate the loss for the corresponding sample. For all samples used in this training round, the regression loss and classification loss of the target bounding boxes for different samples are weighted and averaged according to the number of foreground samples to obtain the model accuracy loss.

[0066] Model distillation loss refers to the loss of a model belonging to dataset DS. i The sample, using the teacher model TM of the dataset i The unified anomaly detection model M predicts the samples respectively, and the output at each resolution is obtained (TM). i Both M and M output feature dimensions of N at each feature point. i +5). Compare the outputs of the two models, setting different weights for outputs at different resolutions and dimensions, and calculating the MSE (mean-square error) loss for the coordinate dimensions (xc, yc, w, h) and the probability dimensions (P0, P1, ..., P...). Ni Calculate the KL (Kullback-Leibler) divergence loss and perform a weighted average, which is the distillation loss for this dataset. Calculate the weighted average of the distillation losses for all datasets to obtain the model distillation loss.

[0067] S204, Modification of the Unified Anomaly Detection Model Inference.

[0068] (1) Resolution masking strategy.

[0069] For each category in the dataset, a reasonable resolution range is selected. Results output at unreasonable resolutions are filtered out by setting the probability of that category to 0. For example, large targets are filtered out at high resolutions, while small targets are filtered out at low resolutions. Resolution control effectively reduces false positives in the model.

[0070] (2) Processing the same category in different datasets For cases where the same anomaly category exists across different datasets and the model's output bounding boxes overlap, both business characteristics and image characteristics can be analyzed to determine whether merging is necessary. If the business characteristics and image characteristics are similar, they will be treated as belonging to the same anomaly category for deduplication. If the business characteristics or image characteristics differ, they will be treated as belonging to multiple anomaly categories, with deduplication performed separately for each category. This strategy effectively improves the model's stability during application, reduces false positives due to overlapping boxes, avoids false negatives that deviate from expectations, and enhances the user experience.

[0071] This application integrates images from a full-scene inspection of power equipment, personnel, and the environment. Based on a model structure of shared backbone + multi-resolution + multi-detector head, a unified anomaly detection model for full-scene anomaly detection in power inspection is trained. In model application, it can reduce resource consumption and improve inference speed. Addressing the different optimal resolutions of different scenario datasets, a unified resolution backbone network and a resolution-based output box selection strategy are integrated, ensuring high accuracy in anomaly detection across various scenarios. For the same anomaly category in different datasets, this application performs independent or merged deduplication processing based on business characteristics and image characteristics, ensuring the model outputs meet user expectations and reducing false positives or false negatives due to overlapping boxes of the same category.

[0072] The following is an application example of the full-scenario anomaly detection model for power inspection in this application.

[0073] 1. Dataset preparation.

[0074] (1) Prepare the following power inspection dataset.

[0075] DS1: Defects in power transmission drone and helicopter inspection equipment, including 40 types of defects such as foundation settlement, vibration damper slippage, insulator damage, and missing bolt pins, totaling 80,000 sheets; DS2: Defects in power transmission channel camera inspection equipment, including 6 types of defects such as tower cranes, excavators, and missing signs, totaling 10,000 images; DS3: Defects in power distribution equipment, including 15 types of defects such as tower top damage, missing insulation covers for wire clamps, broken insulators, and missing pins for bolts, totaling 100,000 sheets; DS4: Defects in power equipment, including 16 types of defects such as oil stains on component surfaces, discoloration of silicone in breather, abnormal readings of pointer meters, and broken insulators, totaling 40,000 sheets; DS5: Violations by personnel, including 16 types of violations such as climbing over or under fences, leaving escalators unattended, and smoking, totaling 60,000 tickets; DS6: Environmental hazards in passageways and stations, including 7 categories of environmental hazards such as smoke, rodents, and operating equipment, totaling 20,000 sheets.

[0076] (2) Training set slices.

[0077] For each category in the dataset, a reasonable resolution range is pre-set. For boxes outside the reasonable range, such as those at 4608 resolution, slices of common large-scale anomaly categories like foundation settlement in DS1, tower cranes in DS2, and escalators in DS5 are discarded. The reasonable resolution range may differ for the same category in different datasets; therefore, separate settings are set for the 100 anomaly categories across the six datasets.

[0078] 2. Train separate anomaly detection models for each DSi dataset.

[0079] (1) The structure of the standalone anomaly detection model consists of a backbone, a detection head, and post-processing, and uses the same multi-resolution scale 4608 / 3072 / 2048 / 1024 (long side size after image scaling).

[0080] (2) Modify the backbone of the individual anomaly detection model, using the Swin model as the backbone. Add a 2D flash attention operator to improve GPU utilization and optimize model speed. Add a 2D multi-directional rotation position encoding mechanism to improve model performance.

[0081] (3) Modify the detection head based on ScaledYOLOv4-P6. Replace the batch-norm layer in the detection head with a layer-norm layer. For each location in the feature map at each resolution, predict the classification probability 1-P of each class of object in the dataset for each anchor. obj P obj P1, P obj P2, ..., P obj P Ni Given the bounding box coordinates xc, yc, w, and h, and setting the number of anchors to 4 per resolution, the output dimension of each position in the last layer of the model is 16. (N i +5), which means 16 output candidate boxes.

[0082] (4) Following the detection head is the post-processing step, which performs NMS (Non-Maximum Suppression) on boxes of the same category to remove overlapping boxes and obtain the final model output. At this point, the trained model TM is obtained. i .

[0083] 3. Train a unified anomaly detection model M.

[0084] (1) Unified anomaly detection model structure.

[0085] A model structure with a shared backbone and multiple detector heads is adopted.

[0086] Backbone: Select the same multi-resolution scale 4608 / 3072 / 2048 / 1024 as the teacher model, and choose the same backbone structure as the teacher model.

[0087] Detection Headers: The backbone is followed by six detection heads H1, H2, ..., H6, each targeting a different category in one of the six datasets. These heads have the same structure as the detection heads in their respective teacher models. Resolution filtering strategies are included.

[0088] (2) Strategies for using training samples First, the weights of each dataset are preset. The six datasets are assigned weights according to their importance: W1=0.2, W2=0.15, W3=0.2, W4=0.15, W5=0.15, and W6=0.15. Within each dataset, the weights of each category are preset to be equal and evenly distributed, i.e., dataset DS. i N i Each category has a weight of 1 / N i .

[0089] When selecting each sample, first randomly select a dataset based on probability, then select a category within that dataset based on probability, and then randomly select samples. Repeat this process until eight images are selected as a batch for the next model training.

[0090] After each epoch (training round) of model training, the recognition performance for each category is calculated on the validation set. Let the dataset be DS. i N i The AP (Average Precision) for each category is as follows: AP i1 AP i2 AP iNi So in the next round of training, N i The weights for each category are 1.1-AP. i1 1.1-AP i2 ... 1.1-AP iNi Then normalization is performed. In this way, the class with better performance will require less training resources, while the class with poorer performance will require more samples for training.

[0091] (3) Modify the training loss function.

[0092] Total loss function = α Model accuracy loss + β Model distillation loss Model accuracy loss: for datasets DS i For the given samples, extract the feature dimensions corresponding to the last layer of the model output that correspond to the dataset, and then classify each target class according to the model's predicted classification probability 1-P. obj P obj P1, P obj P2, ..., P obj P Ni The target bounding box coordinates (xc, yc, w, h) are matched with the information of the sample labeled boxes to calculate the loss for the corresponding sample. Based on the number of samples of each category in the validation set, a weighted average of the target bounding box regression loss and classification loss is calculated to obtain the model accuracy loss.

[0093] Model distillation loss: for datasets DS i The sample, using the teacher model TM of the dataset i The unified model M and the sample are predicted separately to obtain the output at each location at each resolution, i.e., the classification probability 1-P. obj P obj P1, P obj P2, ..., P obj P Ni The coordinates of the target bounding box are xc, yc, w, and h. The MSE loss is calculated using the coordinates xc, yc, w, and h output by both models, with respect to the probability distribution 1-P. obj P obj P1, P obj P2, ..., P obj P Ni Calculate the KL divergence loss. For outputs with different resolutions and dimensions, assign different weights and perform a weighted average to obtain the distillation loss for that dataset. Then, calculate the distillation loss for all datasets based on their dataset weights W. i Calculate the weighted average to obtain the model distillation loss.

[0094] 4. Modification of model inference.

[0095] (1) Resolution masking strategy.

[0096] For each category in the dataset, a reasonable resolution range is pre-defined. Boxes outside this range, such as common large-scale anomalies like foundation settlement in DS1, tower cranes in DS2, and escalators in DS5 at 4608 resolution, are masked, and the corresponding P values ​​for each category are reduced. j Setting it to 0 reduces the probability by adding it to the background probability. The reasonable resolution range may differ for the same category across different datasets; therefore, settings were applied separately for each of the 100 anomaly classes across the six datasets.

[0097] (2) Processing of the same category in different datasets.

[0098] The output results from each detection head are merged. For cases where the same anomaly category exists in different datasets and the output boxes overlap, both business characteristics and image characteristics are analyzed to determine whether merging is necessary. If the business characteristics and image characteristics are similar, they will be processed as the same anomaly category, and a weighted average denomination will be applied for deduplication. For example, "transmission equipment defect - missing pin" and "distribution network equipment defect - missing pin". If the business characteristics or image characteristics are different, they will be processed as multiple anomaly categories, and a weighted average denomination will be applied separately for each category. For example, "transmission equipment defect - insulator damage", "distribution equipment defect - insulator damage", and "substation equipment defect - insulator damage" have the same name but different image features, so they are treated as different categories and processed independently using NMS (non-maximum suppression).

[0099] like Figure 3 The diagram shown is a schematic of a full-scene anomaly detection system for power patrol according to this application, which may include: The data module is used to acquire images of power inspection scenes; The detection module is used to input the power inspection scene image into the unified anomaly detection model to obtain the category information and location information of the anomaly. The unified anomaly detection model adopts the ScaledYOLOv4 structure, and for different scenarios, corresponding detection heads are connected after the backbone network of the ScaledYOLOv4 structure. The method for calculating the loss function during the training of the unified anomaly detection model includes: Different scenario datasets with labeled defects are input into separate anomaly detection models to obtain intermediate detection results; The intermediate detection results are used as the benchmark for calculating the loss function during the training of the unified anomaly detection model.

[0100] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of each block is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple blocks may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0101] Furthermore, in the various embodiments of the present invention, the modules can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0102] This application also provides an electronic device, which may include one or more processors, memory and communication interfaces.

[0103] The memory, communication interface, and processor are coupled together. For example, the memory, communication interface, and processor can be coupled together via a bus.

[0104] The communication interface is used for data transmission with other devices. The memory stores computer program code. This computer program code includes computer instructions, which, when executed by the processor, cause the electronic device to perform the steps of the aforementioned power inspection full-scene anomaly detection method.

[0105] The processor can be a processor or controller, such as a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The processor can be used to support an electronic device in performing the method steps provided in the above embodiments.

[0106] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. These buses can be categorized as address buses, data buses, control buses, etc.

[0107] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the above-described power inspection full-scene anomaly detection method.

[0108] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage media known in the art.

[0109] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A power patrol all-scene anomaly detection method, characterized in that, The method comprises the following steps: acquiring a power inspection scene image; inputting the power inspection scene image into a unified anomaly detection model to obtain category information and position information of an anomaly occurrence; the unified anomaly detection model adopts a ScaledYOLOv4 structure, and corresponding detection heads are connected after the backbone network of the ScaledYOLOv4 structure for different scenes; a loss function calculation method in the training process of the unified anomaly detection model comprises: correspondingly inputting different scene data sets with labeled defects into separate anomaly detection models to obtain intermediate detection results; the intermediate detection results are used as a calculation benchmark of the loss function in the training process of the unified anomaly detection model.

2. The method of claim 1, wherein, The different scene data sets with labeled defects input into the separate anomaly detection models further comprise the following processing method: scaling the images in the different scene data sets with labeled defects to different resolution sizes, then respectively performing image slicing, and corresponding the position information and category information of the defect target frame in the scaled images to the sliced images.

3. The method of claim 1, wherein, The separate anomaly detection model adopts a ScaledYOLOv4 structure.

4. The method of claim 3, wherein, A backbone network correction method in the separate anomaly detection model comprises: replacing the batch normalization layer with a network without the batch normalization layer; and / or, adding an image two-dimensional fast attention operator; and / or, adding a 2D multi-directional rotation position encoding mechanism.

5. The method of claim 3, wherein the method further comprises: A detection head correction method in the separate anomaly detection model comprises: respectively adopting target existence probability correlation formula prediction, performing unified probability and position information prediction on all image position fixed reference points under multiple resolution scales, and implementing output dimensions corresponding to different scene data sets according to the corresponding different scene data sets with labeled defects.

6. The method of claim 1, wherein, The loss function of the unified anomaly detection model comprises: wherein, the total loss function is the loss function adopted by the unified anomaly detection model, α is a weight coefficient of the model accuracy loss, and β is a weight coefficient of the model distillation loss; the model distillation loss takes the intermediate detection result as a calculation benchmark.

7. The method of claim 1, wherein, In the process of obtaining the category information and the position information of the anomaly occurrence, the following steps are further included: after the unified anomaly detection model outputs the preliminary detection result, the preliminary detection result output under the preset resolution is filtered and shielded; and preliminary detection results with similar characteristics are merged and deduplicated, and preliminary detection results with different characteristics are respectively retained according to different scenes.

8. A power patrol all-scene anomaly detection system, characterized by, The method comprises the following steps: a data module is configured to acquire a power inspection scene image; a detection module is configured to input the power inspection scene image into a unified anomaly detection model to obtain category information and position information of an anomaly occurrence; the unified anomaly detection model adopts a ScaledYOLOv4 structure, and corresponding detection heads are connected after the backbone network of the ScaledYOLOv4 structure for different scenes; a loss function calculation method in the training process of the unified anomaly detection model comprises: correspondingly inputting different scene data sets with labeled defects into separate anomaly detection models to obtain intermediate detection results; the intermediate detection results are used as a calculation benchmark of the loss function in the training process of the unified anomaly detection model.

9. An electronic device, comprising: The method comprises the following steps: The electronic device comprises a memory, one or more processors; the memory is coupled with the processor; wherein the memory has computer program code stored therein, the computer program code comprises computer instructions, when the computer instructions are executed by the processor, the electronic device executes the steps of the power patrol all-scene anomaly detection method as claimed in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium has a computer program stored therein, when the computer program is executed by the processor, the steps of the power patrol all-scene anomaly detection method as claimed in any one of claims 1-7 are implemented.

Images