Power transmission line high-altitude anomaly detection method combining positive and negative sample algorithms

Abstract

The invention relates to the field of image recognition, and provides a positive and negative sample algorithm combined power transmission line high-altitude anomaly detection method. The objective of the invention is to improve the accuracy and robustness of detection. By comprehensively utilizing the advantages of the positive sample algorithm and the negative sample algorithm, the method effectively solves the problems of background complexity and abnormal data collection. Firstly, a detection picture and a base picture are cut and masked and focused on a power transmission line area; next, a positive sample algorithm is used to identify an abnormal region, while a negative sample algorithm is focused on detecting a specific type of anomaly. Through a multi-stage filtering mechanism including target detection filtering and template matching filtering, false alarms are effectively reduced, and the detection effect is improved. According to the method, a memseg algorithm and a YOLO model are adopted in model training, and an anomaly filtering model is introduced to detect common misinformation targets such as birds and vegetation. Finally, by combining positive and negative sample results, high-precision and high-recall-rate detection performance is realized.

Description

Technical Field

[0001] The present invention relates to the fields of image recognition, artificial intelligence and high-altitude anomaly detection of power transmission lines, and provides a high-altitude anomaly detection method for power transmission lines combining positive and negative sample algorithms. Background Art

[0002] In the field of image recognition and artificial intelligence, high-altitude anomaly detection of transmission lines is an important application scenario, which is mainly used to timely detect abnormal conditions on transmission lines, such as hanging foreign objects and line damage, so as to avoid power accidents. Existing high-altitude anomaly detection methods for transmission lines can be mainly divided into two categories: positive sample algorithms and negative sample algorithms.

[0003] Positive Sample Algorithm

[0004] The core idea of ​​the positive sample algorithm is to establish a normal sample library, which contains multiple images of high-altitude transmission line scenes without abnormalities. During detection, the image to be detected is subjected to feature extraction and comparison with the images in the normal sample library, and the normal image with the most similar features to the image to be detected is found as the base map, and the abnormal area is found through operations such as fusion and segmentation. Specifically, the features (such as color, texture, shape, etc.) of the image to be detected and the images in the normal sample library are usually extracted, and then the normal image with the most similar features to the image to be detected is selected for fusion, and the abnormal area is identified through segmentation operations.

[0005] However, the positive sample algorithm faces great challenges in the high-altitude scenes of power transmission lines. Since the changes in lighting conditions and cloud cover in the high-altitude background are very complex, these changes will increase the difference between the base map in the normal sample library and the image to be detected, thereby affecting the detection accuracy of the algorithm. In addition, due to the complexity of the high-altitude scenes of power transmission lines, it is difficult for the positive sample algorithm to balance the recall rate and precision during the model iteration process, and it is easy to miss or misdetect.

[0006] Negative Sample Algorithm

[0007] The negative sample algorithm collects known abnormal data, labels them, and then uses these labeled data to train target detection models (such as YOLO, SSD, Mask R-CNN, etc.) to detect high-altitude abnormalities in power transmission lines. The advantage of this method is that it can perform targeted detection on known types of abnormalities with high detection accuracy.

[0008] However, the negative sample algorithm also has some obvious defects. First, it is very difficult to collect abnormal data because the occurrence of anomalies is a low-probability event, and it is difficult to obtain a sufficient number of abnormal samples. Second, there are a wide variety of anomalies, which are difficult to enumerate. Even if a good model is trained, it may not be able to cover all abnormal situations. Finally, since the negative sample algorithm is trained based on known anomalies, its detection effect is poor for unknown anomaly types.

[0009] Deficiencies of the prior art

[0010] In summary, the positive sample algorithm and the negative sample algorithm each have their own advantages and disadvantages, and both have certain limitations in the detection of high-altitude anomalies of transmission lines. Although the positive sample algorithm can detect unknown anomalies, it is difficult to simultaneously ensure the detection accuracy and recall rate in complex backgrounds; although the negative sample algorithm can perform high-precision detection for known anomalies, it is difficult to cover all abnormal situations, and its detection ability for unknown anomalies is poor.

[0011] In addition, existing high-altitude anomaly detection methods for transmission lines often only use a single detection algorithm, lack further filtering and optimization of the detection results, are easily affected by background interference and false alarms, resulting in low detection accuracy and recall rate.

[0012] Therefore, how to combine the advantages of the positive sample algorithm and the negative sample algorithm and propose a more accurate and robust high-altitude anomaly detection method for transmission lines has become an urgent problem to be solved by those skilled in the art. Summary of the invention

[0013] The purpose of the present invention is to solve the problems that the existing positive sample algorithm is difficult to balance the recall rate and accuracy under changing background conditions, and the negative sample algorithm has problems in data collection and anomaly enumeration. By combining the positive and negative sample algorithms and adding a filtering module, the accuracy and recall rate of high-altitude anomaly detection of transmission lines are improved.

[0014] To achieve the above object, the present invention adopts the following technical solutions:

[0015] The present invention provides a high-altitude anomaly detection method for transmission lines combining positive and negative sample algorithms, including the following steps:

[0016] Step 1: Process the detection image detect_image and its corresponding base image set base_images and ROI information to obtain the cropped detection image cut_detect_image and the cropped base image set cut_base_images, as well as the masked cropped detection image cm_detect_image and the cropped base image set cm_base_images;

[0017] Step 2: Perform positive sample algorithm processing on the cropped mask detection image cm_detect_image and the set of cropped mask base images cm_base_images to obtain the abnormal segmentation map out_maskM;

[0018] Step 3: Perform negative sample anomaly detection on the cropped mask detection image cm_detect_image to obtain the list of negative sample anomaly detection bounding boxes ab_bboxs;

[0019] Step 4: Calculate the abnormal region contour and its area based on the abnormal binary map ab_maskM. If there is a contour with an area greater than the threshold th_area or the list of negative sample anomaly bounding boxes ab_bboxs is not empty, proceed to Step 5; otherwise, end the detection;

[0020] Step 5: Perform object detection on the cropped detection image cut_detect_image to obtain the list of filtered object bounding boxes fi_bboxs;

[0021] Step 6: Filter out the anomaly bounding boxes in the list of negative sample anomaly bounding boxes ab_bboxs that have a high overlap with the list of filtered object bounding boxes fi_bboxs, and restore the coordinates of the remaining negative sample anomaly bounding boxes to the original image;

[0022] Step 7: Filter out the regions in the positive sample abnormal region ab_maskM that overlap with the list of filtered object bounding boxes fi_bboxs;

[0023] Step 8: Convert the positive sample abnormal result ab_maskM into a rectangular box and restore it to the original image,

[0024] to obtain the positive sample abnormal bounding box src_po_bbox,

[0025] Step 9: Delete the positive sample abnormal bounding box src_po_bbox whose overlap degree ios with the negative sample abnormal bounding box src_ab_bbox is greater than the threshold th_ios;

[0026] Step 10: Perform template matching filtering to remove false alarms in the positive sample abnormal bounding boxes that contain line equipment;

[0027] Step 11: Merge the filtered positive sample abnormal bounding boxes and negative sample abnormal bounding boxes. If there are abnormal bounding boxes, output the abnormal detection result; otherwise, end the detection.

[0028] In the above technical solution, Step 1 includes the following steps:

[0029] Step 1.1: According to the ROI information, calculate the circumscribed rectangle of the ROI (x min , y min , x max , y max), where ROI is the high-altitude part of the transmission line corridor, (x min , y min ) is the upper left corner point of the circumscribed rectangle of ROI, (x max , y max ) is the lower right corner point of the circumscribed rectangle of ROI;

[0030] Step 1.2: Crop detect_image and base_images according to the circumscribed rectangle to obtain cut_detect_image and cut_base_images;

[0031] Step 1.3: Set the pixel values of the areas outside ROI to zero to obtain the cropped mask detection image cm_detect_image and the set of cropped mask base images cm_base_images.

[0032] In the above technical solution, Step 2 includes:

[0033] Step 2.1: Uniformly scale cm_detect_image and cm_base_images to the size of MxM to obtain the uniformly sized cropped mask detection cm_detect_imageM and the set of cropped mask base images cm_base_imagesM;

[0034] Step 2.2: Use the decoder to extract features from the cropped mask detection image cm_detect_imageM and the set of cropped mask base images cm_base_imagesM and calculate the similarity. Select the best cropped mask base image cm_base_imageM according to the similarity, corresponding to the original image of the best base image good_base_image;

[0035] Step 2.3: Send the cropped mask detection image cm_detect_imageM and the best cropped mask base image cm_base_imageM into the positive sample algorithm for comparison detection to obtain the abnormal detection result image, and perform closing operation to obtain the abnormal binary image ab_maskM.

[0036] In the above technical solution, the coordinates of each abnormal box ab_bbox in Step 3 are where is the upper left corner point of the abnormal box, is the lower right corner point of the abnormal box.

[0037] In the above technical solution, Step 6 includes the following steps:

[0038] Step 6.1: Calculate the overlap degree los between the negative sample abnormal box ab_bbox and the filtered target box fi_bbox ab,fi :

[0039]

[0040] The coordinates of each filtering box fi_bbox are is the upper left corner point of the filtering box, is the lower right corner point of the filtering box, and the coordinates of the abnormal box ab_bbox are where is the upper left corner point of the abnormal box, is the lower right corner point of the abnormal box;

[0041] Step 6.2, if ios ab,fi is greater than the threshold th_ioss, then delete the negative sample abnormal box ab_bbox;

[0042] Step 6.3, restore the coordinates of the negative sample abnormal boxes that are not deleted to the original image to obtain the negative sample abnormal box src_ab_bbox, and the coordinates of src_ab_bbox are expressed as

[0043]

[0044] In the above technical solution, step 7 includes the following steps:

[0045] Step 7.1, draw the binary image fi_mask of the abnormal filtering area according to the list of filtering target boxes fi_bboxs;

[0046] Step 7.2, scale fi_mask to the MxM size to obtain fi_maskM;

[0047] Step 7.3, perform an AND operation on ab_maskM and fi_maskM to filter out the abnormal areas that overlap with the filtering target boxes.

[0048] In the above technical solution, step 8 includes the following steps:

[0049] Step 8.1, find the abnormal contours of ab_maskM and their areas, retain the contours with an area greater than th_aera, and find the circumscribed rectangle boxes po_bbox corresponding to the contours;

[0050] Step 8.2, restore the coordinates of po_bbox to the original image to obtain the positive sample abnormal box src_po_bbox, and the coordinates of src_po_bbox are expressed as;

[0051]

[0052] where M is the size;

[0053] In the above technical solution, step 9 includes the following steps:

[0054] Step 9.1, calculate the overlap degree ios between the positive sample abnormal box src_po_bbox and the negative sample abnormal box src_ab_bbox ab，po :

[0055]

[0056] Step 9.2, if ios ab，po is greater than the threshold th_ios, then delete the positive sample abnormal box src_po_bbox.

[0057] In the above technical solution, step 10 includes the following steps:

[0058] Step 10.1, obtain the abnormal template region image tmpl_image and the abnormal background region image bg_image;

[0059] Step 10.2, perform template matching of the abnormal template region image on the background region image to obtain the maximum matching coefficient max_val;

[0060] Step 10.3, if max_val is greater than the threshold th_tmpl, then delete the positive sample abnormal box.

[0061] In the above technical solution, step 10.1 includes the following steps:

[0062] Obtain the positive sample abnormal box Keep the target center unchanged and expand the width and height outwards according to the ratio R o respectively to obtain the upper left corner coordinates and the lower right corner coordinates of the abnormal template region as follows:

[0063]

[0064] Among them, W and H are the width and height of the original image respectively, and obtain the abnormal template region image tmpl_image on the detect_image according to the above region;

[0065] Step 10.2 includes the following steps:

[0066] At the upper left corner coordinates and the lower right corner coordinates of the abnormal template region, further expand according to the ratio Rbg to obtain the upper left corner coordinates and the lower right corner coordinates of the abnormal background region image as follows:

[0067]

[0068] Among them, W and H are the width and height of the original image respectively. The background region image bg_image is obtained on the best base image of the original image good_base_image according to the above region.

[0069] Since the present invention adopts the above technical means, it has the following beneficial effects:

[0070] 1. By combining the positive sample algorithm and the negative sample algorithm, the problems of complex background and difficult collection of abnormal data in the detection of high-altitude abnormalities of transmission lines are solved, and the effects of improving the detection accuracy and recall rate are achieved.

[0071] By combining the positive sample algorithm and the negative sample algorithm, the present invention can effectively cope with the problem of background complexity caused by changes in illumination and clouds in the high-altitude scene of transmission lines, and at the same time overcome the challenge of difficult collection of abnormal data in the negative sample algorithm. The positive sample algorithm can detect unknown abnormalities through the difference detection with the best base image, while the negative sample algorithm improves the detection accuracy of known abnormalities by training known abnormal data. The combination of the two significantly improves the recall rate on the premise of ensuring the accuracy.

[0072] 2. By introducing a target detection filtering module, the problem of false alarms of known non-abnormal targets (such as birds, vegetation, passenger planes, etc.) is solved, and the effect of reducing the false alarm rate is achieved.

[0073] After the present invention detects an abnormality, a target detection filtering module is introduced, which specifically filters the common false alarm targets (such as birds, vegetation, passenger planes, etc.) in the transmission line scene. By comparing the abnormal detection box with the detection boxes of these known non-abnormal targets and using the Intersection over Self (IoS) index, the false alarms are effectively eliminated, thereby significantly reducing the false alarm rate.

[0074] 3. By the template matching filtering module, the problem of false alarms of line equipment in the positive sample algorithm is solved, and the effect of further improving the detection accuracy is achieved.

[0075] After the positive sample algorithm detects the abnormal area, the present invention further introduces a template matching filtering module. By matching the similarity between the abnormal area and the background area, the false alarms caused by line equipment are identified and eliminated. This technical means effectively improves the detection accuracy and reduces the situation of false positives.

[0076] 4. By means of multi-channel data materials and semi-supervised training methods, the training problem of the positive sample model in a complex background is solved, and the effect of improving the detection effect of the positive sample difference area is achieved.

[0077] During the training process of the positive sample model of the present invention, multi-channel data materials are adopted, including abnormal data generated based on the SAM dataset and real abnormal samples, and a semi-supervised training method is used to improve the ability of the model to detect different regions in complex backgrounds, so that the model can more accurately identify abnormalities in the high-altitude scene of transmission lines.

[0078] 5. By means of unified cropping and masking processing, the problems of different image sizes and background interference are solved, achieving the effects of improving the robustness and consistency of the algorithm.

[0079] By performing unified cropping and masking processing on the detection image and the base map, the present invention effectively eliminates background interference, ensures the consistency of the input image, and improves the robustness of the algorithm and the stability of the detection effect.

[0080] 6. By combining the positive sample and negative sample algorithms and adding a post-stage filtering module, the problem that a single algorithm is difficult to balance precision and recall rate is solved, achieving the effect of improving the recall rate while ensuring precision.

[0081] The present invention combines the positive sample and negative sample algorithms and adds a target detection filtering and template matching filtering module after detection, effectively solving the problem that a single algorithm is difficult to balance precision and recall rate. Experimental results show that the present invention achieves a recall rate of up to 97.04% and a precision of 98.11% on the test set, significantly superior to the effects of using only the positive sample or negative sample algorithms alone. Description of the Drawings

[0082] Figure 1 : Overall process flow diagram;

[0083] Figure 2 : Flow chart of the template matching filtering module;

[0084] Figure 3 : General algorithm flow diagram;

[0085] Figure 4 : Schematic diagram of roi drawing, the left figure is one of the base maps, and the right figure is the detection map;

[0086] Figure 5 : Schematic diagram of the cropped map (left figure) and the cropped mask map (right figure);

[0087] Figure 6 : Best cropped mask base map of MxM size (left), cropped mask map (middle), and abnormal detection result map (right);

[0088] Figure 7 : Normal base map (left) and detection map (right), schematic diagram of the final effect of positive sample abnormal detection;

[0089] Figure 8 : The detection image after cropping the mask according to the ROI is detected by the negative sample anomaly detection model to detect the tower crane strut (common anomaly);

[0090] Figure 9 : Detection effect diagrams of birds (left) and vegetation (right);

[0091] Figure 10 : The maximum similarity region (green box in the left figure) in the background region of the base map after the anomaly box detected by the positive sample algorithm on the detection image is expanded (the small figure in the right figure). In this example, the maximum similarity is only 0.348, and it is considered an anomaly and this anomaly box is not filtered. Specific implementation manners

[0092] The following will give a detailed description of the embodiments of the present invention. Although the present invention will be described and explained in conjunction with some specific implementation manners, it should be noted that the present invention is not limited to these implementation manners only. On the contrary, any modifications or equivalent replacements made to the present invention should be covered within the scope of the claims of the present invention.

[0093] In addition, in order to better illustrate the present invention, numerous specific details are given in the following specific implementation manners. Those skilled in the art will understand that the present invention can also be implemented without these specific details.

[0094] The present invention proposes a method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms, which can better realize the detection of high-altitude anomalies of transmission lines.

[0095] Step 1: Obtain the detection image detect_image and the corresponding base image set (base_images) and ROI information, where the base image is an image without anomalies, and the ROI is the high-altitude part of the transmission line channel, such as Figure 4 shown. The base image set refers to a list of several base images similar to detect_image and without anomalies.

[0096] Calculate the circumscribed rectangle (x min , y min , x max , y max ) of the ROI and crop the detection image detect_image and the base image set (base_images) to obtain the cropped detection image cut_detect_image and the cropped base image set cut_base_images; then set the pixel values of the regions outside the ROI to zero to obtain the cropped mask detection image cm_detect_image and the cropped mask base image set cm_base_images; where (x min , y minis the upper left corner point of the ROI circumscribed rectangle, (x max , y max ) is the lower right corner point of the ROI circumscribed rectangle.

[0097] Step 2: Use the cropped mask detection image cm_detect_image and the set of cropped mask base images cm_base_images to perform the positive sample algorithm, obtaining the abnormal segmentation image out_mask. The detected abnormal area is 255 and the normal area is 0. The size of the image is the same as that of cm_detect_image. The specific operation of the positive sample algorithm is as follows:

[0098] Step 2.1: Uniformly scale the sizes of the cropped mask detection image cm_detect_image and the set of cropped mask base images Cm_base_images to MxM (in this example, 768x768 is used), obtaining the uniformly sized cropped mask detection image cm_detect_imageM and the set of cropped mask base images cm_base_imagesM;

[0099] Step 2.2: Use the decoder (in this example, the officially pre-trained ResNet18 model) to extract features (the output of the last convolutional layer) from the uniformly sized cropped mask detection image cm_detect_imageM and the set of cropped mask base images

[0100] cm_base_imagesM. Compare the features of

[0101] cm_detect_imageM with the features extracted from the set of cropped mask base images cm_base_imagesM one by one to find the best cropped mask base image cm_base_imageM. Then, for the original base image, it is good_base_image; in this example, the simplest mean square error discrimination is adopted. This method reflects the similarity degree by calculating the mean square error between the two features (the smaller the mean square error, the higher the similarity degree, and the corresponding base image is used as the best base image).

[0102] Step 2.3: Feed the cropped mask detection image cm_detect_imageM and the best cropped mask base image cm_base_imageM into the positive sample algorithm for comparison detection to obtain the positive sample algorithm anomaly detection result image. Then, perform a closing operation (in this example, use the morphologyEx method in opencv to perform the closing operation, where the kernel size is 9x9) to fill the gaps in the abnormal area to obtain the abnormal binary image ab_maskM. Here, ab_maskM is a binary image with size MxM, where the pixel value 255 represents an abnormal value and 0 represents a normal value. The positive sample algorithm can be a deep learning algorithm or a traditional image positive sample anomaly detection algorithm. In this example, the memseg normal sample anomaly detection algorithm is used.

[0103] Step 3: Perform negative sample anomaly detection on the cropped mask detection image cm_detect_image to obtain a list of negative sample anomaly detection bounding boxes ab_bboxs. The coordinates of each anomaly box ab_bbox are where is the upper left corner point of the anomaly box, is the lower right corner point of the anomaly box.

[0104] Step 4: Calculate the contour and its corresponding area of the abnormal area based on the abnormal binary image ab_maskM. If there is a contour with an area greater than the threshold th_area (empirical value 15) or the list of negative sample algorithm anomaly detection bounding boxes ab_bboxs is not empty, jump to Step 5; otherwise, end the detection.

[0105] Step 5: Start the target detection filtering module. First, perform target detection on the cropped detection image cut_detect_image to obtain a list of filtered target bounding boxes fi_bboxs. The coordinates of each filtered box fi_bbox are , where is the upper left corner point of the filtered box, is the lower right corner point of the filtered box. This target detection is mainly for targets that are prone to false alarms in transmission lines, such as birds, vegetation, airliners, etc.

[0106] Step 6: The target detection filtering module filters the negative sample abnormal bounding boxes and restores the coordinates of the remaining negative sample abnormal bounding boxes to the original image. If the list of negative sample abnormal bounding boxes ab_bboxs is empty, jump to Step 8; otherwise, compare each abnormal bounding box ab_bbox in ab_bboxs with each target bounding box fi_bbox in fi_bboxs in turn to calculate ios (Intersection over self). If ios is greater than the threshold th_ios (0.5 in this example), delete the abnormal bounding box ab_bbox from the list of negative sample abnormal bounding boxes ab_bboxs. The difference between ios and iou is that iou is the ratio of the intersection area to the union area, while ios is the ratio of the intersection area to its own area. If the abnormal bounding box ab_bbox is set as The filtering bounding box fi_bbox is where is the upper left coordinate of ab_bbox, is the lower right coordinate of ab_bbox, and the same applies to fi_bbox. Then, the ios of the abnormal bounding box ab_bbox and the filtering bounding box fi_bbox is expressed as follows:

[0107]

[0108] Through the above operations, there may be unfiltered negative sample abnormal bounding boxes left. At this time, these negative sample abnormal bounding boxes need to be restored to the original image. Since the negative sample abnormal bounding boxes are detected for targets on the cropped mask detection image cm_detect_image obtained by cropping the mask according to the roi in the original image, if the negative sample abnormal bounding box ab_bbox is set as Then the coordinates of the negative sample abnormal bounding box src_ab_bbox restored to the original image can be expressed as

[0109]

[0110]

[0111] Step 7: The target detection filtering module filters the positive sample abnormal regions. If there is an abnormal region contour in the positive sample abnormal result ab_maskM with an area greater than the threshold th_area (empirical value 15), draw the binary image fi_mask of the abnormal filtering region according to the size of the cropped mask detection image cm_detect_image and the list of filtering target bounding boxes fi_bboxs. Then, scale the binary image fi_mask to the MxM size using the nearest neighbor method to obtain fi_maskM. Finally, perform the AND operation on ab_maskM and fi_maskM, that is, when the pixel values at the same position are both 255, the pixel value at that position in ab_maskM remains 255; otherwise, it is set to 0.

[0112] Step 8: Transform the abnormal contour of the positive sample ab_maskM into a rectangular box and restore it to the original image. Find the abnormal contour and its area of ab_maskM, retain the contours with an area greater than th_aera, and find the corresponding bounding rectangle po_bbox of the contour as the positive sample abnormal box under the MxM size. Since the size of ab_maskM is the same as the scaled and cropped mask detection image cm_detect_imageM, and cm_detect_imageM is obtained by scaling the cropped mask detection image cm_detect_image by MxM, and cm_detect_image is obtained by cropping detect_image according to the bounding rectangle (x min , y min , x max , y max ) and adding a mask. In summary, let the bounding rectangle of the abnormal contour be po_bbox as Then the coordinates of the positive sample abnormal box src_po_bbox restored to the original image can be expressed as

[0113]

[0114] Step 9: Delete the positive sample abnormal box src_po_bbox whose intersection over union (IoU) with the negative sample abnormal box is greater than the threshold th_ios;

[0115] Step 10: Template matching filtering module to filter out false alarms in the positive sample abnormal boxes that contain line equipment. The specific method is as follows:

[0116] Step 10.1 Obtain the abnormal template region image. Obtain the positive sample abnormal box src_po_bbox Keep the target center fixed and expand outward by a ratio R o (0.3 in this example) for the width and height respectively to obtain the upper left corner coordinates and the lower right corner coordinates which are expressed as follows:

[0117]

[0118] where, W and H are the width and height of the original image respectively. Obtain the abnormal template region image tmpl i mage according to the above region on detect_image.

[0119] Step 10.2 Obtain the abnormal background region image. Refer to the upper left corner coordinates and the lower right corner coordinates of the abnormal template region and further expand by a ratio R bg(Take 0.3 in this example) Expand to obtain the upper-left coordinate of the abnormal background area map and the lower-right coordinate which are expressed as follows:

[0120]

[0121] where W and H are the width and height of the original image respectively. Obtain the background area map bgimage according to the above area on the best base image of the original image good_base_image.

[0122] Step 10.3 Perform template matching of the abnormal template area map on the background area map. During the template matching process, use the abnormal template area map as the template and the background area map as the background. The matching method uses the template algorithm with the normalized correlation coefficient, and select the maximum coefficient max_val as the output. The larger the coefficient max_val, the greater the possibility that there is an area similar to the abnormal template area map in the background area map.

[0123] Step 10.4 Delete the positive sample abnormal box according to the threshold. If max_val is greater than the threshold th_tmpl (take 0.8 in this example), delete the positive sample abnormal box as a false alarm generated by the device.

[0124] Step 11: Merge the filtered positive sample abnormal boxes and negative sample abnormal boxes. If there is an abnormal box, it means an abnormality is detected, and then output the abnormal detection result. If there is no abnormal box, jump to end the detection;

[0125] Model training:

[0126] 1. Training of the positive sample abnormal detection model. In this example, use the memseg positive sample library abnormal detection model.

[0127] 1.1 In terms of data preparation, there are mainly two considerations. On the one hand, it is the generation of abnormalities randomly based on the objects in the sam dataset. On the other hand, it is the collection of real abnormal samples. The former considers the generalization of unknown abnormalities, and the latter considers the real abnormal situations;

[0128] 1.2 During the training process, it should be noted that the generation of abnormal data is during the training process rather than generated in advance. The data given to the data processing module is two types of data. One is the detection map and the base map with abnormalities, and the other is the detection map, the base map and the abnormal binary map label without abnormalities. For the former, there is no need to generate abnormalities anymore, while for the latter, abnormalities need to be randomly obtained from the abnormal material library in real life and the sam dataset (use sam0 and sam99 in this example) and fused on the detection map to generate, and at the same time give the generated abnormal binary map label.

[0129] 1.3 The positive sample anomaly detection model aims to find out the abnormal differences between the detected image and the base map.

[0130] 2. Training of the negative sample anomaly detection model. The yolo model is used in this example.

[0131] 2.1 The data mainly consists of three parts, namely: data without anomalies, collected anomalous data, and generated anomalous data, with a ratio of 1:1:3. The generated data is obtained by fusing the sam dataset and network materials as anomalies on the data without anomalies, and at the same time, the positions and sizes of the corresponding anomalies can be obtained without further annotation; the collected anomalous data is from the data collected at the transmission line site, including cranes, tower cranes, line entanglements, floating objects, etc., which need to be manually annotated.

[0132] 2.2 When training the model, it is necessary to crop and mask the detected image according to the roi.

[0133] 2.3 The difference from general object detection is that here the anomaly types are not distinguished for labeling, but all anomaly categories are regarded as one category (including randomly generated anomalies), aiming to let the model learn the feature differences between anomalies and the transmission line background as much as possible.

[0134] 3. Training of the anomaly filtering model. The yolo model is used in this example. Collect and label non-anomalous images in the high-altitude scene, including birds, vegetation, airliners, etc., and train the anomaly filtering model using different labels for each category.

[0135] In this application, the positive sample algorithm and the negative sample algorithm are combined, combining the characteristics that the positive sample algorithm is good at detecting anomalies different from the base map and the negative sample algorithm is good at detecting known and enumerable anomalies. Selecting the high-precision models of both for combination effectively solves the problem of improving the recall while ensuring the precision (limiting the false alarm rate). On the test set, when using the positive sample anomaly detection alone, the recall is 91.23% and the precision is 87.12%; when using the negative sample anomaly detection alone, the recall is 81.33% and the precision is 98.16%; selecting the high-precision positive sample anomaly detection model and negative sample anomaly detection model for combination and adding a post-stage filtering module, finally both the precision and the recall are effectively improved, with the recall being 97.04% and the precision being 98.11%.

Claims

1. A method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms, characterized in that, It includes the following steps: Step 1: Process the detection image detect_image, its corresponding base image set base_images, and ROI information to obtain the cropped detection image cut_detect_image and the cropped base image set cut_base_images, as well as the cropped detection image cm_detect_image and the cropped base image set cm_base_images after mask processing; Step 2: Perform positive sample algorithm processing on the cropped mask detection image cm_detect_image and the cropped mask base image set cm_base_images to obtain the abnormal segmentation map ab_maskM; Step 3: Perform negative sample abnormal detection on the cropped mask detection image cm_detect_image to obtain the negative sample abnormal detection box list ab_bboxs; Step 4: Calculate the abnormal region contour and its area based on the abnormal binary map ab_maskM. If there is a contour with an area greater than the threshold th_area or the negative sample abnormal box list ab_bboxs is not empty, go to Step 5; otherwise, end the detection; Step 5: Perform object detection on the cropped detection image cut_detect_image to obtain the filtered object box list fi_bboxs; Step 6: Filter the abnormal boxes in the negative sample abnormal box list ab_bboxs with a high overlap degree with the filtered object box list fi_bboxs, and restore the coordinates of the remaining negative sample abnormal boxes to the original image; Step 7: Filter the regions in the positive sample abnormal region ab_maskM that overlap with the filtered object box list fi_bboxs; Step 8: Convert the filtered positive sample abnormal result ab_maskM into a rectangular box and restore it to the original image to obtain the positive sample abnormal box src_po_bbox; Step 9: Delete the positive sample abnormal box src_po_bbox with an overlap degree ios greater than the threshold th_ios with the negative sample abnormal box src_ab_bbox; Step 10: Perform template matching filtering to remove false alarms of line equipment included in the positive sample abnormal boxes again; Step 11: Merge the filtered positive sample abnormal boxes and negative sample abnormal boxes. If there are abnormal boxes, output the abnormal detection result; otherwise, end the detection.

2. The method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms according to claim 1, characterized in that Step 1 includes the following steps: Step 1.

1. Calculate the circumscribed rectangle (x min , y min , x max , y max ) of the ROI according to the ROI information, where the ROI is the high-altitude part of the transmission line corridor, (x min , y min ) is the upper left corner point of the circumscribed rectangle of the ROI, and (x max , y max ) is the lower right corner point of the circumscribed rectangle of the ROI; Step 1.2: Crop detect_image and base_images according to the bounding rectangle to obtain cut_detect_image and cut_base_images; Step 1.3: Set the pixel values of the regions outside the ROI to zero to obtain the cropped mask detection image cm_detect_image and the cropped mask base image set cm_base_images.

3. A method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms according to claim 1, characterized in that, Step 2 includes: Step 2.1: Uniformly scale cm_detect_image and cm_base_images to the size of MxM to obtain the cropped mask detection cm_detect_imageM of the unified size and the set of cropped mask base images cm_base_imagesM; Step 2.2: Use the decoder to extract features from the cropped mask detection image cm_detect_imageM and the set of cropped mask base images cm_base_imagesM and calculate the similarity. Select the best cropped mask base image cm_base_imageM according to the similarity, corresponding to the original image of the best base image good_base_image; Step 2.3: Send the cropped mask detection image cm_detect_imageM and the best cropped mask base image cm_base_imageM into the positive sample algorithm for comparison detection to obtain the abnormal detection result image, and perform closing operation to obtain the abnormal binary image ab_maskM.

4. A method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms according to claim 1, characterized in that, The coordinates of each abnormal box ab_bbox in step 3 are where is the upper left corner point of the abnormal box, is the lower right corner point of the abnormal box.

5. A method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms, characterized in that Step 6 includes the following steps: Step 6.1, calculate the overlap degree ios between the negative sample abnormal box ab_bbox and the filtered target box fi_bbox ab，fi : The coordinates of each filtering box fi_bbox are which is the upper left corner point of the filtering box, which is the lower right corner point of the filtering box. The coordinates of the abnormal box ab_bbox are where is the upper left corner point of the abnormal box, which is the lower right corner point of the abnormal box; Step 6.2, if ios ab,fi is greater than the threshold th_ios, then delete the negative sample abnormal box ab_bbox, Step 6.3: Restore the coordinates of the un-deleted negative sample anomaly boxes to the original image to obtain the negative sample anomaly box src_ab_bbox, and the coordinates of src_ab_bbox are expressed as 6. The method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms according to claim 3, characterized in that, Step 7 includes the following steps: Step 7.1: Draw the abnormal filtering region binary image fi_mask according to the filtered target box list fi_bboxs; Step 7.2: Scale fi_mask to the size of MxM to obtain fi_maskM; Step 7.3: Perform an AND operation on ab_maskM and fi_maskM to filter out the abnormal regions overlapping with the filtered target boxes.

7. A method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms according to claim 3, characterized in that, Step 8 includes the following steps: Step 8.

1. Obtain the abnormal contour and its area of ab_maskM, retain the contours with an area greater than th_aera, and find the corresponding bounding rectangle po_bbox of the contour. po_bbox is the upper left corner coordinates of po_bbox, and the lower right corner coordinates of po_bbox; Step 8.2, restore the coordinates of po_bbox to the original image to obtain the positive sample abnormal box src_po_bbox, and the coordinates of src_po_bbox are expressed as; Where M is the size.

8. A method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms according to claim 7, characterized in that, Step 9 includes the following steps: Step 9.1: Calculate the overlap degree ios between the positive sample abnormal box src_po_bbox and the negative sample abnormal box src_ab_bbox ab，po : Step 9.

2. If ios ab，po is greater than the threshold th_ios, then delete the positive sample abnormal box src_po_bbox.

9. A method for detecting high-altitude anomalies of transmission lines by combining positive and negative sample algorithms according to claim 3, characterized in that, Step 10 includes the following steps: Step 10.1: Obtain the abnormal template region image tmpl_image and the abnormal background region image bg_image; Step 10.2: Perform template matching on the abnormal template region image on the background region image to obtain the maximum matching coefficient max_val; Step 10.3: If max_val is greater than the threshold th_tmpl, delete the positive sample abnormal box.

10. According to the method for detecting high-altitude abnormalities of transmission lines combining positive and negative sample algorithms described in claim 3, characterized in that, Step 10.1 includes the following steps: Obtain positive sample abnormal bounding boxes Keep the target center fixed and scale by ratio R o Expand the width and height respectively to obtain the upper-left coordinates of the abnormal template region and the lower-right coordinates Shown as follows: Among them, W and H are respectively the width and height of the original image, and the abnormal template region image tmpl_image is obtained on the detect_image according to the above regions; Step 10.2 includes the following steps: The upper left corner coordinates of the abnormal template area and the lower right corner coordinates are further expanded according to the ratio R bg to obtain the upper left corner coordinates of the abnormal background area map and the lower right corner coordinates which are expressed as follows: Among them, W and H are the width and height of the original image respectively, and the background region image bg_image is obtained on the best base original image good_base_image according to the above region.

Images