A model-assisted semi-automatic power inspection target detection labeling method
The model-assisted semi-automatic power line inspection target detection and annotation method, which combines manual annotation with detector prediction and optimizes the annotation set, solves the problems of high cost and noise in power line inspection, and achieves high-efficiency, low-cost, high-quality annotation, which is suitable for the highly specialized field of power line inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2026-02-06
- Publication Date
- 2026-06-09
AI Technical Summary
In the field of power line inspection, the cost of target detection and labeling is high and the detector prediction is noisy, which increases the cost of verification. It is difficult to balance labeling quality and efficiency and cannot meet the requirements of high precision.
A model-assisted semi-automatic power line inspection target detection and annotation method is adopted. The annotation set is optimized by initial manual annotation, detector training and inference, prediction difference comparison and manual discrimination and screening, forming a pseudo-annotation set, and iteratively optimizing the annotation quality.
It significantly reduces labor costs, improves annotation quality, adapts to high-resolution images and multi-scale target detection, and is suitable for highly specialized power line inspection fields, providing high-quality annotation data support.
Smart Images
Figure CN122176273A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of machine vision and power line inspection, specifically relating to a model-assisted semi-automatic power line inspection target detection and annotation method. Technical Background With the large-scale application of deep learning technology in the field of power line inspection, the performance of core tasks such as defect identification and key component detection of power transmission and distribution equipment has improved, making the reliance on high-quality labeled data increasingly significant. Power line inspection scenarios require the detection of more than ten types of equipment and defects, including glass insulators, suspension clamps, insulator spontaneous breakage, and clamp corrosion. The morphological characteristics and defect manifestations of different targets have strong industry-specific features, which places stringent requirements on the accuracy of labeled data.
[0002] Object detection is a fundamental task in power line inspection machine vision systems, but its data annotation time and manpower costs remain high. Existing research shows that manual annotation of a single power equipment detection box takes an average of 10 seconds, and subsequent professional review to ensure annotation accuracy consumes an additional 5 seconds. To allow the model to fully learn the subtle features and defect patterns of power equipment, tens of thousands of detection boxes typically need to be annotated for a single object category, directly driving up the total annotation cost.
[0003] This cost issue is even more pronounced in the field of power grid inspection. Power grid inspection data involves core information about power grid equipment, and the annotation work cannot be completed by ordinary annotators. It must rely on domain experts or senior technicians with professional knowledge of the power industry. These personnel need to accurately distinguish the differences between various insulator and clamp models and accurately identify hidden defects such as spontaneous explosion and corrosion. The professional threshold is much higher than that of general visual inspection scenarios, further exacerbating the cost pressure of annotation.
[0004] In recent years, the basic performance and generalization ability of target detection algorithms have been continuously improved. A small number of labeled samples are sufficient to support the detector in building preliminary detection capabilities, enabling the identification of relatively simple power equipment such as insulators and suspension clamps. Theoretically, detector predictions can replace some manual labeling, thereby reducing costs. However, in actual power inspection scenarios, the output results of detectors generally contain noise: first, the detection frame positioning is biased, making it difficult to accurately define small targets such as insulator self-explosion points and rusted areas of clamps; second, target category misjudgment easily occurs, confusing composite insulators with glass insulators, or identifying normal parts as defective parts. These noises cause the labeling quality of model predictions to fall far short of the high-precision requirements of power inspection, necessitating manual review and correction.
[0005] However, if all the power inspection image annotation results output by the detector are manually reviewed, it will generate a lot of additional professional verification costs, creating a new contradiction of "reducing annotation costs but increasing verification costs".
[0006] In summary, the power line inspection field urgently needs a target detection and annotation method that is adapted to the characteristics of the industry, which can effectively control the dual costs of manual annotation and professional verification, while ensuring the accuracy of the annotation data and meeting the needs of deep learning models for high-quality annotation data in power line inspection. Summary of the Invention
[0007] To address the technical challenges of high annotation costs, increased verification costs due to noise in detector predictions, and the difficulty in balancing annotation quality and efficiency in existing power line inspection target detection technologies, this invention proposes a model-assisted semi-automatic power line inspection target detection annotation method. The specific technical solution is as follows: S1. Acquire power inspection images containing power transmission and distribution equipment, and label the power equipment targets and defect targets in the images to obtain an initial reference image set and an initial reference label set; further update the detector based on the initial reference label set and the initial reference image set; the label includes the index of the image to which the target belongs, the coordinates of the target detection box, and the category to which the target belongs; S2, Collect power inspection images containing power transmission and distribution equipment to be labeled, forming an image set to be labeled; Input the image set to be labeled into the detector to obtain the basic label set; S3: Based on the basic annotation set, the image set to be annotated, the reference image set, and the reference annotation set, the detector is trained to obtain the trained detector; the image set to be annotated is input into the trained detector to obtain the predicted annotation set; S4, taking the power inspection images to be labeled as units, based on the intersection-union ratio calculation results of the detection boxes of each label in the basic label set and each label in the prediction label set and the category comparison results, the corresponding labels in the prediction label set and the basic label set are assigned to the matching label set or the non-matching label set. S5, sequentially determine whether each label in the mismatched label set is adopted. If the result is adopted, update the basic label set according to the adopted labels; otherwise, do not update the basic label set. Finally, use the updated basic label set as the optimized pseudo label set. S6 uses the optimized pseudo-label set as the new base label set, and repeats S3-S6 until the proportion of unadopted labels in this round of iteration exceeds the threshold. At this point, the loop ends and the optimized pseudo-label set is output, which is the final labeling result of the image to be labeled.
[0008] Furthermore, when looping through S3-S6, the detector in S3 is the trained detector obtained from the previous loop.
[0009] Furthermore, in step S4, the condition for determining that a label belongs to the set of matching labels is: ; in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. To match the labeled set, For intersection, union, and comparison, As constraints, These represent the image index, bounding box coordinates, and target category of target j, respectively. To set the maximum threshold for intersection-union ratio, These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing existence and "and" respectively.
[0010] Furthermore, in S4, the mismatched annotation set includes four sub-annotation sets: new target, category adjustment, deleted target, and border adjustment.
[0011] Furthermore, the condition for determining whether an annotation belongs to the newly added target sub-annotation set is: ; in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. To add a new target sub-label set, For intersection, union, and comparison, As constraints, Here, represents the image index and the bounding box coordinates of target j, respectively. To minimize the crossover-union ratio (CUI), These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing "any" and "and" respectively.
[0012] Furthermore, the condition for determining that a label belongs to the category adjustment sub-label set is: ; in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. Adjust the sub-label set for the category. For intersection, union, and comparison, As constraints, These represent the image index, bounding box coordinates, and target category of target j, respectively. To set the maximum threshold for intersection-union ratio, These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing existence and "and" respectively.
[0013] Furthermore, the condition for determining that an annotation belongs to the set of sub-annotations to be deleted is: ; in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. To delete the target sub-label set, For intersection, union, and comparison, As constraints, Here, represents the image index and the bounding box coordinates of target j, respectively. To minimize the crossover-union ratio (CUI), These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing "any" and "and" respectively.
[0014] Furthermore, the condition for determining that an annotation belongs to the set of border adjustment sub-annotations is: ; in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. Adjust the sub-label set for the border. For intersection, union, and comparison, As constraints, These represent the image index, bounding box coordinates, and target category of target j, respectively. To set the maximum threshold for intersection-union ratio, To minimize the crossover-union ratio (CUI), These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing existence and "and" respectively.
[0015] Furthermore, when the next batch of images to be labeled is acquired, the detector in S2 is the detector that has been trained and obtained when the loop stopped last time, the reference image set in S3 contains the initial reference image set and all historical images to be labeled, and the reference annotation set contains the final annotation results of the initial reference annotation set and all historical images to be labeled.
[0016] Compared with the prior art, the present invention has the following significant advantages: (1) Significantly reduced labor costs: Through the semi-automatic mode of “model prediction + manual discrimination of differences”, only manual binary classification of single target sub-graphs is required, without full annotation and verification. The time spent on manual work is much less than that of traditional full manual annotation, significantly reducing the dual costs of annotation and verification. (2) Continuous improvement in annotation quality: By manually judging and correcting model prediction noise, and through scenarios such as IoU threshold division matching, addition, and category adjustment, annotation details are precisely optimized. The quality of pseudo-annotations significantly exceeds the performance of the detector itself, and steadily improves with each iteration. The updated high-quality annotations feed back into the detector training, improving the model's detection performance. In turn, the optimized detector can output better prediction results, forming a positive cycle of "annotation quality - model performance". (3) Strong scene adaptability: It is suitable for high-resolution images and multi-scale target detection needs, and is applicable to vertical fields with high professionalism and high annotation threshold, such as power inspection. It provides high-quality annotation data support for deep learning tasks in various industries and has broad practicality and promotion value. Attached Figure Description
[0017] Figure 1 This is an overall framework diagram of the method of the present invention.
[0018] Figure 2 This refers to the update order of the variables in the method of this invention.
[0019] Figure 3 This refers to changes in detector performance and pseudo-label quality.
[0020] Figure 4 It refers to the pseudo-annotation changes of some images during the auxiliary annotation process. Detailed Implementation
[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.
[0022] Given a set of power inspection images containing power transmission and distribution equipment The purpose of object detection annotation is to build a batch of annotations. Each of the annotations Index of the image to which the target belongs Target bounding box coordinates Target Category Composition, used to represent an image A target object on, where and These represent the coordinates of the top left and bottom right corners of the target bounding box, respectively.
[0023] Based on the above background, this invention proposes a semi-automatic power line inspection target detection and labeling method, which includes nine main steps: initial manual labeling, initial detector training and inference, updated detector training and inference, comparison of prediction differences, manual discrimination and screening, and updated labeling.
[0024] like Figure 1 and Figure 2 As shown, the specific steps are as follows: Step 1: Sample a portion of the collected power inspection images and manually annotate them as a reference annotation set to establish initial annotation standards. The remaining unannotated images are used as data to be annotated. First, a certain proportion of images are sampled from the power line inspection image set as a reference image set. Each target in the image is manually labeled to obtain a reference label set. Each annotation contains the index of the image corresponding to the target, the target bounding box, and the target category.
[0025] The remaining images in the original image dataset are denoted as In subsequent processes, a detector is used to assist manual acquisition of pseudo-labels for the remaining images. .
[0026] in, It represents the union of sets.
[0027] The image sampling process for dividing the reference image set and the data to be labeled can be completely random, or it can be targeted sampling combined with a specific data distribution (such as deliberately selecting a batch of samples that are more difficult, have more targets, and contain more rare classes). Through manual annotation, the basic annotation methods and special cases for various categories can be determined, the annotation standards for each category can be clarified (such as the bounding box drawing range and category determination rules), and the initial annotation specifications can be established as a reference for network learning.
[0028] Step 2: Train the detector based on the reference annotation set to obtain model weights with preliminary detection capabilities. Given detector And initial weights, using the reference dataset Train the detector to obtain weights that have preliminary detection capabilities. : in, The weight parameters represent the detector's weights. These can be publicly available pre-trained weights or randomly initialized weights. The initial values for the weight parameters are set to... ; This describes the process of training a detector. The specific process may vary depending on the detector architecture and training method. This invention does not limit the specific type of detector.
[0029] During the training process, this invention... The system is further divided into a training set and a test set. The training set is used to calculate the loss function associated with the detector. The detector is considered to be trained successfully when a preset number of iterations or convergence is reached. The test set is used to evaluate the generalization ability. Since the trained detector weights will be used for inference and prediction later, the test set can be used to determine the optimal training parameters.
[0030] Step 3: Use the trained detector to infer on unlabeled data to generate initial pseudo-labels. Unlabeled image data Input the initially trained detector and use the weights obtained during training. In unlabeled image data The initial pseudo-labels are obtained by reasoning on the above. : in, This describes the inference process of the detector. This stage requires selecting an appropriate network confidence level based on the actual data. If the focus is on discovering more new objects, a smaller confidence level can be used to retain more prediction results; if the goal is to further improve the annotation quality, a larger confidence level should be used.
[0031] Step 4: Merge the reference annotation set With initial pseudo-labels A fully labeled dataset was constructed with all images, and the detector was retrained to improve model performance. The pseudo-labels obtained through reasoning Compared with manual annotation Merged into the first round of annotations Using a fully labeled dataset The detector network is retrained until a preset number of iterations or the loss function converges, resulting in weights that enhance detection capabilities. .
[0032] The initial weight parameters of the detector at this time It doesn't need to be exactly the same as the first training in step two; it can be used flexibly in practice. Initialization or other random weight initialization.
[0033] During the training process described above, it is still necessary to randomly re-divide the training and test sets. Randomly dividing the dataset can also ensure that differences are generated in multiple training sessions, thus ensuring the model's generalization ability.
[0034] Step 5: Use the re-trained detector again on the unlabeled image data. Inferring from above, generating updated pseudo-labels. The unlabeled image data is re-inputted into the detector after secondary training, and then... Get the updated pseudo-labels .
[0035] Step Six: Compare the differences between the initial pseudo-labels and the updated pseudo-labels. Divide the labels into five categories based on the Intersection over Union (IoU) threshold: matching, added targets, category adjustment, deleted targets, and border adjustment. Construct the corresponding label sets. Initial pseudo-labels Considered as basic annotation The updated pseudo-labels Considered as a prediction label Compare images individually. and The differences are analyzed by constructing five annotation sets according to the following four scenarios. , , , , .
[0036] (1) Matching. A target in the predicted annotation corresponds completely to a base annotation, that is, the intersection-over-union (IoU) ratio between the bounding box of a predicted annotation and the bounding box of a base annotation is greater than a threshold. And they are of the same category.
[0037] in, This represents the predicted label corresponding to target j in the predicted labeling. This represents the base annotation corresponding to target k in the base annotation.
[0038] (2) New target. A target in the predicted annotation does not exist in the base annotation, that is, the intersection-union ratio of the border of a predicted annotation and the borders of all base annotations is less than the threshold. .
[0039] (3) Category adjustment. A target category in the predicted annotation is inconsistent with the base annotation, that is, the intersection-union ratio of the border of a predicted annotation with the border of a base annotation is greater than the threshold. However, the categories are inconsistent.
[0040] (4) Target deletion. A target that has no base label in the predicted annotations, i.e., the intersection-union ratio of the border of a base label and the borders of all predicted labels is less than the threshold. .
[0041] (5) Border Adjustment. A significant difference exists between the border of a predicted target and the base border; specifically, the IOU threshold between the border of a predicted target and the border of a base target is within a certain range. and They are between, and of the same category.
[0042] By manually adjusting the threshold and The above five sets can basically cover and All matching and difference cases.
[0043] Step 7: Crop the labeled regions corresponding to the labeled set into single-target sub-images, manually determine whether to accept the modification, and filter out label noise. Of the five sets obtained in step six, The annotations that correspond to the two predictions being consistent can be considered as a more reasonable part of the basic annotations and can be ignored. The other four sets correspond to the differences between the two annotations, and the annotation result of one of them can be selectively accepted.
[0044] Considering that manually determining the rationality of target detection annotations is time-consuming, but manually performing image binary classification is much faster, this invention will... , , , These four sets are labeled with their corresponding border ranges and then expanded outward by a certain scale to be cropped into single-target sub-images. Each sub-image contains only one main object and one to two borders, corresponding to the basic labeled borders and the predicted labeled borders, respectively.
[0045] After obtaining the cropped sub-images, each sub-image needs to be manually reviewed to determine whether modifications are acceptable. If accepted, the base annotations will be updated according to the predicted annotations; otherwise, the base annotations will remain unchanged. This step allows for manual removal of noise from the annotations and control of annotation quality.
[0046] in, This indicates a manual judgment operation. , , , This represents the set that accepts modification after the judgment.
[0047] If the category or bounding box range results in both the base and predicted annotations of the target do not match reality, manual correction of the annotations is required. This step can also be accomplished by training one or more additional classification models, because the binary classification task of the cropped subimage is much simpler than the original object detection task, and even using a network of similar size can produce better classification results.
[0048] Step 8: Update the original pseudo-labels based on the manual judgment results to form an optimized label set. Update the original pseudo-labels according to the results of manual judgment. The optimized pseudo-label set is obtained. : in, This indicates the function used for updating. During the update process, new functions are added. Target, delete Adjust the goal Adjust the border. The category.
[0049] Manual spot checks are required after the update. The annotations in the code ensure that there are no issues such as incorrect updates.
[0050] Step 9: Repeat the update of labels and model until convergence. The pseudo-label set will be sampled. As new initial pseudo-labels, steps four through eight are repeated, and the detector makes further predictions. It can be used to continuously update the basic annotations. At the end of each loop, all annotations in the five categories of annotation sets (matching, adding targets, category adjustment, deleting targets, and border adjustment) are deleted, and the sets are initialized to empty before entering the next loop.
[0051] When most of the modifications in step seven (e.g., 90%) are manually judged as unacceptable, it means that the quality of the basic annotations generated in this round is high enough that it is difficult to modify them through the current model. At this point, it is determined to be converged, the loop can be terminated, and the currently trained detector and the optimized pseudo-annotation set that is close to the full manual annotation are obtained. The optimized pseudo-annotation set is the final annotation of the image to be annotated.
[0052] After acquiring a new batch of images to be labeled, use this new batch of images as... Input the currently trained detector to get The final annotation will be used as Repeat steps four through eight until convergence is determined. The pseudo-label set generated at this point is the label corresponding to the image to be labeled.
[0053] To verify the effectiveness of the method of the present invention, further experiments were designed, and the specific experimental settings are as follows: 1. Dataset This invention collected 27,001 power grid inspection images with a resolution of 640 to form a dataset to verify the effectiveness of the proposed method. The dataset is labeled with 14 types of equipment and defects, including glass insulators, spontaneous glass insulator explosions, composite insulators, suspension clamps, and suspension clamp corrosion. The dataset is divided into three subsets: training, expansion, and testing. The training set simulates manually labeled data from actual use, while the expansion set simulates unlabeled data; the labels in the expansion set are not visible to the network. The proposed method uses the training and expansion sets to train the detector and evaluates its performance on the test set. In this experiment, the number of images in the training and expansion sets is the same, simulating a 50% labeled and 50% unlabeled scenario. The statistical results of the data distribution in each subset are shown in Table 1.
[0054] Table 1. Data distribution statistics for each subset of the inspection dataset. 2. Evaluation Indicators Average precision (AP) is a key metric for measuring the overall performance of a detector across all confidence thresholds. After matching a certain number of labeled bounding boxes and predicted bounding boxes using a specific algorithm, the results can be categorized into the following cases: (1) True Positive (TP): Samples that are actually positive are correctly predicted as positive, and the labeled objects are detected and the labeled boxes match the predicted boxes.
[0055] (2) True Negative (TN): Samples that are actually negative are correctly predicted as negative, and the labeled background area has no prediction box.
[0056] (3) False Positive (FP): Samples that are actually negative are incorrectly predicted as positive, and the labeled background area appears as a prediction box.
[0057] (4) False Negative (FN): A sample that is actually a positive class is incorrectly predicted as negative. Therefore, precision and recall are defined as follows: When the detector's confidence threshold changes, some low-confidence predicted boxes are discarded, thus affecting the overall precision and recall. A precision-recall curve can be plotted by iterating through all thresholds, using precision and recall as coordinates. The area under this curve represents the detector's overall performance, i.e., the average precision (AP). Since calculating the area under the curve using only integral methods is too time-consuming, current research typically uses fixed sampling points to estimate the area.
[0058] In the above process, the matching of the labeled bounding boxes and the predicted bounding boxes is determined by the Intersection over Union (IoU) threshold. IoU is defined as the area of intersection of two bounding boxes divided by their area of union; only bounding box pairs with an IoU greater than the threshold are considered a match. Different IoU thresholds result in different AP calculations; commonly used thresholds are 0.5 and 0.75, corresponding to... and .
[0059] 3. Network and Training Details This invention selects the commonly used baseline object detection model YOLOv11-Large as the detector, and assists with seven rounds of manual annotation. The first round of annotation includes steps two and three, which only trains the initial detector weights and makes predictions on the extended set. Each subsequent round of annotation includes the complete process of steps four through eight. After training the detector, the results of manual judgment are used to update the pseudo-labels. The code is implemented based on the PyTorch 2.4 framework and CUDA 12.4 environment, and uses four NVIDIA RTX 4090 GPUs for network training. The training parameters for each round are set according to the standard YOLOv11 training process: a stochastic gradient descent (SGD) optimizer is used, with a weight decay factor of 0.0005 and a momentum of 0.937; during training, the learning is initially warmed up to 0.01, and then decays linearly. There are 100 training rounds, a batch size of 32, and an image size of 640.
[0060] 4. Evaluation of Model-Assisted Annotation Effectiveness To quantitatively evaluate the proposed auxiliary annotation method, this experiment assesses its detection performance on both the extended set and the test set after each round of detector training, denoted as follows: and After updating the pseudo-labels based on the detection predictions and manual judgment results, this experiment also calculates the difference between the pseudo-labels and the true labels, denoted as . .
[0061] Figure 3 The changes in detector performance and pseudo-label quality during seven rounds of auxiliary annotation are shown. The curve starts from the second round because the first round of annotation lacks a manual judgment process; the pseudo-labels are the detector's predictions on the extended set. It can be observed that the proposed method effectively improves both detector performance and pseudo-label quality. As the annotation rounds progress, the detector's performance on the extended set steadily improves, but remains relatively low overall. Benefiting from the annotation correction capability provided by manual judgment, the quality of pseudo-labels corrected using lower-performance detection can significantly exceed their original quality (a difference of 6% between detector performance and label quality on the extended set across multiple annotation rounds). (Approximately). The improved extended set of annotations is then used for detector training in the next annotation round, further enhancing the detector's performance. Through this cycle, only manual intervention in the discrimination part is required to achieve synergistic improvement between the two.
[0062] at the same time, Figure 3 The trend indicates that the proposed method has not yet reached its full potential, and the detector performance and annotation quality are expected to improve further with each annotation round. However, it is also noted that the detector's performance on the test set shows a saturation trend with each annotation round, and the performance improvement brought by each retraining significantly decreases.
[0063] Figure 4 The document showcases some image examples from the extended set, which visually demonstrate the improvement in pseudo-annotation quality. Figure 4 In (b), the second round of pseudo-labeling omitted the right-angle hanging plate above the insulator, while the seventh round of labeling has added it. Figure 4 In (a), the second round of pseudo-annotation incorrectly identified the triangular bracket in the center of the figure as an adjustable bracket, and the triangular bracket in the upper left corner of the figure was also not detected. In the seventh round of annotation, the detection box of the triangular bracket was also corrected and supplemented, making it closer to the true annotation.
[0064] 5. Labor Cost Analysis This experiment was used to calculate the manual cost of 7 rounds of auxiliary annotation and compare it with traditional fully manual annotation. Based on existing research, annotating one detection box takes an average of 10 seconds, and verification takes 5 seconds, which is basically consistent with the statistical results in this experiment. Therefore, the time required to annotate the entire extended set of 23,164 targets is 23,164 × 15 = 347,460 seconds. In most scenarios, the proposed method only requires manual judgment on whether to accept modifications; manual annotation is only needed in a small number of cases where both the original and predicted annotations are incorrect. Moreover, the judgment process is performed on a single target sub-graph, and multiple sub-graphs can be judged simultaneously using a simple file preview function. Statistically, the time to judge a single sub-graph is approximately 1 second.
[0065] Table 2 shows the number of sub-images requiring manual judgment in the 7-round auxiliary annotation process (the first round does not involve manual judgment). The total number of sub-images requiring judgment across the four types is 13,889, corresponding to 13,889 seconds of manual time. Considering that manual annotation or adjustments were also used during the auxiliary annotation process, taking approximately 12,000 seconds, the final total time is approximately 25,889 seconds. This time is approximately 7.451% of the total annotation time (25,889 / 347,460), demonstrating that the proposed method significantly reduces the cost of manual annotation.
[0066] If all data is manually labeled, the proposed auxiliary labeling method can achieve 76% of its target. If at least 4000 images need to be labeled in the extended set, the manual annotation time would be 167936 seconds. The proposed method still only takes 25889 / 167936 = 15.416% of the time required for full annotation. The above results show that based on partially labeled images, a large number of unlabeled images, and the auxiliary annotation method proposed in this invention, the cost of traditional full manual annotation can be significantly reduced.
[0067] Table 2. Number of manual judgments for each annotation round. Verification using actual inspection images demonstrates that the model-assisted semi-automatic power line inspection target detection and annotation method of this invention significantly outperforms traditional fully manual annotation methods in terms of labor cost control, annotation quality improvement, and detector performance synergistic optimization, thus validating the effectiveness and practicality of the technical solution. This method can be widely applied to highly specialized vertical fields with high annotation thresholds, such as power transmission and distribution inspection, as well as various general target detection data annotation scenarios, providing a complete, low-cost, and highly reliable technical solution for the efficient acquisition of high-quality labeled data in deep learning tasks.
[0068] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.
Claims
1. A model-assisted semi-automatic power line inspection target detection and annotation method, characterized in that, include: S1. Acquire power inspection images containing power transmission and distribution equipment, and label the power equipment targets and defect targets in the images to obtain an initial reference image set and an initial reference label set; further update the detector based on the initial reference label set and the initial reference image set; the label includes the index of the image to which the target belongs, the coordinates of the target detection box, and the category to which the target belongs; S2, Collect power inspection images containing power transmission and distribution equipment to be labeled, and form an image set to be labeled; The set of images to be labeled is input into the detector to obtain the basic label set; S3, the detector is trained based on the basic annotation set, the image set to be annotated, the reference image set and the reference annotation set to obtain the trained detector; The set of images to be labeled is input into the trained detector to obtain the predicted label set; S4, taking the power inspection images to be labeled as units, based on the intersection-union ratio calculation results of the detection boxes of each label in the basic label set and each label in the prediction label set and the category comparison results, the corresponding labels in the prediction label set and the basic label set are assigned to the matching label set or the non-matching label set. S5, sequentially determine whether each label in the mismatched label set is adopted. If the result is adopted, update the basic label set according to the adopted labels; otherwise, do not update the basic label set. Finally, use the updated basic label set as the optimized pseudo label set. S6 uses the optimized pseudo-label set as the new base label set, and repeats S3-S6 until the proportion of unadopted labels in this round of iteration exceeds the threshold. At this point, the loop ends and the optimized pseudo-label set is output, which is the final labeling result of the image to be labeled.
2. The model-assisted semi-automatic power line inspection target detection and labeling method according to claim 1, characterized in that, When looping through S3-S6, the detector in S3 is the trained detector obtained from the previous loop.
3. The model-assisted semi-automatic power line inspection target detection and annotation method according to claim 1, characterized in that, In step S4, the condition for determining that a label belongs to the set of matching labels is: ； in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. To match the labeled set, For intersection, union, and comparison, As constraints, These represent the image index, bounding box coordinates, and target category of target j, respectively. To set the maximum threshold for intersection-union ratio, These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing existence and "and" respectively.
4. The model-assisted semi-automatic power line inspection target detection and annotation method according to claim 1, characterized in that, In S4, the mismatched annotation set includes four sub-annotation sets: new target, category adjustment, deleted target, and border adjustment.
5. The model-assisted semi-automatic power line inspection target detection and labeling method according to claim 4, characterized in that, The condition for determining whether an annotation belongs to the newly added target sub-annotation set is: ； in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. To add a new target sub-label set, For intersection, union, and comparison, As constraints, Here, represents the image index and the bounding box coordinates of target j, respectively. To minimize the crossover-union ratio (CUI), These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing "any" and "and" respectively.
6. The model-assisted semi-automatic power line inspection target detection and annotation method according to claim 4, characterized in that, The condition for determining whether an annotation belongs to the category adjustment sub-annotation set is: ； in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. Adjust the sub-label set for the category. For intersection, union, and comparison, As constraints, These represent the image index, bounding box coordinates, and target category of target j, respectively. To set the maximum threshold for intersection-union ratio, These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing existence and "and" respectively.
7. The model-assisted semi-automatic power line inspection target detection and annotation method according to claim 4, characterized in that, The condition for determining that an annotation belongs to the set of sub-annotations of the target to be deleted is: ； in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. To delete the target sub-label set, For intersection, union, and comparison, As constraints, Here, represents the image index and the bounding box coordinates of target j, respectively. To minimize the crossover-union ratio (CUI), These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing "any" and "and" respectively.
8. The model-assisted semi-automatic power line inspection target detection and annotation method according to claim 4, characterized in that, The condition for determining that an annotation belongs to the set of border adjustment sub-annotations is: ； in, This represents the label of target j in the prediction label set. This represents the annotation of target k in the basic annotation set. Adjust the sub-label set for the border. For intersection, union, and comparison, As constraints, These represent the image index, bounding box coordinates, and target category of target j, respectively. To set the maximum threshold for intersection-union ratio, To minimize the crossover-union ratio (CUI), These are the prediction annotation set and the basic annotation set, respectively. Both are logical determiners, representing existence and "and" respectively.
9. The model-assisted semi-automatic power line inspection target detection and labeling method according to claim 1, characterized in that, When the next batch of images to be labeled is acquired, the detector in S2 is the detector that has been trained and obtained when the loop stopped last time. The reference image set in S3 contains the initial reference image set and all historical images to be labeled. The reference annotation set contains the final annotation results of the initial reference annotation set and all historical images to be labeled.