Insulator RTV Coating Identification Method and Device Based on Intrinsic Image Decomposition

Abstract

The present invention proposes a method and device for identifying the RTV coating of insulators based on intrinsic image decomposition. Generally, based on the images of insulator strings captured by drone inspections, aiming at the small inter-class differences of insulator RTV, with only subtle color and gloss differences, the direct classification effect is not good. It is proposed to use the reflectance image obtained by intrinsic image decomposition as the input of the classifier, which improves the classification accuracy. On this basis, an ensemble learner composed of three classifiers, namely ResNeSt101, ResNet101, and ViT-Large, is adopted to further improve the classification performance. Aiming at the characteristics of refined inspection operations of drone towers, the insulator string sample annotation method proposed in the present invention, combined with the minimum center distance screening method, can accurately extract a single-target insulator string at the shooting point. Different from the traditional object detection that requires detecting all instances as much as possible, it can be further applied to similar scenarios such as inspection defect identification.

Description

Technical Field

[0001] The present invention relates to the technical fields of power facility inspection, image recognition, insulator detection, etc., and particularly relates to a method and a system device for identifying the RTV coating of insulators based on intrinsic image decomposition. Background Art

[0002] With the rapid development of social economy, the number of pollution sources in various places is increasing continuously. Some insulators in polluted areas need to be sprayed with RTV to improve insulation performance. In recent years, pollution flashover accidents have occurred from time to time due to the unsprayed RTV anti-pollution flashover coating on transmission line insulators, and the pollution flashover threatens the safe and stable operation of transmission lines. At the same time, due to the change of line equipment, the situation that the insulator account is inconsistent with the actual situation is widespread, and it is necessary to spend a lot of manpower to manage the account data.

[0003] In recent years, the inspection by unmanned aerial vehicle (UAV) has been widely carried out in power grid companies and has become an important part of the three-dimensional inspection system, and a large amount of inspection image data has been accumulated. These data can be used to check for missed spraying of the RTV coating on insulators. However, the manual inspection is inefficient, error-prone, and the workload is huge. Therefore, it is necessary to consider using intelligent methods such as machine learning to identify whether the insulator is sprayed with the RTV coating, automatically conduct intelligent inspection on a large amount of inspection image data, explore the value of refined inspection data of UAV tower poles, and improve the data management efficiency.

[0004] In the scenario of refined inspection of UAV tower poles, the pictures at one shooting point often contain multiple strings of insulators. It is difficult to identify and locate the string of insulators at the corresponding installation position. Due to reasons such as camera shooting color difference and improper exposure, there are often only subtle color and luster differences between the insulators with and without the RTV coating, and it is difficult to accurately classify them at a fine-grained level.

[0005] In the existing technologies, there is no fast, effective and accurate identification method for the RTV coating. For example, in the solution of the method and device for detecting the peeling area of the RTV coating of insulators in Chinese Patent CN201510800260, it relies on manual coloring of the defective parts of the RTV coating, and then rubbing and collecting images, which is obviously essentially different from the shooting method of UAV inspection; in the solution of a method for identifying the defects of the RTV coating of insulators in CN201911226484, it can only process the image of a single insulator, and distinguish the RTV coating with obvious gray-scale difference and its damaged parts through the method of threshold decomposition, but obviously it cannot extract the insulator string and identify and judge the insulators with and without the RTV coating. Summary of the Invention

[0006] In view of the defects and deficiencies existing in the prior art, the present invention proposes a method and device for identifying the RTV coating of insulators based on intrinsic image decomposition, which is used to solve the problem of checking whether the RTV coating of insulators should be applied but has not been applied. Generally, based on the inspection of insulators by drones, due to the small inter-class differences of the RTV of insulators, there are only subtle color and gloss differences, and the direct classification effect is not good. It is proposed to use the reflectance image obtained by intrinsic image decomposition as the input of the classifier, which improves the classification accuracy. On this basis, an ensemble learner composed of three classifiers, namely ResNeSt101, ResNet101 and ViT-Large, is adopted to further improve the classification performance. In view of the characteristics of the refined inspection operation of the drone tower, the insulator string sample annotation method proposed by the present invention combined with the minimum center distance screening method can accurately extract a single target insulator string at the shooting point. Different from the traditional object detection that requires detecting all instances as much as possible, it can be further applied to similar scenarios such as inspection defect recognition.

[0007] The present invention specifically adopts the following technical solutions:

[0008] A method for identifying the RTV coating of insulators based on intrinsic image decomposition, characterized by including the following steps:

[0009] Adopt the method of object detection to extract candidate insulator strings; and extract the target insulator string corresponding to the picture at the shooting point of the candidate insulator string; then expand the boundary of the target insulator string, and cut out the local image corresponding to the target insulator string at the corresponding position on the original picture; perform intrinsic image decomposition on the local image; and use the reflectance image obtained by the intrinsic image decomposition as the input to perform fine-grained classification of the insulator RTV to determine whether the insulator is coated with RTV.

[0010] Among them, the object extracted by the object detection method, that is, the original picture is generally the inspection image of the insulator string by the drone. Of course, it can also be applied to other types of shooting images with insulator strings.

[0011] Furthermore, the training sample annotation method for object detection is: a group of insulators in a string shape is used as a target of an insulator string; the entire group of insulators of the jumper wire is used as a target of an insulator string; only the target insulator string corresponding to the shooting point is marked, and other background insulator strings are not marked.

[0012] Further, the YOLOv5-l model is used for insulator target detection, and the YOLOv5-l is optimized as follows: the anchor sizes are recalculated, and the optimized sizes are [110, 410, 117, 494, 157, 297], [151, 588, 516, 141, 821, 200], [485, 467, 665, 274, 839, 375]; the img-size parameter of the model is set to 1024; momentum is set to 0.937, weight_decay is set to 0.0005, giou is set to 0.05, cls is set to 0.5, cls_pw is set to 1.0, obj is set to 1.0, obj_pw is set to 3.0, iou_t is set to 0.2, anchor_t is set to 5.3, hsv_h is set to 0.015, hsv_s is set to 0.75, hsv_v is set to 0.45, degrees is set to 10.0, scale is set to 0.5, perspective is set to 0.0005, fliplr is set to 0.5.

[0013] Further, the minimum center distance method is used to screen out the target insulator string from the candidate insulator strings, and the target insulator string corresponding to a string of photographed point pictures is extracted.

[0014] Further, the center distance calculation formula is:

[0015]

[0016] In the formula, (x, y) is the normalized center point coordinates of the candidate insulator string.

[0017] Further, before splitting the target insulator string, the boundary is expanded, and both the length and width are expanded to 1.1 times that of the target insulator string;

[0018] The expansion method is: expand in both the length and width directions; if the margin space of the original image is sufficient, expand 0.05 times towards the boundary, and if the margin space of the original image is insufficient, only expand to the maximum boundary of the original image.

[0019] Further, the USI 3 D network is used to perform intrinsic image decomposition on the local pictures of the target insulator string to obtain the reflectance image; and the USI 3 D is optimized as follows: λ 1 is set to 15.0, λ 2 is set to 0.15, λ 3 is set to 15, λ 4 is set to 0.15, λ 5 is set to 5.0.

[0020] Further, taking the reflectance image obtained by intrinsic image decomposition as the input, it is independently classified by three classifiers, namely ResNeXt101, ResNet101 and ViT-Large, and then the integrated learner finally determines whether the insulator is coated with RTV.

[0021] Further, the optimization of ResNeXt101 is as follows: groups is set to 32, width_per_group is set to 8, and img_size is set to 512;

[0022] The optimization of ResNet101 is as follows: the loss function uses LabelSmoothLoss; label_smooth_val is set to 0.15, loss_weight is set to 1.0; img_size is set to 512;

[0023] The optimization of ViT-Large is as follows: patch_size is set to 32, and img_size is set to 512;

[0024] The integrated learner adopts the equal-weight voting method, and the weights of the three classifiers ResNeXt101, ResNet101 and ViT-Large are all 1, and the one with the most votes is the final result.

[0025] And, an insulator RTV coating recognition device based on intrinsic image decomposition, which is characterized by including: a memory, a processor, and a computer program stored on the memory and executable on the processor;

[0026] The computer program includes: an insulator target detection module, a target insulator string discrimination module, an insulator string local cropping module, an intrinsic image decomposition module, and an RTV fine-grained classification module;

[0027] The insulator target detection module uses the YOLOv5-l model to detect insulator targets, and is used to extract all candidate insulator strings in the captured image containing the complete insulator string;

[0028] The target insulator string discrimination module uses the minimum center distance method to screen out the target insulator string from the candidate insulator strings, and is used to further screen the candidate insulator strings and extract a string of target insulator strings corresponding to the captured image;

[0029] The insulator string local cropping module is used to expand the boundary of the target insulator string and crop the local image corresponding to the position of the target insulator string on the original image;

[0030] The intrinsic image decomposition module uses USI 3The D network performs intrinsic image decomposition on the local image of the target insulator string, which is used to perform intrinsic image decomposition on the local image of the target insulator string;

[0031] The RTV fine-grained classification module includes an ensemble learner composed of three classifiers: ResNeSt101, ResNet101, and ViT-Large. It uses the reflectance image obtained from intrinsic image decomposition as input to perform fine-grained classification of insulator RTV and determine whether the insulator is coated with RTV.

[0032] In addition, an electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor, when executing the program, implements the steps of the insulator RTV coating recognition method as described above.

[0033] In addition, a non-transitory computer-readable storage medium stores a computer program. The computer program, when executed by a processor, implements the steps of the insulator RTV coating recognition method as described above.

[0034] The present invention and its preferred solutions have the following advantages or beneficial effects:

[0035] Aiming at the problem that there is a small inter-class difference in insulator RTV, only subtle color and gloss differences, and the direct classification effect is not good, the present invention proposes to use the reflectance image of intrinsic image decomposition as the input of the classifier, effectively improving the classification accuracy; on this basis, an ensemble learner composed of three classifiers: ResNeSt101, ResNet101, and ViT-Large is adopted to further improve the classification performance. Aiming at the characteristics of refined operation of unmanned aerial vehicle (UAV) on power towers, the insulator string sample annotation method proposed by the present invention combined with the minimum center distance screening can accurately extract a single target insulator string at the shooting point. Different from the traditional object detection requirements to detect all instances as much as possible, it can be further applied to similar scenarios such as inspection defect recognition. Description of the Drawings

[0036] The following further details the present invention in conjunction with the drawings and specific embodiments:

[0037] Figure 1 It is a schematic flowchart of the insulator RTV coating recognition method according to an embodiment of the present invention. Detailed Embodiments

[0038] In the following, specific embodiments of the present application will be described in detail with reference to the accompanying drawings. According to these detailed descriptions, those skilled in the art can clearly understand the present application and can implement the present application. Without departing from the principle of the present application, the features in various different embodiments can be combined to obtain new implementation manners, or some features in certain embodiments can be replaced to obtain other preferred implementation manners.

[0039] To make the features and advantages of this patent more obvious and understandable, specific embodiments are given below and detailed descriptions are provided in conjunction with the accompanying drawings as follows:

[0040] As Figure 1 shown, this embodiment provides a specific solution for an intelligent recognition method of insulator RTV coating based on intrinsic image decomposition, including the following steps:

[0041] (1) Extract all candidate insulator strings in the picture at the shooting point;

[0042] (2) Further screen the candidate insulator strings to extract a string of target insulator strings corresponding to the picture at the shooting point;

[0043] (3) Expand the boundary of the target insulator string and cut out the local image corresponding to the position of the target insulator string on the original image;

[0044] (4) Perform intrinsic image decomposition on the local image of the target insulator string;

[0045] (5) Use the reflectance image obtained by intrinsic image decomposition as the input to perform fine-grained classification of insulator RTV to determine whether the insulator is coated with RTV.

[0046] It should be noted that the above step flow only represents the specific steps adopted during the implementation of this embodiment, and its sequence does not limit the technical features of the solution of the present invention.

[0047] In step 1, an image containing a complete insulator string taken at close range by a collection tool such as a drone is used as the input, and the YOLOv5-l model is used for insulator target detection.

[0048] The optimizations made to the YOLOv5-l method in the embodiments of the present invention are as follows: Recalculate the anchor sizes, and the optimized sizes are [110, 410, 117, 494, 157, 297], [151, 588, 516, 141, 821, 200], [485, 467, 665, 274, 839, 375]; Set the img-size parameter of the model to 1024; And optimize a series of hyperparameters, set momentum to 0.937, weight_decay to 0.0005, giou to 0.05, cls to 0.5, cls_pw to 1.0, obj to 1.0, obj_pw to 3.0, iou_t to 0.2, anchor_t to 5.3, hsv_h to 0.015, hsv_s to 0.75, hsv_v to 0.45, degrees to 10.0, scale to 0.5, perspective to 0.0005, fliplr to 0.5. The above parameter settings effectively improve the accuracy of insulator target detection and can be directly applied to the massive image data obtained from the current UAV inspection of transmission lines, achieving good results.

[0049] In the embodiments of the present invention, the method for annotating target detection training samples is as follows: Group a series of insulators in the form of "V", "I", "II", double "II", "V+I", etc. as an insulator string target; The entire set of insulators of the jumper is used as an insulator string target; Only annotate the target insulator string corresponding to the shooting point, and do not annotate other background insulator strings.

[0050] In step 2, the minimum center distance is used to screen out the target insulator string from the candidate insulator strings, and the target insulator string corresponding to a string of shooting point pictures is extracted.

[0051] Furthermore, the center distance calculation formula is:

[0052]

[0053] In the formula, (x, y) is the normalized center point coordinates of the candidate insulator string.

[0054] In step 3, before splitting the target insulator string, expand the boundary, and both the length and width are expanded to 1.1 times that of the target insulator string.

[0055] Furthermore, the specific expansion method is: Expand in both the length and width directions respectively; If the margin space of the original image is sufficient, expand 0.05 times towards the boundary, and if the margin space of the original image is insufficient, only expand to the maximum boundary of the original image.

[0056] In step 4, use USI 3The D network performs intrinsic image decomposition on the local image of the target insulator string to obtain the reflectance image.

[0057] In the embodiment of the present invention, the optimization of USI 3 The optimization of D is: λ 1 is set to 15.0, λ 2 is set to 0.15, λ 3 is set to 15, λ 4 is set to 0.15, λ 5 is set to 5.0. This parameter tuning makes the effect of intrinsic image decomposition better.

[0058] In step 5, taking the reflectance image obtained by intrinsic image decomposition as the input, independent classification is performed by three classifiers, namely ResNeXt101, ResNet101, and ViT-Large, and then the integrated learner finally judges whether the insulator is coated with RTV.

[0059] In the embodiment of the present invention, the optimization of ResNeXt101 is: groups is set to 32, width_per_group is set to 8, and img_size is set to 512.

[0060] In the embodiment of the present invention, the optimization of ResNet101 is: the loss function adopts LabelSmoothLoss;, label_smooth_val is set to 0.15, loss_weight is set to 1.0; img_size is set to 512.

[0061] In the embodiment of the present invention, the optimization of ViT-Large is: patch_size is set to 32, and img_size is set to 512.

[0062] In the embodiment of the present invention, the integrated learner adopts the equal-weight voting method, and the weights of the three classifiers, ResNeXt101, ResNet101, and ViT-Large, are all 1. According to the principle of "the minority obeys the majority", the one with more votes is the final result.

[0063] The above method provided in this embodiment can be stored in a computer-readable storage medium in a coded form, implemented in the form of a computer program, input the basic parameter information required for calculation through computer hardware, and output the calculation result.

[0064] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, apparatus, or computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0065] The present invention is described with reference to the flowcharts of methods, devices (apparatuses), and computer program products according to the embodiments of the present invention. It should be understood that each process in the flowchart and the combination of processes in the flowchart can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in one Figure 1 process or multiple processes.

[0066] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in one Figure 1 process or multiple flowcharts.

[0067] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 process or multiple processes.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: still can modify the specific implementation manners of the present invention or make equivalent replacements, and any modification or equivalent replacement without departing from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

[0069] This patent is not limited to the above best implementation manner. Anyone can obtain other various forms of insulator RTV coating recognition methods and devices based on intrinsic image decomposition under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention shall fall within the coverage of this patent.

Claims

1. An insulator RTV coating identification method, characterized in that, it includes the following steps: Extract candidate insulator strings by using the method of object detection; and extract the target insulator string corresponding to the picture of the shooting point of the candidate insulator string; then expand the boundary of the target insulator string, and cut out the local image corresponding to the position of the target insulator string on the original picture; perform intrinsic image decomposition on the local image; and use the reflectance image obtained by the intrinsic image decomposition as the input to perform fine-grained classification of insulator RTV to determine whether the insulator is coated with RTV; Adopt USI 3 Perform intrinsic image decomposition on the local image of the target insulator string using the D network to obtain the reflectance image; and optimize the USI 3 D as follows: λ 1 Set to 15.0, λ 2 Set to 0.15, λ 3 Set to 15, λ 4 Set to 0.15, λ 5 Set to 5.0; Use the reflectance image obtained by intrinsic image decomposition as the input, and perform independent classification by three classifiers of ResNeXt101, ResNet101 and ViT-Large, and then the integrated learner finally judges whether the insulator is coated with RTV; Optimize ResNeXt101 as follows: set groups to 32, width_per_group to 8, and img_size to 512; Optimize ResNet101 as follows: use LabelSmoothLoss as the loss function; set label_smooth_val to 0.15, loss_weight to 1.0; img_size to 512; Optimize ViT-Large as follows: set patch_size to 32, img_size to 512; The integrated learner adopts the equal-weight voting method, and the weights of the three classifiers of ResNeXt101, ResNet101 and ViT-Large are all 1, and the one with more votes is the final result.

2. The insulator RTV coating identification method according to claim 1, characterized in that: The training sample annotation method for the object detection is: a group of insulators of various string types are only used as one insulator string target; the jumper whole-group insulators are used as one insulator string target; only the target insulator string corresponding to the shooting point is annotated, and other background insulator strings are not annotated.

3. The insulator RTV coating identification method according to claim 2, characterized in that: The YOLOv5-l model is used for insulator target detection, and the YOLOv5-l is optimized as follows: the anchor sizes are recalculated, and the optimized sizes are [110, 410, 117, 494, 157, 297], [151, 588, 516, 141, 821, 200], [485, 467, 665, 274, 839, 375]; the img-size parameter of the model is set to 1024; momentum is set to 0.937, weight_decay is set to 0.0005, giou is set to 0.05, cls is set to 0.5, cls_pw is set to 1.0, obj is set to 1.0, obj_pw is set to 3.0, iou_t is set to 0.2, anchor_t is set to 5.3, hsv_h is set to 0.015, hsv_s is set to 0.75, hsv_v is set to 0.45, degrees is set to 10.0, scale is set to 0.5, perspective is set to 0.0005, and fliplr is set to 0.

5.

4. The insulator RTV coating recognition method according to claim 1, characterized in that: The minimum center distance method is used to screen out the target insulator string from the candidate insulator strings, and a string of target insulator strings corresponding to the photographed point pictures is extracted.

5. The insulator RTV coating recognition method according to claim 4, characterized in that: The center distance calculation formula is: Where (x, y) are the normalized central point coordinates of the candidate insulator string.

6. The insulator RTV coating recognition method according to claim 1, characterized in that: Before cutting the target insulator string, the boundary is expanded, and both the length and width are expanded to 1.1 times that of the target insulator string; The expansion method is: expand in both the length and width directions respectively; if the margin space of the original image is sufficient, expand 0.05 times towards the boundary, and if the margin space of the original image is insufficient, only expand to the maximum boundary of the original image.

7. An insulator RTV coating recognition device based on intrinsic image decomposition, characterized in that, The method according to claim 1 is used for insulator RTV coating recognition, including: a memory, a processor, and a computer program stored on the memory and executable on the processor; The computer program includes: an insulator target detection module, a target insulator string discrimination module, an insulator string local cutting module, an intrinsic image decomposition module, and an RTV fine-grained classification module; The insulator target detection module uses the YOLOv5-l model for insulator target detection, and is used to extract all candidate insulator strings in the photographed point pictures containing complete insulator strings; The target insulator string discrimination module uses the minimum center distance method to screen out the target insulator string from the candidate insulator strings, and is used to further screen the candidate insulator strings and extract a string of target insulator strings corresponding to the photographed point pictures; The insulator string local cutting module is used to expand the boundary of the target insulator string and cut out the local image corresponding to the position of the target insulator string on the original image; The intrinsic image decomposition module adopts USI 3 The D network performs intrinsic image decomposition on the local picture of the target insulator string; The RTV fine-grained classification module includes an ensemble learner composed of three classifiers: ResNeSt101, ResNet101, and ViT-Large. Using the reflectance image obtained by intrinsic image decomposition as the input, it performs fine-grained classification of insulator RTV to determine whether the insulator is coated with RTV.

Images