Collaborative analysis method of power insulator strings based on multi-task convolutional neural network

Abstract

The present invention relates to a collaborative analysis method for power insulator strings based on a multi-task convolutional neural network, including: image acquisition, collecting images of insulator strings under different lighting and background conditions; preprocessing; insulator string contour annotation, annotating the image along the edge of the insulator string; deep neural network model construction, constructing a multi-task convolutional neural network model, which has the functions of target recognition and target segmentation; model training, importing the data in S2 into the model constructed by S5, and performing error calculation with the annotated true values ​​of S3 and S4, and feedback learning; using a verification set that has also been annotated with a target frame, direction, and contour for model verification. The present invention can realize the introduction of computer vision / deep learning technology into the field of power transmission line inspection, replace people to complete the analysis of massive images, and at the same time, in order to provide efficiency and accuracy, realize the recognition and analysis of all attributes at one time.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision, deep learning, and power inspection, and specifically to a collaborative analysis method for power insulator strings based on a multi-task convolutional neural network, which is used for locating and segmenting insulator strings during power inspection. Background Art

[0002] my country's large population dictates a significant electricity demand, while its vast territory necessitates a wide range of electricity consumption. High-voltage and ultra-high-voltage transmission enables long-distance, high-capacity, and low-loss power transmission. By the end of 2021, State Grid Corporation of China had completed 15 AC and 13 DC transmission projects, representing a world-leading total of 230 million kilowatts of interprovincial and interregional transmission capacity.

[0003] To ensure stable electricity supply for both residential and industrial use, regular inspections of the extensive power transmission network are necessary. State Grid's requirements necessitate meticulous inspections of every tower in the high- and ultra-high-voltage transmission network.

[0004] Insulator strings are key components on transmission towers and a crucial component of transmission line inspections. As the name suggests, insulator strings insulate transmission lines from the towers, ensuring safe transmission. Defect detection in insulator strings primarily focuses on the insulator string's tilt angle, the condition of the mounting points at each end, and the integrity of the insulator string's discs. Current inspection requirements generate 150 million to 300 million images containing insulator strings for each complete transmission line inspection. Screening this number of images requires a significant investment of manpower and resources.

[0005] With the development of artificial intelligence (AI), computer vision and deep learning-based techniques for identifying and analyzing objects of interest in photos have grown at an astonishing rate, replacing a significant amount of repetitive manual labor. Currently, image recognition focuses solely on identifying and analyzing single attributes. Multiple algorithms must be run in parallel or in cascade to fully identify and analyze all attributes. To introduce computer vision and deep learning technologies into the field of power transmission line inspection, replacing the manual analysis of massive amounts of images while simultaneously achieving efficient and accurate identification and analysis of all attributes, this patent proposes a collaborative insulator string analysis method based on a multi-task convolutional neural network. Summary of the Invention

[0006] The present invention aims to provide a collaborative analysis method for power insulator strings based on a multi-task convolutional neural network, which is characterized by comprising the following steps:

[0007] S1. Image acquisition: collecting images of insulator strings under different lighting and background conditions;

[0008] S1.1 Use drones and visible light payloads to collect data at different locations;

[0009] S1.2 Use a drone with a visible light payload to collect images under different lighting conditions;

[0010] S1.3 uses a drone with a visible light payload to collect images at different distances from the insulator string. The distance is generally set within a range of 2 meters to 35 meters from the insulator string.

[0011] S2. Preprocessing, using three schemes to enrich the dataset;

[0012] S2.1. Perform pixel-level grayscale operations on the RGB channels of the acquired image.

[0013] S2.2. Cropping the original image by randomly selecting the X and Y coordinates of the upper left corner of the new image and the width and height of the new image, and converting the new coordinates based on the cropped offset.

[0014] S2.3. Mosaic the images obtained in S2.1 and S2.2.

[0015] S3. Label the target box and its direction in the image, using the long edge representation method to label the image;

[0016] S4. Annotate the insulator string outline: annotate the image along the edge of the insulator string;

[0017] S5. Deep neural network model construction: Build a multi-task convolutional neural network model with both object recognition and object segmentation capabilities;

[0018] The segmentation branch of the S5.1 model adopts a U-row structure, first downsampling and then upsampling;

[0019] The target recognition branch of the S5.2 model is connected in parallel with the PAFPN structure in the first half of the U-shaped structure (downsampling stage) for rotating target positioning;

[0020] S6. Model training: import the data in S2 into the model built in S5, and calculate the error with the annotated true values ​​in S3 and S4, and feedback learning;

[0021] S7 uses the same validation set with target box, direction and contour annotations for model verification;

[0022] Furthermore, in the above steps S1.1, S1.2, and S1.3, the captured images cover multiple resolution ranges, including but not limited to 640*480, 1280*720, 1024*768, 1920*1080, 2560*1440, 3840*2160, 7680*4320, etc.;

[0023] Furthermore, in S2.1, the grayscale operation of the three RGB channels of the image is performed by using random channel inversion, random parameter Gaussian blur, and 4-neighborhood or 8-neighborhood random Laplace sharpening; the Gamma transform with parameter values ​​between [0,1] is performed, and the operation formula is:

[0024] S=C*R γ

[0025] Where S is the transformed pixel value, C is the Gamma transformation scaling coefficient, which is 1, R is the input pixel, and γ is the transformation parameter, which is [0,1].

[0026] Furthermore, in S2.2, the X and Y coordinates of the upper left corner of the selected new image satisfy the following conditions: X<W original / 2,Y<H original / 2, where W original ,H original The width and height of the original image are W and H, and the width and height of the selected new image satisfy the following conditions X+W<W original ,Y+H<H original , where X, Y are the upper left corner X, Y, and W of the new image selected in 6 original ,H original The width and height of the original image

[0027] Furthermore, in S2.3, two-image mosaic, three-image mosaic, and four-image mosaic stitching methods are randomly adopted;

[0028] Furthermore, in S3, the annotation box is a rectangle. The selection of the annotation box follows the direction of the long side of the insulator string. The image is annotated using the DOTA method. The annotation format is [x1, y1, x2, y2, x3, y3, x4, y4, ClassName, Diffcult], where x n ,y nIdentify the four vertices of the target in sequence. Generally, the upper left corner is taken as the starting coordinate angle, ClassName is the target category, and Diffcult indicates the difficulty of identifying the target. Then convert the DOTA annotation format into a rotation angle identification format according to the long side identification. The format is [class_id,x_center,y_center,long_side,short_side,angle], where class_id identifies the target category index, x_center and y_center are the coordinates of the center point of the target frame, long_side and short_side are the lengths of the long and short sides of the target frame respectively, and angle is the rotation angle of the surface target frame. The angle is expressed by the long side and ranges from 0 to 179 degrees. The positive direction of the X axis is 0 degrees, and the direction close to the negative direction of the X axis is 179 (the negative direction of the X axis is also 0 degrees).

[0029] Furthermore, in S4, software such as Labelme is used to mark the target edge. The marking content includes not only the insulator string itself, but also the hanging points at both ends of the insulator string, including the triangle plate, retractable connecting rod, etc.

[0030] Furthermore, the multi-task network designed by S5 is a deep neural network that simultaneously performs rotation target recognition and target segmentation. It contains two branches: the rotation target recognition branch and the target segmentation branch.

[0031] Furthermore, in S5.1, the target segmentation branch in the multi-task convolutional neural network adopts a U-shaped structure, which adopts the form of downsampling followed by upsampling. The branch ultimately outputs a feature map of the same size as the input image. The U-shaped segmentation network consists of the following parts:

[0032] (1) The training input image is first downsampled by 4-level dilated convolution;

[0033] (2) After 4 levels of deconvolution, a new upsampling is performed to obtain a feature map with the same size as the input training image;

[0034] (3) There is a skip layer between downsampling and upsampling to fuse high-level and low-level features;

[0035] Furthermore, the target segmentation branch in the multi-task convolutional neural network performs a Softmax operation on the channel axis at the final output truncation. The specific formula is as follows:

[0036]

[0037] Furthermore, in S5.1, the target segmentation branch in the multi-task convolutional neural network adopts a weighted two-class cross entropy loss, and the specific formula is as follows:

[0038]

[0039] Furthermore, in S5.2, the rotation target recognition branch in the multi-task convolutional neural network is connected in parallel with the PAFPN structure in the first half of the U-shaped structure of the target segmentation branch (downsampling stage). After a certain amount of convolutional neural network extraction, the output content is the target category, center point coordinates and long side rotation angle. The loss calculation method of the rotation target recognition branch in the multi-task convolutional neural network combines the mean square error loss and cross entropy loss.

[0040] Furthermore, the multi-task convolutional neural network training method designed by S5 adopts the method of alternating training and then joint training. That is, the target detection branch is trained first, and after several epochs, the target segmentation branch is trained in the form of fine-tuning. This is repeated for several cycles and then joint training is performed.

[0041] Furthermore, the error calculation method in S6 refers to that described in 14 and 16, and the feedback learning method adopts the error back propagation learning mechanism;

[0042] Furthermore, S7 uses labeled data that has not participated in training to evaluate the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the collaborative analysis method for power insulator strings based on multi-task convolutional neural networks;

[0044] Figure 2 Schematic diagram of multi-scale images and images with different exposures;

[0045] Figure 3 Schematic diagram of image data preprocessing;

[0046] Figure 4 Collaboratively analyze deep neural networks for object recognition and object segmentation;

[0047] Figure 5 is the detection result of the rotating target;

[0048] Figure 6 This is the result of local insulator string segmentation. DETAILED DESCRIPTION

[0049] In order to further illustrate the collaborative analysis method of power insulator strings based on multi-task convolutional neural networks proposed in the present invention, its specific implementation methods, implementation processes, and implementation contents are further described in detail, but the specific implementation methods, processes, and contents of the present invention are not limited to these.

[0050] Reference Figure 1As shown in the figure, the main process is divided into data collection, data preprocessing, creation of a collaborative analysis neural network, model training, model evaluation, and edge deployment and operation. Labels starting with "B" represent the backbone network, and those starting with "S" and connected by dotted lines represent process decomposition branches.

[0051] Step B1: Collect insulator string data. Collect diverse insulator string data as specific samples for model training. Also collect a portion of data as samples for model evaluation.

[0052] Step B2: Data labeling. The segmentation branch uses contour labeling, and the rotation target detection branch uses rotation target box labeling.

[0053] Step B3: Data preprocessing. Data is augmented by random cropping, random inversion, and two-image or four-image augmentation. This not only increases the dataset size, but also effectively improves the robustness and generalization ability of the model.

[0054] Step B4: Create a multi-task collaborative analysis convolutional neural network. By designing a special deep neural network, it is possible to simultaneously identify the insulator string target and effectively segment the insulator string outline.

[0055] Step B5: Model training. The data collected in step B1 is expanded and enhanced through step B2, and used as training data for the convolutional neural network in step B3 for collaborative analysis, and repeated training is performed.

[0056] Step B6: Model evaluation: Use the evaluation samples collected in B1 to input into the training model obtained in B4, observe the model quality, and determine whether to add more data or more training rounds.

[0057] Step B7: Edge deployment and operation. The model trained in step B4 and evaluated as meeting the requirements in step B5 is deployed to the cloud or edge to perform specific power inspection tasks.

[0058] According to the above method flow, the collaborative analysis method of power insulator strings based on multi-task convolutional neural network is explained in more detail through the following specific embodiments.

[0059] 1. Diverse data collection

[0060] Figure 1In the B1 data collection stage, the stabilized gimbal camera is mounted on a multi-rotor drone or a fixed-wing drone, and the flight is manually controlled to collect data. Steps S11, S12, and S13 are a typical decomposition of the data collection process. By manually adjusting the resolution of the stabilized gimbal camera, pictures of various resolutions are taken. The collected images cover multiple resolution ranges, including but not limited to 640*480, 1280*720, 1024*768, 1920*1080, 2560*1440, 3840*2160, 7680*4320, etc.; in addition to adjusting the resolution and taking more coarse-grained and fine-grained images, the exposure is also adjusted as indicated in step S12. Figure 2 As shown in , by adjusting different exposure times, pictures with different brightness and darkness can be taken at the same location.

[0061] In addition to the above, it is also necessary to control the drone to shoot the insulator strings from more angles and capture more pictures with backgrounds of different complexity.

[0062] Data collection requires collecting training data sets and evaluation data sets separately, with the ratio of the two being 80%:20%.

[0063] 2. Data Labeling

[0064] exist Figure 4 Z2 rotating target recognition branch, the annotation box is a rectangle, the selection method of the annotation box follows the direction of the long side of the insulator string, the image is annotated using the DOTA method, and the annotation format is [x1, y1, x2, y2, x3, y3, x4, y4, ClassName, Diffcult], where x n ,y n Identify the four vertices of the target in sequence. Generally, the upper left corner is taken as the starting coordinate angle, ClassName is the target category, and Diffcult indicates the difficulty of identifying the target. Then convert the DOTA annotation format into a rotation angle identification format according to the long side identification. The format is [class_id,x_center,y_center,long_side,short_side,angle], where class_id identifies the target category index, x_center and y_center are the coordinates of the center point of the target frame, long_side and short_side are the lengths of the long and short sides of the target frame respectively, and angle is the rotation angle of the surface target frame. The angle is expressed by the long side and ranges from 0 to 179 degrees. The positive direction of the X axis is 0 degrees, and the direction close to the negative direction of the X axis is 179 (the negative direction of the X axis is also 0 degrees).

[0065] exist Figure 4In the Z1 branch, software such as Labelme is used to mark the target edge. The marking content includes not only the insulator string itself, but also the hanging points at both ends of the insulator string, including the triangle plate, telescopic connecting rod, etc.

[0066] 3. Enhance the image by preprocessing the data

[0067] The data preprocessing in step B3 includes, but is not limited to, random inversion shown in S31, random cropping shown in S32, and mosaic splicing shown in S33.

[0068] The random flipping in S31 includes horizontal flipping, vertical flipping and two diagonal flippings. By randomly selecting the flipping form, the image is swapped along the vertical center line, the horizontal center line or the diagonal line at the pixel level.

[0069] The random cropping described in S32 includes two methods. One method is the non-discarding method, which randomly crops a picture into two images along a certain horizontal line X1X2 or vertical line Y1Y2. The original picture can also be randomly cropped into four pictures using the horizontal center lines X1X2 and Y1Y2 at the same time. The other method is the discarding method, which is based on the above-mentioned random cropping, and crops all four sides of the image and randomly discards a part.

[0070] The mosaic stitching described in S33 is achieved by combining two or four images according to certain principles. More specifically, a canvas of random size is first created and randomly divided into four parts, called A, B, C, and D areas, each of which has a random size. Four training image data are selected, and the four images are randomly cropped and scaled according to the sizes of the four areas, and then the resulting new images are overlaid on the canvas. Figure 3 It is a typical two-image and four-image mosaic display.

[0071] The image preprocessing and enhancement are not limited to the methods indicated in S21, S22, and S23, and more forms of preprocessing can also be performed.

[0072] 4. Create a multi-task collaborative analysis convolutional neural network

[0073] Referring to step B4, the reason why the present invention is a collaborative analysis method is that it focuses on creating a convolutional neural network that can perform multiple tasks simultaneously. In particular, the convolutional neural network can simultaneously perform the target recognition, positioning, and rotation angle detection tasks of the insulator string (such as Figure 4 Z2 area), on the other hand, the convolutional neural network can also segment the insulator string through the trunk U-shaped structure (such as Figure 4 Z1 area).

[0074] like Figure 4 The figure shows a typical multi-task collaborative analysis convolutional neural network designed for this invention. The Z1 region is the insulator string segmentation network, and the Z2 region is the insulator string rotation recognition branch.

[0075] Figure 4 The multi-task convolutional neural network shown supports arbitrary resolution, but needs to meet one condition, that is, the input resolution W input ,H input It needs to be divisible by 2, 4, 8, 16, and 32 respectively to ensure consistency between downsampling and upsampling in the image segmentation stage.

[0076] The Z1 region first undergoes four downsampling operations. Each downsampling uses the ABA operation, which stands for Atrous Convolution, Batch Normalization (BN), and Activation. After each ABA operation, the size of the feature map obtained is halved compared to the previous layer, that is:

[0077] W l+1 =W l / 2,H l+1 =H l / 2

[0078] Among them, l∈[0,4], W and H represent the width and height information of the feature map respectively.

[0079] Figure 4 As shown in the Z1 area, after 4 downsampling, a high-order feature map P5 is obtained. The size of this feature map is as follows:

[0080] W P5 =W input / 32,H P5 =H input / 32

[0081] Among them, W P5 ,H P5 is the width and height of the P5 feature map, W input ,H input is the width and height of the input image.

[0082] Figure 4 In the Z1 region, the P5 feature map is followed by four upsampling operations, each of which is performed by performing a deconvolution operation (Transposed Convolution). After each upsampling, the resolution is doubled compared to the previous layer, that is:

[0083] W l+1 =W l *2,H l+1 =Hl *2

[0084] Among them, l∈[5,8], W and H represent the width and height information of the feature map respectively.

[0085] Figure 4 In the Z1 region, after P5 is upsampled 4 times, the feature map P9 is obtained, and the resolution of P9 is equivalent to the input image resolution. That is:

[0086] W P9 =W input ,H P9 =H input

[0087] Figure 4 In the Z1 area, there are jump connections between the feature maps P1 and P9, P2 and P8, P3 and P7, and P4 and P6. The ADD operation or the inter-channel Concatenate operation can be used selectively to fuse high-order and low-order features.

[0088] like Figure 4 Region Z2 is the rotation object detection branch.

[0089] 5. Model training phase

[0090] Figure 1 The model training of part B5 adopts the method of alternating training in stages and then merging training. First, the target detection branch is trained ( Figure 4 In the Z2 phase, after 300 epochs of training, the target segmentation branch is trained again, also for 300 epochs. Then, the Z1 and Z2 branches are trained continuously, also for 300 epochs.

[0091] During training, Figure 4 The target segmentation branch identified in the Z1 stage uses the second-class cross entropy loss for feedback adjustment, as follows:

[0092]

[0093] Figure 4 The rotation target position and rotation angle of the rotation target detection branch identified in the Z2 stage use regression loss, and the category determination uses binary cross entropy loss, and the two are feedback-adjusted simultaneously.

[0094] 6. Model Evaluation

[0095] Figure 1 The evaluation in stage B6 uses the collected evaluation data set accounting for 20% to evaluate the trained model and obtain relevant evaluation parameters.

[0096] Figure 4The Z1 target segmentation stage uses PA (pixel accuracy), CPA (category pixel accuracy), and MPA (category average pixel accuracy) as evaluation indicators.

[0097] Figure 4 The Z2 rotation target recognition stage uses mAP, precision and recall as evaluation indicators.

[0098] 7. Edge deployment and operation

[0099] Figure 1 The deployment and operation of the B7 stage deploys the trained model on an edge computing platform, such as NVIDIANANO, NVIDIA XAVIER NX, or Cambrian Siyuan 220, to perform real-time computing and analysis. Figure 5 The following is the result of the rotating target recognition of the insulator string. Figure 6 Shown are the contour segmentation results of the insulator string.

[0100] Although the present invention has shown and described the embodiments of the present invention, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

[0101] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any ordinary technician in this industry can smoothly implement the present invention as shown in the drawings and described above. However, any equivalent changes, modifications and evolutions made by technicians familiar with this profession without departing from the scope of the technical solution of the present invention using the technical content disclosed above are all equivalent embodiments of the present invention. At the same time, any equivalent changes, modifications and evolutions made to the above embodiments based on the essential technology of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A collaborative analysis method for power insulator strings based on a multi-task convolutional neural network, characterized in that: It includes the following steps: S1. Image acquisition, collecting images of insulator strings under different lighting and background conditions; S1.1 Use drones and visible light payloads to collect data in different locations; S1.2 Use UAVs and visible light payloads to collect images under different lighting conditions; S1.3 Use a drone with a visible light payload to collect images at different distances from the insulator string. The distance is generally set within a range of 2 meters to 35 meters from the insulator string. S2. Preprocessing, three schemes are used to enrich the dataset; S2.

1. Perform pixel-level grayscale operations on the three channels of RGB on the collected image; In S2.1, the grayscale operation of the three-channel pixel level of the image RGB is performed by using random channel inversion, random parameter Gaussian blur, 4-neighborhood or 8-neighborhood random Laplace sharpening; the Gamma transformation with parameter values ​​between [0,1] is performed, and the operation formula is: S=C*R γ Where S is the transformed pixel value, C is the Gamma transformation scaling factor, which takes a value of 1, R is the input pixel, and γ is the transformation parameter, which takes a value of [0,1]; S2.

2. Cropping the original image by randomly selecting the X, Y coordinates of the upper left corner of the new image and the width and height W, H of the new image, and converting the coordinates according to the offset after cropping; S2.

3. Mosaic stitching is used for the images obtained in S2.1 and S2.2; S3. Annotation of the target frame and its direction in the image, using the long edge representation method to annotate the image; S4. Annotate the contour of the insulator string, annotate the image along the edge of the insulator string; S5. Deep neural network model construction, build a multi-task convolutional neural network model with the functions of target recognition and target segmentation; The segmentation branch of the S5.1 model adopts a U-row structure, first downsampling and then upsampling; The target recognition branch of the S5.2 model is connected in parallel with the PAFPN structure in the first half of the U-shaped structure (downsampling stage) for rotating target positioning; S6. Model training: import the data in S2 into the model built in S5, and calculate the error with the annotated true values ​​of S3 and S4, and feedback learning; S7 uses the same validation set with target box, direction and contour annotations for model verification.

2. The method for collaborative analysis of power insulator strings based on a multi-task convolutional neural network according to claim 1 is characterized in that: In S1.1, S1.2, and S1.3, the acquired images cover multiple resolution ranges, including but not limited to 640*480, 1280*720, 1024*768, 1920*1080, 2560*1440, 3840*2160, and 7680*4320.

3. The collaborative analysis method for power insulator strings based on a multi-task convolutional neural network according to claim 1 is characterized in that: In S2.2, the X and Y coordinates of the upper left corner of the selected new image meet the following conditions: X<W original / 2,Y<H original / 2, where W original ,H original is the width and height of the original image; The width and height W and H of the selected new image meet the following conditions: X+W<W original ,Y+H<H original , where X, Y are the upper left corner X, Y, and W of the new image selected in 6 original ,H original is the width and height of the original image.

4. The method for collaborative analysis of power insulator strings based on a multi-task convolutional neural network according to claim 1 is characterized in that: In S2.3, two-image mosaic, three-image mosaic, and four-image mosaic stitching methods are randomly adopted.

5. The method for collaborative analysis of power insulator strings based on a multi-task convolutional neural network according to claim 1 is characterized in that: In S3, the annotation box is a rectangle. The selection of the annotation box follows the direction of the long side of the insulator string. The image is annotated using the DOTA method. The annotation format is [x1, y1, x2, y2, x3, y3, x4, y4, ClassName, Diffcult], where x n ,y n Identify the four vertices of the target in sequence. Generally, the upper left corner is taken as the starting coordinate angle, ClassName is the target category, and Diffcult indicates the difficulty of identifying the target. Labelme software is used to mark the edge of the target. In addition to the insulator string body, the marking content also includes the hanging points at both ends of the insulator string, including the triangle plate and the retractable connecting rod. The DOTA marking format needs to be converted into the rotation angle marking format according to the long side marking. The format is as follows: [class_id,x_center,y_center,long_side,short_side,angle], where class_id identifies the target category index, x_center and y_center are the coordinates of the center point of the target box, long_side and short_side are the lengths of the long and short sides of the target box respectively, and angle is the rotation angle of the surface target box. The angle is expressed in long side notation and takes values ​​from 0 to 179 degrees, with 0 degrees in the positive direction of the X axis and 179 degrees close to the negative direction of the X axis.

6. The method for collaborative analysis of power insulator strings based on a multi-task convolutional neural network according to claim 1 is characterized in that: The multi-task network designed by S5 is a deep neural network that performs rotation target recognition and target segmentation simultaneously. It contains two branches, namely the rotation target recognition branch and the target segmentation branch. In S5.1, the target segmentation branch in the multi-task convolutional neural network adopts a U-shaped structure, which takes the form of downsampling first and then upsampling. The branch finally outputs a feature map of the same size as the input image. The U-shaped segmentation network includes the following parts: (1) The training input image is first downsampled by 4-level dilated convolution; (2) After 4 levels of deconvolution operations, a new type of upsampling is performed to obtain a feature map with the same size as the input training image; (3) There is a skip layer between downsampling and upsampling to fuse high-level and low-level features; The target segmentation branch in the multi-task convolutional neural network performs a Softmax operation on the channel axis at the final output truncation. The specific formula is as follows:

7. The method for collaborative analysis of power insulator strings based on a multi-task convolutional neural network according to claim 1, characterized in that: In S5.1, the target segmentation branch in the multi-task convolutional neural network adopts a weighted two-class cross entropy loss, and the specific formula is as follows: In S5.2, the rotated target recognition branch in the multi-task convolutional neural network is connected in parallel with the PAFPN structure in the first half of the U-shaped structure of the target segmentation branch (downsampling stage), and then after a certain convolutional neural extraction, the output content is the target category, center point coordinates and long side rotation angle; the loss calculation method of the rotated target recognition branch in the multi-task convolutional neural network combines the mean square error loss and the cross entropy loss.

8. The method for collaborative analysis of power insulator strings based on a multi-task convolutional neural network according to claim 1, characterized in that: The multi-task convolutional neural network training method designed by S5 adopts the method of alternating training first and then joint training, that is, the target monitoring branch is trained first, and after several epochs, the target segmentation branch is trained in the form of fine-tuning, and this is repeated for several cycles before joint training. The error calculation method in S6 refers to that described in 14 and 16, and the feedback learning method adopts the error back propagation learning mechanism.

9. The method for collaborative analysis of power insulator strings based on a multi-task convolutional neural network according to claim 1, characterized in that: S7 uses labeled data that has not participated in training to evaluate the model.

Images