Insulator equivalent icing thickness classification monitoring method and electronic equipment

Abstract

The invention discloses an insulator equivalent icing thickness classification monitoring method and electronic equipment. The method comprises the following steps: establishing an insulator icing monitoring image library; constructing a transferable classification detection network; respectively marking the positions of the insulators in the images, whether the insulators are iced or not and the icing types of the iced insulators to obtain a data set; obtaining an insulator identification model, an insulator icing judgment model and an insulator icing classification model through staged transfer learning; and identifying the insulator and the icing type thereof by using the insulator icing classification model, and monitoring the equivalent icing thickness of the insulator. According to the method, the identification precision of the insulator icing type can be improved, the method can be used for positioning and detecting the insulator in the image and distinguishing the ice-free state and the icing state of the insulator, the equivalent icing thickness of different icing types can be dynamically classified and monitored in real time, and decision support is provided for intelligent operation and maintenance of the insulator in ice and snow weather. The method can be widely applied to the field of power grid intelligent monitoring.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring of power grids, and in particular to a classification monitoring method for equivalent ice thickness of insulators and electronic equipment. Background Art

[0002] As a basic energy facility in modern society, the importance of safe and stable operation of power transmission lines in extreme weather is becoming increasingly prominent. In cold areas at mid- and high-latitudes, extreme snow and ice weather frequently occurs, and insulators of power transmission lines are often covered with ice. Ice covering shortens the air insulation distance of the insulator and increases the weight of the insulator. Severe ice covering threatens the safe and stable operation of the power grid, causing significant economic losses and.

[0003] Different types of icing and icing degrees bring different risks to insulators, and the corresponding deicing methods are also different. Therefore, classified monitoring of the equivalent ice thickness of insulators can provide more accurate technical support for the decision-making of anti-icing and deicing work of transmission lines. Distinguishing the types of insulator icing is the key prerequisite for classified monitoring of the equivalent ice thickness of insulators in complex backgrounds. The relevant technical solutions mainly provide two types of ideas. The first type of idea is to use artificially specified image features to distinguish the types of icing. For example, patent CN 107729397A gives the color component value range of insulator images covered with rime, rime, wet snow, and dry snow, and uses the color component difference to distinguish the types of icing; patent CN 113591877 A identifies the icing types of insulators of different materials by comparing the color, texture, and gradient similarity of image blocks and templates; the evaluation results of this method are affected by lighting and imaging conditions, and are only applicable to the classification of iced insulators in the laboratory, not to the actual online monitoring images of icing in the power grid. The second type of idea is to use artificial intelligence models to extract image features for distinguishing icing types and classify images. For example, patents CN 115965907 A and CN 118470443A use YOLO network and DDRNet network to identify the type of insulator icing respectively; such methods improve the speed of icing type recognition, but the image recognition effect is limited by the amount of training data and computing resources. If there is a difference in the data distribution of the data set and the test image, the monitoring accuracy of the insulator icing type may be affected. Therefore, there is still a certain gap between the existing methods and the actual application needs of online classification monitoring of icing. Summary of the invention

[0004] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the object of the present invention is to provide an insulator equivalent ice thickness classification monitoring method and electronic equipment based on staged transfer learning.

[0005] The first technical solution adopted by the present invention is:

[0006] An insulator equivalent ice thickness classification monitoring method comprises the following steps:

[0007] Establish an image library for monitoring insulator icing;

[0008] Build and train transferable classification detection networks;

[0009] Annotate the positions of insulators in the image to establish a first data set; use the first data set to fine-tune the pre-trained classification detection network to obtain an insulator recognition model;

[0010] Mark whether the insulator is covered with ice and establish a second data set; use the second data set to fine-tune the insulator recognition model to obtain an insulator icing judgment model;

[0011] The icing types of iced insulators are marked to establish a third data set; the insulator icing judgment model is fine-tuned using the third data set to obtain an insulator icing classification model;

[0012] The insulator icing classification model is used to identify the insulator visible light monitoring images to evaluate the icing thickness of the insulators.

[0013] Furthermore, the step of establishing an insulator icing monitoring image library includes:

[0014] Image acquisition: set the sampling time interval to obtain the image acquisition time; use a visible light camera installed in a fixed position to shoot the insulator at the sampling time interval to obtain the insulator ice monitoring image;

[0015] Automatically mark damaged images: Count the pixel values ​​of each pixel in the insulator ice monitoring image, and take the pixels whose red (R), green (G), and blue (B) components of the pixel values ​​satisfy [(0<R<5)∩(0<R<5)∩(0<R<5)] as black dots; when the number of black dots in the image exceeds the set damage threshold, the image is automatically marked as a damaged image;

[0016] Automatically mark blurred images: Based on the FFT (Fast Fourier Transform) algorithm, the insulator ice monitoring image is converted to the frequency domain to obtain the first frequency domain image; the central low-frequency area of ​​the first frequency domain image is set to zero to obtain the second frequency domain image; based on the IFFT (Inverse Fast Fourier Transform) algorithm, the second frequency domain image is converted to the spatial domain to obtain a transformation image; the amplitude average value of each pixel in the transformation image is calculated to obtain the blurriness; when the blurriness is lower than the set blur threshold, the image is automatically marked as a blurred image;

[0017] Establish an image library: remove images that are automatically marked as damaged or blurred; traverse the insulator icing monitoring images, match the images, file names and acquisition times, and establish an insulator icing monitoring image library.

[0018] Furthermore, the classification detection network includes an image preprocessing unit, a feature extraction unit, a feature sampling unit, an insulator positioning unit and an ice classification unit;

[0019] The construction and training of a transferable classification detection network includes:

[0020] Set up a training image preprocessing unit: take K input images from the insulator ice monitoring image library; set the number of preprocessing times to K; randomly take multiple training images for preprocessing each time; randomly scale, flip and splice the training images; crop the spliced ​​multiple input images into squares; fill the information-free area to obtain K training images;

[0021] Set up feature extraction unit: Use Focus module to slice the R, G, B color feature components of training image to obtain slice features; Use CSP (Cross Stage Partial Network) module to cross-fuse high- and low-dimensional features of training image to obtain fusion features; Use SPP (Spatial Pyramid Pooling) module to perform pooling operation on fusion features to obtain pooling features;

[0022] Set up feature sampling units: use the FPN (Feature Pyramid Network) module to upsample the pooled features to obtain a high-dimensional feature map; use the PAN (Path Aggregation Network) module to downsample the high-dimensional features to obtain a low-dimensional feature map; integrate the high-dimensional feature map and the low-dimensional feature map to obtain a sampling feature map;

[0023] Set the insulator positioning unit: Use s 2 The sampling feature map is divided into grids; the Sigmoid function is used to predict whether each grid contains an insulator; the minimum circumscribed rectangle containing the insulator grid is fitted to obtain the insulator identification box B; the scale parameters and category parameters of the true value box G are extracted from the annotation file; the coordinate offset (σ) of the center point of the insulator identification box B relative to the true value box G is recorded x ,σ y ), width scaling σ w and high scaling σ h ; Calculate shape intersection-over-union (IOU) B,G And the position loss function l box ;

[0024] Set up the ice classification unit: use the Softmax function to predict the category probability distribution Pr(O) of the insulator in the recognition box B; calculate the classification loss function l of the insulator cls ; Calculate the classification recognition confidence f dAnd the confidence loss function l obj ;

[0025] Set up the transfer learning unit: load the classification detection network from the source domain D S The learned source domain parameters θ S ; Input target domain D T K' new images are added; preprocessing, feature extraction and feature sampling are performed on the K' new images to obtain new features; the parameters of the classification detection network are fine-tuned with the new features to make the position loss function l box , classification loss function l cls And the confidence loss function l obj Descend until convergence and obtain the target domain parameter θ T ; Set the source domain parameter θ of the classification detection network S Update to the target domain parameter θ T .

[0026] Furthermore, the objective function of transfer learning in the transfer learning unit is:

[0027] logp(θ T |D T )=logp(D T |θ T )+logp(θ S |D S )+logp(D S )

[0028] In the formula, p(θ S |D S ) is the source domain parameter θ S In the source domain D S The posterior distribution function, p(θ T |D T ) is the target domain parameter θ T In the target domain D T The posterior distribution function, p(D T |θ T ) is the target domain parameter θ T In the target domain D T The conditional distribution function, p(D S ) is the source domain D S The marginal distribution function of .

[0029] Furthermore, the shape intersection over union (IOU) B,G The calculation formula is:

[0030]

[0031] The position loss function l box The calculation formula is:

[0032] 1-IOU B,G

[0033] Where a(BI G) and a(BUG) are the intersection area and union area of ​​B and G, respectively; d is the diagonal length of the minimum circumscribed rectangle of B and G; W and H are the width and height of the input image, respectively;

[0034] The classification loss function l cls The calculation formula is:

[0035] l cls =-∑C i 'log(C i )-(1-C i ')log(1-C i )

[0036] The classification recognition confidence f d The calculation formula is:

[0037] f d =Pr(O)×IOU B,G

[0038] The confidence loss function l obj The calculation formula is:

[0039] l obj =-Σf d,G log(f d,B )-(1-f d,G )log(1-f d,B )

[0040] In the formula, f d,B and f d,G They are the recognition confidence of the recognition frame B and the truth frame G of the insulator respectively.

[0041] Furthermore, the positions of the insulators are marked to establish a first data set; and the first data set is used to fine-tune the pre-trained classification detection network to obtain an insulator recognition model, including:

[0042] Mark the position of the insulator: Take K1 first-annotated images from the insulator ice monitoring image library; traverse the K1 first-annotated images, mark the insulator string in the image with a rectangular box labeled "01-insulator", and obtain the upper left corner vertex coordinates (x1, y1) and the lower right corner vertex coordinates (x2, y2) of the insulator truth box G;

[0043] Establish the first data set: Calculate the scale parameters of the insulator true value frame G, including: the horizontal coordinate true value G of the center point x , the true value of the vertical coordinate G y , width truth value Gw and the height truth value G h ; Convert the label "01-insulator" to the category parameter "0" of the truth box G; Write the scale parameter and category parameter of the insulator truth box G into a TXT format file with the same name as the annotated image to obtain the first annotation set; Store the first annotation set and K1 annotated images in the first path to obtain the first data set;

[0044] Fine-tune the pre-trained classification detection network: load the pre-trained parameters θ0 learned by the transferable classification detection network in the MS COCO dataset; input the first dataset and take the classification number i=1; use the transfer learning unit to fine-tune the pre-trained parameters θ0 for I1 rounds to obtain the first parameters θ1; update the pre-trained parameters θ0 of the classification detection network to the first parameters θ1 to obtain the insulator recognition model.

[0045] Furthermore, the step of marking whether the insulator is covered with ice and establishing a second data set; and fine-tuning the insulator recognition model using the second data set to obtain an insulator icing judgment model includes:

[0046] Mark whether the insulator is covered with ice: from the first data set, take K2 second annotated images; traverse the K2 second annotated images, determine whether the insulator in the annotated frame is covered with ice, and replace the labels of ice-free insulators and iced insulators with "01-none" and "02-iced" respectively;

[0047] Establish the second data set: export a TXT file with the same name as the K2 second annotated images from the first data set; convert the label "02-iced" to the category parameter "1" of the truth box G; replace the category parameter of iced insulators with "1" in the TXT file to obtain the second annotation set; store the second annotation set and the K2 annotated images in the second path to obtain the second data set;

[0048] Fine-tune the insulator identification model: load the insulator identification model and the first parameter θ1; input the second data set and take the classification number i=2; use the transfer learning unit to fine-tune the first parameter θ1 for I2 rounds to obtain the second parameter θ2; update the first parameter θ1 of the classification detection network to the second parameter θ2 to obtain the insulator icing judgment model.

[0049] Furthermore, the icing types of the iced insulators are marked to establish a third data set; the insulator icing judgment model is fine-tuned using the third data set to obtain an insulator icing classification model, including:

[0050] Labeling the icing type of the insulator: from the second data set, taking K3 third labeled images of ice-covered insulators (category parameter is "1"); traversing the K3 third labeled images, determining the icing type of the insulator, and replacing the labels of snow-covered insulators, rime-covered insulators, mixed rime-covered insulators, and rain rime-covered insulators with "02-snow", "03-rime", "04-mixed", and "05-glaze", respectively;

[0051] Establish the third data set: export a TXT file with the same name as the K3 third annotated images from the second data set; convert the labels "02-snow", "03-rime", "04-mixed", and "05-glaze" into the category parameters "1", "2", "3", and "4" of the truth box G respectively to obtain the third annotation set; store the third annotation set and the K3 annotated images in the third path to obtain the third data set;

[0052] Fine-tune the insulator icing judgment model: load the insulator icing judgment model and the second parameter θ2; input the third data set and take the classification number i=5; use the transfer learning unit to fine-tune the second parameter θ2 for I3 rounds to obtain the third parameter θ3; update the second parameter θ2 of the classification detection network to the third parameter θ3 to obtain the insulator icing classification model.

[0053] Furthermore, the insulator ice classification model is used to identify the insulator visible light monitoring image to evaluate the ice thickness of the insulator, including:

[0054] Identify the visible light monitoring image of the insulator: input the real-time visible light monitoring image of the insulator into the insulator icing classification model; output the label and classification confidence of the insulator icing type, including: "no ice", "snow", "rime", "mixed rime", and "rain rime"; output the center point coordinates of the insulator identification box B (B x ,B y ), and the height truth value B h ;

[0055] Assess the ice thickness of the insulator: When the insulator is iced, extract the width B of the identification box of the same insulator in the image when it is ice-free and ice-covered w0 , B w1 ; Calculate the ice thickness I of the insulator under different ice types; When the ice thickness exceeds the thickness threshold, a warning signal is issued.

[0056] Furthermore, the calculation formula for the standard ice thickness is:

[0057]

[0058] Where ρ is the ice density. When the ice type is ice-free, ρ = 0 g / cm 3When the ice type is snow, ρ = 0.4 g / cm 3 When the ice type is rime, take ρ = 0.3g / cm 3 When the icing type is mixed rime, ρ = 0.6 g / cm 3 When the ice type is rime, take ρ = 0.9 g / cm 3 .

[0059] Furthermore, when the ambient temperature is lower than 0°C, the operating environment of the insulator is set to ice and snow weather, and the insulator icing classification monitoring system is started; when the ambient temperature is higher than 0°C, the operating environment of the insulator is set to normal weather, and the insulator icing classification monitoring system is stopped.

[0060] The second technical solution adopted by the present invention is:

[0061] An electronic device comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a classification monitoring method for equivalent ice thickness of insulators as described above.

[0062] The third technical solution adopted by the present invention is:

[0063] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement a method for classifying and monitoring equivalent ice thickness of insulators as described above.

[0064] The fourth technical solution adopted by the present invention is:

[0065] A computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned insulator equivalent ice thickness classification monitoring method.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] (1) The present invention designs a phased transfer learning strategy, which realizes high-precision identification of the type of insulator icing based on identifying the insulator and judging whether it is iced. On the premise of saving monitoring computing resources, the applicability of the insulator icing classification model under variable environmental conditions is improved.

[0068] (2) The method and system provided by the present invention can monitor the operating insulators in real time, automatically clean and process the visible light monitoring images of the insulators in real scenes, greatly reducing manual intervention, lowering the difficulty of labeling and improving operation and maintenance efficiency.

[0069] (3) The technical solution provided by the present invention can also automatically convert the ice thickness of insulators according to the image recognition results of insulators under different ice types, providing a scientific decision-making basis for smart grids to timely and accurately monitor the operating status of equipment and respond to extreme weather. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0071] Figure 1 An overall flow chart of an insulator equivalent ice thickness classification monitoring method based on phased transfer learning provided by an embodiment of the present invention;

[0072] Figure 2 A schematic diagram of constructing an insulator icing classification model based on phased transfer learning provided in an embodiment of the present invention;

[0073] Figure 3 A schematic diagram of manual marking of identification frames of insulators with different ice coating types provided in an embodiment of the present invention;

[0074] Figure 4 A diagram showing the identification results of different ice types of the same insulator in a typical monitoring image provided by an embodiment of the present invention;

[0075] Figure 5 A diagram of the software and hardware composition architecture of an insulator equivalent ice thickness classification monitoring system based on staged transfer learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0077] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0078] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0079] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0080] In view of the problems in the prior art of monitoring the equivalent ice thickness of operating insulators, such as complex background, changeable ice forms, uneven number of monitoring images, resulting in large labeling workload, low recognition accuracy, high data dependence, complex calculation, etc., the present invention provides an insulator equivalent ice thickness classification monitoring scheme based on staged transfer learning.

[0081] Example 1

[0082] like Figure 1 As shown, this embodiment provides a classification monitoring method for insulator equivalent ice thickness based on staged transfer learning, which specifically includes the following steps:

[0083] S1. Establish an insulator icing monitoring image library: collect and clean insulator icing monitoring images in icy and snowy weather to obtain an insulator icing monitoring image library.

[0084] As an implementation manner, step S1 specifically includes the following steps:

[0085] S11, collecting images: setting a sampling time interval to obtain an image collection time; using a visible light camera at a fixed installation position to photograph the insulator at the sampling time interval to obtain an insulator ice coverage monitoring image.

[0086] S12. Automatically mark damaged images: Count the pixel value of each pixel in the insulator ice monitoring image, and take the pixel points whose red (R), green (G), and blue (B) components of the pixel value satisfy [(0<R<5)∩(0<G<5)∩(0<B<5)] as black dots; when the number of black dots in the image is higher than the set damage threshold, the image is automatically marked as a damaged image.

[0087] S13. Automatically mark blurred images: based on the FFT (Fast Fourier Transform) algorithm, the insulator ice monitoring image is converted to the frequency domain to obtain a first frequency domain image; the central low-frequency area of ​​the first frequency domain image is set to zero to obtain a second frequency domain image; based on the IFFT (Inverse Fast Fourier Transform), the second frequency domain image is converted to the spatial domain to obtain a transformation image; the amplitude average of each pixel in the transformation image is calculated to obtain the blurriness; when the blurriness is lower than the set blur threshold, the image is automatically marked as a blurred image.

[0088] The expression of the FFT is:

[0089]

[0090] The expression of IFFT is:

[0091]

[0092] Where f(x,y) is the pixel value of the pixel with coordinates (x,y) in the insulator icing monitoring image, F(u,v) is the frequency value of the pixel with coordinates (u,v) in the first frequency domain image, and M and N are the width and height of the insulator icing monitoring image, respectively.

[0093] S14, establishing an image library: removing images automatically marked as damaged or blurred; traversing the insulator icing monitoring images, matching the images, file names and acquisition times, and establishing an insulator icing monitoring image library.

[0094] S2. Construct a transferable classification detection network: obtain a classification detection network by sequentially setting a training image preprocessing unit, a feature extraction unit, a feature sampling unit, an insulator positioning unit, and an ice classification unit; obtain a transferable classification detection network by setting a transfer learning unit.

[0095] See also Figure 2 In some embodiments, step S2 comprises the following steps:

[0096] S21. Set a training image preprocessing unit: take K input images from the insulator ice monitoring image library; set the number of preprocessing times to K; randomly take 4 training images for preprocessing each time; randomly scale, flip and splice the training images; crop the spliced ​​4 input images into a square; fill the information-free area to obtain K training images.

[0097] S22. Set a feature extraction unit: use the Focus module to slice the R, G, and B color feature components of the training image to obtain slice features; use the CSP (Cross Stage Partial Network) module to cross-fuse the high- and low-dimensional features of the training image to obtain fused features; use the SPP (Spatial Pyramid Pooling) module to pool the fused features to obtain pooled features.

[0098] S23. Set a feature sampling unit: use the FPN (Feature Pyramid Network) module to upsample the pooled features to obtain a high-dimensional feature map; use the PAN (Path Aggregation Network) module to downsample the high-dimensional features to obtain a low-dimensional feature map; integrate the high-dimensional feature map and the low-dimensional feature map to obtain a sampling feature map.

[0099] S24, set the insulator positioning unit: use s 2 The sampling feature map is divided into grids; the Sigmoid function is used to predict whether each grid contains an insulator; the minimum circumscribed rectangle containing the insulator grid is fitted to obtain the insulator identification box B; the scale parameters and category parameters of the true value box G are extracted from the annotation file; the coordinate offset (σ) of the center point of the insulator identification box B relative to the true value box G is recorded x ,σ y ), width scaling σ w and high scaling σ h ; Calculate shape intersection-over-union (IOU) B,G And the position loss function l box .

[0100] The IOU B,G The calculation formula is:

[0101]

[0102] The l box The calculation formula is:

[0103] 1-IOU B,G

[0104] Where a(BI G) and a(BUG) are the intersection area and union area of ​​B and G, respectively; d is the diagonal length of the minimum circumscribed rectangle of B and G; W and H are the width and height of the input image, respectively;

[0105] S25. Set the ice classification unit: use the Softmax function to predict the category probability distribution Pr(O) of the insulator in the recognition box B; calculate the classification loss function of the insulator:

[0106] l cls =-∑C i 'log(C i )-(1-C i ')log(1-C i )

[0107] In the formula, C i , C i ' is the probability of whether the predicted category is the true value, i is the number of categories; calculate the classification recognition confidence:

[0108] f d =Pr(O)×IOU B,G

[0109] Calculate the confidence loss function:

[0110] l obj =-∑f d,G log(f d,B )-(1-f d,G )log(1-f d,B )

[0111] In the formula, f d,B and f d,G They are the recognition confidence of the recognition frame B and the truth frame G of the insulator respectively.

[0112] S26, set up transfer learning unit: load classification detection network from source domain D S The learned source domain parameters θ S ; Input target domain D T K' new images are added; preprocessing, feature extraction and feature sampling are performed on the K' new images to obtain new features; the parameters of the classification detection network are fine-tuned with the new features to make the position loss function l box , classification loss function l cls And the confidence loss function l obj Descend until convergence and obtain the target domain parameter θ T ; Set the source domain parameter θ of the classification detection network S Update to the target domain parameter θ T ; The objective function of the above transfer learning is:

[0113] logp(θ T |D T )=logp(D T |θ T )+logp(θ S |D S )+logp(D S )

[0114] In the formula, p(θ S |D S ) is the source domain parameter θ S In the source domain D S The posterior distribution function, p(θ T |D T ) is the target domain parameter θ T In the target domain D T The posterior distribution function, p(D T |θ T ) is the target domain parameter θ T In the target domain D T The conditional distribution function, p(D S ) is the source domain D S The marginal distribution function of .

[0115] S3. Transfer learning insulator recognition model: mark the position of the insulator, establish the first data set, use the data set to fine-tune the pre-trained classification detection network, and obtain the insulator recognition model.

[0116] See also Figure 2 As an optional implementation, step S3 specifically includes:

[0117] S31. Mark the position of the insulator: take K1 first labeled images from the insulator ice monitoring image library; traverse the K1 first labeled images, mark the insulator string in the image with a rectangular box labeled "01-insulator", and obtain the upper left corner vertex coordinates (x1, y1) and the lower right corner vertex coordinates (x2, y2) of the insulator truth box G.

[0118] S32, establish the first data set: calculate the scale parameters of the insulator true value frame G, including: the horizontal coordinate true value G of the center point x , the true value of the vertical coordinate G y , width truth value G w and the height truth value G h ; Convert the label "01-insulator" to the category parameter "0" of the truth box G; Write the scale parameter and category parameter of the insulator truth box G into a TXT format file with the same name as the annotated image to obtain the first annotation set; Store the first annotation set and K1 annotated images in the first path to obtain the first data set.

[0119] S33, fine-tuning the pre-trained classification detection network: load the pre-trained parameters θ0 learned by the transferable classification detection network in the MS COCO dataset; input the first dataset and take the classification number i=1; use the transfer learning unit to fine-tune the pre-trained parameters θ0 for I1 rounds to obtain the first parameters θ1; update the pre-trained parameters θ0 of the classification detection network to the first parameters θ1 to obtain the insulator recognition model.

[0120] S4. Transfer learning of insulator ice-covered judgment model: mark whether the insulator is covered with ice, establish a second data set, use the data set to fine-tune the insulator recognition model, and obtain the insulator ice-covered judgment model.

[0121] See also Figure 2 As an optional implementation, step S4 specifically includes:

[0122] S41. Labeling whether the insulator is covered with ice: from the first data set, take K2 second labeled images; traverse the K2 second labeled images, determine whether the insulator in the labeled frame is covered with ice, and replace the labels of ice-free insulators and iced insulators with "01-none" and "02-iced" respectively.

[0123] S42. Establish a second data set: export a TXT format file with the same name as the K2 second annotated images from the first data set; convert the label "02-iced" to the category parameter "1" of the truth box G; replace the category parameter of the iced insulator with "1" in the TXT format file to obtain a second annotation set; store the second annotation set and the K2 annotated images in a second path to obtain a second data set.

[0124] S43, fine-tuning the insulator identification model: loading the insulator identification model and the first parameter θ1; inputting the second data set, taking the number of classifications i=2; using the transfer learning unit to fine-tune the first parameter θ1 for I2 rounds to obtain the second parameter θ2; updating the first parameter θ1 of the classification detection network to the second parameter θ2 to obtain the insulator icing judgment model.

[0125] S5. Transfer learning of insulator icing classification model: Label the icing types of iced insulators, establish a third data set, use the data set to fine-tune the icing judgment model, and obtain the insulator icing classification model.

[0126] See also Figure 2 As an optional implementation, step S5 specifically includes:

[0127] S51, labeling the icing type of the insulator: from the second data set, take K3 third labeled images of ice-covered insulators (category parameter is "1"); traverse the K3 third labeled images, determine the icing type of the insulator, and replace the labels of snow-covered insulators, rime-covered insulators, mixed rime-covered insulators, and rain rime-covered insulators with "02-snow", "03-rime", "04-mixed", and "05-glaze", respectively. Figure 3 shown.

[0128] S52. Establish a third data set: export a TXT format file with the same name as the K3 third annotated images from the second data set; convert the labels "02-snow", "03-rime", "04-mixed", and "05-glaze" into category parameters "1", "2", "3", and "4" of the truth box G respectively to obtain a third annotation set; store the third annotation set and the K3 annotated images in a third path to obtain a third data set.

[0129] S53, fine-tuning the insulator icing judgment model: loading the insulator icing judgment model and the second parameter θ2; inputting the third data set, taking the number of classifications i=5; using the transfer learning unit to fine-tune the second parameter θ2 for I3 rounds to obtain the third parameter θ3; updating the second parameter θ2 of the classification detection network to the third parameter θ3 to obtain the insulator icing classification model.

[0130] S6. Monitor the equivalent ice thickness of insulators: Use the insulator ice classification model to identify the visible light monitoring image of insulators and evaluate the ice thickness of insulators.

[0131] See also Figure 2 As an optional implementation, step S6 specifically includes:

[0132] S61, identifying the visible light monitoring image of the insulator: inputting the real-time collected visible light monitoring image of the insulator into the insulator icing classification model; outputting the label of the insulator icing type and the classification confidence, the labels include: "no ice", "snow", "rime", "mixed rime", "rain rime", etc. Figure 4 Output the center point coordinates of the insulator identification frame B (B x ,B y ), and the height truth value B h .

[0133] S62, evaluate the ice thickness of the insulator: when the insulator is iced, extract the width B of the identification box of the same insulator in the image when it is ice-free and ice-covered w0 , B w1 ; Calculate the ice thickness I of the insulator under different ice types; When the ice thickness exceeds the thickness threshold, a warning signal is issued.

[0134] Specifically, the calculation formula of the standard ice thickness is:

[0135]

[0136] Where ρ is the ice density. When the ice type is ice-free, ρ = 0 g / cm 3 When the ice type is snow, ρ = 0.4 g / cm 3 When the ice type is rime, take ρ = 0.3g / cm 3 When the icing type is mixed rime, ρ = 0.6 g / cm 3 When the ice type is rime, take ρ = 0.9 g / cm 3 .

[0137] Example 2

[0138] The present embodiment provides an insulator equivalent ice thickness classification monitoring system based on phased transfer learning. When the ambient temperature is lower than 0°C, the operating environment of the insulator is set to snowy weather, and the insulator ice classification monitoring system is started; when the ambient temperature is higher than 0°C, the operating environment of the insulator is set to normal weather, and the insulator ice classification monitoring system is stopped.

[0139] In some embodiments, see Figure 5 The insulator equivalent ice thickness classification monitoring system based on phased transfer learning includes the following software units:

[0140] Insulator image cleaning software unit: used to clean the collected insulator visible light monitoring images;

[0141] Insulator and ice type recognition software unit: used to identify insulators in images and determine ice type;

[0142] Insulator equivalent ice thickness monitoring software unit: used to calculate the equivalent ice thickness of the insulator according to the ice type of the insulator;

[0143] Human-computer interaction software unit: used to display monitoring results and alarm information, and provide a system operation interface.

[0144] In some embodiments, see Figure 5 The insulator equivalent ice thickness classification monitoring system based on phased transfer learning includes the following hardware:

[0145] at least one set of cameras capable of capturing visible light images of insulators at set sampling time intervals;

[0146] At least one processor, configured to run the software unit described above;

[0147] At least one memory is used to store computer-readable instructions and insulator ice classification monitoring results.

[0148] Since the system is an insulator equivalent ice thickness classification monitoring system based on phased transfer learning in an embodiment of the present invention, and the principle of solving the problem by the system is similar to that of the method, the implementation of the system can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0149] Example 3

[0150] An embodiment of the present invention further provides an electronic device, the electronic device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the following Figure 1 A classification monitoring method for insulator equivalent ice thickness based on staged transfer learning is shown.

[0151] It is understood that the memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the server, etc.

[0152] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.

[0153] Since the electronic device is an electronic device corresponding to an insulator equivalent ice thickness classification monitoring method based on phased transfer learning in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0154] Example 4

[0155] The embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 A classification monitoring method for insulator equivalent ice thickness based on staged transfer learning is shown.

[0156] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0157] Since the storage medium is a storage medium corresponding to an insulator equivalent ice thickness classification monitoring method based on phased transfer learning in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0158] Example 5

[0159] In some possible implementations, various aspects of the method of the embodiment of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the insulator equivalent ice thickness classification monitoring method based on staged transfer learning according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" for executing various embodiments can be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, VisualBasic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0160] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0161] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0162] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A classification monitoring method for equivalent ice thickness of insulators, characterized in that: The following steps are involved: Establish an image library for monitoring insulator icing; Build and train transferable classification detection networks; Marking the positions of the insulators to establish a first data set; using the first data set to fine-tune the pre-trained classification detection network to obtain an insulator recognition model; Mark whether the insulator is covered with ice and establish a second data set; use the second data set to fine-tune the insulator recognition model to obtain an insulator icing judgment model; Mark the icing type of iced insulators and establish the third data set; The third data set is used to fine-tune the insulator icing judgment model to obtain the insulator icing classification model; The insulator icing classification model is used to identify the insulator visible light monitoring images to evaluate the icing thickness of the insulators.

2. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1 is characterized in that: The step of establishing an insulator icing monitoring image library comprises: Image acquisition: set the sampling time interval to obtain the image acquisition time; use a visible light camera installed in a fixed position to shoot the insulator at the sampling time interval to obtain the insulator ice monitoring image; Automatically mark damaged images: Count the pixel values ​​of each pixel in the insulator ice monitoring image, and take the pixels whose red, green, and blue components of the pixel values ​​meet the preset conditions as black dots; when the number of black dots in the image exceeds the set damage threshold, the image is automatically marked as a damaged image; Automatically mark blurred images: Based on the FFT algorithm, the insulator ice monitoring image is converted to the frequency domain to obtain the first frequency domain image; the central low-frequency area of ​​the first frequency domain image is set to zero to obtain the second frequency domain image; based on the IFFT algorithm, the second frequency domain image is converted to the spatial domain to obtain a transformation image; the amplitude average value of each pixel point in the transformation image is calculated to obtain the blurriness; when the blurriness is lower than the set blur threshold, the image is automatically marked as a blurred image; Establish an image library: remove images that are automatically marked as damaged or blurred; traverse the insulator icing monitoring images, match the images, file names and acquisition times, and establish an insulator icing monitoring image library.

3. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1 is characterized in that: The classification detection network includes an image preprocessing unit, a feature extraction unit, a feature sampling unit, an insulator positioning unit and an ice classification unit; The construction and training of a transferable classification detection network includes: Set up a training image preprocessing unit: take K input images from the insulator ice monitoring image library; set the number of preprocessing times to K; randomly take multiple training images for preprocessing each time; randomly scale, flip and splice the training images; Crop the stitched input images into squares; fill the information-free areas to obtain K training images; Set up a feature extraction unit: use the Focus module to slice the R, G, and B color feature components of the training image to obtain slice features; cross-fuse the high- and low-dimensional features of the training image to obtain fused features; perform pooling operations on the fused features to obtain pooling features; Set up feature sampling units: upsample the pooled features to obtain a high-dimensional feature map; downsample the high-dimensional features to obtain a low-dimensional feature map; integrate the high-dimensional feature map and the low-dimensional feature map to obtain a sampling feature map; Set up the insulator positioning unit: divide the sampling feature map; predict whether each grid contains an insulator; fit the minimum circumscribed rectangle containing the insulator grid to obtain the insulator identification box B; extract the scale parameters and category parameters of the true value box G from the annotation file; record the coordinate offset (σ) of the center point of the insulator identification box B relative to the true value box G x ,σ y ), width scaling σ w and high scaling σ h ; Calculate shape intersection-over-union (IOU) B,G And the position loss function l box ; Set up the ice classification unit: predict the class probability distribution Pr(O) of the insulator in the recognition box B; calculate the classification loss function l of the insulator cls ; Calculate the classification recognition confidence f d And the confidence loss function l obj ; Set up the transfer learning unit: load the classification detection network from the source domain D S The learned source domain parameters θ S ; Input target domain D T K' new images are added; preprocessing, feature extraction and feature sampling are performed on the K' new images to obtain new features; the parameters of the classification detection network are fine-tuned with the new features to make the position loss function l box , classification loss function l cls And the confidence loss function l obj Descend until convergence and obtain the target domain parameter θ T ; Set the source domain parameter θ of the classification detection network S Update to the target domain parameter θ T .

4. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 3 is characterized in that: The objective function of transfer learning in the transfer learning unit is: logp(θ T |D T )=logp(D T |θ T )+logp(θ S |D S )+logp(D S ) In the formula, p(θ S |D S ) is the source domain parameter θ S In the source domain D S The posterior distribution function, p(θ T |D T ) is the target domain parameter θ T In the target domain D T The posterior distribution function, p(D T |θ T ) is the target domain parameter θ T In the target domain D T The conditional distribution function of p(D S ) is the source domain D S The marginal distribution function of .

5. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 3 is characterized in that: The shape intersection over union (IOU) B,G The calculation formula is: The position loss function l box The calculation formula is: 1-IOU B,G Where a(BI G) and a(BUG) are the intersection area and union area of ​​B and G, respectively; d is the diagonal length of the minimum circumscribed rectangle of B and G; W and H are the width and height of the input image, respectively; The classification loss function l cls The calculation formula is: l cls =-∑C i 'log(C i )-(1-C i ')log(1-C i ) The classification recognition confidence f d The calculation formula is: f d =Pr(O)×IOU B,G The confidence loss function l obj The calculation formula is: l obj =-Σf d,G log(f d,B )-(1-f d,G )log(1-f d,B ) In the formula, f d,B and f d,G They are the recognition confidence of the recognition frame B and the truth frame G of the insulator respectively.

6. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1 is characterized in that: The steps of marking the positions of the insulators and establishing a first data set; and fine-tuning the pre-trained classification detection network using the first data set to obtain an insulator recognition model include: Take K1 first annotated images from the insulator ice monitoring image library; traverse the K1 first annotated images, annotate the insulator string in the image with a rectangular box labeled "01-insulator", and obtain the upper left corner vertex coordinates (x1, y1) and the lower right corner vertex coordinates (x2, y2) of the insulator truth box G; Calculate the scale parameters of the insulator true value frame G, including: the horizontal coordinate true value G of the center point x , the true value of the vertical coordinate G y , width truth value G w and the height truth value G h ; Convert the label "01-insulator" to the category parameter "0" of the truth box G; Write the scale parameter and category parameter of the insulator truth box G into a TXT format file with the same name as the annotated image to obtain the first annotation set; Store the first annotation set and K1 annotated images in the first path to obtain the first data set; Load the pre-trained parameters θ0 learned by the transferable classification detection network in the MS COCO dataset; input the first dataset and take the classification number i=1; use the transfer learning unit to fine-tune the pre-trained parameters θ0 for I1 rounds to obtain the first parameters θ1; update the pre-trained parameters θ0 of the classification detection network to the first parameters θ1 to obtain the insulator recognition model.

7. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1 is characterized in that: The marking of whether the insulator is covered with ice and establishing a second data set; The second data set is used to fine-tune the insulator identification model to obtain an insulator icing judgment model, including: From the first data set, take K2 second annotated images; traverse the K2 second annotated images, determine whether the insulator in the annotation box is covered with ice, and replace the labels of the ice-free insulator and the ice-covered insulator with "01-none" and "02-iced" respectively; export a TXT format file with the same name as the K2 second annotated images from the first data set; convert the label "02-iced" to the category parameter "1" of the truth box G; replace the category parameter of the ice-covered insulator with "1" in the TXT format file to obtain a second annotation set; store the second annotation set and the K2 annotated images in a second path to obtain a second data set; Fine-tune the insulator identification model: load the insulator identification model and the first parameter θ1; input the second data set and take the classification number i=2; use the transfer learning unit to fine-tune the first parameter θ1 for I2 rounds to obtain the second parameter θ2; update the first parameter θ1 of the classification detection network to the second parameter θ2 to obtain the insulator icing judgment model.

8. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1 is characterized in that: The ice type of the ice-covered insulator marked with ice is used to establish a third data set; The third data set is used to fine-tune the insulator icing judgment model, and the insulator icing classification model is obtained, including: From the second data set, K3 third annotated images of ice-covered insulators are obtained; the K3 third annotated images are traversed to determine the icing type of the insulator, and the labels of snow-covered insulators, rime-covered insulators, mixed rime-covered insulators, and rain rime-covered insulators are replaced with "02-snow", "03-rime", "04-mixed", and "05-glaze", respectively; Export a TXT file with the same name as the K3 third annotated images from the second data set; convert the labels "02-snow", "03-rime", "04-mixed", and "05-glaze" into category parameters "1", "2", "3", and "4" of the truth box G, respectively, to obtain a third annotation set; store the third annotation set and the K3 annotated images in a third path to obtain a third data set; Fine-tune the insulator icing judgment model: load the insulator icing judgment model and the second parameter θ2; input the third data set and take the classification number i=5; use the transfer learning unit to fine-tune the second parameter θ2 for I3 rounds to obtain the third parameter θ3; update the second parameter θ2 of the classification detection network to the third parameter θ3 to obtain the insulator icing classification model.

9. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1 is characterized in that: The insulator ice classification model is used to identify the insulator visible light monitoring image to evaluate the ice thickness of the insulator, including: The real-time collected insulator visible light monitoring image is input into the insulator icing classification model; the label of the insulator icing type and the classification confidence are output; the center point coordinates of the insulator identification frame B (B x ,B y ), and the height truth value B h ; When the insulator is covered with ice, the width B of the identification box when the same insulator is ice-free and ice-covered is extracted. w0 , B w1 ; Calculate the ice thickness I of the insulator under different ice types; When the ice thickness exceeds the thickness threshold, a warning signal is issued.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method described in any one of claims 1 to 9.

Images