Insulator equivalent ice thickness classification monitoring method and electronic device

Abstract

The application discloses a kind of insulator equivalent ice thickness classification monitoring method and electronic equipment, wherein method includes: establishing insulator icing monitoring image library;Migratable classification detection network is constructed;The position of image insulator, whether insulator is iced, the icing type of iced insulator is labeled respectively, and data set is obtained;Through stage migration learning, insulator recognition model, insulator icing judgment model and insulator icing classification model are obtained;With insulator icing classification model, insulator and its icing type are identified, and the equivalent ice thickness of insulator is monitored.The application can improve the recognition accuracy of insulator icing type, which can not only be used for positioning detection insulator in image and distinguishing its ice-free and icing state, but also can dynamically classify and monitor the equivalent ice thickness of different icing types in real time, provide decision support for intelligent operation and maintenance of insulator under ice and snow weather.The application can be widely applied in power grid intelligent monitoring field.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring of power grids, and in particular to a classification monitoring method for equivalent ice thickness of insulators and electronic equipment. Background Art

[0002] As fundamental energy infrastructure in modern society, transmission lines are increasingly crucial for safe and stable operation in extreme weather conditions. In cold regions of mid- and high-latitudes, extreme snow and ice conditions frequently occur, and ice often forms on transmission line insulators. This ice formation shortens the air insulation distance of insulators and increases their weight. Severe ice accumulation can threaten the safe and stable operation of power grids, causing significant economic losses and damage.

[0003] Different ice types and degrees pose varying risks to insulators, necessitating different de-icing methods. Therefore, classifying and monitoring the equivalent ice thickness of insulators can provide more accurate technical support for decision-making in transmission line anti-icing and de-icing efforts. Distinguishing insulator ice types is a key prerequisite for classifying and monitoring equivalent ice thickness in complex environments. Related technical solutions primarily offer two approaches. The first utilizes manually specified image features to distinguish ice types. For example, patent CN 107729397A specifies color component value ranges for images of insulators covered with rime, rime, wet snow, and dry snow, using color component differences to distinguish ice types. Patent CN 113591877 A identifies ice types on insulators of different materials by comparing the color, texture, and gradient similarities of image blocks with templates. However, the evaluation results of these methods are affected by lighting and imaging conditions and are only applicable to the classification of iced insulators in the laboratory, not to actual online monitoring images of power grid ice. The second approach utilizes artificial intelligence models to extract image features that distinguish ice types and classify the images. For example, patents CN 115965907 A and CN 118470443 A use the YOLO network and DDRNet network, respectively, to identify insulator ice types. While these methods improve the speed of ice type identification, image recognition performance is limited by the amount of training data and computing resources. If the data distribution of the dataset and the test images differ, the accuracy of insulator ice type monitoring may be affected. Therefore, existing methods still fall short of the practical application requirements of online ice classification monitoring. Summary of the Invention

[0004] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide an insulator equivalent ice thickness classification monitoring method and electronic equipment based on staged transfer learning.

[0005] The first technical solution adopted by the present invention is:

[0006] A classification monitoring method for equivalent ice thickness of insulators comprises the following steps:

[0007] Establish an image library for monitoring insulator icing;

[0008] Build and train transferable classification and detection networks;

[0009] Marking the locations of insulators in the image to establish a first data set; using the first data set to fine-tune the pre-trained classification detection network to obtain an insulator recognition model;

[0010] Mark whether the insulator is covered with ice and establish a second data set; use the second data set to fine-tune the insulator recognition model to obtain an insulator icing judgment model;

[0011] The icing types of iced insulators are marked to establish a third data set. The third data set is used to fine-tune the insulator icing judgment model to obtain an insulator icing classification model.

[0012] An insulator icing classification model is used to identify insulator visible light monitoring images to assess the icing thickness of insulators.

[0013] Furthermore, the step of establishing an insulator icing monitoring image library includes:

[0014] Image acquisition: Set the sampling time interval to obtain the image acquisition time; use a visible light camera installed in a fixed position to shoot the insulator at the sampling time interval to obtain the insulator ice monitoring image;

[0015] Automatically mark damaged images: Count the pixel values ​​of each pixel in the insulator ice monitoring image and take the pixels whose red (R), green (G), and blue (B) components satisfy [(0 < R < 5) ∩ (0 < R < 5) ∩ (0 < R < 5)] as black dots. When the number of black dots in the image exceeds the set damage threshold, the image is automatically marked as damaged.

[0016] Automatically label blurred images: The insulator ice monitoring image is converted to the frequency domain using the FFT (Fast Fourier Transform) algorithm to obtain a first frequency domain image. The central low-frequency region of the first frequency domain image is set to zero to obtain a second frequency domain image. The second frequency domain image is converted to the spatial domain using the IFFT (Inverse Fast Fourier Transform) algorithm to obtain a transformation image. The amplitude average of each pixel in the transformation image is calculated to obtain the degree of blur. When the degree of blur is lower than a set blur threshold, the image is automatically labeled as a blurred image.

[0017] Establish an image library: remove images automatically marked as damaged or blurred; traverse the insulator icing monitoring images, match the images, file names and acquisition times, and establish an insulator icing monitoring image library.

[0018] Furthermore, the classification detection network includes an image preprocessing unit, a feature extraction unit, a feature sampling unit, an insulator positioning unit and an ice classification unit;

[0019] The construction and training of a transferable classification detection network includes:

[0020] Set up a training image preprocessing unit: take K input images from the insulator ice monitoring image library; set the number of preprocessing times to K; randomly take multiple training images for preprocessing each time; randomly scale, flip, and splice the training images; crop the spliced ​​multiple input images into squares; fill the information-free areas, and obtain K training images;

[0021] Set up the feature extraction unit: use the Focus module to slice the R, G, and B color feature components of the training image to obtain slice features; use the CSP (Cross Stage Partial Network) module to cross-fuse the high- and low-dimensional features of the training image to obtain fused features; use the SPP (Spatial Pyramid Pooling) module to pool the fused features to obtain pooled features;

[0022] Set up feature sampling units: use the FPN (Feature Pyramid Network) module to upsample the pooled features to obtain a high-dimensional feature map; use the PAN (Path Aggregation Network) module to downsample the high-dimensional features to obtain a low-dimensional feature map; integrate the high-dimensional feature map and the low-dimensional feature map to obtain a sampled feature map;

[0023] Set the insulator positioning unit: Use s 2 The sampling feature map is divided into grids; the Sigmoid function is used to predict whether each grid contains an insulator; the minimum bounding rectangle containing the insulator grid is fitted to obtain the insulator identification box B; the scale parameters and category parameters of the true value box G are extracted from the annotation file; the coordinate offset (σ) of the center point of the insulator identification box B relative to the true value box G is recorded. x ,σ y ), width scaling σ w and high scaling σ h ; Calculate shape intersection-over-union (IOU) B,G And the position loss function l box ;

[0024] Set up the ice classification unit: use the Softmax function to predict the category probability distribution Pr(O) of the insulator in the recognition box B; calculate the classification loss function l of the insulator cls ; Calculate the classification recognition confidence f dAnd the confidence loss function l obj ;

[0025] Set up the transfer learning unit: load the classification detection network from the source domain D S The learned source domain parameters θ S ; Input target domain D T K' new images are added; preprocessing, feature extraction and feature sampling are performed on the K' new images to obtain new features; the parameters of the classification detection network are fine-tuned with the new features to make the position loss function l box , classification loss function l cls And the confidence loss function l obj Descend until convergence and obtain the target domain parameter θ T ; The source domain parameters θ of the classification detection network S Update to the target domain parameters θ T .

[0026] Furthermore, the objective function of transfer learning in the transfer learning unit is:

[0027] logp(θ T |D T )=logp(D T |θ T )+logp(θ S |D S )+logp(D S )

[0028] Where, p(θ S |D S ) is the source domain parameter θ S In the source domain D S The posterior distribution function, p(θ T |D T ) is the target domain parameter θ T In the target domain D T The posterior distribution function, p(D T |θ T ) is the target domain parameter θ T In the target domain D T The conditional distribution function, p(D S ) is the source domain D S The marginal distribution function of .

[0029] Furthermore, the shape intersection-over-union (IOU) B,G The calculation formula is:

[0030]

[0031] The position loss function l box The calculation formula is:

[0032] 1-IOU B,G

[0033] Where a(BI G) and a(BUG) are the intersection area and union area of ​​B and G, respectively; d is the diagonal length of the minimum circumscribed rectangle of B and G; W and H are the width and height of the input image, respectively.

[0034] The classification loss function l cls The calculation formula is:

[0035] l cls =-∑C i 'log(C i )-(1-C i ')log(1-C i )

[0036] The classification recognition confidence f d The calculation formula is:

[0037] f d =Pr(O)×IOU B,G

[0038] The confidence loss function l obj The calculation formula is:

[0039] l obj =-Σf d,G log(f d,B )-(1-f d,G )log(1-f d,B )

[0040] Where, f d,B and f d,G are the recognition confidence of the identification frame B and the ground truth frame G of the insulator respectively.

[0041] Furthermore, the positions of the insulators are marked to establish a first data set; and the first data set is used to fine-tune a pre-trained classification detection network to obtain an insulator recognition model, including:

[0042] Mark the position of the insulator: Take K1 first-labeled images from the insulator ice monitoring image library; traverse the K1 first-labeled images and mark the insulator string in the image with a rectangular box labeled "01-insulator" to obtain the coordinates of the upper left corner vertex (x1, y1) and the lower right corner vertex coordinates (x2, y2) of the insulator truth box G;

[0043] Establish the first data set: Calculate the scale parameters of the insulator truth frame G, including: the center point horizontal coordinate truth value G x , the true value of the vertical coordinate G y , width truth value Gw and the height truth value G h ; Convert the label "01-insulator" to the category parameter "0" of the ground truth box G; write the scale parameter and category parameter of the insulator ground truth box G into a TXT format file with the same name as the annotated image to obtain the first annotation set; store the first annotation set and K1 annotated images in the first path to obtain the first data set;

[0044] Fine-tune the pre-trained classification detection network: load the pre-trained parameters θ0 learned by the transferable classification detection network on the MS COCO dataset; input the first dataset and take the number of categories i = 1; use the transfer learning unit to fine-tune the pre-trained parameters θ0 for I1 rounds to obtain the first parameters θ1; update the pre-trained parameters θ0 of the classification detection network to the first parameters θ1 to obtain the insulator recognition model.

[0045] Furthermore, the step of marking whether the insulator is covered with ice and establishing a second data set; and fine-tuning the insulator recognition model using the second data set to obtain an insulator icing judgment model includes:

[0046] Labeling insulators to see if they are covered with ice: Take K2 second-annotated images from the first dataset; traverse the K2 second-annotated images to determine whether the insulators in the annotated frame are covered with ice, and replace the labels of ice-free insulators with "01-none" and iced insulators with "02-iced", respectively.

[0047] Create the second dataset: Export a TXT file with the same name as the K2 second annotated images from the first dataset; convert the label "02-iced" to the category parameter "1" of the ground truth box G; replace the category parameter of iced insulators with "1" in the TXT file to obtain the second annotated set; store the second annotated set and the K2 annotated images in the second path to obtain the second dataset;

[0048] Fine-tune the insulator identification model: load the insulator identification model and the first parameter θ1; input the second data set and take the number of classifications i = 2; use the transfer learning unit to fine-tune the first parameter θ1 for I2 rounds to obtain the second parameter θ2; update the first parameter θ1 of the classification detection network to the second parameter θ2 to obtain the insulator icing judgment model.

[0049] Furthermore, the icing types of the iced insulators are marked to establish a third data set; the insulator icing judgment model is fine-tuned using the third data set to obtain an insulator icing classification model, including:

[0050] Label the icing type of insulators: From the second dataset, obtain K3 third-annotated images of iced insulators (class parameter is "1"). Traverse the K3 third-annotated images to determine the icing type of the insulators, and replace the labels of snow-covered insulators, rime-covered insulators, mixed rime-covered insulators, and rain rime-covered insulators with "02-snow", "03-rime", "04-mixed", and "05-glaze", respectively.

[0051] Create a third dataset: Export a TXT file with the same name as the K3 third annotated images from the second dataset; convert the labels "02-snow", "03-rime", "04-mixed", and "05-glaze" into the category parameters "1", "2", "3", and "4" of the ground truth box G, respectively, to obtain the third annotated set; store the third annotated set and the K3 annotated images in a third path to obtain the third dataset;

[0052] Fine-tune the insulator icing judgment model: load the insulator icing judgment model and the second parameter θ2; input the third data set and take the classification number i = 5; use the transfer learning unit to fine-tune the second parameter θ2 for I3 rounds to obtain the third parameter θ3; update the second parameter θ2 of the classification detection network to the third parameter θ3 to obtain the insulator icing classification model.

[0053] Furthermore, the use of the insulator ice classification model to identify the insulator visible light monitoring image to assess the ice thickness of the insulator includes:

[0054] Identify insulator visible light monitoring images: Input the real-time collected insulator visible light monitoring images into the insulator icing classification model; output the labels of the insulator icing types and the classification confidence, including: "no ice", "snow", "rime", "mixed rime", and "rain rime"; output the center point coordinates of the insulator identification box B (B x ,B y ), and the height truth value B h ;

[0055] Assess the thickness of ice on insulators: When an insulator is iced, extract the width B of the identification box of the same insulator in the image when it is ice-free and ice-covered. w0 、B w1 ; Calculate the ice thickness I of the insulator under different ice types; When the ice thickness exceeds the thickness threshold, an early warning signal is issued.

[0056] Furthermore, the calculation formula for the standard ice thickness is:

[0057]

[0058] Where ρ is the ice density. When the ice type is ice-free, ρ = 0 g / cm 3, when the icing type is snow, take ρ=0.4g / cm 3 , when the icing type is rime, take ρ=0.3g / cm 3 , when the icing type is mixed rime, take ρ=0.6g / cm 3 , when the icing type is glaze, take ρ=0.9g / cm 3 .

[0059] Further, when the ambient temperature is lower than 0 DEG C, the operating environment of the insulator is set as ice and snow weather, and the insulator icing classification monitoring system is started to operate; when the ambient temperature is higher than 0 DEG C, the operating environment of the insulator is set as normal weather, and the insulator icing classification monitoring system is stopped to operate.

[0060] The second technical scheme adopted by the application is:

[0061] An electronic device, comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the insulator equivalent icing thickness classification monitoring method as described above.

[0062] The third technical scheme adopted by the application is:

[0063] A computer readable storage medium, the storage medium storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by a processor to implement the insulator equivalent icing thickness classification monitoring method as described above.

[0064] The fourth technical scheme adopted by the application is:

[0065] A computer program product or computer program, comprising computer instructions stored in a computer readable storage medium. The processor of the computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the insulator equivalent icing thickness classification monitoring method as described above.

[0066] Compared with the prior art, the application has the following beneficial effects:

[0067] (1) The application designs a phased transfer learning strategy, which realizes high-precision identification of the icing type of the insulator on the basis of identifying the insulator and judging whether it is iced. Under the premise of saving monitoring computing resources, the applicability of the insulator icing classification model under variable environmental conditions is improved.

[0068] (2) The method and system provided by the present invention can monitor the operating insulators in real time, automatically clean and process the visible light monitoring images of the insulators in real scenes, greatly reducing manual intervention, lowering the difficulty of labeling and improving operation and maintenance efficiency.

[0069] (3) The technical solution provided by the present invention can also automatically convert the ice thickness of insulators based on the image recognition results of insulators under different ice types, providing a scientific decision-making basis for smart grids to timely and accurately monitor the operating status of equipment and respond to extreme weather. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0071] Figure 1 An overall flow chart of a method for classifying and monitoring equivalent ice thickness of insulators based on phased transfer learning provided by an embodiment of the present invention;

[0072] Figure 2 A schematic diagram of constructing an insulator icing classification model based on staged transfer learning provided by an embodiment of the present invention;

[0073] Figure 3 A schematic diagram of manual marking of identification frames for insulators with different ice coverage types provided by an embodiment of the present invention;

[0074] Figure 4 This is a diagram showing the identification results of different ice types on the same insulator in a typical monitoring image provided by an embodiment of the present invention;

[0075] Figure 5 A diagram of the software and hardware architecture of an insulator equivalent ice thickness classification monitoring system based on staged transfer learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0077] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, front, back, left, right and the like, is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0078] In the description of the present application, one or more is meant to be one or more, more than two is meant to be two or more, greater than, less than, more than, etc. are understood to not include the number, above, below, within, etc. are understood to include the number. If the first, second is described, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of technical features indicated.

[0079] In the description of the present application, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be broadly understood, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0080] In view of the problems of large annotation workload, low recognition accuracy, high data dependency and complex calculation caused by complex background, variable ice shape, uneven number of monitoring images, etc. in monitoring the equivalent ice thickness of running insulators in the prior art, the present application provides an insulator equivalent ice thickness classification monitoring scheme based on staged transfer learning.

[0081] Embodiment 1

[0082] As shown in Figure 1 The present embodiment provides an insulator equivalent ice thickness classification monitoring method based on staged transfer learning, which specifically includes the following steps:

[0083] S1, establish an insulator icing monitoring image library: collect and clean insulator icing monitoring images in snowy weather to obtain an insulator icing monitoring image library.

[0084] As an implementation manner, step S1 specifically includes the following steps:

[0085] S11, image acquisition: set a sampling time interval to obtain an image acquisition time; use a visible light camera with a fixed installation position to take pictures of the insulator at the sampling time interval to obtain the insulator icing monitoring images.

[0086] S12. Automatically mark damaged images: Count the pixel values ​​of each pixel in the insulator ice monitoring image, and take the pixels whose red (R), green (G), and blue (B) components of the pixel values ​​satisfy [(0<R<5)∩(0<G<5)∩(0<B<5)] as black dots; when the number of black dots in the image is higher than the set damage threshold, the image is automatically marked as a damaged image.

[0087] S13. Automatically mark blurred images: convert the insulator ice monitoring image into the frequency domain based on the FFT (Fast Fourier Transform) algorithm to obtain a first frequency domain image; set the central low-frequency area of ​​the first frequency domain image to zero to obtain a second frequency domain image; convert the second frequency domain image into the spatial domain based on the IFFT (Inverse Fast Fourier Transform) to obtain a transformation image; calculate the amplitude average of each pixel point in the transformation image to obtain the blurriness; when the blurriness is lower than the set blur threshold, automatically mark the image as a blurred image.

[0088] The expression of the FFT is:

[0089]

[0090] The expression of IFFT is:

[0091]

[0092] Where f(x,y) is the pixel value of the pixel with coordinates (x,y) in the insulator icing monitoring image, F(u,v) is the frequency value of the pixel with coordinates (u,v) in the first frequency domain image, and M and N are the width and height of the insulator icing monitoring image, respectively.

[0093] S14. Establish an image library: remove images automatically marked as damaged or blurred; traverse the insulator icing monitoring images, match the images, file names, and acquisition times, and establish an insulator icing monitoring image library.

[0094] S2. Construct a transferable classification detection network: obtain a classification detection network by sequentially setting a training image preprocessing unit, a feature extraction unit, a feature sampling unit, an insulator positioning unit, and an ice classification unit; obtain a transferable classification detection network by setting a transfer learning unit.

[0095] See also Figure 2 In some embodiments, step S2 includes the following steps:

[0096] S21. Set up a training image preprocessing unit: take K input images from the insulator icing monitoring image library; set the number of preprocessing times to K; randomly take 4 training images for preprocessing each time; randomly scale, flip and splice the training images; crop the spliced ​​4 input images into a square; fill the information-free area to obtain K training images.

[0097] S22. Set up a feature extraction unit: use the Focus module to slice the R, G, and B color feature components of the training image to obtain slice features; use the CSP (Cross Stage Partial Network) module to cross-fuse the high- and low-dimensional features of the training image to obtain fused features; use the SPP (Spatial Pyramid Pooling) module to pool the fused features to obtain pooled features.

[0098] S23. Set up a feature sampling unit: use the FPN (Feature Pyramid Network) module to upsample the pooled features to obtain a high-dimensional feature map; use the PAN (Path Aggregation Network) module to downsample the high-dimensional features to obtain a low-dimensional feature map; integrate the high-dimensional feature map and the low-dimensional feature map to obtain a sampling feature map.

[0099] S24, set the insulator positioning unit: use s 2 The sampling feature map is divided into grids; the Sigmoid function is used to predict whether each grid contains an insulator; the minimum bounding rectangle containing the insulator grid is fitted to obtain the insulator identification box B; the scale parameters and category parameters of the true value box G are extracted from the annotation file; the coordinate offset (σ) of the center point of the insulator identification box B relative to the true value box G is recorded. x ,σ y ), wide scaling σ w and high scaling σ h ; Calculate shape intersection-over-union (IOU) B,G And the position loss function l box .

[0100] The IOU B,G The calculation formula is:

[0101]

[0102] The said box The calculation formula is:

[0103] 1-IOU B,G

[0104] Where a(BI G) and a(BUG) are the intersection area and union area of ​​B and G, respectively; d is the diagonal length of the minimum circumscribed rectangle of B and G; W and H are the width and height of the input image, respectively.

[0105] S25. Set up the ice classification unit: use the Softmax function to predict the class probability distribution Pr(O) of the insulator in the recognition box B; calculate the classification loss function of the insulator:

[0106] l cls =-∑C i 'log(C i )-(1-C i ')log(1-C i )

[0107] Where C i 、C i ' is the probability of whether the predicted category is the true value, i is the number of categories; calculate the classification recognition confidence:

[0108] f d =Pr(O)×IOU B,G

[0109] Calculate the confidence loss function:

[0110] l obj =-∑f d,G log(f d,B )-(1-f d,G )log(1-f d,B )

[0111] Where, f d,B and f d,G are the recognition confidence of the identification frame B and the ground truth frame G of the insulator respectively.

[0112] S26, set up transfer learning unit: load the classification detection network from the source domain D S The learned source domain parameters θ S ; Input target domain D T K' new images are added; preprocessing, feature extraction and feature sampling are performed on the K' new images to obtain new features; the parameters of the classification detection network are fine-tuned with the new features to make the position loss function l box , classification loss function l cls And the confidence loss function l obj Descend until convergence and obtain the target domain parameter θ T ; The source domain parameters θ of the classification detection network S Update to the target domain parameters θ T ; The objective function of the above transfer learning is:

[0113] logp(θ T |D T )=logp(D T |θ T )+logp(θ S |D S )+logp(D S )

[0114] Where, p(θ S |D S ) is the source domain parameter θ S In the source domain D S The posterior distribution function, p(θ T |D T ) is the target domain parameter θ T In the target domain D T The posterior distribution function, p(D T |θ T ) is the target domain parameter θ T In the target domain D T The conditional distribution function, p(D S ) is the source domain D S The marginal distribution function of .

[0115] S3. Transfer learning insulator recognition model: Mark the positions of the insulators, establish the first dataset, and use the dataset to fine-tune the pre-trained classification detection network to obtain the insulator recognition model.

[0116] See also Figure 2 As an optional implementation, step S3 specifically includes:

[0117] S31. Mark the position of the insulator: Take K1 first-annotated images from the insulator ice monitoring image library; traverse the K1 first-annotated images, mark the insulator string in the image with a rectangular box labeled "01-insulator", and obtain the upper left corner vertex coordinates (x1, y1) and the lower right corner vertex coordinates (x2, y2) of the insulator truth box G.

[0118] S32. Establish the first data set: calculate the scale parameters of the insulator true value frame G, including: the true value of the horizontal coordinate of the center point G x , vertical coordinate true value G y , width truth value G w and the height truth value G h ; Convert the label "01-insulator" to the category parameter "0" of the ground truth box G; write the scale parameter and category parameter of the insulator ground truth box G into a TXT format file with the same name as the annotated image to obtain the first annotation set; store the first annotation set and K1 annotated images in the first path to obtain the first data set.

[0119] S33. Fine-tune the pre-trained classification detection network: load the pre-trained parameters θ0 learned by the transferable classification detection network in the MS COCO dataset; input the first dataset and take the classification number i=1; use the transfer learning unit to fine-tune the pre-trained parameters θ0 for I1 rounds to obtain the first parameters θ1; update the pre-trained parameters θ0 of the classification detection network to the first parameters θ1 to obtain the insulator recognition model.

[0120] S4. Transfer learning of insulator icing judgment model: Label whether the insulator is covered with ice, establish a second data set, and use this data set to fine-tune the insulator recognition model to obtain the insulator icing judgment model.

[0121] See also Figure 2 As an optional implementation, step S4 specifically includes:

[0122] S41. Labeling whether insulators are covered with ice: Take K2 second labeled images from the first dataset; traverse the K2 second labeled images to determine whether the insulators in the labeled frame are covered with ice, and replace the labels of ice-free insulators with "01-none" and iced insulators with "02-iced", respectively.

[0123] S42. Establish a second data set: export a TXT file with the same name as the K2 second annotated images from the first data set; convert the label "02-iced" to the category parameter "1" of the truth box G; replace the category parameter of iced insulators with "1" in the TXT file to obtain a second annotated set; store the second annotated set and the K2 annotated images in a second path to obtain a second data set.

[0124] S43. Fine-tune the insulator identification model: load the insulator identification model and the first parameter θ1; input the second data set and take the number of classifications i=2; use the transfer learning unit to fine-tune the first parameter θ1 for I2 rounds to obtain the second parameter θ2; update the first parameter θ1 of the classification detection network to the second parameter θ2 to obtain the insulator icing judgment model.

[0125] S5. Transfer learning insulator icing classification model: Label the icing types of iced insulators, establish a third dataset, and use this dataset to fine-tune the icing judgment model to obtain the insulator icing classification model.

[0126] See also Figure 2 As an optional implementation, step S5 specifically includes:

[0127] S51. Label the icing type of the insulator: From the second dataset, obtain K3 third labeled images of ice-covered insulators (category parameter is "1"); traverse the K3 third labeled images to determine the icing type of the insulator, and replace the labels of snow-covered insulators, rime-covered insulators, mixed rime-covered insulators, and rain rime-covered insulators with "02-snow", "03-rime", "04-mixed", and "05-glaze", respectively. Figure 3 shown.

[0128] S52. Establish a third data set: export a TXT file with the same name as the K3 third annotated images from the second data set; convert the labels "02-snow", "03-rime", "04-mixed", and "05-glaze" into the category parameters "1", "2", "3", and "4" of the ground truth box G, respectively, to obtain a third annotated set; store the third annotated set and the K3 annotated images in a third path to obtain a third data set.

[0129] S53. Fine-tune the insulator icing judgment model: load the insulator icing judgment model and the second parameter θ2; input the third data set and take the number of classifications i=5; use the transfer learning unit to fine-tune the second parameter θ2 for I3 rounds to obtain the third parameter θ3; update the second parameter θ2 of the classification detection network to the third parameter θ3 to obtain the insulator icing classification model.

[0130] S6. Monitor the equivalent ice thickness of insulators: Use the insulator ice classification model to identify the visible light monitoring image of insulators and evaluate the ice thickness of insulators.

[0131] See also Figure 2 As an optional implementation, step S6 specifically includes:

[0132] S61. Identify the visible light monitoring image of the insulator: input the real-time collected visible light monitoring image of the insulator into the insulator icing classification model; output the label of the insulator icing type and the classification confidence, the labels include: "no ice", "snow", "rime", "mixed rime", "rain rime", etc. Figure 4 Output the center point coordinates of the insulator identification frame B (B x ,B y ), and the height truth value B h .

[0133] S62. Evaluate the ice thickness of the insulator: When the insulator is iced, extract the width B of the identification box of the same insulator in the image when it is ice-free and ice-covered. w0 、B w1 ; Calculate the ice thickness I of the insulator under different ice types; When the ice thickness exceeds the thickness threshold, an early warning signal is issued.

[0134] Specifically, the calculation formula for the standard ice thickness is:

[0135]

[0136] Where ρ is the ice density. When the ice type is ice-free, ρ = 0 g / cm 3 When the ice type is snow, take ρ = 0.4g / cm 3 When the ice type is rime, take ρ = 0.3g / cm 3 When the icing type is mixed rime, take ρ = 0.6 g / cm 3 When the ice type is rime, take ρ = 0.9 g / cm 3 .

[0137] Example 2

[0138] This embodiment provides an insulator equivalent ice thickness classification monitoring system based on staged transfer learning. When the ambient temperature is lower than 0°C, the operating environment of the insulator is set to snowy weather, and the insulator ice classification monitoring system is started; when the ambient temperature is higher than 0°C, the operating environment of the insulator is set to normal weather, and the insulator ice classification monitoring system is stopped.

[0139] In some embodiments, see Figure 5 The insulator equivalent ice thickness classification monitoring system based on phased transfer learning includes the following software units:

[0140] Insulator image cleaning software unit: used to clean the collected insulator visible light monitoring images;

[0141] Insulator and icing type recognition software unit: used to identify insulators in images and determine icing types;

[0142] Insulator equivalent ice thickness monitoring software unit: used to calculate the equivalent ice thickness of the insulator according to the ice type of the insulator;

[0143] Human-computer interaction software unit: used to display monitoring results and alarm information, and provide a system operation interface.

[0144] In some embodiments, see Figure 5 The insulator equivalent ice thickness classification monitoring system based on phased transfer learning includes the following hardware:

[0145] at least one set of cameras capable of capturing visible light images of insulators at set sampling time intervals;

[0146] At least one processor, configured to run the software unit described above;

[0147] At least one memory is used to store computer-readable instructions and insulator ice classification monitoring results.

[0148] Since the system is an insulator equivalent ice thickness classification monitoring system based on phased transfer learning in an embodiment of the present invention, and the principle of solving the problem by the system is similar to that of the method, the implementation of the system can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0149] Example 3

[0150] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following Figure 1 A classification monitoring method for insulator equivalent ice thickness based on staged transfer learning is shown.

[0151] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0152] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0153] Since the electronic device is an electronic device corresponding to an insulator equivalent ice thickness classification monitoring method based on phased transfer learning in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0154] Example 4

[0155] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 A classification monitoring method for insulator equivalent ice thickness based on staged transfer learning is shown.

[0156] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0157] Since the storage medium is a storage medium corresponding to an insulator equivalent ice thickness classification monitoring method based on phased transfer learning in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0158] Example 5

[0159] In some possible implementations, various aspects of the methods of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is used to cause the computer device to execute the steps of the insulator equivalent ice thickness classification and monitoring method based on staged transfer learning according to various exemplary embodiments of the present application as described above in this specification. The executable computer program code or "code" used to implement the various embodiments may be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0160] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0161] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0162] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A classification monitoring method for equivalent ice thickness of insulators, characterized in that: The following steps are involved: Establish an image library for monitoring insulator icing; Build and train transferable classification and detection networks; Marking the positions of the insulators to establish a first data set; using the first data set to fine-tune the pre-trained classification detection network to obtain an insulator recognition model; Mark whether the insulator is covered with ice and establish a second data set; use the second data set to fine-tune the insulator recognition model to obtain an insulator icing judgment model; Mark the icing type of iced insulators and create a third data set; The third dataset is used to fine-tune the insulator icing judgment model and obtain the insulator icing classification model. An insulator icing classification model is used to identify insulator visible light monitoring images to assess the insulator icing thickness. The classification detection network includes an image preprocessing unit, a feature extraction unit, a feature sampling unit, an insulator positioning unit and an ice classification unit; The construction and training of a transferable classification detection network includes: Set up a training image preprocessing unit: take K input images from the insulator icing monitoring image library; set the number of preprocessing times to K; randomly take multiple training images for preprocessing each time; randomly scale, flip, and splice the training images; Crop the stitched input images into squares and fill the uninformative areas to obtain K training images. Set up a feature extraction unit: use the Focus module to slice the R, G, and B color feature components of the training image to obtain slice features; cross-fuse the high- and low-dimensional features of the training image to obtain fused features; perform pooling on the fused features to obtain pooled features; Set up feature sampling units: upsample the pooled features to obtain a high-dimensional feature map; downsample the high-dimensional features to obtain a low-dimensional feature map; integrate the high-dimensional feature map and the low-dimensional feature map to obtain a sampling feature map; Set up the insulator positioning unit: divide the sampling feature map; predict whether each grid contains an insulator; fit the minimum circumscribed rectangle containing the insulator grid to obtain the insulator identification box B; extract the scale parameters and category parameters of the true value box G from the annotation file; record the coordinate offset (σ) of the center point of the insulator identification box B relative to the true value box G x ,σ y ), width scaling σ w and high scaling σ h ; Calculate shape intersection-over-union (IOU) B,G And the position loss function l box ; Set up the ice classification unit: predict the class probability distribution Pr(O) of the insulator in the recognition box B; calculate the classification loss function l of the insulator cls ; Calculate the classification recognition confidence f d And the confidence loss function l obj ; Set up the transfer learning unit: load the classification detection network from the source domain D S The learned source domain parameters θ S ; Input target domain D T K' new images are added; preprocessing, feature extraction and feature sampling are performed on the K' new images to obtain new features; the parameters of the classification detection network are fine-tuned with the new features to make the position loss function l box , classification loss function l cls And the confidence loss function l obj Descend until convergence and obtain the target domain parameter θ T ; The source domain parameters θ of the classification detection network S Update to the target domain parameters θ T .

2. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1, characterized in that: The step of establishing an insulator icing monitoring image library includes: Image acquisition: Set the sampling time interval to obtain the image acquisition time; use a visible light camera installed in a fixed position to shoot the insulator at the sampling time interval to obtain the insulator ice monitoring image; Automatically mark damaged images: Count the pixel values ​​of each pixel in the insulator ice monitoring image and define the pixels whose red, green, and blue components meet preset conditions as black dots. When the number of black dots in the image exceeds the set damage threshold, the image is automatically marked as damaged. Automatically mark blurred images: The insulator ice monitoring image is converted to the frequency domain using the FFT algorithm to obtain a first frequency domain image; the central low-frequency region of the first frequency domain image is set to zero to obtain a second frequency domain image; the second frequency domain image is converted to the spatial domain using the IFFT algorithm to obtain a transformation image; the amplitude average of each pixel in the transformation image is calculated to obtain the degree of blur; when the degree of blur is lower than a set blur threshold, the image is automatically marked as a blurred image; Establish an image library: remove images automatically marked as damaged or blurred; traverse the insulator icing monitoring images, match the images, file names and acquisition times, and establish an insulator icing monitoring image library.

3. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1, characterized in that: The objective function of transfer learning in the transfer learning unit is: logp(θ T |D T )=logp(D T |θ T )+logp(θ S |D S )+logp(D S ) Where, p(θ S |D S ) is the source domain parameter θ S In the source domain D S The posterior distribution function, p(θ T |D T ) is the target domain parameter θ T In the target domain D T The posterior distribution function, p(D T |θ T ) is the target domain parameter θ T In the target domain D T The conditional distribution function of p(D S ) is the source domain D S The marginal distribution function of .

4. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1, characterized in that: The shape intersection-over-union (IOU) B,G The calculation formula is: The position loss function l box The calculation formula is: 1-IOU B,G Where a(B∩G) and a(B∪G) are the intersection area and union area of ​​B and G respectively, d is the diagonal length of the minimum circumscribed rectangle of B and G, W and H are the width and height of the input image respectively; The classification loss function l cls The calculation formula is: l cls =-∑C i 'log(C i )-(1-C i ')log(1-C i ) The classification recognition confidence f d The calculation formula is: f d =Pr(O)×IOU B,G The confidence loss function l obj The calculation formula is: l obj =-∑f d,G log(f d,B )-(1-f d,G )log(1-f d,B ) Where, f d,B and f d,G are the recognition confidence of the identification frame B and the ground truth frame G of the insulator respectively.

5. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1, characterized in that: The steps of marking the positions of the insulators to establish a first data set and fine-tuning a pre-trained classification detection network using the first data set to obtain an insulator recognition model include: Take K1 first-labeled images from the insulator ice monitoring image library; traverse the K1 first-labeled images and label the insulator string in the image with a rectangular box labeled "01-insulator" to obtain the coordinates of the upper left corner vertex (x1, y1) and the lower right corner vertex coordinates (x2, y2) of the insulator truth box G; Calculate the scale parameters of the insulator true value frame G, including: the center point horizontal coordinate true value G x , the true value of the vertical coordinate G y , width truth value G w and the height truth value G h ; Convert the label "01-insulator" to the category parameter "0" of the ground truth box G; write the scale parameter and category parameter of the insulator ground truth box G into a TXT format file with the same name as the annotated image to obtain the first annotation set; store the first annotation set and K1 annotated images in the first path to obtain the first data set; Load the pre-trained parameters θ0 learned by the transferable classification detection network on the MS COCO dataset; input the first dataset and take the number of classifications i = 1; use the transfer learning unit to fine-tune the pre-trained parameters θ0 for I1 rounds to obtain the first parameters θ1; update the pre-trained parameters θ0 of the classification detection network to the first parameters θ1 to obtain the insulator recognition model.

6. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1, characterized in that: marking whether the insulator is covered with ice and establishing a second data set; The second dataset is used to fine-tune the insulator identification model to obtain an insulator icing judgment model, including: From the first dataset, K2 second annotated images are obtained; the K2 second annotated images are traversed to determine whether the insulators in the annotated frame are covered with ice, and the labels of ice-free insulators and ice-covered insulators are replaced with "01-none" and "02-iced", respectively; a TXT file with the same name as the K2 second annotated images is exported from the first dataset; the label "02-iced" is converted to the category parameter "1" of the ground truth frame G; the category parameter of ice-covered insulators is replaced with "1" in the TXT file to obtain a second annotated set; the second annotated set and the K2 annotated images are stored in a second path to obtain a second dataset; Fine-tune the insulator identification model: load the insulator identification model and the first parameter θ1; input the second data set and take the number of classifications i = 2; use the transfer learning unit to fine-tune the first parameter θ1 for I2 rounds to obtain the second parameter θ2; update the first parameter θ1 of the classification detection network to the second parameter θ2 to obtain the insulator icing judgment model.

7. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1, characterized in that: The ice type of the ice-covered insulator marked is used to establish a third data set; The third dataset was used to fine-tune the insulator icing judgment model, and an insulator icing classification model was obtained, including: From the second dataset, K3 third-annotated images of iced insulators are obtained; the K3 third-annotated images are traversed to determine the icing type of the insulators, and the labels of snow-covered insulators, rime-covered insulators, mixed rime-covered insulators, and rain rime-covered insulators are replaced with "02-snow", "03-rime", "04-mixed", and "05-glaze", respectively; Export a TXT file with the same name as the K3 third annotated images from the second dataset; convert the labels "02-snow", "03-rime", "04-mixed", and "05-glaze" into the category parameters "1", "2", "3", and "4" of the ground truth box G, respectively, to obtain the third annotated set; store the third annotated set and the K3 annotated images in a third path to obtain the third dataset; Fine-tune the insulator icing judgment model: load the insulator icing judgment model and the second parameter θ2; input the third data set and take the classification number i = 5; use the transfer learning unit to fine-tune the second parameter θ2 for I3 rounds to obtain the third parameter θ3; update the second parameter θ2 of the classification detection network to the third parameter θ3 to obtain the insulator icing classification model.

8. The method for classifying and monitoring equivalent ice thickness of insulators according to claim 1, characterized in that: The insulator ice classification model is used to identify the insulator visible light monitoring image to evaluate the ice thickness of the insulator, including: Input the real-time collected insulator visible light monitoring image into the insulator icing classification model; output the label of the insulator icing type and the classification confidence; output the center point coordinates of the insulator identification box B (B x ,B y ), and the height truth value B h ; When the insulator is covered with ice, the width B of the identification box of the same insulator in the image when it is ice-free and ice-covered is extracted. w0 、B w1 ; Calculate the ice thickness I of the insulator under different ice types; When the ice thickness exceeds the thickness threshold, an early warning signal is issued.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

Images