A line icing detection method based on deep learning

Abstract

The invention discloses a line icing detection method based on deep learning, which belongs to the technical field of image processing. The method comprises the following steps: obtaining an original icing image, extracting icing features in the image, extracting global features of the icing image in combination with a transfer learning model, fusing multi-channel extracted features to obtain an icing type recognition result of the image, and obtaining an icing density in a current state. The icing area in the segmented image is detected, the model segmentation result is optimized, and the horizontal icing thickness and the vertical icing thickness are inferred using the segmented icing area. The equivalent icing thickness of the line in the current icing state is calculated using an equivalent area method, and different equivalent icing thickness optimization calculation formulas are used according to the icing environment of the line, so as to adapt to the icing detection requirements of the line in different environments. The invention can improve the real-time and stability of icing detection through image processing and analysis based on deep learning, thereby helping to reduce the icing threat of various lines, reduce maintenance costs, and ensure stable operation.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for detecting ice coating on a line. Background Art

[0002] Icing disaster is one of the common natural disasters in life. In severe cases, it will cause accidents such as power line flashover and power outage, tower collapse, etc., posing a threat to the safe and stable operation of key infrastructure such as power grids. The current methods of ice detection can be mainly divided into two categories: actual field testing and remote image detection. Actual field measurements mainly include direct measurement methods, weighing methods, etc., and these methods almost all rely on line patrol personnel to manually measure information such as line ice thickness, or require additional devices such as tension sensors to be installed on monitoring facilities. These methods generally have problems such as large workload, high cost and complex operation, so they are difficult to be flexibly applied to line ice status monitoring in actual environments.

[0003] The image-based ice detection method collects and analyzes ice images by installing image acquisition equipment on poles or other monitoring facilities to determine the ice status of the line. Among them, there is a method of using drones to obtain line ice images from multiple perspectives to measure the ice thickness. Although drones can be flexibly used to obtain line ice status from more perspectives, the implementation cost is high and it is difficult to achieve all-weather real-time monitoring; there is also a method of extracting line edge information from line ice images using wavelet transform and morphological edge detection methods to confirm the ice status, but this method is easily affected by the environmental background; in addition, the method of using median filtering and image enhancement technology to improve image quality and then using the Canny edge detection algorithm to identify the line edge can improve some situations, but it is still difficult to completely avoid the interference of complex background noise.

[0004] In the task of line icing detection based on deep learning, the performance of identifying and detecting ice-covered areas in ice-covered images based on semantic segmentation models is significantly better than edge detection algorithms in eliminating background noise interference. Moreover, most methods based on images to obtain ice thickness information fail to consider the impact of different types of ice density, and different types of ice have significantly different effects on the line with the same ice thickness. Making full use of ice density information is crucial for applications in practical environments. The convolutional neural network model based on deep learning can well judge the type of ice in the image, thereby providing important density information for the calculation of ice thickness, and obtaining ice thickness at different angles through different viewing angles can better infer the uniform ice thickness surrounding the line. Therefore, making full use of the deep learning model can significantly improve the performance of line ice thickness detection. Summary of the invention

[0005] The problem to be solved by the present invention is to provide a line icing detection method in view of the shortcomings of the background technology, including the identification of icing types, calculating the equivalent icing thickness of the line using semantic segmentation technology, and optimizing and adjusting the line icing thickness calculation method for nighttime and when the line is invisible from a side view, so as to realize line icing detection in complex environments.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] A line ice detection method based on deep learning specifically comprises the following steps:

[0008] Step S1, obtaining the original ice image of the line taken by the ice monitoring equipment, and making a relevant data set;

[0009] Step S2, using histogram equalization to enhance the brightness features in the original ice-covered image, taking the enhanced image as a brightness feature map, and obtaining its average brightness value and brightness standard deviation, respectively encoding them into feature tensors consistent with the dimensions of the brightness feature map, and splicing them with the brightness feature map in the channel dimension to obtain a composite brightness feature map;

[0010] Step S3, using the horizontal local binary pattern H-LBP to process the original ice-covered image to obtain its roughness texture feature map, and using the gray level co-occurrence matrix to obtain the contrast and homogeneity information of the original ice-covered image, respectively encoding them into feature tensors consistent with the dimensions of the roughness texture feature map, and splicing them with the roughness texture feature map in the channel dimension to obtain a composite roughness feature map;

[0011] Step S4, constructing a multi-branch line ice type recognition model IceNet-T, including a trunk branch, a brightness branch and a roughness branch, inputting the original ice image into the trunk branch, inputting the composite brightness feature map into the brightness branch, inputting the composite roughness feature map into the roughness branch, fusing the multi-branch extraction features, and obtaining the ice type recognition result;

[0012] Step S5, using the semantic segmentation model SCTNet to detect and segment the ice-covered line area in the original ice-covered image, and combining the segmentation results of the multi-scale conditional random field MSCRF optimization model to improve the segmentation accuracy; wherein MSCRF is Multi-Scale Conditional Random Field;

[0013] Step S6: According to the segmentation results of the main view and the side view lines in the original ice image by the semantic segmentation model, the horizontal ice thickness and the vertical ice thickness are calculated respectively. At the same time, the ice density is obtained by combining the ice type recognition result. The equivalent ice thickness of the current line is calculated using the equivalent area method, which includes the optimization calculation of the environmental conditions such as visible from the side view, invisible from the side view, and low light at night. The horizontal ice thickness is the long diameter. a ; Vertical ice thickness is short diameter b .

[0014] As a further preferred solution of the line icing detection method based on deep learning of the present invention, in step S2, a composite brightness feature map is obtained, which specifically includes the following steps:

[0015] Step S2.1: Use the histogram equalization method to enhance the brightness feature map in the original ice-covered image. The enhancement formula is as follows: ;in, is the position in the image after equalization The gray value of the pixel at is the position in the original ice image The gray value of the pixel at L is the number of gray levels, where b bit image, L =2 b , is the gray value of the original ice-covered image The number of pixels, N is the total number of pixels in the image, The gray value of the original ice-covered image is less than or equal to The cumulative distribution function of pixels;

[0016] Calculate the average brightness value and brightness standard deviation of the original ice-covered image using the following formula: ;in, represents the average brightness value, represents the brightness standard deviation, H.W. and C are the height, width and number of channels of the input image, Indicates the input image The brightness value corresponding to the position;

[0017] Step S2.2: Construct a feature tensor consisting of the average brightness value and the brightness standard deviation, that is, expand these two values ​​into a matrix of the same size as the image, and concatenate them with the original image data in the channel dimension. The formula is as follows: ;in, Represents the average brightness value feature tensor consistent with the original image dimension, Represents the brightness standard deviation feature tensor consistent with the original image dimension, Represents a matrix with all elements set to 1. The average brightness feature tensor and brightness standard deviation feature tensor after the expansion dimension are concatenated with the brightness feature map in the channel dimension. The formula is as follows: in, represents the final composite brightness feature tensor, Concat represents the channel dimension concatenation function, They respectively represent the enhanced brightness feature map, the feature tensor of the average brightness value after dimension expansion, and the feature tensor of the brightness standard deviation.

[0018] As a further preferred solution of the line icing detection method based on deep learning of the present invention, in step S3, a composite roughness feature map is obtained, which specifically includes the following steps:

[0019] Step S3.1, use the horizontal local binary pattern H-LBP to process the original ice-covered image to obtain its roughness texture feature map. The horizontal local binary pattern focuses on the texture information in the horizontal direction. The formula is as follows: in, Indicates location The H-LBP calculation results at Indicates the position in the image The pixel value at P is the horizontal neighborhood range, is a binarization function, that is: ;

[0020] The H-LBP formula consists of two parts. The left side of the plus sign is a comparison between the target pixel and the horizontal neighboring pixels on the left, and the right side of the plus sign is a comparison between the target pixel and the horizontal neighboring pixels on the right. The comparison results are binarized and assigned weights. , thus encoding the result into a binary number, the result of H-LBP Is an integer representing the horizontal texture mode, the final H×W H-LBP feature map calculated from an image of size T It can be expressed as: ;

[0021] Step S3.2, in addition to the roughness texture feature map obtained by H-LBP, the composite roughness texture feature map also includes the feature tensor formed by expanding the contrast and homogeneity calculated using the gray level co-occurrence matrix, where the contrast can characterize the roughness of the ice-covered image, and the homogeneity can characterize the smoothness of the ice-covered image. The specific calculation formula is as follows: ;in, Contrast The contrast ratio is Ct , Homogeneity Homogeneity Hg , LRepresents the number of gray levels, is the gray-level co-occurrence matrix, which indicates that the gray value is x and y The joint occurrence probability of pixel pairs at a certain distance and direction;

[0022] Step S3.3: Obtaining the roughness texture feature map T And the contrast of the original ice image Ct and homogeneity Hg After that, you need to Ct and Hg Expand to T The same dimension, so as to concatenate it with the channel dimension, the expanded formula is as follows: ;in, H and W is the roughness texture feature map T The height and width dimensions, represents the contrast feature tensor after the expanded dimension, represents the homogeneity feature tensor after the expanded dimension, is a matrix with all elements set to 1. The roughness texture feature map is concatenated with the contrast and homogeneity feature tensors after encoding and expansion in the channel dimension. The formula is as follows: ;in, Represents the generated composite roughness feature map, Concat is the channel dimension concatenation function, and the concatenated It can enhance the image's ability to express texture features.

[0023] As a further preferred embodiment of the line icing detection method based on deep learning of the present invention, in step S4, a three-branch icing type recognition model IceNet-T is constructed for the composite brightness feature map and the composite roughness feature map extracted in steps S2 and S3, combined with the original icing image, and the final icing type recognition result is obtained by extracting and fusing the three-branch results, which specifically includes the following steps:

[0024] Step S4.1, input the original ice-covered image into the trunk branch. The trunk branch is composed of the main feature extraction network of the deep learning transfer model, and its output layer structure is fine-tuned to adapt it to the ice type recognition task. The original ice-covered image is input into the trunk branch to extract the global ice-covered features. The extraction process can be expressed by the following formula:

[0025] ;in, I represents the input original ice-covered image, It represents the main network feature extraction process of the migration model. Indicates replacing the fully connected output layer of the original migration model. is the new bias term, ReLU represents the ReLU activation function, and finally the global features extracted by the trunk branch are obtained ;

[0026] Step S4.2: Input the composite brightness feature map and the composite roughness feature map into the brightness branch and the roughness branch respectively. In addition to adjusting parameters to adapt to the dimensions of the composite feature maps received by each branch, the network structures used in the feature extraction part are similar, both consisting of 1 initialization module, 4 main feature extraction modules and 1 classifier. The specific feature extraction process can be expressed by the following formula: in, represents the composite brightness feature map, represents the composite roughness feature map, Init Represents the initialization module, M Represents the main feature extraction module. Each main feature extraction module contains an SE channel attention layer. Classifier represents the classifier, the subscripts represent the branches they belong to, and the superscripts represent different modules. Finally, the brightness branch and the roughness branch obtain feature maps respectively. and ; Among them, SE stands for Squeeze-and-Excitation;

[0027] Step S4.3: Obtain the extraction results of the three branches , and Finally, the features extracted by different branches are fused together through the weighted summation method, which can be expressed as follows: in, It represents the features after the fusion of three branches. They represent the weight parameters of the main branch, brightness branch, and roughness branch respectively, and the sum of the weights is 1. Result It represents a probability distribution result obtained by converting the fused features by the Softmax function, where the one with the largest probability is the final ice type recognition result.

[0028] As a further preferred solution of the line ice detection method based on deep learning of the present invention, in step S5, the multi-scale conditional random field MSCRF used to optimize the semantic segmentation result specifically includes the following steps:

[0029] Step S5.1: According to the network structure of SCTNet, semantic segmentation feature maps of different scales are extracted from the multi-layer network of its encoder and decoder. MSCRF refines and optimizes the segmentation results by combining the spatial and color information of the multi-scale feature maps. The multi-scale feature maps extracted by different convolutional layers of the semantic segmentation model have different dimensions. These feature maps need to be adjusted to a uniform size through preprocessing and then combined with the target feature map. Figure 1 The preprocessing process can be expressed as follows: in, Resize is the adjusted multi-scale feature map. Function to adjust the dimension. H and W are the height and width of the target size, n is the number of feature maps that need to be processed, and the final multi-scale feature map can be represented as a set: ;

[0030] Step S5.2: MSCRF inputs the target feature map and multi-scale feature map that need to be optimized P , by minimizing an energy function, the labels of adjacent pixels in the target feature map with similar color and position are made consistent. The segmentation result of the target feature map is optimized through multiple iterative updates. The iterative optimization formula is as follows: in, U express Unary , that is, the target feature map to be optimized, Represents feature maps of different scales, n Indicates the number of feature maps that need to be processed, and Represent spatial Gaussian weight and bilateral Gaussian weight, which can represent the spatial information and color information in the feature map respectively. The Softmax function is used to normalize the weights. k After iterations, the final output optimized feature map for: Among them, the Concat function concatenates the iterative results of feature maps of different scales in the channel dimension, and the Reshape function reshapes the concatenated iterative results so that their channel dimensions are the same as the original input target feature map. U The same, get the final optimization result .

[0031] As a further preferred embodiment of the line icing detection method based on deep learning of the present invention, in step S6, the equivalent ice thickness of the line is calculated using the equivalent area method, including the following steps:

[0032] Step S6.1: Use the semantic segmentation model to segment the main view and side view lines in the original ice-covered image to obtain the segmented areas of the lines and the ice-covered area of ​​the same line under the same view. Its horizontal ice thickness The following relationship exists: ; The diameter of the bare wire at this viewing angle is known , the pixel area of ​​the bare line is obtained by semantic segmentation and the pixel area under ice cover The horizontal ice thickness under ice-covered conditions can be calculated using the following formula: : in, This is the major diameter parameter required in the equivalent area method. a Similarly, the ice thickness in the vertical direction can be obtained by using the segmentation result of the side view line in the original ice image, which is the short diameter parameter required in the equivalent area method. b ;

[0033] Step S6.2, equivalent area method: the irregular cross section of the ice-covered line is equivalent to a standard circle with the same area, and the radius of the standard circle is subtracted from the radius of the bare wire to obtain the equivalent uniform ice thickness of the line. First, the irregular ice-covered cross section is converted into a regular ellipse, and the area of ​​the ellipse is made close to the original irregular cross-sectional area by increasing the adjustment parameters. The ellipse is further equivalent to a standard circle with the same cross-sectional area, and finally the equivalent uniform ice thickness is obtained. The process can be expressed by the following formula: in, is the required equivalent uniform ice thickness, r is the bare wire radius, is the ice density, a is the major diameter parameter, b is the short diameter parameter. The left side of the equation represents the equivalent standard circle area minus the bare wire cross-sectional area. The right side of the equation in brackets represents the ellipse area minus the bare wire cross-sectional area, which represents the regular ice cross-sectional area outside the bare wire. The error is adjusted by the ice density and a constant parameter to obtain the following equivalent uniform ice thickness calculation formula: Among them, ice density It can be calculated based on the identification results of ice cover type;

[0034] Step S6.3: In view of the difficulty of observing the side view line due to low light at night, the equivalent ice thickness calculation formula is optimized and adjusted. The ice cover state of the main view and the side view of the same line at the last moment of the day on the same day is used to infer the ice cover state of the line at the side view at night, and then the short path parameter at night is calculated. ;

[0035] Calculate the ratio of the long and short diameter parameters at the last moment of the day : ;in, is the long-path parameter at the last moment of the day; is the short path parameter at the last moment of the day;

[0036] The nighttime long-path parameters can be obtained based on the ice coverage status of the line from the main viewing angle at night. , combined with the ratio of the long and short diameter parameters at the last moment of the day , and further calculate the short-path parameters at night: ;

[0037] The ratio of the major and minor diameter parameters at the last moment of the day and the long-path parameters at night The optimized calculation formula for equivalent ice thickness at night is obtained: ;

[0038] Step S6.4: For the case where the line from the side view is almost invisible or it is difficult to extract effective information due to the line layout, a large number of statistical observations are made on the linear relationship between the long and short diameters when the line is visible from the side view, and a polynomial function relationship is used to describe the relationship between the long and short diameters when the line is visible from the side view. The ice state from the side view can be inferred from the ice state based on the main view, and a large number of long and short diameter parameter data sets when the line is visible from the side view are collected. ,in N is the sample size, and the polynomial function is defined as follows: in, Indicates the short path parameter when the side view is visible. Indicates the long diameter parameter when the side view is visible. n represents the order of the polynomial, Represents the coefficients of each polynomial; determines the coefficients of the polynomial by the least squares method to make the function close to the data points; uses a large number of Fitting regular functions to data , and then the long diameter parameter is calculated based on the detection under the invisible state of the side view , calculate its short diameter parameter : By regular function and the major diameter parameter The optimization calculation formula for the equivalent ice thickness in the invisible state from the side view is obtained: .

[0039] As a further preferred embodiment of a line icing detection method based on deep learning of the present invention, a line icing detection system is included, wherein the line icing detection system includes: a preprocessing module, an icing type recognition module, an icing segmentation module and an equivalent icing thickness calculation module;

[0040] A preprocessing module is used to obtain the original ice image captured by the ice monitoring equipment, and obtain a composite brightness feature map and a composite roughness feature map through a brightness preprocessing module and a roughness preprocessing module respectively;

[0041] The ice type recognition module is a three-branch convolutional neural network IceNet-T, which includes a trunk branch, a brightness branch, and a roughness branch. It is used to input the original ice image into the trunk branch, and input the composite brightness feature map and composite roughness feature map obtained by preprocessing into the brightness branch and the roughness branch respectively. The ice features extracted by the three branches are integrated to obtain the ice type recognition result.

[0042] The ice segmentation module is used to train the semantic segmentation model SCTNet using the labeled data set, and optimize the ice segmentation results through the multi-scale conditional random field MSCRF in the segmentation result optimization module SegBoost to obtain a more accurate ice area, and extract the pixel area of ​​the ice area in the main view and the ice area in the side view;

[0043] The equivalent ice thickness calculation module is used to obtain the ice density parameters based on the ice type recognition results, and then deduce the major and minor diameter parameters required by the equivalent ice calculation formula based on the pixel areas of the ice areas in the main and side perspectives identified by the ice segmentation module, and then calculate the uniform equivalent ice thickness. The ice thickness calculation formula is optimized and adjusted for nighttime and side perspective routes that are invisible, and finally the equivalent ice thickness calculation results under different environmental conditions are obtained.

[0044] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0045] The line icing detection method of the present invention uses a multi-branch convolutional neural network model to extract and fuse multiple icing features. The brightness branch can focus on the illumination changes and overall brightness patterns in the image, which is helpful to identify the reflection characteristics of different icing types. The roughness branch can capture the details of the surface texture and reflect the physical texture of different icing types. This multi-branch approach ensures the comprehensiveness of feature extraction and improves the ability to distinguish icing types.

[0046] The line ice detection method of the present invention uses semantic segmentation technology to segment and extract the ice-covered area in the image, and uses multi-scale conditional random field MSCRF to extract multi-scale spatial, color and other feature information in the segmentation process, and optimizes the segmentation result. This method can carefully capture the subtle changes in ice coverage, improve the accuracy of the detection results, and improve its performance without increasing the model training cost, so that the pixel area of ​​the ice-covered area can be more accurately extracted, and then the equivalent ice thickness can be accurately calculated;

[0047] The line icing detection method of the present invention can calculate the equivalent icing thickness of the line under normal circumstances, and also realizes the calculation of the equivalent icing thickness at night by utilizing the icing law during the day, thereby ensuring that the icing state of the line can be monitored 24 hours a day; in addition, for the calculation of the equivalent icing thickness in the state where only a single line can be observed and is not visible from a side view, the present invention calculates the horizontal icing thickness (long diameter) in the state where the line is visible from a side view by statistically analyzing the horizontal icing thickness (long diameter) in the state where the line is visible from a side view a ) and vertical ice thickness (short diameter b ), and use the obtained linear relationship to infer the vertical icing state of the line when it is invisible from the side view, and then successfully infer the equivalent icing thickness under this state. This method can also use the icing state from the side view to infer the icing state from the main view, and can make flexible adjustments according to the actual shooting conditions of the icing monitoring equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 This is a flow chart of the line icing detection method of the present invention;

[0050] Figure 2 This is a structural diagram of the overall model of the line icing detection method of the present invention;

[0051] FIG3 (a) is a schematic diagram of a main viewing angle circuit and a side viewing angle circuit in an embodiment of the present invention;

[0052] FIG3 (b) is a schematic diagram of the long and short diameters of the ice-covered cross section of the line in an embodiment of the present invention;

[0053] Figure 4 This is a structural diagram of the IceNet-T model of the line icing type recognition module of the present invention;

[0054] Figure 5 This is a model structure diagram of the ice segmentation module of the present invention;

[0055] Figure 6 It is a comparison curve of verification accuracy and loss value of the IceNet-T model comparison experiment of the present invention;

[0056] Figure 7 This is a diagram showing the effect of the ice segmentation module of the present invention on segmenting the main view and side view lines in the original ice image;

[0057] Figure 8The figure is a display diagram of the detection result of the actual ice-covered image according to the embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the application is further elaborated in detail below in conjunction with the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in the field on this embodiment belong to the protection scope of the present invention. At the same time, for the step numbering in the embodiment of the present invention, it is only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0059] In one embodiment of the present invention, a line icing detection method is as follows: Figure 1 As shown, the following steps are included:

[0060] Step S1, obtaining the original ice image of the line taken by the ice monitoring equipment, and making a relevant data set;

[0061] Step S2, using histogram equalization to enhance the brightness features in the original ice-covered image, taking the enhanced image as a brightness feature map, and obtaining its average brightness value and brightness standard deviation, respectively encoding them into feature tensors consistent with the dimensions of the brightness feature map, and splicing them with the brightness feature map in the channel dimension to obtain a composite brightness feature map, specifically comprising the following steps:

[0062] Step S2.1: Use the histogram equalization method to enhance the brightness feature map in the original ice-covered image. The enhancement formula is as follows: ;in, is the position in the image after equalization The gray value of the pixel at is the position in the original ice image The gray value of the pixel at L is the number of gray levels, where b bit image, L =2 b , is the gray value of the original ice-covered image The number of pixels, N is the total number of pixels in the image, The gray value of the original ice-covered image is less than or equal to The cumulative distribution function of pixels;

[0063] Calculate the average brightness value and brightness standard deviation of the original ice-covered image using the following formula: ;in, represents the average brightness value, represents the brightness standard deviation, H.W. and C are the height, width and number of channels of the input image, Indicates the input image The brightness value corresponding to the position;

[0064] Step S2.2: Construct a feature tensor consisting of the average brightness value and the brightness standard deviation, that is, expand these two values ​​into a matrix of the same size as the image, and concatenate them with the original image data in the channel dimension. The formula is as follows: ;in, Represents the average brightness value feature tensor consistent with the original image dimension, Represents the brightness standard deviation feature tensor consistent with the original image dimension, Represents a matrix with all elements set to 1. The average brightness feature tensor and brightness standard deviation feature tensor after the expansion dimension are concatenated with the brightness feature map in the channel dimension. The formula is as follows: in, represents the final composite brightness feature tensor, Concat represents the channel dimension concatenation function, They respectively represent the enhanced brightness feature map, the feature tensor of the average brightness value after dimension expansion, and the feature tensor of the brightness standard deviation.

[0065] Step S3, using the horizontal local binary pattern (H-LBP) to process the original ice-covered image to obtain its roughness texture feature map, and using the gray level co-occurrence matrix to obtain the contrast and homogeneity information of the original ice-covered image, respectively encoding them into feature tensors consistent with the dimensions of the roughness texture feature map, and splicing them with the roughness texture feature map in the channel dimension to obtain a composite roughness feature map, specifically comprising the following steps:

[0066] Step S3.1, use the horizontal local binary pattern H-LBP to process the original ice-covered image to obtain its roughness texture feature map. The horizontal local binary pattern focuses on the texture information in the horizontal direction. The formula is as follows: in, Indicates location The H-LBP calculation results at Indicates the position in the image The pixel value at P is the horizontal neighborhood range, is a binarization function, that is: ;

[0067] The H-LBP formula consists of two parts. The left side of the plus sign is a comparison between the target pixel and the horizontal neighboring pixels on the left, and the right side of the plus sign is a comparison between the target pixel and the horizontal neighboring pixels on the right. The comparison results are binarized and assigned weights. , thus encoding the result into a binary number, the result of H-LBP Is an integer representing the horizontal texture mode, the final H×W H-LBP feature map calculated from an image of size T It can be expressed as: ;

[0068] Step S3.2, in addition to the roughness texture feature map obtained by H-LBP, the composite roughness texture feature map also includes the feature tensor formed by expanding the contrast and homogeneity calculated using the gray level co-occurrence matrix, where the contrast can characterize the roughness of the ice-covered image, and the homogeneity can characterize the smoothness of the ice-covered image. The specific calculation formula is as follows: ;in, Contrast The contrast ratio is Ct , Homogeneity Homogeneity Hg , L Represents the number of gray levels, is the gray-level co-occurrence matrix, which indicates that the gray value is x and y The joint occurrence probability of pixel pairs at a certain distance and direction;

[0069] Step S3.3: Obtaining the roughness texture feature map T And the contrast of the original ice image Ct and homogeneity Hg After that, you need to Ct and Hg Expand to T The same dimension, so as to concatenate it with the channel dimension, the expanded formula is as follows: ;in, H and W is the roughness texture feature map T The height and width dimensions, represents the contrast feature tensor after the expanded dimension, represents the homogeneity feature tensor after the expanded dimension, is a matrix with all elements set to 1. The roughness texture feature map is concatenated with the contrast and homogeneity feature tensors after encoding and expansion in the channel dimension. The formula is as follows: ;in, Represents the generated composite roughness feature map, Concat is the channel dimension concatenation function, and the concatenated It can enhance the image's ability to express texture features.

[0070] Step S4, constructing a multi-branch line ice type recognition model IceNet-T, including a trunk branch, a brightness branch and a roughness branch, inputting the original ice image into the trunk branch, inputting the composite brightness feature map into the brightness branch, inputting the composite roughness feature map into the roughness branch, fusing the multi-branch extraction features, and obtaining the ice type recognition result, specifically including the following steps:

[0071] Step S4.1, input the original ice-covered image into the trunk branch. The trunk branch is composed of the main feature extraction network of the deep learning transfer model, and its output layer structure is fine-tuned to adapt it to the ice type recognition task. The original ice-covered image is input into the trunk branch to extract the global basic ice-covered features. The extraction process can be expressed by the following formula:

[0072] ;in, I represents the input original ice-covered image, It represents the main network feature extraction process of the migration model. Indicates replacing the fully connected output layer of the original migration model. is the new bias term, ReLU represents the ReLU activation function, and finally the global features extracted by the trunk branch are obtained ;

[0073] Step S4.2: Input the composite brightness feature map and the composite roughness feature map into the brightness branch and the roughness branch respectively. In addition to making necessary parameter adjustments to adapt to the dimensions of the composite feature maps received by each branch, the network structures used in the feature extraction part are similar, both consisting of 1 initialization module, 4 main feature extraction modules and 1 classifier. The specific feature extraction process can be expressed by the following formula: in, represents the composite brightness feature map, represents the composite roughness feature map, Init Represents the initialization module, M Represents the main feature extraction module. Each main feature extraction module contains an SE channel attention layer. Classifier represents the classifier, the subscripts of the above items represent their respective branches, and the superscripts represent different modules. Finally, the brightness branch and the roughness branch obtain feature maps respectively. and ; Among them, SE stands for Squeeze-and-Excitation;

[0074] Step S4.3: Obtain the extraction results of the three branches , and Finally, the features extracted by different branches are fused together through the weighted summation method, which can be expressed as follows: in, It represents the features after the fusion of three branches. They represent the weight parameters of the main branch, brightness branch, and roughness branch respectively, and the sum of the weights is 1. Result It represents a probability distribution result obtained by converting the fused features by the Softmax function, where the one with the largest probability is the final ice type recognition result.

[0075] Step S5, using the semantic segmentation model SCTNet to detect and segment the ice-covered line area in the original ice-covered image, and combining the segmentation results of the multi-scale conditional random field MSCRF optimization model to improve the segmentation accuracy, specifically including the following steps:

[0076] Step S5.1: According to the network structure of SCTNet, semantic segmentation feature maps of different scales are extracted from the multi-layer network of its encoder and decoder. MSCRF refines and optimizes the segmentation results by combining the spatial and color information of the multi-scale feature maps. The multi-scale feature maps extracted by different convolutional layers of the semantic segmentation model have different dimensions. These feature maps need to be adjusted to a uniform size through preprocessing and then combined with the target feature map. Figure 1 The preprocessing process can be expressed as follows: in, Resize is the adjusted multi-scale feature map. Function to adjust the dimension. H and W are the height and width of the target size, n is the number of feature maps that need to be processed, and the final multi-scale feature map can be represented as a set: ;

[0077] Step S5.2: MSCRF inputs the target feature map and multi-scale feature map that need to be optimized P , by minimizing an energy function, the labels of adjacent pixels in the target feature map with similar color and position are made consistent. Through multiple iterative updates, the segmentation result of the target feature map can be optimized. The iterative optimization formula is as follows: in, U express Unary , that is, the target feature map to be optimized, Represents feature maps of different scales, n Indicates the number of feature maps that need to be processed, and Represent spatial Gaussian weight and bilateral Gaussian weight, which can represent the spatial information and color information in the feature map respectively. The Softmax function is used to normalize the weights. kAfter iterations, the final output optimized feature map for: Among them, the Concat function concatenates the iterative results of feature maps of different scales in the channel dimension, and the Reshape function reshapes the concatenated iterative results so that their channel dimensions are the same as the original input target feature map. U The same, get the final optimization result .

[0078] Step S6: Calculate the horizontal ice thickness (long diameter) of the main view and side view lines in the original ice image according to the segmentation results of the semantic segmentation model. a ) and vertical ice thickness (short diameter b ), and the ice density is obtained by combining the ice type identification results, and the equivalent ice thickness of the current line is calculated using the equivalent area method, including the optimization calculation of environmental conditions such as visible from side angles, invisible from side angles, and low light at night, which specifically includes the following steps:

[0079] Step S6.1: Use the semantic segmentation model to segment the main view and side view lines in the original ice-covered image to obtain the segmented areas of the lines and the ice-covered area of ​​the same line under the same view. Its horizontal ice thickness The following relationship exists: ; The diameter of the bare wire at this viewing angle is known , the pixel area of ​​the bare line is obtained by semantic segmentation and the pixel area under ice cover The horizontal ice thickness under ice-covered conditions can be calculated using the following formula: : in, This is the major diameter parameter required in the equivalent area method. a Similarly, the ice thickness in the vertical direction can be obtained by using the segmentation result of the side view line in the original ice image, which is the short diameter parameter required in the equivalent area method. b ;

[0080] Step S6.2, equivalent area method: the irregular cross section of the ice-covered line is equivalent to a standard circle with the same area, and the radius of the standard circle is subtracted from the radius of the bare wire to obtain the equivalent uniform ice thickness of the line. First, the irregular ice-covered cross section is approximated as a regular ellipse, and the area of ​​the ellipse is made as close as possible to the original irregular cross-sectional area by increasing the adjustment parameters. The ellipse is further equivalent to a standard circle with the same cross-sectional area, and finally the equivalent uniform ice thickness is obtained. The process can be expressed by the following formula: in, is the required equivalent uniform ice thickness, r is the bare wire radius, is the ice density, a is the major diameter parameter, b is the short diameter parameter. The left side of the equation represents the equivalent standard circle area minus the bare wire cross-sectional area. The right side of the equation in brackets represents the approximate ellipse area minus the bare wire cross-sectional area, which represents the regular ice cross-sectional area outside the bare wire. This part adjusts the error through the ice density and a constant parameter, thus obtaining the following equivalent uniform ice thickness calculation formula: Among them, ice density It can be calculated based on the identification results of ice cover type;

[0081] Step S6.3: In view of the difficulty of observing the side view line due to low light at night, the equivalent ice thickness calculation formula is optimized and adjusted. The ice cover state of the main view and the side view of the same line at the last moment of the day on the same day is used to infer the ice cover state of the line at the side view at night, and then the short path parameter at night is calculated. ;

[0082] Calculate the ratio of the long and short diameter parameters at the last moment of the day : ;in, is the long-path parameter at the last moment of the day; is the short path parameter at the last moment of the day;

[0083] The nighttime long-path parameters can be obtained based on the ice coverage status of the line from the main viewing angle at night. , combined with the ratio of the long and short diameter parameters at the last moment of the day , and further calculate the short-path parameters at night: ;

[0084] The ratio of the major and minor diameter parameters at the last moment of the day and the long-path parameters at night The optimized calculation formula for equivalent ice thickness at night is obtained: ;

[0085] Step S6.4: For the case where the line from the side view is almost invisible or it is difficult to extract effective information due to the line layout, a large number of statistical observations are made on the linear relationship between the long and short diameters when the line is visible from the side view, and a polynomial function relationship is used to approximate the relationship between the long and short diameters when the line is visible from the side view. The ice state from the side view can be inferred from the ice state based on the main view, and a large number of long and short diameter parameter data sets when the line is visible from the side view are collected. ,in N is the sample size, and the polynomial function is defined as follows: in, Indicates the short path parameter when the side view is visible. Indicates the long diameter parameter when the side view is visible.n represents the order of the polynomial, Represents the coefficients of each polynomial; determines the coefficients of the polynomial by the least squares method, making the function as close to the data points as possible; using a large number of Fitting regular functions to data , and then the long diameter parameter is calculated based on the detection under the invisible state of the side view , calculate its short diameter parameter : By regular function and the major diameter parameter The optimization calculation formula for the equivalent ice thickness in the invisible state from the side view is obtained: .

[0086] The overall model structure of the present invention is as follows Figure 2 As shown in the figure, the model as a whole consists of three modules, and the specific construction process is as follows:

[0087] Firstly, the original ice image is obtained by using the ice monitoring equipment, and then preprocessed in the ice type recognition module. The composite brightness feature map and composite roughness feature map are obtained by the brightness preprocessing module and the roughness preprocessing module respectively.

[0088] Then, the original ice-covered image, the composite brightness feature map, and the composite roughness feature map are input into the backbone branch, brightness branch, and roughness branch of IceNet-T, respectively, as follows: Figure 4 As shown, the global features, brightness features and roughness features of the ice-covered image are extracted, and the features extracted by the three branches are fused to obtain the ice-covered type recognition result;

[0089] At the same time, the original ice-covered image is input into the ice-covered segmentation module, and the main view line and the side view line in the ice-covered image are segmented by SCTNet, and then the MSCRF designed in the segmentation optimization module SegBoost is used for optimization to obtain the final ice-covered area optimization segmentation result; FIG3 (a) is a schematic diagram of the main view line and the side view line in an embodiment of the present invention; FIG3 (b) is a schematic diagram of the long and short diameters of the ice-covered cross section of the line in an embodiment of the present invention;

[0090] Finally, in the equivalent ice thickness calculation module, the horizontal and vertical ice thickness of the line ice cross section are calculated according to the segmentation results of the ice segmentation module, that is, the long diameter a and short path b , and then use the ice type identification result to obtain the ice density, substitute the major and minor diameter parameters and ice density into the ice thickness calculation formula, optimize and adjust the ice thickness calculation formula according to special environmental conditions such as night and invisible lines from side angles, and finally obtain the equivalent ice thickness calculation results under different environmental conditions.

[0091] In this embodiment, the detailed process of the SegBoost module optimizing the output results of the semantic segmentation model is as follows: Figure 5 As shown in the figure, each training model will be iterated and updated multiple times. The figure shows the detailed update process of the conditional random field (CRF) of the single-scale feature map. All feature maps extracted from different convolutional layers of SCTNet are updated with a round of CRF, which is the entire process of one MSCRF iteration. The final segmentation effect is shown in the figure. Figure 7 As shown, the first column is the original ice-covered image, the second column is the ice-covered area labels of the corresponding main view and side view lines in the original ice-covered image, and the third column is the final output ice-covered area segmentation result.

[0092] In this embodiment, the network structures adopted by the brightness branch and the roughness branch of the IceNet-T model are similar. The difference is that the dimensions of the feature maps received by the two branches are different, which makes the model parameter settings of the two branches different. The main part of the trunk branch selects the feature extraction network of the migration deep learning model MobileNet-V3. In order to adapt it to the task of ice cover type recognition, its output network layer is appropriately adjusted, and the final output of the three branches is a one-dimensional feature vector. The weighted normalized sum of the feature vectors output by the three branches can obtain the fused probability distribution result, among which the one with the largest probability value is the final ice cover type recognition result.

[0093] In this embodiment, the ice-covered areas of the main view and side view routes obtained by semantic segmentation can be used to calculate the long-diameter parameters. a and short diameter parameters b According to the identification result of ice type, the current ice density can be obtained. ρ Normally, the density of rime ice is 0.7-0.9 g / cm 3 The density of rime ice is 0.1~0.4g / cm 3 The density of mixed rime ice is 0.2~0.6g / cm 3 , and then the corresponding equivalent ice thickness calculation formula can be used according to different environmental conditions, including visible from the side view, invisible from the side view, and nighttime environmental conditions. The result of the formula calculation is the final equivalent ice thickness of the line.

[0094] In addition, advanced classification models such as EfficientNet, MobileOne, RepViT, and ResNeXt were used to train and test the ice type recognition dataset, and the accuracy and loss value curves of these models on the validation set were recorded and compared with IceNet-T. The experimental results are shown in the figure. Figure 6As shown, it can be seen that the verification accuracy of IceNet-T can be stabilized at around 98%, which is higher than the other models, and the model stability of IceNet-T is also better than the other models.

[0095] At the same time, ablation experiments were conducted on the three branches of IceNet-T. The trunk branch, brightness branch, and roughness branch were used separately for ice type recognition experiments, and the parameter quantity, model size, and verification accuracy were recorded. The detailed results are shown in Table 1:

[0096] Table 1

[0097] method Number of parameters (M) Model size (M) Accuracy (%) Luminance branch 0.87 3.44 89.78 Roughness branch 0.87 3.43 80.28 Main branch 1.52 5.94 92.03 IceNet-T 3.26 12.81 98.67

[0098] It can be seen that the complete IceNet-T can achieve an accuracy of 98.67% in identifying ice types, which is better than any of the branches.

[0099] Finally, based on the proposed line detection method, a test experiment was conducted on line ice images, including tests on ice conditions such as visible from side angles, invisible from side angles, and low light at night. The experimental segmentation effect is as follows: Figure 8 The corresponding line detection output results are shown in Table 2, which include information such as horizontal ice thickness, vertical ice thickness, equivalent ice thickness, ice type, and lighting environment judgment. Figure 8 The red coverage area is used to display the ice-covered area of ​​the main view segmented by the model, and the yellow coverage area is used to display the ice-covered area of ​​the side view. Through these experimental results, it can be found that the line ice-covered detection method of the embodiment of the present invention can well realize the detection of ice-covered line information such as ice-covered type and ice-covered thickness.

[0100] Table 2

[0101]

[0102] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A line icing detection method based on deep learning, characterized in that: The specific steps include: Step S1, obtaining the original ice image of the line taken by the ice monitoring equipment, and making a relevant data set; Step S2, using histogram equalization to enhance the brightness features in the original ice-covered image, taking the enhanced image as a brightness feature map, and obtaining its average brightness value and brightness standard deviation, respectively encoding them into feature tensors consistent with the dimensions of the brightness feature map, and splicing them with the brightness feature map in the channel dimension to obtain a composite brightness feature map; Step S3, using the horizontal local binary pattern H-LBP to process the original ice-covered image to obtain its roughness texture feature map, and using the gray level co-occurrence matrix to obtain the contrast and homogeneity information of the original ice-covered image, respectively encoding them into feature tensors consistent with the dimensions of the roughness texture feature map, and splicing them with the roughness texture feature map in the channel dimension to obtain a composite roughness feature map; Step S4, constructing a multi-branch line ice type recognition model IceNet-T, including a trunk branch, a brightness branch and a roughness branch, inputting the original ice image into the trunk branch, inputting the composite brightness feature map into the brightness branch, inputting the composite roughness feature map into the roughness branch, fusing the multi-branch extraction features, and obtaining the ice type recognition result; Step S5, using the semantic segmentation model SCTNet to detect and segment the ice-covered line area in the original ice-covered image, and combining the segmentation results of the multi-scale conditional random field MSCRF optimization model to improve the segmentation accuracy; wherein MSCRF is Multi-Scale Conditional Random Field; Step S6: According to the segmentation results of the main view and the side view lines in the original ice image by the semantic segmentation model, the horizontal ice thickness and the vertical ice thickness are calculated respectively. At the same time, the ice density is obtained by combining the ice type recognition result. The equivalent ice thickness of the current line is calculated using the equivalent area method, which includes the optimization calculation of the environmental conditions such as visible from the side view, invisible from the side view, and low light at night. The horizontal ice thickness is the long diameter. a ; Vertical ice thickness is short diameter b .

2. The line icing detection method based on deep learning according to claim 1 is characterized in that: In step S2, a composite brightness feature map is obtained, which specifically includes the following steps: Step S2.1: Use the histogram equalization method to enhance the brightness feature map in the original ice-covered image. The enhancement formula is as follows: ;in, is the position in the image after equalization The gray value of the pixel at is the position in the original ice image The gray value of the pixel at L is the number of gray levels, where b bit image, L =2 b , is the gray value of the original ice-covered image The number of pixels, N is the total number of pixels in the image, The gray value of the original ice-covered image is less than or equal to The cumulative distribution function of pixels; Calculate the average brightness value and brightness standard deviation of the original ice-covered image using the following formula: ;in, represents the average brightness value, represents the brightness standard deviation, H.W. and C are the height, width and number of channels of the input image, Indicates the input image The brightness value corresponding to the position; Step S2.2: Construct a feature tensor consisting of the average brightness value and the brightness standard deviation, that is, expand these two values ​​into a matrix of the same size as the image, and concatenate them with the original image data in the channel dimension. The formula is as follows: ;in, Represents the average brightness value feature tensor consistent with the original image dimension, Represents the brightness standard deviation feature tensor consistent with the original image dimension, Represents a matrix with all elements set to 1. The average brightness feature tensor and brightness standard deviation feature tensor after the expansion dimension are concatenated with the brightness feature map in the channel dimension. The formula is as follows: in, represents the final composite brightness feature tensor, Concat represents the channel dimension concatenation function, They respectively represent the enhanced brightness feature map, the feature tensor of the average brightness value after dimension expansion, and the feature tensor of the brightness standard deviation.

3. The line icing detection method based on deep learning according to claim 1 is characterized in that: In step S3, a composite roughness characteristic map is obtained, which specifically includes the following steps: Step S3.1, use the horizontal local binary pattern H-LBP to process the original ice-covered image to obtain its roughness texture feature map. The horizontal local binary pattern focuses on the texture information in the horizontal direction. The formula is as follows: in, Indicates location The H-LBP calculation results at Indicates the position in the image The pixel value at P is the horizontal neighborhood range, is a binarization function, that is: ; The H-LBP formula consists of two parts. The left side of the plus sign is a comparison between the target pixel and the horizontal neighboring pixels on the left, and the right side of the plus sign is a comparison between the target pixel and the horizontal neighboring pixels on the right. The comparison results are binarized and assigned weights. , thus encoding the result into a binary number, the result of H-LBP Is an integer representing the horizontal texture mode, the final H×W H-LBP feature map calculated from an image of size T It can be expressed as: ； Step S3.2, in addition to the roughness texture feature map obtained by H-LBP, the composite roughness texture feature map also includes the feature tensor formed by expanding the contrast and homogeneity calculated using the gray level co-occurrence matrix, where the contrast can characterize the roughness of the ice-covered image, and the homogeneity can characterize the smoothness of the ice-covered image. The specific calculation formula is as follows: ;in, Contrast The contrast ratio is Ct , Homogeneity Homogeneity Hg , L Represents the number of gray levels, is the gray-level co-occurrence matrix, which indicates that the gray value is x and y The joint occurrence probability of pixel pairs at a certain distance and direction; Step S3.3: Obtaining the roughness texture feature map T And the contrast of the original ice image Ct and homogeneity Hg After that, you need to Ct and Hg Expand to T The same dimension, so as to concatenate it with the channel dimension, the expanded formula is as follows: ;in, H and W Roughness texture feature map T The height and width dimensions, represents the contrast feature tensor after the expanded dimension, represents the homogeneity feature tensor after the expanded dimension, is a matrix with all elements set to 1. The roughness texture feature map is concatenated with the contrast and homogeneity feature tensors after encoding and expansion in the channel dimension. The formula is as follows: ;in, Represents the generated composite roughness feature map, Concat is the channel dimension concatenation function, and the concatenated It can enhance the image's ability to express texture features.

4. The line icing detection method based on deep learning according to claim 1 is characterized in that: In step S4, a three-branch ice type recognition model IceNet-T is constructed for the composite brightness feature map and composite roughness feature map extracted in steps S2 and S3, combined with the original ice image, and the three-branch results are extracted and fused to obtain the final ice type recognition result, which specifically includes the following steps: Step S4.1, input the original ice-covered image into the trunk branch. The trunk branch is composed of the main feature extraction network of the deep learning transfer model, and its output layer structure is fine-tuned to adapt it to the ice type recognition task. The original ice-covered image is input into the trunk branch to extract the global ice-covered features. The extraction process can be expressed by the following formula: ;in, I represents the input original ice-covered image, It represents the main network feature extraction process of the migration model. Indicates replacing the fully connected output layer of the original migration model. is the new bias term, ReLU represents the ReLU activation function, and finally the global features extracted by the trunk branch are obtained ; Step S4.2: Input the composite brightness feature map and the composite roughness feature map into the brightness branch and the roughness branch respectively. In addition to adjusting parameters to adapt to the dimensions of the composite feature maps received by each branch, the network structures used in the feature extraction part are similar, both consisting of 1 initialization module, 4 main feature extraction modules and 1 classifier. The specific feature extraction process can be expressed by the following formula: in, represents the composite brightness feature map, represents the composite roughness feature map, Init Represents the initialization module, M Represents the main feature extraction module. Each main feature extraction module contains an SE channel attention layer. Classifier represents the classifier, the subscripts represent the branches to which they belong, and the superscripts represent different modules. Finally, the brightness branch and the roughness branch obtain feature maps respectively. and ; Among them, SE stands for Squeeze-and-Excitation; Step S4.3: Obtain the extraction results of the three branches , and Finally, the features extracted by different branches are fused together through the weighted summation method, which can be expressed as follows: in, It represents the features after the fusion of three branches. They represent the weight parameters of the main branch, brightness branch, and roughness branch respectively, and the sum of the weights is 1. Result It represents a probability distribution result obtained by converting the fused features by the Softmax function, where the one with the largest probability is the final ice type recognition result.

5. The line icing detection method based on deep learning according to claim 1, characterized in that: In step S5, the multi-scale conditional random field MSCRF used to optimize the semantic segmentation result specifically includes the following steps: Step S5.1: According to the network structure of SCTNet, semantic segmentation feature maps of different scales are extracted from the multi-layer network of its encoder and decoder. MSCRF refines and optimizes the segmentation results by combining the spatial and color information of the multi-scale feature maps. The multi-scale feature maps extracted by different convolutional layers of the semantic segmentation model have different dimensions. These feature maps need to be adjusted to a uniform size through preprocessing, and then input into MSCRF together with the target feature map for iterative optimization. The preprocessing process can be expressed by the following formula: in, Resize is the adjusted multi-scale feature map. Function to adjust the dimension. H and W are the height and width of the target size, n is the number of feature maps that need to be processed, and the final multi-scale feature map can be represented as a set: ; Step S5.2: MSCRF inputs the target feature map and multi-scale feature map that need to be optimized P , by minimizing an energy function, the labels of adjacent pixels in the target feature map with similar color and position are made consistent. The segmentation result of the target feature map is optimized through multiple iterative updates. The iterative optimization formula is as follows: in, U express Unary , that is, the target feature map to be optimized, Represents feature maps of different scales, n Indicates the number of feature maps that need to be processed, and Represent spatial Gaussian weight and bilateral Gaussian weight, which can represent the spatial information and color information in the feature map respectively. The Softmax function is used to normalize the weights. k After iterations, the final output optimized feature map for: Among them, the Concat function concatenates the iterative results of feature maps of different scales in the channel dimension, and the Reshape function reshapes the concatenated iterative results so that their channel dimensions are the same as the original input target feature map. U The same, get the final optimization result .

6. The line icing detection method based on deep learning according to claim 1, characterized in that: In step S6, the equivalent ice thickness of the line is calculated using the equivalent area method, including the following steps: Step S6.1: Use the semantic segmentation model to segment the main view and side view lines in the original ice-covered image to obtain the segmented areas of the lines and the ice-covered area of ​​the same line under the same view. Its horizontal ice thickness The following relationship exists: ; The diameter of the bare wire at this viewing angle is known , the pixel area of ​​the bare line is obtained by semantic segmentation and the pixel area under ice cover The horizontal ice thickness under ice-covered conditions can be calculated using the following formula: : in, This is the major diameter parameter required in the equivalent area method. a Similarly, the ice thickness in the vertical direction can be obtained by using the segmentation result of the side view line in the original ice image, which is the short diameter parameter required in the equivalent area method. b ; Step S6.2, equivalent area method: the irregular cross section of the ice-covered line is equivalent to a standard circle with the same area, and the radius of the standard circle is subtracted from the radius of the bare wire to obtain the equivalent uniform ice thickness of the line. First, the irregular ice-covered cross section is converted into a regular ellipse, and the area of ​​the ellipse is made close to the original irregular cross-sectional area by increasing the adjustment parameters. The ellipse is further equivalent to a standard circle with the same cross-sectional area, and finally the equivalent uniform ice thickness is obtained. The process can be expressed by the following formula: in, is the required equivalent uniform ice thickness, r is the bare wire radius, is the ice density, a is the major diameter parameter, b is the short diameter parameter. The left side of the equation represents the equivalent standard circle area minus the bare wire cross-sectional area. The right side of the equation in brackets represents the ellipse area minus the bare wire cross-sectional area, which represents the regular ice cross-sectional area outside the bare wire. The error is adjusted by the ice density and a constant parameter, thus obtaining the following equivalent uniform ice thickness calculation formula: Among them, ice density It can be calculated based on the identification results of ice cover type; Step S6.3: In view of the difficulty of observing the side view line due to low light at night, the equivalent ice thickness calculation formula is optimized and adjusted. The ice cover state of the main view and the side view of the same line at the last moment of the day on the same day is used to infer the ice cover state of the line at the side view at night, and then the short path parameter at night is calculated. ; Calculate the ratio of the long and short diameter parameters at the last moment of the day : ;in, is the long-path parameter at the last moment of the day; is the short path parameter at the last moment of the day; The nighttime long-path parameters can be obtained based on the ice coverage status of the line from the main viewing angle at night. , combined with the ratio of the long and short diameter parameters at the last moment of the day , and further calculate the short-path parameters at night: ; The ratio of the major and minor diameter parameters at the last moment of the day and the long-path parameters at night The optimized calculation formula for equivalent ice thickness at night is obtained: ; Step S6.4: For the case where the line from the side view is almost invisible or it is difficult to extract effective information due to the line layout, a large number of statistical observations are made on the linear relationship between the long and short diameters when the line is visible from the side view, and a polynomial function relationship is used to describe the relationship between the long and short diameters when the line is visible from the side view. The ice state from the side view can be inferred from the ice state based on the main view, and a large number of long and short diameter parameter data sets when the line is visible from the side view are collected. ,in N is the sample size, and the polynomial function is defined as follows: in, Indicates the short path parameter when the side view is visible. Indicates the long diameter parameter when the side view is visible. n represents the order of the polynomial, Represents the coefficients of each polynomial; determines the coefficients of the polynomial by the least squares method to make the function close to the data points; uses a large number of Fitting the data to a regular function , and then the long diameter parameter is calculated based on the detection under the invisible state of the side view , calculate its short diameter parameter : By regular function and the major diameter parameter The optimization calculation formula for the equivalent ice thickness in the invisible state from the side view is obtained: .

7. The line icing detection method based on deep learning according to claim 1, characterized in that: It comprises a line icing detection system, which comprises: a pre-processing module, an icing type identification module, an icing segmentation module and an equivalent icing thickness calculation module; A preprocessing module is used to obtain the original ice image captured by the ice monitoring equipment, and obtain a composite brightness feature map and a composite roughness feature map through a brightness preprocessing module and a roughness preprocessing module respectively; The ice type recognition module is a three-branch convolutional neural network IceNet-T, which includes a trunk branch, a brightness branch, and a roughness branch. It is used to input the original ice image into the trunk branch, and input the composite brightness feature map and composite roughness feature map obtained by preprocessing into the brightness branch and the roughness branch respectively. The ice features extracted by the three branches are integrated to obtain the ice type recognition result. The ice segmentation module is used to train the semantic segmentation model SCTNet using the labeled data set, and optimize the ice segmentation results through the multi-scale conditional random field MSCRF in the segmentation result optimization module SegBoost to obtain a more accurate ice area, and extract the pixel area of ​​the ice area in the main view and the ice area in the side view; The equivalent ice thickness calculation module is used to obtain the ice density parameters based on the ice type recognition results, and then deduce the major and minor diameter parameters required by the equivalent ice calculation formula based on the pixel areas of the ice areas in the main and side perspectives identified by the ice segmentation module, and then calculate the uniform equivalent ice thickness. The ice thickness calculation formula is optimized and adjusted for nighttime and side perspective routes that are invisible, and finally the equivalent ice thickness calculation results under different environmental conditions are obtained.

Images