Power transmission line icing thickness detection method based on deep learning and dual-light image
Through deep learning algorithms and dual-light image technology, the problems of low efficiency and low accuracy of ice-covered thickness detection in transmission lines are solved, and accurate positioning and thickness measurement of ice-covered areas are realized, providing fast and accurate ice-covered thickness detection. As a result, the ability to judge the power grid safety has been enhanced.
Patent Information
- Application Number
- CN202410815255.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2024-11-08
AI Technical Summary
The existing transmission line ice-cover thickness detection methods are inefficient, with personnel safety risks, low accuracy and cumbersome operation procedures, making it difficult to achieve fast and accurate ice-covering monitoring, especially in severe weather and complex natural environments, which are prone to missed inspections or errors. It affects the safe and stable operation of the power grid.
Using deep learning algorithms and dual-light image detection methods, the original data set is constructed by acquiring and preprocessing the ice-covered image of the transmission line, and the segmentation process is performed using a single-stage instance segmentation algorithm and YOLACT network, combining multi-direction Sobel edge operators Design and infrared image data, perform boundary extraction and image fusion, and calculate the standard thickness of the transmission line ice covering.
Accurate positioning and contour extraction of ice-covered areas is achieved, which reduces calculation complexity, improves detection accuracy and accuracy, reduces background interference and noise impacts, provides fast and accurate ice-covered thickness measurement results, and provides power grid maintenance. Provide a basis for safety judgment.
Smart Images

Figure CN118918164A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power transmission line detection, and in particular to a method for detecting ice thickness of power transmission lines based on deep learning and dual-light images. Background Art
[0002] The earliest research on transmission line icing at home and abroad can be traced back to the mid-20th century. However, due to the constraints of geographical environment and weather conditions, the traditional manual inspection method is inefficient, the repeated inspection cycle is lengthy, and it is difficult to meet actual needs, so it has not been widely promoted. With the advancement of science and technology, ice thickness detection methods have gradually evolved, but they still face many challenges. The limitations of these methods seriously restrict the power system's real-time monitoring and rapid response capabilities to icing conditions, potentially threatening the safe and stable operation of the power grid. In addition, some existing automated detection systems often rely on specific hardware equipment, such as load sensors and strain gauges, which not only have high installation and maintenance costs, but are also difficult to promote and apply on a large scale and cannot meet the needs of wide coverage. At the same time, these systems are unable to cope with the monitoring needs of transmission lines in a wide area. It is difficult to make full use of the massive amount of information collected to conduct in-depth law mining and trend prediction, thus affecting the foresight and initiative of ice prevention and control. More importantly, in a complex and changeable natural environment, traditional methods are often difficult to accurately identify and locate iced areas, especially in the case of strong background interference and poor lighting conditions, which can easily lead to missed detections and misjudgments, resulting in inaccurate ice risk assessment and affecting the scientific nature of decision-making. These deficiencies not only increase the difficulty and cost of power grid operation and maintenance, but also lay hidden dangers for potential ice disasters. Therefore, the development of a more advanced and reliable ice thickness detection method is the focus of current research.
[0003] However, the common solutions currently used have many shortcomings, including: the currently commonly used manual direct measurement method is intuitive, but it has problems of low efficiency and personnel safety risks, especially in severe weather conditions, which increases the difficulty and risk factor of the operation; secondly, although the indirect image comparison method avoids the risks of manual measurement to a certain extent, it is not accurate and the operation process is cumbersome, making it difficult to ensure the reliability and timeliness of the measurement results. Summary of the invention
[0004] The present invention is proposed in view of the problems of low efficiency and personnel safety risks in the prior art when detecting ice thickness on transmission lines, increased difficulty and risk factor of operations under severe weather conditions, low accuracy and cumbersome operating procedures, and difficulty in ensuring the reliability and timeliness of measurement results.
[0005] Therefore, the problem to be solved by the present invention is how to provide a method for detecting ice thickness of transmission lines based on deep learning algorithm and dual-light images, so as to realize a method for quickly and contactlessly judging the ice condition of ice-covered lines.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0007] In the first aspect, an embodiment of the present invention provides a method for detecting ice thickness of transmission lines based on deep learning and dual-light images, which includes collecting ice images of transmission lines and performing image preprocessing to construct an original data set; segmenting the original data set, constructing a pre-segmentation model and performing calculations to obtain a mask of the ice-covered transmission line area; collecting infrared image data of the transmission line based on a multi-directional Sobel edge operator design and performing boundary extraction to obtain a mask of the original transmission line area; performing camera calibration and image fusion on the ice image and the infrared image data and performing calculations to obtain the standard thickness of ice on the transmission line.
[0008] As a preferred solution of the method for detecting ice thickness of transmission lines based on deep learning and dual-light images described in the present invention, wherein: the construction of the original data set is to use standardized dual-light optical equipment to collect ice images of transmission lines and perform image preprocessing to construct the original data set; the image preprocessing includes size transformation, brightness transformation, color enhancement, contrast enhancement and image denoising.
[0009] As a preferred solution of the transmission line ice thickness detection method based on deep learning and dual-light images described in the present invention, wherein: the construction of the pre-segmentation model includes a single-stage instance segmentation algorithm, YOLACT network generation, data set, evaluation index and model training; the single-stage instance segmentation algorithm includes the following steps: using the polar coordinate representation coding mask method, classifying and regressing each pixel in the target centroid area to the dense distance between the centroid and the contour; extracting features from the feature extraction backbone network, and inputting them into the classification branch and the regression branch, the classification branch includes category classification and polar centerness; taking a point in the feature map as the polar coordinate center point, generating N rays at a fixed angle, calculating the ray distance from each target edge to the center point, and calculating according to the ray distance to obtain the optimal mask; retaining Q best samples in each FPN structure, filtering according to the classification confidence score and the polar centerness product, wherein the distance between the sample point and the N rays is known, and the coordinates of the corresponding object edge points are connected one by one starting from P to form the contour of the corresponding instance.
[0010] As a preferred solution of the transmission line ice thickness detection method based on deep learning and dual-light images described in the present invention, wherein: the original transmission line area mask obtained includes a multi-directional Sobel edge operator design, infrared image binarization based on the law of the law, image filtering based on a nonlinear operator, a topological boundary tracking algorithm, and original transmission area MASK extraction; the multi-directional Sobel edge operator design adopts a multi-directional Sobel operator, adds A templates in different directions on the basis of the traditional Sobel operator, and extracts edge information from the templates in the directions of A1, A2, A3 and A4 respectively. The specific formula of the template is as follows:
[0011]
[0012] The closer to the pixel point, the larger the weighted value. The weighted difference of the neighborhood of each pixel point on the image is calculated. When the weighted difference reaches an extreme value, it is set as the threshold. If the weighted difference of the pixel point is greater than the threshold, the corresponding pixel point is retained. If the weighted difference of the pixel point is less than the threshold, the corresponding pixel point is removed to obtain the edge information of the transmission line.
[0013] As a preferred solution of the transmission line ice thickness detection method based on deep learning and dual-light images described in the present invention, the transmission line ice thickness includes a physical model for calculating the transmission line ice thickness, optical RGB calibration and infrared camera calibration, optical RGB-infrared transmission line area MASK registration and fusion, and transmission line ice standard thickness measurement; the transmission line ice thickness calculation physical model includes the following steps: when the camera optical axis is orthogonal to the line axis direction, the line edges are parallel to each other, assuming that the total number of pixels in the ice area in the x1-x2 area is P D , the total number of pixels in the original transmission line area is P d , the calculation formula for the equivalent thickness of ice covering the transmission line is as follows:
[0014]
[0015] Where t is the equivalent thickness of ice on the transmission line; D is the equivalent diameter of the ice-covered area; d is the original transmission line diameter; P D P is the total number of pixels in the ice-covered area in the x1-x2 region; d is the total number of pixels in the original transmission line area.
[0016] As a preferred solution of the transmission line ice thickness detection method based on deep learning and dual-light images described in the present invention, the optical RGB-infrared transmission line area MASK registration and fusion is due to the relationship between relative translation, rotation, different scaling and distortion between the images of the same object obtained by different sensors and different imaging modes. Therefore, before using the optical RGB image and the infrared image to perform transmission line area mask fusion, strict geometric registration must be ensured, and the optical RGB-infrared binocular imaging device with parallel optical axes is image registered; the registration error is calculated as follows:
[0017]
[0018] Among them, σ x is the registration error; f is the focal length; d c is the baseline length; I pix is the pixel size; D target For a specific target distance; D optimal is the target distance.
[0019] As a preferred solution of the transmission line ice thickness detection method based on deep learning and dual-light images described in the present invention, in which: in the actual ice scene, the ice image does not always meet the parallelism of the upper and lower edges of the line. Therefore, when shooting the line or ice image, it is necessary to ensure that the number of pixels of the line or ice area and the image size meet the line or the entire ice area, the radial pixel number and image size ratio is B1~B2, and the straight-line distance from the optical center position of the camera to the line is C1~C2; the calculation formula of the standard ice thickness of the transmission line is as follows:
[0020]
[0021] Where B0 is the standard thickness of ice covering the transmission line; ρ is the estimated density of ice covering; K s is the ice-covered shape coefficient; R is the ice-covered radius; r is the original transmission line radius; t dx is the equivalent thickness of ice covering the transmission line; d is the original diameter of the transmission line.
[0022] In the second aspect, in order to further solve the safety problems existing in the detection of ice thickness, the present invention provides a transmission line ice thickness detection system based on deep learning and dual-light images, which includes: an ice image module, which is used to collect ice images of transmission lines and perform image preprocessing, construct an original data set, and use a pre-segmentation model to calculate to obtain an ice-covered transmission line area mask; an infrared image module, which is used to collect infrared image data of transmission lines based on a multi-directional Sobel edge operator design and perform boundary extraction to obtain an original transmission line area mask; an ice thickness module, which is used to calculate the equivalent ice thickness of the transmission line by combining the ice-covered transmission line area mask and the original transmission line area mask, obtain the ice radius and calculate to obtain the standard ice thickness of the transmission line.
[0023] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for detecting ice thickness of transmission lines based on deep learning and dual-light images as described in the first aspect of the present invention is implemented.
[0024] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the method for detecting ice thickness of transmission lines based on deep learning and dual-light images as described in the first aspect of the present invention is implemented.
[0025] The beneficial effects of the present invention are as follows: the present invention adopts a single-stage instance segmentation algorithm and a YOLACT network, and utilizes a polar coordinate representation method for coding masks, which not only improves the accuracy of segmentation, but also significantly reduces the computational complexity, realizes accurate positioning and contour extraction of ice-covered areas, and provides accurate regional information for subsequent ice thickness calculation; designs a multi-directional Sobel edge operator, makes edge detection more comprehensive and accurate, enhances the ability to recognize the outline of transmission lines, effectively reduces the influence of background interference and noise, and improves the accuracy of the original transmission line area mask; through the fusion of dual-spectrum images, fully utilizes the respective advantages of visible light and infrared light, enhances the ability to recognize ice conditions, and the thickness calculation method based on the physical model takes into account the ice shape and density factors, improves the accuracy of measurement, and introduces a standardized ice thickness calculation formula to make the measurement result more universal and comparable, realizes non-contact, fast and accurate ice thickness measurement of ice-covered lines, and provides a basis for line safety assessment for maintenance personnel. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0027] Figure 1 Schematic diagram of the search starting point of the Square boundary tracking algorithm in Example 1.
[0028] Figure 2 Schematic diagram of the tracking boundary direction of the Square boundary tracking algorithm in Example 1.
[0029] Figure 3 Schematic diagram of the limitations of the Square boundary tracking algorithm in Example 1.
[0030] Figure 4 This is a schematic diagram of an actual ice-covered scene in Example 1.
[0031] Figure 5 This is a schematic diagram of the calculation principle of the equivalent ice thickness in Example 1.
[0032] Figure 6 This is a flow chart for calculating the standard ice thickness of the transmission line in Example 1. DETAILED DESCRIPTION
[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.
[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0035] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0036] Example 1
[0037] Reference Figure 1 to Figure 6 , which is the first embodiment of the present invention, provides a method for detecting ice thickness of a power transmission line based on deep learning and dual-light images, comprising the following steps:
[0038] S1: Collect ice-covered images of transmission lines and perform image preprocessing to construct the original data set.
[0039] Preferably, the original data set is constructed by using a standardized dual-light optical device to collect ice-covered images of power transmission lines and perform image preprocessing to construct the original data set.
[0040] Furthermore, image preprocessing includes size transformation, brightness transformation, color enhancement, contrast enhancement and image denoising.
[0041] Specifically, the size transformation is to change the length and width of the image based on the size modification strategy to achieve image scaling modification.
[0042] Specifically, the brightness transformation is to calculate the adjustment coefficient based on the RGB space brightness adjustment algorithm, and use adjustment means to adjust the brightness. The adjustment means include: adjusting based on the current RGB value, that is, the RGB value of the pixel after adjustment is the product of the RGB value of the pixel before adjustment and the adjustment coefficient.
[0043] Regardless of the influence of the RGB value, the R, G, and B values of each point are always adjusted in the same way, that is, the RGB value of the pixel after adjustment is the sum of the RGB value of the pixel before adjustment and the adjustment coefficient.
[0044] Specifically, color enhancement is to divide the grayscale of the image into M different regions using a grayscale layering method, so that each region corresponds to a color.
[0045] Specifically, contrast enhancement is to enhance the contrast of an image by using histogram equalization. A mapping curve is formulated through the grayscale distribution histogram of the image, and the mapping curve is used to transform the pixel grayscale values of the entire image, thereby improving the contrast of the image.
[0046] Furthermore, by using the traditional histogram equalization method to process the image, the pixels of the processed image will occupy as many gray levels as possible and be evenly distributed.
[0047] Specifically, the image denoising is to perform denoising processing on the image according to the wavelet transform, put the ice-covered part of the transmission line into the wavelet coefficient greater than the absolute value Q1, and put the background image into the background coefficient less than or equal to the absolute value Q1, so as to reduce the influence of the background on the edge detection of the transmission line.
[0048] Furthermore, the wavelet transform comprises the following steps: calculating the wavelet decomposition coefficients using the wavelet transform image.
[0049] Thresholds are set for wavelet coefficients based on different scales, and wavelet coefficients less than the threshold Q2 are set to zero to obtain wavelet coefficients greater than or equal to the threshold Q2, thereby removing low-frequency signals and eliminating most of the noise in the image.
[0050] The image is reconstructed using inverse wavelet transform to obtain the denoised image.
[0051] Specifically, since the ice-covered images of power transmission lines are affected by ambient lighting and noise factors, the ice-covered images are preprocessed to unify the size of the ice-covered images and improve the image quality, which is beneficial to subsequent feature extraction.
[0052] S2: Segment the original data set, build a pre-segmentation model and perform calculations to obtain the mask of the ice-covered transmission line area.
[0053] Preferably, building a pre-segmentation model includes a single-stage instance segmentation algorithm, YOLACT network generation, data sets, evaluation indicators, and model training.
[0054] Specifically, the single-stage instance segmentation algorithm includes the following steps: using the polar coordinate representation encoding mask method, each pixel in the target centroid region is classified and regressed to the dense distance between the centroid and the contour.
[0055] Features are extracted from the feature extraction backbone network and input into the classification branch and regression branch. The classification branch includes category classification and polar centerness.
[0056] A point in the feature map is used as the polar coordinate center point, and N rays are generated at a fixed angle, where the angle increment is 10 degrees and N is 36. The ray distance from each target edge to the center point is calculated, and the optimal mask is obtained based on the ray distance.
[0057] The 1000 best samples are retained in each FPN structure and filtered according to the product of classification confidence score and polar centerness. The distance between the sample point and N rays is known, and the coordinates of the corresponding object edge points are connected one by one starting from 0° to form the contour of the corresponding instance.
[0058] Specifically, the YOLACT network generation includes the following steps: inputting the remote sensing image into the YOLACT network, extracting image features in the feature backbone network and inputting them into the prototype mask generation network branch and the mask coefficient branch.
[0059] The prototype mask generation network branch is used to generate a set of prototype masks with the same size as the original image through a fully convolutional network.
[0060] The mask coefficient branch is used to generate the mask coefficient of each instance, predict the category of the candidate box, and the bounding box position.
[0061] The prototype mask is linearly combined with the corresponding mask coefficients to generate an instance mask, which is then cropped using the predicted bounding box to obtain the instance segmentation result.
[0062] Specifically, the data set is to label the RGB image of ice-covered power lines taken by a dual-light camera with LabelImg to obtain a labeled data set, wherein the label format of the labeled data set is a PNG image.
[0063] Furthermore, each PNG image label corresponds to a mask of the target area of the original image. PNG image labels are all binary images, in which the white part represents the foreground of the corresponding source image, that is, the area of interest, and the black part represents the background. A self-written program is used to convert the XML file generated by LabelImg annotation into a PNG image label.
[0064] Specifically, the evaluation index is to calculate the AP values of all categories and average them to get the mAP value. The calculation formula of the mAP value is as follows:
[0065]
[0066] Among them, mAP is the mAP value of each category; C is the number of target categories, including pig category and pig_bad category; AP i It is the average precision of the i-th category, which is used to quantitatively evaluate the performance of the algorithm. The AP value is obtained by calculating the recall rate and the average precision between 0 and 1.
[0067] Specifically, the model training includes the following steps: taking the PNG image label and the original image as input images and setting the model training parameters, which include image size, backbone network, batch size, learning rate, and number of iterations.
[0068] Preprocess and resize the input image.
[0069] The input image is processed using the ResNet backbone network and FPN to obtain multi-scale features, which are then input into the parallel detection module for coefficient combination prediction.
[0070] The p3 feature map output by the feature pyramid is input into the prototype network to generate a pre-segmentation model and predict the prototype mask of the image.
[0071] The fast non-maximum suppression method is used to remove the interference detection box, and the prototype mask is combined to generate the prediction box and the mask of the ice-covered transmission line area.
[0072] The loss functions of the detection box, classification and mask are calculated respectively, and the model weights are updated based on the gradient backpropagation method.
[0073] Repeat the above process until the model meets the stability conditions or other requirements.
[0074] Specifically, by constructing a pre-segmentation model, it is possible to ensure high accuracy when processing conductor images containing towers with similar straight-line features under complex backgrounds. It is possible to accurately segment conductor images under complex backgrounds of woodlands, fog, land, and partial freezing. It is also possible to perform conductor image segmentation considering different resolutions, conductor bends, and irregular ice coverage.
[0075] S3: Based on the multi-directional Sobel edge operator design, the infrared image data of the transmission line is collected and the boundary is extracted to obtain the original transmission line area mask.
[0076] Preferably, obtaining the original transmission line area mask includes multi-directional Sobel edge operator design, infrared image binarization based on the great law, image filtering based on nonlinear operators, a boundary tracing algorithm of topological structure, and original transmission area MASK extraction.
[0077] Specifically, the multi-directional Sobel edge operator is designed to use a multi-directional Sobel operator, that is, two templates in different directions are added on the basis of the traditional Sobel operator, and the templates in the directions of 0°, 90°, 45° and 135° are used to extract edge information. The specific formula of the template is as follows:
[0078]
[0079] Furthermore, the closer to the pixel point, the larger the weighted value. The weighted difference of the neighborhood of each pixel point on the image is calculated. When the weighted difference reaches an extreme value, it is set as the threshold. If the weighted difference of the pixel point is greater than the threshold, the corresponding pixel point is retained. If the weighted difference of the pixel point is less than the threshold, the corresponding pixel point is removed to obtain the edge information of the transmission line for subsequent regional mask screening.
[0080] Specifically, since the direction of the transmission line in the infrared image is uncertain, and the classic Sobel operator only uses templates in the horizontal and vertical directions, the accuracy of the extracted transmission line edge is low. Therefore, the present invention uses a multi-directional Sobel operator to more accurately extract the edge information of the transmission line in the infrared image.
[0081] Specifically, the infrared image binarization based on the big law includes the following steps: graying the infrared image data to obtain a grayscale image.
[0082] Based on the Otsu algorithm, the grayscale image is binarized and divided into L grayscale levels. The calculation formula for the probability of each grayscale level is as follows:
[0083]
[0084] Among them, p i is the probability of each gray level; ni is the number of pixels with gray level i; N is the total number of pixels in the image; L is the number of gray levels.
[0085] The grayscale image is divided into C0 and C1 based on the grayscale value, where C0 is [1, 2, ..., k], C1 is [k+1, ..., L], the binarization threshold is k, and the calculation formula for the occurrence probability of C0 and C1 is as follows:
[0086]
[0087] Among them, ω0 is the probability of occurrence of class C0; ω1 is the probability of occurrence of class C1; p i is the probability of each gray level; k is the binarization threshold; ω(k) is the probability of occurrence less than the binarization threshold k.
[0088] The calculation formula for the mean of C0 and C1 is as follows:
[0089]
[0090] Among them, μ0 is the mean of class C0; μ1 is the mean of class C1; ω(k) is the probability of occurrence less than the binarization threshold k; ω0 is the probability of occurrence of class C0; ω1 is the probability of occurrence of class C1; p i is the probability of each gray level; μ T is the sum of the classes.
[0091] The calculation formula for the variance of class C0 and class C1 is as follows:
[0092]
[0093] in, is the variance of class C0; is the variance of class C1.
[0094] The discriminant index is used to judge the effect of binarization. The calculation formula of the discriminant index is as follows:
[0095]
[0096] in, is the intra-class variance; is the between-class variance; is the population variance; since κ=1+λ,η=λ / (1+λ), when the discriminant index λ is the largest, the discriminant index λ, the discriminant index κ and the discriminant index η are all at their maximum values, and the calculation formula for the relationship between the discriminant indexes is as follows:
[0097]
[0098] Among them, λ is the discriminant index of the binarization effect.
[0099] Furthermore, due to the population variance It has nothing to do with the selection of the binarization threshold k, so when the between-class variance When it is maximum, the discriminant index λ, the discriminant index κ and the discriminant index η are all at their maximum values, and the calculation formula for the relationship between the inter-class variance and the binarization threshold is as follows:
[0100]
[0101] Among them, k is the binarization threshold; k * is the optimal binarization threshold, and the optimal binarization threshold k is obtained by traversing the binarization threshold k * , to achieve the best infrared image binarization processing.
[0102] Specifically, by using the threshold calculated by the Otsu algorithm, the inter-class variance between the background image and the foreground image segmented according to the threshold can be maximized, and the Otsu algorithm has a certain degree of adaptability and can achieve relatively ideal binarization under different lighting conditions.
[0103] Specifically, image filtering based on nonlinear operators is when the infrared sensor is capturing images. When the imaging element or transmission system is unstable, the visualized infrared image is prone to generate dark and bright spots of random positions and varying sizes, which are generally called noise points. The noise points will interfere with the subsequent acquisition of edge arrays. In order to reduce the impact of the noise points, image filtering is used to denoise the image.
[0104] Specifically, the boundary tracing algorithm of the topological structure includes the determination of the starting point, the boundary judgment criterion and the search termination criterion.
[0105] Furthermore, the determination of the starting point plays a decisive role in the boundary search of certain images, and different starting points may lead to tracking of different boundaries.
[0106] Furthermore, the boundary discrimination criterion is the core of the boundary tracking algorithm, which is used to guide the next search direction of the algorithm and is related to the accuracy of the boundary search algorithm, in which the Square algorithm is used for calculation.
[0107] Further, such as Figure 1 to Figure 3 The figure shows the Square boundary tracking algorithm and its limitations. The search direction of the Square algorithm is from bottom to top and from left to right. The first target pixel encountered is the starting point, and the direction of the arrow is the forward direction. The Square algorithm includes the following steps: If the pixel is not a target pixel, then go to the adjacent pixel to the right.
[0108] If the pixel is the target pixel, turn left and enter the adjacent pixel.
[0109] Repeat the above steps until you reach the starting point again.
[0110] The target pixel points passed through the above steps are the boundaries of the target image.
[0111] Specifically, the original transmission area MASK extraction includes the following steps: the boundaries of the transmission lines in the infrared image are extracted in the form of a chain code array through a boundary tracing algorithm to obtain edge data.
[0112] Guided by the acquired edge data, the edge information is used to filter out the correct boundary of the transmission line area, that is, the boundary chain code array.
[0113] Calculate the minimum circumscribed rectangle of the boundary chain code array to obtain the optimal original transmission line mask.
[0114] Furthermore, the minimum enclosing rectangle includes the following steps: Since the polygon M has 4 vertices in the X-axis direction and the Y-axis direction, which correspond to the maximum values of the horizontal coordinate and the maximum values of the vertical coordinate, perpendicular lines are drawn to the four vertices in the X-axis and Y-axis directions. The specific formula is as follows:
[0115]
[0116] Among them, y max is the maximum value of the vertical coordinate; min is the minimum value of the vertical coordinate; x max is the maximum value of the horizontal axis; x min is the minimum value of the horizontal axis; M ymax is the maximum ordinate value among all vertices of polygon M; ymin is the minimum ordinate value among all vertices of polygon M; xmax is the maximum value of the horizontal coordinate among all vertices of polygon M; xmin is the minimum value of the horizontal coordinate among all the vertices of polygon M.
[0117] Use four straight lines to form a circumscribed rectangle. If one of the sides of the circumscribed rectangle coincides with a side of polygon M, the area of the current circumscribed rectangle is recorded and recorded as the minimum value. If the circumscribed rectangle and polygon M do not have overlapping sides at this time, the area of the current circumscribed rectangle is not recorded, and the straight line is rotated clockwise until it has overlapping sides with polygon M. The area of the new circumscribed rectangle is calculated at this time. If the area of the new circumscribed rectangle is less than the previously saved minimum value, the area of the circumscribed rectangle is updated, otherwise it is discarded. Repeat the steps until the rotation angle of the straight line is greater than 90 degrees. The circumscribed rectangle corresponding to the saved minimum value is the minimum circumscribed rectangle of polygon M.
[0118] Specifically, by utilizing the multi-directional Sobel and Otsu binarization algorithms, the adaptability to complex scenes is enhanced, the edge information of the transmission lines can be accurately extracted, and better binarization effects can be achieved under different lighting conditions.
[0119] S4: Perform camera calibration and image fusion and calculation on the ice-covered image and infrared image data to obtain the standard thickness of ice covering the transmission line.
[0120] Preferably, calculating the ice thickness of the transmission line includes calculating the physical model of the ice thickness of the transmission line, optical RGB calibration and infrared camera calibration, optical RGB-infrared transmission line area MASK registration and fusion, and measuring the standard thickness of ice on the transmission line.
[0121] Specifically, the physical model for calculating the ice thickness of transmission lines includes the following steps: based on the principle of pixel similarity, research is conducted on the calculation of the equivalent ice thickness of transmission lines based on ice images, that is, the ratio of the number of pixels corresponding to the ice-covered area and the line area is equal to the ratio of the equivalent thickness of the ice-covered area to the line diameter.
[0122] like Figure 4 and Figure 5 The figure shows the calculation principle of equivalent ice thickness. When the camera optical axis is orthogonal to the line axis, the line edges are parallel to each other. Assuming that the total number of pixels in the ice area in the x1-x2 region is P D , the total number of pixels in the original transmission line area is P d According to the parallelogram area formula and pixel similarity relationship, the calculation formula for the equivalent thickness of ice covering the transmission line is as follows:
[0123]
[0124] Where t is the equivalent thickness of ice on the transmission line; D is the equivalent diameter of the ice-covered area; d is the original transmission line diameter; P D P is the total number of pixels in the ice-covered area in the x1-x2 region; d is the total number of pixels in the original transmission line area.
[0125] Based on the image-based ice equivalent thickness calculation method and combined with the internal and external parameters of the camera, the input ice-covered transmission line area mask is corrected, registered and fused to obtain the mask pixel information of the "transmission line + ice-covered" area, and then the ice thickness is obtained.
[0126] Furthermore, in actual ice-covered scenes, ice-covered images do not always satisfy the parallelism between the upper and lower edges of the line. When the optical axis of the camera is at a certain angle to the axis of the transmission line, the captured ice-covered image of the transmission line has perspective distortion, that is, objects appear larger when near and smaller when far away. Therefore, it is necessary to consider inverse perspective transformation of the input image. Secondly, due to the different optical axes and size offsets of the optical RGB camera and the infrared camera, it is necessary to consider image registration and fusion to facilitate the subsequent judgment of pixel information of the mask of the relevant area.
[0127] Specifically, optical RGB calibration and infrared camera calibration are studies on the calibration process of camera internal and external parameters, where camera calibration includes internal parameters and external parameters.
[0128] Furthermore, calculating the intrinsic parameters includes the following steps: preparing a calibration plate, on which there are a series of points with known coordinates, wherein the points with known coordinates are obtained by measurement or generated by software.
[0129] Aim the camera at the calibration plate and take a series of pictures, where the pictures cover the entire plane of the calibration plate. Fix the camera on a tripod and take 30 pictures of the chessboard calibration plate at different distances and angles.
[0130] Feature points on the calibration plate are extracted from the captured images, where the feature points are obtained through various feature extraction algorithms, such as SIFT algorithm or SURF algorithm.
[0131] The intrinsic parameters of the camera are calculated by minimizing the reprojection error, usually using the Levenberg-Marquardt optimization algorithm.
[0132] Specifically, the intrinsic parameters include focal length, principal point coordinates, pixel size, skew coefficient, and distortion, which accurately describe the imaging process of the camera and perform various complex image processing tasks.
[0133] Furthermore, the extrinsic parameters include pitch angle and yaw angle.
[0134] Specifically, when a three-dimensional stereoscopic image is projected onto a two-dimensional plane, the image may experience perspective distortion. Two originally parallel lines in the stereoscopic space intersect at a point in the two-dimensional plane, namely, the vanishing point. If two parallel lines in the three-dimensional space are parallel to the image plane, the vanishing point is located at infinity at the image plane, otherwise the vanishing point is located at a finite distance from the image plane, namely, when there is a yaw angle, the horizontal coordinate of the vanishing point is offset; when there is a pitch angle, the vertical coordinate of the vanishing point is offset.
[0135] Furthermore, the calculation formula of the yaw angle is as follows:
[0136]
[0137] Among them, γ is the camera yaw angle; f is the camera focal length; l is the distance between the camera and the transmission line; u1 is the horizontal coordinate of the vanishing point of the imaging plane when the camera has a yaw angle and a pitch angle; m is the size of the imaging plane.
[0138] Furthermore, the calculation formula of the pitch angle is as follows:
[0139]
[0140] Wherein, θ is the pitch angle of the camera; v1 is the ordinate of the vanishing point of the imaging plane when the camera has a yaw angle and a pitch angle.
[0141] Specifically, the optical RGB-infrared transmission line area mask registration and fusion is that there are relative translation, rotation, different scaling and distortion relationships between the images of the same object obtained by different sensors and different imaging modes. Therefore, before using the optical RGB image and the infrared image for the transmission line area mask fusion, strict geometric registration must be ensured. The optical RGB-infrared binocular imaging device with parallel optical axes is imaged. The main limitation is that the image registration is only for a specific target distance D target efficient.
[0142] Furthermore, the registration error is calculated as follows:
[0143]
[0144] Among them, σ x is the registration error; f is the focal length; d c is the baseline length; I pix is the pixel size; D target For a specific target distance; D optimal is the target distance, indicating that the image alignment error is 0. If the optical axes are parallel, that is, the target distance D optimal =∞, accurate alignment is only possible when the target object is far away from the camera. target If the value is less than ∞, there will be a registration error. The calibration methods are optical RGB calibration and infrared camera calibration. Attention should be paid to the image scale factor and the offset of infrared relative to visible light image.
[0145] Further, such as Figure 6 The figure shows the process of measuring the standard ice thickness of transmission lines. In actual ice scenes, not all ice images are parallel to the upper and lower edges of the line. Therefore, when shooting lines or ice images, it is necessary to ensure that the number of pixels and image size of the line or ice area meet the requirements of the line or the entire ice area. The ratio of radial pixel number to image size is 1:2 to 1:3, and the straight-line distance from the optical center of the camera to the line is 50 cm to 200 cm.
[0146] Specifically, before calculating the standard ice thickness of the transmission line, the equivalent ice thickness of the transmission line is calculated to obtain the effective ice radius, and then the standard ice thickness of the transmission line is calculated.
[0147] Specifically, the calculation of the equivalent thickness of ice on the transmission line is to extract the ice-covered transmission line area and the actual transmission line area in the RGB optical image and the infrared image respectively, and then correct, register and align the two mask images in combination with the calculated resolution parameters, intrinsic parameters and extrinsic parameters of the infrared camera and the RGB optical camera, and mark the accurate "transmission line + ice-covered" area mask. By adapting the pixel-level characterization results of the transmission line area to the actual size, and combining the physical model for calculating the ice thickness of the transmission line with the "transmission line + ice-covered" area mask pixel information, the calculation formula for the equivalent thickness of ice on the transmission line is as follows:
[0148]
[0149] Among them, t dx is the equivalent thickness of ice covering the transmission line; d is the original transmission line diameter; P D is the total number of pixels in the ice-covered transmission line area; P d is the total number of pixels in the original transmission line area.
[0150] Furthermore, due to the diversity of transmission line icing types and differences in shooting angles, the standard thickness of transmission line icing is calculated by combining the identified icing type results and the equivalent thickness of transmission line icing, so as to provide an important reference for subsequent transmission line maintenance decisions.
[0151] Specifically, according to the "Regulations for Survey of Icing on Overhead Transmission Lines" (DL / T 5509-2015), the calculation formula for the standard thickness of ice on transmission lines is as follows:
[0152]
[0153] Where B0 is the standard thickness of ice covering the transmission line; ρ is the estimated density of ice covering; K s is the ice-covered shape coefficient; R is the ice-covered radius, in mm; r is the original transmission line radius, in mm; t dx is the equivalent thickness of ice covering the transmission line; d is the original diameter of the transmission line.
[0154] Furthermore, the ice shape coefficient is determined by calculation and analysis of local measured ice data. For areas without measured data, it can be selected in accordance with regulations based on the identified ice type results. When small ice cover occurs, a low value of the ice shape coefficient should be used; when large ice cover occurs, a high value of the ice shape coefficient should be used.
[0155] Specifically, by utilizing the physical model of equivalent ice thickness and the camera calibration method, a mathematical relationship between ice thickness and image features was constructed. Combined with the regional mask information of dual-light images, the accurate measurement of the standard ice thickness of transmission lines was achieved, providing a key reference for transmission line maintenance.
[0156] The present embodiment also provides a transmission line ice thickness detection system based on deep learning and dual-light images, including: an ice image module, which is used to collect ice images of transmission lines and perform image preprocessing, construct an original data set, and use a pre-segmentation model to calculate to obtain an ice-covered transmission line area mask; an infrared image module, which is used to collect infrared image data of transmission lines based on a multi-directional Sobel edge operator design and perform boundary extraction to obtain an original transmission line area mask; an ice thickness module, which is used to calculate the equivalent ice thickness of the transmission line by combining the ice-covered transmission line area mask and the original transmission line area mask, obtain the ice radius and calculate to obtain the standard ice thickness of the transmission line.
[0157] This embodiment also provides a computer device, which is applicable to the case of a method for detecting ice thickness on a transmission line based on deep learning and dual-light images, and includes: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the method for detecting ice thickness on a transmission line based on deep learning and dual-light images as proposed in the above embodiment.
[0158] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.
[0159] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for detecting ice thickness of a power transmission line based on deep learning and dual-light images as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a disk or an optical disk.
[0160] In summary, the present invention adopts a single-stage instance segmentation algorithm and a YOLACT network, and uses a polar coordinate representation method for coding masks, which not only improves the segmentation accuracy, but also significantly reduces the computational complexity, realizes accurate positioning and contour extraction of the ice-covered area, and provides accurate regional information for subsequent ice thickness calculation; a multi-directional Sobel edge operator is designed to make edge detection more comprehensive and accurate, enhance the ability to recognize the outline of the transmission line, effectively reduce the influence of background interference and noise, and improve the accuracy of the original transmission line area mask; through the fusion of dual-spectrum images, the respective advantages of visible light and infrared light are fully utilized, the ability to recognize ice conditions is enhanced, and the thickness calculation method based on the physical model takes into account the ice shape and density factors, which improves the measurement accuracy. By introducing a standardized ice thickness calculation formula, the measurement results have stronger universality and comparability, and non-contact, fast and accurate ice thickness measurement of ice-covered lines is realized, providing a basis for line safety assessment for maintenance personnel.
[0161] Example 2
[0162] Referring to Tables 1 to 3, which are the second embodiment of the present invention, this embodiment is different from the first embodiment in that, in order to verify its beneficial effects, operating data and related instructions of the present invention in an actual environment are provided.
[0163] In this example, the original data set is constructed by using standardized dual-light optical equipment to collect ice-covered images of transmission lines and performing size transformation, brightness transformation, color enhancement, contrast enhancement and image denoising. Table 1 shows the size transformation strategy.
[0164] Table 1 Size transformation strategy table
[0165] Flag Parameters Brief Notes effect INTER_NEAREST 0 Nearest neighbor interpolation INTER_LINEAR 1 Bilinear interpolation INTER_CUBIC 2 Bicubic interpolation INTER_AREA 3 Preferred for image reduction, image enlargement is similar to nearest INTER_LANCZOS4 4 Lanczos interpolation INTER_LINEAR_EXACT 5 Bit-accurate bilinear interpolation INTER_MAX 6 Mask interpolation
[0166] The ice-covered transmission line images are preprocessed by using the size transformation strategy and other processing measures in the above table, the original data set is constructed, and the pre-segmentation model is trained using the model training parameters, where Table 2 shows the model training parameter table, and then the mask of the ice-covered transmission line area is obtained to achieve accurate segmentation of the conductor image.
[0167] Table 2 Model training parameters
[0168] Parameter name Parameter Value Image size 640*640 Backbone network ResNet50+FPN Batch size 3 Learning Rate 0.0001 Iterations 360000
[0169] When calculating the standard ice thickness of the transmission line, this example collects data from a region without measured data and selects the ice shape coefficient based on the identified ice type results, as shown in Table 3. When a small ice phenomenon occurs, a lower value of the ice shape coefficient should be used; when a large ice phenomenon occurs, a higher value of the ice shape coefficient should be used.
[0170] Table 3 Ice shape coefficient and estimated density
[0171]
[0172] The present invention processes the ice-covered image and the infrared image of the transmission line according to the above table to obtain the mask of the ice-covered transmission line area and the mask of the original transmission line area, and then calculates the equivalent thickness of the ice on the transmission line and calculates according to the effective ice-covered radius to obtain the standard thickness of the ice on the transmission line, thereby enhancing the ability to identify the ice situation, realizing non-contact, fast and accurate ice thickness measurement of the ice-covered line, and providing a basis for line safety evaluation for subsequent maintenance personnel.
[0173] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A method for detecting ice thickness on power transmission lines based on deep learning and dual-light images, characterized in that: include: Collect ice-covered images of transmission lines and perform image preprocessing to construct the original data set; Segmenting the original data set, building a pre-segmentation model and performing calculations to obtain a mask of the ice-covered transmission line area; Based on the multi-directional Sobel edge operator design, the infrared image data of the transmission line is collected and the boundary is extracted to obtain the original transmission line area mask; Camera calibration and image fusion are performed on the ice-covered image and the infrared image data, and calculation is performed to obtain the standard thickness of ice covering the transmission line.
2. The method for detecting ice thickness of power transmission lines based on deep learning and dual-light images according to claim 1, characterized in that: The construction of the original data set is to collect ice-covered images of power transmission lines using standardized dual-light optical equipment and perform image preprocessing to construct the original data set; The image preprocessing includes size conversion, brightness conversion, color enhancement, contrast enhancement and image denoising.
3. The method for detecting ice thickness of power transmission lines based on deep learning and dual-light images according to claim 2, characterized in that: The construction of the pre-segmentation model includes a single-stage instance segmentation algorithm, YOLACT network generation, data set, evaluation index and model training; The single-stage instance segmentation algorithm includes the following steps: Using the polar coordinate representation encoding mask method, each pixel in the target centroid area is classified and regressed to the dense distance between the centroid and the contour; Extract features from the feature extraction backbone network and input them into the classification branch and regression branch. The classification branch includes category classification and polar centerness. Take a point in the feature map as the polar coordinate center point, generate N rays at a fixed angle, calculate the ray distance from each target edge to the center point, and calculate based on the ray distance to obtain the optimal mask; In each FPN structure, Q best samples are retained and filtered according to the product of classification confidence score and polar centerness. The distance between the sample point and N rays is known, and the coordinates of the corresponding object edge points are connected one by one starting from P to form the contour of the corresponding instance.
4. The method for detecting ice thickness of power transmission lines based on deep learning and dual-light images according to claim 3, characterized in that: The method of obtaining the original transmission line area mask includes designing a multi-directional Sobel edge operator, binarizing an infrared image based on the Great Law, filtering an image based on a nonlinear operator, tracing a boundary algorithm of a topological structure, and extracting the original transmission area mask; The multi-directional Sobel edge operator is designed to use a multi-directional Sobel operator, adding A templates in different directions on the basis of the traditional Sobel operator, and extracting edge information from the templates in directions A1, A2, A3 and A4 respectively. The specific formula of the template is as follows: The closer to the pixel point, the larger the weighted value. The weighted difference of the neighborhood of each pixel point on the image is calculated. When the weighted difference reaches the extreme value, it is set as the threshold. If the weighted difference of the pixel point is greater than the threshold, the corresponding pixel point is retained. If the weighted difference of the pixel point is less than the threshold, the corresponding pixel point is removed to obtain the edge information of the transmission line.
5. The method for detecting ice thickness of power transmission lines based on deep learning and dual-light images according to claim 4, characterized in that: The transmission line ice thickness includes a physical model for calculating the transmission line ice thickness, optical RGB calibration and infrared camera calibration, optical RGB-infrared transmission line area MASK registration and fusion, and transmission line ice standard thickness measurement; The physical model for calculating ice thickness of transmission lines comprises the following steps: Since the edges of the line are parallel to each other when the camera optical axis is orthogonal to the line axis, it is assumed that the total number of pixels in the ice-covered area in the x1-x2 region is P D , the total number of pixels in the original transmission line area is P d , the calculation formula for the equivalent thickness of ice covering the transmission line is as follows: Where t is the equivalent thickness of ice on the transmission line; D is the equivalent diameter of the ice-covered area; d is the original transmission line diameter; P D P is the total number of pixels in the ice-covered area in the x1-x2 region; d is the total number of pixels in the original transmission line area.
6. The method for detecting ice thickness of power transmission lines based on deep learning and dual-light images according to claim 5, characterized in that: The optical RGB-infrared power line area mask registration and fusion is because there are relative translation, rotation, different scaling and distortion relationships between the images of the same object obtained by different sensors and different imaging modes. Therefore, before using the optical RGB image and the infrared image to perform power line area mask fusion, strict geometric registration must be ensured, and image registration must be performed on the optical RGB-infrared binocular imaging device with parallel optical axes; The registration error is calculated as follows: Among them, σ x is the registration error; f is the focal length; d c is the baseline length; I pix is the pixel size; D target For a specific target distance; D optimal is the target distance.
7. The method for detecting ice thickness of power transmission lines based on deep learning and dual-light images according to claim 6, characterized in that: In actual ice-covered scenes, ice-covered images are not always parallel to the upper and lower edges of the line. Therefore, when shooting lines or ice-covered images, it is necessary to ensure that the number of pixels and image size of the line or ice-covered area meet the requirements of the line or the entire ice-covered area. The ratio of radial pixel number to image size is B1~B2, and the straight-line distance from the camera optical center to the line is C1~C2. The calculation formula for the standard ice thickness of the transmission line is as follows: Where B0 is the standard thickness of ice covering the transmission line; ρ is the estimated density of ice covering; K s is the ice-covered shape coefficient; R is the ice-covered radius; r is the original transmission line radius; t dx is the equivalent thickness of ice covering the transmission line; d is the original diameter of the transmission line.
8. A power transmission line ice thickness detection system based on deep learning and dual-light images, based on the power transmission line ice thickness detection method based on deep learning and dual-light images according to any one of claims 1 to 7, characterized in that: include, The ice-covered image module is used to collect ice-covered images of transmission lines and perform image preprocessing, construct the original data set, and use the pre-segmentation model to calculate and obtain the mask of the ice-covered transmission line area; Infrared image module, which is used to collect infrared image data of transmission lines and perform boundary extraction based on multi-directional Sobel edge operator design to obtain the original transmission line area mask; The ice thickness module is used to calculate the equivalent ice thickness of the transmission line by combining the ice-covered transmission line area mask and the original transmission line area mask, obtain the ice radius and perform calculations to obtain the standard ice thickness of the transmission line.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for detecting ice thickness of transmission lines based on deep learning and dual-light images as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting ice thickness of a power transmission line based on deep learning and dual-light images as described in any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Line icing detection method based on deep learning
CN119169535A
Method and device for determining icing thickness of power line, medium and program
CN119579676A
Power transmission line floating ice monitoring method and system, electronic equipment and medium
CN119762435A
A power transmission line ice floe monitoring method and system, electronic device and medium
CN119762435B
Power transmission line icing type detection method and system based on hyperspectral satellite remote sensing
CN119848687A
