A method for extracting features of transmission line conductors based on image segmentation

Abstract

A method for extracting features of transmission line conductors based on image segmentation is disclosed. The method includes: coarsely segmenting the conductor portion of an image using an image segmentation model; further processing using a local thresholding method to obtain key points of the transmission line, resulting in a key point map; obtaining an expanded region for the key points and determining the main direction of the key points; obtaining the feature extraction region for the key points; and using a feature extraction algorithm to obtain the conductor features of the image. This invention uses an image segmentation model and a local thresholding method to obtain and extract key points of the transmission line conductors from images captured by a visual monitoring device. It can extract not only points with drastic changes, such as corners, but also points with relatively gentle changes. The subsequent determination of the main direction of the key points primarily serves to reduce errors caused by rotation of the shooting perspective.

Description

Technical Field

[0001] This invention belongs to the category of multi-view Figure 3 In the field of dimensional reconstruction technology, this invention relates in particular to a method for extracting features of transmission line conductors based on image segmentation. Background Technology

[0002] Accurately and efficiently reconstructing realistic three-dimensional scenes from two-dimensional images has long been a challenging problem. However, with the development of computer graphics, image processing, and other computer technologies, a clear solution has emerged. Stereo vision is one important solution, using inverse imaging techniques to extract depth information from images and then reconstruct the three-dimensional spatial structure of objects. The quality of image feature extraction, a crucial component of stereo vision, is a significant factor influencing the reconstruction of the three-dimensional spatial structure of the scene.

[0003] Image feature extraction, as one of the most fundamental and important topics in computer vision, has significant applications in areas such as image target recognition and video detection. Image feature extraction mainly involves extracting information such as angles, brightness, and contours from an image using specific techniques. In the power transmission field, traditional algorithms such as SIFT, SUFT, and ORB are primarily used for extracting features from power transmission line images. However, these algorithms suffer from problems such as a limited number of feature points and poor feature point quality during the extraction process, which severely hinders the application of power transmission line images. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a method for extracting features of transmission line conductors based on image segmentation.

[0005] To achieve the above objectives, the transmission line conductor feature extraction method based on image segmentation provided by the present invention includes the following steps performed in sequence:

[0006] 1) Use an image segmentation model to coarsely segment the conductor portion in the image captured by the power transmission line visualization monitoring equipment to obtain the coarsely segmented power transmission conductor image;

[0007] 2) The image of the transmission line after coarse segmentation is further processed using the local thresholding method to obtain the key points of the transmission line. At the same time, in the image captured by the transmission line visualization monitoring equipment, all pixel values ​​except the key points of the transmission line are set to 0 to obtain the key point map of the transmission line.

[0008] 3) In the key point map of the transmission line, take the key point of the transmission line of a single transmission line as the center, obtain the expanded area of ​​the key point of the transmission line, then calculate the gradient histogram matrix within the expanded area of ​​the key point of the transmission line, and smooth the gradient histogram matrix. Then, use the key point main direction determination method to determine the main direction of the key point of the transmission line through the smoothed gradient histogram matrix.

[0009] 4) In the key point map of the transmission line, take the key point of the transmission line of a single transmission line as the center, obtain the key point feature extraction area of ​​the transmission line. Then, with the help of the main direction of the key point of the transmission line, use the transmission line feature extraction algorithm to extract each 1*4*4*8 key point feature from the key point feature extraction area of ​​the transmission line. Merge all the extracted key point features of the transmission line to finally obtain the transmission line features of the image.

[0010] In step 1), the transmission line visualization monitoring equipment refers to the monitoring equipment installed on existing transmission line towers that can acquire visual images, including night vision cameras, visible light cameras and video recording equipment;

[0011] The image captured by the power transmission line visualization monitoring device is a matrix of 1944*2592*3, where 1944 is the number of horizontal pixels, 2592 is the number of vertical pixels, and 3 represents the three channels of the image, which respectively represent the values ​​of the three colors red (R), green (G), and blue (B) in the pixel. The value range is 0-255, and each pixel has these three color values.

[0012] The image segmentation model uses deep neural network models including DeeplabV3+, CPS and U2PL, and the data used to train the model includes the category of power transmission lines.

[0013] The coarse segmentation involves using images captured by the power transmission line visualization monitoring equipment as input to an image segmentation model. The model segments the image by assigning a category label to each pixel in the input image to indicate its category. If the pixel belongs to the power transmission line category, its value is retained, while the values ​​of pixels belonging to other categories are assigned to 0. This process yields the coarsely segmented image of the power transmission line.

[0014] In step 2), the method for further processing the coarsely segmented transmission line image using the local thresholding method to obtain key points of the transmission line is as follows: First, the second channel values ​​of the coarsely segmented transmission line image are sorted from smallest to largest to obtain sorted second channel values. Then, the sorted second channel values ​​are used to obtain a local threshold. The local threshold is obtained by taking the average of the last 10 sorted second channel values ​​and subtracting the average of the first 10 sorted second channel values ​​from the average of the first 10 sorted second channel values, and then multiplying the difference by 0.78. Finally, pixels with values ​​greater than the local threshold are retained as key points of the transmission line.

[0015] The key point map of the transmission line is a set of second channel values ​​of all key points of the transmission line, represented as a matrix of 1944*2592*1. In this matrix, the element values ​​of non-key point coordinates of the transmission line are 0, and the other values ​​are the second channel values ​​of the key points of the transmission line.

[0016] In step 3), the expansion area of ​​the key points of the transmission line is an image area that expands three grids upwards, downwards, leftwards, and rightwards, centered on the key points of the transmission line of a single transmission line. If the expansion area of ​​the key points of the transmission line exceeds the key point image of the transmission line, the excess part is removed.

[0017] The calculation of the gradient histogram matrix within the expanded region of key points on the transmission line involves calculating the gradient histogram matrix for each key point. Each key point has a 1*36 gradient histogram matrix, representing the division of 360 degrees into 36 equal parts: 0-10 degrees, 10-20 degrees, ..., 350-360 degrees. Each element of the gradient histogram matrix represents the cumulative gradient value within each degree range. Then, the gradient and gradient direction of each pixel within the expanded region of the key points are selected and added to the corresponding element of the gradient histogram matrix. The formulas for calculating the gradient and gradient direction of each pixel within the expanded region of the key points are as follows:

[0018]

[0019] θ=tan -1 ((M[x+1,y]-M[x-1,y])÷(M[x,y+1]-M[x,y-1]))

[0020] Where G is the gradient of a pixel, θ is the gradient direction of a pixel, M is the key point map matrix of the transmission line, and x and y are the horizontal and vertical coordinates of each pixel in the key point map matrix M of the transmission line in the extended region of the key points of the transmission line, respectively.

[0021] The gradient histogram smoothing process refers to traversing the gradient histogram and smoothing the data for each element encountered and its surrounding elements. The smoothing process is shown in the following formula:

[0022] g0 = Gs[idx-1] ÷ 2

[0023] g1 = Gs[(idx+1)%36]÷2

[0024] g2=Gs[idx-2]÷2

[0025] g3=Gs[(idx+2)%36]÷2

[0026] Gs1[idx]=(6×Gs[idx]+g0+g1+g2+g3)÷8

[0027] Where Gs is the gradient histogram, idx is the element index of the gradient histogram Gs, and Gs1 is the smoothed gradient histogram, which is obtained by iterating through all elements of the gradient histogram Gs by changing the value of the element index idx; g0 is the contribution value of the left neighbor element to the smoothed gradient histogram element at index idx, and g1 is the contribution value of the right neighbor element to the smoothed gradient histogram element at index idx. 2 represents the contribution of the left neighboring element to the smoothed gradient histogram element at index idx, and g3 represents the contribution of the right neighboring element to the smoothed gradient histogram element at index idx. If idx-1 and id-2 are less than 0, then Gs[idx-1] and Gs[idx-2] represent taking values ​​from the reverse direction of the gradient histogram Gs. (idx+2)%36 is to ensure that the matrix index can always take values ​​between 0 and 35 when taking values. % is the modulo operator.

[0028] The method for determining the principal direction of key points of the transmission line is to take the index Mi of the maximum value of the smoothed gradient histogram matrix Gs1, and then process the maximum value in the smoothed gradient histogram matrix Gs1 to obtain the principal direction of the key points of the transmission line; the specific method is shown in the following formula:

[0029] Ls = Gs1[(Mi-1)%36]

[0030] Lm = Gs1[(Mi+1)%36]

[0031] Ll=Gs1[Mi]+0.5×(Lm-Ls)÷(Lm-2*Gs1[Mi]+Ls)%36

[0032] D = 360 - (Ll) × 360 ÷ 36

[0033] Where D represents the principal direction of the key point of the transmission line, Gs1 is the smoothed gradient histogram, Mi is the index of the maximum value of the smoothed gradient histogram Gs1, and % represents the modulo operator. Ls is the left gradient direction correction value of the principal direction of the key point of the transmission line, Lm is the right gradient direction correction value of the principal direction of the key point of the transmission line, and LI is the corrected gradient direction. (Mi-1)%36 and (Mi+1)%36 are used to ensure that the index can be between 0 and 35 when taking values.

[0034] In step 4), the key point feature extraction region of the transmission line is an image region that expands 32 grids upwards, downwards, leftwards, and rightwards, centered on the key point of the transmission line of a single transmission line. If the range of the key point feature extraction region of the transmission line exceeds the key point image of the transmission line, the excess part is removed.

[0035] The aforementioned power transmission line feature extraction algorithm is a method for extracting features from key points of a single power transmission line. It takes the key point of the power transmission line as input and the features of that key point as output. The output consists of 4*4 seed points, each with 8 directions. This represents dividing a circle into 8 parts, each 45°, with the 8 directions sequentially represented as 0-45°, 45-90°, ..., 315-360°, ultimately represented as a 1*4*4*8 seed point matrix Z. The specific operation is as follows: First, each element of the key point feature extraction region of the power transmission line is mapped and feature extracted, resulting in the index x1 of the second dimension of the seed point matrix, and the index x1 of the third dimension of the seed point matrix. The key features of the transmission line are determined by four variables: the index y1 of the dimension, the gradient G1 of the element in the key feature extraction region of the transmission line, and the gradient direction θ1 of the element in the key feature extraction region of the transmission line. Then, the direction algebra F of the gradient is determined by the gradient direction θ1 of the element in the key feature extraction region of the transmission line. Finally, the value of the gradient G1 of the element in the key feature extraction region of the transmission line is accumulated into the elements of the second dimension with index x1, the third dimension with index y1, and the fourth dimension with index F in the seed point matrix Z. After traversing all the element positions of the key feature extraction region of the transmission line, the final 1*4*4*8 seed point matrix is ​​the key feature of the transmission line.

[0036] The mapping process in the transmission line feature extraction algorithm involves transforming the 32*32 key point feature extraction region of the transmission line into a 4*4 seed point matrix Z. The transformation formula is as follows:

[0037]

[0038]

[0039] x1 = xx - xx%1

[0040] y1 = yy - yy%1

[0041] Where x1 and y1 are the indices of the second and third dimensions of the transformed seed point matrix, respectively. f y f The subscript for the key point feature extraction region of the transmission line is 0-31, where 31 represents the size of the key point feature extraction region, and D represents the main direction of the key point of the transmission line. 2 is an empirical value; xx and yy are the indices of the second and third dimensions of the seed point matrix after conversion from floating-point type. Here, yy or xx is a floating-point number, and taking the remainder after dividing by 1 will give the decimal part of yy or xx.

[0042] The feature extraction algorithm for transmission lines describes feature extraction processing performed on elements within the feature extraction region of each key point on the transmission line. The goal is to extract the gradient and gradient direction of each element, as shown in the specific extraction formula below:

[0043] x p =px+x f

[0044] y p =py+y f

[0045] d x =(M[x p +1, y p ]-M[x p -1, y p ]) 2

[0046] d y =(M[x p y p +1]-M[x p y p -1]) 2

[0047] d z =(M[x p y p ]-M[px,py]) 2

[0048]

[0049] θ1=tan -1 (d x ÷d y )

[0050] Where G1 is the gradient of an element in the feature extraction region of the key points of the transmission line, θ1 is the gradient direction of an element in the feature extraction region of the key points of the transmission line, px and py are the coordinates of the key points of the transmission line, and M is the key point map matrix of the transmission line; x p y p d represents the subscript of the key point features of the transmission line in the key point map matrix of the transmission line; x d y d z These are the gradients of key features of transmission lines in the x, y, and z directions, respectively.

[0051] The method for determining the direction algebra F of the gradient through the gradient direction θ1 of the element is as follows: divide the gradient direction θ1 of the element by 45, and then take the integer part to obtain the direction algebra F of the gradient.

[0052] The extracted key point features of the transmission line are a 1*4*4*8 matrix. After merging, the resulting image has a transmission line feature matrix of size mo*4*4*8, where mo is the number of key points of the transmission line.

[0053] The image segmentation-based method for extracting features of transmission line conductors provided by this invention has the following advantages: By using an image segmentation model and a local thresholding method, key points of the transmission line conductors in images captured by a visual monitoring device are obtained and extracted. Compared to traditional algorithms that combine image segmentation models and local thresholding methods, this method can extract not only points with drastic changes, such as corner points, but also points with relatively gentle changes. Furthermore, the primary function of determining the main direction of the key points is to reduce errors caused by rotation of the shooting perspective. Attached Figure Description

[0054] Figure 1 This is a flowchart of the transmission line conductor feature extraction method based on image segmentation provided by the present invention;

[0055] Figure 2 This invention selects an image taken by a power transmission line visualization monitoring device in a certain city;

[0056] Figure 3 This is a coarsely segmented image of the power transmission line captured by this invention;

[0057] Figure 4 This is a key point diagram of the power transmission line obtained by the present invention. Detailed Implementation

[0058] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention, but this does not constitute a limitation on the scope of protection of the present invention.

[0059] In this invention, for clarity, the following explanation is provided: A matrix refers to a combination of multi-dimensional elements. The elements in a matrix are obtained as "matrix [first dimension subscript, second dimension subscript, ..., nth dimension subscript]".

[0060] like Figure 1 As shown, the image segmentation-based method for extracting features of transmission line conductors provided in this embodiment includes the following steps performed in sequence:

[0061] 1) Use image segmentation models for, for example Figure 2 The image captured by the power transmission line visualization monitoring equipment shown is coarsely segmented to obtain, as... Figure 3 The image shown is a coarsely segmented image of the power transmission line;

[0062] The aforementioned power transmission line visualization monitoring equipment refers to existing monitoring equipment installed on power transmission line towers that can acquire visual images, including night vision cameras, visible light cameras, and video recording equipment.

[0063] The image captured by the power transmission line visualization monitoring device is a matrix of 1944*2592*3, where 1944 is the number of pixels horizontally, 2592 is the number of pixels vertically, and 3 represents the three channels of the image, which respectively represent the values ​​of the three colors red (R), green (G), and blue (B) in the pixel. The value range is 0-255, and each pixel has these three color values.

[0064] The image segmentation model refers to a pre-trained deep neural network model capable of identifying and segmenting power transmission lines. It employs deep neural network models including DeepLabV3+, CPS, and U2PL, and the training data for this model should include the category of power transmission lines. This embodiment uses a pre-trained DeepLabV3+ deep neural network model.

[0065] The coarse segmentation involves using images captured by the power transmission line visualization monitoring equipment as input to an image segmentation model. This model segments the image, assigning a category label to each pixel to indicate its class. If the pixel belongs to the power transmission line category, its value is retained; otherwise, a value of 0 is assigned. This yields the coarsely segmented image of the power transmission line. In this embodiment, the coarsely segmented image of the power transmission line is a matrix of size 1944*2592.

[0066] 2) The coarsely segmented transmission line image is further processed using the local thresholding method to obtain key points of the transmission line. Simultaneously, in the image captured by the transmission line visualization monitoring equipment, all pixel values ​​except for the key points of the transmission line are set to 0, resulting in the following: Figure 4 The diagram shows the key points of the power transmission line.

[0067] The method for further processing the coarsely segmented transmission line image using the local thresholding method to obtain key points of the transmission line is as follows: First, the second channel values ​​of the coarsely segmented transmission line image are sorted from smallest to largest to obtain sorted second channel values. Then, a local threshold is obtained using the sorted second channel values. The local threshold is obtained by subtracting the average of the last 10 sorted second channel values ​​from the average of the first 10 sorted second channel values, and then multiplying the difference by 0.78. Finally, pixels greater than the local threshold are retained as key points of the transmission line. Here, the key points of the transmission line are represented by their pixel coordinates in the image captured by the transmission line visualization monitoring equipment. The pixel coordinates of the key points of the transmission line can be represented as a 2*N two-dimensional matrix P. The first dimension of the two-dimensional matrix P stores the horizontal axis coordinates of the key points of the transmission line in the coarsely segmented transmission line image, and the second dimension stores the horizontal and vertical axis coordinates of the key points of the transmission line in the coarsely segmented transmission line image. In this embodiment, the pixel coordinates of the key points of the transmission line are as follows:

[0068] [[0, 0, 0, 0, 0, 01, 1, 1, 1, 1······, 811]

[0069] [1013, 1014, 1015, 1016, 1024, ..., 2368]]

[0070] A total of 29,835.

[0071] The key point map of the transmission line is a collection of the second channel values ​​of all key points of the transmission line, which can be represented as a matrix of 1944*2592*1. In this matrix, the element values ​​of non-key point coordinates of the transmission line are 0, and the other values ​​are the second channel values ​​of the key points of the transmission line.

[0072] 3) In the key point map of the transmission line, take the key point of the transmission line of a single transmission line as the center, obtain the expanded area of ​​the key point of the transmission line, then calculate the gradient histogram matrix within the expanded area of ​​the key point of the transmission line, and smooth the gradient histogram matrix. Then, use the key point main direction determination method to determine the main direction of the key point of the transmission line through the smoothed gradient histogram matrix.

[0073] The expanded area of ​​the key points of the transmission line is an image area that expands three grids upwards, downwards, leftwards, and rightwards from the key points of the transmission line of a single transmission line. If the expanded area of ​​the key points of the transmission line exceeds the key point image of the transmission line, the excess part is removed.

[0074] The calculation of the gradient histogram matrix within the expanded region of key points on the transmission line involves calculating the gradient histogram matrix for each key point. Each key point has a 1*36 gradient histogram matrix, representing the division of 360 degrees into 36 equal parts: 0-10 degrees, 10-20 degrees, ..., 350-360 degrees. Each element of the gradient histogram matrix represents the cumulative gradient value within each degree range. Then, the gradient and gradient direction of each pixel within the expanded region of the key points are selected and added to the corresponding element of the gradient histogram matrix. The formulas for calculating the gradient and gradient direction of each pixel within the expanded region of the key points are as follows:

[0075]

[0076] θ=tan -1 ((M[x+1,y]-M[x-1,y])÷(M[x,y+1]-M[x,y-1]))

[0077] Where G is the gradient of a pixel, θ is the gradient direction of a pixel, M is the key point map matrix of the transmission line, and x and y are the horizontal and vertical coordinates of each pixel in the key point map matrix M of the transmission line key point expansion region, respectively. Taking the key point of the transmission line with coordinates [0, 1013] as an example, the gradient histogram matrix of the expansion region of the key point of the transmission line is as follows:

[0078] [0.0, 0.0, 0.0, 9.707337878916224, 1.943991221090204, 0.0, 0.0, 0.0, 1.9286422752809753, 11.142466847318598, 0.0, 0.0, 0.0, 0.0, ...]

[0079] The gradient histogram smoothing process refers to traversing the gradient histogram and smoothing the data for each element encountered and its surrounding elements. The smoothing process is described in the following formula:

[0080] g0 = Gs[idx-1] ÷ 2

[0081] g1 = Gs[(idx+1)%36]÷2

[0082] g2=Gs[idx-2]÷2

[0083] g3=Gs[(idx+2)%36]÷2

[0084] Gs1[idx]=(6×Gs[idx]+g0+g1+g2+g3)÷8

[0085] Where Gs is the gradient histogram, idx is the element index of the gradient histogram Gs, and Gs1 is the smoothed gradient histogram. The smoothed gradient histogram Gs1 can be obtained by iterating through all elements in the gradient histogram Gs by changing the value of the element index idx. g0 represents the contribution of the left neighboring element to the smoothed gradient histogram element at index idx; g1 represents the contribution of the right neighboring element; g2 represents the contribution of the left adjacent element; and g3 represents the contribution of the right adjacent element. Additionally, if idx-1 and id-2 are less than 0, then Gs[idx-1] and Gs[idx-2] represent values ​​taken from the gradient histogram Gs in the reverse direction. For example, Gs[-1] represents the second-to-last value of the gradient histogram Gs, and Gs[-2] represents the second-to-last value. (idx+2)%36 ensures that the matrix index always falls between 0 and 35 when taking values. % represents the modulo operator.

[0086] Perform the above operation on each element of the gradient histogram matrix, and assign the result to the smoothed gradient histogram matrix at the corresponding index. After completing the traversal, the smoothed gradient histogram matrix is ​​obtained. Here, the smoothed gradient histogram matrix is ​​also a 1*36 matrix. Taking the key point of the transmission line with coordinates [0, 1013] as an example, the smoothed gradient histogram matrix Gs1 is obtained as follows:

[0087] [42.19897461, 11.8046875, 7.07299805, 17.53466797, 25.29150391, 30.88330078, 27.82861328, 12.35571289, 14.38598633, 34.57495117, 3.55908203, 2.78369141, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3.71801758, 4.30078125]

[0088] The method for determining the principal direction of key points on the transmission line is to take the index of the maximum value of the smoothed gradient histogram matrix Gs1 as Mi = 0, and then process the maximum value in the smoothed gradient histogram matrix Gs1 to obtain the principal direction of the key points on the transmission line. The specific method is shown in the following formula:

[0089] Ls = Gs1[(Mi-1)%36]

[0090] Lm = Gs1[(Mi+1)%36]

[0091] Ll=Gs1[Mi]+0.5×(Lm-Ls)÷(Lm-2*Gs1[Mi]+Ls)%36

[0092] D = 360 - (Ll) × 360 ÷ 36

[0093] Where D represents the principal direction of the key point of the transmission line, Gs1 is the smoothed gradient histogram, Mi is the index of the maximum value of the smoothed gradient histogram Gs1, and % represents the modulo operator. Ls is the left gradient direction correction value of the principal direction of the key point of the transmission line, Lm is the right gradient direction correction value of the principal direction of the key point of the transmission line, and LI is the corrected gradient direction. (Mi-1)%36 and (Mi+1)%36 are used to ensure that the index can be between 0 and 35 when taking values. By processing the smoothed gradient histogram Gs1 of the first key point of the transmission line, the principal direction D of the key point of the transmission line is obtained as 271.06 degrees.

[0094] 4) In the key point map of the transmission line, take the key point of the transmission line of a single transmission line as the center, obtain the key point feature extraction area of ​​the transmission line. Then, with the help of the main direction of the key point of the transmission line, use the transmission line feature extraction algorithm to extract each 1*4*4*8 key point feature from the key point feature extraction area of ​​the transmission line. Merge all the extracted key point features of the transmission line to finally obtain the transmission line features of the image.

[0095] The key point feature extraction region for transmission lines is an image area centered on the key point of a single transmission line, expanding upwards, downwards, leftwards, and rightwards by 32 grids. If the range of the key point feature extraction region exceeds the key point image, the excess portion is removed. Taking a key point of a transmission line with coordinates [0, 1013] as an example, the obtained key point feature extraction region size is 32*32. Its values ​​are as follows:

[0096] [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 244, 229, 140, 99, 131, 132, 232, 244, 201, 206, 253, 241, 149, 102, 137, 129, 212, 241],

[0097] ······

[0098] [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 201, 217, 244, 211, 204, 130, 110, 166, 232, 229, 211, 216, 234, 229, 221, 131]]

[0099] The aforementioned power transmission line feature extraction algorithm is a method for extracting features from key points of a single power transmission line. It takes the key point of the power transmission line as input and the features of that key point as output. The output consists of 4*4 seed points, each with 8 directions. This represents dividing a circle into 8 parts, each with a 45° angle. The 8 directions are represented sequentially as 0-45°, 45-90°, ..., 315-360°, ultimately represented as a 1*4*4*8 seed point matrix Z. The specific operation is as follows: First, each element of the key point feature extraction region of the power transmission line is mapped and its features are extracted. The process yields four variables: the index x1 of the second dimension of the seed point matrix, the index y1 of the third dimension of the seed point matrix, the gradient G1 of the elements in the key point feature extraction region of the transmission line, and the gradient direction θ1 of the elements in the key point feature extraction region of the transmission line. Then, the directional algebra F of the gradient is determined using the gradient direction θ1 of the elements in the key point feature extraction region of the transmission line. Finally, the values ​​of the gradient G1 of the elements in the key point feature extraction region of the transmission line are accumulated and added to the elements of the seed point matrix Z with indices x1 in the second dimension, y1 in the third dimension, and F in the fourth dimension. After traversing all element positions in the key point feature extraction region of the transmission line, the final 1*4*4*8 seed point matrix represents the key point features of the transmission line.

[0100] The mapping process in the transmission line feature extraction algorithm involves transforming the 32*32 key point feature extraction region of the transmission line into a 4*4 seed point matrix Z. The transformation formula is as follows:

[0101]

[0102]

[0103] x1 = xx - xx%1

[0104] y1 = yy - yy%1

[0105] Where x1 and y1 are the indices of the second and third dimensions of the transformed seed point matrix, respectively. f y f The subscript for the key point feature extraction region of the transmission line is 0-31, where 31 represents the size of the key point feature extraction region, and D represents the main direction of the key point of the transmission line. 2 is an empirical value; xx and yy are the indices of the second and third dimensions of the seed point matrix after conversion from floating-point type. Here, yy or xx is a floating-point number, and taking the remainder after dividing by 1 will give the decimal part of yy or xx.

[0106] The feature extraction algorithm for power transmission lines describes feature extraction processing performed on elements within the feature extraction region of each key point on the power transmission line. The goal is to extract the gradient and gradient direction of each element. The specific extraction formula is as follows:

[0107] x p =px+x f

[0108] y p =py+y f

[0109] d x =(M[x p +1, y p ]-M[x p -1, y p ]) 2

[0110] d y =(M[x p y p +1]-M[x p y p -1]) 2

[0111] d z =(M[x p y p ]-M[px,py]) 2

[0112]

[0113] θ1=tan -1 (d x ÷d y )

[0114] Where G1 is the gradient of an element in the feature extraction region of the key points of the transmission line, θ1 is the gradient direction of an element in the feature extraction region of the key points of the transmission line, px and py are the coordinates of the key points of the transmission line, and M is the key point map matrix of the transmission line; x p y p d represents the subscript of the key point features of the transmission line in the key point map matrix of the transmission line. x d y d z These are the gradients of key features of transmission lines in the x, y, and z directions.

[0115] The method for determining the gradient direction algebra F by the gradient direction θ1 of an element is as follows: divide the gradient direction θ1 of the element by 45, and then take the integer value to obtain the gradient direction algebra F. The direction algebra F here is only for easy quantization and writing into the seed point matrix Z. The essence is to determine the range of directions in which the gradient direction θ1 of the element is located, and then add the gradient value to that direction.

[0116] Taking the key point feature extraction region of a power transmission line with coordinates [0, 1013] as an example, the above operation is performed on each element of the key point feature extraction region set of the key point with coordinates [0, 1013] to calculate the key point feature of each key point. The final result is a 1*4*4*8 conductor feature matrix, as shown below:

[0117] [[[0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0]],[[0,0,0,0,0,0,0,0],[9231.64973779,0,0,0,0,0,0],[0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0], [30650.00400483, 17336.82430725, 0, 0, 0, 0, 0, 0], [17922.1006282, 24586.74653197, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0], [[0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0]]]

[0118] Finally, after merging the key features of all transmission lines, the resulting image has a matrix of conductor features of size 29835*4*4*8.

[0119] The resulting duct features can be used for feature matching of different images and camera parameter estimation in 3D reconstruction.

[0120] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for extracting features of transmission line conductors based on image segmentation, characterized in that: The image segmentation-based method for extracting features of transmission line conductors includes the following steps performed in sequence: 1) Use an image segmentation model to coarsely segment the conductor portion in the image captured by the power transmission line visualization monitoring equipment to obtain the coarsely segmented power transmission conductor image; 2) The image of the transmission line after coarse segmentation is further processed using the local thresholding method to obtain the key points of the transmission line. At the same time, in the image captured by the transmission line visualization monitoring equipment, all pixel values ​​except the key points of the transmission line are set to 0 to obtain the key point map of the transmission line. 3) In the key point map of the transmission line, take the key point of the transmission line of a single transmission line as the center, obtain the expanded area of ​​the key point of the transmission line, then calculate the gradient histogram matrix within the expanded area of ​​the key point of the transmission line, and smooth the gradient histogram matrix. Then, use the key point main direction determination method to determine the main direction of the key point of the transmission line through the smoothed gradient histogram matrix. 4) In the key point map of the transmission line, the key point of the transmission line is taken as the center to obtain the key point feature extraction area of ​​the transmission line. Then, with the help of the main direction of the key point of the transmission line, the transmission line feature extraction algorithm is used to extract each 1*4*4*8 key point feature from the key point feature extraction area of ​​the transmission line. All the extracted key point features of the transmission line are merged to finally obtain the transmission line features of the image. In step 2), the method for further processing the coarsely segmented transmission line image using the local thresholding method to obtain key points of the transmission line is as follows: First, the second channel values ​​of the coarsely segmented transmission line image are sorted from smallest to largest to obtain sorted second channel values. Then, the sorted second channel values ​​are used to obtain a local threshold. The local threshold is obtained by taking the average of the last 10 sorted second channel values ​​and subtracting the average of the first 10 sorted second channel values ​​from the average of the first 10 sorted second channel values, and then multiplying the difference by 0.

78. Finally, pixels with values ​​greater than the local threshold are retained as key points of the transmission line. The key point map of the transmission line is a set of second channel values ​​of all key points of the transmission line, represented as a matrix of 1944*2592*1. In this matrix, the element values ​​of non-key point coordinates of the transmission line are 0, and the other values ​​are the second channel values ​​of the key points of the transmission line.

2. The method for extracting features of transmission line conductors based on image segmentation according to claim 1, characterized in that: In step 1), the transmission line visualization monitoring equipment refers to the monitoring equipment installed on existing transmission line towers that can acquire visual images, including night vision cameras, visible light cameras and video recording equipment; The image captured by the power transmission line visualization monitoring device is a matrix of 1944*2592*3, where 1944 is the number of horizontal pixels, 2592 is the number of vertical pixels, and 3 represents the three channels of the image, which respectively represent the values ​​of the three colors red (R), green (G), and blue (B) in the pixel. The value range is 0-255, and each pixel has these three color values. The image segmentation model uses deep neural network models including DeeplabV3+, CPS and U2PL, and the data used to train the model includes the category of power transmission lines. The coarse segmentation involves using images captured by the power transmission line visualization monitoring equipment as input to an image segmentation model. The model segments the image by assigning a category label to each pixel in the input image to indicate its category. If the pixel belongs to the power transmission line category, its value is retained, while the values ​​of pixels belonging to other categories are assigned to 0. This process yields the coarsely segmented image of the power transmission line.

3. The method for extracting features of transmission line conductors based on image segmentation according to claim 1, characterized in that: In step 3), the expansion area of ​​the key points of the transmission line is an image area that expands three grids upwards, downwards, leftwards, and rightwards, centered on the key points of the transmission line of a single transmission line. If the expansion area of ​​the key points of the transmission line exceeds the key point image of the transmission line, the excess part is removed. The calculation of the gradient histogram matrix within the expanded region of key points on the transmission line involves calculating the gradient histogram matrix for each key point. Each key point has a 1*36 gradient histogram matrix, representing the division of 360 degrees into 36 equal parts: 0-10 degrees, 10-20 degrees, ..., 350-360 degrees. Each element of the gradient histogram matrix represents the cumulative gradient value within each degree range. Then, the gradient and gradient direction of each pixel within the expanded region of the key points are selected and added to the corresponding element of the gradient histogram matrix. The formulas for calculating the gradient and gradient direction of each pixel within the expanded region of the key points are as follows: θ=tan -1 ((M[x+1,y]-M[x-1,y])÷(M[x,y+1]-M[x,y-1])) Where G is the gradient value of a pixel, θ is the gradient direction of a pixel, M is the key point map matrix of the transmission line, and x and y are the horizontal and vertical coordinates of each pixel in the key point map matrix M of the transmission line in the extended region of the key points of the transmission line, respectively. The gradient histogram smoothing process refers to traversing the gradient histogram, that is, multiplying each element of the gradient histogram by 6, then adding the sum of the previous element and the next element divided by 36, and adding the sum of the previous two elements and the next two elements divided by 36. The result is divided by 16 to obtain the smoothed result of that element. The result is then assigned to the gradient histogram. After completing the traversal, the smoothed gradient histogram can be obtained. The method for determining the main direction of key points of the transmission line is to take the index of the maximum value of the smoothed gradient histogram matrix S as i. The specific determination method uses the following formula: Ls = S[i-1] ÷ 36 Lm = S[i-1] ÷ 36 Ll=S[i]+0.5×(Ls-Lm)÷(Ls-2*S[i]+Lm)% 36 D = 360 - (Ll) × 360 ÷ 36 Where D is the main direction of the key point of the transmission line, S is the smoothed gradient histogram matrix, i is the maximum value of the smoothed gradient histogram matrix S, and % is the remainder sign.

4. The method for extracting features of transmission line conductors based on image segmentation according to claim 1, characterized in that: In step 4), the key point feature extraction region of the transmission line is an image region that expands n grids upward, downward, left, and right, centered on the key point of the transmission line of a single transmission line. Here, n is an empirical value, ranging from 3 to 256. If the range of the key point feature extraction region of the transmission line exceeds the key point image of the transmission line, the excess part is removed. The aforementioned power transmission line feature extraction algorithm is a method for extracting features from key points of a single power transmission line. It takes the key point of the power transmission line as input and the features of that key point as output. The output consists of 4*4 seed points, each with 8 directions. This represents dividing a circle into 8 parts, each 45°, with the 8 directions sequentially represented as 0-45°, 45-90°, ..., 315-360°, ultimately represented as a 1*4*4*8 seed point matrix Z. The specific operation is as follows: First, each element of the key point feature extraction region of the power transmission line is mapped and feature extracted, resulting in the index x1 of the second dimension of the seed point matrix, and the index x1 of the third dimension of the seed point matrix. The key features of the transmission line are determined by four variables: the index y1 of the dimension, the gradient G1 of the element in the key feature extraction region of the transmission line, and the gradient direction θ1 of the element in the key feature extraction region of the transmission line. Then, the direction algebra F of the gradient is determined by the gradient direction θ1 of the element in the key feature extraction region of the transmission line. Finally, the value of the gradient G1 of the element in the key feature extraction region of the transmission line is accumulated into the elements of the seed point matrix Z with the second dimension index x1, the third dimension index y1, and the fourth dimension index F. After traversing all the element positions of the key feature extraction region of the transmission line, the final 1*4*4*8 seed point matrix is ​​the key feature of the transmission line. The mapping process in the transmission line feature extraction algorithm involves transforming the 32*32 key point feature extraction region of the transmission line into a 4*4 seed point matrix Z. The transformation formula is as follows: x1 = xx - xx%1 y1 = yy - yy%1 Where x1 and y1 are the indices of the second and third dimensions of the transformed seed point matrix, respectively. f ,y f The subscript for the key point feature extraction region of the transmission line is 0-31, representing the size of the key point feature extraction region. Here, D represents the principal direction of the key point of the transmission line. And 2 are empirical values; The feature extraction algorithm for transmission lines describes feature extraction processing performed on elements within the feature extraction region of each key point on the transmission line. The goal is to extract the gradient and gradient direction of each element, as shown in the specific extraction formula below: x p =px+x f and p =py+y f d x =(M[x p +1,y p ]-M[x p -1,y p ]) 2 d y =(M[x p ,y p +1]-M[x p ,y p -1]) 2 d z =(M[x p ,y p ]-M[px,py]) 2 θ1=tan -1 (d x ÷d y ) Where G1 is the gradient of the element in the key point feature extraction region of the transmission line, θ1 is the gradient direction of the element in the key point feature extraction region of the transmission line, px,py are the coordinates of the key point of the transmission line, and M is the key point map matrix of the transmission line. The method for determining the direction algebra F of the gradient through the gradient direction θ1 of the element is as follows: divide the gradient direction θ1 of the element by 45, and then take the integer part to obtain the direction algebra F of the gradient. The extracted key point features of the transmission line are a matrix of size 1*4*4*8. After merging, the conductor features of the resulting image are a matrix of size 29835*4*4*8.

Images