An overhead power transmission line joint tube identification and positioning method

Abstract

The application discloses an overhead power transmission line joint pipe identification and positioning method, which comprises collecting joint pipe images, expanding the number of target samples, labeling the joint pipe and making a joint pipe rotation target data set; a joint pipe rotation target detection model is constructed, a model rotation boundary box loss function is improved, a feature extraction and fusion network is optimized, an anchor frame parameter is adjusted in combination with a K-means clustering algorithm; a rotation target algorithm identifies an RGB image, outputs joint pipe target confidence and a rotation frame information, and maps the information to a joint pipe depth map; a joint pipe center point pixel coordinate depth value is extracted, a joint pipe physical width and height are estimated according to a pixel coordinate and a world coordinate corresponding relationship, and a center line on a depth point through a joint pipe center point and parallel to a long side is linearly fitted, and a joint pipe is reconstructed in a three-dimensional depth space. The application can provide a terminal joint pipe autonomous identification and accurate positioning reference scheme for a multi-rotor unmanned aerial vehicle carrying a digital X-ray imaging device to implement X-ray photographing detection on the joint pipe.

Description

Technical Field

[0001] This invention relates to an auxiliary invention method for in-service non-destructive testing of overhead power transmission line splice pipes using UAV X-rays, and particularly to a method for identifying and locating overhead power transmission line splice pipes that integrates a rotating target detection algorithm and RGB-D information. Background Technology

[0002] Splicing conduits are crucial electrical crimping fittings in overhead transmission lines, serving both to withstand conductor tension and to conduct current. They are typically connected to transmission lines using a hydraulic crimping process, and once crimped, they cannot be disassembled. Numerous reports have surfaced of transmission line structural detachment caused by problems with the crimping quality of splicing conduits. Therefore, to ensure the stable operation of transmission lines, effective in-service non-destructive testing of splicing conduits has become paramount.

[0003] X-ray digital imaging technology is currently widely used in the power industry for quality inspection and fault assessment. Currently, X-ray inspection of power crimp fittings requires manual labor to move the X-ray imaging plate and X-ray machine to a high position and fix it at the splice location. This process is time-consuming, labor-intensive, and carries the risks of working at height and radiation hazards, making it unsuitable for current power equipment inspection needs. To overcome these risks, State Grid XX Power, XX Power, Southern Power Grid XX Technology, and XX University, among others, have developed drone-borne X-ray inspection equipment for in-service inspection of splice fittings, addressing the shortcomings of manual X-ray inspection. However, currently, professional drone pilots are still required to precisely suspend the imaging equipment at the splice location, requiring the pilot to maintain high concentration during operation; improper operation can easily lead to accidents.

[0004] With the rapid development of deep learning, target detection algorithms based on horizontal anchor boxes have been applied to the precise landing of UAVs using visual navigation. However, they are not effective for identifying splice tubes with large aspect ratios and arbitrary orientations. Instance segmentation methods, on the other hand, have high annotation costs and cannot be quickly applied to practical splice tube identification projects. This invention proposes the SS-R3Det model, which can solve the problem of identifying splice tubes with large aspect ratios and arbitrary orientations. This invention combines the recognition results with binocular RGB-D depth information, simultaneously achieving splice tube depth, width, and height measurement, as well as three-dimensional spatial reconstruction of the splice tube centerline. This provides technical support and reference for achieving autonomous identification and precise positioning of splice tubes at the end of UAVs carrying X-ray equipment.

[0005] The specific patent prior art documents mentioned above are as follows:

[0006] 1) "An Automatic Reading Method for Pointer Instruments Based on Rotating Target Detection", Patent No. CN202210807428.1. This invention relates to the field of visual measurement technology. It constructs an improved rotating target detection network based on R3Det to obtain the pointer detection box, which features high accuracy. After obtaining the pointer skeleton and dial scale, the reading is calculated using the angle method to achieve automatic reading. The automatic reading method for pointer instruments based on rotating target detection proposed in this invention can accurately locate the pointer area and perform directional erosion to remove interference and fit the straight line where the pointer is located within this area, greatly improving the accuracy of pointer extraction, especially showing good detection effect for narrow pointers.

[0007] 2) "A Target Detection Method Considering Rotating Targets in Remote Sensing Images," Patent No. CN202110684873.9. This invention provides a rotating target detector for rotating targets in remote sensing images, including a feature extraction stage and a detection box refinement stage. In the algorithm's detection, the first-stage network uses horizontal regression boxes to obtain faster speed and more region proposal boxes. In the regression box refinement stage, rotated anchor boxes are used to adapt to dense scenes. This invention solves the problem of low detection accuracy caused by the dense arrangement of ship targets in remote sensing image ship detection. This invention can obtain detection results with aspect ratios and dimensions closer to the actual shape of the ship, and is suitable for military and civilian remote sensing image ship detection operations.

[0008] 3) "A rotational correction imaging method, device, and medium for environmental testing", patent number CN202310645194.X. This invention predicts the object category using a single-stage rotating target detection network S2A-Net and outlines the object's position using a rotating frame; it uses a binocular depth camera to acquire the object's depth map and calculate its three-dimensional coordinates; and it obtains the sample's attitude information relative to the coordinate system of the robotic arm base by rotating the rectangular frame.

[0009] 4) "An Image Processing Method Based on Machine Vision", Patent No. CN202310066520.1. This invention discloses an image processing method based on machine vision, comprising: acquiring an RGB image and a binocular depth image of a shooting scene to form an RGBD image; using a deep learning method to obtain a monocular depth image of the shooting scene from the RGBD image; identifying outliers in the binocular depth image; and modifying the depth values ​​of each outlier in the binocular depth image based on the monocular depth image to obtain the final depth image. This embodiment quickly achieves the fusion of the monocular and binocular depth images, improving the quality of the depth image. Summary of the Invention

[0010] To address the aforementioned technical problems, the purpose of this invention is to provide a method for identifying and locating overhead power line splice pipes that integrates a rotating target detection algorithm and RGB-D information.

[0011] The objective of this invention is achieved through the following technical solution:

[0012] A method for identifying and locating overhead power transmission line splice conduits includes:

[0013] Step A: Collect images of the splice tube, increase the number of target samples, label the splice tube, and create a splice tube rotation target dataset;

[0014] Step B involves constructing a target detection model for rotating splice tubes, improving the model's rotating bounding box loss function, optimizing the feature extraction and fusion network, and combining it with the K-means clustering algorithm to adjust the anchor box parameters.

[0015] Step C, the rotating target algorithm, is used to identify RGB images, output the target confidence score and rotation frame information of the splice tube, and map the information to the splice tube depth map;

[0016] Step D: Extract the pixel coordinate depth value of the center point of the splice tube, and estimate the physical width and height of the splice tube based on the correspondence between pixel coordinates and world coordinates;

[0017] Step E involves linearly fitting the depth point along the center line parallel to the long side of the splice tube, and reconstructing the splice tube in three-dimensional depth space.

[0018] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0019] This invention addresses the identification and positioning of splice tubes with large aspect ratios and arbitrary orientations. It constructs an SS-R3Det identification model for splice tubes, achieving high-precision identification of rotating target splice tubes. By integrating binocular depth information and designing a post-processing algorithm, it completes the measurement of splice tube depth, width, and height, as well as the reconstruction of the splice tube centerline in three-dimensional space, thus achieving precise positioning of the splice tube. This invention can provide technical support and reference for UAVs carrying X-ray equipment to achieve autonomous identification and precise positioning of end-effector splice tubes. Attached Figure Description

[0020] Figure 1 This is a flowchart of the splice tube identification and positioning process;

[0021] Figure 2 This is the basic framework for connector identification and positioning;

[0022] Figure 3 This is a schematic diagram of the SS-R3Det feature extraction module;

[0023] Figure 4a and 4b Here is an example of the target identification result and its depth map for the rotating splice tube;

[0024] Figure 5a , 5b Figures 5c, 5d, and 5e are example diagrams of three-dimensional spatial straight line fitting of the centerline of the splicing pipe;

[0025] Figure 6 This is a picture of the actual connector;

[0026] Figure 7 This is an X-ray image of the splice tube obtained from a drone. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in further detail below with reference to the embodiments and accompanying drawings.

[0028] like Figure 1 , Figure 2 The diagram shows the flowchart of an overhead power transmission line splice identification and localization method that integrates a rotating target detection algorithm and RGB-D information, including:

[0029] Step S100: Collect images of the splice tube, expand the number of target samples, and label the splice tubes to create a splice tube rotation target dataset.

[0030] Step S110: Construct a target detection model for rotating splice tubes, improve the model's rotating bounding box loss function, optimize the feature extraction and fusion network, combine K-means clustering algorithm, and adjust anchor box parameters.

[0031] Step S120: The rotating target algorithm is used to identify RGB images, output the target confidence of the splice tube and the rotating frame information, and map the information to the splice tube depth map;

[0032] Step S130: Extract the pixel coordinate depth value of the center point of the splice tube, and estimate the physical width and height of the splice tube based on the correspondence between pixel coordinates and world coordinates.

[0033] Step S140 involves linear fitting of the depth point along the center line parallel to the long side of the splice tube, and reconstructing the splice tube in three-dimensional depth space.

[0034] In step S100 above:

[0035] Images of the splice tubes captured under different conditions were collected. New samples were generated by expanding the target sample count using the Augmentor image enhancement technique. Specific enhancement techniques included color enhancement / reduction, contrast enhancement / reduction, perspective distortion (vertical and oblique directions), elastic distortion, and scaling. The splice tubes were labeled with rotating bounding boxes using roLabelImg, and the labeling results (x, y, w, h, θ) were converted into DOTA format (x1, y1, x2, y2, x3, y3, x4, y4) representing the coordinates of the four vertices of the rotating bounding box, thus creating a rotating target dataset for the splice tubes.

[0036] The above step S110 specifically includes:

[0037] 1) The bounding box loss function is KLD, which can dynamically adjust the training weights to adapt to the target scale of the large aspect ratio of the connecting tube.

[0038] First, convert the labeled rotating rectangle (x, y, w, h, θ) of the splice tube into a two-dimensional Gaussian distribution (μ, Σ):

[0039] μ=(x,y) T (1)

[0040]

[0041] In the above formula, R is the rotation matrix, Λ is the covariance matrix, whose eigenvalues ​​are half the length and width of the rotated rectangle, μ is the mean vector, which is determined by the center coordinates of the rotation matrix, and ∑ is the covariance matrix after rotation.

[0042] The distance between different Gaussian distributions is calculated using KLD, and its specific expression is shown in equation (3).

[0043]

[0044] N P N t Let x represent the distribution of the predicted bounding box and the ground truth bounding box, respectively. The first term on the right-hand side of the equation is related to the center coordinate x of the predicted bounding box. p ,y p The second item relates to the length, width, and rotation angle w of the prediction box. p ,h p ,θ p The details are as follows:

[0045]

[0046]

[0047]

[0048] Here Δx = x p -xt ,Δy=y p -y t ,Δθ=θ p -θ t .

[0049] The bounding box loss function is now:

[0050]

[0051] 2) Optimize the feature extraction and fusion network of the R3Det model through the feature fusion ASFF module.

[0052] In the SS-R3Det feature extraction module, such as Figure 3 As shown, for the feature map output by each layer of FPN, the features from other layers are resized and incorporated into the ASFF layer feature map through upsampling and subsampling, and then weighted and summed bit by bit:

[0053]

[0054] in This represents the ASFF output feature map y. l At position (i,j), x l Represents the features of the l-th layer. This represents the feature map resized from layer s to layer t, where α, β, and γ are learnable parameters representing the importance of each feature map layer. This represents the values ​​of α, β, γ at position (i, j). These can be shared scalars across channels or a single value for each channel. satisfy

[0055] 3) Combine the K-means clustering algorithm to adjust the anchor frame parameters.

[0056] This invention uses the K-means clustering method to cluster the width and height of the original data, and optimizes the anchor frame size parameter settings based on the clustering results.

[0057] K objects are randomly selected as initial cluster centers. Then, the Euclidean distance between each object and each seed cluster center is calculated. Each object is then assigned to the nearest cluster center until the cluster centers no longer change. At this point, the sum of squared errors is locally minimized, as shown in equation (9).

[0058]

[0059] Here, p is the sample, M is the sample dataset, and O is the O(n) value. j It is the j-th cluster center, and E is the sum of squared errors.

[0060] The above step S120 specifically includes:

[0061] The rotating target algorithm is used to identify RGB images captured by a stereo depth camera. The stereo depth camera outputs an RGB image and a depth map. The RGB image is input into the rotating target detection model. When the result is greater than the confidence threshold, the detection result information (cx, cy, w, h, θ) is output; otherwise, a new RGB image is detected, and the result is mapped to the depth map to extract the depth value at the corresponding location (e.g., depth map). Figure 4a and 4b ).

[0062] The above step S130 specifically includes:

[0063] 1) Extract the depth value of the pixel coordinates of the center point of the splice tube. The specific steps are as follows:

[0064] Compare the sizes of w and h, taking the direction of the larger one as the long side, and set the search factor search_factor. The search threshold δ = long_side * search_factor, where...

[0065] long_side=max(w,h) (10)

[0066] Randomly sample randnum values ​​within the range (-δ, δ) as the sampling offset sample_offset. Randomly sample randnum RGB image pixels passing through the center point and parallel to the long side. The pixel coordinates (sample_x, sample_y) are determined by the following two formulas.

[0067] sample_x=cx+sample_offset*math.cosθ (11)

[0068] sample_y=cy+sample_offset*math.sinθ (12)

[0069] The corresponding depth value can be calculated by combining the pixel coordinates (sample_x, sample_y) with the depth map depth_data.

[0070] sample_depth=depth_data[int(sample_y), int(sample_x)] (13)

[0071] Assuming there are num_depth_zero missing points among randnum depth values, the average depth value is...

[0072]

[0073] At this time d oThis represents the pixel coordinate depth value of the splice tube center point, i.e., the distance between the splice tube center point and the binocular camera center point.

[0074] 2) Based on the correspondence between pixel coordinates and world coordinates, estimate the physical width and height w_ps and h_ps of the connector. The specific steps are as follows:

[0075] The target result (x, y, w, h, θ) is converted into the pixel coordinates of the four vertices of the rotating rectangle image of the connecting tube (x1, y1, x2, y2, x3, y3, x4, y4), and then the pixel coordinates are converted into world coordinates (X). W1 ,Y W1 ,X W2 ,Y W2 ,X W3 ,Y W3 ,X W4 ,Y W4 Based on the distance between two adjacent vertices in the world coordinates of the rotated rectangle, w_ps and h_ps can be calculated.

[0076]

[0077]

[0078] In (15) and (16), the world coordinates (X) W ,Y W The pixel coordinates (u,v) can be obtained by conversion from equation (17).

[0079]

[0080] In equation (17), the first matrix on the right-hand side is the camera extrinsic parameter matrix, where R and T are the rotation and translation amounts, respectively. The second matrix is ​​the camera intrinsic parameter matrix (camera inherent properties), where f is the focal length, dx and dy are the pixel sizes, u0 and v0 are the positions of the camera coordinate center in the pixel coordinate system, and Z... C The distance from the object point to the camera optical center in the camera coordinate system corresponding to pixel (u,v) is given.

[0081] The above step S140 specifically includes:

[0082] Let the search range search_factor = 0.5 and the search threshold δ = long_side * search_factor. Extract all values ​​in the range (-δ, δ) as the offset value sample_offset. Substitute them into equations (11), (12), and (13) to extract the depth value data corresponding to the center line of the splice pipe, and remove the missing points with a depth value of zero to form the point cloud depth data of the center line of the splice pipe.

[0083] Propose the three-dimensional spatial linear equation of the point cloud depth data of the splice centerline:

[0084]

[0085] In the formula, x and y are pixel coordinates, d is the corresponding depth value, and a, b, and c are undetermined coefficients of a line in three-dimensional space. After transforming the equation, we get:

[0086]

[0087]

[0088] in

[0089] A straight line in space can be considered as the intersection of two planes. Fitting a straight line in space can be considered as fitting the equations of two planes. According to the least squares fitting principle, we first calculate the sum of squares of the residuals between the predicted and measured values ​​in equations (19) and (20):

[0090] N1=∑(x i -k1d i -m1) 2 (twenty two)

[0091] N2=∑(y i -k2d i -m2) 2 (twenty three)

[0092] To minimize equations (22) and (23), their partial derivatives with respect to the four coefficients k1,m1,k2,m2 must be zero.

[0093]

[0094]

[0095]

[0096]

[0097] Combining equations (24), (25), (26), and (27), we can obtain:

[0098]

[0099] Combining equation (21), and setting x0, y0, d0 as the RGB pixel coordinates and depth values ​​corresponding to the center point of the connecting tube, x i ,y i ,d i From the point cloud depth data of the centerline of the splice pipe extracted above, the fitted three-dimensional straight line of the centerline of the splice pipe can be obtained (e.g., Figure 5a , 5b 5c, 5d and 5e).

[0100] While the embodiments disclosed in this invention are as described above, the content is merely for the purpose of facilitating understanding of the invention and is not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and variations in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the scope of patent protection for this invention shall still be determined by the scope defined in the appended claims.

Claims

1. A method for identifying and locating overhead transmission line splice conduits, characterized in that, The method includes the following steps: Step A: Collect images of the splice tube, increase the number of target samples, label the splice tubes, and create a splice tube rotation target dataset; Step B involves constructing a target detection model for rotating splice tubes, improving the model's rotating bounding box loss function, optimizing the feature extraction and fusion network, and adjusting the anchor box parameters by combining the K-means clustering algorithm. Step C, the rotating target algorithm, is used to identify RGB images, output the target confidence of the splice tube and the rotating frame information, and map the information to the splice tube depth map; Step D: Extract the pixel coordinate depth value of the center point of the splice tube, and estimate the physical width and height of the splice tube based on the correspondence between pixel coordinates and world coordinates; Step E involves linear fitting of the depth point along the center line parallel to the long side of the splice tube, and reconstructing the splice tube in three-dimensional depth space. In step A, the splicing pipe is labeled using a rotating rectangular frame (roLabelImg), and the labeling results are then... Convert to DOTA format represented by the coordinates of the four vertices of the rotating frame. Create a dataset of rotating targets for the splicing tube; In step B, the SS-R3Det model for detecting rotating targets in splicing pipes uses R3Det as the basic model and KLD as the bounding box loss function. The feature extraction and fusion network of the R3Det model is optimized through the feature fusion ASFF module. Based on the K-means clustering algorithm, the width and height distribution of splicing pipe samples is analyzed, and the anchor frame size parameter is adjusted to improve the model's high-precision detection of rotating targets in splicing pipes with large aspect ratios. In step C, RGB images and depth maps are captured by a binocular depth camera, and the RGB images are input into the rotating target detection model of the connector tube. In step D, the physical width of the connector is estimated based on the correspondence between pixel coordinates and world coordinates. and high Specifically, it includes: target result Convert to pixel coordinates of the four vertices of the rotating rectangle image of the splice tube Then convert the pixel coordinates to world coordinates. The distance between two adjacent vertices of the rotated rectangle in world coordinates is calculated. and : (6) （7） In (6) and (7), world coordinates with pixel coordinates Obtained by transformation from equation (8), （8） In equation (8), the first matrix on the right-hand side is the camera extrinsic parameter matrix. These are the rotation and translation amounts, respectively; the second matrix is ​​the camera intrinsic parameter matrix. Focal length For pixel size, , This represents the position of the camera's center in the pixel coordinate system. For pixels The distance from the object point to the camera optical center in the corresponding camera coordinate system.

2. The method for identifying and locating overhead transmission line splice pipes as described in claim 1, characterized in that, In step C, the detection result information is output when the confidence level of the splice tube target is greater than the confidence threshold. When the confidence level of the target of the connecting tube is less than the confidence threshold, a new RGB image is re-detected, and the output detection result information is mapped to the depth map to extract the depth value at the corresponding position.

3. The method for identifying and locating overhead transmission line splice pipes as described in claim 1, characterized in that, The extraction of the depth value of the pixel coordinates of the center point of the splice tube in step D includes: Compare and Size, with the larger of the values ​​as the longer side, and a search factor is set. Search threshold ,in （1） In scope Random sampling The value is used as the sampling offset. Random sampling passing through the center point and parallel to the long side Each RGB image pixel, pixel coordinates Determined by the following two formulas, （2） （3） Based on pixel coordinates Combined with depth map Calculate the corresponding depth value: （4） Assumption There are missing points with a depth value of zero. The average depth value that is not zero at this point: （5） at this time This represents the pixel coordinate depth value of the splice tube center point, i.e., the distance between the splice tube center point and the binocular camera center point.

Images