Method and system for detecting icing thickness of power transmission line

Abstract

The invention relates to the field of vision measurement, in particular to a power transmission line icing thickness detection method and system, and the method comprises the steps: obtaining a multi-position image through the movement of an unmanned aerial vehicle, constructing a cost feature fusing the multi-view matching cost for each parallax of each pixel point, evaluating the multi-view texture prominence of the pixel point, screening a reference pixel point, and carrying out the detection of the icing thickness of a power transmission line. And finally, guiding an aggregation path of an SGM algorithm by using distribution of reference pixel points, and completing multi-view cost fusion and thickness detection. According to the method, the problems of weak texture and visual angle sensitivity caused by special optical characteristics of the icing surface are effectively solved, the accuracy, robustness and practicability of icing thickness detection of the power transmission line are remarkably improved, and reliable technical guarantee is provided for safe operation of a power grid.

Description

Technical Field

[0001] This invention relates to the field of visual measurement, specifically to a method and system for detecting the thickness of ice accretion on power transmission lines. Background Technology

[0002] In winter or in high-altitude, high-humidity areas, transmission lines are highly susceptible to icing due to rain, snow, and freezing weather. Icing significantly increases the mechanical load on towers, leading to serious accidents such as conductor galloping, damage to ground wire supports, insulator flashover, and even tower collapse and line breakage, posing a significant threat to power grid safety. Therefore, accurate and efficient monitoring of icing thickness on transmission lines is a crucial technical link for achieving early warning of power grid ice disasters, guiding scientific de-icing operations, and ensuring the safe and stable operation of the power grid, and has extremely important practical significance.

[0003] Existing binocular vision-based methods for detecting icing thickness face significant challenges in practical applications, primarily due to the unique optical properties of iced surfaces. Icing on transmission lines exhibits strong refraction and reflection of ambient light, resulting in weak texture on the iced surface in images. More importantly, this optical effect makes the texture or structure of the icing highly dependent on the image acquisition viewpoint. At some views, the texture features of the icing may be clearly visible, while at others, they may completely disappear due to specular reflection or light penetration, appearing as textureless areas. This viewpoint-dependent texture instability makes it difficult for conventional binocular vision matching algorithms (such as Semi-Global Matching, SGM) to obtain stable and accurate initial matching and aggregation costs when calculating the disparity between corresponding points in the left and right images. Ultimately, this leads to large depth calculation errors and unreliable icing thickness measurements. Summary of the Invention

[0004] To address the above problems, this invention provides a method and system for detecting the icing thickness of power transmission lines.

[0005] The present invention provides a method and system for detecting the icing thickness of transmission lines, which adopts the following technical solution: One embodiment of the present invention provides a method for detecting the icing thickness of transmission lines, the method comprising the following steps: The drone is equipped with a binocular camera and an IMU unit. The IMU unit is used to obtain the drone's position, and the binocular camera is used to collect images at each position. For any disparity 'a' of any pixel at any position A, obtain the disparity 'b' corresponding to disparity 'a' at any position B. The matching cost of the pixel at disparity 'b' in the image acquired at position B is recorded as the viewpoint cost at position B. The viewpoint cost of disparity 'a' at all positions is used as the cost feature of disparity 'a'. The matching cost is obtained by binocular ranging technology. The cost feature difference of the same pixel at the same location under different parallaxes is used as the multi-view texture salience of the pixel. In the image acquired at this location, the pixels with the largest multi-view texture salience are used as reference pixels. The SGM algorithm is used to obtain the aggregate cost of all pixels in the image acquired at each location under all parallaxes; the distance between the power transmission line and the UAV is obtained by fusing the aggregate costs of all pixels at all locations under all parallaxes, and the ice thickness is detected by using the distance. The initial cost of each disparity in the SGM algorithm is the mean of all viewpoint costs in the cost features of each disparity. The aggregation path in the SGM algorithm is determined by the distribution of reference pixels.

[0006] Preferably, the specific steps for obtaining the disparity b corresponding to disparity a at any position B are as follows: The image acquired at position A is denoted as FA, and the image acquired at position B is denoted as FB. For any pixel Q in FA, the corresponding pixel Q0 in image FB is obtained. Under parallax a, the depth Z(Q) of pixel Q at position A is Z(Q) = (f×b) / a, where f is the camera focal length and b is the baseline length of the stereo camera. All positions are in the same plane, and the position where the binocular camera faces the power line is denoted as the reference position. The angle between the displacement of the reference position and the adjacent position and the horizontal plane is denoted as c1. Given c1 and the depth Z(Q) of pixel Q at position A, the depth Z(Q0) of pixel Q0 at position B is obtained according to geometric knowledge. Z(Q0) and b satisfy the formula Z(Q0)=(f×b) / b. b is calculated according to this formula.

[0007] Preferably, the specific steps for obtaining the cost feature difference of the same pixel at the same location under different disparities are as follows: For the cost features of the same pixel at the same location under all disparities, the cosine similarity between every two cost features is calculated, and the mean of the cosine similarities between all cost features is calculated. The difference is negatively correlated with the mean.

[0008] Preferably, the specific steps for path aggregation in the SGM algorithm are as follows: For any pixel in any image, obtain several candidate paths starting from that pixel. The ratio of the sum of the multi-view texture salience of all reference pixels on the candidate path to the sum of the multi-view texture salience of all pixels on the candidate path is recorded as the optimization degree of the candidate path. Select a set of target paths from all candidate paths, where the mean of the optimization degree of all candidate paths in the target path set is maximized and the angular characteristics of the candidate paths in the target path set differ the most; the angular characteristics refer to the angular size of the direction in which each candidate path is located. All paths in the target path combination are used as several aggregated paths set for each pixel in the SGM algorithm.

[0009] Preferably, the specific steps of fusing the aggregation cost of all pixels at all locations under all disparities are as follows: The position where the binocular camera is facing the power line is denoted as reference position P1; for the two positions at both ends of the UAV's trajectory, one of the positions is denoted as g. D1: Obtain the position adjacent to position g and closest to the reference position P1 along the motion trajectory, and denote it as g1; fuse the aggregation cost of all pixels in the image acquired by g under all disparities into the aggregation cost of the image acquired by g1; D2: Use position g1 as position g, and repeat D1 until g1 is the reference position P1; D3: For the two positions at both ends of the drone's trajectory, take the other position as g, re-execute D1 and D2, and obtain the fusion aggregation cost of all pixels in the image acquired by P1 under all parallaxes.

[0010] Preferably, the specific steps for detecting the ice thickness are as follows: For the image acquired at reference position P1, the fusion aggregation cost of each pixel in the image under all disparities is calculated, and the disparity with the minimum aggregation cost is obtained as the optimal disparity of each pixel. The depth of each pixel is obtained based on the optimal disparity of each pixel using binocular vision technology. The connected components of the icy power transmission line within the image are segmented. The mean depth of the two edge pixels m1 and m2 of the connected component is denoted as M, which represents the distance between the power transmission line and the drone. Given the formula: Actual height of object = (object pixel height × actual distance) / camera focal length; take M as the actual distance, and the Euclidean distance between m1 and m2 as the object pixel height, substitute them into the formula to get the actual height of the object, subtract the cross-sectional diameter of the transmission line from the actual height of the object and divide by 2 to get the ice thickness.

[0011] Preferably, the specific steps for selecting a set of target path combinations from all candidate paths are as follows: Select several candidate paths with replacement from all candidate paths and combine them into a path combination; traverse all selection and combination methods to obtain several path combinations. For any path combination, obtain the average optimization degree of all candidate paths in the path combination, sort all candidate paths by angular features from smallest to largest, calculate the absolute value of the difference between two adjacent angular features, and calculate the mean of the absolute values ​​of the differences between all adjacent angular features; calculate the screening index of the path combination, which is positively correlated with the average optimization degree and the mean of the absolute values ​​of the differences, respectively. The path combination with the highest selection metric among all path combinations is selected as the target path combination.

[0012] Preferably, the specific steps of fusing the aggregated cost of all pixels in the image acquired by g under all parallaxes into the aggregated cost of the image acquired by g1 are as follows: For any pixel q in the image acquired by g, for the corresponding pixel q0 in the image acquired at g1, and for any disparity a1 of pixel q at position g1 corresponding to disparity b1, the aggregation cost f(q, a1) of pixel q in disparity a1 is fused into the aggregation cost f(q0, b1) of pixel q0 in disparity b1. The fusion result is equal to the mean of f(q, a1) and f(q0, b1), and this mean is replaced by the aggregation cost of pixel q0 in disparity b1.

[0013] Preferably, the specific steps for obtaining the corresponding pixel are as follows: For any two images FA and FB, and for any pixel Q in image FA; Corner detection and matching are performed on images FA and FB. The homography matrix is ​​solved using the RANSAC algorithm based on the coordinates of the matched corners. The coordinates of the pixels Q in FA are then subjected to a radial transformation using the homography matrix. The pixel in image FB that has the closest Euclidean distance to the coordinates after the radial transformation is denoted as the same pixel Q0.

[0014] Another embodiment of the present invention provides a transmission line icing thickness detection system, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement all the steps of the above-described transmission line icing thickness detection method.

[0015] The beneficial effects of the technical solution of the present invention are: This invention significantly improves the robustness and accuracy of stereo matching in weak texture and view-sensitive regions. In existing technologies, conventional SGM algorithms struggle to obtain accurate initial matching costs and aggregate costs when dealing with scenes like icy roads where weak textures are caused by ambient light refraction and reflection, and where texture appearance changes with viewpoint. This invention constructs cost features for each disparity (i.e., the set of viewpoint costs for that disparity across all positions), so that the judgment of the quality of a disparity no longer depends on the matching results from a single viewpoint, but rather integrates matching information from multiple viewpoints. Even if the icy texture disappears or performs poorly at certain viewpoints, as long as effective texture information is presented at some viewpoints, the cost features of that disparity can be effectively constructed, thus providing a reliable data foundation for subsequent optimization and fundamentally overcoming the limitations of single-viewpoint binocular vision.

[0016] This invention utilizes multi-view texture saliency as the basis for selecting reference pixels. Pixels with significant differences in matching costs across multiple views (i.e., high texture saliency) are considered to have more reliable cost features and are selected as reference pixels. Furthermore, the aggregation path of the SGM algorithm is determined by the distribution of these reliable reference pixels. This means that the cost aggregation process prioritizes pixel paths with reliable texture information and high matching quality, effectively avoiding the propagation of incorrect matching costs from unreliable regions. It guides the aggregation algorithm to focus on regions with high information quality in the image, thereby obtaining a more accurate and reliable disparity map in complex scenes.

[0017] This invention does not simply average the disparity results from multiple viewpoints. Instead, it deeply fuses multi-view information into the initial cost of each location, each pixel, and each disparity before the cost aggregation stage of the SGM algorithm. Finally, the aggregated costs from all locations are fused to obtain a globally optimal fusion result. This "cost-level" deep fusion allows disparities that cannot be effectively detected in a single viewpoint due to baseline or viewpoint limitations to be correctly calculated using supplementary information from other viewpoints, thereby indirectly expanding the effective disparity detection range. The resulting distance between the power transmission line and the drone is more accurate, laying a solid foundation for subsequent precise detection of icing thickness.

[0018] In summary, this invention effectively solves the problems of weak texture and viewing angle sensitivity caused by the special optical properties of icing surfaces, and significantly improves the accuracy, robustness and practicality of icing thickness detection for transmission lines. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the steps of a method for detecting ice thickness on power transmission lines according to an embodiment of the present invention. Figure 2 This is a geometric schematic diagram of all locations relative to the transmission line according to the present invention; Figure 3 This is a schematic diagram of a power transmission line. Detailed Implementation

[0021] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method and system for detecting icing thickness on transmission lines according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] The following description, in conjunction with the accompanying drawings, details the specific scheme of the method and system for detecting the icing thickness of power transmission lines provided by the present invention.

[0024] Example 1: Please see Figure 1 The diagram illustrates a flowchart of a method for detecting the icing thickness of a power transmission line according to an embodiment of the present invention. The method includes the following steps: Step S101: The drone is equipped with a binocular camera and an IMU unit. The IMU unit is used to obtain the position of the drone during its movement, and the binocular camera is used to collect images at each position.

[0025] A remotely controlled drone is used to inspect power transmission lines. The drone is equipped with a binocular camera, consisting of two cameras (left and right). It also carries an IMU (Inertial Measurement Unit), which mainly comprises an accelerometer and a gyroscope. In this example, the sampling frequency of the accelerometer and gyroscope is 1 kHz, and the sampling frequency of the binocular camera is 24 Hz.

[0026] The drone's binocular cameras are horizontally oriented. The remote operator issues a command to measure the icing thickness. Upon receiving the command, the drone (or the CCD sensor used for imaging) sets its position to [0,0,0] and creates a three-dimensional Cartesian coordinate system XYZ with this position as the origin. The X and Y axes are in a horizontal plane, and the Z axis points vertically upwards. Simultaneously, the drone is prohibited from left or right movement (e.g., by disabling the left / right movement joystick, or by using a PID algorithm to control the speed of left / right movement to zero). The purpose is to ensure that the drone moves within a single plane (the plane perpendicular to the length of the power line). In a simplest and easiest implementation, the drone simply moves vertically.

[0027] Simultaneously, starting from position [0,0,0], the UAV uses inertial navigation technology to obtain its motion trajectory based on accelerometer and gyroscope data collected by the IMU. The motion trajectory is a coordinate sequence composed of coordinate points originating from the origin.

[0028] In this embodiment, several coordinate points are sampled along the motion trajectory, and these coordinate points are used as the positions of the UAV during its movement. One of these positions must be designated as the reference position, where the camera's field of view is directly facing the power line.

[0029] As an example, the coordinates of the drone after moving 10 centimeters along the trajectory are taken as a position (including the origin). In this embodiment, a total of 6 positions are sampled. It is preferred that the number of positions is greater than 3. In other embodiments, other methods can be used to sample several positions, ensuring that the movement path length of all positions is not too large (e.g., less than 1 meter) to avoid excessive differences in the viewpoint of different positions.

[0030] like Figure 2 As shown, P1, P2, P3, and P4 are four of the locations obtained in this example, and P0 represents any point on the power transmission line. At P1, the camera's field of view is directly facing the power transmission line, which means P1 serves as a reference position.

[0031] It should be noted that in this embodiment, since the camera is not in close contact with the power line to acquire images, the distance between the camera and the power line is much greater than the diameter of the power line. Therefore, in this embodiment, the camera (or the CCD sensor used for imaging) is regarded as a point mass, and the ambient light "emitted" from any position on the power line is incident parallel to the camera.

[0032] To further clarify, angle c1 is known, and c1 equals the angle between the displacement from P1 to P2 and the horizontal plane. The coordinates of P1, P2, P3, and P4 are known (obtained through inertial navigation technology) and lie in the same plane. The coordinates of point P0 are unknown. Based on this, for line segments P1PO, P2PO, P3PO, and P4PO, if the length of any one of these line segments is given, then the lengths of all other line segments (including angles c2 and c3) can be uniquely determined. This conclusion forms the basis for subsequent calculations and can be derived through geometric reasoning, as follows: (1) such as Figure 2 As stated above, if line segment P1PO is denoted as L, and in triangle P1POP2, the lengths of two sides (line segment P1PO and line segment P1P2) and an included angle (c1) are known, then all parameters of triangle P1POP2 are known and unique (including the length of line segment P2PO and the included angle P1P2P0).

[0033] (2) Since P1, P2, and P3 are known, the angle (counterclockwise angle) between line segment P1P2 and line segment P2P3 is also known. Subtract the included angle P1P2P0 from the angle between P1P2 and P2P3 to get c2.

[0034] (3) In triangle P2POP3, if the lengths of two sides (segment P2PO and segment P2P3) and an included angle (c2) are known, then all parameters of triangle P2POP3 are known and unique (including the length of segment P3PO and the included angle P2P3P0).

[0035] (4) And so on, when L is known, Figure 2 All line segments in the shown geometric figure are known. Furthermore, when L is unknown, but the length of any one of line segments P1PO, P2PO, P3PO, and P4PO is known, L can be calculated in reverse. Figure 2 All line segments in the geometric figure shown are known.

[0036] The above all utilize well-known geometric knowledge; this embodiment will not provide specific derivation formulas and solution methods.

[0037] In special cases, when all positions are not on the same plane, for example, when the least squares method is used to fit a plane based on all positions and the average distance from all positions to the plane is greater than 2.5 cm, the remote operator controls the drone to change the drone's attitude and trajectory, and then reissues the ice thickness measurement command.

[0038] For any given location, and the image captured by the drone at that location, this image refers to the left and right images captured by the left and right cameras in the binocular camera system, respectively. The left and right images exhibit parallax. Both the left and right images are color images; in this embodiment, only the L channel of the left and right images in the Lab color space is retained. Therefore, all images mentioned subsequently refer to grayscale images of the L channel.

[0039] This embodiment uses the left image as an example. In existing technologies, when using a binocular camera (or using the left and right images) for ranging, the matching cost (i.e., the initial matching cost) of any pixel in the left image under any disparity is calculated using Census transform (window size 3×3) and Hamming distance. The initial matching cost of all pixels under all disparities is aggregated using a multi-path aggregation strategy using the SGM (Semi-Global Matching) algorithm to obtain the aggregated cost of all pixels under all disparities. Then, the optimal disparity d of any pixel is obtained using the Winner-Take-All (WTA) strategy. Finally, the depth of any pixel is obtained using the formula Z=(f×b) / d. Here, Z represents the depth of any pixel (the depth of the object from the camera plane), f is the camera focal length, and b is the baseline length of the left and right cameras. The focal lengths of the left and right cameras are the same, and the focal length and baseline length are calibrated and are known quantities. After the UAV moves to a certain position, the depth of each pixel in the acquired image (left image) can be obtained. This embodiment measures the icing thickness based on this depth. In addition, in this embodiment, on the one hand, the distance between the drone and the power transmission line is relatively far (it is not close to the power transmission line, and the camera can be regarded as a point mass), and on the other hand, in order to save computation, this embodiment sets the parallax range to a small range, such as [0, 64]. That is, any parallax mentioned later refers to any integer within this parallax range. In other embodiments, the parallax range can be set to other values, but this embodiment does not make specific limitations.

[0040] However, because icing on transmission lines has a strong refractive and reflective effect on environmental optical fibers, the icing not only exhibits weak texture characteristics, but the texture or structure also varies with the image acquisition perspective. For example, at some perspectives, the texture or structure is not clearly visible, which makes it impossible for the conventional binocular visual ranging and SGM aggregation methods to obtain accurate and reliable optimal parallax.

[0041] In this embodiment, based on the existing binocular visual ranging and SGM aggregation methods, the motion characteristics and viewpoint change characteristics of the UAV are combined to further obtain a more accurate and reliable optimal disparity.

[0042] Step S102: For any disparity a of any pixel at any position A, obtain the disparity b corresponding to disparity a at any position B. The matching cost of the pixel at disparity b in the image acquired at position B is recorded as the view cost at position B. The view cost of disparity a at all positions is used as the cost feature of disparity a.

[0043] For any position A and an image FA acquired at A (taking the left image as an example), any disparity of any pixel Q in image FA is represented as a.

[0044] Obtain the pixel Q0 in image FB corresponding to pixel Q in FA.

[0045] For any position B and the image FB acquired at B (taking the left image as an example), obtain the disparity b corresponding to disparity a at position B; The matching cost of pixel Q0 in FB under disparity b is obtained (the method of obtaining it has been described in step S101), and is denoted as the viewing cost of disparity a at position B.

[0046] Thus, for any pixel at position A, its disparity 'a' corresponds to a viewpoint cost at any position. The viewpoint costs of disparity 'a' at all positions are used as the cost features of disparity 'a'.

[0047] It should be noted that the cost feature is a vector, for example, the viewpoint costs at positions P1, P2, P3, P4, etc. are arranged into a vector.

[0048] The cost feature describes the matching cost of any pixel at any location under any parallax from different viewpoints.

[0049] Specifically, when positions B and A are the same, pixel Q0 and pixel Q are the same pixel in the same image. In this case, the viewpoint cost is the cost of pixel Q in the image acquired at position A under the parallax a.

[0050] As an example, obtaining the pixel Q0 in image FB corresponding to pixel Q in FA includes the following methods: Corner detection (using the Sift corner detection algorithm) and corner matching (using the approximate nearest neighbor matching algorithm) are performed on images FA and FB. The homography matrix is ​​solved using the RANSAC algorithm based on the coordinates of the matched corners. The coordinates of the pixels Q in FA are then radiated using the homography matrix. The pixel in image FB that has the nearest Euclidean distance (or coincides) with the coordinates after the radiated transformation is denoted as pixel Q0.

[0051] This embodiment uses the Sift corner detection algorithm to detect corners. The contrast threshold in this algorithm is set to 0.03 and the edge threshold is set to 15. In addition, the FLANN (Fearless Nearest Neighbor) matching algorithm is used for corner matching.

[0052] As an example, the method for obtaining the disparity b corresponding to disparity a at position B includes: Positions A and B are known quantities; under disparity a, the depth Z(Q) of pixel Q at position A is (f × b) / a. Pixel Q is equivalent to... Figure 2 In the equation P0, depth Z(Q) is equivalent to Figure 2 The length of the line segment from any position on the trajectory to P0 is given. At this point, for position B, the length of the line segment from B to P0 can be uniquely determined according to the conclusion of step S101.

[0053] For pixel Q0 in the image captured at point B, pixel Q0 and pixel Q represent the same object (i.e., pixels with the same name). The length of the line segment from B to P0 is taken as the depth of pixel Q0 at point B, denoted as Z(Q0). Since Z(Q0) = (f × b) / b, the unknown quantity b in this formula can be solved.

[0054] Note that the purpose of this example is not to obtain the position of P0, and the depth Z(Q) is not the final depth to be obtained in this embodiment. The purpose of this example is to obtain the disparity of a disparity in image FA in image FB, which is equivalent to obtaining the length b of an object of length a in image FA in FB.

[0055] Thus, based on geometric relationships, the disparity b corresponding to disparity a at position B has been obtained. Further, the cost feature of any pixel at any position with any disparity is obtained. It should be noted that images acquired at different locations may contain pixels with the same name (e.g., Q and Q0 mentioned above). The cost features under the corresponding disparities (e.g., disparities a and b) are theoretically identical, thus requiring only one calculation. However, this embodiment still recalculates the cost features using the method described above. The purpose is to enable matching between images acquired at different locations during cost feature calculation (i.e., the corner detection, corner matching, and radiative transformation processes described above). Because images acquired at different locations have different viewing angles, there is a matching error (i.e., an error in obtaining pixel Q0), resulting in different actual cost features. Recalculating the cost features allows images acquired at different locations to match each other, effectively reducing the impact of matching errors and avoiding the possibility of introducing large errors when only one calculation is needed.

[0056] Step S103: The cost feature difference of the same pixel at the same location under different parallaxes is used as the multi-view texture salience of the pixel. In the image acquired at this location, the pixels with the largest multi-view texture salience are used as reference pixels.

[0057] The above obtains the cost features of any pixel at any location under any disparity. For the cost features of the same pixel at the same location under all disparities, cosine similarity is calculated between every two cost features. The mean of the cosine similarity between all cost features is represented as x. The multi-view texture saliency of the pixel is obtained, and its value is negatively correlated with x.

[0058] In other embodiments, to reduce computation, the cost features of two adjacent disparities (i.e. two disparities that differ by 1) are calculated to have cosine similarity, and the mean of the cosine similarity of all adjacent disparities is represented as x.

[0059] As an example, let exp(-x) be the multi-view texture saliency, where exp() represents an exponential function with the natural constant as the base.

[0060] Multi-view texture saliency describes whether the object corresponding to each pixel exhibits obvious texture structure information from multiple viewpoints. Specifically, the smaller x is, the more significant the cost features of the pixel under different parallaxes. This further indicates that the same object has different and diverse matching costs under different parallaxes captured from different viewpoints. The diversity of matching costs is determined by the texture differences between pixels (between the pixel and pixels under different parallaxes in the right image). Therefore, it means that the pixel can exhibit obvious texture structure information when viewed from multiple viewpoints. In other words, because obvious texture structure information is exhibited from multiple viewpoints, the pixel has different and diverse texture matching situations with pixels under different parallaxes in the right image under different viewpoints (for example, the refraction or reflection of ambient light by ice causes different texture manifestations under different viewpoints), resulting in different matching costs (i.e., smaller x). In this case, the multi-view texture saliency is larger.

[0061] The larger x is, the less obvious the difference in cost features of the pixel under different parallaxes. This further indicates that the same object has the same or indistinguishable matching cost under different parallaxes captured from different viewpoints. It also indicates that the pixel does not present texture structure information when viewed from multiple viewpoints, or the texture cannot be clearly captured in most viewpoints (e.g., it cannot be captured due to the refraction or reflection of ambient light by ice). In this case, the multi-view texture prominence is small.

[0062] In summary, pixels with higher multi-view texture saliency are more helpful in evaluating matching costs under different parallaxes from different viewpoints, or in other words, the cost features of that pixel under different parallaxes are more accurate and reliable. Pixels with lower multi-view texture saliency may not be able to evaluate matching costs under different parallaxes from different viewpoints (for example, the texture structure of this pixel may not be captured at most viewpoints due to refraction or reflection, resulting in distorted and inaccurate matching costs with no reference value, only increasing interference), indicating that the cost features of this pixel under different parallaxes are not reliable.

[0063] For all pixels in an image acquired at any location, each pixel corresponds to a multi-view texture saliency. In this embodiment, the softmax formula is used to normalize the multi-view texture saliency of all pixels in an image. The multi-view texture saliency mentioned thereafter refers to the normalized multi-view texture saliency.

[0064] The N0 pixels with the highest multi-view texture prominence in the image are designated as reference pixels. In this embodiment, N0 is set to one-quarter (rounded up) of the total number of pixels in the image.

[0065] Step S104: Use the SGM algorithm to obtain the aggregate cost of all pixels in the image acquired at each location under all disparities. The initial cost in the SGM algorithm is the mean of all matching costs in the cost features. The aggregation path in the SGM algorithm is determined by the reference pixel.

[0066] The SGM (Semi-Global Matching) algorithm is a classic stereo matching algorithm that approximates two-dimensional global optimization by aggregating one-dimensional paths in multiple directions, significantly reducing computational complexity while maintaining accuracy. In this embodiment, for an image acquired at any given location, the SGM algorithm is used to aggregate the cost of all pixels in the image at each location across all disparities.

[0067] Specifically, the SGM algorithm takes the initial cost of all pixels in the image under all disparities as input and outputs the aggregated cost of all pixels in the image under all disparities. The initial cost of each pixel under each disparity is the mean of all matching costs in the cost features of each pixel under each disparity.

[0068] In addition, when running the SGM algorithm, several aggregation paths need to be set for each pixel, and these aggregation paths are composed of several reference pixels.

[0069] As an example, the method for obtaining the aggregation path is as follows: For any pixel in the same image, a horizontal ray is drawn from that pixel as the starting point. The ray rotates 15° around the pixel as the center. After each rotation, the pixels in the 9×9 neighborhood of the pixel formed by the ray constitute a candidate path. The angle of the rotation process is recorded as the angular feature of the candidate path. The ratio of the sum of the multi-view texture saliency of all reference pixels on the candidate path to the sum of the multi-view texture saliency of all pixels on the candidate path is recorded as the optimization degree of the candidate path.

[0070] From all candidate paths, select four candidate paths with replacement and combine them into a path combination. Iterate through all the selection methods and from all the resulting path combinations, select a target path combination that maximizes the mean optimization of all candidate paths and maximizes the difference in angular characteristics between the candidate paths.

[0071] A higher mean of the optimization score indicates greater multi-view texture prominence of the pixel and higher accuracy of the pixel's cost features (or initial cost). Conversely, a lower mean of the optimization score indicates less multi-view texture prominence of the pixel, making it unreliable for the aggregation process of the initial cost. The greatest difference in angular features between candidate paths ensures cost aggregation from a larger pixel distribution range.

[0072] All paths in the target path combination are used as several aggregated paths set for each pixel in the SGM algorithm.

[0073] As an example, the method for obtaining the target path combination is as follows: For any path combination, obtain the average optimization degree of all candidate paths in the path combination, sort all candidate paths by angular features from smallest to largest, calculate the absolute value of the difference between two adjacent angular features, and divide the mean of the absolute values ​​of the differences between all adjacent angular features by 90° to obtain the distribution feature of the path combination. The purpose of dividing by 90° is to remove the dimensions and order of magnitude. The average optimization degree and the mean of the distribution feature are used as the screening index for path combinations.

[0074] The path combination with the highest selection metric among all path combinations is selected as the target path combination.

[0075] In particular, if none of the paths in the target path combination pass through the reference pixel, that is, when the average optimization degree is equal to 0, it is still regarded as an aggregated path.

[0076] Thus, for each image acquired at each location, the aggregate cost of all pixels in the image under all parallaxes has been obtained.

[0077] Step S105: Combine the aggregation costs of all pixels at all locations under all parallaxes.

[0078] For the two positions at both ends of the drone's trajectory, one of the positions is g (e.g., the position with the smallest path length to the reference position P1).

[0079] (1) Obtain the position adjacent to position g and closest to reference position P1 along the motion trajectory, and denot it as g1; merge the aggregation cost in the image collected by g into the aggregation cost in the image collected by g1.

[0080] (2) Use g1 as position g and repeat (1) until g1 is the reference position P1.

[0081] (3) For the two positions at both ends of the UAV's trajectory, take the other position as g and re-execute (1)~(2).

[0082] The above process achieves the goal of fusing the aggregation cost of all pixels at all locations under all parallaxes.

[0083] As an example, the method for fusing the aggregation cost in the image acquired by g into the aggregation cost in the image acquired by g1 is as follows:

[0084] For any pixel q in the image acquired by g, obtain the corresponding pixel q0 in the image acquired at g1 (see step S102 for details). Obtain the disparity b1 corresponding to any disparity a1 of pixel q at position g1 (see step S102 for details), and fuse the aggregation cost f(q, a1) of pixel q at disparity a1 into the aggregation cost f(q0, b1) of pixel q0 at disparity b1.

[0085] In one example, the fusion result is equal to the mean of f(q, a1) and f(q0, b1).

[0086] In another example, the fusion result is equal to the sum of w1×f(q, a1) and w2×f(q0, b1). w1 and w2 are obtained as follows: for the average optimization degree W1 of the target path combination corresponding to pixel q, and the average optimization degree W2 of the target path combination corresponding to pixel q0, normalize W1 and W2 using the softmax formula to obtain w1 and w2.

[0087] The purpose is to ensure that the aggregation cost obtained under the optimal combination of target paths with a larger average is more reliable. Therefore, when fusing, we focus on the aggregation cost with greater reliability to avoid the problem that the final fusion result is unreliable when the aggregation cost in the image at a certain location is inaccurate or unreliable.

[0088] The fusion result serves as the new aggregation cost of pixel q0 in disparity b1. This new aggregation cost is used in subsequent calculations, which means it participates in the repeated execution process of (1).

[0089] Specifically, if b1 exceeds the disparity range (defined in step S101), f(q0, b1) cannot be obtained. In this case, b1 is also treated as a value within the disparity range, and the fusion result is equal to f(q, a1). Based on this, it can be seen that even if the disparity range is set relatively small in step S101, disparity analysis with a large disparity range can still be achieved through multi-view fusion.

[0090] Step S106: Obtain the distance between the power transmission line and the drone using the fused aggregation cost, and use the distance to detect the icing thickness.

[0091] The above process fuses the aggregated costs of all pixels at all locations under all disparities into the image acquired at reference location P1. For the aggregated cost of each pixel in this image under all disparities (i.e., the new aggregated cost after fusion, hereinafter referred to as the fused aggregated cost), the disparity with the minimum aggregated cost is obtained as the optimal disparity for each pixel, which is the optimal disparity d mentioned in step S101. The depth of each pixel and the corresponding depth map of the image are obtained using the formula Z=(f×b) / d.

[0092] In the image acquired by P1, the depth of each pixel represents the distance between the power transmission line and the drone at different locations.

[0093] In the image acquired at reference position P1, the connected region of the icy power transmission line is obtained. For any vertical cross-section of the connected region, for two pixels where this cross-section intersects with the upper and lower edges of the connected region, such as... Figure 3 In the given information, the pixel with the smallest depth is obtained by summing the pixels along the line connecting pixels m1 and m2. Figure 3 m3 in the middle.

[0094] As an alternative example, suppose the cross-section of an ice-covered transmission line is a circle. Figure 3 In this diagram, m4 is considered the center of the circle. The radius r of the circle is equal to the difference between the average depth of m1 and m2 and the depth of m3. The icing thickness (horizontal icing thickness) is obtained by subtracting the cross-sectional radius of the transmission line from the radius r.

[0095] The cross-sectional radius of the transmission line is measured in advance and is a known quantity.

[0096] In another example, let M be the average depth of m1 and m2. Given the formula: Actual height of object = (Object pixel height × Actual distance) / Focal length, let M be the actual distance, the Euclidean distance between m1 and m2 be the object pixel height, and the focal length be f. Substituting these values ​​into the formula yields the actual height of the object. Subtract the cross-sectional diameter of the power line from the actual height, and then divide by 2 to obtain the icing thickness (vertical icing thickness).

[0097] In another example, the icing thickness (vertical icing thickness) can also be obtained by making the drone's binocular camera vertically upward and then following all the steps above.

[0098] In another example, for all vertical cross-sections of the power transmission line connected domain within the image, the average value of the ice thickness obtained from the above example is used as the ice thickness for this example.

[0099] In another example, the above process uses the left image as an example to obtain the icing thickness, and then uses the right image as an example to obtain the icing thickness. The average of the two icing thicknesses is used as the icing thickness in this example.

[0100] In another example, after deleting several rows and columns from each of the four sides of the image, the image width and height are reduced by 20% (the number of rows and columns deleted is rounded down) to obtain the ice thickness. The purpose of this method is to reduce the amount of computation.

[0101] In another example, considering the inherent error in ice thickness measurement—one source of which is the inability to guarantee the camera is directly facing every point on the ice-covered power line—this error, within ±8% of the ice thickness, is negligible. In this example, the operator can remotely control a drone to issue multiple ice thickness measurement commands from different distances from the power line. The average of all obtained ice thickness measurements is then calculated, thus increasing the measurement time to improve accuracy.

[0102] This concludes the example.

[0103] In summary, this embodiment, by utilizing specific measurement methods, further accurately measured the icing thickness of transmission lines from multiple perspectives.

[0104] Example 2: Step S101 of Embodiment 1 includes: the camera's field of view is facing the power transmission line, including the following steps: After the camera captures each image (taking the left image as an example), a rectangular Region of Interest (ROI) is pre-drawn on that image. The center of the high region coincides with the center of the image. The width of the region is 30% of the image width (rounded up), and the height is 10% of the image height.

[0105] The connected components of the power transmission line are segmented. For the two edges of the connected components (the edges opposite to the ice-covered power transmission line), the two closest pixels on these two edges are taken as a pair of opposite edge pixels. The average value of the position of the opposite edge pixels is recorded as the center pixel. The center pixels of all opposite edge pixels are connected to obtain the central axis of the ice-covered power transmission line. When the major axis of the rectangular bounding box of the central axis is horizontal and the top and bottom edges of the bounding box pass through the ROI region, it is determined that the camera's field of view is facing the power transmission line.

[0106] When the camera's field of view is facing the power line, mark the location described by the drone as P1, and sample a location every 10 centimeters forward and backward from P1 along the movement trajectory.

[0107] In special cases where the camera's field of view is not directly facing the power line, the remote operator manipulates the drone to change its attitude and trajectory, and then reissues the command to measure the icing thickness.

[0108] As an example, methods for segmenting the connected components of icing-covered transmission lines include: The image is input into a semantic segmentation network (e.g., DeepLabV3), and the semantic segmentation network outputs the semantic region of each iced power transmission line, that is, the connected domain of the iced power transmission line. This embodiment takes any one connected domain of the iced power transmission line as an example for description.

[0109] As another example, methods for segmenting the connected components of icing-covered transmission lines include: The image is enhanced using Limit Contrast Adaptive Histogram Equalization (CLAHE), and then all edges in the image are obtained using the Canny algorithm. In the Canny algorithm, the length of the Gaussian filter kernel is 3, and the high and low thresholds are 120 and 50, respectively.

[0110] In this embodiment, only edges with an edge width greater than 50% of the image width are retained. The image dimensions in this embodiment are 1024×1024. Preferably, when remotely controlling a drone to inspect power transmission lines, personnel should avoid including towers and other close-up objects in the binocular camera's field of view to prevent incorrect extraction of the connected components of the icy power transmission line. For any two edges, the average Euclidean distance between all opposite pixels on these two edges is recorded as the matching value of the two edges. For any given edge, the edge with the smallest matching value among all other edges is considered as a pair of opposite edges. The two nearest endpoints of these two edges are connected to form a closed region, which serves as the connected component of the icy power transmission line.

[0111] As an example, the acquisition of the connected domain of the icing transmission line described in step S106 of Embodiment 1 is based on a depth map, and the specific process is the same as the example above.

[0112] Example 3: This embodiment provides a transmission line icing thickness detection system. The system includes a drone equipped with a binocular camera and an IMU unit. The binocular camera is used to acquire images, and the IMU unit is used to locate the drone's position. The drone includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor reads the images and the drone's position, and implements all the steps of all the above embodiments when running the computer program.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting the icing thickness of transmission lines, characterized in that, The method includes the following steps: The drone is equipped with a binocular camera and an IMU unit. The IMU unit is used to obtain the drone's position, and the binocular camera is used to collect images at each position. For any disparity 'a' of any pixel at any position A, obtain the disparity 'b' corresponding to disparity 'a' at any position B. The matching cost of the pixel at disparity 'b' in the image acquired at position B is recorded as the viewpoint cost at position B. The viewpoint cost of disparity 'a' at all positions is used as the cost feature of disparity 'a'. The matching cost is obtained by binocular ranging technology. The cost feature difference of the same pixel at the same location under different parallaxes is used as the multi-view texture salience of the pixel. In the image acquired at this location, the pixels with the largest multi-view texture salience are used as reference pixels. The SGM algorithm is used to obtain the aggregate cost of all pixels in the image acquired at each location under all parallaxes; the distance between the power transmission line and the UAV is obtained by fusing the aggregate costs of all pixels at all locations under all parallaxes, and the ice thickness is detected by using the distance. The initial cost of each disparity in the SGM algorithm is the mean of all viewpoint costs in the cost features of each disparity. The aggregation path in the SGM algorithm is determined by the distribution of reference pixels.

2. The method for detecting icing thickness on transmission lines according to claim 1, characterized in that, The specific steps for obtaining the disparity b corresponding to disparity a at any position B are as follows: The image acquired at position A is denoted as FA, and the image acquired at position B is denoted as FB. For any pixel Q in FA, the corresponding pixel Q0 in image FB is obtained. Under parallax a, the depth Z(Q) of pixel Q at position A is Z(Q) = (f×b) / a, where f is the camera focal length and b is the baseline length of the stereo camera. All positions are in the same plane, and the position where the binocular camera faces the power line is denoted as the reference position. The angle between the displacement of the reference position and the adjacent position and the horizontal plane is denoted as c1. Given c1 and the depth Z(Q) of pixel Q at position A, the depth Z(Q0) of pixel Q0 at position B is obtained according to geometric knowledge. Z(Q0) and b satisfy the formula Z(Q0)=(f×b) / b. b is calculated according to this formula.

3. The method for detecting icing thickness on transmission lines according to claim 1, characterized in that, The specific steps for obtaining the cost feature differences of the same pixel at the same location under different disparities are as follows: For the cost features of the same pixel at the same location under all disparities, the cosine similarity between every two cost features is calculated, and the mean of the cosine similarities between all cost features is calculated. The difference is negatively correlated with the mean.

4. The method for detecting icing thickness on transmission lines according to claim 1, characterized in that, The specific steps for path aggregation in the SGM algorithm are as follows: For any pixel in any image, obtain several candidate paths starting from that pixel. The ratio of the sum of the multi-view texture salience of all reference pixels on the candidate path to the sum of the multi-view texture salience of all pixels on the candidate path is recorded as the optimization degree of the candidate path. Select a set of target paths from all candidate paths, where the mean of the optimization degree of all candidate paths in the target path set is maximized and the angular characteristics of the candidate paths in the target path set differ the most; the angular characteristics refer to the angular size of the direction in which each candidate path is located. All paths in the target path combination are used as several aggregated paths set for each pixel in the SGM algorithm.

5. The method for detecting icing thickness on transmission lines according to claim 1, characterized in that, The specific steps involved in fusing the aggregation cost of all pixels at all locations under all disparities are as follows: The position where the binocular camera is facing the power line is denoted as reference position P1; for the two positions at both ends of the UAV's trajectory, one of the positions is denoted as g. D1: Obtain the position adjacent to position g and closest to the reference position P1 along the motion trajectory, and denote it as g1; fuse the aggregation cost of all pixels in the image acquired by g under all disparities into the aggregation cost of the image acquired by g1; D2: Use position g1 as position g, and repeat D1 until g1 is the reference position P1; D3: For the two positions at both ends of the drone's trajectory, take the other position as g, re-execute D1 and D2, and obtain the fusion aggregation cost of all pixels in the image acquired by P1 under all parallaxes.

6. The method for detecting icing thickness on transmission lines according to claim 5, characterized in that, The specific steps involved in detecting the ice thickness are as follows: For the image acquired at reference position P1, the fusion aggregation cost of each pixel in the image under all disparities is calculated, and the disparity with the minimum aggregation cost is obtained as the optimal disparity of each pixel. The depth of each pixel is obtained based on the optimal disparity of each pixel using binocular vision technology. The connected components of the icy power transmission line within the image are segmented. The mean depth of the two edge pixels m1 and m2 of the connected component is denoted as M, which represents the distance between the power transmission line and the drone. Given the formula: Actual height of object = (object pixel height × actual distance) / camera focal length; take M as the actual distance, and the Euclidean distance between m1 and m2 as the object pixel height, substitute them into the formula to get the actual height of the object, subtract the cross-sectional diameter of the transmission line from the actual height of the object and divide by 2 to get the ice thickness.

7. The method for detecting icing thickness on transmission lines according to claim 4, characterized in that, The specific steps involved in selecting a set of target path combinations from all candidate paths are as follows: Select several candidate paths with replacement from all candidate paths and combine them into a path combination; traverse all selection and combination methods to obtain several path combinations. For any path combination, obtain the average optimization degree of all candidate paths in the path combination, sort all candidate paths by angular features from smallest to largest, calculate the absolute value of the difference between two adjacent angular features, and calculate the mean of the absolute values ​​of the differences between all adjacent angular features. Calculate the screening index for the path combination, wherein the screening index is positively correlated with the average degree of preference and the mean of the absolute value of the difference, respectively; The path combination with the highest selection metric among all path combinations is selected as the target path combination.

8. The method for detecting icing thickness on transmission lines according to claim 5, characterized in that, The specific steps involved in fusing the aggregated cost of all pixels in the image acquired by g under all disparities into the aggregated cost of the image acquired by g1 are as follows: For any pixel q in the image acquired by g, for the corresponding pixel q0 in the image acquired at g1, and for any disparity a1 of pixel q at position g1 corresponding to disparity b1, the aggregation cost f(q, a1) of pixel q in disparity a1 is fused into the aggregation cost f(q0, b1) of pixel q0 in disparity b1. The fusion result is equal to the mean of f(q, a1) and f(q0, b1), and this mean is replaced by the aggregation cost of pixel q0 in disparity b1.

9. A method for detecting icing thickness on transmission lines according to claim 2 or 8, characterized in that, The specific steps for obtaining the corresponding pixel are as follows: For any two images FA and FB, and for any pixel Q in image FA; Corner detection and matching are performed on images FA and FB. The homography matrix is ​​solved using the RANSAC algorithm based on the coordinates of the matched corners. The coordinates of the pixels Q in FA are then subjected to a radial transformation using the homography matrix. The pixel in image FB that has the closest Euclidean distance to the coordinates after the radial transformation is denoted as the same pixel Q0.

10. A transmission line icing thickness detection system, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor runs the computer program, it implements all the steps of the method for detecting the icing thickness of a transmission line as described in any one of claims 1 to 9.

Images