Transmission line recognition and positioning method based on depth vision

Abstract

The method for identifying and locating power lines based on depth vision belongs to the field of machine vision. It solves the problem that the recognition effect of power lines in existing methods is easily interfered with and the positioning effect is poor. The specific process of the present invention is: collecting the depth image of the power line, and performing depth filtering, mapping the filtered depth image into a depth grayscale image; performing median filtering and Gaussian filtering on the depth grayscale image to obtain the filtered depth grayscale image; and performing edge detection on the filtered depth grayscale image through an edge detection algorithm to obtain a binary image of the power line contour; performing hough feature fitting in the binary image to obtain the starting points and end points of multiple straight lines in the binary image; after screening multiple straight lines, fitting is performed to realize the recognition of the power line, and the positioning of the power line is completed by calculating the coordinates of the midpoint of each power line in the depth camera coordinate system. The present invention is mainly used for the identification and positioning method of power lines.

Description

Technical Field

[0001] The present invention belongs to the field of recognition and positioning based on machine vision. Background Art

[0002] With the rapid development of drone technology over the past decade, drones, due to their flexible and maneuverable capabilities in three-dimensional space, have played an increasingly important role in the inspection and maintenance of high-voltage transmission lines. Compared to traditional manual inspection methods involving operators wearing specialized equipment on transmission lines and towers, aerial work robot systems such as drones offer numerous advantages, including speed, efficiency, and improved safety for maintenance personnel. Consequently, aerial work robots for live-line operations, which combine machine vision with drone technology, have attracted increasing attention from researchers both domestically and internationally, demonstrating their significant research and application potential.

[0003] Aerial robotic systems, such as drones, that perform automated aerial work tasks typically rely on accurate power line identification and positioning information. However, the actual environment surrounding power lines and towers is complex and highly variable. Traditional image recognition methods for power lines have significantly reduced accuracy when identifying power lines against complex backgrounds and may not function properly due to sunlight. Furthermore, the strong electromagnetic environment near high-voltage power lines can cause significant electromagnetic and communication interference to onboard sensors such as IMUs, GPS, and magnetic compasses, making their positioning accuracy insufficient to meet the requirements of actual automated operations. In particular, for tasks requiring high precision, drones and other aerial robotic systems must reach precise locations close to power lines for tasks such as replacing components and clearing debris, while maintaining a safe distance from the power lines to prevent accidents. Even when manually operating drones to perform semi-autonomous live-line operations, untrained operators often struggle to maintain control of the drone's posture. Therefore, whether in automated or semi-automated operations, inaccurate identification of power lines and low relative positioning accuracy can lead to mission failure or even damage to power lines or the drone's own crash. To prevent these situations and ensure the safety and efficiency of aerial work, it's crucial to research autonomous power line identification and positioning methods for aerial work platforms using their own sensors during live-line operations.

[0004] Existing related research, such as patent application CN201811502886, proposes a transmission line recognition method based on a grayscale image edge mapping parameter-free algorithm. However, this method suffers from two problems: First, a monocular image can only identify transmission lines, but cannot determine their relative position to itself without relying on other sensors and methods. Second, the complex background environment makes it very easy to misidentify. For example, in a natural environment, the algorithm can easily misidentify the edge features of the horizon as a transmission line, reducing recognition accuracy. Patent application CN202110440584 proposes a transmission line target detection method based on an improved number of weighted deconvolution layers. This method does not fully consider the impact of natural lighting on recognition. In natural conditions in the wild, the camera will inevitably be continuously interfered with by sunlight and its reflections. When facing the sun or facing strong background reflections, the detection and recognition of transmission lines will be seriously affected. Patent applications CN202110498974 and CN202110672117 propose methods for identifying power lines using deep learning and neural networks. However, these methods require high platform computing power and consume more power, which is detrimental to the drone's endurance and difficult to deploy on some low-computing drones. Patent application CN202010068432 proposes a method for identifying and locating power lines that combines a binocular camera and a depth camera. This method requires the drone to be flown to a very small distance of 0.5-1m from the power line in order to operate normally, and its operating conditions are very limited. Furthermore, this method uses multiple sensors, increasing its cost. During the recognition process, traditional color image visual recognition is still strongly interfered with by sunlight, and the depth image in this method is only used for filtering, which fails to fully utilize the advantage of depth images that they are not affected by complex backgrounds and lighting at long distances. In summary, the problems of power line recognition being susceptible to interference and positioning being poor in existing methods urgently need to be addressed. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem that the transmission line identification effect is easily interfered with and the positioning effect is poor in the existing methods. The present invention provides a transmission line identification and positioning method based on depth vision.

[0006] A power line recognition method based on depth vision includes the following steps:

[0007] Step 1: Use a depth camera to collect a depth image of the power line and transmit it to an onboard processor; the onboard processor performs depth filtering on the input depth image to obtain a filtered depth image;

[0008] Step 2: Map the filtered depth image into a depth grayscale image using an inverse normalization algorithm;

[0009] Step 3: Perform median filtering and Gaussian filtering on the mapped depth grayscale image to obtain a filtered depth grayscale image;

[0010] Step 4: Perform edge detection on the filtered depth grayscale image using an edge detection algorithm to obtain a binary image of the transmission line outline;

[0011] Step 5: Perform hough feature fitting in the binary image of the transmission line contour to obtain the starting points and end points of multiple straight lines in the binary image;

[0012] Step 6. Map the starting point and end point of each straight line in the binary image back to the depth grayscale image filtered in step 3, calculate the distance between each straight line and the depth camera, and filter the straight lines in the depth grayscale image mapped back to step 3 according to the distance between each straight line and the depth camera and the spatial position between each straight line. Take the two straight lines that meet the straight line screening conditions as a group; then fit the two straight lines in each group as the upper edge line and the lower edge line of the same transmission line, and obtain a transmission line corresponding to the two straight lines in each group, thereby realizing the identification of the transmission line.

[0013] Preferably, the specific implementation of step 1, obtaining the filtered depth image is:

[0014] Compare the depth value depth(i,j) of each pixel in the incoming depth image with the preset background depth P depth_max Compare and perform depth filtering to obtain a filtered depth image;

[0015] The expression for depth filtering of the depth value depth(i,j) of each pixel is:

[0016]

[0017] Among them, depth(i,j) is the depth value of the pixel point with the horizontal coordinate i and the vertical coordinate j in the pixel plane in the image coordinate system.

[0018] Preferably, in step 2, mapping the filtered depth image into a depth grayscale image by the inverse normalization algorithm is implemented as follows:

[0019] P(i,j)=255-ε×depth(i,j);

[0020] Among them, P(i,j) is the grayscale value after depth(i,j) mapping, and ε is the normalization operator.

[0021] Preferably, ε=255 / P depth_max .

[0022] Preferably, the fourth step of performing edge detection on the filtered depth grayscale image using an edge detection algorithm to obtain a binary image of the transmission line contour is implemented as follows:

[0023] First, the image gradient of each pixel of the filtered depth grayscale image is calculated by the Canny operator, and then the image gradient of each pixel is used to perform a non-maximum suppression operation on the filtered depth grayscale image to determine whether each pixel in the filtered depth grayscale image is an edge pixel or a non-edge pixel. Then, all edge pixels in the filtered depth grayscale image are mapped to 1 and all non-edge pixels are mapped to 0, thereby obtaining a binary image of the transmission line contour.

[0024] Preferably, in step 4, determining whether each pixel in the depth grayscale image is an edge pixel or a non-edge pixel is implemented as follows:

[0025] The image gradient of each pixel of the filtered deep grayscale image is compared with the preset image gradient threshold. If the image gradient of the pixel is greater than or equal to the preset image gradient threshold, the pixel is determined to be an edge pixel; if the gradient of the pixel is less than the preset image gradient threshold, the pixel is determined to be a non-edge pixel.

[0026] Preferably, in step 6, the starting point and end point of each straight line in the binary image are mapped back to the depth grayscale image filtered in step 3, and the distance between each straight line and the depth camera is calculated as follows:

[0027] The starting point and end point of each straight line in the binary image are taken as the diagonal lines of a parallelogram, and the depth values of all pixels in the area corresponding to the parallelogram in the binary image are selected, and the depth values of the pixels in the area are greater than the preset background depth P. depth_max The pixels in the area are removed; the depth values of the remaining pixels in the area are sorted from high to low, and then some pixels are discarded according to a preset ratio in the direction of high to low sorting, and outlier pixels in the direction of high to low sorting are removed; finally, the average depth value of the remaining pixels is taken as the distance between the line and the depth camera.

[0028] Preferably, in step 6, the implementation method of grouping two straight lines that meet the straight line screening condition is as follows:

[0029] When the distance difference between any two straight lines and the depth camera satisfies the preset depth error threshold range h1 and the spatial height difference between the any two straight lines satisfies the preset height error threshold range h2, the corresponding two straight lines are regarded as a group.

[0030] The positioning method implemented by the power line recognition method based on depth vision is to calculate the coordinates of the midpoint of each power line in the depth camera coordinate system, thereby completing the positioning of the power line. The specific process is as follows:

[0031] First, take the average of the distances between the two straight lines corresponding to each transmission line and the depth camera, and use the average distance as the depth depth′ of the midpoint of the transmission line from the origin of the depth camera coordinates;

[0032] Secondly, the depth depth′ and the depth camera parameters are used to perform coordinate transformation to solve the coordinates of the midpoint of the transmission line in the depth camera coordinate system.

[0033] Preferably, the coordinate transformation is performed using the depth depth′ and the depth camera parameters to solve the coordinates of the midpoint of the transmission line in the depth camera coordinate system:

[0034]

[0035] Among them, X, Y and Z are the coordinates of the x-axis, y-axis and z-axis of the midpoint of the transmission line in the depth camera coordinate system respectively; and the optical axis direction of the depth camera is the z-axis direction, the horizontal right direction of the depth camera is the x-axis direction, and the direction perpendicular to the plane where the z-axis and x-axis are located and pointing downward is the y-axis direction; u is the horizontal coordinate of the pixel at the midpoint of the transmission line in the pixel plane in the image coordinate system, v is the vertical coordinate of the pixel at the midpoint of the transmission line in the pixel plane in the image coordinate system, and f x is the focal length of the camera in the x-axis direction in the camera body coordinate system, f y is the focal length of the camera in the y-axis direction in the camera coordinate system, c x is the horizontal coordinate of the corresponding pixel point in the pixel plane mapped to the image coordinate system by the camera principal point, c y The vertical coordinate of the pixel point corresponding to the camera principal point mapped to the pixel plane in the image coordinate system.

[0036] The beneficial effects brought by the present invention are:

[0037] First, in the solution of the present invention, in terms of power line identification, this solution maps the depth image output by the depth camera into a depth grayscale image for power line identification, effectively avoiding the influence of complex background environment and lighting on identification, and can clearly observe the shape characteristics of the power line, thereby improving the stability of identification. The edge detection and feature matching method is adopted, combining edge features and depth information to more accurately identify power lines. In terms of power line positioning, taking advantage of the higher accuracy of the depth camera output value, the average value method is adopted to further reduce measurement errors and improve positioning accuracy.

[0038] Second, compared to existing solutions that combine binocular cameras and depth cameras, this solution reduces the number of sensors required. A single depth camera is all that's needed, effectively saving costs. Compared to methods that use binocular cameras to identify power lines and then use stereo vision for positioning, this method uses depth images mapped to grayscale images, which are more stable, highlight power line features, and produce smaller distance errors measured by the depth camera. Therefore, this solution improves power line identification and positioning accuracy while reducing costs.

[0039] This depth-vision-based power line identification and positioning method is specifically designed for use with aerial work platforms (AWPs), such as drones, that perform automated live-line maintenance and inspection work. It fully considers the complex background environment of high voltage and strong magnetic fields, as well as persistent sunlight interference, that AWPs face when operating autonomously in real-world environments. This method addresses practical challenges faced by existing autonomous power line identification and positioning technologies. It closely matches the actual conditions in which this system performs aerial missions, demonstrating broad applicability.

[0040] This method leverages the advantages of depth vision, which is immune to interference from strong magnetic fields and illuminated color images, to accurately identify and locate power transmission lines solely through depth vision. This significantly enhances the autonomous operation capabilities of unmanned aerial work platforms, reduces installation and computing costs, and enables their widespread application across various models. It can significantly improve the safety and efficiency of autonomous operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 1 is a schematic diagram of a flow chart of the power transmission line recognition method based on depth vision according to the present invention;

[0042] Figure 2 is a schematic diagram of a flow chart of the power transmission line positioning method based on depth vision according to the present invention;

[0043] Figure 3 This is a working diagram of the power line recognition method based on deep vision;

[0044] Figure 4 : is the experimental effect diagram of the positioning experiment performed by the method of the present invention; wherein, Figure 4 a is the original depth image captured by the depth camera; Figure 4 b is the depth grayscale image obtained after inverse normalization filtering; Figure 4 c is the binarized image after edge detection; Figure 4 d is the position of the identified transmission line in the real world. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0046] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0047] Example 1

[0048] See also Figure 1 The first embodiment of the present invention describes a method for identifying power lines based on depth vision, which includes the following steps:

[0049] Step 1: Use the depth camera to collect the depth image of the power line and transmit it to the onboard processor; the onboard processor performs depth filtering on the input depth image to obtain the filtered depth image; specifically, the depth value depth(i,j) of each pixel in the input depth image is compared with the preset background depth P depth_max Compare and perform depth filtering to obtain a filtered depth image; the expression for depth filtering of the depth value depth(i,j) of each pixel is:

[0050]

[0051] Where depth(i,j) is the depth value of the pixel at the horizontal coordinate i and the vertical coordinate j in the pixel plane in the image coordinate system;

[0052] When performing depth filtering, the algorithm normalizes all pixel depth values outside the set threshold range to the background depth to filter out most of the background noise and highlight the pixels corresponding to the transmission line;

[0053] Step 2: Map the filtered depth image into a depth grayscale image using the inverse normalization algorithm; specifically,

[0054] P(i,j)=255-ε×depth(i,j);

[0055] Among them, P(i,j) is the grayscale value after depth(i,j) mapping, ε=255 / P depth_max is the normalization operator; usually ε=255 / P depth_max The background depth threshold is ε = 255 / P depth_maxIt is determined that in the algorithm, through the normalization operation, the depth image is linearly mapped to the grayscale value according to the depth value; further through the reverse operation, the pixels with smaller depth in the depth image are mapped to pixels with larger grayscale value and higher brightness, so as to further highlight the characteristics of the nearby location area where the transmission line is located, which is convenient for the subsequent detection and recognition algorithm.

[0056] Step 3: Perform median filtering and Gaussian filtering on the mapped depth grayscale image to obtain a filtered depth grayscale image. In this step, the measurement noise generated by the camera is further filtered by median filtering and Gaussian filtering to obtain a filtered depth grayscale image.

[0057] Step 4. Perform edge detection on the filtered depth grayscale image through an edge detection algorithm to obtain a binary image of the transmission line outline. Specifically, first calculate the image gradient of each pixel of the filtered depth grayscale image through the Canny operator, and then use the image gradient of each pixel to perform a non-maximum suppression operation on the filtered depth grayscale image to determine whether each pixel in the filtered depth grayscale image is an edge pixel or a non-edge pixel. Then, all edge pixels in the filtered depth grayscale image are mapped to 1 and all non-edge pixels are mapped to 0, thereby obtaining a binary image of the transmission line outline.

[0058] The method for determining whether each pixel in the depth grayscale image is an edge pixel or a non-edge pixel is as follows:

[0059] Compare the image gradient of each pixel of the filtered deep grayscale image with a preset image gradient threshold. If the image gradient of the pixel is greater than or equal to the preset image gradient threshold, the pixel is determined to be an edge pixel; if the gradient of the pixel is less than the preset image gradient threshold, the pixel is determined to be a non-edge pixel.

[0060] Step 5: Perform hough feature fitting in the binary image of the transmission line contour to obtain the starting points and end points of multiple straight lines in the binary image;

[0061] Step 6. Map the starting point and end point of each straight line in the binary image back to the depth grayscale image filtered in step 3, calculate the distance between each straight line and the depth camera, and filter the straight lines in the depth grayscale image filtered in step 3 according to the distance between each straight line and the depth camera and the spatial position between the straight lines. Take the two straight lines that meet the straight line screening conditions as a group; then fit the two straight lines in each group as the upper edge line and the lower edge line of the same transmission line, and obtain a transmission line corresponding to the two straight lines in each group, thereby realizing the recognition of the transmission line;

[0062] Specifically, in step 6, the starting point and end point of each line in the binary image are mapped back to the depth grayscale image filtered in step 3, and the distance between each line and the depth camera is calculated as follows:

[0063] The starting point and end point of each straight line in the binary image are taken as the diagonal lines of a parallelogram, and the depth values of all pixels in the area corresponding to the parallelogram in the binary image are selected, and the depth values of the pixels in the area are greater than the preset background depth P. depth_max The pixels in the area are removed; the depth values of the remaining pixels in the area are sorted from high to low, and then some pixels are discarded according to a preset ratio in the direction of high to low sorting, and outlier pixels in the direction of high to low sorting are removed; finally, the average depth value of the remaining pixels is taken as the distance between the line and the depth camera.

[0064] During the calculation of this sixth step, the depth values of all pixels within the parallelogram containing the edge are first selected, removing the background depth. The remaining depth values are then sorted, and the largest portion of them is discarded in a certain proportion to prevent overlapping transmission lines from affecting the actual depth calculation. Outliers whose depth values differ significantly from the majority are also removed. Finally, the average of the remaining depth values is used as the calculated depth value for the line, effectively improving the accuracy of the distance between the line and the depth camera.

[0065] Specifically, in step 6, the implementation method of grouping two straight lines that meet the straight line screening conditions is as follows:

[0066] When the distance difference between any two lines and the depth camera meets the preset depth error threshold range h1, and the spatial height difference between the two lines meets the preset height error threshold range h2, the corresponding two lines are considered a group. In application, the value range of h1 and h2 can be determined according to the application conditions in the actual application scenario.

[0067] Figure 3 In the figure, after processing the depth image of the transmission line captured by the depth camera, a binary image is obtained, and the starting point and end point of the two straight lines in the binary image are A and B respectively. The midpoint of the transmission line corresponding to the two straight lines in the pixel plane in the image coordinate system is N(u,v), and the coordinates of the midpoint of the transmission line in the depth camera coordinate system are M(X,Y,depth′), and the depth depth′ of the transmission line from the depth camera.

[0068] Example 2:

[0069] See also Figure 2This embodiment 2 describes a positioning method implemented by the power line recognition method based on depth vision. The positioning method calculates the coordinates of the midpoint of each power line in the depth camera coordinate system to complete the positioning of the power line. The specific process is as follows:

[0070] First, take the average of the distances between the two straight lines corresponding to each transmission line and the depth camera, and use the average distance as the depth depth′ of the midpoint of the transmission line from the origin of the depth camera coordinates;

[0071] Secondly, the coordinate transformation is performed using the depth 'depth' and the depth camera parameters to solve the coordinates of the midpoint of the transmission line in the depth camera coordinate system. Specifically, the coordinate transformation is performed using the depth 'depth' and the depth camera parameters to solve the coordinates of the midpoint of the transmission line in the depth camera coordinate system:

[0072]

[0073] Among them, X, Y and Z are the coordinates of the x-axis, y-axis and z-axis of the midpoint of the transmission line in the depth camera coordinate system respectively; and the optical axis direction of the depth camera is the z-axis direction, the horizontal right direction of the depth camera is the x-axis direction, and the direction perpendicular to the plane where the z-axis and x-axis are located and pointing downward is the y-axis direction; u is the horizontal coordinate of the pixel at the midpoint of the transmission line in the pixel plane in the image coordinate system, v is the vertical coordinate of the pixel at the midpoint of the transmission line in the pixel plane in the image coordinate system, and f x is the focal length of the camera in the x-axis direction in the camera body coordinate system, f y is the focal length of the camera in the y-axis direction in the camera coordinate system, c x is the horizontal coordinate of the corresponding pixel point in the pixel plane mapped to the image coordinate system by the camera principal point, c y The ordinate of the pixel corresponding to the camera's principal point in the pixel plane in the image coordinate system is mapped to the principal point of the camera. The principal point of the camera's principal point is the center point of the camera's field of view, which can also be understood as the center point of the image.

[0074] in, is the camera intrinsic parameter matrix.

[0075] When the camera is installed normally, the front surface of the camera should be vertically parallel to the front plane of the mounting platform, and the bottom surface of the camera should be parallel to the plane of the mounting platform. If the camera and the mounting platform are installed at a certain angle, the conversion relationship between the camera coordinate system and the drone coordinate system can be established by coordinate transformation.

[0076] In specific applications, one can first select one or more transmission lines of greatest interest based on the specific application. After identifying the corresponding transmission line, coordinate transformation is used to determine the relative position of the transmission line and the camera. When the camera is properly mounted, the front surface of the camera is generally vertically parallel to the front plane of the mounting platform, and the bottom surface of the camera is parallel to the plane of the mounting platform. If the camera is mounted at an angle to the drone, coordinate transformation can be used to establish a conversion relationship between the camera coordinate system and the drone coordinate system. The mounting platform can be a drone.

[0077] After the positioning method of the present invention outputs the coordinates of the midpoint of each transmission line in the depth camera coordinate system, it provides a stable and reliable source of relative position data for upper-level aerial work tasks such as visual servoing. The depth information between the transmission line and the camera obtained and the actual distance between the rotorcraft manipulator system and the transmission line can be used to establish a proportional control mapping, and the position information of the transmission line in the pixel coordinate system and the actual height, left and right distance between the rotorcraft manipulator system and the transmission line can be used to establish a proportional control mapping, thereby realizing hybrid visual servo control of the rotorcraft manipulator aerial work tasks. Specifically, assuming that the current actual position of the rotorcraft manipulator system is (x′, y′, z′), and the hybrid visual position information of the transmission line obtained by machine vision technology is (X, Y, depth′), we can design a hybrid visual servo proportional guidance law as follows:

[0078]

[0079] Among them, k depth ,k x ,k y Respectively represent the proportional guidance parameters in the z-axis, x-axis and y-axis directions in the depth camera coordinate system, diag(k depth ,k u ,k v ) represents a diagonal matrix consisting of proportional guidance parameters in three directions; image height and image width Represents the camera image height and width respectively.

[0080] Based on the hybrid visual servo proportional guidance law designed above, the desired signal (x d ,y d ,z d), which is then fed into the position and attitude loops' underlying controllers for further computation, resulting in a hybrid visual servo control law based on hybrid visual position information feedback. It should be noted that the method of the present invention aims to provide effective hybrid visual position information data, and its applicability includes, but is not limited to, systems such as drones and robots, and can be combined with common underlying control methods such as PID.

[0081] At this point, the detailed description of the specific implementation scheme of the power line identification and positioning method based on depth vision proposed in the present invention for hybrid visual servo position control of a rotorcraft performing aerial live-line work tasks in a real environment has been completed. The effect diagram of the power line identification and positioning method based on depth vision in an experimental environment can be seen in Figure 4 .

[0082] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention. It should be understood that many modifications may be made to the illustrative embodiments, and that other arrangements may be devised, without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in ways other than those described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be employed in conjunction with other described embodiments.

Claims

1. A power line recognition method based on depth vision, characterized in that: The method comprises the following steps: Step 1: Use a depth camera to collect a depth image of the power line and transmit it to an onboard processor; the onboard processor performs depth filtering on the input depth image to obtain a filtered depth image; Step 2: Map the filtered depth image into a depth grayscale image using an inverse normalization algorithm; Step 3: Perform median filtering and Gaussian filtering on the mapped depth grayscale image to obtain a filtered depth grayscale image; Step 4: Perform edge detection on the filtered depth grayscale image using an edge detection algorithm to obtain a binary image of the transmission line outline; specifically, the image gradient of each pixel of the filtered depth grayscale image is calculated using the Canny operator, and then the image gradient of each pixel is used to perform a non-maximum suppression operation on the filtered depth grayscale image to determine whether each pixel in the filtered depth grayscale image is an edge pixel or a non-edge pixel, and then all edge pixels in the filtered depth grayscale image are mapped to 1 and all non-edge pixels are mapped to 0, thereby obtaining a binary image of the transmission line outline; Step 5: Perform hough feature fitting in the binary image of the transmission line contour to obtain the starting points and end points of multiple straight lines in the binary image; Step 6. Map the starting point and end point of each straight line in the binary image back to the depth grayscale image filtered in step 3, calculate the distance between each straight line and the depth camera, and filter the straight lines in the depth grayscale image filtered in step 3 according to the distance between each straight line and the depth camera and the spatial position between the straight lines. Take the two straight lines that meet the straight line screening conditions as a group; then fit the two straight lines in each group as the upper edge line and the lower edge line of the same transmission line, and obtain a transmission line corresponding to the two straight lines in each group, thereby realizing the recognition of the transmission line; The method for calculating the distance between each line and the depth camera is as follows: The starting point and end point of each straight line in the binary image are taken as the diagonal lines of a parallelogram, and the depth values of all pixels in the area corresponding to the parallelogram in the binary image are selected, and the depth values of the pixels in the area are greater than the preset background depth P. depth_max Pixel culling; After the depth values of the remaining pixels in the area are sorted from high to low, some pixels are discarded according to a preset ratio in the high-to-low sorting direction, and outlier pixels in the high-to-low sorting direction are removed; finally, the average depth value of the remaining pixels is used as the distance between the line and the depth camera.

2. The power line recognition method based on depth vision according to claim 1, characterized in that: Step 1: The specific implementation method of obtaining the filtered depth image is: Compare the depth value depth(i,j) of each pixel in the incoming depth image with the preset background depth P depth_max Compare and perform depth filtering to obtain a filtered depth image; The expression for depth filtering of the depth value depth(i,j) of each pixel is: Among them, depth(i,j) is the depth value of the pixel point with the horizontal coordinate i and the vertical coordinate j in the pixel plane in the image coordinate system.

3. The power line recognition method based on depth vision according to claim 1, characterized in that: In step 2, the implementation method of mapping the filtered depth image into a depth grayscale image by the inverse normalization algorithm is: P(i,j)=255-ε×depth(i,j); Among them, P(i,j) is the grayscale value after depth(i,j) mapping, and ε is the normalization operator.

4. The power line recognition method based on depth vision according to claim 3, characterized in that: e=255 / P depth_max 。 5. The power line recognition method based on depth vision according to claim 1, characterized in that: In step 4, the implementation method of determining whether each pixel in the depth grayscale image is an edge pixel or a non-edge pixel is as follows: The image gradient of each pixel of the filtered deep grayscale image is compared with the preset image gradient threshold. If the image gradient of the pixel is greater than or equal to the preset image gradient threshold, the pixel is determined to be an edge pixel; if the gradient of the pixel is less than the preset image gradient threshold, the pixel is determined to be a non-edge pixel.

6. The power transmission line identification method based on depth vision according to claim 1, characterized in that: In step 6, the two lines that meet the line screening conditions are grouped together as follows: When the distance difference between any two straight lines and the depth camera satisfies the preset depth error threshold range h1 and the spatial height difference between the any two straight lines satisfies the preset height error threshold range h2, the corresponding two straight lines are regarded as a group.

7. A positioning method implemented by using the power line recognition method based on depth vision according to any one of claims 1 to 6, characterized in that: This positioning method calculates the coordinates of the midpoint of each transmission line in the depth camera coordinate system to complete the positioning of the transmission line. The specific process is as follows: First, take the average of the distances between the two straight lines corresponding to each transmission line and the depth camera, and use the average distance as the depth depth′ of the midpoint of the transmission line from the origin of the depth camera coordinates; Secondly, the depth depth′ and the depth camera parameters are used to perform coordinate transformation to solve the coordinates of the midpoint of the transmission line in the depth camera coordinate system.

8. The positioning method implemented by the power line recognition method based on depth vision according to claim 7 is characterized in that: The coordinate transformation is performed using depth′ and the depth camera parameters to solve the coordinates of the midpoint of the transmission line in the depth camera coordinate system: Among them, X, Y and Z are the coordinates of the x-axis, y-axis and z-axis of the midpoint of the transmission line in the depth camera coordinate system respectively; and the optical axis direction of the depth camera is the z-axis direction, the horizontal right direction of the depth camera is the x-axis direction, and the direction perpendicular to the plane where the z-axis and x-axis are located and pointing downward is the y-axis direction; u is the horizontal coordinate of the pixel at the midpoint of the transmission line in the pixel plane in the image coordinate system, v is the vertical coordinate of the pixel at the midpoint of the transmission line in the pixel plane in the image coordinate system, and f x is the focal length of the camera in the x-axis direction in the camera body coordinate system, f y is the focal length of the camera in the y-axis direction in the camera coordinate system, c x is the horizontal coordinate of the corresponding pixel point in the pixel plane mapped to the image coordinate system by the camera principal point, c y The vertical coordinate of the pixel point corresponding to the camera principal point mapped to the pixel plane in the image coordinate system.

Images