Live-line robot aerial bypass automatic operation method

Abstract

The application provides an aerial bypass automatic operation method of a live working robot, which is based on the nesting relationship and rectangular features of a two-dimensional code positioning frame, three positioning frames of a two-dimensional code are screened out through image preprocessing, contour extraction, nested layer detection, rectangular judgment and other steps, then the two-dimensional code is directionally corrected, the extracted two-dimensional points are mapped to three-dimensional point cloud space points for analysis and processing, three-dimensional space center point coordinates and a plane normal vector are obtained, finally, the x-axis and y-axis in the target space are fitted to realize accurate identification of the target. According to the offset processing of the fixed mechanical structure position of the target two-dimensional code and the aerial clamp dismounting port, the arm driving head can be accurately inserted into the dismounting port for operation.

Description

Technical Field

[0001] This invention relates to the field of live-line robots, specifically a method for automatic overhead bypass operation of a live-line working robot. Background Technology

[0002] In many applications, such as industrial automation, robot vision, and augmented reality, accurate target identification and localization are essential. QR codes, due to their large information storage capacity and ease of recognition, are frequently used as targets. Existing QR code recognition technologies primarily focus on decoding the QR code information, and their accuracy in accurately identifying and locating QR codes in complex environments remains limited. Some traditional recognition methods may be affected by image noise, lighting variations, and QR code deformation, leading to decreased recognition accuracy. In practical applications, traditional recognition methods lack robustness to image noise and interference, easily misidentifying noise or other irrelevant contours as QR code bounding boxes, resulting in recognition errors. Furthermore, the lack of an effective bounding box filtering mechanism makes it difficult to accurately extract the true bounding box in complex images, affecting the QR code's positioning accuracy. Traditional methods for QR code orientation correction and spatial coordinate axis fitting are not accurate enough to meet the demands of high-precision target recognition. Summary of the Invention

[0003] To address the problems of existing technologies, this invention provides an automatic bypass handling method for live-line working robots, enabling efficient and precise disassembly of aerial bypasses in a live environment. Utilizing super-resolution 3D LiDAR for global modeling and coarse localization improves the accuracy and reliability of positioning. Through multi-step image preprocessing, precise contour extraction and filtering, QR code orientation correction, and 3D coordinate calculation, accurate target identification and localization are achieved. Finally, the end effector tool accurately inserts into the disassembly port to complete the bypass disassembly operation, significantly improving work efficiency and reducing the risks associated with manual operation.

[0004] This invention provides an automatic bypass operation method for live-line working robots, including positioning, automatic identification, and disassembly, the specific process of which is as follows:

[0005] 1) During the positioning process, perform global modeling of the 3D LiDAR, select points for calculation and coarse positioning, adjust the camera on the live-line working robot and collect data;

[0006] 2) Automatic identification process: A target multimodal information identification and localization method is employed, specifically including:

[0007] 2.1) Image and point cloud data acquisition: Visible light images and point cloud data are acquired by an area array laser on a robotic arm; the area array laser camera emits laser beams in a specific mode to scan the target area, while the camera records the intensity and phase information of the reflected light, thereby generating high-quality visible light images and point cloud data;

[0008] 2.2) Preprocess the image;

[0009] 2.3) Extract the image contours and filter the positioning boxes;

[0010] 2.4) Correct the orientation of the QR code to ensure it is in a standard horizontal or vertical position;

[0011] 2.5) 3D coordinate calculation and offset processing: Extract the point cloud data on the target QR code from the collected point cloud data, analyze and calculate the 3D coordinate position of the center point of the QR code frame, and add a fixed offset to the center point of the bypass disassembly horn mouth based on the distance between the target and the fixed structural component between the target and the disassembly horn mouth.

[0012] 2.6) Plane normal vector and coordinate axis fitting: The RANSAC plane fitting algorithm is used to obtain the normal vector of the QR code plane;

[0013] 3) Precision operation and bypass dismantling: The control arm end effector tool moves precisely to the position of the bypass dismantling horn opening according to the calculated position and attitude information; the arm control drive winch starts to release the line, gradually dismantling the bypass line from the hanging clamp, completing the aerial bypass dismantling operation; the position and status of the robotic arm are monitored in real time throughout the operation.

[0014] Further improvements include the following steps in the positioning process described in step 1):

[0015] 1.1) Global Modeling of 3D LiDAR: A super-resolution 3D LiDAR is used to scan the live-line operation scene of the aerial power distribution network to obtain point cloud data of the entire operation area; the LiDAR emits laser beams at high frequency to conduct all-round detection of the surrounding environment and obtain dense point cloud information; by preprocessing the point cloud data, a rough global model of the aerial operation scene is performed.

[0016] 1.2) Point selection calculation and coarse positioning: Using global point cloud data, a specific algorithm is used to perform point selection calculation, focusing on the feature points of the aerial wire clamp; through the analysis and matching of these feature points, the position of the aerial wire clamp is calculated; the control unit issues instructions based on the calculation results to drive the robotic arm to move to the coarse positioning position;

[0017] 1.3) Adjust the camera and collect data: After the robotic arm reaches the coarse positioning position, adjust the area array laser camera on the robotic arm to align it with the target area for recognition; the area array laser camera starts working and simultaneously collects RGB images and 3D point cloud data.

[0018] Further improvements are made, and the image preprocessing process in step 2.2) specifically includes:

[0019] 2.21) Scaling: Scaling the input RGB image;

[0020] 2.22) Grayscale conversion: Converting a color image to a grayscale image to remove color information;

[0021] 2.23) Filtering: Set the convolution kernel, first perform sharpening filtering, which enhances the edges and details in the image and highlights the outline features of the QR code; then perform Gaussian blur processing on the grayscale image, which uses the Gaussian function to smooth the image.

[0022] 2.24) Morphological operations: Morphological operations are performed on the filtered image; first, the binary image is eroded using specific structuring elements to remove small noise and burrs; then, a dilation operation is performed to restore the main outline of the QR code; at the same time, a morphological gradient operation is performed to detect the edge features of the image and further highlight the QR code's bounding box.

[0023] 2.25) Adaptive threshold binarization: An adaptive method is used to determine the preprocessing parameters.

[0024] Further improvements, the morphological operation in step 2.24) specifically involves: first, performing an erosion operation on the binary image, using specific structuring elements to erode the image and remove small noise and burrs; then performing a dilation operation to restore the main outline of the QR code; simultaneously, performing a morphological gradient operation to detect the edge features of the image and further highlight the positioning box of the QR code.

[0025] Further improvements are made to the adaptive threshold binarization described in step 2.25, which specifically involves using an adaptive threshold algorithm during the binarization process. This algorithm automatically adjusts the binarization threshold based on the pixel distribution in the local area of ​​the image. For image areas with higher brightness, the binarization threshold is increased accordingly; for image areas with lower brightness, the binarization threshold is decreased.

[0026] Further improvements are made to the contour extraction and bounding box filtering method described in step 2.3), which is as follows:

[0027] 2.31) Contour detection and extraction: Use the findContour function in the OpenCV library to extract contours from the preprocessed image and obtain all contour information in the image;

[0028] 2.32) Positioning box filtering: Nesting layer detection. The nesting layer of each contour is counted by the returned vector hierarchy information. Each element in the vector hierarchy stores an array containing 4 integers, which represent the index numbers of the next contour, the previous contour, the parent contour, and the inner contour of the i-th contour, respectively. When the number of contour nesting is greater than or equal to 3, it is used as a candidate contour.

[0029] 2.33) Positioning box filtering: Rectangle judgment, calculate the perimeter and area of ​​each candidate contour, check whether the ratio between the square of the perimeter and the area is close to 16, and filter out positioning boxes that are close to 16.

[0030] 2.34) Final screening: There must be three layers of rectangles in the nested contours. Through these two screening steps, contours that do not meet the conditions are excluded, and the remaining ones are the three positioning boxes of the QR code; if the number of remaining contours is not three, the detection is judged as a failure, and the arm-mounted laser camera is adjusted to re-collect data for detection.

[0031] Further improvements are made to the QR code orientation correction process described in step 2.4): The minAreaRect function from the OpenCV library is used to find the smallest bounding rectangle for the selected positioning boxes, and the rotation angle of this rectangle is obtained. Since the three positioning boxes are basically located on the same plane, their rotation angles can be considered to be the same, i.e., this angle is the rotation angle of the QR code. The orientation of the QR code is corrected based on this rotation angle.

[0032] Further improvements are made to the RANSAC algorithm described in step 2.6), which uses random sampling and iteration to fit the best-fitting planar model from the point cloud data, thereby obtaining the normal vector of the plane. Based on the corrected rotation angle, the x-axis and y-axis point clouds on the plane are extracted, and line fitting is performed to obtain the x-axis and y-axis in the target space. Line fitting can employ the least squares method, minimizing the sum of squared distances from points to lines to obtain the optimal line parameters.

[0033] The beneficial effects of this invention are as follows:

[0034] 1. The method for automatic identification, positioning and dismantling of overhead bypasses in a dual-arm live-line working robot based on multimodal information fusion of the present invention achieves efficient and accurate dismantling of overhead bypasses in a live-line environment.

[0035] 2. Global modeling and coarse positioning are performed using super-resolution 3D LiDAR, which improves the accuracy and reliability of positioning;

[0036] 3. By performing multi-step preprocessing on the image, precise contour extraction and filtering, as well as QR code orientation correction and three-dimensional coordinate calculation, accurate target identification and positioning were achieved.

[0037] 4. The end effector of the robotic arm can accurately insert into the disassembly port to complete the bypass disassembly operation, which greatly improves the work efficiency and reduces the risk of manual operation. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is the overall flowchart of the automatic identification, positioning, and disassembly algorithm for aerial bypass of a dual-arm live-line working robot;

[0040] Figure 2 This is a flowchart of the target multimodal information identification and localization algorithm;

[0041] Figure 3 This is a flowchart of an image preprocessing algorithm;

[0042] Figure 4 This is a flowchart of the contour extraction and bounding box filtering algorithm;

[0043] Figure 5 This is an example diagram of the center point of the automatic target recognition algorithm for the dual-arm robot in actual operation of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Overall approach:

[0046] This invention, based on the nesting relationship and rectangular features of QR code positioning boxes, filters out three positioning boxes for the QR code through image preprocessing, contour extraction, nesting layer detection, and rectangle judgment. Then, the QR code is oriented and corrected. The extracted 2D points are mapped onto a 3D point cloud for analysis, obtaining the coordinates of the 3D center point and the plane normal vector. Finally, the x-axis and y-axis in the target space are fitted, achieving accurate target recognition. By offsetting the target QR code against the fixed mechanical structure of the aerial clamp disassembly port, the arm drive head can be precisely inserted into the disassembly port for operation.

[0047] Specific steps:

[0048] like Figure 1 The diagram shows the main flowchart of the automatic aerial bypass identification, positioning, and disassembly method for a dual-arm live-line working robot based on multimodal information fusion, as described in this invention. First, a coarse global model of the aerial working scene is created using a super-resolution 3D LiDAR. After point selection calculations using the global point cloud, the position of the aerial cable clamp is obtained. Then, the control unit issues commands to move the robotic arm to the coarse positioning position and adjusts the on-arm array camera to start taking pictures, acquiring RGB images and 3D point cloud data. Then, the process begins as follows... Figure 2 The target multimodal information recognition and localization method shown is used to obtain the three-dimensional coordinates of the center point, the plane normal vector, and the x and y axes. The control arm's end effector tool is then positioned to precisely identify the bypass disassembly port. The arm's control drive winch then releases the cable, completing the aerial bypass disassembly.

[0049] in, Figure 2 The specific steps of the target multimodal information identification and localization method shown are as follows:

[0050] 1) Acquire visible light images and point cloud data using an area-array laser on the arm;

[0051] 2) Preprocess the acquired image data;

[0052] 3) Use the findContour function in the OpenCV library to extract contours from the preprocessed image, obtain all contour information, and filter the QR code positioning boxes;

[0053] 4) Use the minAreaRect function in the OpenCV library to find the smallest bounding rectangle of the selected positioning box, obtain its rotation angle, and use it as the rotation angle of the QR code for orientation correction;

[0054] 5) Extract point cloud data from the target QR code on the collected point cloud data, analyze and calculate the three-dimensional coordinate position of the center point of the QR code frame, and add a fixed offset to the center point of the bypass disassembly horn mouth based on the distance between the target and the fixed structural component between the target and the disassembly horn mouth.

[0055] 6) Use RANSAC plane fitting to obtain the plane normal vector of the QR code, and extract the x-axis and y-axis point clouds on the plane according to the corrected rotation angle. Perform line fitting to obtain the x-axis and y-axis in the target space.

[0056] like Figure 3 As shown, the steps of the image preprocessing method in step 2) are as follows:

[0057] 21) Scaling: Appropriately scale the input RGB image to reduce computation and improve processing efficiency;

[0058] 22) Grayscale conversion: Converting a color image to a grayscale image removes color information and simplifies the representation of image data;

[0059] 23) Filtering: Set a 3x3 convolution kernel, first perform sharpening filtering to enhance the edges and details in the image and make the image clearer; then perform Gaussian blur processing on the grayscale image to reduce image noise and smooth the image.

[0060] 24) Morphological operations: First, perform erosion on the binary image to remove small noise and burrs, then perform dilation to restore the main outline of the QR code and make the positioning box clearer; morphological gradient operations detect image edge features.

[0061] 25) Adaptive Threshold Binarization: An adaptive method is used to determine preprocessing parameters. During binarization, an adaptive thresholding algorithm is used, with the threshold setting determined by the distribution of surrounding neighboring pixels. The binarization threshold for image regions with higher brightness is usually higher, while the binarization threshold for image regions with lower brightness will be correspondingly lower. Local image regions with different brightness, contrast, and texture will have corresponding local binarization thresholds, improving the accuracy of binarization.

[0062] like Figure 4 As shown, the steps of the contour extraction and bounding box filtering method in step 3) are as follows:

[0063] 31) Contour detection and extraction: Use the findContour function in the OpenCV library to obtain all contours in the image. These contours may include the bounding box of the QR code and other irrelevant contours.

[0064] 32) Box Filtering: Nesting Level Detection. The nesting level of each contour is counted using the returned hierarchy vector. Each element in the hierarchy vector stores an array of four integers, representing the index numbers of the i-th contour's preceding, preceding, parent, and nested contours. Since QR code boxes have a three-level nesting feature, and considering the influence of noise and other factors, contours with a nesting level greater than or equal to 3 are considered candidate contours.

[0065] 33) Positioning Box Filtering: Rectangle Judgment. Calculate the perimeter and area of ​​each candidate contour, and check if the ratio between the square of the perimeter and the area is close to 16. Assume 'a' is the side length of a square, the perimeter of the square is 4a, and the area is 'a'. 2 The square of the perimeter is 16a 2 The ratio of the square of the perimeter to the area is 16. This is used to determine whether the outline is approximately rectangular.

[0066] 34) There must be three layers of rectangles in the nested contours. Through these two steps of filtering, contours that do not meet the conditions are eliminated, and what remains are basically the three positioning boxes of the QR code. If the number of remaining contours is not three, the detection is judged as a failure, and the arm-mounted laser camera is adjusted to re-collect data for detection.

[0067] A specific implementation scenario of the present invention is as follows: Figure 5 As shown:

[0068] 1. Overview of Implementation Scenarios:

[0069] In the maintenance and repair of power systems, it is often necessary to dismantle overhead bypass cables. This invention presents a multimodal information fusion-based dual-arm live-line working robot method for automatic identification, positioning, and dismantling of overhead bypass cables, aiming to achieve safe, efficient, and precise dismantling of overhead bypass cables in live environments. The implementation process of this method is described in detail below with specific implementation scenarios.

[0070] 2. Specific implementation steps:

[0071] 21) Global Modeling and Coarse Localization

[0072] 211) Global Modeling of 3D LiDAR

[0073] A super-resolution 3D LiDAR was used to scan the overhead power distribution network live-line work scenario, acquiring point cloud data of the entire work area. The LiDAR emitted laser beams at a high frequency to conduct omnidirectional detection of the surrounding environment, obtaining dense point cloud information. By preprocessing the point cloud data, such as removing noise points and filtering, a rough global model of the overhead work scenario was created.

[0074] 212) Point selection calculation and coarse positioning

[0075] Using global point cloud data, a specific algorithm is employed to perform point selection calculations, focusing on the feature points of the aerial wire clamp. Through analysis and matching of these feature points, the position of the aerial wire clamp is accurately calculated. Based on the calculation results, the control unit issues commands to drive the robotic arm to move to the coarse positioning position.

[0076] 213) Adjust the camera and collect data

[0077] After the robotic arm reaches the coarse positioning position, the area-array laser camera on the robotic arm is adjusted to align with the target area for identification. The area-array laser camera then begins to operate, simultaneously acquiring RGB images and 3D point cloud data. The RGB images provide rich visual information, while the 3D point cloud data contains depth information of the target object, laying the foundation for subsequent precise identification and localization.

[0078] 22) Target multimodal information recognition and localization method

[0079] 221) Image and point cloud data acquisition

[0080] Visible light images and point cloud data are acquired using an area-array laser camera mounted on an arm. The area-array laser camera emits laser beams in a specific pattern to scan the target area, while simultaneously recording the intensity and phase information of the reflected light, thereby generating high-quality visible light images and point cloud data.

[0081] 222) Image Preprocessing

[0082] a) Scaling: Appropriately scaling the input RGB image can reduce the computational load of subsequent processing without losing too much image detail, thus improving processing efficiency.

[0083] b) Grayscale conversion: Converting a color image to a grayscale image to remove color information. In this embodiment, a weighted average method is used for grayscale processing, that is, the three RGB channels are weighted and averaged according to the human eye's sensitivity to different colors to obtain the grayscale value.

[0084] c) Filtering: Set a 3x3 convolution kernel and perform a sharpening filter first. Sharpening filters enhance edges and details in the image, making it clearer and highlighting the outline features of the QR code. Then, apply a Gaussian blur to the grayscale image. Gaussian blur uses the Gaussian function to smooth the image, reducing noise and making it smoother.

[0085] d) Morphological Operations: Morphological operations are performed on the filtered image. First, an erosion operation is performed on the binary image, using a specific structuring element to remove small noise and burrs. Then, a dilation operation is performed to restore the main outline of the QR code, making the bounding box clearer. Simultaneously, a morphological gradient operation is performed to detect edge features of the image, further highlighting the QR code's bounding box.

[0086] e) Adaptive Threshold Binarization: An adaptive method is used to determine the preprocessing parameters. During binarization, an adaptive thresholding algorithm is employed. This algorithm automatically adjusts the binarization threshold based on the pixel distribution of local image regions. For image regions with higher brightness, the binarization threshold is increased accordingly; for image regions with lower brightness, the binarization threshold is decreased. This ensures that local image regions with different brightness, contrast, and texture receive appropriate binarization processing, improving the accuracy of binarization.

[0087] 223) Contour extraction and bounding box filtering

[0088] a) Contour detection and extraction: The findContour function from the OpenCV library is used to extract contours from the preprocessed image, obtaining all contour information in the image. These contours may include the bounding box of the QR code and other irrelevant contours.

[0089] b) Box Filtering: Nesting Level Detection. The nesting level of each contour is counted using the returned hierarchy vector. Each element in the hierarchy vector stores an array of four integers, representing the index numbers of the i-th contour's preceding, preceding, parent, and nested contours. Since QR code boxes have a three-level nesting characteristic, and considering the influence of noise and other factors, contours with a nesting level greater than or equal to 3 are considered candidate contours.

[0090] c) Positioning box filtering: Rectangle judgment. Calculate the perimeter and area of ​​each candidate contour, and check if the ratio between the square of the perimeter and the area is close to 16. Assume 'a' is the side length of a square, the perimeter of the square is 4a, and the area is a. 2 The square of the perimeter is 16a 2 The ratio of the square of the perimeter to the area is 16. This is how to determine whether the outline is approximately rectangular.

[0091] d) Final filtering: There must be three layers of rectangles in the nested contours. Through these two filtering steps, contours that do not meet the conditions are eliminated, and what remains are basically the three positioning boxes of the QR code. If the number of remaining contours is not three, the detection is considered a failure, and the arm-mounted laser camera is adjusted to re-collect data for detection.

[0092] 224) QR code orientation correction

[0093] The selected bounding boxes are used with the `minAreaRect` function from the OpenCV library to find the smallest bounding rectangle, and the rotation angle of this rectangle is also obtained. Since the three bounding boxes are basically located on the same plane, their rotation angles can be considered to be the same, i.e., this angle is the rotation angle of the QR code. Based on this rotation angle, the orientation of the QR code is corrected so that the QR code is in a standard horizontal or vertical position.

[0094] 225) Three-dimensional coordinate calculation and offset processing

[0095] Point cloud data from the target QR code is extracted from the collected point cloud data. A specific algorithm is used to analyze and calculate the 3D coordinates of the center point of the QR code frame. Based on the distance between the target and the fixed structural component at the bypass disassembly horn opening, a fixed offset is added to the center point of the bypass disassembly horn opening. For example, by pre-measuring a fixed distance d between the target and the disassembly horn opening, an offset d is added along a specific direction to obtain the 3D coordinates of the center point of the bypass disassembly horn opening, based on the calculated 3D coordinates of the QR code center point.

[0096] 226) Fitting plane normal vectors and coordinate axes

[0097] The RANSAC plane fitting algorithm is used to obtain the normal vector of the QR code plane. The RANSAC algorithm fits the best-fitting plane model from point cloud data through random sampling and iteration, thus obtaining the plane's normal vector. Based on the corrected rotation angle, the x-axis and y-axis point clouds on the plane are extracted, and line fitting is performed to obtain the x-axis and y-axis in the target space. Line fitting can use the least squares method, minimizing the sum of squared distances from points to lines to obtain the optimal line parameters.

[0098] 23) Precision Operations and Bypass Dismantling

[0099] The control arm's end effector, driven by the tool, precisely moves to the calculated center point and orientation of the bypass dismantling horn. The arm's control drive winch then begins releasing the bypass cable, gradually detaching it from the cable clamp, completing the overhead bypass dismantling operation. Throughout the entire process, the robotic arm's position and status are monitored in real time to ensure the safety and accuracy of the operation.

[0100] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, for the device embodiments, the above descriptions are merely preferred embodiments of the present invention. Since they are fundamentally similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention, without departing from the principle of the present invention, should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for automatic overhead bypass operation of a live-line working robot, characterized in that: The process includes positioning, automatic identification, and disassembly, and is detailed below: 1) In the positioning process, perform global modeling of 3D LiDAR, select points for calculation and coarse positioning, adjust the camera on the live-line working robot and collect data; 2) Automatic identification process: A target multimodal information identification and localization method is employed, specifically including: 2.1) Image and point cloud data acquisition: Visible light images and point cloud data are acquired by an area array laser on a robotic arm; the area array laser camera emits laser beams in a specific mode to scan the target area, while the camera records the intensity and phase information of the reflected light, thereby generating high-quality visible light images and point cloud data; 2.2) Preprocess the image; 2.3) Extract the image contours and filter the positioning boxes; 2.4) Correct the orientation of the QR code to ensure it is in a standard horizontal or vertical position; 2.5) 3D coordinate calculation and offset processing: Extract the point cloud data on the target QR code from the collected point cloud data, analyze and calculate the 3D coordinate position of the center point of the QR code frame, and add a fixed offset to the center point of the bypass disassembly horn mouth based on the distance between the target and the fixed structural component between the target and the disassembly horn mouth. 2.6) Plane normal vector and coordinate axis fitting: The RANSAC plane fitting algorithm is used to obtain the normal vector of the QR code plane; 3) Precision operation and bypass dismantling: The control arm end effector tool moves precisely to the position of the bypass dismantling horn opening center point based on the calculated position and attitude information; the arm control drive winch starts to release the line, gradually dismantling the bypass line from the hanging clamp, completing the aerial bypass dismantling operation; the position and status of the robotic arm are monitored in real time throughout the entire operation. The specific method for step 2.3 is as follows: 2.31) Contour detection and extraction: Use the findContour function in the OpenCV library to extract contours from the preprocessed image and obtain all contour information in the image; 2.32) Positioning box filtering: Nesting layer detection. The nesting layer of each contour is counted by the returned vector hierarchy information. Each element in the vector hierarchy stores an array containing 4 integers, which represent the index numbers of the next contour, the previous contour, the parent contour, and the inner contour of the i-th contour, respectively. When the number of contour nesting is greater than or equal to 3, it is used as a candidate contour. 2.33) Positioning box filtering: Rectangle judgment, calculate the perimeter and area of ​​each candidate contour, check whether the ratio between the square of the perimeter and the area is close to 16, and filter out positioning boxes that are close to 16. 2.34) Final screening: There must be three layers of rectangles in the nested contours. Through these two screening steps, contours that do not meet the conditions are excluded, and the remaining ones are the three positioning boxes of the QR code; if the number of remaining contours is not three, the detection is judged as a failure, and the arm-mounted laser camera is adjusted to re-collect data for detection. Step 2.4) The QR code orientation correction process is as follows: the minAreaRect function in the OpenCV library is used to find the smallest bounding rectangle for the selected positioning boxes, and the rotation angle of the rectangle is obtained at the same time; since the three positioning boxes are basically located on the same plane, it is assumed that their rotation angles are the same, that is, this angle is the rotation angle of the QR code, and the orientation of the QR code is corrected according to this rotation angle.

2. The automatic bypass operation method for live-line working robots according to claim 1, characterized in that: Step 1) The positioning process specifically includes the following steps: 1.1) Global Modeling of 3D LiDAR: A super-resolution 3D LiDAR is used to scan the live-line operation scene of the aerial power distribution network to obtain point cloud data of the entire operation area; the LiDAR emits laser beams at high frequency to conduct all-round detection of the surrounding environment and obtain dense point cloud information; by preprocessing the point cloud data, a rough global model of the aerial operation scene is performed. 1.2) Point selection calculation and coarse positioning: Using global point cloud data, a specific algorithm is used to perform point selection calculation, focusing on the feature points of the aerial wire clamp; through the analysis and matching of these feature points, the position of the aerial wire clamp is calculated; the control unit issues instructions based on the calculation results to drive the robotic arm to move to the coarse positioning position; 1.3) Adjust the camera and collect data: After the robotic arm reaches the coarse positioning position, adjust the area array laser camera on the robotic arm to align it with the target area for recognition; the area array laser camera starts working and simultaneously collects RGB images and 3D point cloud data.

3. The automatic bypass operation method for live-line working robots according to claim 1, characterized in that: Step 2.2) The image preprocessing process is specifically as follows: 2.21) Scaling: Scaling the input RGB image; 2.22) Grayscale conversion: Converting a color image to a grayscale image to remove color information; 2.23) Filtering: Set the convolution kernel, first perform sharpening filtering, which enhances the edges and details in the image and highlights the outline features of the QR code; then perform Gaussian blur processing on the grayscale image, which uses the Gaussian function to smooth the image. 2.24) Morphological operations: Morphological operations are performed on the filtered image; first, the binary image is eroded using specific structuring elements to remove small noise and burrs; then, a dilation operation is performed to restore the main outline of the QR code; at the same time, a morphological gradient operation is performed to detect the edge features of the image and further highlight the QR code's bounding box. 2.25) Adaptive threshold binarization: An adaptive method is used to determine the preprocessing parameters.

4. The automatic bypass operation method for live-line working robots according to claim 3, characterized in that: The morphological operation described in step 2.24) is as follows: First, an erosion operation is performed on the binary image, using a specific structuring element to erode the image and remove small noise and burrs; then, a dilation operation is performed to restore the main outline of the QR code; at the same time, a morphological gradient operation is performed to detect the edge features of the image and further highlight the positioning box of the QR code.

5. The automatic bypass operation method for live-line working robots according to claim 3 or 4, characterized in that: Step 2.25) The adaptive threshold binarization specifically involves: during the binarization process, an adaptive threshold algorithm is used, which automatically adjusts the binarization threshold according to the pixel distribution in the local area of ​​the image; for image areas with higher brightness, the binarization threshold is increased accordingly; for image areas with lower brightness, the binarization threshold is decreased.

6. The automatic bypass operation method for live-line working robots according to claim 1, characterized in that: Step 2.6) The RANSAC plane fitting algorithm fits the best-fitting plane model from the point cloud data through random sampling and iteration, thereby obtaining the normal vector of the plane; the x-axis and y-axis point clouds on the plane are extracted according to the corrected rotation angle, and line fitting is performed to obtain the x-axis and y-axis in the target space; the line fitting adopts the least squares method, and the optimal line parameters are obtained by minimizing the sum of squared distances from the point to the line.

Images