Aerial bypass automatic operation method of hot-line work robot

Abstract

The invention provides a hot-line work robot air bypass automatic work method which comprises the following steps: based on a nesting relation and rectangular characteristics of two-dimensional code positioning frames, screening out three positioning frames of a two-dimensional code through the steps of preprocessing an image, extracting a contour, detecting the number of nesting layers, judging rectangles and the like, and then, correcting the direction of the two-dimensional code; and mapping the extracted two-dimensional points to three-dimensional point cloud space points for analysis processing to obtain three-dimensional space center point coordinates and plane normal vectors, and finally fitting an x axis and a y axis in a target space to realize accurate identification of the target. According to the target two-dimensional code and the position of a fixed mechanical structure of the aerial cable clamp dismounting opening, offset processing is carried out, and the purpose that the arm driving head is accurately inserted into the dismounting opening can be achieved.

Description

Technical Field

[0001] The invention relates to the field of live-line robots, in particular to an aerial bypass automatic operation method for live-line working robots. Background Art

[0002] In many application scenarios, such as industrial automation, robotic vision, and augmented reality, accurate target identification and positioning are required. QR codes are often used as targets due to their large information storage capacity and easy recognition. Existing QR code recognition technology mainly focuses on decoding the QR code information, and still has certain limitations in accurately identifying and positioning the QR code in complex environments. Some traditional recognition methods may be affected by factors such as image noise, illumination changes, and QR code deformation, resulting in reduced recognition accuracy. In practical applications, traditional recognition methods are not robust enough to image noise and interference, and are prone to misinterpreting noise or other irrelevant contours as the QR code's positioning frame, resulting in recognition errors. In addition, there is a lack of an effective positioning frame screening mechanism, making it difficult to accurately extract the true positioning frame in complex images, affecting the positioning accuracy of the QR code. Traditional methods are not accurate enough in correcting the orientation of the QR code and fitting the spatial coordinate axes, and cannot meet the needs of high-precision target recognition. Summary of the Invention

[0003] To address the problems of the prior art, the present invention provides a method for automatically handling aerial bypasses for live working robots, enabling efficient and precise removal of aerial bypasses in live environments. Global modeling and coarse positioning using a super-resolution three-dimensional lidar improve positioning accuracy and reliability. Precise target identification and positioning are achieved through multi-step image preprocessing, precise contour extraction and screening, QR code orientation correction, and three-dimensional coordinate calculation. Finally, the end-of-arm drive head tool accurately inserts into the removal port to complete the bypass removal operation, significantly improving operational efficiency and reducing the risks of manual operation.

[0004] The present invention provides an automatic aerial bypass operation method for a live working robot, including positioning, automatic identification and disassembly. The specific process is as follows:

[0005] 1) Positioning process: 3D lidar global modeling, point selection calculation and coarse positioning, adjustment of the camera on the live working robot and data collection;

[0006] 2) Automatic identification process: Using the target multimodal information identification and positioning method, specifically including:

[0007] 2.1) Image and point cloud data acquisition: The area array laser on the robotic arm collects visible light images and point cloud data. The area array laser camera emits a laser beam in a specific pattern to scan the target area, while the camera records the intensity and phase information of the reflected light, thereby generating high-quality visible light images and point cloud data.

[0008] 2.2) Preprocess the image;

[0009] 2.3) Extract the image contour and select the positioning frame;

[0010] 2.4) Correct the QR code orientation to ensure it is in the standard horizontal or vertical position;

[0011] 2.5) 3D Coordinate Calculation and Offset Processing: Extract the point cloud data on the target QR code from the collected point cloud data, analyze and calculate the 3D coordinate position of the center point of the QR code frame, and add a fixed offset to the center point of the bypass disassembly bell mouth based on the distance between the fixed structure of the target and the disassembly bell mouth;

[0012] 2.6) Plane normal vector and coordinate axis fitting: Use the RANSAC plane fitting algorithm to obtain the normal vector of the QR code plane;

[0013] 3) Precision Operation and Bypass Removal: The robot arm's end-drive tool is controlled to precisely move to the bypass removal bell's center point position and posture information based on the calculated information. The arm controls the drive winch to begin paying out the line, gradually removing the bypass line from the wire clip to complete the aerial bypass removal operation. Throughout the operation, the robot arm's position and status are monitored in real time.

[0014] Further improvement, the positioning process in step 1) specifically includes the following steps:

[0015] 1.1) 3D LiDAR Global Modeling: Ultra-high-resolution 3D LiDAR is used to scan live aerial distribution network operations, acquiring point cloud data for the entire operation area. The LiDAR emits laser beams at a high frequency, comprehensively detecting the surrounding environment and generating dense point cloud information. This point cloud data is then pre-processed to create a rough global model of the aerial operation scene.

[0016] 1.2) Point selection calculation and coarse positioning: Utilizing global point cloud data, a specific algorithm is employed to perform point selection calculations, focusing on the characteristic points of the aerial wire clamp. The position of the aerial wire clamp is calculated by analyzing and matching these characteristic points. Based on the calculation results, the control unit issues instructions to drive the robotic arm to the coarse positioning position.

[0017] 1.3) Adjust the camera and collect data: After the robotic arm reaches the coarse positioning position, the area array laser camera on the robotic arm is adjusted to align with the target area for identification. The area array laser camera begins to operate, simultaneously collecting RGB images and 3D point cloud data.

[0018] Further improvement, the image preprocessing process in step 2.2) is specifically as follows:

[0019] 2.21) Scaling: Scale the input RGB image;

[0020] 2.22) Grayscale: Convert a color image to a grayscale image to remove color information;

[0021] 2.23) Filtering: Set the convolution kernel and first perform a sharpening filter. Sharpening filters enhance the edges and details in the image to highlight the contour features of the QR code. Then, perform a Gaussian blur on the grayscale image. Gaussian blur uses a Gaussian function to smooth the image.

[0022] 2.24) Morphological Operations: Perform morphological operations on the filtered image. First, perform an erosion operation on the binary image, using a specific structuring element to remove small noise and glitches. Then, perform a dilation operation to restore the main outline of the QR code. Simultaneously, perform a morphological gradient operation to detect edge features in the image and further highlight the positioning frame of the QR code.

[0023] 2.25) Adaptive threshold binarization: An adaptive method is used to determine the preprocessing parameters.

[0024] Further improvement, the morphological operation in step 2.24) is specifically as follows: first, an erosion operation is performed on the binary image, using a specific structural element to erode the image to remove small noise and burrs in the image; then, an expansion operation is performed to restore the main outline of the QR code; at the same time, a morphological gradient operation is performed to detect the edge features of the image and further highlight the positioning frame of the QR code;

[0025] Further improvement, the adaptive threshold binarization described in step 2.25) is specifically as follows: during the binarization process, an adaptive threshold algorithm is used, which automatically adjusts the binarization threshold according to the pixel distribution in the local area of ​​the image; for image areas with higher brightness, the binarization threshold is correspondingly increased; for image areas with lower brightness, the binarization threshold is reduced.

[0026] Further improvement, the contour extraction and positioning frame screening method in step 2.3) is as follows:

[0027] 2.31) Contour detection and extraction: Use the findContour function in the OpenCV library to extract the contours of the preprocessed image and obtain all the contour information in the image;

[0028] 2.32) Positioning box screening: Nesting layer detection, counting the nesting layers of each contour through the returned vector hierarchy information. Each element in the vector hierarchy stores an array containing 4 int integers, representing the index number of the next contour, the previous contour, the parent contour, and the embedded contour of the i-th contour respectively. Contours with a nesting number greater than or equal to 3 are considered candidate contours;

[0029] 2.33) Positioning box screening: Rectangle determination, calculate the perimeter and area of ​​each candidate contour, check whether the ratio between the square of the perimeter and the area is close to 16, and screen out positioning boxes with a ratio close to 16;

[0030] 2.34) Final screening: There must be three layers of rectangles within the nested contours. This two-step screening eliminates contours that do not meet the criteria, leaving only the three positioning frames of the QR code. If the number of remaining contours is less than three, the detection is considered a failure, and the arm's area array laser camera is adjusted to recollect data for testing.

[0031] As a further improvement, the QR code orientation correction process described in step 2.4) involves using the minAreaRect function in the OpenCV library to find the minimum bounding rectangle of the selected positioning frames and simultaneously obtaining the rotation angle of this rectangle. Since the three positioning frames are essentially in the same plane, their rotation angles can be assumed to be the same, i.e., the rotation angle of the QR code. The QR code orientation is corrected based on this rotation angle.

[0032] As a further improvement, the RANSAC algorithm described in step 2.6) uses random sampling and iteration to fit the most appropriate plane model from the point cloud data, thereby obtaining the plane normal vector. Based on the corrected rotation angle, the x-axis and y-axis point clouds on the plane are deducted and linearly fitted to obtain the x-axis and y-axis in the target space. Linear fitting can be performed using the least squares method to minimize the sum of the squared distances from the points to the line to obtain the optimal line parameters.

[0033] The beneficial effects of the present invention are:

[0034] 1. The multimodal information fusion method of the present invention for automatic identification, positioning and disassembly of aerial bypass by a dual-arm live working robot realizes efficient and accurate disassembly of aerial bypass in a live environment.

[0035] 2. Using super-resolution 3D LiDAR for global modeling and coarse positioning improves positioning accuracy and reliability;

[0036] 3. Accurate target identification and positioning are achieved through multi-step image preprocessing, precise contour extraction and screening, QR code direction correction and three-dimensional coordinate calculation.

[0037] 4. The end-drive head tool of the robotic arm can be accurately inserted into the disassembly port to complete the bypass disassembly operation, which greatly improves the operation efficiency and reduces the risk of manual operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is the overall flow chart of the automatic identification, positioning and disassembly algorithm for the aerial bypass of a dual-arm live-working robot;

[0040] Figure 2 It is a flow chart of the target multimodal information recognition and positioning algorithm;

[0041] Figure 3 It is the flow chart of the image preprocessing algorithm;

[0042] Figure 4 It is the flow chart of the contour extraction and positioning box screening algorithm;

[0043] Figure 5 This is an example diagram of the center point of the automatic target recognition algorithm for the dual-arm robot in actual operation of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] Overall idea:

[0046] Based on the nested relationship and rectangular features of the QR code positioning frames, the present invention selects the three positioning frames of the QR code through steps such as image preprocessing, contour extraction, nested layer detection, and rectangle judgment. The QR code is then oriented and mapped to three-dimensional point cloud space points for analysis and processing, obtaining the coordinates of the three-dimensional center point and the plane normal vector. Finally, the x-axis and y-axis in the target space are fitted to achieve accurate target identification. By offsetting the target QR code and the fixed mechanical structure position of the aerial wire clamp disassembly port, the arm-driven head can be accurately inserted into the disassembly port.

[0047] Specific steps:

[0048] like Figure 1 As shown in the figure, it is the main flow chart of the method for automatic identification, positioning and disassembly of aerial bypass of a dual-arm live-working robot with multimodal information fusion of the present invention. First, a rough global modeling of the aerial working scene is performed using a super-resolution three-dimensional laser radar, and the position of the aerial hanging wire clamp is obtained after point selection calculation processing using the global point cloud. At this time, the control unit issues a command to let the robot arm reach the rough positioning position, and adjusts the array camera on the arm to start taking pictures, collecting RGB images and three-dimensional point cloud data, and then starts as follows Figure 2 The target's multimodal information identification and positioning method is shown, obtaining the 3D coordinates of the center point, the plane normal vector, and the x- and y-axes. The end-of-arm drive head tool is controlled to precisely identify the bypass removal opening. The arm then controls the winch to pay out the line, completing the aerial bypass removal.

[0049] in, Figure 2 The specific steps of the target multimodal information identification and positioning method shown are as follows:

[0050] 1) Collect visible light images and point cloud data through the area array laser on the arm;

[0051] 2) Preprocessing the collected image data;

[0052] 3) Use the findContour function in the OpenCV library to extract the contour of the preprocessed image, obtain all contour information, and filter the QR code positioning frame;

[0053] 4) Use the minAreaRect function in the OpenCV library to find the minimum enclosing rectangle of the filtered positioning frame, obtain its rotation angle, and use this as the rotation angle of the QR code for direction correction;

[0054] 5) Extract the point cloud data on the target QR code from the collected point cloud data, analyze and calculate the three-dimensional coordinate position of the center point of the QR code frame, and add a fixed offset to the center point position of the bypass disassembly bell mouth based on the distance between the fixed structure of the target and the disassembly bell mouth;

[0055] 6) Use RANSAC plane fitting to obtain the plane normal vector of the QR code. According to the corrected rotation angle, deduct the x-axis and y-axis point clouds on the plane, and perform straight line fitting respectively to obtain the x-axis and y-axis in the target space.

[0056] like Figure 3 As shown, the steps of the image preprocessing method 2) are as follows:

[0057] 21) Scaling: Appropriately scale the input RGB image to reduce the amount of calculation and improve processing efficiency;

[0058] 22) Grayscale conversion: converting a color image into a grayscale image, removing color information and simplifying image data representation;

[0059] 23) Filtering: Set a convolution kernel of 3x3 and perform a sharpening filter to enhance the edges and details of the image to make it clearer. Then perform Gaussian blurring on the grayscale image to reduce image noise and smooth the image.

[0060] 24) Morphological operation: First, perform an erosion operation on the binary image to remove small noise and burrs in the image, then perform a dilation operation to restore the main outline of the QR code and make the positioning frame clearer; morphological gradient operation detects image edge features;

[0061] 25) Adaptive Threshold Binarization: This method uses an adaptive method to determine preprocessing parameters. During the binarization process, an adaptive threshold algorithm is used, with the threshold setting determined by the distribution of pixels in the surrounding neighborhood. Brighter image regions typically have a higher binarization threshold, while lower brightness regions have a correspondingly lower threshold. Local image regions with varying brightness, contrast, and texture will have corresponding local binarization thresholds, improving binarization accuracy.

[0062] like Figure 4 As shown, the steps of the 3) contour extraction and positioning frame screening method are as follows:

[0063] 31) Contour detection and extraction: Use the findContour function in the OpenCV library to obtain all contours in the image. These contours may include the positioning box of the QR code and other irrelevant contours.

[0064] 32) Positioning box screening: Nesting layer detection. The number of nesting layers of each contour is counted using the returned vector hierarchy information. Each element in the vector hierarchy stores an array containing 4 int integers, representing the index numbers of the next contour, the previous contour, the parent contour, and the embedded contour of the i-th contour. Since the QR code positioning box has three nested layers, and considering the influence of factors such as noise, contours with a nesting number greater than or equal to 3 can be considered as candidate contours;

[0065] 33) Positioning box screening: rectangle judgment. Calculate the perimeter and area of ​​each candidate contour and check whether the ratio between the square of the perimeter and the area is close to 16. Assume that a is the side length of the square, the perimeter of the square is 4a, and the area is a. 2 , the square of the perimeter is 16a 2 , the ratio of the square of the perimeter to the area is 16. In this way, we can determine whether the contour is approximately rectangular;

[0066] 34) Nested contours must contain three layers of rectangles. These two steps eliminate unqualified contours, leaving only the three positioning boxes of the QR code. If the number of remaining contours is less than three, the detection is considered a failure, and the arm's area array laser camera is adjusted to recollect data for detection.

[0067] A specific implementation scenario of the present invention is as follows Figure 5 As shown:

[0068] 1. Implementation scenario overview:

[0069] During power system maintenance and overhaul, it's often necessary to remove overhead bypass cables. The present invention's method for automatically identifying, locating, and removing overhead bypass cables using a multimodal information-fusion dual-arm live-working robot is designed to enable the safe, efficient, and precise removal of overhead bypass cables in live environments. The following describes the implementation of this method in detail, combining specific implementation scenarios.

[0070] 2. Specific implementation steps:

[0071] 21) Global modeling and coarse positioning

[0072] 211) 3D LiDAR Global Modeling

[0073] A high-resolution 3D LiDAR scans live aerial distribution network operations, acquiring point cloud data for the entire work area. The LiDAR emits laser beams at a high frequency, comprehensively surveying the surrounding environment and generating dense point cloud information. Preprocessing the point cloud data, such as removing noise points and filtering, allows for a rough global modeling of the aerial work scene.

[0074] 212) Point selection calculation and rough positioning

[0075] Using global point cloud data, a specific algorithm is employed to perform point selection calculations, focusing on the characteristic points of the aerial wire clamp. By analyzing and matching these characteristic points, the position of the aerial wire clamp is accurately calculated. Based on the calculation results, the control unit issues commands to drive the robotic arm to the coarse positioning position.

[0076] 213) Adjust the camera and collect data

[0077] After the robotic arm reaches the coarse positioning position, the area array laser camera on the robotic arm is adjusted to align with the target area for identification. The area array laser camera begins operating, simultaneously capturing RGB images and 3D point cloud data. The RGB image provides rich visual information, while the 3D point cloud data contains depth information of the target object, providing the basis for subsequent precise identification and positioning.

[0078] 22) Target multimodal information identification and positioning method

[0079] 221) Image and point cloud data acquisition

[0080] The area array laser on the arm collects visible light images and point cloud data. The area array laser camera emits a laser beam in a specific pattern to scan the target area, while the camera records the intensity and phase information of the reflected light, generating high-quality visible light images and point cloud data.

[0081] 222) Image preprocessing

[0082] a) Scaling: Appropriately scale the input RGB image to reduce the computational complexity of subsequent processing without losing too much image details, thereby improving processing efficiency.

[0083] b) Grayscale conversion: Convert the color image to a grayscale image to remove color information. In this embodiment, the grayscale conversion is performed using a weighted average method, that is, based on the human eye's sensitivity to different colors, the three RGB channels are weighted averaged to obtain a grayscale value.

[0084] c) Filtering: A convolution kernel of 3 x 3 is set, and a sharpening filter is first applied. This filter enhances edges and details, making the image clearer and highlighting the outline of the QR code. Then, a Gaussian blur is applied to the grayscale image. This smoothes the image using a Gaussian function, reducing noise and creating a smoother image.

[0085] d) Morphological Operations: Morphological operations are performed on the filtered image. First, an erosion operation is performed on the binary image, using specific structuring elements to remove small noise and glitches. Then, a dilation operation is performed to restore the main outline of the QR code and make the positioning frame clearer. Simultaneously, a morphological gradient operation is performed to detect edge features in the image and further highlight the positioning frame of the QR code.

[0086] e) Adaptive Threshold Binarization: An adaptive method is used to determine preprocessing parameters. During the binarization process, an adaptive threshold algorithm is used. This algorithm automatically adjusts the binarization threshold based on the pixel distribution in the local image region. For areas with higher brightness, the binarization threshold is increased accordingly; for areas with lower brightness, the binarization threshold is decreased. This ensures that local image regions with varying brightness, contrast, and texture receive appropriate binarization processing, improving binarization accuracy.

[0087] 223) Contour extraction and positioning frame screening

[0088] a) Contour detection and extraction: Use the findContour function in the OpenCV library to extract contours from the preprocessed image and obtain all contour information in the image. These contours may include the positioning frame of the QR code and other irrelevant contours.

[0089] b) Positioning Box Screening: Nesting Level Detection. The returned vector hierarchy information counts the number of nesting levels of each contour. Each element in the vector hierarchy stores an array of four int integers, representing the index numbers of the next contour, the previous contour, the parent contour, and the embedded contours of the i-th contour. Because the QR code positioning box has three nested layers, and considering the influence of factors such as noise, contours with a nested level of three or more are considered candidate contours.

[0090] c) Positioning box screening: rectangle judgment. Calculate the perimeter and area of ​​each candidate contour and check whether the ratio between the square of the perimeter and the area is close to 16. Assume that a is the side length of the square, the perimeter of the square is 4a, and the area is a 2 , the square of the perimeter is 16a 2 , the ratio of the square of the perimeter to the area is 16. In this way, we can determine whether the contour is approximately rectangular.

[0091] d) Final screening: The nested contours must contain three layers of rectangles. This two-step screening eliminates contours that do not meet the criteria, leaving only the three positioning boxes of the QR code. If the number of remaining contours is less than three, the detection is considered a failure, and the arm's area array laser camera is adjusted to re-collect data for testing.

[0092] 224) QR code orientation correction

[0093] Use the minAreaRect function in the OpenCV library to find the minimum enclosing rectangle of the selected positioning frames and obtain the rotation angle of this rectangle. Since the three positioning frames are essentially on the same plane, their rotation angles can be assumed to be the same, which is the rotation angle of the QR code. Based on this rotation angle, the QR code is oriented and corrected to maintain a standard horizontal or vertical position.

[0094] 225) 3D coordinate calculation and offset processing

[0095] The point cloud data for the target QR code is extracted from the collected point cloud data. A specific algorithm is used to analyze and calculate the 3D coordinates of the center point of the QR code frame. Based on the distance between the target and the disassembly bell, a fixed offset is added to the center point of the bypass disassembly bell. For example, if the fixed distance d between the target and the disassembly bell is previously measured, the offset d is added along a specific direction to the calculated 3D coordinates of the QR code center point to obtain the 3D coordinates of the bypass disassembly bell center point.

[0096] 226) Plane normal vector and coordinate axis fitting

[0097] The RANSAC plane fitting algorithm is used to obtain the normal vector of the QR code plane. The RANSAC algorithm uses random sampling and iteration to fit the most appropriate plane model from the point cloud data, thereby obtaining the plane normal vector. Based on the corrected rotation angle, the x-axis and y-axis point clouds on the plane are deducted and straight line fitting is performed on each axis to obtain the x-axis and y-axis in the target space. The least squares method can be used to minimize the sum of the squared distances between the points and the line to obtain the optimal line parameters.

[0098] 23) Precision operation and bypass disassembly

[0099] The tooling at the end of the arm controls the actuator to precisely move to the bypass removal bell's center point and posture based on the calculated position. The arm then controls the capstan to begin paying out the bypass line, gradually removing it from the wire clip, completing the aerial bypass removal operation. Throughout the entire operation, the robotic arm's position and status are monitored in real time to ensure safety and accuracy.

[0100] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, the above is only a preferred embodiment of the present invention. Since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this. Any technical personnel familiar with this technical field is within the technical scope disclosed by the present invention. For ordinary technical personnel in this technical field, changes or replacements that can be easily thought of should be covered within the protection scope of the present invention without departing from the principle of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for automatic aerial bypass operation of a live working robot, characterized by: Including positioning, automatic identification and disassembly, the specific process is as follows: 1) Positioning process: 3D lidar global modeling, point selection calculation and coarse positioning, adjustment of the camera on the live working robot and data collection; 2) Automatic identification process: Using the target multimodal information identification and positioning method, specifically including: 2.1) Image and point cloud data acquisition: The area array laser on the robotic arm collects visible light images and point cloud data. The area array laser camera emits a laser beam in a specific pattern to scan the target area, while the camera records the intensity and phase information of the reflected light, thereby generating high-quality visible light images and point cloud data. 2.2) Preprocess the image; 2.3) Extract the image contour and select the positioning frame; 2.4) Correct the QR code orientation to keep it in the standard horizontal or vertical position; 2.5) 3D Coordinate Calculation and Offset Processing: Extract the point cloud data on the target QR code from the collected point cloud data, analyze and calculate the 3D coordinate position of the center point of the QR code frame, and add a fixed offset to the center point of the bypass disassembly bell mouth based on the distance between the fixed structure of the target and the disassembly bell mouth; 2.6) Plane normal vector and coordinate axis fitting: Use the RANSAC plane fitting algorithm to obtain the normal vector of the QR code plane; 3) Precision Operation and Bypass Removal: The robot arm's end-drive tool is controlled to precisely move to the bypass removal bell's center point position and posture information based on the calculated information. The arm controls the drive winch to begin paying out the line, gradually removing the bypass line from the wire clip to complete the aerial bypass removal operation. Throughout the operation, the robot arm's position and status are monitored in real time.

2. The method for automatic aerial bypass operation of a live working robot according to claim 1, characterized in that: Step 1) The positioning process specifically includes the following steps: 1.1) 3D LiDAR Global Modeling: Ultra-high-resolution 3D LiDAR is used to scan live aerial distribution network operations, acquiring point cloud data for the entire operation area. The LiDAR emits laser beams at a high frequency, comprehensively detecting the surrounding environment and generating dense point cloud information. This point cloud data is then pre-processed to create a rough global model of the aerial operation scene. 1.2) Point selection calculation and coarse positioning: Utilizing global point cloud data, a specific algorithm is employed to perform point selection calculations, focusing on the characteristic points of the aerial wire clamp. The position of the aerial wire clamp is calculated by analyzing and matching these characteristic points. Based on the calculation results, the control unit issues instructions to drive the robotic arm to the coarse positioning position. 1.3) Adjust the camera and collect data: After the robotic arm reaches the coarse positioning position, the area array laser camera on the robotic arm is adjusted to align with the target area for identification. The area array laser camera begins to operate, simultaneously collecting RGB images and 3D point cloud data.

3. The method for automatic aerial bypass operation of a live working robot according to claim 1, characterized in that: The image preprocessing process in step 2.2) is specifically as follows: 2.21) Scaling: Scale the input RGB image; 2.22) Grayscale: Convert a color image to a grayscale image to remove color information; 2.23) Filtering: Set the convolution kernel and first perform a sharpening filter. Sharpening filters enhance the edges and details in the image to highlight the contour features of the QR code. Then, perform a Gaussian blur on the grayscale image. Gaussian blur uses a Gaussian function to smooth the image. 2.24) Morphological Operations: Perform morphological operations on the filtered image. First, perform an erosion operation on the binary image, using a specific structuring element to remove small noise and glitches. Then, perform a dilation operation to restore the main outline of the QR code. Simultaneously, perform a morphological gradient operation to detect edge features in the image and further highlight the positioning frame of the QR code. 2.25) Adaptive threshold binarization: An adaptive method is used to determine the preprocessing parameters.

4. The method for automatic aerial bypass operation of a live working robot according to claim 3, characterized in that: The morphological operation described in step 2.24) is specifically as follows: first, an erosion operation is performed on the binary image, and a specific structural element is used to erode the image to remove small noise and burrs in the image; then an expansion operation is performed to restore the main outline of the QR code; at the same time, a morphological gradient operation is performed to detect the edge features of the image to further highlight the positioning frame of the QR code.

5. The method for automatic aerial bypass operation of a live working robot according to claim 3 or 4, characterized in that: The adaptive threshold binarization described in step 2.25) is specifically as follows: during the binarization process, an adaptive threshold algorithm is used, which automatically adjusts the binarization threshold according to the pixel distribution in the local area of ​​the image; for image areas with higher brightness, the binarization threshold is correspondingly increased; for image areas with lower brightness, the binarization threshold is reduced.

6. The method for automatic aerial bypass operation of a live working robot according to claim 1, characterized in that: The contour extraction and positioning frame screening method described in step 2.3) is specifically as follows: 2.31) Contour detection and extraction: Use the findContour function in the OpenCV library to extract the contours of the preprocessed image and obtain all the contour information in the image; 2.32) Positioning box screening: Nesting layer detection, counting the nesting layers of each contour through the returned vector hierarchy information. Each element in the vector hierarchy stores an array containing 4 int integers, representing the index number of the next contour, the previous contour, the parent contour, and the embedded contour of the i-th contour respectively. Contours with a nesting number greater than or equal to 3 are considered candidate contours; 2.33) Positioning box screening: Rectangle determination, calculate the perimeter and area of ​​each candidate contour, check whether the ratio between the square of the perimeter and the area is close to 16, and screen out positioning boxes with a ratio close to 16; 2.34) Final screening: There must be three layers of rectangles within the nested contours. This two-step screening eliminates contours that do not meet the criteria, leaving only the three positioning frames of the QR code. If the number of remaining contours is less than three, the detection is considered a failure, and the arm's area array laser camera is adjusted to recollect data for testing.

7. The method for automatic aerial bypass operation of a live working robot according to claim 1, characterized in that: Step 2.4) describes the QR code orientation correction process: Using the minAreaRect function in the OpenCV library, the minimum bounding rectangle (MBR) of the selected positioning frames is found, and the rotation angle of this rectangle is obtained. Since the three positioning frames are essentially in the same plane, their rotation angles are assumed to be the same, i.e., the rotation angle of the QR code. This rotation angle is then used to correct the QR code's orientation.

8. The method for automatic aerial bypass operation of a live working robot according to claim 1, characterized in that: The RANSAC algorithm described in step 2.6) uses random sampling and iteration to fit the most appropriate plane model from the point cloud data, thereby obtaining the plane normal vector. Based on the corrected rotation angle, the x-axis and y-axis point clouds on the plane are deducted and linearly fitted to obtain the x-axis and y-axis in the target space. Linear fitting can be performed using the least squares method to minimize the sum of the squared distances from the points to the line to obtain the optimal line parameters.

Images