A live-line lead extension method for a high-voltage live-line work robot

Abstract

The application discloses a kind of high voltage hot-line work robot live wire connection wire operation method, it is related to live-line work field, including wire stripper, lead fastening device, lead clamp etc. front end actuator, live-line work platform, visual module, main and auxiliary mechanical arm.The application effectively improves the efficiency of live wire connection wire operation by visual algorithm;And through the main and auxiliary mechanical arm of six degrees of freedom, special front end actuator is carried, improves the operation flexibility of robot;With mechanical arm instead of human operation, improve operation safety.The application designs the visual algorithm of judging lead posture, solves the problem of weak texture small target missed detection by RGB-D camera through pixel level image processing;By simplifying lead target, interested target is regarded as rigid body, solve the problem of lead target posture is not clear;Utilize the pose estimation network based on RGB image, solve the target recognition and pose estimation problem under the condition of complex working environment, target is shielded.

Description

Technical Field

[0001] This invention relates to the field of live-line working, specifically a method for live-line connection of leads in a 10kV power distribution line. Background Technology

[0002] As high-voltage working robots enter the power maintenance industry, there is a growing expectation that robots can replace human workers in various aspects to ensure personnel safety. However, in this era where robot intelligence is still relatively low, many delicate operations still require experienced operators. For example, in live-line connection work, the grasping and lifting of the lead wire, and the installation of the lead wire to the main line, cannot be efficiently performed by robots alone. Therefore, the development of new high-voltage live-line working robots for live-line connection operations has become one of the challenges that needs to be addressed. Summary of the Invention

[0003] To address the aforementioned problems, the present invention aims to provide a method for live-line connection of high-voltage power lines using a robot, thereby improving the efficiency and safety of live-line operations on triangularly arranged power distribution lines. Through the collaboration of dual arms and computer vision, the robot can grasp, lift, and connect the power lines.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] The high-voltage live-line working robot on which the method is based includes a live-line working platform; two collaborative robotic arms; and a live-line working front-end actuator including a wire stripper, a wire fastening device, a wire clamp, and a vision module.

[0006] A method for live wire connection includes the following steps:

[0007] Step 1: The live-line working platform moves to the predetermined working position, with the main and auxiliary robotic arms each carrying a vision module; the main robotic arm, equipped with the wire stripper's front-end actuator, arrives at the preparatory position of the phase to be worked on; the main line is identified and located through the vision algorithm, and the wire stripper is used to complete the insulation removal operation on the main line.

[0008] Step 2: The main robotic arm switches the front-end actuator to the lead wire fastening device, and the auxiliary robotic arm, equipped with the lead wire clamp front-end actuator, moves to the lead wire gripping preparation position. The visual algorithm determines the lead wire posture and grips the rear end of the lead wire from the insulation part.

[0009] The specific process of the visual algorithm in step 2 is as follows:

[0010] The key to determining the lead wire attitude is enabling the robot to perceive the attitude of the insulated section of the lead wire and optimally grasp the rear end of the deinsulated section, moving it to the same attitude as the deinsulated section of the main wire. Small segments of the electric field line have high rigidity, so a small segment of the deinsulated section can be treated as a rigid body. The auxiliary robotic arm, carrying the lead wire fastening device and vision module, moves to the lead wire grasping preparation position (i.e., the side of the lead wire) and uses the vision module to capture the lead wire connection operation scene. First, the image is fed into the network, which predicts the features of the deinsulated section of the lead wire and performs axial expansion of the target pixels at the pixel level. The attitude estimation network calculates the semantic segmentation information of the target and the vector field pointing to the target key points. Subsequently, RANSAC is used to... Voting calculates the target key points from the direction vector field and generates the mean and covariance of the spatial distribution of the key points. Finally, using the EPnP algorithm, the position of the target key points in the image of the vision module is known. The pixel coordinates of the target key points are converted into coordinates in the camera coordinate system using the camera intrinsic matrix and perspective projection model. The coordinates in the camera coordinate system are converted into three-dimensional coordinates in the world coordinate system using the camera extrinsic parameters, so that the robot can perceive its posture in three-dimensional space.

[0011] Step 3: The main robotic arm, equipped with the lead wire fastening device, reaches below the main line. The angle of the lead wire fastening device is adjusted according to the direction of the main line. The auxiliary robotic arm drags the lead wire to the upper side of the main line. The main robotic arm completes the work of fixing the main line and the lead wire to the lead wire fastening device.

[0012] Step 3 is as follows:

[0013] Step 31: The auxiliary robotic arm aligns the part of the lead wire with the part of the main wire with the part of the main wire with the insulation stripped. The specific method is as follows: The algorithm is used to obtain the posture of the main wire with the insulation stripped part (A) and use it as a target. The auxiliary robotic arm corrects the posture of the lead wire with the insulation stripped part (B) through the target so that the posture of (B) is the same as that of (A) and is aligned with it.

[0014] Step 32: The main robotic arm lifts the lead wire fastening device in a straight line and, relying on the funnel structure of the lead wire fastening device, allows the deinsulated part of the main wire to slide into the gripper assembly.

[0015] Step 33: The auxiliary robotic arm adjusts the attitude of the lead wire, slides the uninsulated part of the lead wire into the gripper assembly, and the lead wire fastening device opens the clamping motor to fix the main lead wire.

[0016] Step 4: The main robotic arm descends with the lead wire fastening device, clamping the wire onto the main wire and the lead wire. The main robotic arm retracts, and the auxiliary robotic arm releases the lead wire and retracts. The main and auxiliary robotic arms move to the reset position, and the live wire connection operation is completed.

[0017] Live-line connection work is dangerous and involves complex environments, and there has long been a desire to use robots to replace manual labor in this task. Existing technologies mainly focus on standardizing the live-line connection process and innovating in the identification of power lines, with very little research on lead identification and attitude estimation, preventing live-line working robots from achieving full automation. Compared with existing technologies, this invention provides a new process for autonomous live-line connection by a live-line working robot, achieving unmanned live-line work; this invention designs a new visual algorithm for determining lead attitude, solving the problem of missed detection of small targets with weak textures using RGB-D cameras through pixel-level image processing; by simplifying the lead target, treating the target of interest as a rigid body, it solves the problem of unclear lead target attitude; and by utilizing an RGB image-based attitude estimation network, it solves the problem of target identification and attitude estimation in complex working environments and when the target is occluded. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the triangular arrangement of power poles for this 10kV power distribution line;

[0019] Figure 2 This is a schematic diagram of the insulation removal operation of the wire stripper of the present invention;

[0020] Figure 3 This is a schematic diagram of the working operation of the present invention, which includes a lead clamp and a lead fastening device for connecting leads.

[0021] Figure 4 This is a schematic diagram of the lead wire fastening device involved in the present invention;

[0022] Figure 5 This is a flowchart of the lead attitude estimation algorithm described in this invention;

[0023] Figure 2 and Figure 3 In the diagram: 1 is the live-line working platform, 2 is the main robotic arm, 3 is the front-end actuator of the wire stripper, 4 is the lead wire, 5 is the front-end actuator of the lead wire fastening device, 6 is the front-end actuator of the lead wire clamp, 7 is the auxiliary robotic arm, and 8 is the main line to be worked on.

[0024] Figure 4 The fastening device includes a gripper mounting base 9, a funnel-shaped guide frame 10, and a gripper assembly 11. Detailed Implementation

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0026] Step 1: The live-line working platform 1 moves to the predetermined working position. The main robotic arm 2, carrying a vision module and equipped with the wire stripper front-end actuator 3, arrives at the working phase preparation position. The main line is identified and located through the vision algorithm, and the wire stripper completes the insulation removal operation on the main line.

[0027] Step 2: The main robotic arm 2 switches its front-end mechanism to the lead wire fastening device front-end actuator 5, and the auxiliary robotic arm 7, equipped with the lead wire clamp front-end actuator 6, moves to the lead wire gripping preparation position. The lead wire posture is determined by the visual algorithm, and the lead wire is gripped at the rear end of the insulated part.

[0028] Step 3: The main robotic arm 2, equipped with the lead wire fastening device 5, reaches below the main line 8. The angle of the lead wire fastening device 5 is adjusted according to the direction of the main line. The auxiliary robotic arm 7 drags the lead wire 4 to the upper side of the main line 8. The main robotic arm 2 completes the work of fixing the main line 8 and the lead wire 4 to the lead wire fastening device 5.

[0029] The specific steps for tightening the lead wires in step 3 are as follows:

[0030] Step 31: The auxiliary robotic arm 7 drags the lead wire to align the deinsulated part of the lead wire with the deinsulated part of the main wire;

[0031] Step 32: The main robotic arm 2 holds the lead wire fastening device 5 and aligns it with the main wire insulation part. Relying on the funnel structure 2 of the lead wire fastening device, the insulation part of the main wire slides into the gripper assembly 11.

[0032] Step 33: The auxiliary robotic arm 7 adjusts the attitude of the lead wire, slides the uninsulated part of the lead wire into the gripper assembly 11, and clamps the wire clamp 11 through the gripper mounting base 9 of the lead wire fastening device to fix the main lead wire.

[0033] Step 4: The main robotic arm 2 descends with the lead wire fastening device, leaving the gripper assembly 11 on the main wire and the lead wire. The main robotic arm 2 retracts, and the auxiliary robotic arm 7 releases the lead wire and retracts. The main and auxiliary robotic arms move to the reset state.

[0034] The specific implementation methods of the visual algorithms mentioned in steps 2 and 3 are as follows:

[0035] The deinsulated portion of the lead wire is denoted as (B), and the deinsulated portion of the main wire is denoted as (A). The visual algorithm in this invention adopts a method for judging the poses of (B) and (A) from a single image. The designed scheme is divided into three stages: the deinsulated portion recognition and extraction stage, the feature pixel extension stage, and the target feature pose estimation stage.

[0036] 1. The auxiliary robotic arm, carrying the lead wire fastening device and vision module, moves to the lead wire grasping preparation position (i.e., the side of the lead wire) and uses a camera to capture the lead wire receiving operation scene; the image is preprocessed to 640×640 and fed into the Darknet network. The Darknet network divides the image into an S×S grid, and each cell is responsible for detecting the (B) and (A) features whose center point falls within that cell; Darknet is a lightweight CNN architecture composed of multiple convolutional and pooling layers. Through multiple convolutional and pooling operations, Darknet gradually reduces the size of the feature map and increases the number of channels, extracting higher-level local features of the image while preserving semantic information. The system detects and locates global features; it matches bounding boxes with each feature to achieve feature detection and localization; each cell predicts a bounding box and its confidence score, which includes the probability of the target contained in the bounding box (Pr(OBJ)) and the accuracy of the bounding box; the accuracy of the bounding box is characterized by the Intersection over Union (IOU) between the predicted box and the actual box; therefore, the confidence score can be defined as Pr(OBJ) × IOU; the predicted value of the bounding box can be represented by (x, y, w, h, c), where (x, y) represents the center coordinates of the bounding box, w and h represent the width and height of the bounding box, and c represents its confidence score; its loss function is as follows:

[0037] Loss=a×lossobj+b×lossrect+c×lossclc

[0038] Where `lossobj` is the confidence loss, representing the probability of the bounding box containing the target; `lossrect` is the bounding box loss, representing the accuracy of the bounding box; `lossclc` is the classification loss, representing the accuracy of different target detections; `a`, `b`, and `c` represent the weights of `lossobj`, `lossrect`, and `lossclc`, respectively; finally, the predicted bounding boxes (B) and (A) are output; then, the non-maximum suppression (NMS) method is used to sort the bounding boxes, and each bounding box is traversed one by one. The bounding box with the highest confidence is selected, and the intersection-union ratio (IOU) of the remaining bounding boxes with the bounding box with the highest confidence is calculated. Bounding boxes with a high degree of overlap with the bounding box with the highest confidence are suppressed; the IOU is calculated as follows:

[0039] w = min(x 1.1 +w1,x 1.2 +w2)-max(x 1.1 ,x 1.2 )

[0040] h = min(y 1.1 +h1,y 1.2 +h2)-max(y 1.1 ,y 1.2 )

[0041] overlap_area=w×h(w>0; h>0)

[0042]

[0043] Where (w,h) represents the width and height of the overlapping bounding box, (x1,y1) and (x2,y2) represent the center coordinates of the bounding box with the highest confidence and the currently compared bounding box, respectively, (w1,h1) and (w2,h2) represent the width and height of the bounding box with the highest confidence and the currently compared bounding box, respectively, overlap_area represents the area of ​​the overlapping bounding box, total_area represents the total area occupied by the two bounding boxes, and IOU represents the intersection-union ratio of the two bounding boxes. Based on the calculated IOU, the degree of overlap between the two bounding boxes is judged. If the IOU is greater than a preset value, the two bounding boxes are considered to have a large overlap and need to be suppressed. The best bounding box is selected by NMS calculation and comparison to improve the detection accuracy.

[0044] Axial pixel expansion is performed on (A) and (B) to leave space for the robotic arm to grasp; the target pixels obtained from feature recognition are encapsulated and extracted. The outline of the target should be a rectangular area. The central axis of the rectangular area along its length is found by image thinning. The target is expanded in two directions along this central axis by pixel expansion. The specific implementation is as follows: the image is converted into a binary image with a black background (=0) and a target rectangular area pixel value (=255). Using the principle of erosion and dilation, the rectangular mask is eroded along the left and right directions of the wide side. After multiple iterations, the central axis of the rectangle along its length is found. The target pixel area is expanded by the central axis.

[0045] A pose estimation network is used to predict the orientation of pixels, which involves semantic segmentation and vector field prediction. For each pixel p, the network outputs a semantic label that associates it with a specific object and a two-dimensional keypoint X representing the distance from pixel p to the object. k The direction of the vector V k (p); Direction vector V k (p) is defined as follows:

[0046]

[0047] Where V k (p) represents the direction vector of the key point, x k , p represents the pixel coordinates of the keypoint and pixel P, ||x k -p||2 indicates x k The normal distance to P;

[0048] Key points for generating hypotheses based on the RANSAC strategy: First, obtain the relevant pixels of the target through semantic segmentation labels; then, randomly select the direction vectors of two pixels and use their intersection as the keypoint X. kThe hypothesis h k,i Repeat N times to generate a set of hypotheses {h} k,i |i=1,2,…,N}. Then the assumed voting score w k,i Defined as:

[0049]

[0050] Where w k,i h represents the score of the i-th vote. k,i Let O represent the hypothetical point of the i-th key point, and let O represent the index function. k,i -p) T Let ||h| be the transpose matrix of the direction vectors of the hypothetical point and the target pixel. k,i -p||2 indicates x k The normative distance to P, θ is a threshold (its value is 0.99), and p∈O means that pixel p belongs to target O. When w k,i A larger value indicates a more persuasive hypothesis; the resulting hypothesis characterizes the spatial probability distribution of keypoints in the image, with the mean and covariance of the keypoints as follows:

[0051]

[0052]

[0053] Where μ k It is mean, ∑ k It's covariance, w k,i h represents the score of the i-th vote. k,i It is the hypothetical point of the i-th key point;

[0054] Given the two-dimensional keypoint positions of each target, its six-dimensional pose can generally be calculated using the PnP algorithm. The PnP algorithm takes a set of known 2D-3D point pairs as input and solves for the camera's rotation and translation matrix by minimizing the reprojection error. Its expression is as follows:

[0055]

[0056] Where P is a point in the real world, P' is the corresponding point on the image, K is the camera intrinsic parameter (where α and β are the scaling focal lengths), s is the skew parameter, (u0, v0) is the optical center, and w is the scaling factor of the image point. It is the camera rotation and translation matrix; however, the traditional PnP algorithm is not robust to noise and outliers and is not efficient, so this algorithm uses the EPnP algorithm instead of the PnP algorithm.

[0057] EPnP is an optimization of the traditional PnP algorithm. It improves computational efficiency and accuracy by introducing additional constraints and optimization strategies. The core idea of ​​the EPnP algorithm is to project 3D points onto the camera coordinate system and use linear least squares to solve for the camera pose. The EPnP algorithm obtains an initial camera pose estimate by solving a system of linear equations, and then uses iterative optimization to further improve the accuracy of the estimate. We use the EPnP algorithm to solve for keypoints, given the average value μ of k = 1, ..., K. k The sum of the covariance matrix ∑ k We compute its 6D pose by minimizing the Mahalanobis distance.

[0058]

[0059]

[0060] Where X k These are the three-dimensional coordinates of the key points. Yes, X k Two-dimensional projection, where π is the perspective projection function. EPnP adjusts the parameters based on four key points. The initialization process minimizes the covariance matrix trajectory. Then, the Levenberg-Marquardt algorithm is used to reduce reprojection errors, ultimately enabling the robot to perceive the pose information of (A) and (B).

[0061] The above embodiments are only used to illustrate the present invention. The structure and connection method of each component can be varied. Based on the technical solution of the present invention, any improvements and equivalent transformations made to some of the technical features according to the present invention should not be excluded from the protection scope of the present invention.

Claims

1. A method for live-line connection operation of a high-voltage live-line working robot, characterized in that, Includes the following steps: Step 1: The live-line working platform moves to the predetermined working position. Both the main robotic arm and the auxiliary robotic arm are equipped with vision modules. The main robotic arm, equipped with the wire stripper front-end actuator, arrives at the preparatory position of the phase to be worked on. The high-voltage live-line working robot identifies and locates the main line through a vision algorithm and completes the insulation removal operation on the main line using the wire stripper. Step 2: The main robotic arm switches the front-end actuator to the lead wire fastening device, and the auxiliary robotic arm, equipped with the lead wire clamp front-end actuator, moves to the lead wire gripping preparation position. The visual algorithm determines the lead wire posture and grips the rear end of the lead wire from the insulation part. The specific process of the visual algorithm in step 2 is as follows: A short segment of the power line is rigid; a segment of the power line with its insulation removed is treated as a rigid body. The auxiliary robotic arm, carrying the lead wire fastening device and vision module, moves to the lead wire grasping preparation position, i.e., the side of the lead wire, and uses the vision module to capture the lead wire connection operation scene. First, the captured image of the lead wire connection operation scene is fed into the network. The network predicts the features of the lead wire with the insulation removed and performs axial expansion of the target pixels at the pixel level. The pose estimation network calculates the semantic segmentation information of the target and the vector field pointing to the target key points. The target key points are calculated from the direction vector field using RANSAC Voting, and the mean and covariance of the spatial distribution of the key points are generated. Finally, using the EPnP algorithm, the position of the target lead wire key points in the image in the vision module is known. The pixel coordinates of the target lead wire key points are converted into coordinates in the vision module coordinate system using the intrinsic parameter matrix of the vision module and the perspective projection model. The coordinates in the vision module coordinate system are converted into three-dimensional coordinates in the world coordinate system using the extrinsic parameters of the vision module, enabling the high-voltage live-line working robot to perceive the posture in three-dimensional space. Step 3: The main robotic arm, equipped with the lead wire fastening device, reaches below the main line. The angle of the lead wire fastening device is adjusted according to the direction of the main line. The auxiliary robotic arm drags the lead wire to the upper side of the main line. The main robotic arm completes the work of fixing the main line and the lead wire to the lead wire fastening device. Step 3 is as follows: Step 31: The auxiliary robotic arm aligns the part of the lead wire with the part of the main wire with the part of the main wire with the insulation stripped; the posture of the main wire with the insulation stripped part A is obtained and used as a target. The auxiliary robotic arm uses the target to correct the posture of the lead wire with the insulation stripped part B, so that the posture of the insulation part B is the same as that of the insulation part A and is aligned with it. Step 32: The main robotic arm supports the wire fastening device and rises in a straight line. Relying on the funnel structure of the wire fastening device, the deinsulated part of the main wire slides into the gripper assembly. Step 33: The auxiliary robotic arm adjusts the attitude of the lead wire, slides the uninsulated part of the lead wire into the gripper assembly, and the gripper mounting base opens the clamping motor to fix the main lead wire. Step 4: The main robotic arm descends with the lead wire fastening device, clamping the wire onto the main wire and the lead wire. The main robotic arm retracts, and the auxiliary robotic arm releases the lead wire and retracts. The main and auxiliary robotic arms move to the reset position, and the live wire connection operation is completed.

2. The method for live-line connection operation of a high-voltage live-line working robot according to claim 1, characterized in that: In step 2, the auxiliary robotic arm uses a visual algorithm to determine the orientation of the lead wire and adjusts the actuator at the front end of the lead wire clamp to grasp the lead wire.

3. A method for live-line connection operation of a high-voltage live-line working robot according to claim 2, characterized in that: The visual algorithm mentioned in step 2 is the lead wire pose estimation algorithm, which is the calibration algorithm for the binocular vision module.

4. The method for live-line connection operation of a high-voltage live-line working robot according to claim 1, characterized in that: The lead wire fastening device includes: a gripper mounting base, a gripper assembly, a positioning post, and a funnel-shaped guide frame; the tightness of the gripper assembly is controlled by a motor in the gripper mounting base. The gripper mounting base assembly is connected to a high-voltage insulated extension arm of a high-voltage live-line working robot; the gripper mounting assembly, positioning column, and guide frame serve as the main body of the lead wire fastening quick-connect device, and the gripper assembly serves as the sub-body of the device.

5. A method for live-line connection operation of a high-voltage live-line working robot according to claim 4, characterized in that: In step 3, the lead wire and the main wire are on both sides of the lead wire fastening device and fall into the clamping jaw assembly of the lead wire fastening device.

6. A method for live-line connection operation of a high-voltage live-line working robot according to claim 3, characterized in that: The specific steps of the lead wire attitude estimation method are as follows: the uninsulated part of the lead wire is denoted as insulation part B, and the uninsulated part of the main wire is denoted as insulation part A; S1. The auxiliary robotic arm, carrying the lead wire fastening device and vision module, moves to the lead wire grasping preparation position, i.e., the side of the lead wire, and uses the vision module to capture the lead wire receiving operation scene. The image of the lead wire receiving operation scene is preprocessed to 640×640 and fed into the Darknet network. The Darknet network divides the image into an S×S grid, and each cell is responsible for detecting the insulation part B and insulation part A features whose center point falls within that cell. Anchor boxes are used to match each feature. Each cell predicts the bounding box and the confidence of the bounding box. The confidence includes the probability of the target contained in the bounding box. The accuracy of the bounding box; the accuracy of the bounding box is measured by the ratio of the predicted bounding box to the actual bounding box. To characterize; confidence level is defined as The predicted value of the bounding box is represented by (x, y, w, h, c), where (x, y) represents the center coordinates of the bounding box, w and h represent the width and height of the bounding box, and c represents its confidence score; its loss function is as follows: ； in The confidence loss represents the probability that the target is contained within the bounding box. The bounding box loss represents the accuracy of the bounding box. The classification loss represents the accuracy of different target detections; , , They represent , , The weights are assigned; finally, the predicted bounding boxes for insulating parts B and A are output; then, non-maximum suppression (NMS) is used to sort the bounding boxes, and each bounding box is traversed one by one. The bounding box with the highest confidence is selected, and the intersection-union ratio (IUU) of the remaining bounding boxes with the bounding box with the highest confidence is calculated. Suppress bounding boxes that have a high overlap with the bounding box with the highest confidence level; calculate as follows: ； ； ； in This indicates the width and height of the overlapping box. , These represent the center point coordinates of the bounding box with the highest confidence level and the currently compared bounding box, respectively. , These represent the width and height of the bounding box with the highest confidence level and the currently compared bounding box, respectively. Indicates the area of ​​the overlapping frame. This represents the total area occupied by the two bounding boxes. This indicates the intersection-union ratio of the two bounding boxes; according to The calculation results are used to determine the degree of overlap between the two bounding boxes. If the value is greater than the preset value, suppression is performed; the best bounding box is selected by NMS calculation and comparison. S2. Perform axial pixel expansion on insulating parts A and B; convert the image into a binary image with a black background (=0) and a target rectangular region pixel value (=255). Using the principle of erosion and dilation, erode the rectangular mask along the left and right sides of the width. After multiple iterations, find the central axis along the length of the rectangle; expand the target pixel area through the central axis. S3. Use a pose estimation network to predict the orientation of pixels, that is, to perform semantic segmentation and vector field prediction on pixels; For each pixel p, the network outputs a semantic label that associates it with a specific object and two-dimensional keypoints representing the connection from pixel p to the object. The direction vector Direction vector The definition is as follows: ； in The direction vector representing the key point. , This represents the pixel coordinates of the key point and pixel P. express The normal distance to P; Key points for generating hypotheses based on the RANSAC strategy: First, obtain the relevant pixels of the target through semantic segmentation labels; then, randomly select the direction vectors of two pixels and use their intersection as keypoints. The assumption Repeat N times to generate a set of hypotheses Then assume the voting score Defined as: ； in Indicates the first Each vote score Indicates the first Assumptions of key points, Indicates the index function, This represents the transpose matrix of the direction vectors of the hypothetical point and the target pixel. express The normative distance to P, It is a threshold. Represents pixels Belongs to the target ; in Indicates the index function, It is a threshold. Represents pixels Belongs to the target ;when A larger value indicates a more persuasive hypothesis; the resulting hypothesis characterizes the spatial probability distribution of keypoints in the image, with the mean and covariance of the keypoints as follows: ； ； in It means, It is covariance; Indicates the first Each vote score It is the first Assumptions of key points; Given the 2D keypoint positions of each target, its 6D pose is calculated using the PnP algorithm. The PnP algorithm takes a set of known 2D-3D point pairs as input and solves for the rotation and translation matrix of the vision module by minimizing the reprojection error. Its expression is as follows: ； in It is a point in the real world. These are the corresponding points on the image. These are intrinsic parameters of the vision module. and It's about scaling the focal length. It's the skew parameter. It is an optical center. It is the scaling factor of the image points. It is the rotation and translation matrix of the visual module; the EPnP algorithm is used instead of the PnP algorithm; The key points are solved using the EPNP algorithm, given... average Covariance Matrix Its 6D pose is calculated by minimizing the Mahalanobis distance. ; ； in These are the three-dimensional coordinates of the key points. Yes Two-dimensional projection, It is a perspective projection function; EPNP adjusts the parameters based on four key points. Initialization is performed to minimize the covariance matrix trajectory; the Levenberg-Marquardt algorithm is used to reduce reprojection error; and the robot is enabled to perceive the attitude information of insulating parts A and B.

Images