Vision-guided automated screw tightening robot operation method

Abstract

This invention relates to a vision-guided automatic screw tightening robot method, which solves the technical problems of how to more accurately align the end effector with the bolt and how to improve the reliability of automatic bolt tightening operations in existing high-voltage line maintenance robots. First, a mobile vehicle moves to the area to be repaired. Second, a detection result image is output using a YOLOv8 target detection model, and the center coordinates of the target bolt are obtained from the detection result image. Next, coordinate transformation is performed to obtain the spatial position of the target bolt in the coordinate system of the six-axis collaborative robot arm base. Then, the robot arm is controlled to move the end effector to this spatial position. Finally, the end effector clamps the target bolt, and the end flange of the six-axis collaborative robot arm rotates to tighten the target bolt. This invention is widely used in bolt maintenance on high-voltage transmission lines.

Description

Technical Field

[0001] This invention relates to the field of surveying and maintenance technology for overhead high-voltage transmission lines, and more specifically, to a vision-guided automatic screw tightening robot operation method. Background Technology

[0002] Overhead high-voltage transmission lines are the main facilities for power distribution in my country. They require preventative maintenance, including regular comprehensive surveys and repairs to prevent potential problems.

[0003] Currently, preventative maintenance of high-voltage transmission lines relies primarily on manual labor. Trained personnel move along cables tens of meters above the ground to inspect and repair external damage, such as wire corrosion or loose pins. This typically requires workers to operate while the power is on. Live-line work is the most direct and effective method of transmission line maintenance, reducing power outages and improving power supply safety and reliability. However, this also increases the risk of electric shock for workers. For safety, workers need to wear heavy electromagnetic shielding suits, which increases the intensity of the work and reduces efficiency. Furthermore, relying solely on protective clothing cannot completely eliminate all safety hazards. Besides the most common risk of electric shock, workers also face injuries from falls, cuts from damaged components, collisions with structural parts, and falling tools.

[0004] Towers are the primary targets for surveying and maintenance. Crossarms are crucial components of towers, secured with bolts. Besides crossarms, other iron accessories on the tower, such as clamps and fork beams, are also fixed with bolts. Tightening these bolts is necessary to prevent them from loosening.

[0005] To reduce the accident risks for on-site line operators, the solution of using line maintenance robots to replace manual labor has advantages such as high efficiency, good operability, high flexibility, no terrain restrictions, and reduced casualties. It can effectively tighten bolts. Referring to the invention patent application CN116914623A, entitled "Transmission Line Repair Robot Based on Multi-Axis Robotic Arm," this robot moves forward by rotating wheels that are attached to the transmission line, and automatically performs the work at the target location.

[0006] Referring to the invention patent application with publication number CN115498550A, a vertical lifting bolt tightening mechanism for live-line working of power distribution networks is disclosed, in which the end effector adopts a sleeve structure. However, how to more accurately align the end effector with the bolt and how to improve the reliability of automatic bolt tightening operation are technical problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0007] This application aims to address the technical challenges of how to more accurately align the end effector with bolts and improve the reliability of automatic bolt tightening operations using existing high-voltage line maintenance robots. It provides a vision-guided automatic screw tightening robot operation method.

[0008] This application provides a vision-guided automatic screw tightening robot operation method. The automatic screw tightening robot includes a mobile vehicle, a six-axis collaborative robotic arm, an end effector, and a depth camera. The six-axis collaborative robotic arm is fixedly connected to the mobile vehicle and has an end flange. The end effector is connected to the end flange of the six-axis collaborative robotic arm, and the depth camera is connected to the end effector through a camera bracket. The operation method includes the following steps:

[0009] The first step is to construct a YOLOv8 target detection model for bolts;

[0010] The second step involves moving the mobile vehicle along the two power lines to the tower position, then the six-axis collaborative robotic arm moves the end effector to the initial working area close to the target bolt; then the depth camera acquires RGB and depth images of the target bolt.

[0011] Preprocess the RGB image;

[0012] The preprocessed RGB image is input into the YOLOv8 object detection model, and the YOLOv8 object detection model outputs the detection result image.

[0013] The center coordinates of the target bolt are obtained from the detection result image;

[0014] The third step is coordinate transformation;

[0015] The target bolt center coordinates obtained in the second step are converted into the absolute spatial position in the six-axis collaborative robot arm base coordinate system;

[0016] The fourth step is to control the movement of the robotic arm and move the end effector to its absolute spatial position.

[0017] Fifth step: The end effector moves to clamp the target bolt;

[0018] The sixth step is to rotate the end flange of the six-axis collaborative robotic arm to tighten the target bolts.

[0019] Preferably, the YOLOv8 target detection model is the YOLOv8-Seg target detection model; in the second step, the process of obtaining the center coordinates of the target bolt based on the detection result image is to calculate the center coordinates of the target bolt mask.

[0020] Preferably, the end effector includes a housing, a flange bearing housing, a rear support block, a connector, a drive motor, a first-stage bevel gear, a second-stage bevel gear, a worm gear, a left bearing, a right bearing, an integrated worm gear transmission component, a support shaft, a rear bearing, a front bearing, a disc, and six sets of clamping mechanisms. The housing includes a housing body and a circular front cover plate. The circular front cover plate is fixedly connected to the front of the housing body, the flange bearing housing is fixedly connected to the rear of the housing body, the rear support block is fixedly connected to the rear of the housing body, and the connector is fixedly connected to the rear support block. The connector is provided with a flange. The drive motor is connected to the flange bearing housing, and the output shaft of the drive motor passes through the flange bearing housing. The first-stage bevel gear, the second-stage bevel gear, the worm gear, and the integrated worm gear transmission component are located inside the housing body.

[0021] The first-stage bevel gear is connected to the output shaft of the drive motor, the second-stage bevel gear is connected to the worm gear, and the first-stage bevel gear meshes with the second-stage bevel gear. One end of the worm gear is rotatably connected to the housing through the left bearing, and the other end of the worm gear is rotatably connected to the housing through the right bearing. The rear end of the support shaft is rotatably connected to the rear support block through the rear bearing.

[0022] The rear end of the worm gear integrated transmission component is provided with a worm gear part and a bearing positioning groove, the front end of the worm gear integrated transmission component is provided with a circular cavity, the inner side wall of the circular cavity is provided with an internal gear ring part, and the circular cavity is provided with a disc limiting groove.

[0023] The disc is placed in the disc limiting groove of the worm gear integrated transmission component. The disc can slide relative to the worm gear integrated transmission component. The disc has six bearing holes evenly distributed along the circumference.

[0024] The front bearing is set in the bearing positioning groove of the worm gear integrated transmission component, and the front end of the support shaft is connected to the front bearing; the worm meshes with the worm wheel part of the worm gear integrated transmission component;

[0025] The clamping mechanism includes a cam positioning shaft, a cam, a cylindrical gear, a clamping shaft, a front bearing, and a rear bearing. The cylindrical gear is fixedly connected to the cam positioning shaft, the cam is fixedly connected to the cam positioning shaft, and the clamping shaft is fixedly connected to the cam. The front bearing is connected to the front end of the cam positioning shaft, and the rear bearing is connected to the rear end of the cam positioning shaft. The rear bearing is connected to the bearing hole of the disc. The circular front cover plate has a central window and six bearing chambers, which are located around the central window and evenly distributed along the circumference. The front bearing of the clamping mechanism is set in the bearing chamber. The cylindrical gear of the clamping mechanism meshes with the internal gear ring of the integrated worm gear transmission component. The clamping shaft of the clamping mechanism extends from the central window of the circular front cover plate.

[0026] Preferably, in the fourth step, the movement of the robotic arm is based on discrete path points obtained by global obstacle avoidance path planning of the robotic arm using the RRT-Connect algorithm.

[0027] Preferably, the process of global obstacle avoidance path planning for the robotic arm based on the RRT-Connect algorithm is as follows:

[0028] Define the target angles that each joint of the robotic arm needs to rotate. The current joint angle of the robotic arm is First, a fifth-order polynomial is used to interpolate and plan the joint angles, setting the start time of each trajectory segment as... The termination time is Joint position The function that changes with time is defined as follows:

[0029]

[0030] The position, velocity, and acceleration constraints for the initial and final states are set as follows:

[0031] The initial state is:

[0032] ,

[0033] The termination status is:

[0034] ,

[0035] Constructing the solution coefficients The system of linear matrix equations is shown below:

[0036] ;

[0037] By representing the target angle of the final target attitude Substituting the above equations into the system of equations, we solve the linear system and the coefficient vector is obtained. This allows us to obtain the desired angles of each joint that are continuous and smooth along the time axis. And the corresponding expected angular velocity and angular acceleration curves;

[0038] The robotic arm's joint motors are driven by a position closed-loop PID control method, and the actual angle feedback from the encoders of each joint is collected in real time. and compare it with the expected angle Compare and calculate the angle tracking error. :

[0039] ;

[0040] Based on this error signal, the controller performs linear weighting with the proportional, integral, and derivative terms respectively, and finally calculates the torque control amount required to drive each joint. :

[0041] ;

[0042] In the formula, This represents the proportional gain coefficient. This represents the integral adjustment gain coefficient; This represents the differential adjustment gain coefficient.

[0043] The beneficial effect of this application is that the robot can more accurately identify the position of the target bolt, and under the movement of the six-axis robotic arm, the end effector can more accurately dock with the bolt, enabling more reliable automatic bolt tightening operations.

[0044] The end effector adaptively locks the target bolt, making it suitable for bolts of various sizes. The encircling clamping mechanism facilitates more accurate locking of the target bolt.

[0045] It has obstacle avoidance capabilities and can adapt to complex on-site environments.

[0046] It has a higher level of intelligence.

[0047] Further features and aspects of the present invention will be clearly described in the following detailed description with reference to the accompanying drawings. Attached Figure Description

[0048] Figure 1 This is a structural diagram of the robot;

[0049] Figure 2 yes Figure 1 The diagram shows the camera coordinate system, the end effector flange coordinate system, and the base coordinate system involved in the control process of the robot shown.

[0050] Figure 3 This is a flowchart of a vision-guided automatic screw tightening robot operation method;

[0051] Figure 4 This is the architecture diagram of the improved YOLOv8-Seg object detection model;

[0052] Figure 5 This is an isometric view of the end effector;

[0053] Figure 6 yes Figure 5 The front view of the end effector is shown;

[0054] Figure 7 yes Figure 5 Top view of the end effector shown;

[0055] Figure 8 yes Figure 7 A cross-sectional view along the AA direction;

[0056] Figure 9yes Figure 5 The diagram shows the internal transmission mechanism of the end effector.

[0057] Figure 10 yes Figure 5 The diagram shown is a structural diagram of the end effector after removing the rear support block, connector, and drive motor.

[0058] Figure 11 yes Figure 10 An axonometric view of the structure shown from another perspective;

[0059] Figure 12 yes Figure 10 The internal transmission mechanism structure diagram shown is shown.

[0060] Figure 13 This is a structural diagram of the transmission mechanism;

[0061] Figure 14 yes Figure 13 An axonometric view of the structure shown from another perspective;

[0062] Figure 15 This is a structural diagram showing the connection between the rear support block and the connecting piece, with a support shaft mounted on the rear support block;

[0063] Figure 16 It is an isometric drawing of the integrated worm gear transmission component;

[0064] Figure 17 yes Figure 16 Another isometric view of the integrated worm gear transmission component shown;

[0065] Figure 18 yes Figure 15 The diagram shows the structure in which the support shaft is connected to the rear support block via a bearing.

[0066] Figure 19 This is a schematic diagram of a structure in which a disc is placed in a circular cavity at the front end of an integrated worm gear transmission component;

[0067] Figure 20 yes Figure 13 The structure shown is a diagram in which the disk is placed in the integrated worm gear transmission component, and the cam positioning shaft of the clamping mechanism is connected to the disk through the rear bearing.

[0068] Figure 21 yes Figure 13 The diagram shows the connection relationships of the cam positioning shaft, cylindrical gear, cam, front bearing, and rear bearing in the clamping mechanism.

[0069] Figure 22 It is an axonometric drawing of the circular front cover;

[0070] Figure 23This is a schematic diagram of a structure with six clamping axes surrounding the smallest opening;

[0071] Figure 24 This is a schematic diagram of a structure with six clamping axes surrounding the maximum opening;

[0072] Figure 25 This is a diagram showing the state of the end effector clamping the target bolt.

[0073] Explanation of symbols in the diagram:

[0074] 1. Housing; 1-1. First fixing post; 1-2. Second fixing post; 1-3. Camera bracket connecting part; 2. Circular front cover plate; 2-1. Central window; 2-2. Bearing chamber; 2-3. Receiving groove; 3. Screw; 4. Flange bearing seat; 5. Rear support block; 6. Connecting piece; 6-1. Flange; 7. Drive motor; 8. First-stage bevel gear; 9. Second-stage bevel gear; 10. Worm gear; 11. Left bearing; 12. Right bearing; 13. Integrated worm gear transmission component; 13-1. Worm gear part; 13-2. Internal gear ring part; 13-3. Bearing positioning groove; 13-4. Central boss. 13-5. Disc limiting groove; 14. Support shaft; 15. Rear bearing; 16. Front bearing; 17. Cam positioning shaft; 18. Cam; 19. Cylindrical gear; 20. Clamping shaft; 21. Disc; 22. Front bearing; 23. Rear bearing; 24. Camera; 25. Bolt; 100. Mobile vehicle; 101. Frame; 102. First traveling wheel; 103. Second traveling wheel; 104. Third traveling wheel; 105. Fourth traveling wheel; 200. Six-axis collaborative robotic arm; 201. End flange; 300. End effector; 400. Depth camera; 500. Camera bracket. Detailed Implementation

[0075] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] The specific embodiments described below are merely preferred embodiments of this application, and the scope of protection of this application is not limited thereto. Those skilled in the art can make modifications or variations based on the principles, concepts, and spirit of this application, and the resulting technical solutions should all be covered within the scope of protection of this application.

[0077] like Figure 1As shown, the automatic screw tightening robot includes a mobile vehicle 100, a six-axis collaborative robotic arm 200, an end effector 300, and a depth camera 400. The mobile vehicle 100 includes a frame 101, a first traveling wheel 102, a second traveling wheel 103, a third traveling wheel 104, a fourth traveling wheel 105, and a traveling wheel drive motor. The mobile vehicle 100 is placed on two power lines, with the first traveling wheel 102 and the third traveling wheel 104 pressing on the first power line, and the second traveling wheel 103 and the fourth traveling wheel 105 pressing on the second power line. Then, the traveling wheel drive motor is started, and the first traveling wheel 102, the second traveling wheel 103, the third traveling wheel 104, and the fourth traveling wheel 105 rotate, thereby moving the mobile vehicle 100 on the two power lines.

[0078] Since the six-axis collaborative robotic arm 200 is fixedly mounted on the frame 101, the movement of the mobile vehicle 100 takes the six-axis collaborative robotic arm 200 with it.

[0079] The end effector 300 is fixedly mounted on the end flange of the six-axis collaborative robot arm 200, and the end flange 201 of the six-axis collaborative robot arm 200 is the sixth joint of the robot arm.

[0080] The camera bracket 500 is fixedly connected to the end effector 300, and the depth camera 400 is fixedly connected to the camera bracket 500.

[0081] like Figures 5 to 24 As shown, the end effector 300 includes a housing, a flange bearing seat 4, a rear support block 5, a connector 6, a drive motor 7, a first-stage bevel gear 8, a second-stage bevel gear 9, a worm gear 10, a left bearing 11, a right bearing 12, an integrated worm gear transmission component 13, a support shaft 14, a rear bearing 15, a front bearing 16, six clamping mechanisms, and a disc 21. The housing includes a body 1 and a circular front cover 2, which is fixedly mounted on the front of the body 1 by screws 3. The flange bearing seat 4 is fixedly connected to the rear of the body 1. The rear of the body 1 is provided with a first fixing post 1-1 and a second fixing post 1-2, and the rear support block 5 is fixedly connected to the first fixing post 1-1 and the second fixing post 1-2. The connector 6 is fixedly connected to the rear support block 5, and the connector 6 is provided with a flange 6-1. The drive motor 7 is fixedly mounted on the flange bearing housing 4, which is equipped with a bearing. The output shaft of the drive motor 7 passes through the bearing in the flange bearing housing 4 and rotates under the support of the bearing. The primary bevel gear 8, the secondary bevel gear 9, the worm gear 10, and the integrated worm gear transmission component 13 are located inside the housing 1. The camera bracket connection part 1-3 on the housing 1 is used to mount the depth camera.

[0082] The first-stage bevel gear 8 is connected to the output shaft of the drive motor 7, and the second-stage bevel gear 9 is connected to the worm gear 10. The first-stage bevel gear 8 and the second-stage bevel gear 9 mesh. One end of the worm gear 10 is rotatably connected to the housing 1 via the left bearing 11, and the other end of the worm gear 10 is rotatably connected to the housing 1 via the right bearing 12. The integrated worm gear transmission component 13 has a worm wheel part 13-1 and an internal gear ring part 13-2, and the worm gear 10 meshes with the worm wheel part 13-1. The rear end of the support shaft 14 is rotatably connected to the rear support block 5 via the rear bearing 15, and the integrated worm gear transmission component 13 is connected to the front end of the support shaft 14 via the front bearing 16.

[0083] The worm gear integrated transmission component 13 has a worm gear portion 13-1 at its rear end, a bearing positioning groove 13-3 at the middle of its rear end, and a circular chamber at its front end. The inner wall of this circular chamber has an internal gear ring portion 13-2, and the circular chamber also has a disc limiting groove 13-5. The front bearing 16 is positioned in the bearing positioning groove 13-3.

[0084] The disc 21 is placed in the disc limiting groove 13-5 of the worm gear integrated transmission component 13, and the disc 21 can slide relative to the worm gear integrated transmission component 13 in the radial direction. The disc 21 is provided with six bearing holes for mounting bearings.

[0085] The clamping mechanism includes a cam positioning shaft 17, a cam 18, a cylindrical gear 19, a clamping shaft 20, a front bearing 22, and a rear bearing 23. The cylindrical gear 19 is fixedly connected to the cam positioning shaft 17, the cam 18 is fixedly connected to the cam positioning shaft 17, and the clamping shaft 20 is fixedly connected to the cam 18. The inner ring of the front bearing 22 is connected to the front end of the cam positioning shaft 17, and the inner ring of the rear bearing 23 is connected to the rear end of the cam positioning shaft 17. There are a total of six clamping mechanisms. The rear bearings 23 are installed in the bearing holes of the disc 21, and six rear bearings 23 are evenly distributed along the circumference on the disc 21. The disc 21 supports the cam positioning shaft 17 through the rear bearings 23.

[0086] The circular front cover plate 2 has a central window 2-1 and six bearing chambers 2-2, which are located around the central window 2-1. When assembling the clamping mechanism, the front bearings 22 are placed in the bearing chambers 2-2, and the circular front cover plate 2 supports the cam positioning shaft 17 via the front bearings 22. Since the six bearing chambers 2-2 are evenly distributed circumferentially, the six front bearings 22 are also evenly distributed circumferentially. The cylindrical gear 19 of the clamping mechanism meshes with the internal gear ring portion 13-2 of the worm gear integrated transmission component 13. The circular front cover plate 2 is fixedly connected to the housing 1, and the clamping shafts 20 of the clamping mechanism extend from the central window 2-1, forming a structure with six clamping shafts 20 evenly distributed circumferentially.

[0087] A central boss 13-4 is provided in the circular cavity at the front end of the worm gear integrated transmission component 13. The function of the central boss 13-4 is to provide a contact surface for the target bolt. When the actuator moves to the target bolt and the six clamping shafts 20 surround the target bolt, if the displacement of the actuator is too large, the target bolt will abut against the central boss 13-4, ensuring that the target bolt is within the enclosure range of the six clamping shafts 20 and improving reliability. Six receiving grooves 2-3 can be provided on the circular front cover plate 2, and the front end of the cam positioning shaft 17 is located in the receiving grooves 2-3, which provide space.

[0088] When the drive motor 7 is started, its output shaft drives the first-stage bevel gear 8 to rotate. The first-stage bevel gear 8, through the second-stage bevel gear 9, drives the worm gear 10 to rotate. The worm gear 10 then drives the integrated worm gear transmission component 13 to rotate. The internal gear ring 13-2 acts on the cylindrical gear 19 of the clamping mechanism, causing the cylindrical gear 19 to rotate. The cam positioning shaft 17 then drives the cam 18 to rotate, and the clamping shaft 20 rotates along with the cam 18. This allows the six clamping shafts 20 to gradually converge and contract, reducing the opening formed by the six clamping shafts. Reversing the drive motor 7 allows the six clamping shafts 20 to gradually expand, increasing the opening formed by the six clamping shafts. The movement of the six clamping shafts 20 is a stepless motion. Figure 23 As shown, the opening formed by the six clamping axes is the smallest. Figure 24 As shown, the opening formed by the six clamping axes is the largest. Figure 25 As shown, the six clamping shafts 20 gradually converge and contract to clamp the hexagonal head of the bolt 25. The six clamping shafts 20 clamp the six sides of the hexagonal head respectively, thereby tightening the hexagonal head of the bolt. It can be seen that the six clamping shafts 20 can adaptively lock the hexagonal head of the bolt, and the movement is stepless and gradual, which can adapt to bolts of different sizes.

[0089] refer to Figure 3 The vision-guided automatic screw tightening robot operation method includes the following steps:

[0090] The first step is to construct a bolt target detection model.

[0091] The YOLOv8 object detection model was adopted and trained using a large dataset.

[0092] Furthermore, the YOLOv8-Seg object detection model can be used. The process of training the YOLOv8-Seg object detection model is as follows:

[0093] First, a bolt dataset for high-voltage transmission line environments was constructed by capturing images of bolts on towers under different lighting conditions and angles. Roboflow annotation tool was used to annotate the hexagonal heads of the bolts in the images with polygonal shapes. Data augmentation was performed using the brightness and contrast adjustment, rotation, flipping, and cropping operations provided by Roboflow annotation tool to expand the sample size. The training, validation, and test sets were then divided in a 7:2:1 ratio. During the training phase, the input image size was set to 1280×1280, the SGD optimizer was selected, and the initial learning rate was set to 0.01 with a momentum factor of 0.937. The training process employed a joint loss function for optimization, including classification loss, bounding box regression loss, and mask segmentation loss. After 500 iterations, the weight file with the highest mAP value on the validation set was selected as the final inference model.

[0094] The YOLOv8-Seg object detection model can be improved. The improved YOLOv8-Seg object detection model includes a backbone network, a neck network, and a head network. The backbone network is responsible for feature extraction from the input image. On top of the CSPDarknet architecture, a C2f module (CSP Bottleneck with 2 convolutions) is used to replace the traditional C3 module to enrich the gradient flow. Furthermore, to address the issues of high background noise on high-voltage power line towers and weak texture of bolt targets, a hybrid attention mechanism module is embedded after the C2f module in the backbone network. This module adaptively suppresses background response and enhances the extraction capability of hexagonal bolt texture features through dual weighting of channel and spatial dimensions. At the end, an SPPF module is used to expand the receptive field to adapt to target scale changes at different shooting distances. Thus, the backbone network includes a first Conv module, a first C2f module, a first hybrid attention mechanism module, a second Conv module, a second C2f module, a second hybrid attention mechanism module, and an SPPF module.

[0095] The neck network employs a PANet (Path Aggregation Network) structure to achieve bidirectional fusion of deep semantic information and shallow texture information. The network concatenates feature maps of different scales along the channel dimension through a top-down upsampling path and a bottom-up convolutional path. This structure ensures that the three-layer feature maps output by the network, namely P3, P4, and P5, retain both high-resolution geometric information beneficial for small object localization and deep semantic features conducive to interference resistance.

[0096] The head network employs an anchor-free decoupled design. For each feature map scale (P3, P4, P5), it includes two parallel branches: a detection branch and a segmentation branch. The detection branch outputs the bounding box coordinates and class confidence of the bolt, achieving coarse target localization. The segmentation branch does not directly output the semantic category but predicts a set of mask coefficients. Simultaneously, the network utilizes the highest-resolution P3 branch to generate a globally shared prototype mask.

[0097] After deployment, the YOLOv8-Seg target detection model or its improved version can obtain the high-precision center of the hexagonal head of a bolt when applied.

[0098] The second step is to perform a rough positioning intervention.

[0099] The mobile trolley 100 is placed on two power lines and moves along them. Once it reaches the crossarm or other metal attachment position on the tower, the six-axis collaborative robotic arm 200 moves its end effector to an initial working area approximately 500mm-800mm from the target bolt. Then, the controller instructs the depth camera 400 to activate, which acquires RGB and depth images of the target area and sends them to the controller.

[0100] To eliminate interference from strong light or shadows at high altitudes, the controller preprocesses the RGB image, specifically by employing a contrast-limited adaptive histogram equalization algorithm to perform grayscale stretching on the L channel of the RGB image. This algorithm, by segmenting the image into blocks and calculating local histograms, significantly enhances the gradient features of bolt edges while suppressing background noise interference from high-voltage power line towers, thus providing high-quality input data for subsequent segmentation models.

[0101] The preprocessed RGB image is input into the improved YOLOv8-Seg object detection model, which outputs the detection result image and generates a pixel-level binary mask for the target bolt hexagonal head. The mask accurately depicts the geometric outline of the bolt's hexagonal head, effectively eliminating background pixels.

[0102] Calculate the geometric centroid of the hexagonal head mask using the principle of image moments. Using this as the precise pixel center of the bolt surface, the calculation formula is as follows:

[0103] , ;

[0104] In the formula, This indicates the total number of pixels in the mask. This represents the sum of the x-coordinates of all pixels in the mask. This represents the sum of the vertical coordinates of all pixels in the mask. This centroid calculation method allows for the determination of the center based on the overall geometry of the bolt, effectively eliminating the problem of bounding box center offset caused by tilted shooting angles or localized reflections.

[0105] The image pixel center coordinates obtained so far That is, the center coordinates of the target bolt.

[0106] The third step is coordinate transformation.

[0107] refer to Figure 2 The camera coordinate system is (X) camera ,Y camera Z camera The coordinate system of the end effector flange of the robotic arm is (X). end ,Y end Z end The base coordinate system is (X). base ,Y base Z base ).

[0108] In obtaining the pixel center of the bolt hexagon head Then, combining the camera's intrinsic parameter matrix with the depth value corresponding to the center point, the pinhole imaging model is used to determine the center coordinates of the image pixels. Convert to 3D spatial coordinates in camera coordinate system Let the depth value corresponding to the center point acquired by the depth camera be... The camera intrinsic parameter matrix is Then the coordinates in the camera coordinate system It is calculated using the following formula:

[0109]

[0110] In the formula, , For camera focal length, , The principal point coordinates are then determined. Next, through pre-performed hand-eye calibration, the transformation matrix of the camera coordinate system relative to the robotic arm's end flange coordinate system is obtained:

[0111]

[0112] The transformation matrix includes the rotation matrix. Translation vector , This is a 3×3 rotation matrix describing the attitude deflection relationship between the depth camera coordinate system and the end effector flange coordinate system of the robotic arm. This is a 3×3 translation vector describing the spatial offset of the depth camera's optical center relative to the center of the robotic arm's end flange. This translation vector is determined by the physical dimensions and installation position of the camera mount 500. At this point, the three-dimensional spatial coordinates in the camera coordinate system are... Convert to three-dimensional spatial coordinates in the end-effector coordinate system of the robotic arm Coordinates in the end flange coordinate system Represented as:

[0113]

[0114] Finally, the system reads the current angles of each joint of the robotic arm. The forward kinematics equations are constructed using the DH parameter table. The homogeneous transformation matrix of the end effector flange coordinate system relative to the base coordinate system is also shown. Defined as the product of the transformation matrices of each link, as shown in the following equation:

[0115]

[0116] in, This represents the transformation matrix of the i-th link coordinate system relative to the (i-1)-th link coordinate system. Its specific form is determined by the link parameters (joint angles). Linkage offset Link length Linkage torsion angle The decision is as follows:

[0117]

[0118] By calculating the matrix product mentioned above, the pose matrix of the end flange center in the base coordinate system can be obtained. Then, the target bolt coordinates can be uniformly transformed to the absolute spatial position in the robot arm base coordinate system using the following formula. :

[0119]

[0120] The fourth step is to control the movement of the robotic arm and move the end effector to the target bolt.

[0121] The controller calls the inverse kinematics solver to calculate the required rotation angles of each joint of the robotic arm, and uses this to guide the output of the drive motors at each joint. The robotic arm then begins to move, moving the end effector 300 to its absolute spatial position. That is, move to the target bolt.

[0122] If there are obstacles on the robotic arm, global obstacle avoidance path planning is performed based on the RRT-Connect algorithm to avoid them. First, two search trees are initialized, one being the initial tree rooted at the current joint angle of the robotic arm. The target tree with the joint angles corresponding to the target pose as the root nodes. In each iteration k of the algorithm, the two trees are expanded alternately. Specifically, a random sampling point is generated within the joint space of the robotic arm. And traverse the current tree to find the distance The nearest node Next, control the direction of tree growth, from Towards Direction with fixed step size Extend the algorithm to find new candidate nodes. The calculation process is shown in the following formula:

[0123]

[0124] In generation During the process, the system synchronously calls the AABB collision detection algorithm to simplify the links of the robotic arm and obstacles such as high-voltage towers into bounding box models, and determines whether the generated path segments interfere with the environment. If no collision is detected, then... It is officially added to the current search tree. A new node is added when a tree successfully expands to include a new node. Then, the Greedy Connect strategy is immediately triggered. At this point, the other tree no longer undergoes random sampling, but instead directly uses the tree just generated. The target gravity point is iteratively expanded continuously. This greedy expansion process continues until an obstacle collision is detected or the distance between the ends of two trees is less than a preset convergence threshold. When the following conditions are met:

[0125]

[0126] The system determines that the two search trees have been successfully connected and then stops the search. Finally, the algorithm backtracks from the connection point of each tree back to the starting point. and target point The extracted two path point sequences are then concatenated to form a complete initial collision-free joint space path. .

[0127] To more accurately align the end effector with the target bolt, the central axis of the end effector and the axis of the target bolt must be strictly collinear in space. This is achieved by defining the target rotation angles required for each joint of the robotic arm. In order to ensure that the robotic arm moves from the current joint angle Smooth transition to the target angle To avoid mechanical vibration caused by sudden acceleration changes, the discrete path point set is first processed by time parameterization. A fifth-order polynomial is then used for interpolation planning of the joint angles, and the start time of each trajectory segment is set as... The termination time is Joint position The function that changes with time is defined as follows:

[0128]

[0129] Based on the boundary constraints of the robotic arm's motion, in order to achieve smooth start-stop control, the position, velocity, and acceleration constraints for the initial and final states are set as follows:

[0130] The initial state is:

[0131] ,

[0132] The termination status is:

[0133] ,

[0134] Based on the above boundary conditions, construct the solution coefficients. The system of linear matrix equations is shown below:

[0135] ;

[0136] By representing the target angle of the final target attitude Substituting the above equations into the system of equations, we solve the linear system and the coefficient vector is obtained. This allows us to obtain the desired angles of each joint that are continuous and smooth along the time axis. The corresponding expected angular velocity and angular acceleration curves are used to ensure that the robotic arm achieves no sudden changes in velocity and continuous acceleration as it approaches the target bolt.

[0137] After completing the trajectory planning, in order to ensure that each joint of the robotic arm can accurately track the desired trajectory curve, To eliminate lag and steady-state errors during actual movement, a position closed-loop PID control method is used to drive the joint motors of the robotic arm, and the actual angle feedback from the encoders of each joint is collected in real time. and compare it with the expected angle Compare and calculate the angle tracking error. :

[0138] ;

[0139] Based on this error signal, the controller performs linear weighting with the proportional, integral, and derivative terms respectively, and finally calculates the torque control amount required to drive each joint. Its mathematical control law model is shown in the following equation:

[0140] ;

[0141] In the formula, This represents the proportional gain coefficient, used to respond to the current error; This represents the integral adjustment gain coefficient, used to eliminate steady-state accumulated error; This represents the differential adjustment gain coefficient, used to predict error trends and suppress overshoot.

[0142] Through the aforementioned closed-loop control, the system dynamically generates control torque, driving each joint to move continuously along the planned trajectory, thereby achieving precise tracking and autonomous operation of the end effector along the desired path. At this point, each joint has precisely reached the target angle calculated from inverse kinematics. The axis of the end flange of the robotic arm naturally coincides with the axis of the target bolt, and the distance from the bolt head is known. It only needs to move in a straight line along the Z-axis of the coordinate system of the end flange of the robotic arm to move to the target bolt. In the initial state, the six clamping axes of the end effector 300 are in the maximum opening state (the opening diameter is larger than the diagonal diameter of the hexagonal head of the target bolt), so the six clamping axes completely surround the target bolt.

[0143] The fifth step is to activate the end effector 300 to lock the target bolt.

[0144] The end effector 300 is an adaptive clamping mechanism with variable diameter functionality. It compensates for errors caused by visual positioning through an opening-envelope and closing-locking method. When the drive motor of the end effector 300 operates, the opening between the six clamping shafts gradually decreases, and the six clamping shafts gradually come into contact with and clamp the hexagonal head of the target bolt. The operating current of the drive motor can be monitored. To control the clamping force, when the current It grows larger and exceeds the preset clamping threshold. If the bolt is seized, the drive motor will immediately stop working.

[0145] The sixth step is to tighten the target bolt.

[0146] The end flange 201 of the six-axis collaborative robotic arm 200 is rotated at a certain angle to tighten the target bolt. Then, the drive motor of the end effector 300 reverses, and the six clamping axes loosen their clamping on the hexagonal head of the target bolt. Finally, the robotic arm retracts in a straight line along the Z-axis in the opposite direction to complete the operation.

[0147] It can read the real-time output torque of the 6th joint of the robotic arm. The level of the rotation can be judged by the change in torque. If it is in the engagement stage, It will remain at a low level without any sudden changes, and when it enters the bonding and tightening stage, It will rise sharply. For the set threshold, when > When the time is right, it is determined that the tightening is complete.

[0148] It should be noted that if the YOLOv8 target detection model is selected, the center of the detection box in the detection result image output by the YOLOv8 target detection model is the two-dimensional image coordinate of the target bolt.

[0149] It should be noted that a sleeve and a drive motor for driving the sleeve can be used instead of the aforementioned end effector. After the sleeve is aligned with the target bolt, the sleeve is rotated to tighten the bolt.

Claims

1. A vision-guided automatic screw tightening robot operation method, characterized in that, The automatic screw tightening robot includes a mobile vehicle, a six-axis collaborative robotic arm, an end effector, and a depth camera. The six-axis collaborative robotic arm is fixedly connected to the mobile vehicle and has an end flange. The end effector is connected to the end flange of the six-axis collaborative robotic arm, and the depth camera is connected to the end effector via a camera bracket. The operation method includes the following steps: The first step is to construct a YOLOv8 target detection model for bolts; The second step involves moving the mobile vehicle along the two power lines to the tower position, then the six-axis collaborative robotic arm moves the end effector to the initial working area close to the target bolt; then the depth camera acquires RGB and depth images of the target bolt. Preprocess the RGB image; The preprocessed RGB image is input into the YOLOv8 object detection model, and the YOLOv8 object detection model outputs the detection result image. The center coordinates of the target bolt are obtained from the detection result image; The third step is coordinate transformation; The target bolt center coordinates obtained in the second step are converted into the absolute spatial position in the six-axis collaborative robot arm base coordinate system; The fourth step is to control the movement of the robotic arm and move the end effector to its absolute spatial position. Fifth step: The end effector moves to clamp the target bolt; The sixth step is to rotate the end flange of the six-axis collaborative robotic arm to tighten the target bolts.

2. The vision-guided automatic screw tightening robot operation method according to claim 1, characterized in that, The YOLOv8 target detection model is the YOLOv8-Seg target detection model; in the second step, the process of obtaining the center coordinates of the target bolt based on the detection result image is to calculate the center coordinates of the target bolt mask.

3. The vision-guided automatic screw tightening robot operation method according to claim 2, characterized in that, The end effector includes a housing, a flange bearing seat, a rear support block, a connector, a drive motor, a primary bevel gear, a secondary bevel gear, a worm gear, a left bearing, a right bearing, an integrated worm gear transmission component, a support shaft, a rear bearing, a front bearing, a disc, and six clamping mechanisms. The housing includes a housing body and a circular front cover plate. The circular front cover plate is fixedly connected to the front of the housing body. The flange bearing seat is fixedly connected to the rear of the housing body. The rear support block is fixedly connected to the rear of the housing body. The connector is fixedly connected to the rear support block, and the connector has a flange. The drive motor is connected to the flange bearing seat, and the output shaft of the drive motor passes through the flange bearing seat. The primary bevel gear, secondary bevel gear, worm gear, and integrated worm gear transmission component are located inside the housing. The first-stage bevel gear is connected to the output shaft of the drive motor, the second-stage bevel gear is connected to the worm gear, and the first-stage bevel gear meshes with the second-stage bevel gear. One end of the worm gear is rotatably connected to the housing through a left bearing, and the other end of the worm gear is rotatably connected to the housing through a right bearing. The rear end of the support shaft is rotatably connected to the rear support block through a rear bearing. The rear end of the integrated worm gear transmission component is provided with a worm gear part and a bearing positioning groove, and the front end of the integrated worm gear transmission component is provided with a circular cavity. The inner side wall of the circular cavity is provided with an internal gear ring part, and the circular cavity is provided with a disc limiting groove. The disk is placed in the disk limiting groove of the worm gear integrated transmission component, and the disk can slide relative to the worm gear integrated transmission component. The disk is provided with six bearing holes evenly distributed along the circumference. The front bearing is set in the bearing positioning groove of the worm gear integrated transmission component, and the front end of the support shaft is connected to the front bearing; the worm meshes with the worm wheel part of the worm gear integrated transmission component. The clamping mechanism includes a cam positioning shaft, a cam, a cylindrical gear, a clamping shaft, a front bearing, and a rear bearing. The cylindrical gear is fixedly connected to the cam positioning shaft, the cam is fixedly connected to the cam positioning shaft, and the clamping shaft is fixedly connected to the cam. The front bearing is connected to the front end of the cam positioning shaft, and the rear bearing is connected to the rear end of the cam positioning shaft. The rear bearing is connected to the bearing hole of the disc. The circular front cover plate has a central window and six bearing chambers. The six bearing chambers are located around the central window and are evenly distributed along the circumference. The front bearing of the clamping mechanism is disposed in the bearing chamber. The cylindrical gear of the clamping mechanism meshes with the internal gear ring of the worm gear integrated transmission component. The clamping shaft of the clamping mechanism extends from the central window of the circular front cover plate.

4. The vision-guided automatic screw tightening robot operation method according to claim 2 or 3, characterized in that, In the fourth step, the movement of the robotic arm is based on discrete path points obtained by global obstacle avoidance path planning of the robotic arm using the RRT-Connect algorithm.

5. The vision-guided automatic screw tightening robot operation method according to claim 4, characterized in that, The process of global obstacle avoidance path planning for the robotic arm based on the RRT-Connect algorithm is as follows: Define the target angles that each joint of the robotic arm needs to rotate. The current joint angle of the robotic arm is First, a fifth-order polynomial is used to interpolate and plan the joint angles, setting the start time of each trajectory segment as... The termination time is Joint position The function that changes with time is defined as follows: The position, velocity, and acceleration constraints for the initial and final states are set as follows: The initial state is: ， The termination status is: ， Constructing the solution coefficients The system of linear matrix equations is shown below: ； By representing the target angle of the final target attitude Substituting the above equations into the system of equations, we solve the linear system and the coefficient vector is obtained. c 0 . . . c 5 T This allows us to obtain the desired angles of each joint that are continuous and smooth along the time axis. And the corresponding expected angular velocity and angular acceleration curves; The robotic arm's joint motors are driven by a position closed-loop PID control method, and the actual angle feedback from the encoders of each joint is collected in real time. and compare it with the expected angle Compare and calculate the angle tracking error. : ； Based on this error signal, the controller performs linear weighting with the proportional, integral, and derivative terms respectively, and finally calculates the torque control amount required to drive each joint. : ； In the formula, This represents the proportional gain coefficient. This represents the integral adjustment gain coefficient; This represents the differential adjustment gain coefficient.

Images