Automatic fastening system and method for overhead line system cantilever bolt

Abstract

The invention discloses an automatic fastening system and method for overhead line system cantilever bolts. The system comprises a maintenance car, a visual ranging parking unit, a laser radar sensing unit, a mechanical arm execution unit, a force feedback control unit, a tail end execution unit, a digital twin unit and an operation management platform. The method comprises the following steps: S1, guiding a maintenance car to stop below a cantilever of the overhead line system; s2, scanning an overhead line system cantilever, generating a three-dimensional model and positioning a bolt; s3, planning a collision-free path; s4, smooth alignment of the bolt sleeve is achieved; s5, bolt fastening is completed according to the preset torque; s6, synchronously displaying the operation progress, and storing whole-process operation data; and S7, performing anomaly detection and task continuous transmission on the whole process, and performing data archiving and early warning. Intelligent and automatic bolt fastening operation is achieved for the first time in the field of railway contact network maintenance, and an effective solution is provided for improving the railway operation and maintenance efficiency and safety.

Description

Technical Field

[0001] The present invention relates to the technical field of railway catenary maintenance automation, and particularly to an automatic tightening system and method for catenary wrist arm bolts. Background Art

[0002] The catenary is a core component of electrified railways, responsible for providing stable power supply to trains. Its operating status directly affects the safety and efficiency of train operation. In particular, the stability of the catenary wrist arm bolts in the traction power supply system is an important factor ensuring the reliable operation of electrified railways. Currently, the maintenance of the catenary mainly relies on manual operations. For the tightening operation of catenary wrist arm bolts, maintenance work vehicles (such as DPT-type maintenance vehicles) or manual ladder trucks are generally used.

[0003] Manual operations can only cover a limited line length in a single inspection, making it difficult to meet the large-scale railway maintenance needs and resulting in low operation efficiency. In the high-altitude operation environment, manual operations are prone to accidents due to fatigue, adverse weather conditions, or tool slippage, with insufficient safety. The tightening quality depends on the experience of the operators, lacking a unified standardized detection and operation process, leading to unstable operation quality. Currently, although some equipment has achieved semi-automatic functions, the automation operation of key links such as bolt tightening has not been broken through, with a low degree of automation. And the operation data during the inspection process lacks effective collection and recording, unable to achieve comprehensive monitoring and analysis of the operation process, and the data management is weak.

[0004] In summary, the existing technologies have obvious deficiencies in terms of efficiency, safety, operation quality, and automation level, which restrict the further development of railway catenary operation and maintenance. Summary of the Invention

[0005] In order to solve the problems of low efficiency, poor safety, unstable quality, and low degree of automation existing in the current catenary wrist arm bolt tightening operation, the present invention proposes an automatic tightening system and method for catenary wrist arm bolts. By integrating lidar positioning, visual servo, and multi-degree-of-freedom robotic arm force feedback technologies, an intelligent and high-precision bolt tightening operation process is constructed from the parking of the maintenance vehicle to the automatic tightening of the bolts to solve the above problems.

[0006] The present application discloses an automatic tightening system for catenary wrist arm bolts, including:

[0007] Maintenance vehicle: including a maintenance work vehicle (12), on which a lidar (8), a first depth camera (9), and a lifting and slewing platform (11) are provided, and a robotic arm assembly (10) is provided on the lifting and slewing platform (11);

[0008] Visual ranging parking unit: used to guide the maintenance vehicle to stop below the catenary wrist arm;

[0009] LiDAR sensing unit: used for three-dimensional scanning and modeling of the catenary boom to obtain the spatial position of the bolts.

[0010] Robotic arm execution unit: used for planning a collision-free path according to the spatial position of the bolts and performing bolt alignment.

[0011] Force feedback control unit: used for real-time monitoring of the contact force between the end of the robotic arm and the bolt to achieve compliant alignment of the bolt sleeve.

[0012] End effector unit: used for tightening the bolt according to the preset torque.

[0013] Digital twin unit: used for real-time mapping of the bolt tightening operation process to achieve virtual-real synchronous monitoring.

[0014] Job management platform: used for task distribution, execution monitoring and data management, and coordinating the collaborative work of each unit through the ROS-Kafka communication protocol.

[0015] Preferably, the robotic arm assembly (10) includes a slide rail (104), on which a lifting column (3) is arranged, a robotic arm (102) is arranged at the top of the lifting column (3), and a tightening gun (101) is arranged at the end of the robotic arm (102).

[0016] The present application also discloses an automatic tightening method for catenary boom bolts, which is realized based on the above automatic tightening system for catenary boom bolts, and includes the following steps:

[0017] S1. Identify the parking target through the target detection algorithm, and combine the depth ranging algorithm to calculate the distance between the maintenance vehicle and the parking target in real time to achieve parking guidance for the maintenance vehicle.

[0018] S2. Perform three-dimensional scanning on the catenary environment, and complete the spatial structure modeling of the catenary boom by using offline modeling and online correction.

[0019] S3. Plan a collision-free path of the robotic arm in the model space through the path planning algorithm and the bounding box method, and use the visual servo control algorithm to achieve bolt recognition and alignment.

[0020] S4. Achieve compliant alignment between the bolt sleeve and the bolt by setting the Z-axis force threshold and the rotation search strategy.

[0021] S5. Tighten the bolt according to the preset torque and real-time feedback the data of the bolt tightening operation.

[0022] S6. Real-time map the entire process of the bolt tightening operation through virtual-real synchronization and store the operation data of the entire process.

[0023] S7. Perform anomaly detection and task resumption transmission on the entire process of bolt tightening operation, and conduct data archiving and warning.

[0024] Preferably, the S1 includes the following steps:

[0025] S11. Use a depth camera to collect real-time image data;

[0026] S12. Use the YOLO algorithm to identify parking targets;

[0027] S13. Use a depth ranging algorithm to calculate the three-dimensional coordinates of the parking target and output the real-time distance between the maintenance vehicle and the target;

[0028] S14. Prompt the maintenance vehicle driver with voice and image markings about the parking distance information to guide the maintenance vehicle driver to complete the parking operation.

[0029] Preferably, the S2 includes the following steps:

[0030] S21. Draw a 3D model of the standard catenary boom spatial structure, import it into the simulation platform to obtain offline catenary boom point cloud data, and establish an offline model of the standard catenary boom spatial structure;

[0031] S22. Use a lidar to scan the on-site catenary boom spatial structure to obtain on-site catenary boom point cloud data;

[0032] S23. Use the bounding box filtering algorithm to obtain the point cloud data of a single catenary boom spatial structure;

[0033] S24. Use the ICP registration algorithm to achieve the registration and splicing of the point cloud data of the multi-view catenary boom spatial structure, and perform registration correction by integrating the offline model obtained in S21;

[0034] S25. Use the greedy projection triangulation algorithm to perform three-dimensional reconstruction on the spliced point cloud data of the catenary boom spatial structure, and construct the geometric and topological structures of the real catenary model.

[0035] Preferably, the S3 includes the following steps:

[0036] S31. According to the bolt position information, plan a collision-free path of the robotic arm in the three-dimensional model through the RRT path planning algorithm, and use the bounding box method to monitor the potential collision between the robotic arm and the surrounding structures in real time;

[0037] S32. Adjust the slide rail and lifting column of the robotic arm assembly according to the collision-free path to compensate for the parking error of the maintenance vehicle and make the end of the robotic arm reach the preset operation area of the bolt;

[0038] S33. Collect bolt images and use the visual servo control algorithm to align the sleeve at the end of the robotic arm with the bolt.

[0039] Preferably, S4 includes the following steps:

[0040] Control the robotic arm to move linearly close to the bolt, and continuously monitor the z-axis contact force through the force sensor. When the z-axis contact force is greater than the preset threshold, stop the linear motion.

[0041] Start the end of the robotic arm to rotate around the z-axis in the tightening direction, and continuously monitor the z-axis contact force through the force sensor until the bolt sleeve is compliant and aligned with the bolt.

[0042] Preferably, S5 includes the following steps:

[0043] Use a tightening gun to apply torque in stages according to the preset torque to complete the bolt tightening.

[0044] Use a six-axis force sensor to continuously monitor the torque, rotation angle, and environmental parameters, and upload the data to the operation management platform.

[0045] Preferably, S6 includes the following steps:

[0046] S61. Through the fusion of lidar and multi-camera vision, construct a three-dimensional twin model with millimeter accuracy, and synchronously map the motion trajectory of the robotic arm and the bolt state.

[0047] S62. Compress the real-time video stream using H.265 encoding and transmit it to the operation management platform to dynamically display the operation progress.

[0048] S63. Integrate the real-time data including the pose of the robotic arm, torque value, and environmental parameters, and dynamically display them in the form of labels on the twin model interface.

[0049] S64. Encrypt and store the full-process video and operation data according to the timestamp, and replay the key nodes by retrieving the historical records.

[0050] Preferably, the anomaly detection in S7 includes:

[0051] Warning level: Pop-up a front-end prompt, and the task execution speed is reduced.

[0052] Pause level: Record the breakpoint and pause the operation.

[0053] Emergency stop level: Trigger the hardware emergency stop, and the audible and visual alarm is activated.

[0054] Advantages of the present invention:

[0055] (1) Improve operation efficiency: Significantly shorten the operation time through automated operations and reduce the personnel input.

[0056] (2) Enhance operation safety: Adopt robotic operations to avoid the safety hazards of high-altitude operations.

[0057] (3) Implement standardized operations: Use intelligent algorithms to ensure the accuracy and consistency of bolt tightening and guarantee the operation quality.

[0058] (4) Strengthen data management: Collect and store operation data in real time to provide support for subsequent analysis and maintenance.

[0059] (5) Adapt to complex environments: The system has good environmental adaptability and can operate stably under extreme weather conditions. Description of the Drawings

[0060] Figure 1 Schematic diagram of the catenary boom bolt automatic tightening system according to an embodiment of the present invention;

[0061] Figure 2 Schematic diagram of the catenary and maintenance vehicle structure according to an embodiment of the present invention;

[0062] Figure 3 Schematic diagram of the robotic arm assembly structure according to an embodiment of the present invention;

[0063] Figure 4 Schematic diagram of the ranging and parking unit operation process according to an embodiment of the present invention;

[0064] Figure 5 Schematic diagram of the catenary boom spatial structure modeling process and intention according to an embodiment of the present invention;

[0065] Figure 6 Schematic diagram of the ICP algorithm process according to an embodiment of the present invention;

[0066] Figure 7 Schematic diagram of the greedy projection triangulation algorithm process according to an embodiment of the present invention;

[0067] Figure 8 Schematic diagram of the end structure of the robotic arm according to an embodiment of the present invention;

[0068] Figure 9 Schematic diagram of the bolt compliance alignment process according to an embodiment of the present invention.

[0069] The reference numerals are as follows:

[0070] 1 - insulator, 2 - horizontal boom, 3 - inclined boom, 4 - positioning pipe, 5 - positioning pipe support, 6 - inclined boom support, 7 - pillar, 8 - lidar, 9 - first depth camera, 10 - robotic arm assembly, 11 - lifting and slewing platform, 12 - maintenance operation vehicle, 101 - tightening gun, 102 - robotic arm, 103 - lifting column, 104 - slide rail. Detailed Embodiments

[0071] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and examples.

[0072] The present application embodiment discloses an automatic fastening system for contact network arm bolts, such as Figure 1 and Figure 2 As shown, it includes a maintenance vehicle, a visual ranging parking unit, a lidar perception unit, a robotic arm execution unit, a force feedback control unit, an end execution unit, a digital twin unit and an operation management platform.

[0073] like Figure 2 As shown, the key structures of the contact network include insulator 1, flat arm 2, inclined arm 3, positioning tube 4, positioning tube support 5, inclined arm support 6, and support 7. The maintenance vehicle includes a maintenance vehicle 12, on which a laser radar 8, a first depth camera 9 and a lifting and rotating platform 11 are arranged, and a mechanical arm assembly 10 is arranged on the lifting and rotating platform 11. Figure 3 As shown, the robot arm assembly 10 includes a slide rail 104 , a lifting column 3 is arranged on the slide rail 104 , a robot arm 102 is arranged on the top of the lifting column 3 , and a tightening gun 101 is arranged at the end of the robot arm 102 .

[0074] The visual ranging parking unit is used to guide the maintenance vehicle to park under the contact network arm. The laser radar perception unit is used to perform three-dimensional scanning and modeling of the contact network arm to obtain the spatial position of the bolt. The robotic arm execution unit is used to plan a collision-free path according to the spatial position of the bolt and perform bolt alignment. The force feedback control unit is used to monitor the contact force between the end of the robotic arm and the bolt in real time to achieve smooth alignment of the bolt sleeve. The end execution unit is used to complete the bolt tightening according to the preset torque. The digital twin unit is used to map the bolt tightening operation process in real time to achieve virtual and real synchronous monitoring. The operation management platform is used for task issuance, execution monitoring and data management, and coordinates the collaborative work of various units through the ROS-Kafka communication protocol.

[0075] The embodiment of the present application also discloses a method for automated tightening of contact network arm bolts, which is implemented based on the above-mentioned automated tightening system for contact network arm bolts. First, the maintenance vehicle starts to run, and at the same time, the operation management system will issue specific maintenance tasks. When the maintenance vehicle stops after driving to the optimal maintenance working position with the assistance of the visual ranging parking unit, the laser radar 8 starts scanning the contact network and completes the three-dimensional matching and modeling of the contact network. According to the modeling results, it is necessary to manually control the bolt tightening work platform to make it reach the specific maintenance position. At the same time, the laser radar 8 continues to refine the alignment and cooperates with the guide rail to fine-tune the position of the work platform to ensure that the robotic arm can eventually reach the position of the bolt.

[0076] After the platform reaches the optimal working position, the robotic arm execution unit will automatically perform operations such as online path planning and sleeve replacement under the premise of collision detection. During this process, it is necessary for manual judgment whether there are split pins and lock washers on the bolt components. If so, they need to be manually removed and reinstalled after the tightening work of the entire bolt component is completed. Through the vision servo and force feedback control unit, the robotic arm can align with the nut, and the other end of the robotic arm can automatically align with the nut through the pose of the nut. After the bolt alignment work is completed, the industrial control computer issues a tightening instruction for the end execution unit to perform the bolt tightening operation, and judges whether it is tightened in place through torque feedback. After that, the robotic arm will automatically locate other bolts in the same connector according to the position of the tightened bolts and complete the tightening in sequence.

[0077] After completing the tightening work of all bolts in a connector, it is necessary to manually control the operation platform to move to the maintenance position of the next bolt connector again and repeat the above tightening operation. During the tightening period, the operation management platform will synchronize the state of the robotic arm with the behavior of the robotic arm in the physical world in the form of animation, that is, the digital twin unit, which is convenient for operators to understand the operation state. After completing all the bolt tightening tasks of a catenary boom, the robotic arm will automatically return to the initial position, and the maintenance vehicle will also drive to the position of the next catenary boom to continue the next round of maintenance tasks.

[0078] In a specific embodiment, the automatic tightening method for catenary boom bolts includes the following steps:

[0079] S1. In the vision ranging parking unit, the parking target is identified through the target detection algorithm, and the distance between the maintenance vehicle and the parking target is calculated in real time in combination with the depth ranging algorithm to realize the parking guidance of the maintenance vehicle. The core hardware of the vision ranging parking unit is the Zed Mini depth camera, which has the performance of waterproof and dustproof, and the protection level reaches IP67.

[0080] As Figure 4 shown, the specific process is as follows:

[0081] S11. When the maintenance vehicle approaches the catenary, the first depth camera 9 is used to collect real-time image data.

[0082] S12. The YOLOv5 algorithm is used to identify the parking target.

[0083] S13. The depth ranging algorithm is used to calculate the three-dimensional coordinates of the parking target and output the real-time distance between the maintenance vehicle and the target. Among them, the depth ranging algorithm calculates the parking target through two right triangles according to the spatial relationship.

[0084] S14. The inspection vehicle is equipped with a display screen in the driver's cab, which can display distance data in real time, prompt distance information through voice (such as "3 meters away from the parking point"), and help the driver adjust the vehicle position through image marking to guide the driver to complete the parking operation.

[0085] The whole process ensures that the inspection vehicle can accurately park at the best operation position of the catenary arm, providing an efficient and safe parking guidance solution for catenary maintenance operations.

[0086] S2. In the lidar sensing unit, the lidar 8 is used to perform three-dimensional scanning on the catenary environment, and the spatial structure modeling of the catenary arm is completed by using offline modeling and online correction. The core task of the lidar sensing unit is to perform high-precision three-dimensional scanning on the catenary environment, generate point cloud data and perform modeling, providing accurate reference for the path planning and operation of the robotic arm. In this embodiment, the lidar 8 selects the Livox Mid-70 lidar, which has high resolution and excellent ranging performance, and is especially suitable for three-dimensional perception of complex catenary environments. The spatial structure of the catenary arm has a unified standard, and the spatial structures of the catenary arms on the same line are generally similar as a whole, and can usually be reflected by a standard offline model of the catenary arm spatial structure. However, due to the influence of long-term operation and natural factors on the catenary arm structure, there is a risk of partial deformation in its structure. Therefore, this embodiment uses two steps of offline modeling and online correction to complete the spatial structure modeling of the catenary arm, and the modeling process is as Figure 5 shown. The specific steps are as follows:

[0087] S21. Offline modeling, draw a 3D model of the standard catenary arm spatial structure, import it into the simulation platform to obtain offline catenary arm point cloud data and perform data processing (filtering, denoising), and establish an offline model of the standard catenary arm spatial structure.

[0088] S22. Online correction, use the Livox Mid-70 lidar to scan the on-site catenary arm spatial structure, obtain on-site catenary arm point cloud data and perform data processing (filtering, denoising).

[0089] S23. Adopt the bounding box filtering algorithm to obtain the point cloud data of a single catenary arm spatial structure.

[0090] S24. Adopt the ICP registration algorithm to realize the registration and splicing of the multi-view catenary arm spatial structure point cloud data, and perform registration correction by integrating the offline model obtained in S21.

[0091] Based on the on-site catenary boom spatial structure point cloud data obtained by updating the Livox Mid-70 lidar with an online model, the ICP registration algorithm is used to achieve the online registration and splicing of the catenary boom spatial structure point cloud, and it is fused with the offline catenary boom model to realize the online correction of the catenary boom model. The ICP algorithm process is as Figure 6 shown, and the specific algorithm process is as follows:

[0092] According to the Euclidean distance between points of the ICP registration algorithm for judgment, assume that the points with the closest Euclidean distance in two point clouds are used as corresponding points.

[0093] Matrix calculation: Use the ordered point pairs extracted from the corresponding point pairs for matrix calculation to obtain the rotation matrix and translation vector.

[0094] Transform the point cloud: Apply the calculated rotation matrix and translation vector to the target point cloud.

[0095] Calculate the objective function: Use the Euclidean distance between point pairs as the objective function.

[0096] Judge the relationship between the objective function and the threshold to determine whether the next round of iteration is needed.

[0097] S25. Use the greedy projection triangulation algorithm to perform 3D reconstruction on the spliced catenary boom spatial structure point cloud data, and construct the geometric and topological structures of the real catenary model. The 3D model is displayed and analyzed through the visualization software Vrep to achieve the all-round observation and analysis of the catenary boom spatial structure, and provide support for maintenance and management work.

[0098] The point cloud greedy projection triangulation algorithm is a surface reconstruction algorithm based on greedy projection. Its core idea is to project the point cloud onto a 2D plane and connect the projected point cloud into triangles to construct a surface model. The greedy triangulation algorithm process is as Figure 7 shown, and specifically includes:

[0099] Perform mesh filtering operations on the input point cloud to reduce noise and redundant data.

[0100] Use the greedy projection algorithm to project the point cloud onto a plane to obtain the projected point cloud and the corresponding normal vectors.

[0101] Perform triangulation according to the projected point cloud and normal vectors, connect adjacent point clouds, and construct a surface model.

[0102] Perform post-processing on the surface model, including operations such as smoothing and edge preservation, to improve the reconstruction effect.

[0103] Since each catenary shape is different, the template matching algorithm (ICP registration algorithm) is used to register the real-time point cloud with the offline model library, correct the deviation, and generate an accurate three-dimensional structure of the catenary. Template matching means generating the point cloud data of several of the most common catenary shapes under ideal conditions, marking the positions of the Luo Shun connectors, and matching the actually collected point cloud data with the template. The one with the highest matching degree is the current catenary model. The positions of the bolt connectors corresponding to the template are transmitted to the robotic arm for path planning, providing a reference for subsequent bolt tightening operations.

[0104] S3. In the robotic arm execution unit, through the path planning algorithm and the bounding box method, plan a collision-free path of the robotic arm in the model space, and use the visual servo control algorithm to realize the recognition and alignment of the bolts.

[0105] S31. According to the bolt position information, plan a collision-free path of the robotic arm in the three-dimensional model through the RRT (Rapidly-exploring Random Tree) path planning algorithm, and use the bounding box (BoundingBox) method to monitor the potential collisions between the robotic arm and the surrounding structures in real time.

[0106] The RRT path planning algorithm explores in the expansion direction by randomly generating nodes in the configuration space, quickly planning a collision-free safe path in a complex environment to ensure that the robotic arm can safely reach near the target bolt point. At the same time, real-time collision detection is carried out through the bounding box method to eliminate the deviation in actual operation, prevent potential risks, and ensure the safety and efficiency of the entire operation process. The specific process of the robotic arm bolt tightening path planning and collision detection includes:

[0107] S311. Initial setting and model preparation, define the initial pose and target pose of the robotic arm in the three-dimensional model obtained in S2, providing the necessary starting point and ending point for path planning.

[0108] S312. The RRT path planning algorithm is particularly suitable for motion planning problems in high-level spaces. Its basic principle is to randomly sample in the state space and gradually construct a tree until a path between the starting state and the target state is found. The basic algorithm steps are as follows:

[0109] a. Take the starting state as the only node of the tree, and this node is the root node of the tree.

[0110] b. Randomly sample a point from the state space as the target state. This target state may be the end point of the path planning or an intermediate target point.

[0111] c. Select the nearest neighbor node of each randomly sampled target state from the tree, and generate a new node connecting this node and the target state according to the kinematic model or constraints. This new node becomes a new member of the tree.

[0112] d. Connect the new node to the nearest neighbor node to form a path segment. This connection is made along a straight line or curve between the two points.

[0113] e. Continuously repeat steps b to d until the termination condition is reached. This termination condition is to find a path from the start state to the target state.

[0114] f. If a path from the start state to the target state is found, trace back from the target node to the root node through the connection relationship of the tree to obtain the complete path.

[0115] S313. The bounding box is a simple geometric shape, usually a rectangle or cube, used to approximately represent the shape and position of an object. In robotic arm collision detection, a bounding box is set for each component of the robotic arm, such as the links of the robotic arm and the end of the robotic arm, and corresponding bounding boxes are also set for the possible obstacles in the surrounding environment. Then, it is judged whether a collision occurs by detecting whether these bounding boxes intersect. The steps of the bounding box in robotic arm collision detection are as follows:

[0116] Bounding box modeling: Model each component of the robotic arm and the objects in the surrounding environment, and create corresponding bounding boxes for them respectively.

[0117] Collision detection: During the movement of the robotic arm, real-time detect whether the bounding box of the robotic arm intersects with the bounding boxes of the surrounding objects to judge whether a collision occurs.

[0118] Collision avoidance or mitigation: If it is detected that a collision is about to occur or has occurred, corresponding measures can be taken to avoid the collision or mitigate the impact of the collision, such as stopping the movement of the robotic arm, adjusting the path planning, etc.

[0119] S314. Cooperative control execution: On the basis of ensuring path safety, the two robotic arms move towards the target bolt point simultaneously according to the planned cooperative path.

[0120] S315. Real-time monitoring and path adjustment: During the execution process, real-time monitor the movement state of the robotic arm. If there is a deviation, use the RRT path planning algorithm to make an immediate adjustment of the path.

[0121] Through three-dimensional space modeling and precise collaborative control, the robot arm can achieve safe and efficient operation in the complex catenary arm environment. The RRT path planning algorithm is adopted, and its core advantage is that it can quickly explore high-dimensional state space. By means of random sampling and gradual expansion of tree structure, it can efficiently generate feasible paths under complex constraints, which is especially suitable for dynamic operation scenarios with real-time requirements. At the same time, the path planning process integrates bounding box collision detection, which can realize real-time collision monitoring of the path and provide safety protection for the movement of the robot arm.

[0122] S32, adjusting the slide rail 104 and the lifting column 103 of the robot arm assembly 10 according to the collision-free path, compensating for the parking error of the maintenance vehicle, and making the end of the robot arm reach the preset working area of ​​the bolt.

[0123] like Figure 3 As shown, in order to meet the requirements of working at different heights, the robot arm 102 is installed on an adjustable slide rail 104 to form a robot arm assembly 10.

[0124] The slide rail 104 can not only be accurately adjusted according to the actual height of the target bolt, but also effectively compensate for the longitudinal deviation problem caused by the parking position error of the maintenance vehicle, and can perform precise compensation within a preset range (1.2m in the Z direction and 1.6m in the Y direction).

[0125] The main operation process of the system is as follows: First, the laser radar sensing unit performs a three-dimensional scan of the contact network structure, obtains the spatial position information of the bolts and completes the modeling; then, the robot arm assembly 10 automatically adjusts to the target position of the bolts by linking the slide rail 104 and the lifting column 103 according to the path planning algorithm. The robot arm 102 moves from the starting state to the target area of ​​the bolt according to the preset path, and after approaching the target position, the visual servo control algorithm is enabled to achieve accurate identification and automatic alignment of the bolts.

[0126] S33, collecting the bolt image through the second depth camera, and aligning the sleeve at the end of the robot arm with the bolt using a visual servo control algorithm.

[0127] Specifically, the structure of the end of the robotic arm is as follows Figure 8 As shown, the tightening gun 101 is arranged on the end of the mechanical arm, and a sleeve is arranged on the tightening gun 101, a force sensor is arranged between the tightening gun 101 and the end of the mechanical arm, and a second depth camera and a high-precision radar are also arranged on the end of the mechanical arm. The second depth camera model in this embodiment is Realsense D435i, and the high-precision radar model is Mech-Eye NANO ULTRA.

[0128] In this embodiment, a position-based visual servo PBVS (Precision Ball Valve System) is adopted, which calculates using the difference between the three-dimensional position information of an object and the required position information. Its target positioning and control capabilities in complex environments are particularly prominent, and the robotic arm is controlled through the error of the pose information of the target. By using the template matching algorithm in the OpenCV computer vision library, the pose of the target bolt can be accurately estimated, thus achieving rapid positioning and recognition of the target. In addition, UR_RTDE (Universal Robots Real-Time Data Exchange) realizes real-time communication between the UR robotic arm and the industrial control computer through the TCP / IP protocol. This efficient data transmission mechanism not only ensures the real-time nature of control instructions but also improves the response speed and control accuracy of the entire system. In this way, efficient and accurate target positioning and control can be achieved in complex operating environments.

[0129] The algorithm logic for implementing visual servo based on OpenCV+UR_RTDE is as follows:

[0130] Image acquisition: Use the second depth camera to collect image information.

[0131] Image preprocessing: Process the grayscale image of the acquired image to increase the contrast between the bolt and the environment.

[0132] Image feature extraction and pose estimation: Collect catenary bolt images through the second depth camera, obtain image data using the second camera, and track the movement of key points through the KLT optical flow tracking algorithm in the OpenCV library; obtain dense depth data using depth information. Use the key points and dense depth as visual features to track the CAD format bolt model, and adopt template matching to estimate the pose of the target bolt.

[0133] Control strategy: Calculate the movement speed of the robotic arm joints based on the difference between the estimated pose information and the desired pose information, and send speed commands through UR_RTDE to achieve the pose control of the robotic arm.

[0134] By the above steps, the end of the robotic arm can be directly facing the bolt. However, there are inevitably certain errors in visual positioning. Therefore, a force feedback control unit is introduced to effectively sleeve the fastening sleeve onto the bolt.

[0135] S4. In the force feedback control unit, by setting the Z-axis force threshold and rotation search strategy, the compliant alignment between the bolt sleeve and the bolt is achieved. After the visual servo is completed, the precise compliant alignment of the end of the robotic arm with the bolt is a crucial step. This process not only ensures the accurate alignment between the bolt sleeve and the bolt but also lays the foundation for the success of the bolt fastening operation.

[0136] In the operation mode of achieving bolt compliant alignment, the robotic arm first approaches near the bolt through linear motion. It uses a force sensor to monitor the z-axis force when contacting the bolt. Once the force value exceeds the preset threshold, the robotic arm stops the linear motion. This mode allows the robotic arm to make fine adjustments when approaching the bolt to adapt to possible small deviations or uneven surfaces. Subsequently, the robotic arm starts to rotate clockwise around the end z-axis. During this period, it still continuously collects force information. Through precise force control strategies, the robotic arm can adjust the rotation angle and force in real time to ensure that when the bolt sleeve contacts the bolt, both the torque and the force are within the control range, neither too large to cause component damage nor too small to result in inaccurate sleeve docking.

[0137] This compliant operation mode utilizes fine compliant force control to ensure the accuracy and safety of the alignment process. By setting the rotation step size and direction, the robotic arm can gradually approach the target state with the minimum rotation error until the bolt sleeve is completely aligned with the bolt, completing the compliant alignment task. The flowchart of the bolt compliant alignment task is as Figure 9 shown and specifically includes the following steps:

[0138] End pose acquisition: Obtain the end pose information through the RTDE Receive interface and set the target position to a position 20 centimeters directly above the current end pose along the z-axis.

[0139] Asynchronous linear movement: Use the moveL() method of the RTDE Control interface for linear movement. When the force sensor detects that the z-axis force exceeds the threshold, stop the movement through the stopL() method.

[0140] Rotation search strategy, which specifically includes the following steps:

[0141] Initial position, force control, and threshold setting: During rotation, monitor the force in real time through the force sensor and set the starting position, torque, and force thresholds for rotation search;

[0142] Rotation execution: Rotate clockwise in 1° steps, continuously monitor the force sensor data, and once it exceeds the threshold, stop rotating and execute bolt tightening.

[0143] The force feedback control unit ensures the precise docking of the bolt sleeve and the bolt through accurate force control and a set threshold judgment mechanism. At the same time, a compliant control strategy is introduced to enable the robotic arm to adapt to small pose deviations during contact and avoid structural damage caused by rigid contact. The force feedback control unit also has excellent real-time response capabilities, can continuously monitor the sensor feedback data, and dynamically adjust the motion state of the robotic arm, thereby improving the alignment efficiency and operation stability. In addition, high-frequency data interaction and control command issuance with the robotic arm are achieved through the RTDE interface, greatly reducing manual intervention and significantly enhancing the intelligence and automation level of the overall system.

[0144] S5. In the end effector unit, the bolt is tightened according to the preset torque, and the data of the bolt tightening operation is fed back in real time. The specific process includes: using a tightening gun to apply torque in stages according to the preset torque to complete the bolt tightening; using a six-axis force sensor to continuously monitor the torque, rotation angle, and environmental parameters, and upload the data to the operation management platform.

[0145] After the bolt is inserted, the end effector unit tightens the bolt. The end effector unit is the core module for automatic bolt tightening, which can accurately complete bolt tightening according to the preset torque and feed back operation data in real time to ensure quality. The end effector unit consists of a tightening gun and a multi-modal sensor, and has high-precision control and environmental adaptability.

[0146] The tightening gun 101 is driven by a servo motor, with a torque range covering 5 - 200 N·m, and the precision control is within ±2%. It supports multi-stage torque application, namely the pre-tightening stage, the main tightening stage, and the final tightening stage, to avoid overloading or loosening. The sleeve adopts a quick-change design, can be adapted to bolt specifications from M10 to M24, and ensures stable contact through magnetic attraction and elastic pads to prevent slipping.

[0147] The multi-modal sensor continuously monitors the torque, rotation angle, and six-axis force data. The six-axis force sensor detects the offset force of the sleeve and dynamically adjusts the end pose to ensure the coaxial alignment of the bolt and the sleeve.

[0148] After tightening, the tightening data of the corresponding bolt is uploaded to the database. The data feedback and management automatically record the torque curve, angle, and environmental parameters of each bolt, generate a unique operation ID, and bind it to the position. Abnormal data (such as torque exceeding the limit or angle deviation) triggers an alarm and marks "pending re-inspection". The operation report is uploaded to the operation management platform through the network, supporting real-time monitoring and historical traceability.

[0149] S6. In the digital twin unit, the entire process of the bolt tightening operation is mapped in real time through virtual-real synchronization, and the operation data of the entire process is stored. At the same time, based on the combination of video stream and data stream, visual monitoring and historical traceability are realized in the operation management platform.

[0150] S61. Full-process 3D modeling. By fusing lidar and image recognition, a 3D twin model with millimeter precision is constructed, achieving a millimeter-level accuracy. Dynamically match the actual movement trajectories of devices such as robotic arms and tightening guns, and synchronously display the operation progress in the twin model, mapping the movement trajectories of the robotic arms and the bolt states.

[0151] S62. Real-time video stream fusion. Compress the real-time video stream using H.265 encoding and transmit it to the operation management platform with low latency via the 5G network. Dynamically display the operation progress and support multi-view switching and zooming to view details.

[0152] S63. Data synchronization and interaction. Integrate real-time data including the pose and torque value of the robotic arm and environmental parameters, and dynamically display them in the form of labels in the twin model interface.

[0153] S64. Storage and interaction. Encrypt and store the full-process video and operation data according to the timestamp, with a retention period of ≥1 year. Historical records can be quickly retrieved by bolt number, position coordinates, or operation time, and key nodes (such as alarm events and torque overrun) can be replayed.

[0154] S7. In the operation management platform, perform anomaly detection and task resumption for the full process of bolt tightening operations, and conduct data archiving and early warning. The operation management platform is based on a hierarchical architecture and the ROS-Kafka communication protocol, and coordinates and monitors through the visual ranging parking unit, lidar sensing unit, robotic arm execution unit, force feedback control unit, end effector unit, and digital twin unit to achieve intelligent management of the catenary boom maintenance task, specifically including the following steps:

[0155] S71. Task creation and distribution.

[0156] Front-end configuration: The operator selects the target boom section (such as "section A3") through the Vue interface, and the system automatically loads the bolt list of this section (including position and specifications), and batch sets parameters (such as torque 50 N·m and priority).

[0157] Instruction encapsulation: The backend generates a unique task ID (format: TASK_date_sequence number), encapsulates the parameters into a Protobuf protocol (Protocol Buffers, an efficient serialization format) message, and sends it to the ROS execution layer through the bolt_tighten_command topic of Kafka.

[0158] Dynamic partition scheduling: Emergency instructions (such as emergency stop) exclusively occupy Kafka partition 0 (response latency ≤10 ms), and regular tasks are load-balanced according to hash partitioning to ensure real-time processing of high-priority tasks.

[0159] S72. Execution and monitoring.

[0160] Instruction Parsing and Control: The ROS-Kafka bridging node receives messages and deserializes them to generate robotic arm control instructions (target coordinates, torque values), driving the robotic arm to move to the bolt position (positioning error ±5mm). The robotic arm performs path planning (using the RRT algorithm) and collision detection (based on the bounding box technique) to ensure safe operation.

[0161] Real-time Feedback and Visualization: The torque sensor transmits data back at a frequency of 50Hz, and the front-end dashboard dynamically displays the actual torque value; the video stream is superimposed on the digital twin model to synchronously display the actions of the robotic arm and the status of the bolt (green / red markings).

[0162] Abnormality Detection, specifically including:

[0163] Warning Level (positioning error 1 - 2cm): A front-end pop-up window prompts, and the task execution speed is reduced.

[0164] Pause Level (torque continuously below the threshold by 95% for 5 times): Record the breakpoint and pause the operation.

[0165] Emergency Stop Level (collision risk): Trigger the hardware emergency stop (response ≤20ms), and the audible and visual alarm is activated.

[0166] S73, Manual Intervention and Resume Transmission.

[0167] Manual Takeover: After an abnormality is triggered, the operator switches to the "manual mode" and adjusts the pose of the robotic arm or the angle of the nozzle through the front-end virtual joystick.

[0168] Task Resume Transmission: After manual confirmation of the repair, the system resumes the operation from the breakpoint, and the backend reissues the resume transmission instruction to the robotic arm execution unit.

[0169] S74, Data Archiving and Early Warning.

[0170] Structured Storage:

[0171] The bolt tightening result (reason for success / failure, operator ID) is written into the Tightening_Records table in MySQL, supporting second-level retrieval by timestamp or bolt ID.

[0172] The abnormality log (such as E101 code) is associated with the bolt ID and archived in a separate database.

[0173] Intelligent Early Warning:

[0174] Daily Analysis: The offline data analysis module scans the Bolt_Info table, marks the bolts with loosening ≥3 times within 30 days, and automatically generates replacement work orders and pushes them to the procurement system.

[0175] Long-term Trend: Bolts with an annual loosening frequency exceeding twice the average of the same type are marked as high-risk, triggering a special inspection task.

[0176] In a specific embodiment, the arm information table (Arm_Info) is shown in Table 1, the bolt information table (Bolt_Info) is shown in Table 2, and the tightening operation record table (Tightening_Records) is shown in Table 3.

[0177] Table 1 Arm information table (Arm_Info)

[0178]

[0179] Table 2 Bolt information table (Bolt_Info)

[0180]

[0181]

[0182] Table 3 Tightening operation record table (Tightening_Records)

[0183]

[0184] The above has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification only illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. An automatic tightening system for catenary wrist bolts, characterized in that Including: Maintenance vehicle: including a maintenance operation vehicle (12), on which a lidar (8), a first depth camera (9) and a lifting and slewing platform (11) are provided, and a robotic arm assembly (10) is provided on the lifting and slewing platform (11); Visual ranging parking unit: used to guide the maintenance vehicle to park under the catenary boom; Lidar sensing unit: used to perform three-dimensional scanning and modeling of the catenary boom to obtain the spatial position of the bolts; Robotic arm execution unit: used to plan a collision-free path according to the bolt spatial position and perform bolt alignment; Force feedback control unit: used to monitor the contact force between the end of the robotic arm and the bolt in real time to achieve compliant alignment of the bolt sleeve; End execution unit: used to complete bolt tightening according to the preset torque; Digital twin unit: used to map the bolt tightening operation process in real time to achieve virtual-real synchronous monitoring; Operation management platform: used for task distribution, execution monitoring and data management, and coordinating the collaborative work of each unit through the ROS-Kafka communication protocol.

2. The catenary boom bolt automatic fastening system according to claim 1, wherein The robotic arm assembly (10) includes a slide rail (104), on which a lifting column (3) is provided, a robotic arm (102) is provided at the top of the lifting column (3), and a tightening gun (101) is provided at the end of the robotic arm (102).

3. An automatic fastening method for catenary boom bolts, characterized in that, Implemented based on the catenary boom bolt automatic tightening system according to any one of claims 1-2, including the following steps: S1. Identify the parking target through the target detection algorithm, and combine the depth ranging algorithm to calculate the distance between the maintenance vehicle and the parking target in real time to achieve parking guidance for the maintenance vehicle; S2. Perform three-dimensional scanning on the catenary environment, and complete the modeling of the catenary boom spatial structure by using offline modeling and online correction; S3. Through the path planning algorithm and the bounding box method, plan a collision-free path of the robotic arm in the model space, and use the visual servo control algorithm to achieve bolt recognition and alignment; S4. By setting the Z-axis force threshold and the rotation search strategy, achieve compliant alignment of the bolt sleeve and the bolt; S5. Tighten the bolt according to the preset torque and feedback the data of the bolt tightening operation in real time; S6. Map the entire process of the bolt tightening operation in real time through virtual-real synchronization and store the operation data of the entire process; S7. Perform anomaly detection and task resumption transmission on the entire process of the bolt tightening operation, and perform data archiving and warning.

4. The automated tightening method for the catenary wrist arm bolts according to claim 3, characterized in that, The S1 includes the following steps: S11. Use the depth camera to collect real-time image data; S12. Use the YOLO algorithm to identify the parking target; S13. Use the depth ranging algorithm to calculate the three-dimensional coordinates of the parking target and output the real-time distance between the maintenance vehicle and the target; S14. Prompt the maintenance vehicle driver with the parking distance information through voice and image marking to guide the maintenance vehicle driver to complete the parking operation.

5. The automatic tightening method for the catenary wrist arm bolts according to claim 4, characterized in that, The S2 includes the following steps: S21. Draw a 3D model of the standard catenary boom spatial structure, import it into the simulation platform to obtain the offline catenary boom point cloud data, and establish an offline model of the standard catenary boom spatial structure; S22. Use the lidar to scan the on-site catenary boom spatial structure to obtain the on-site catenary boom point cloud data; S23. Use the bounding box filtering algorithm to obtain the single spatial structure point cloud data of the catenary mast arm; S24. Use the ICP registration algorithm to realize the registration and splicing of the multi-view spatial structure point cloud data of the catenary mast arm, and fuse the offline model obtained in S21 for registration correction; S25. Use the greedy projection triangulation algorithm to perform 3D reconstruction on the spliced spatial structure point cloud data of the catenary mast arm, and construct the geometric and topological structures of the real catenary model.

6. The automatic fastening method for the catenary boom bolt according to claim 5, characterized in that, The said S3 includes the following steps: S31. According to the bolt position information, plan the collision-free path of the robotic arm in the 3D model through the RRT path planning algorithm, and use the bounding box method to monitor the potential collision between the robotic arm and the surrounding structures in real time; S32. Adjust the slide rail and lifting column of the robotic arm assembly according to the collision-free path to compensate for the parking error of the maintenance vehicle, so that the end of the robotic arm reaches the preset operation area of the bolt; S33. Collect the bolt image, and use the visual servo control algorithm to align the sleeve at the end of the robotic arm with the bolt.

7. The automatic fastening method for the catenary boom bolt according to claim 6, characterized in that The said S4 includes the following steps: Control the robotic arm to move linearly close to the bolt, and monitor the z-axis contact force in real time through the force sensor. When the z-axis contact force is greater than the preset threshold, stop the linear motion; Start the rotation of the end of the robotic arm around the z-axis in the tightening direction, and continuously monitor the z-axis contact force through the force sensor until the bolt sleeve is compliantly aligned with the bolt.

8. The automatic fastening method for the catenary wrist arm bolt according to claim 7, characterized in that, The said S5 includes the following steps: Use the tightening gun to apply torque in stages according to the preset torque to complete the bolt tightening; Use the six-axis force sensor to monitor the torque, rotation angle and environmental parameters in real time, and upload the data to the operation management platform.

9. The automatic tightening method for the catenary wrist arm bolt according to claim 8, characterized in that, The said S6 includes the following steps: S61. Through the fusion of lidar and multi-camera vision, construct a three-dimensional twin model with millimeter accuracy, and synchronously map the motion trajectory of the robotic arm and the bolt state; S62. Compress the real-time video stream using H.265 encoding and transmit it to the operation management platform to dynamically display the operation progress; S63. Integrate the real-time data including the pose of the robotic arm, torque value and environmental parameters, and dynamically display them in the form of labels on the twin model interface; S64. Encrypt and store the full-process video and operation data according to the time stamp, and replay the key nodes by retrieving the historical records.

10. The automatic tightening method for the catenary boom bolt according to claim 9, wherein, The anomaly detection in the said S7 includes: Warning level: Pop-up prompt at the front end, and the task execution speed is reduced; Pause level: Record the breakpoint and pause the operation; Emergency stop level: Trigger the hardware emergency stop, and the audible and visual alarm is started.

Images