Contact net cantilever robot assembling method based on hand-eye-force integration

Abstract

The invention discloses a hand-eye-force integration-based catenary cantilever robot assembly method, which comprises the following steps of: synthesizing image sequences of different exposures into a high-quality image based on a self-adaptive multi-exposure fusion technology, identifying a workpiece contour in the image based on a deep learning framework, and calculating a pose of an identified part in a camera coordinate system; the two-dimensional camera and the three-dimensional camera are matched to carry out accurate scanning, an accurate pose deviation matrix of an actual workpiece relative to a theoretical model is obtained through a nearest point cloud registration algorithm, and a corrected moving target pose is calculated; and the robot moves towards the position and posture of the moving target, when the robot is about to make contact with the assembly workpiece, the impedance control mode is switched to conduct cantilever installation, and moment data are continuously monitored in the whole process till installation is completed. According to the method, the robot can intelligently cope with uncertainty in the cantilever assembly process, and the assembly success rate and the assembly efficiency are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of automated assembly technology, and in particular to a method for assembling a contact network wrist robot based on a hand-eye-force integrated system. Background Technology

[0002] As a key load-bearing structure in the overhead contact system of electrified railways, the assembly quality of the cantilever arm system directly affects power supply safety and train operation stability. Currently, the assembly of cantilever arms mainly relies on manual operation, which has inherent shortcomings such as low efficiency, high labor intensity, and difficulty in ensuring consistent quality. Promoting the automation and intelligentization of its assembly process is an inevitable requirement for the development of railway technology.

[0003] Industrial robots provide an ideal execution platform for solving this problem. However, applying robots to actual assembly scenarios of wrist arms faces two major technological challenges: First, the perception challenge. Wrist arms and their auxiliary parts (such as insulators and bushings) are mostly smooth metal components, which are prone to strong reflections under industrial lighting conditions, leading to difficulties in feature extraction and large fluctuations in positioning accuracy for traditional 2D vision recognition algorithms. Second, the execution challenge. Precision assembly is a process with multiple uncertainties (such as part machining tolerances, robot absolute positioning errors, and tooling fixture repeatability errors). Relying solely on teach-and-playback or vision-guided position control is prone to assembly jamming due to error accumulation in micron-level hole-shaft mating, lacking the "compliance" and "adaptive" capabilities possessed by human technicians. Summary of the Invention

[0004] In order to overcome the above-mentioned problems in the existing technology, the present invention proposes an assembly method for a contact network wrist robot based on hand-eye-force integration.

[0005] The technical solution adopted by this invention to solve its technical problem is: a method for assembling a contact network wrist robot based on hand-eye-force integration, comprising the following steps: Step 1, System Calibration and Parameter Initialization: Determine the spatial transformation relationship between the camera coordinate system and the robot end effector coordinate system, establish the precise mapping relationship between the force sensor coordinate system and the robot end effector coordinate system, and verify the calibration results; Step 2, visual coarse localization: Based on adaptive multi-exposure fusion technology, image sequences with different exposures are synthesized into high-quality images. The workpiece contour in the image is identified based on a deep learning framework. Combined with the depth information of the 3D camera, the pose of the identified part in the camera coordinate system is calculated and then converted into a pose of the grasping target that can be used by the robot. Step 3, Robot Motion and Compliant Grasping: Based on the target pose obtained in Step 2, the robot moves to the grasping point and performs the grasping action, while monitoring the force sensor data in real time. After the grasping is completed, the robot moves to a safe height directly above the wrist arm base assembly point. Step 4, Visual fine positioning: The assembly hole area is roughly located by a 2D camera, and this area is used as the region of interest to guide the 3D camera to perform precise scanning. The precise pose deviation matrix of the actual workpiece relative to the theoretical model is obtained by the nearest point cloud registration algorithm. Step 5, Online pose compensation: The pose deviation matrix obtained in step 4 is inverted and applied to the robot to achieve the planned theoretical assembly target pose, generating the corrected motion target pose; Step 6, Impedance control mode switching and assembly status monitoring: The robot moves towards the target pose obtained in step 5. When the robot is about to come into contact with the assembly workpiece, it switches to impedance control mode to install the wrist arm. The torque data is continuously monitored throughout the process until the installation is completed.

[0006] The above-mentioned method for assembling a contact network wrist robot based on hand-eye-force integration, specifically step 2, the adaptive multi-exposure fusion technology, involves the two-dimensional camera operating at a fixed viewpoint using an exposure time sequence { t 1, t 2, ..., t n}Collected image set{ I 1, I 2, ..., I n}, for each pixel location in the image ( x , y ), calculate the fusion weights in its images with different exposures. W k ( x , y High dynamic range fused images are obtained by weighted averaging.

[0007] The above-mentioned method for assembling a contact network wrist robot based on hand-eye-force integration, wherein the fusion weights W k ( x , y The specific calculation formula is as follows: Among them, I k(x, y) This represents the normalized gray value at position (x, y) in the k-th image. This represents the gradient magnitude at that location, a parameter used to control the appropriateness of exposure. This refers to parameters used to control the appropriateness of exposure. The specific calculation formula for high dynamic range fused images is as follows: Among them, I HDR Let be the gray value of the fused high dynamic range image at (x, y).

[0008] In the aforementioned method for assembling a contact network wrist robot based on a hand-eye-force integrated system, step 4, visual precision positioning, is fundamentally about solving the actual scanned point cloud. P real Point cloud of theoretical CAD model P model ={ q j Optimal rigid body transformation between} T deviation The following optimization problem is established: T deviation = Subject to Introduce bidirectional distance constraints: , Normal vector consistency constraint: , Curvature compatibility constraints: Where, ω i These are weighting coefficients. This represents the nearest neighbor correspondence. In the point cloud of the theoretical CAD model, and p i The corresponding nearest neighbor, R is the rotation matrix, t is the translation vector, p i For the actual scanned point cloud P real The coordinates of the i-th three-dimensional point (x) i , y i , z i ), q i For the theoretical CAD model point cloud P model In and p i The corresponding coordinates of the i-th 3D point The maximum allowable distance threshold, The angle between the maximum running normal vectors, For the maximum operating curvature difference, , These are all normal vectors of the corresponding points. This represents the curvature value at the corresponding point.

[0009] The above-mentioned method for assembling a contact network wrist robot based on hand-eye-force integration, specifically the impedance control mode in step 6, is as follows: the robot end effector is equivalent to a six-degree-of-freedom mass-spring-damped system, whose behavior is determined by inertia, damping and stiffness parameters. The assembly process is managed by a finite state machine, which switches between search state, contact alignment state, insertion state and completion state based on the real-time readings of the torque sensor.

[0010] The above-mentioned method for assembling a contact network wrist robot based on hand-eye-force integration, specifically the wrist robot installation process in the impedance control mode in step 6 is as follows: In the search state, the robot end effector moves slowly along the assembly axis while monitoring the contact force. If a significant lateral force or torque is detected, it switches to the contact alignment state; if axial resistance is mainly detected, it directly switches to the insertion state. Contact alignment state: Based on the feedback of lateral force or torque, the robot actively adjusts its end-effector posture in a plane perpendicular to the axis to achieve self-alignment of the hole shaft. When the lateral force is reduced to below the safety threshold, the alignment is considered complete and the robot enters the insertion state. Insertion state: The robot maintains the current alignment posture and completes the insertion process of the part with a constant or slowly increasing axial force. The torque data is continuously monitored throughout the process. If an abnormal lateral force is detected again, it returns to the contact alignment state. If the axial force suddenly jumps and exceeds the preset positioning threshold, the assembly is determined to be successful and the robot enters the completion state. Completion status: The robot stops moving, releases its gripper, and records a successful assembly message.

[0011] In the above-described method for assembling a contact network wrist robot based on a hand-eye-force integrated system, the adjustment amount of the robot's end effector motion in the unaligned state is expressed as: = K p · F side in, Adjust the amount of exercise. F side It is a lateral force. K p This is the proportionality coefficient.

[0012] The beneficial effects of this invention are: ① This invention constructs a hierarchical, interference-resistant, and self-correcting visual guidance system. First, it proposes a hierarchical perception architecture of "global coarse localization - local fine localization," which quickly locks the target through deep learning and then performs precise measurement using 3D point clouds, balancing efficiency and accuracy, and solving the contradiction between "not being fast enough" and "not being accurate enough" in industrial scenarios. ② The innovation lies in constructing a "interference-resistant and robust" technology chain for harsh industrial environments, solving the recognition and registration problems caused by strong reflectivity of metal parts and poor point cloud quality. At the coarse localization level, it innovatively adopts adaptive multi-exposure fusion technology to provide the "optimal input" target for subsequent deep learning networks, actively generating images that suppress highlights and enhance features. At the fine localization level, it develops a multi-constraint weighted registration method, which intelligently selects the most reliable point pair correspondence by comprehensively utilizing distance, normal vector, and curvature triple geometric constraints, effectively resisting interference from noise and similar local features. ③ An online closed-loop correction mechanism of "measurement-compensation-verification" was established. By inversely compensating for workpiece deviations observed visually to the robot's motion commands, the pose error between the theoretical model and the physical world was systematically eliminated, significantly improving the system's engineering practicality and economy. ④ A compliant assembly strategy based on impedance control and finite state machines was proposed, establishing a collaborative mechanism between visual "global map" and force-sensory "local navigation," enabling the robot to intelligently cope with uncertainties in the wrist arm assembly process. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the process of this invention; Figure 2 This is a schematic diagram of the coarse and fine positioning process of the present invention; Figure 3 This is a schematic diagram of the finite state machine transition for compliant assembly according to the present invention; Figure 4 This is a schematic diagram of the experimental test results of an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] like Figure 1 As shown in the figure, this embodiment discloses an assembly method for a contact network wrist robot based on hand-eye-force integration, which specifically includes the following steps: Step 1: Hardware System Integration and Setup. Select a six-DOF industrial robot with a rated load greater than 300 kg, ensuring its repeatability meets sub-millimeter requirements. Securely install the robot at the workstation. Install the hardware on the robot's sixth-axis flange in the following order: First, install a high-precision six-dimensional force / torque sensor, ensuring its connection surface with the flange is clean and tight to guarantee accurate force data transmission. Second, install a custom connector above the force sensor; the other side of this connector is used to install a flexible gripper or quick-change tool tray. All connectors must be precision machined to ensure overall connection rigidity and accuracy. Finally, compactly install the vision module integrating 2D and 3D vision cameras onto the end effector body, forming an "eye on hand" configuration.

[0016] Auxiliary system configuration: A programmable ring-shaped multispectral light source is installed within the robot's working cell to provide a stable lighting environment for the vision system that can adapt to different reflective conditions. Simultaneously, an industrial control computer is deployed to integrate and run core software such as robot control, vision processing, and force control management.

[0017] Step 2: System Calibration and Parameter Initialization. A high-precision checkerboard or dot array calibration board is fixed within the robot's workspace. The robot's end effector is manually or programmatically controlled to move the hand-eye camera to fifteen to twenty different spatial poses, acquiring a clear image of the calibration board in each pose. Using a hand-eye calibration algorithm (such as the Tsai-Lenz method), all acquired images and robot pose data are processed to accurately determine the spatial transformation relationship between the camera coordinate system and the robot's end effector coordinate system. After calibration, verification is required to ensure the projection error is less than one pixel. Using the cusp method or four-point method, the precise position and orientation of the flexible gripper's grasping center or assembly tool tip in the robot's end effector coordinate system are accurately determined. This point is defined as the robot's tool center point.

[0018] Zero-point calibration was performed on the six-dimensional force / torque sensor, and a gravity compensation procedure was executed to eliminate the influence of the end effector's own gravity on the force readings. A precise mapping relationship was established between the force sensor coordinate system and the robot's end effector coordinate system to ensure that the forces and torques detected by the sensor can be correctly converted to the robot's base coordinate system for interpretation.

[0019] Step 3: Visual Coarse Localization. In the railway installation environment, the metal material on the surface of the cantilever assembly produces strong specular reflections, resulting in overexposed or underexposed areas in traditional single-exposure images, severely affecting the accuracy of feature extraction and recognition. To solve this problem, this system adopts adaptive multi-exposure fusion technology.

[0020] The robot moves to the preset photo-taking position, and the vision system automatically controls the camera to quickly acquire multiple images of the workpiece at a series of different exposure times (e.g., short, medium, and long exposures).

[0021] A high dynamic range image fusion algorithm is used to synthesize a sequence of images with different exposures into a high-quality image with rich detail and no overexposed or underexposed areas. The fused image is then input into a pre-trained deep learning instance segmentation network, which automatically identifies target parts such as insulators and bushings in the image and outputs their pixel-level precise contours.

[0022] By combining the depth information from the 3D camera, the 3D position and orientation of the identified part in the camera coordinate system are calculated. Using the obtained hand-eye transformation relationship, the pose is transformed to the robot's base coordinate system, resulting in a target pose that the robot can directly use for grasping.

[0023] With the camera positioned at a fixed viewpoint, the exposure time sequence { t 1, t 2, ..., t n}Collected image set{ I 1, I 2, ..., I n For each pixel location in the image ( x , y ), calculate the fusion weights in its images with different exposures. W k ( x , y The weighting function W k ( x , y Taking into account both pixel exposure rationality and local contrast, it is expressed as follows: In the formula: I k ( x , y ) indicates the first k Image at position ( x , y The normalized gray value at ) This represents the gradient magnitude at that location, and is a parameter used to control the appropriateness of the exposure.

[0024] Final high dynamic range fused image I HDR The weighted average is obtained as follows: In the formula: IHDR For the fused high dynamic range image in ( x , y The grayscale value at ().

[0025] A deep learning architecture based on Mask R-CNN is used to achieve accurate identification and segmentation of parts. The network output includes the target's class probability, bounding box coordinates, and pixel-level segmentation mask.

[0026] For the binary mask obtained from segmentation M ( x , y Combined with depth maps acquired by depth cameras D ( x , y The initial pose of the target is calculated using principal component analysis. First, all 3D point sets within the mask region are extracted. P ={ p i},in p i =[ x i , y i , z i ] T .

[0027] Calculate the centroid of the point set: Construct the covariance matrix: By performing eigenvalue decomposition on ∑, the main orientation of the target is obtained, and then the 6-DOF pose in the camera coordinate system is calculated.

[0028] Step 4: Robot Motion and Compliant Grasping. The robot motion planner designs a collision-free motion path based on the target's pose. The robot moves along the path to the grasping point and controls the flexible gripper to perform the grasping action. During this process, force sensor data is monitored in real time. If abnormal contact force or torque is detected, the movement is immediately paused and the posture is fine-tuned to ensure the reliability and non-damage of the grasping process.

[0029] Step 5: Visual Precision Localization. The core of precision localization is solving the actual scanned point cloud. P real Point cloud of theoretical CAD model P model ={ q j Optimal rigid body transformation between} T deviationAfter successfully grasping the part, the robot moves it to a safe height directly above the assembly point on the wrist arm base. First, the 2D camera is triggered to quickly photograph the assembly area. Image processing algorithms (such as Hough circle detection) are used to roughly locate the area of ​​the assembly hole, which is then used as the region of interest to guide the 3D camera for precise scanning, improving efficiency.

[0030] Within the region of interest defined by 2D vision, a 3D structured light camera is triggered to scan and acquire high-density 3D point cloud data at the assembly reference point. An iterative nearest-neighbor point cloud registration algorithm is then executed: the scanned actual point cloud is iteratively aligned with the point cloud of the standard computer-aided design model of the cantilever arm base. In each iteration, the algorithm finds the nearest neighbor point in the model point cloud for each point in the actual point cloud and calculates an optimal rigid body transformation (including rotation and translation) that minimizes the overall distance error between the two sets of point clouds. This process continues iteratively until the transformation matrix converges to a preset accuracy threshold. The final output transformation matrix is ​​the accurate pose deviation matrix of the actual workpiece relative to the theoretical model.

[0031] The recent point-to-point cloud registration algorithm has established the following optimization problem: Make it satisfy: T deviation = Subject to The following constraints are introduced: Two-way distance constraint: Normal vector consistency constraint: Curvature compatibility constraints: In the formula: ω i These are weighting coefficients. This represents the nearest neighbor correspondence. In the point cloud of the theoretical model and p i The corresponding nearest neighbor, R For rotation matrix, t It is a translation vector. p i For actual scanned point cloud P real The first in i Coordinates of three-dimensional points ( x i , y i , z i ), q iPoint cloud for standard CAD models P m odel Zhongyu p i The corresponding number i Three-dimensional point coordinates, This is the maximum allowed distance threshold (to remove incorrect matching point pairs that are too far apart). The angle between the maximum running normal vectors (ensuring that the matching points have similar local surface orientations). To maximize the difference in curvature (ensuring that the matching points have similar local surface shapes). , These are all normal vectors of the corresponding points. This represents the curvature value at the corresponding point.

[0032] Step 6: Online pose compensation. Invert the pose deviation matrix calculated above to obtain the transformation matrix used for robot motion compensation: .

[0033] Applying the compensation matrix to the robot's pre-planned theoretical assembly target pose generates a corrected, new robot motion target pose: In the formula: The theoretical assembly pose (4×4 homogeneous transformation matrix) is based on the CAD model. The actual target pose after compensation.

[0034] The specific processes for coarse and fine positioning are as follows: Figure 2 As shown, the specific process is as follows: (1) Visual guidance begins: system initialization, including camera parameter settings, robot positioning, loading of pre-trained deep learning models and CAD data.

[0035] (2) Image acquisition, multi-exposure sequence acquisition: industrial camera acquires images of the working area to ensure stable lighting conditions; to address the problem of metal reflection, 3-5 image sequences are quickly acquired with different exposure times (5ms, 10ms, 20ms).

[0036] (3) HDR image fusion: Apply weighting functions to perform pixel-level fusion to generate high-quality images without overexposure or underexposure.

[0037] (4) Mask R-CNN instance segmentation: Input HDR fused image, apply deep learning network to perform object detection and pixel-level segmentation, output component category labels, bounding boxes and accurate segmentation mask.

[0038] (5) Is the pose reliable? Determine the judgment criteria. If the conditions are not met, return to re-acquire the image. If the conditions are met, proceed to capture execution.

[0039] (6) Robot performs grasping: Plan a collision-free grasping path and control the flexible gripper to complete the grasping wrist arm action.

[0040] (7) Successful capture: Determine the judgment criteria. If the capture fails, return to start the process again. If the capture is successful, the coarse positioning is completed and the fine positioning stage begins.

[0041] (8) The robot moves to the vicinity of the assembly point: it moves the wrist arm to a safe position 50-100mm away from the assembly point (the base of the column).

[0042] (9) 2D vision rapid ROI positioning: quickly locate the approximate area of ​​the assembly hole (duckbill) and output the coordinate range of the region of interest.

[0043] (10) 3D camera fine scanning to obtain point cloud: high-resolution 3D scanning is performed in the region of interest to obtain local high-precision point cloud data.

[0044] (11) Point cloud quality check: Determine the inspection standard. If the quality is not up to standard, return to 2D positioning. If the quality is up to standard, proceed to point cloud registration.

[0045] (12) Perform constraint ICP registration: apply distance constraints, normal constraints and curvature constraints.

[0046] (13) Whether it converges: Determine the convergence condition. If it does not converge, adjust the corresponding point pair constraints and re-register. If it converges, calculate the final transformation matrix.

[0047] (14) Calculate the pose deviation matrix: output the optimal transformation matrix from the wrist arm to the theoretical model, and calculate the inverse transformation.

[0048] (15) Update robot target pose: Apply compensation and calculate compensated pose.

[0049] (16) Perform pose compensation motion: The robot moves along the compensated path to the new target pose.

[0050] (17) Accuracy verification after compensation: Set the verification method and establish the verification standard. If the verification fails, return to the starting point of the fine positioning and reprocess. If the verification passes, the fine positioning is completed.

[0051] Step 7: Impedance Control Mode Switching and Assembly Status Monitoring. The robot begins to move towards the compensated new target pose. When the robot is about to make contact with the assembly workpiece (e.g., only two millimeters away from the target point), the robot controller smoothly switches from a high-stiffness position control mode to an omnidirectional compliant control mode based on the impedance model.

[0052] In impedance control mode, the robot's end effector is equivalent to a virtual mass-spring-damped system, with the following dynamic equations: in, F ext The external force detected by the torque sensor. M d , B d , K d These are the set inertia, damping, and stiffness matrices, respectively. , , For the desired position, velocity, and acceleration, X , , For the actual position, velocity, and acceleration.

[0053] The behavior of the mass-spring-damped system is determined by inertia, damping, and stiffness parameters, which are pre-set to values ​​that enable the robot to exhibit gentle and compliant characteristics. The core of the mass-spring-damped system lies in dynamically generating pose correction Δ by measuring contact forces. X = X - X d This enables the robot to "comply" with environmental constraints in its movement.

[0054] The assembly process is managed by a finite state machine, which intelligently switches between different states based on real-time readings from force / torque sensors, such as... Figure 3 As shown: Search status: The robot end effector moves slowly along the assembly axis while monitoring contact forces. If a significant lateral force is detected... If a torque indicates eccentric contact, the system transitions to a "contact alignment state"; if a steadily increasing axial resistance is primarily detected... F z If it does, it will directly switch to "insertion state".

[0055] Contact Alignment State: Based on feedback from lateral force / torque, the robot actively adjusts its end effector posture in a plane perpendicular to the axis to eliminate eccentric contact and achieve self-alignment of the hole and shaft. When the lateral force decreases below the safety threshold, alignment is considered complete, and the robot transitions to the "insertion state".

[0056] The exercise adjustment amount is expressed as follows: = K p · Fside In the formula: Adjust the amount of exercise. F side It is a lateral force. K p This is the proportionality coefficient.

[0057] Insertion state: The robot maintains its current alignment posture with a constant or slowly increasing axial force. F zd The insertion process of the part is completed. Force data is continuously monitored throughout the process. If an abnormal lateral force is detected again, the system returns to the "contact alignment state" to release the jamming. If the axial force suddenly increases and exceeds the preset positioning threshold... If the assembly is successful, the process will proceed to the "completed state".

[0058] Completion status: The robot stops moving, releases its gripper, records successful assembly information, and the process ends.

[0059] Step 8: Full-process data recording. The system automatically records key data throughout the entire assembly process, including: original images, processed point clouds, six-dimensional force / torque curves, robot motion trajectories, and state machine switching sequences.

[0060] Step 9: System Performance Analysis and Parameter Optimization. Based on the accumulated assembly process data, statistical analysis is performed on the system performance, and key indicators such as success rate and average assembly time are calculated.

[0061] By utilizing the data analysis results, impedance control parameters, state machine switching thresholds, and vision processing parameters are iteratively optimized to provide support for further improving the system's assembly efficiency, success rate, and robustness.

[0062] Based on the above method, this embodiment builds an experimental platform, which consists of the following parts: an ABB six-degree-of-freedom industrial robot, a composite vision system consisting of a Hikvision MV-CH050-10UM (2D) and a Lingyunguang 3D structured light camera integrated at the end effector; an adaptive two-finger flexible gripper; and an industrial control computer running the Ubuntu system and ROS (Robot Operating System) framework for integrating and implementing algorithms.

[0063] Based on the above experimental platform, visual positioning accuracy, assembly success rate, and efficiency were tested. Details are as follows: Visual positioning accuracy test: The visual positioning measurement of the arm assembly hole was repeated 20 times on a fixed fixture. The comparison between this method and the traditional 2D vision method, using the measurement value of the laser tracker as a benchmark, is shown in Table 1 below.

[0064] Table 1 Assembly success rate and efficiency test: To evaluate the overall performance, under the condition of introducing an initial positional deviation of ±2mm, 50 insulator assembly experiments were conducted using three strategies. The assembly results are shown in Table 2 below.

[0065] Table 2 Experiments have shown that the method of the present invention, based on visually precise pre-positioning, uses force control only for fine-tuning, which greatly shortens the blind search time. At the same time, thanks to its intelligent compliance, it achieves the highest assembly success rate. Figure 4 Typical force curves of the robot end effector during the assembly process are shown under three strategies. The force curve of the method of this invention is the smoothest and there is no impact peak.

[0066] Figure 4 This demonstrates the axial force measured by the robot's end effector's six-dimensional force / torque sensor under three different assembly strategies, with the same initial deviation conditions. F z A typical curve showing how the process changes over time. This curve clearly reflects the differences in assembly dynamics and performance under different strategies.

[0067] Figure 4 The calculation results show that the force curve of pure position control exhibits high risk, and its sharp force peaks prove that this method cannot adapt to small alignment errors, which can easily lead to assembly failure or part damage. The force curve of pure force control search is the smoothest and safest, but the long search phase results in extremely low efficiency. The force curve of the method of this invention combines high efficiency and high safety. It inherits the smooth characteristics of pure force control search and avoids impact forces; at the same time, because the precise positioning of vision greatly shortens the search range of force control, its assembly time is much shorter than that of pure force control search, achieving the best balance of performance.

[0068] This invention successfully solves two core challenges in precision assembly—"inaccurate vision" and "unstable assembly"—through a closed-loop collaboration of vision and force sensing. It is not a simple improvement on existing technology, but rather a systematic integrated innovation that achieves a qualitative leap in assembly performance, providing solid technical support and a feasible practical path for realizing high-precision, high-reliability, and high-efficiency robotic operations in uncertain environments.

[0069] Compared with existing technologies, the method of this invention achieves a qualitative leap in visual guidance and force control, moving from "series-independent" to "closed-loop fusion." Traditional methods typically treat vision and force control as two independent processes, with vision exiting after initial positioning. This results in the robot becoming stuck due to a lack of "tactile feedback" when faced with micron-level assembly deviations caused by part tolerances and positioning errors. This invention, through a layered strategy of "global visual positioning and local fine-tuning of force control," deeply integrates the macroscopic pose guidance of vision with the microscopic compliance adjustment of force, giving the robot both "keen eyes" and "skillful hands." This not only effectively solves the problem of unstable identification and positioning of complex parts with strong reflectivity but also endows the robot with the ability to actively perceive contact states and intelligently adjust assembly posture. This fundamentally breaks through the limitations of single-sensor operation, significantly increasing the assembly success rate from approximately 62% in traditional methods to over 90%, achieving a balance between high precision and high reliability.

[0070] This invention successfully overcomes the dilemma of the trade-off between "precision" and "efficiency" in precision assembly. While a pure force-controlled search strategy can guarantee eventual success, its blind search in unknown environments results in an average assembly time of up to 12.8 seconds, which is insufficient to meet industrial cycle time requirements. This invention utilizes vision for sub-millimeter-level precise positioning, pre-positioning the robot's starting assembly point within the optimal working range of force control. This allows force control to make only extremely minor compensation adjustments, significantly reducing the average assembly time and improving efficiency by over 50%. Furthermore, this research is not merely an isolated algorithm improvement, but rather provides a complete solution integrating anti-reflective vision, intelligent force control strategies, and a unified software platform. This system-level integration ensures high robustness and reusability of the technology, making it applicable not only to wrist arm assembly but also providing a replicable and scalable advanced paradigm for precision assembly scenarios throughout the manufacturing industry, demonstrating significant engineering application value.

[0071] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its scope and protection, and such modifications or equivalent substitutions should also be considered to fall within the protection scope of the present invention.

Claims

1. A catenary cantilever robot assembly method based on hand-eye-force integration, characterized in that, Comprising the following steps: Step 1, system calibration and parameter initialization: determine the spatial transformation relationship between the camera coordinate system and the robot end coordinate system, establish the accurate mapping relationship between the force sensor coordinate system and the robot end coordinate system, and verify the calibration results; Step 2, visual coarse positioning: based on adaptive multi-exposure fusion technology, different exposure image sequences are synthesized into high-quality images, based on a deep learning framework, the workpiece contour in the image is identified, combined with the depth information of the three-dimensional camera, the pose of the identified part in the camera coordinate system is calculated, and then converted into a grasping target pose available for the robot; Step 3, robot motion and compliant grasping: the robot moves to the grasping point based on the grasping target pose obtained in step 2, and performs the grasping action, and monitors the force sensor data in real time, and moves to the wrist arm base assembly point above the safety height after the grasping is completed; Step 4, visual fine positioning: the two-dimensional camera roughly locates the assembly hole area, and guides the three-dimensional camera to perform accurate scanning with the area as the region of interest, and the accurate pose deviation matrix of the actual workpiece relative to the theoretical model is obtained through the nearest point point cloud registration algorithm; Step 5, online pose compensation: the pose deviation matrix obtained in step 4 is inversely applied to the robot to realize the planned theoretical assembly target pose, and a corrected motion target pose is generated; Step 6, impedance control mode switching and assembly state monitoring: the robot moves to the motion target pose obtained in step 5, and when the robot and the assembly workpiece are about to contact, the impedance control mode is switched to install the wrist arm, and the torque data is continuously monitored until the installation is completed.

2. The hand-eye-force integrated catenary boom robot assembly method according to claim 1, wherein, The step 2 adaptive multi-exposure fusion technology is specifically: a two-dimensional camera collects an image set { t 1, t 2, …, t n} under a fixed viewpoint with an exposure time sequence { I 1, I 2, …, I n}, for each pixel position ( x , y ) in the image, a fusion weight W k ( x , y ) in different exposure images of the pixel position is calculated, and a high dynamic range fusion image is obtained through weighted average.

3. The hand-eye-force integrated catenary boom robot assembly method according to claim 2, wherein, The fusion weight W k ( x , y ) The specific calculation formula is: where I k(x, y) denotes the normalized gray value of the k-th image at position (x, y), denotes the gradient magnitude at this position, denotes a parameter for the reasonability of controlling exposure; The specific calculation formula of the high dynamic range fused image is: where I HDR is the luminance value of the fused high dynamic range image at (x, y).

4. The hand-eye-force integrated catenary arm robot assembly method according to claim 1, characterized in that, The step 4 visual fine positioning core is to solve the actual scanning point cloud P real The optimal rigid transformation between the theoretical CAD model point cloud P model ={ q j} and the actual scanning point cloud T deviation The following optimization problem is established: T deviation = Subject to Introducing bi-directional distance constraints: , Normal vector consistency constraint: , Curvature compatibility constraint: where ω i is a weighting coefficient, is the nearest neighbor correspondence, is the theoretical CAD model point cloud corresponding to p i , R is a rotation matrix, t is a translation vector, p i is the i-th three-dimensional point coordinate (x i , y i , z i ) in the actual scanned point cloud P real , q i is the i-th three-dimensional point coordinate corresponding to p model in the theoretical CAD model point cloud P i , is the maximum allowed distance threshold, is the maximum running normal vector angle, is the maximum running curvature difference, , are the normal vectors of the corresponding points, is the curvature value at the corresponding point.

5. The hand-eye-force integrated catenary arm robot assembly method according to claim 1, wherein, The impedance control mode in step 6 is specifically: the robot end is equivalent to a six-degree-of-freedom mass-spring-damper system, whose behavior is determined by inertia, damping and stiffness parameters, and the assembly process is managed by a finite state machine, which switches between search state, contact alignment state, insertion state and completion state according to the real-time readings of the torque sensor.

6. The hand-eye-force integrated catenary arm robot assembly method according to claim 1, wherein, The wrist arm installation process in step 6 under the impedance control mode is specifically: in the search state, the robot end slowly moves along the assembly axis direction while monitoring the contact force, and if significant lateral force or torque is detected, it will enter the contact alignment state; if mainly axial resistance is detected, it will directly enter the insertion state; Contact alignment state: the robot actively adjusts its end pose in the plane perpendicular to the axis according to the feedback of lateral force or torque, realizes the self-alignment of the hole axis, and when the lateral force decreases below the safety threshold, it is considered that the alignment is completed, and enters the insertion state; Insertion state: the robot maintains the current alignment pose, and completes the insertion process of the part with constant or slowly increasing axial force, and continuously monitors the torque data throughout the process; if abnormal lateral force is detected again, it will return to the contact alignment state; if the axial force suddenly jumps and exceeds the preset in-place threshold, it is judged that the assembly is successful, and enters the completion state; Completion state: the robot stops moving, releases the gripper, and records the assembly success information.

7. The hand-eye-force integrated catenary boom robot assembly method of claim 6, wherein, The robot end motion adjustment amount in the disalignment state is represented as: = K p · F side wherein, is the motion adjustment amount, F side is the lateral force, K p is the proportionality factor.

Images