Visual migration mechanical arm double-arm cooperative assembly control method and system

Abstract

The present application relates to the technical field of intelligent control of industrial robots, and particularly relates to a vision transfer mechanical arm dual-arm cooperative assembly control method and system. The method comprises the following steps: collecting the action posture of the mechanical arm dual arms and the relative pose information between the dual arms in real time; identifying the process switching node of the current assembly process stage; constructing the master-slave role dynamic control logic of the dual-arm cooperation, setting the initial motion control parameters of the dual-arm cooperation based on the collected dual-arm posture and relative pose information, and determining the initial master-slave role division of the dual arms. The present application realizes seamless connection of processes, adaptation to various tasks and high-precision stable assembly through posture node identification technology, dynamic adjustment cooperation technology and vision transfer technology, so as to improve the assembly process efficiency, the mechanical arm adaptation ability and the overall assembly quality.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for industrial robots, and in particular to a vision-transfer-based dual-arm collaborative assembly control method and system for robotic arms. Background Technology

[0002] In existing technologies, the collaborative assembly control of robotic arms has significant technical limitations. Specifically, traditional control methods often rely on offline programming or fixed trigger conditions to achieve process switching, lacking dynamic and accurate monitoring of the real-time motion posture and three-dimensional relative pose of the two arms. Assembly processes are often interrupted due to node recognition delays or misjudgments. The master-slave role allocation is mostly fixed, and motion priority and positioning accuracy parameters require manual adjustment by stopping the machine. It cannot automatically adapt to the load requirements and accuracy standards of the target process, making it difficult to cope with the rapid switching of diverse assembly tasks. At the same time, the assembly process lacks a complete closed-loop control mechanism, relying only on initial positioning guidance. It cannot capture assembly accuracy deviations and changes in part status in real time, nor can it dynamically adjust collaborative parameters. It is prone to assembly quality degradation or workpiece damage due to positional deviations and unstable clamping, making it difficult to achieve high-precision and high-stability assembly operations with dual-arm collaboration. Summary of the Invention

[0003] Therefore, it is necessary to provide a vision-transfer robotic arm dual-arm collaborative assembly control method and system to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, a vision-transfer-based robotic arm dual-arm collaborative assembly control method is provided, the method comprising the following steps: Step S1: Real-time acquisition of the movement postures of the two robotic arms and the relative position information between the two arms; identification of the process switching nodes in the current assembly process stage; Step S2: Construct the master-slave dynamic control logic for dual-arm collaboration. Based on the collected dual-arm posture and relative pose information, set the initial master-slave collaboration motion control parameters for dual-arm collaboration and determine the initial master-slave role division of dual arms. Step S3: When a process switching node is identified, adjust the movement priority and positioning accuracy parameters of the two arms in the master-slave role control mechanism according to the collaboration requirements of the target process to complete the master-slave role switching; Step S4: When the robotic arm performs the assembly operation according to the current role control parameters, it also controls the dual-arm collaborative assembly parameters based on the real-time collected assembly accuracy and part status information until the current assembly operation is completed.

[0005] The present invention also provides a vision-transfer robotic arm dual-arm collaborative assembly control system for the above-mentioned vision-transfer robotic arm dual-arm collaborative assembly control method, the vision-transfer robotic arm dual-arm collaborative assembly control system comprising: The posture node recognition module is used to collect the movement postures of the two arms of the robotic arm and the relative posture information between the two arms in real time; and to identify the process switching nodes of the current assembly process stage. The master-slave logic initialization module is used to construct the master-slave role dynamic control logic for dual-arm collaboration. Based on the collected dual-arm posture and relative pose information, it sets the initial master-slave collaboration motion control parameters and determines the initial master-slave role division of the dual arms. The role switching adjustment module is used to adjust the movement priority and positioning accuracy parameters of the two arms in the master-slave role control mechanism according to the collaboration requirements of the target process when a process switching node is detected, so as to complete the master-slave role switching. The collaborative assembly parameter control module is used to control the collaborative assembly parameters of the two arms based on real-time collected assembly accuracy and part status information when the robotic arm performs the assembly operation according to the current role control parameters, until the current assembly operation is completed.

[0006] The beneficial effects of this invention are as follows: I. Improve the accuracy and efficiency of process connection. The posture node recognition module collects and analyzes the posture and relative position information of the two arms in real time to achieve accurate positioning of process switching nodes, ensures seamless connection of each assembly process, reduces assembly stagnation caused by node recognition delay, and significantly improves the continuity and efficiency of the overall assembly process.

[0007] Second, enhance the dynamic adaptability of dual-arm collaboration. The master-slave logic initialization module and the role switching adjustment module work together to build a dynamic master-slave control logic that can flexibly adjust the movement priority and positioning accuracy parameters of the two arms according to the collaboration requirements of different processes. This enables the robotic arm to quickly adapt to diverse assembly task requirements and achieve dynamic optimization of the collaboration mode.

[0008] Third, to ensure high precision and stability of assembly operations, the collaborative assembly parameter control module dynamically adjusts the collaborative parameters of the two arms based on real-time feedback of assembly precision and part status information. This effectively controls positional and angular deviations during the assembly process, while stabilizing the clamping state of parts, significantly improving the precision control level of assembly operations and the consistency of overall assembly quality. Attached Figure Description

[0009] Figure 1 A flowchart illustrating the steps of a vision-transfer robotic arm dual-arm collaborative assembly control method. Figure 2 To obtain a schematic diagram of the spatial position of the two arms; Figure 3 This is a schematic diagram of a robotic arm with two arms; The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0010] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0011] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0012] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0013] To achieve the above objectives, please refer to Figures 1 to 3 A vision-transfer robotic arm dual-arm collaborative assembly control method, the method comprising the following steps: Preferably, step S1: Real-time acquisition of the movement postures of the two arms of the robotic arm and the relative posture information between the two arms; identification of the process switching node of the current assembly process stage; Optionally, step S1 includes: Collect rotational angle data of each joint of both arms and simultaneously obtain the spatial position coordinates of both arms; Based on the rotation angle data and spatial position coordinates, the straight-line distance and relative angle between the end effectors of the two arms are calculated to form the motion state data of the two arms; Based on the motion state data of the two arms, the spatial position features of the parts to be assembled and the geometric state features of the assembly interface are extracted. The spatial location features and geometric state features are compared with the preset feature thresholds for each process stage. When the part position enters the workspace corresponding to the next process and the interface status meets the switching conditions, it is marked as a process switching node.

[0014] Please see Figure 2In this embodiment, eight absolute photoelectric encoders (600P / R resolution, 1000Hz sampling frequency) of model E6B2-CWZ6C are installed at eight joints of the robotic arm to collect rotational data of each joint in real time. Simultaneously, two BASLERacA2500-14uc binocular vision sensors 101 (calibration error ≤0.02mm, frame rate 30fps) are used to obtain the three-dimensional spatial coordinates (unit: mm) of the link in the base coordinate system {0} through hand-eye calibration technology.

[0015] It should be noted that, based on the collected joint rotation angle data and link spatial coordinates, the coordinates of the end effector of the left robotic arm 102, P1(x1,y1,z1), and the coordinates of the end effector of the right robotic arm 103, P2(x2,y2,z2), are determined. The straight-line distance between the two is obtained through spatial geometric calculations, and the relative angle is obtained through vector calculations (where V1 is the vector from P1 to the origin of the left arm link, and V2 is the vector from P2 to the origin of the right arm link). The rotation angle data, link spatial coordinates, end effector straight-line distance, and relative angle are integrated into a dual-arm motion state dataset (stored in CSV format, with timestamps accurate to the millisecond level).

[0016] It should be noted that, based on the dual-arm motion state dataset, the center coordinates (x0, y0, z0), volume (120mm × 80mm × 50mm), and surface normal vector N (0.2, 0.3, 0.5) of the part to be assembled are extracted as spatial position features through image pixel analysis and 3D reconstruction functions of the binocular vision sensor; the diameter (30mm), depth (20mm), inner surface roughness Ra=0.6μm, and end face circular runout tolerance (0.02mm) of the assembly interface are extracted as geometric state features through the precision measurement mode of the vision sensor.

[0017] In another embodiment, the preset workspace thresholds for parts in processes 1 to 2 are x∈[50,200]mm, y∈[30,180]mm, and z∈[20,150]mm. The assembly interface switching conditions are diameter deviation ≤ ±0.05mm, end face circular runout tolerance ≤ 0.03mm, and surface roughness Ra≤ 0.8μm. The extracted part center coordinates (x0, y0, z0) are compared with the workspace thresholds one by one, and the diameter, circular runout tolerance, and surface roughness Ra of the assembly interface are compared with the switching condition thresholds. When the part center coordinates fall completely into the workspace range of the next process and all interface status parameters meet the switching conditions, the system automatically marks this moment as a process switching node. The node information includes a timestamp, current process number 1, next process number 2, and the above-mentioned key feature parameters.

[0018] Preferably, step S2: construct the master-slave role dynamic control logic for dual-arm collaboration, based on the collected dual-arm posture and relative pose information, set the initial master-slave collaboration motion control parameters for dual-arm collaboration, and determine the initial master-slave role division of dual arms; Most importantly, the dynamic control logic for the master-slave roles in the dual-arm collaboration and the setting of initial parameters include: Construct a master-slave role dynamic control logic, which includes the role determination relationship, the parameter mapping correspondence relationship and the boundary constraint limitation condition. The role determination relationship is bound to the structural characteristic parameters of the two arms, the parameter mapping correspondence pre-stores the motion control parameter range under different division of labor modes, and the boundary constraint limitation condition integrates the motion limit value of the two arm joints. The robot receives the collected posture and relative pose information of the two arms, extracts the spatial position distribution and relative distance of the end effectors of the two arms, compares the structural characteristic parameters of the two arms through the correlation relationship of role determination, and temporarily designates the robot arm with high-precision operating tools as the initial master role and the robot arm with heavy-duty clamping structure as the initial slave role. Based on the overlap of the workspace of the two arms in the relative pose information, the fine positioning parameter range of the initial protagonist color and the clamping stability parameter range of the initial slave character are called through the correspondence of parameter mapping. Combined with the joint rotation angle in the current posture of the two arms, the parameter range is adaptively compressed to determine the specific value of the initial motion control parameters. The initial role assignments and corresponding motion control parameters are written into the execution cache of the master-slave role dynamic control logic to complete the logic construction and parameter initialization.

[0019] In this embodiment, a visual sensing system (resolution 1920×1080 pixels, spatial positioning accuracy ±0.01mm, sampling frequency 200Hz) is used to construct the master-slave dynamic control logic. This logic includes the correlation of role determination, the correspondence of parameter mapping, and the limiting conditions of boundary constraints. The correlation of role determination is bound to the structural characteristic parameters of the two arms (A arm repeatability accuracy ±0.02mm, tool end diameter 5mm; B arm maximum clamping force 500N, load limit 40kg). The correspondence of parameter mapping pre-stores the fine positioning parameter range of the master role (position deviation ±0.02mm, angle deviation ±0.1°) and the clamping stability parameter range of the slave role (clamping force 80-150N, clamping deformation ≤0.03mm). The limiting conditions of boundary constraints integrate the limit values ​​of the joint motion of the two arms (shoulder rotation angle -180° to 180°, elbow rotation angle 0° to 120°, wrist rotation angle -90° to 90°). For example, the visual sensing system receives and collects the posture and relative pose information of the two arms, extracts the spatial coordinates of the end effector of arm A (X=320mm, Y=250mm, Z=180mm) and the spatial coordinates of the end effector of arm B (X=350mm, Y=260mm, Z=180mm), calculates the relative distance between them as 31.62mm, and compares the structural characteristic parameters of the two arms through the correlation of role determination. Arm A, which has high-precision tool operation, is tentatively designated as the initial master role, and arm B, which has a heavy-duty clamping structure, is tentatively designated as the initial slave role. Based on relative pose information, the overlap of the workspace of the two arms is calculated to be 60%. The fine positioning parameter range of the initial protagonist color and the clamping stability parameter range of the initial follower color are called through the correspondence of parameter mapping. Combined with the joint angles in the current posture of the two arms (arm A shoulder 30°, elbow 45°, wrist 20°; arm B shoulder 25°, elbow 50°, wrist 15°), spatial geometric operations are used to adaptively compress the parameter range to determine the specific values ​​of the initial motion control parameters (arm A position deviation ±0.015mm, angle deviation ±0.08°). It should be noted that the clamping force of arm B is 90-130N and the clamping deformation is ≤0.025mm; the initial role division (arm A is the master role and arm B is the slave role) and the corresponding motion control parameters are written into the execution buffer of the master-slave role dynamic control logic through the industrial bus to complete the logic construction and parameter initialization.

[0020] Optionally, the following operations may be performed when setting the initial motion control parameters in step S2: Obtain the weight of the part to be assembled, and determine the adjustment range of the clamping force by combining it with the structural strength parameters of the robotic arm; After the clamping force adjustment range parameters are set, the relative position of the clamping point of the robotic arm and the center of gravity of the part is collected, and an alignment detection signal is generated. If the alignment of the alignment detection signal does not meet the requirements, the clamping force parameters are readjusted.

[0021] In one embodiment, a high-precision weighing sensor of model HBMC16i (measuring range 0-50kg, accuracy ±0.01kg) is installed on the inner side of the clamping surface of the robotic arm from the gripping end of the role, and the actual weight of the part to be assembled is collected in real time as 3.5kg; the structural strength parameters of the robotic arm are retrieved, including the yield strength of the double arm connecting rod material of 300MPa, the maximum bearing torque of the joint of 50N・m, and the allowable stress of the end effector clamping structure of 250MPa. Combined with the actual weight of the part, the adjustment range of the clamping force is calculated and determined to be 80N-120N.

[0022] In another embodiment, after the clamping force adjustment range parameters are written to the force control module of the robotic arm's main controller via an industrial bus, two KeyenceIL-300 laser displacement sensors (measuring range 0-300mm, resolution 0.01mm, sampling frequency 500Hz) are activated to align with the clamping points of the left and right robotic arms and the center of gravity marker of the part, respectively. The sensors collect data on the horizontal distance of 8mm and the vertical distance of 5mm from the left clamping point to the center of gravity of the part, and the horizontal distance of 7.8mm and the vertical distance of 5.2mm from the right clamping point to the center of gravity of the part. Alignment detection is then generated based on this distance data. The alignment criteria for this signal are: horizontal deviation of the clamping point from the center of gravity ≤ ±1mm and vertical deviation ≤ ±1mm. The test results show that the horizontal deviation of the left clamping point exceeds the standard range by 0.2mm. The clamping force correction program is immediately started, and the clamping force of the left robotic arm is adjusted from the initial setting of 100N to 105N, while the clamping force of the right robotic arm remains unchanged at 100N. The relative position data is collected again through the laser displacement sensor until the alignment detection signal shows that the horizontal and vertical deviations of the clamping point from the center of gravity of the part meet the judgment criteria. The final setting of the clamping force parameters is then completed.

[0023] Of particular importance, the correction process for the clamping force parameter includes: By visual transfer, the center of gravity marker on the surface of the part and the gripping contact point of the gripper at the end of the robot arm are identified, and their coordinate data in three-dimensional space are extracted. Based on the coordinate data, the straight-line distance between the gripping contact point and the center of gravity marker of the part is calculated, and the offset distance value is determined. Based on the number and distribution structure of the claws of the robotic arm gripper, a mapping relationship between offset distance and gripping force compensation value is established, where claws that offset towards the center of gravity correspond to positive compensation, and claws that offset away from the center of gravity correspond to reverse compensation. The clamping force adjustment of each claw is calculated according to the mapping relationship, and the force control joint of the clamping device is driven to perform the adjustment so that the clamping force of each claw is distributed proportionally until the point of application of the resultant clamping force is confirmed to coincide with the coordinate of the center of gravity of the part through mechanical calculation. After the adjustment is completed, the posture data of the part in the clamping state is continuously collected through visual transfer, and the position change of the feature points on the edge of the part is extracted; if the position change of the feature points is within the clamping stability threshold allowed by the robot arm structure, it is determined that the part has no tilt or displacement. After confirming that the clamping state is stable, the current clamping force parameters of each claw are stored in the parameter cache area of ​​the control logic and locked as the final clamping force parameters of the part.

[0024] In one embodiment, a BASLERacA4096-30um industrial vision camera (resolution 4096×3000 pixels, frame rate 30fps, calibration error ≤0.01mm) is used in conjunction with vision transfer technology to capture real-time images of the part to be assembled and the gripper at the end of the robotic arm. The system accurately identifies the pre-set center of gravity marker A (coordinates X=250mm, Y=180mm, Z=120mm) on the surface of the part, as well as the gripping contact points B1 (248mm, 178mm, 120mm), B2 (252mm, 178mm, 120mm), B3 (252mm, 182mm, 120mm), and B4 (248mm, 182mm, 120mm) of the four evenly distributed claw petals (angle interval 90°) of the gripper. The complete coordinate data of all points are obtained through the three-dimensional coordinate extraction module built into the vision system.

[0025] It should be noted that, based on these coordinate data, spatial geometric calculations show that the straight-line distance from each clamping contact point to the center of gravity marker A is 2.83 mm, thus determining a uniform offset distance value of 2.83 mm. Based on the 4-jaw ring distribution structure of the gripper, a fixed mapping relationship between the offset distance and the clamping force compensation value is established. A clamping force compensation of 5 N is set for every 1 mm of offset distance. It is determined that the jaws pointing towards the center of gravity receive positive compensation, while those moving away from the center of gravity receive reverse compensation. Using an initial clamping force of 100 N as a baseline, the adjustment amount for each jaw is calculated: jaws B1 and B3, pointing towards the center of gravity, each receive 14.15 N of positive compensation, resulting in a clamping force of 114.15 N; jaws B2 and B4, moving away from the center of gravity, each receive 14.15 N of reverse compensation, resulting in a clamping force of 85.85 N.

[0026] In another embodiment, the gripper's built-in FUTEKLCM300 force-controlled joint (control accuracy ±0.1N, response time ≤10ms) drives each claw to precisely perform adjustment actions until mechanical calculations confirm that the point of application of the clamping force completely coincides with the coordinates of the center of gravity marker A. After adjustment, the vision camera maintains continuous acquisition, and the gripping posture data of the part is captured in real time using visual transfer technology. The positional changes of four preset feature points (C1-C4) on the edge of the part are extracted. The gripping stability threshold set by the robotic arm is ≤0.05mm for the positional changes of the feature points. The actual acquisition results show that the positional changes of the four feature points are all controlled within 0.03mm. After confirming that the part is not tilted or displaced, the final clamping force parameters of each claw (B1=114.15N, B2=85.85N, B3=114.15N, B4=85.85N) are transmitted to the parameter buffer of the master-slave dynamic control logic via high-speed industrial Ethernet. The set of parameters is locked by system control commands and used as the fixed clamping force parameters for assembling this type of part.

[0027] Preferably, step S3: when a process switching node is identified, the motion priority and positioning accuracy parameters of the two arms in the master-slave role control mechanism are adjusted according to the collaboration requirements of the target process to complete the master-slave role switching; Optionally, when a process switching node is detected in step S3, adjusting the movement priority of the two arms includes: Identify the operating structure of the master robot arm and the heavy-duty clamping structure of the slave robot arm; The weight of the parts and the required operational precision are obtained through visual transfer. Configure the operation response threshold for the master character robotic arm according to the operation accuracy requirements, and configure the clamping response threshold for the slave character robotic arm according to the weight of the part. Real-time monitoring of the joint angles between the main character's robotic arm and the slave character's robotic arm; when the joint angle of the main character's robotic arm touches the limit of the operating structure, switch to the limit adaptation response threshold.

[0028] In one embodiment, an intelligent vision sensor (resolution 1280×1024 pixels, recognition accuracy ±0.01mm, response time ≤10ms) is used to identify the dual-arm structure of the robotic arm in real time, determining that the left robotic arm is the main role operating structure equipped with an electric gripper (repeat positioning accuracy ±0.02mm, maximum gripping force 50N), and the right robotic arm is the slave role heavy-duty gripping structure equipped with a heavy-duty hydraulic gripper (maximum gripping force 500N, load capacity ≤50kg). It should be noted that, through this vision sensor combined with vision transfer technology, the weight data of the part to be assembled (verified to be 8kg by the sensor's built-in weighing module) and the operational accuracy requirements of the current target process are simultaneously collected (positioning accuracy ≤ ±0.03mm, motion synchronization accuracy ≤ 50ms). Based on these operational accuracy requirements, an operational response threshold is configured for the left robotic arm of the master character (joint start response time ≤ 20ms, position feedback response delay ≤ 10ms). Based on the 8kg weight parameter of the part, a clamping response threshold is configured for the right robotic arm of the slave character (hydraulic system pressure build-up response time ≤ 30ms, clamping force stabilization response delay ≤ 15ms). In another embodiment, eight incremental encoders (1024P / R resolution, 500Hz sampling frequency) are installed at each joint of the two arms to monitor the joint rotation data of the master and slave robotic arms in real time. The operating structure limit angle of the shoulder joint of the master robot is set to 90°. When the encoder detects that the joint rotation angle reaches the 90° limit value, the operation response threshold of the master robot is automatically switched to the limit adaptation response threshold (joint start response time ≤15ms, position feedback response delay ≤8ms). At the same time, the clamping response threshold of the slave robot remains unchanged, ensuring that the movement priority of the two arms after the process switch is accurately matched with the target process requirements.

[0029] Optionally, the adjustment of the positioning accuracy parameters of the two arms in the master-slave role control mechanism in step S3 is specifically as follows: Collect the normal angle data between the force sensor gripper and the assembly interface, and map the normal angle data into the joint angle correction amount of the gripper; The positioning accuracy parameters are adjusted based on the correction amount to make the end positioning deviation match the positioning tolerance of its own structure, and the parallelism of the mating surface is checked simultaneously, generating a parallelism correction signal and feeding it back to the controller.

[0030] In one embodiment, a visual sensing system with integrated force feedback (resolution 1280×1024 pixels, angle measurement accuracy ±0.05°, sampling frequency 300Hz) is used to collect the normal angle data between the force sensing gripper and the assembly interface at the end of the master and slave robot arms in real time. The measured normal angle between the master gripper and the assembly interface is 2.3°, and the normal angle between the slave gripper and the assembly interface is 1.8°. It should be noted that, through the preset angle-joint angle mapping rule (each 1° normal angle corresponds to a 0.06° joint angle correction), the main character's 2.3° normal angle is mapped to a 0.138° gripper joint angle correction, and the character's 1.8° normal angle is mapped to a 0.108° joint angle correction. Based on this correction, the positioning accuracy parameters of the two arms are adjusted. The initial allowable value for the main character's end-effector positioning deviation is ±0.05mm, and the structural positioning tolerance is ±0.02mm. After adjustment, the main character's end-effector positioning deviation is controlled within ±0.02mm. Similarly, the initial allowable value for the character's end-effector positioning deviation is ±0.08mm, and the structural positioning tolerance is ±0.04mm. After adjustment, the positioning deviation is controlled within ±0.04mm, ensuring that the end-effector positioning deviation and the structural positioning tolerance are precisely matched. For example, four sets of symmetrical feature points on the assembly mating surface are extracted using a visual sensing system. These four sets of feature points are evenly distributed in a ring around the center of the assembly interface, with a central angle of 90° between adjacent feature points. They are feature points P1 (15mm from the center, 0° direction), P2 (15mm from the center, 90° direction), P3 (15mm from the center, 180° direction), and P4 (15mm from the center, 270° direction). The angle between the normal vectors of the lines connecting P1 and P3 and P2 and P4 is calculated. The measured angle between the normal vectors of the master and slave grippers and the assembly interface mating surface is 1.2°. The preset parallelism standard value is ≤0.3°. Since the measured value exceeds the standard range, the system generates a parallelism correction signal (including parameters with a clockwise correction direction and a correction angle of 0.9°).

[0031] Optionally, checking the parallelism of the mating surfaces includes: Extract the coordinates of the edge feature points of the mating surface and calculate the angle between the normal vectors of the lines connecting the feature points; The included angle is compared with a preset parallelism standard value. When it exceeds the standard range, a correction signal is generated. After receiving the signal, the controller adjusts the joint angle of the gripper according to the preset step size until the included angle meets the requirements.

[0032] In one embodiment, a visual sensing system (resolution 1920×1080 pixels, coordinate extraction accuracy ±0.01mm, sampling frequency 200Hz) is used to continuously photograph the assembly mating surface, extracting the coordinates of four sets of symmetrical feature points on the edge of the mating surface. These four sets of feature points are diagonally symmetrically distributed with reference to the assembly interface center O (X=300mm, Y=200mm, Z=150mm), and are respectively feature point A (X=290mm, Y=190mm, Z=150mm). Feature points A (X=310mm, Y=210mm, Z=150mm), B (X=310mm, Y=190mm, Z=150mm), C (X=310mm, Y=190mm, Z=150mm), and D (X=290mm, Y=210mm, Z=150mm) are identified. The normal vector N1 of the line connecting feature points A and B, and the normal vector N2 of the line connecting feature points C and D are calculated using spatial vector operations. The angle between the two normal vectors is found to be 1.5°. For example, the preset standard value for the parallelism of the mating surfaces is ≤0.3°. The calculated 1.5° normal vector angle is compared with this standard value to confirm that the measured angle exceeds the standard range. The vision sensing system immediately generates a parallelism correction signal, which contains the core parameters of a correction direction of counterclockwise, a single adjustment step of 0.1°, and a target angle of 0.3°. After receiving the correction signal, the main controller of the robotic arm drives the joint of the gripper to perform an angle adjustment operation according to the preset single adjustment step of 0.1°. After each adjustment, the feature point coordinates are re-extracted and the normal vector angle is calculated through the vision sensing system. After 12 cycles of adjustment, the final measured normal vector angle is 0.25°, which meets the preset parallelism standard value requirement, and the joint angle adjustment operation is stopped.

[0033] Please see Figure 3 The robotic arm includes a dual-arm structure, which comprises a large arm and a forearm. Step S3 also includes performing the following operations when planning an obstacle avoidance path based on the dual-arm structure: Obtain the deployment angles of the upper and lower arms of both arms, and calculate the turning avoidance radius corresponding to different link lengths; For a robotic arm with an omnidirectional rotating structure, plan the continuous rotation trajectory of the shoulder joint; For robotic arms with folding and limiting structures, plan the segmented folding trajectory of the elbow joint; The distance between links is monitored in real time by visual migration, and the coordinates of trajectory nodes are dynamically adjusted to maintain a safe threshold.

[0034] In one embodiment, the unfolding angles of the upper arm 201 and lower arm 202 of the robotic arm are acquired in real time. The unfolding angle of the left robotic arm's upper arm is 120° and the unfolding angle of its lower arm is 90°, while the unfolding angle of the right robotic arm's upper arm is 110° and the unfolding angle of its lower arm is 85°. Given that the lengths of the upper arms of both arms are 600mm and the lengths of the lower arms are 450mm, the rotation avoidance radii corresponding to different link lengths are calculated using trigonometric functions. The rotation avoidance radius of the left robotic arm is 812mm, and the rotation avoidance radius of the right robotic arm is 786mm. For the left robotic arm, which uses an omnidirectional rotation structure 204, a continuous rotation trajectory of the shoulder joint is planned, with the rotation angle range set to -180° to 180° and the trajectory movement speed set to 30° / s, ensuring that the link does not touch the surrounding tooling during the rotation process. In another embodiment, for the right robotic arm using the folding limiting structure 205, the segmented folding trajectory of the elbow joint is planned, which is divided into three folding intervals. The first segment folds from 85° to 60°, the second segment folds from 60° to 35°, and the third segment folds from 35° to 10°. The movement speed of each segment is 20° / s, and a 50ms dwell time is set between adjacent segments to ensure smooth movement. It should be noted that the distance between the links of the two robotic arms is monitored in real time by using visual transfer technology combined with a high-speed visual acquisition system (1280×720 pixels resolution, 50fps frame rate). The safety threshold for the link distance is set at 50mm. When the distance between the left and right robotic arms links drops to 42mm, the spatial coordinates of the current trajectory nodes are extracted (X=450mm, Y=320mm, Z=500mm for the left robotic arm elbow node and X=520mm, Y=330mm, Z=500mm for the right robotic arm elbow node). Based on the difference in the length of the two robotic arms links, the offset of the adjacent trajectory nodes is calculated to be 15mm. The coordinates of the trajectory nodes are corrected according to this offset, and the coordinates of the right robotic arm elbow node are adjusted to X=535mm, Y=330mm, Z=500mm, so that the link distance is restored to 57mm, maintaining the safety threshold requirement.

[0035] Optionally, the dynamic adjustment of trajectory node coordinates includes: Set a safety threshold benchmark for the linkage spacing, collect spacing data in real time, and compare it with the benchmark. When the spacing is less than the baseline, extract the spatial coordinates of the current node of the trajectory; Calculate the offset between adjacent nodes based on the difference in link length; The trajectory position is adjusted by correcting the trajectory coordinates based on the offset to meet the structural constraints of the linkage motion.

[0036] In this embodiment, the safety threshold benchmark for the distance between the two arm links of the robotic arm is set to 60mm. Visual transfer technology is used in conjunction with a high-precision visual sensing system (resolution 1280×1024 pixels, distance measurement accuracy ±0.5mm, sampling frequency 200Hz) to collect the actual distance data between the two arm links in real time, and the collected distance data is compared with the safety threshold benchmark in real time. When the actual distance between the two arm links is detected to drop to 52mm (less than the safety threshold of 60mm), the spatial coordinates of the current trajectory node are accurately extracted by the coordinate extraction module of the vision sensing system. The coordinates of the trajectory node of the left robotic arm elbow are (X=380mm, Y=250mm, Z=420mm), and the coordinates of the trajectory node of the right robotic arm elbow are (X=430mm, Y=260mm, Z=420mm). It should be noted that the left robotic arm has a top arm length of 650mm and a forearm length of 480mm, while the right robotic arm has a top arm length of 630mm and a forearm length of 460mm. Based on the difference in the lengths of the two robotic arms, the offset between adjacent trajectory nodes is calculated to be 12mm through spatial geometric operations. The offset direction is away from the positive X-axis direction of the left robotic arm. The coordinates of the right robotic arm elbow trajectory node are corrected according to this 12mm offset, and the original coordinates (X=430mm, Y=260mm, Z=420mm) are adjusted to (X=442mm, Y=260mm, Z=420mm). The trajectory position is updated synchronously, so that the distance between the two robotic arms links is restored to 64mm after adjustment, which meets the structural constraint requirements for avoiding collisions during the movement of the robotic arm links.

[0037] Of particular importance is that the dual-arm structure includes the elbow linkage intersection area, and step S3 also includes handling the risk of motion interference by following the procedure: Identify the interference risk level in the elbow linkage intersection area; For robotic arms with low backlash deceleration function, start the micro-programming sequence of the elbow joint and adjust in the reverse direction according to the preset step size; For robotic arms with high load deceleration function, start the shoulder joint lifting program and raise it to the preset height; The spacing is continuously monitored, and when the spacing exceeds the critical value, a recovery signal is sent to the parameter adjustment module.

[0038] In this embodiment, a visual transfer combined with a high-precision visual monitoring system (resolution 1920×1080 pixels, spacing measurement accuracy ±0.3mm, sampling frequency 300Hz) is used to collect the actual spacing data of the intersection area of ​​the elbow linkages of the robotic arm in real time. The preset interference risk level judgment criteria are: spacing ≤30mm is high risk, 30mm<spacing≤40mm is medium risk, and spacing>40mm is low risk. The measured spacing of the intersection area is 28mm, which is judged as a high interference risk level. It should be noted that the left robotic arm is clearly a structure with low backlash deceleration function (backlash accuracy ≤ 0.02°). Immediately start the micro-programming of its elbow joint, set the single adjustment step size to 0.5°, and make segmented adjustments in the counterclockwise direction opposite to the current movement direction. After each adjustment, pause for 30ms to stabilize the posture. In another embodiment, the right robotic arm is a structure with high load deceleration function (maximum load capacity ≤ 80kg), and the shoulder joint lifting program is started simultaneously. The preset lifting height is 15mm, and it is lifted upward at a constant speed of 5mm / s. The vision monitoring system is kept in continuous acquisition state, and the critical distance for interference release is set to 45mm. When the actual distance of the elbow linkage intersection area is detected to increase to 48mm (exceeding the critical value), the system generates a recovery signal containing the current posture parameters, which is sent in real time to the parameter adjustment module of the robotic arm master-slave role control logic through the industrial control bus to complete the closed-loop processing of interference risk.

[0039] Preferably, in step S4: when the robotic arm performs the assembly operation according to the current role control parameters, it simultaneously controls the dual-arm collaborative assembly parameters based on the real-time collected assembly accuracy and part status information until the current assembly operation is completed.

[0040] Optionally, step S4 includes: The main character robotic arm executes the assembly operation according to the current character control parameters, decomposes the control parameters into motion commands for each joint, and combines them with its own joint transmission characteristics to convert them into joint rotation angles and motion speeds, driving the end effector to perform assembly actions; Real-time acquisition of assembly accuracy information; extraction of positional and angular deviations between the end effector of the main robot arm and the assembly interface; conversion of positional and angular deviations into joint fine-tuning amounts according to the joint transmission ratio of the main robot arm; and adjustment of its fine positioning parameters based on the fine-tuning amounts. The robot arm synchronously collects part status information to identify part displacement and surface contact status. The adjustment range is determined based on the clamping force range of its clamping structure design, and the clamping force parameters are adjusted according to the displacement of the part and the surface contact state.

[0041] In one embodiment, the main robotic arm performs assembly operations according to the currently set operation response thresholds (joint start response time ≤ 15ms, position feedback response delay ≤ 8ms). The control parameters are decomposed into motion commands for the shoulder, elbow, and wrist joints. Combined with its own joint transmission characteristics (shoulder joint transmission ratio 1:10, elbow joint transmission ratio 1:8, wrist joint transmission ratio 1:5), these are converted into specific joint rotation angles (shoulder 35°, elbow 60°, wrist 25°) and movement speeds (shoulder 25° / s, elbow 30° / s, wrist 35° / s), driving the end effector to perform assembly actions. Specifically, a vision sensing system (1920×1080 pixels resolution, position measurement accuracy ±0.01mm, angle measurement accuracy ±0.02°, sampling frequency 200Hz) is used to collect assembly accuracy information in real time. The positional deviations (X-axis 0.03mm, Y-axis 0.02mm, Z-axis 0.04mm) and angular deviations (pitch angle 0.2°, yaw angle 0.15°, roll angle 0.1°) between the end of the main character's robotic arm and the assembly interface are extracted. These deviations are converted into joint fine-tuning amounts (shoulder 0.3°, elbow 0.2°, wrist 0.15°) according to the corresponding joint transmission ratio. Based on these fine-tuning amounts, the fine positioning parameters of the main character's robotic arm are adjusted from the initial ±0.03mm to ±0.01mm. In another embodiment, the robotic arm synchronously collects part status information through an integrated force-sensing clamping system (force measurement accuracy ±0.5N, displacement detection accuracy ±0.02mm, sampling frequency 150Hz), identifying the part's X-axis displacement of 0.05mm, Y-axis displacement of 0.03mm, and the contact pressure between the part surface and the clamping device of 90N. Given that the robotic arm's clamping structure is designed to clamp with a force range of 80N-120N, the clamping force adjustment range is determined to be 85N-115N. Based on the identified part displacement and contact pressure, the clamping force is adjusted from the initial 100N to 95N to ensure the part maintains a stable posture during assembly.

[0042] Of particular importance is that when the assembly is determined to be complete in step S4, the following operations are performed: Visual migration is used to detect the engagement depth between the connector and the mounting hole, and the engagement status between the buckle and the slot. Extract the standard values ​​for depth and edge alignment of the structural design; The detection data is compared with the standard value. When the depth meets the standard and the alignment meets the requirements, the force feedback signal is triggered for acquisition. The force feedback signal is compared with the structural assembly force threshold. If the two match, an assembly completion command is generated.

[0043] In one embodiment, visual transfer technology is used in conjunction with a high-precision visual inspection system (resolution 1920×1080 pixels, depth measurement accuracy ±0.02mm, edge recognition accuracy ±0.01mm, sampling frequency 250Hz) to detect the engagement depth between the connector and the mounting hole in the assembly process in real time, and at the same time identify the engagement edge fit status of the buckle and the slot. It should be noted that the actual engagement depth between the connector and the mounting hole is set to 15mm, and the edge fit gap between the buckle and the slot is ≤0.05mm. In another embodiment, the standard value for the depth of the assembly structure design is extracted as 15mm ± 0.1mm, and the standard value for edge alignment is ≤ 0.1mm. The measured bonding depth of 15mm and the edge fitting gap of ≤ 0.05mm are compared with the standard values ​​for depth and edge alignment, respectively. After confirming that both meet the requirements, the force feedback acquisition module built into the end of the robotic arm (force measurement accuracy ± 0.5N, response time ≤ 10ms) is triggered to acquire the current assembly force feedback signal. The measured force feedback signal value is 82N. The preset assembly force threshold of the structure is retrieved as 80N ± 5N. The measured force feedback signal of 82N is compared with the assembly force threshold to confirm that the two are within the matching range.

[0044] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0045] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A vision-transfer-based robotic arm dual-arm collaborative assembly control method, characterized in that, Includes the following steps: Step S1: Real-time acquisition of the movement postures of the two robotic arms and the relative pose information between the two arms; Identify the process switching node at the current assembly stage; Step S1 includes: Collect rotational angle data of each joint of both arms and simultaneously obtain the spatial position coordinates of both arms; Based on the rotation angle data and spatial position coordinates, the straight-line distance and relative angle between the end effectors of the two arms are calculated to form the motion state data of the two arms; Based on the motion state data of the two arms, the spatial position features of the parts to be assembled and the geometric state features of the assembly interface are extracted. The spatial location features and geometric state features are compared with the preset feature thresholds for each process stage. When the part position enters the workspace corresponding to the next process and the interface state meets the switching conditions, it is marked as a process switching node. Step S2: Construct the master-slave dynamic control logic for dual-arm collaboration. Based on the collected dual-arm posture and relative pose information, set the initial master-slave collaboration motion control parameters for dual-arm collaboration and determine the initial master-slave role division of dual arms. Step S3: When a process switching node is identified, adjust the movement priority and positioning accuracy parameters of the two arms in the master-slave role control mechanism according to the collaboration requirements of the target process to complete the master-slave role switching; In step S3, when a process switching node is detected, adjusting the movement priority of the two arms includes: Identify the operating structure of the master robot arm and the heavy-duty clamping structure of the slave robot arm; The weight of the parts and the required operational precision are obtained through visual transfer. Configure the operation response threshold for the master robot arm according to the operation accuracy requirements, and configure the clamping response threshold for the slave robot arm according to the weight of the part. Real-time monitoring of the joint angle between the main character's robotic arm and the slave character's robotic arm; when the joint angle of the main character's robotic arm touches the limit of the operating structure, switch to the limit adaptation response threshold. Specifically, adjusting the positioning accuracy parameters of the two arms in the master-slave role control mechanism in step S3 is as follows: Collect the normal angle data between the force sensor gripper and the assembly interface, and map the normal angle data into the joint angle correction amount of the gripper; The positioning accuracy parameters are adjusted based on the correction amount to make the end positioning deviation match the positioning tolerance of its own structure, and the parallelism of the mating surface is checked simultaneously, generating a parallelism correction signal and feeding it back to the controller. Step S4: When the robotic arm performs the assembly operation according to the current role control parameters, it also controls the dual-arm collaborative assembly parameters based on the real-time collected assembly accuracy and part status information until the current assembly operation is completed.

2. The vision-transfer-based robotic arm dual-arm collaborative assembly control method according to claim 1, characterized in that, When setting the initial motion control parameters in step S2, perform the following operations: Obtain the weight of the part to be assembled, and determine the adjustment range of the clamping force by combining it with the structural strength parameters of the robotic arm; After the clamping force adjustment range parameters are set, the relative position of the clamping point of the robotic arm and the center of gravity of the part is collected, and an alignment detection signal is generated. If the alignment of the alignment detection signal does not meet the requirements, the clamping force parameters are readjusted.

3. The vision-transfer-based robotic arm dual-arm collaborative assembly control method according to claim 1, characterized in that, Verifying the parallelism of the mating surfaces includes: Extract the coordinates of the edge feature points of the mating surface and calculate the angle between the normal vectors of the lines connecting the feature points; The included angle is compared with a preset parallelism standard value. When it exceeds the standard range, a correction signal is generated. After receiving the signal, the controller adjusts the joint angle of the gripper according to the preset step size until the included angle meets the requirements.

4. The vision-transfer-based robotic arm dual-arm collaborative assembly control method according to claim 1, characterized in that, The robotic arm includes a dual-arm structure, which comprises a large arm and a forearm. Step S3 also includes performing the following operations when planning an obstacle avoidance path based on the dual-arm structure: Obtain the deployment angles of the upper and lower arms of both arms, and calculate the turning avoidance radius corresponding to different link lengths; For a robotic arm with an omnidirectional rotating structure, plan the continuous rotation trajectory of the shoulder joint; For robotic arms with folding and limiting structures, plan the segmented folding trajectory of the elbow joint; The distance between links is monitored in real time by visual migration, and the coordinates of trajectory nodes are dynamically adjusted to maintain a safe threshold.

5. The vision-transfer-based robotic arm dual-arm collaborative assembly control method according to claim 4, characterized in that, Dynamic adjustment of trajectory node coordinates includes: Set a safety threshold benchmark for the linkage spacing, collect spacing data in real time, and compare it with the benchmark. When the spacing is less than the baseline, extract the spatial coordinates of the current node of the trajectory; Calculate the offset between adjacent nodes based on the difference in link length; The trajectory position is adjusted by correcting the trajectory coordinates based on the offset to meet the structural constraints of the linkage motion.

6. The vision-transfer-based robotic arm dual-arm collaborative assembly control method according to claim 1, characterized in that, Step S4 includes: The main character robotic arm executes the assembly operation according to the current character control parameters, decomposes the control parameters into motion commands for each joint, and combines them with its own joint transmission characteristics to convert them into joint rotation angles and motion speeds, driving the end effector to perform assembly actions; Real-time acquisition of assembly accuracy information; extraction of positional and angular deviations between the end effector of the main robot arm and the assembly interface; conversion of positional and angular deviations into joint fine-tuning amounts according to the joint transmission ratio of the main robot arm; and adjustment of its fine positioning parameters based on the fine-tuning amounts. The robot arm synchronously collects part status information to identify part displacement and surface contact status. The adjustment range is determined based on the clamping force range of its clamping structure design, and the clamping force parameters are adjusted according to the displacement of the part and the surface contact state.

7. A vision-transfer-based robotic arm dual-arm collaborative assembly control system, characterized in that, For performing the vision transfer-based dual-arm collaborative assembly control method for robotic arms as described in claim 1, the vision transfer-based dual-arm collaborative assembly control system includes: The posture node recognition module is used to collect the movement postures of the two arms of the robotic arm and the relative posture information between the two arms in real time; and to identify the process switching nodes of the current assembly process stage. The master-slave logic initialization module is used to construct the master-slave role dynamic control logic for dual-arm collaboration. Based on the collected dual-arm posture and relative pose information, it sets the initial master-slave collaboration motion control parameters and determines the initial master-slave role division of the dual arms. The role switching adjustment module is used to adjust the movement priority and positioning accuracy parameters of the two arms in the master-slave role control mechanism according to the collaboration requirements of the target process when a process switching node is detected, so as to complete the master-slave role switching. The collaborative assembly parameter control module is used to control the collaborative assembly parameters of the two arms based on real-time collected assembly accuracy and part status information when the robotic arm performs the assembly operation according to the current role control parameters, until the current assembly operation is completed.

Images