Robot teaching method, system, device and medium based on visual pose capture

Abstract

The application provides a robot teaching method and system based on visual pose capture, equipment and medium, and belongs to the technical field of robot teaching and trajectory control. The method comprises the following steps: based on an original teaching trajectory, a constant arc length sampling method is used to calculate a cumulative arc length of the trajectory and linear interpolation is performed between adjacent sampling points to obtain a plurality of key points; a mechanical arm moves to a target pose corresponding to the key points in sequence, and a current pose of a teaching tool is captured in real time at each key point; position errors and attitude errors are calculated based on the current pose and a recording pose of the original teaching trajectory, and a correction amount is generated to iteratively update the pose of the mechanical arm until a preset error threshold is met to obtain a corrected key point corresponding relationship; a local affine compensation model is established according to the corrected key point corresponding relationship, and a compensation trajectory is generated; and the end of the mechanical arm completes trajectory reproduction based on the compensation trajectory. The application improves the trajectory reproduction accuracy and reduces the time cost of closed-loop correction.

Description

Technical Field

[0001] This invention belongs to the field of robot teaching and trajectory control technology, and particularly relates to a robot teaching method, system, device and medium based on visual pose capture. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Most industrial robot teaching methods involve using a teach pendant for trajectory planning, where the robotic arm replicates the pre-set target trajectory path. Alternatively, the robot can be manually guided by dragging its end effector, recording the manually guided path for replication. These methods are inefficient for complex workpiece structures, require highly skilled operators, and present significant challenges in human-machine interaction.

[0004] Existing technologies propose visual capture-based teaching methods, such as acquiring the spatial pose information of a handheld teaching tool using an infrared camera and generating a replicating trajectory for the robotic arm. However, defects in the installation and calibration relationship between the teaching tool and the robotic arm's end effector still exist, leading to inconsistencies between the taught trajectory and the actual execution trajectory of the robotic arm. Furthermore, the trajectory replication process relies on the robotic arm's body coordinate system or kinematic model for error compensation, making it susceptible to errors and resulting in insufficient accuracy in reproducing complex trajectories. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention proposes a robot teaching method, system, device, and medium based on visual pose capture, in order to solve the problems of low trajectory efficiency, low accuracy, and difficulty in teaching complex teaching processes due to calibration relationships and error influences.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention discloses a robot teaching method based on visual pose capture, comprising: The original teaching trajectory of the teaching tool is obtained, and the probe calibration of the teaching tool is performed to determine the fixed spatial relationship between the tip of the teaching tool and the rigid body feature set coordinate system. TCP calibration and hand-eye calibration of the robotic arm are performed to establish the spatial transformation relationship between the robotic arm base coordinate system and the motion capture coordinate system. Based on the original teaching trajectory, the cumulative arc length of the trajectory is calculated using the equal arc length sampling method, and several key points are obtained by linear interpolation between adjacent sampling points. The robotic arm moves sequentially to the target pose corresponding to the key points, and captures the current pose of the teaching tool in real time at each key point; based on the current pose and the recorded pose of the original teaching trajectory, the position error and attitude error are calculated, and a correction amount is generated to iteratively update the robotic arm pose until the preset error threshold is met to obtain the corrected key point correspondence. Based on the corrected key point correspondence, a local affine compensation model is established and a compensation trajectory is generated; The robotic arm end effector replicates the trajectory based on the compensation trajectory.

[0007] Secondly, this invention discloses a robot teaching system based on visual pose capture, comprising: The calibration module is configured to: acquire the original teaching trajectory of the teaching tool, perform probe calibration on the teaching tool to determine the fixed spatial relationship between the tip of the teaching tool and the rigid body feature set coordinate system, and perform TCP calibration and hand-eye calibration on the robotic arm to establish the spatial transformation relationship between the robotic arm base coordinate system and the motion capture coordinate system. The key point extraction module is configured to: calculate the cumulative arc length of the trajectory based on the original teaching trajectory using an equal arc length sampling method and perform linear interpolation between adjacent sampling points to obtain several key points; The closed-loop correction module is configured as follows: the robotic arm moves sequentially to the target pose corresponding to the key point, and captures the current pose of the teaching tool in real time at each key point; the position error and attitude error are calculated based on the current pose and the recorded pose of the original teaching trajectory, and a correction amount is generated to iteratively update the robotic arm pose until the preset error threshold is met to obtain the corrected key point correspondence. The weighted affine compensation module is configured to: establish a local affine compensation model and generate a compensation trajectory based on the corrected key point correspondence; The robotic arm control module is configured to: enable the robotic arm end effector to reproduce the trajectory based on the compensation trajectory.

[0008] Thirdly, the present invention discloses an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when run by the processor, complete the steps of the above-mentioned robot teaching method based on visual pose capture.

[0009] Fourthly, the present invention discloses a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-described robot teaching method based on visual pose capture.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention achieves high-precision acquisition of the pose of the teaching tool through optical infrared vision capture, establishes a unified coordinate transformation relationship through probe calibration, TCP calibration and hand-eye calibration, reduces the time overhead caused by point-by-point servoing of long trajectories through key point closed-loop correction, and improves the accuracy and smoothness of complex trajectory reproduction through sliding window weighted affine compensation, thereby enabling the robotic arm to efficiently, accurately and continuously reproduce the teaching trajectory.

[0011] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0012] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0013] Figure 1 This is a flowchart of the robot teaching method based on visual pose capture as described in Embodiment 1 of the present invention.

[0014] Figure 2 This is a structural diagram of the robot teaching system based on visual pose capture as described in Embodiment 2 of the present invention.

[0015] Figure 3 This is a structural diagram of the teaching tool described in Embodiment 2 of the present invention.

[0016] Figure 4 This is a schematic diagram showing the coordinate relationship between the robotic arm, the teaching tool, and the optical infrared vision capture module as described in Embodiment 2 of the present invention.

[0017] Figure 5 This is a schematic diagram of the simulated actual industrial robotic arm trajectory reproduction scenario described in Embodiment 2 of the present invention. Detailed Implementation

[0018] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0020] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0021] Currently, robotic arm teaching is developing towards rapid programming teaching. This involves employing high-precision, natural, and efficient human-computer interaction methods to achieve precise control of multi-degree-of-freedom collaborative robotic arms. To improve the naturalness and flexibility of robot teaching, existing vision-capture-based teaching methods have improved the convenience of human-computer interaction to some extent, but still suffer from insufficient accuracy and efficiency. Specifically: First, the installation and calibration relationship between the teaching tool and the robotic arm end effector is not perfect, making it difficult to guarantee the consistency between the taught trajectory and the actual execution trajectory of the robotic arm; second, in the trajectory reproduction process, error compensation usually relies on the robotic arm's body coordinate system or kinematic model, which is easily affected by probe calibration errors, TCP calibration errors, hand-eye calibration errors, and cumulative errors of the robotic arm itself, resulting in insufficient accuracy in reproducing complex trajectories; third, for continuous long trajectories, existing methods still have shortcomings in balancing reproduction accuracy, execution efficiency, and smoothness.

[0022] In order to solve the problems and defects of the prior art, the present invention provides a robot teaching method, system, device and medium based on visual pose capture, so as to improve the accuracy, efficiency and smoothness of teaching and reproducing complex trajectories.

[0023] Example 1 In one or more embodiments, to address the problems of low teaching efficiency, insufficient reproduction accuracy, and poor smoothness of continuous long trajectories in existing robot teaching technologies for complex trajectory scenarios, a robot teaching method based on visual pose capture is disclosed, such as... Figure 1 As shown, it includes the following steps: Step S101: Obtain the original teaching trajectory of the teaching tool, perform probe calibration on the teaching tool to determine the fixed spatial relationship between the tip of the teaching tool and the rigid body feature set coordinate system, and perform TCP calibration and hand-eye calibration on the robotic arm to establish the spatial transformation relationship between the robotic arm base coordinate system and the motion capture coordinate system.

[0024] As one implementation method, this invention first establishes a high-precision, low-latency optical motion capture system as the global observation benchmark for the entire teaching task. In terms of physical spatial layout, the robotic arm to be taught is used as the work center, and a rigid truss is built around it, with 10 industrial-grade infrared optical cameras arranged in a square arrangement, ensuring that the optical axis center of each camera converges and aligns with the core area of ​​the robotic arm's workspace. This highly redundant multi-view stereo vision layout not only expands the effective capture range but also effectively solves the problem of line-of-sight occlusion that may occur during the robotic arm's movement, ensuring that at least three cameras can capture the target at any given time. During the system initialization phase, T-shaped and L-shaped high-precision calibration tools are used to scan within the field of view. By calculating the camera's intrinsic parameters (distortion model) and extrinsic parameters (rotation and translation matrices), a unified spatial coordinate system calibration is performed on the camera system composed of these 10 infrared cameras, thereby establishing a unique, highly linear motion capture system coordinate system, providing an absolute spatial reference benchmark for all subsequent pose measurements.

[0025] The operator holds a teaching tool and teaches along a predetermined trajectory. The optical infrared vision capture module collects the temporal pose data of the tip of the teaching tool in the motion capture coordinate system in real time and transmits it to the computer workstation. The computer workstation preprocesses the collected teaching trajectory, including median filtering, Gaussian smoothing, and curve fitting, to remove noise and generate the original teaching trajectory.

[0026] The teaching pendant is calibrated with probes, and the robotic arm is calibrated using TCP and hand-eye calibration to establish the spatial transformation relationship between the robotic arm base coordinate system and the motion capture coordinate system. Specifically, this includes: The teaching pendant is equipped with reflective markers to form a rigid body feature set that can be recognized by motion capture. Specifically, four reflective markers are set on the teaching pendant, and these four markers constitute a minimum rigid body feature set. A probe calibration algorithm is used to determine the fixed spatial relationship between the tip of the teaching pendant and the rigid body coordinate system, and the Kabsch rigid body registration algorithm is used to correct the initial attitude deviation caused by the error in the placement of the reflective markers, thereby obtaining the accurate three-dimensional pose of the tip of the teaching pendant in the motion capture coordinate system.

[0027] In this embodiment, a probe calibration algorithm is used to determine the fixed spatial offset relationship of the teaching pendant tip relative to the rigid body feature set coordinate system. Simultaneously, to address initial attitude deviations that may be caused by installation errors of reflective markers, the Kabsch rigid body registration algorithm is used to correct the teaching pendant's attitude, thereby obtaining the precise three-dimensional pose of the teaching pendant tip in the motion capture coordinate system. The corrected teaching pendant pose data is transmitted to the computer workstation via the SDK for subsequent trajectory processing and robot control.

[0028] Specifically, let the first The rigid body transformation of the teaching tool's rigid body relative to the motion capture coordinate system is as follows: ,in, Let be a rotation matrix. Let the translation vector be: Let the fixed coordinates of the tip of the teaching tool in the teaching tool coordinate system be... The position of the fixed pivot point in space in the motion capture coordinate system is: Then we have: By combining multiple frames of data to form an overdetermined system of equations, and solving it using the least squares method, the position offset of the teaching tool tip in the tool coordinate system can be obtained. This completes the probe calibration.

[0029] For teaching pendant posture correction, let the set of coordinates of reflective marker points in the ideal model of the teaching pendant be . The actual set of coordinates of the reflective marker points obtained is as follows: The optimal rotation matrix is ​​solved using the Kabsch rigid body registration algorithm. With translation vector , so that the objective function The minimum value is obtained, thereby eliminating the initial attitude deviation caused by the error in the placement of reflective markers.

[0030] The hand-eye calibration adopts an eye-on-hand external calibration method. The teaching tool, after being calibrated by the probe and corrected for the posture, is installed on the end flange of the robotic arm. By controlling the robotic arm to reach multiple spatial positions with different postures, the pose of the robotic arm TCP in the robotic arm base coordinate system and the pose of the teaching tool in the motion capture coordinate system are recorded simultaneously. Based on multiple sets of pose data, the rigid transformation relationship between the robotic arm base coordinate system and the motion capture coordinate system is solved.

[0031] Specifically, let the pose of the end effector flange of the robotic arm relative to the coordinate system of the robotic arm base be... The pose of the teaching tool relative to the motion capture coordinate system is The teaching tool is fixed relative to the end flange of the robotic arm. The transformation to be determined by the motion capture coordinate system relative to the robotic arm base coordinate system is: Then we have: By collecting data from multiple sets of different postures and solving them jointly, the transformation matrix between the motion capture coordinate system and the robotic arm base coordinate system can be obtained. This completes the spatial alignment between the motion capture coordinate system and the robot arm base coordinate system. After this step, the teaching trajectory acquired by the optical infrared vision capture module can be accurately converted into an executable trajectory in the robot arm base coordinate system.

[0032] Step S102: Based on the original teaching trajectory, calculate the cumulative arc length of the trajectory using the equal arc length sampling method and perform linear interpolation between adjacent sampling points to obtain several key points. Specifically: Let the original trajectory point set be The total arc length of the trajectory is approximated by the sum of the Euclidean distances between adjacent sampling points:

[0033] in, The total arc length of the trajectory; This represents the number of original trajectory points; Let be the coordinates of the i-th original trajectory sampling point.

[0034] By calculating the cumulative arc length of the trajectory and performing linear interpolation between adjacent sampling points, key points that are approximately uniformly distributed along the entire trajectory can be obtained.

[0035] Specifically, the cumulative arc length of the trajectory is obtained by summing the distances between adjacent sampling points. , Then the first The cumulative arc length of the original trajectory sampling points is in, The cumulative arc length is used to determine the target arc length position of the key point on the entire trajectory.

[0036] Subsequently, based on the target arc length position obtained from equal arc length sampling, m key points are extracted. The target arc length interval between adjacent key points is then:

[0037] Then the first The cumulative arc length of the target corresponding to each key point is: ,in For the target arc length interval, Number of key points; Subsequently, in the cumulative arc length sequence, we search for the condition that satisfies... Adjacent original trajectory sampling points and The coordinates of the kth keypoint are obtained by linear interpolation as follows. :

[0038] Therefore, adjacent sampling points corresponding to the target arc length position are sequentially found in the cumulative arc length sequence, and the spatial coordinates of the key points are determined by linear interpolation, thereby obtaining key points distributed along the entire trajectory with equal arc lengths. This process avoids the problem of uneven spatial distribution caused by directly taking points according to time or original sampling sequence, thus ensuring that the extracted key points have more balanced spatial coverage on the entire trajectory, which is more conducive to subsequent trajectory shape representation and local compensation modeling.

[0039] Step S103: The robotic arm moves sequentially to the target pose corresponding to the key point, and captures the current pose of the teaching tool in real time at each key point; the position error and attitude error are calculated based on the current pose and the recorded pose of the original teaching trajectory, and a correction amount is generated to iteratively update the robotic arm pose until the corrected key point correspondence is obtained by meeting the preset error threshold.

[0040] The deviation between the current pose and the recorded pose is obtained in real time, and visual closed-loop correction is performed on the robotic arm to obtain the corrected key point correspondence. The correction amount is generated based on the position error and posture error, and the robotic arm pose is iteratively updated until a preset error threshold is met, thus obtaining the corrected key point correspondence.

[0041] Specifically, the correction amount is generated according to the following steps: 1: During the closed-loop calibration process at key points, the target position is set as... The current location is The position error is obtained by subtracting the current position from the target position:

[0042] Extract the quaternions for the target and current pose. To ensure the shortest path rotation requires the dot product to be less than 0, the quaternions are inverted to guarantee smooth motion. Then, the quaternions are converted into corresponding rotation matrices. Given the target pose, the rotation matrix is... Current attitude rotation matrix Then, the corresponding error rotation matrix is ​​calculated as follows:

[0043] The error rotation matrix is ​​converted into a rotation vector, denoted as the 3D pose error vector in the motion capture coordinate system. Preferably, for the error rotation matrix... ,set up The corresponding rotation axis angle is The axis of rotation is a unit vector. Then the rotation vector is defined as: The rotation angle can be obtained from the trace of the rotation matrix. ,when The axis of rotation is Therefore, the rotation vector is represented as follows: .

[0044] 2: Transform the error into the robotic arm base coordinate system using the calculated hand-eye calibration matrix:

[0045]

[0046] in, This refers to the position error of the robot arm base coordinate system. This represents the attitude error of the robot arm's base coordinate system.

[0047] 3: Generate the correction vector: Multiplying the error in the robot arm's base coordinate system by the amplification gain and the time constant step size of the control cycle, we convert it into a correction control increment for the current state, expressed as:

[0048]

[0049] in, This is the position correction amount; This is the attitude correction amount; For position gain, this example sets ; To control the time constant step size of the control cycle, the sampling frequency of the motion capture system is 125Hz, therefore, given... ; For attitude gain, this example sets .

[0050] 4: Add correction values ​​based on the current TCP state of the robotic arm.

[0051] To avoid control oscillations caused by system delays, the system chooses to superimpose corrections based on the real-time TCP feedback position of the robotic arm as a reference, rather than directly superimposing them on historical trajectory points.

[0052] Obtain the robot's current true TCP pose and perform position correction. Attitude correction: Convert the robot's current TCP rotation vector into a rotation matrix. The calculated attitude correction is also converted into a rotation matrix. Then, left-multiply to update the current pose. The latest rotation matrix Convert it into a rotation vector.

[0053] Since the error is calculated in the motion capture coordinate system, the impact of the cumulative error in the robot arm's body coordinate system on the reproduction accuracy can be reduced.

[0054] Step S104: Based on the corrected key point correspondence, establish a local affine compensation model and generate a compensation trajectory; the robotic arm end effector completes trajectory reproduction based on the compensation trajectory.

[0055] Based on the corrected correspondence of key points, a local affine compensation model is established, specifically as follows: Let the source point be The target point after compensation is The compensation result is then represented as follows:

[0056] in, for The affine transformation matrix, It is a translation vector.

[0057] A system of linear equations is established to determine the correspondence between key points within each local window, and the local affine parameters are solved using the least squares method. and .

[0058] Compared to rigid body compensation models that only consider rotation and translation, affine compensation models can also describe local scale changes and shear errors, making them more suitable for compensating for non-uniform spatial distortions in complex trajectory reproduction processes.

[0059] Furthermore, the original teaching trajectory is segmented using a sliding window approach, and the local affine transformation parameters are solved for each window. For the trajectory point to be compensated, let it correspond to the first... The arc distance between the centers of the windows is Then the weight of each window can be defined as:

[0060] in, For the weight distribution parameters, For the current trajectory point to be compensated to the th The arc length distance between the centers of each active window is calculated. For each trajectory point to be compensated, the compensation result under each local window affine model is calculated, and the compensation results are weighted and fused according to the weights to obtain the final compensation value of the trajectory point. This generates a continuous and smooth compensation trajectory, thereby reducing the trajectory jump phenomenon caused by traditional fixed segment compensation.

[0061] Step S105: Generate control commands based on the compensation trajectory to drive the robotic arm to complete trajectory reproduction.

[0062] This embodiment first obtains the compensated target trajectory and uniformly represents it in the robot's base coordinate system. Then, during execution, it calculates the position error and attitude error by combining the current feedback pose of the robotic arm with the target pose. For the key point closed-loop correction part, the error is first calculated in the motion capture coordinate system, and then the error direction is transformed to the robot's base coordinate system based on the hand-eye calibration results. After that, proportional control is used to calculate the single-cycle position correction and attitude correction, and these are superimposed on the current TCP pose to generate the next desired pose. Finally, it is sent to the robotic arm for execution through the URScript's Servoj interface.

[0063] This invention utilizes an optical infrared vision capture module to acquire high-precision spatial pose information of the teaching tool. Through probe calibration, TCP calibration, and hand-eye calibration, it establishes the coordinate transformation relationship between the teaching tool, the motion capture system, and the robotic arm. Combined with key point closed-loop correction and weighted affine compensation, it reduces the time overhead of closed-loop correction while ensuring trajectory reproduction accuracy and reduces pose abrupt changes caused by traditional segmented compensation. This enables the robotic arm to reproduce the teaching trajectory with high precision, continuity, and smoothness, making it suitable for scenarios such as complex surface processing, flexible assembly, and rapid robot teaching.

[0064] Example 2 In one or more embodiments, a robot teaching system based on visual pose capture is disclosed, such as... Figure 2 As shown, it includes an optical infrared vision capture module, a teaching tool, a computer workstation, and a robotic arm.

[0065] The computer workstation includes a computer host, a display terminal, and host computer control software running on the computer host. The host computer control software is used to receive and process the pose data collected by the optical infrared vision capture module and generate control commands for the robotic arm.

[0066] In this embodiment, the host computer control software includes a calibration module, a key point extraction module, a closed-loop correction module, a weighted affine compensation module, and a robotic arm control module. The calibration module is configured to: acquire the original teaching trajectory of the teaching tool, perform probe calibration on the teaching tool to determine the fixed spatial relationship between the tip of the teaching tool and the rigid body feature set coordinate system, and perform TCP calibration and hand-eye calibration on the robotic arm to establish the spatial transformation relationship between the robotic arm base coordinate system and the motion capture coordinate system. The key point extraction module is configured to: calculate the cumulative arc length of the trajectory based on the original teaching trajectory using an equal arc length sampling method and perform linear interpolation between adjacent sampling points to obtain several key points; The closed-loop correction module is configured as follows: the robotic arm moves sequentially to the target pose corresponding to the key point, and captures the current pose of the teaching tool in real time at each key point; the position error and attitude error are calculated based on the current pose and the recorded pose of the original teaching trajectory, and a correction amount is generated to iteratively update the robotic arm pose until the preset error threshold is met to obtain the corrected key point correspondence. The weighted affine compensation module is configured to: establish a local affine compensation model and generate a compensation trajectory based on the corrected key point correspondence; The robotic arm control module is configured to: enable the robotic arm end effector to reproduce the trajectory based on the compensation trajectory.

[0067] Preferably, the optical infrared vision capture module includes 10 optical infrared cameras, each fixed in a room 4 meters above the ground by a three-way pan-tilt bracket. During setup, the cameras are arranged in a square pattern above the site (the square has a side length of 7 meters), with the focus center of each camera aligned with the robotic arm. After setup, the spatial coordinate system of the motion capture vision system composed of these 10 cameras needs to be calibrated using T-shaped and L-shaped rods to ensure that the computer workstation can effectively and accurately acquire the spatial pose of reflective markers or rigid bodies.

[0068] The optical infrared visual capture module is connected to the computer workstation via a network cable. Each optical infrared camera is connected to the input port of a data exchange device via a network cable, and then connected to the output port of a conversion device via a main network cable, which in turn connects to the computer workstation. In this embodiment, the optical infrared visual capture module acquires data at a frame rate of 125 FPS.

[0069] like Figure 3 As shown, the teaching tool is a handheld teaching tool with four reflective markers. These four reflective markers form a minimum rigid body feature set, which is used for identification and tracking by the optical infrared vision capture module.

[0070] Preferably, the robotic arm communicates with the computer workstation via TCP / IP, and a teaching pendant is mounted on the end flange of the robotic arm. After completing the probe calibration and attitude correction of the teaching pendant, as well as the TCP calibration and hand-eye calibration of the robotic arm, the computer workstation generates trajectory control commands for the robotic arm based on the processed teaching trajectory, and drives the end effector of the robotic arm to perform motion along the target trajectory.

[0071] The coordinate relationship between the robotic arm, the teaching pendant, and the optical-infrared vision capture module, such as... Figure 4As shown. To establish the spatial mapping relationship between the motion capture coordinate system and the robotic arm base coordinate system, this embodiment adopts a hand-eye calibration method with the eye outside the hand. Specifically, the teaching pendant, after probe calibration and Kabsch attitude correction, is installed on the end flange of the robotic arm, controlling the robotic arm to move to multiple different postures, and simultaneously recording the pose of the robotic arm TCP in the robotic arm base coordinate system and the pose of the teaching pendant in the motion capture coordinate system.

[0072] Furthermore, such as Figure 5 As shown, the present invention will be described in detail by applying the proposed solution to a practical process scenario. In the application, the system includes an optical infrared vision capture module 1, a robotic arm 2, a workpiece to be coated with glue 3, a handheld teaching tool 4, and a glue application gun 5.

[0073] During the teaching phase, the operator holds the handheld teaching tool 4 and teaches the motion along the target adhesive application trajectory of the workpiece 3 to be processed. The optical infrared vision capture module 1 collects the spatial pose information of the handheld teaching tool 4 in real time through multiple infrared optical cameras, and transmits the pose data to the computer workstation via the SDK. The computer workstation preprocesses, transforms, and models the collected teaching trajectory data to obtain the target trajectory that the robotic arm can execute.

[0074] During the reproduction phase, a glue gun 5 is mounted at the end of the robotic arm 2. The computer workstation generates control commands based on the robot's reproduced trajectory after calibration, key point closed-loop correction, and sliding window weighted affine compensation. These commands drive the robotic arm 2 to move the glue gun 5 at its end to perform glue application along the surface of the workpiece 3. The system continuously acquires the real-time pose of the teaching tool or end effector in the motion capture coordinate system through the optical infrared vision capture module 1 and compares it with the corresponding pose of the recorded trajectory. This enables key point closed-loop correction and trajectory compensation, thereby reducing the impact of probe calibration, TCP calibration, hand-eye calibration, and robotic arm body errors on the accuracy of trajectory reproduction. Through this method, the present invention can accurately map complex spatial trajectories obtained through manual teaching to the execution trajectory of the glue gun 5 at the end of the robotic arm. While improving the teaching efficiency of complex trajectories, it also considers the accuracy, continuity, and smoothness of trajectory reproduction, making it suitable for operations such as glue application, spraying, grinding, polishing, and curved surface machining.

[0075] In this embodiment, key point extraction adopts an equal arc length sampling method to make key points as evenly distributed as possible along the entire teaching trajectory; visual loop closure correction is based on the error between the current real-time pose and the target pose in the motion capture coordinate system; sliding window weighted affine compensation achieves a continuous and smooth transition of the trajectory through the weighted fusion of multiple local window compensation results.

[0076] Through the above system structure and method, this invention utilizes optical infrared visual capture technology to achieve high-precision acquisition of the pose of the teaching tool. It establishes a unified coordinate relationship through probe calibration, TCP calibration, and hand-eye calibration. It generates a continuous and smooth robotic arm reproduction trajectory through key point closed-loop correction and sliding window weighted affine compensation, thereby effectively improving the accuracy, efficiency, and smoothness of complex trajectory teaching and reproduction.

[0077] Example 3 This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they complete the steps of the above-described robot teaching method based on visual pose capture.

[0078] Example 4 This embodiment provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-described robot teaching method based on visual pose capture.

[0079] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A robot teaching method based on visual pose capture, characterized in that, include: The original teaching trajectory of the teaching tool is obtained, and the probe calibration of the teaching tool is performed to determine the fixed spatial relationship between the tip of the teaching tool and the rigid body feature set coordinate system. TCP calibration and hand-eye calibration of the robotic arm are performed to establish the spatial transformation relationship between the robotic arm base coordinate system and the motion capture coordinate system. Based on the original teaching trajectory, the cumulative arc length of the trajectory is calculated using the equal arc length sampling method, and several key points are obtained by linear interpolation between adjacent sampling points. The robotic arm moves sequentially to the target pose corresponding to the key points, and captures the current pose of the teaching tool in real time at each key point; Based on the current pose and the recorded pose of the original teaching trajectory, the position error and posture error are calculated, and a correction amount is generated to iteratively update the pose of the robotic arm until the preset error threshold is met to obtain the corrected key point correspondence. Based on the corrected key point correspondence, a local affine compensation model is established and a compensation trajectory is generated; The robotic arm end effector replicates the trajectory based on the compensation trajectory.

2. The robot teaching method based on visual pose capture as described in claim 1, characterized in that, The equal arc length sampling method is specifically as follows: Let the original trajectory point set be The total arc length of the trajectory is: in, The total arc length of the trajectory; This represents the number of original trajectory points; Let i be the coordinates of the i-th original trajectory sampling point; If m key points are extracted, the target arc length interval between adjacent key points is: Where m is the number of key points. The target arc length interval.

3. The robot teaching method based on visual pose capture as described in claim 1, characterized in that, The calculation of the cumulative arc length of the trajectory and the linear interpolation between adjacent sampling points are specifically as follows: No. The cumulative arc length of the original trajectory sampling points is in, , Let i be the coordinates of the i-th original trajectory sampling point; No. The cumulative arc length of the target corresponding to each key point is: ,in For the target arc length interval, Number of key points; In the cumulative arc length sequence, find the one that satisfies Adjacent original trajectory sampling points and The coordinates of the kth keypoint are obtained by linear interpolation as follows: in, Let be the coordinates of the kth key point.

4. The robot teaching method based on visual pose capture as described in claim 1, characterized in that, The position error and attitude error are calculated based on the recorded pose of the current pose and the original teaching trajectory, including: in, For the target location, Current position For positional error, Let be the target attitude rotation matrix. The rotation matrix is ​​the current attitude. Let be the attitude error matrix.

5. The robot teaching method based on visual pose capture as described in claim 1, characterized in that, The correction amount is generated based on the position error and attitude error. Specifically, the error in the robot arm base coordinate system is multiplied by the amplification gain and the time constant step size of the control cycle, and converted into a correction control increment for the current state. The expression is as follows: in, This is the position correction amount; This is the attitude correction amount; For position gain; To control the time constant step size of the cycle; For attitude gain; This refers to the position error of the robot arm base coordinate system; This represents the attitude error of the robot arm's base coordinate system.

6. The robot teaching method based on visual pose capture as described in claim 1, characterized in that, The step of establishing a local affine compensation model based on the corrected key point correspondence includes: The local affine model is represented as: in, Let be the affine transformation matrix. It is a translation vector. To compensate for the outcome, As the source point; The original teaching trajectory is segmented using a sliding window method. The local affine transformation parameters of each window are solved by the least squares method to obtain the compensation results of the trajectory points to be compensated under the affine model of each local window. For each trajectory point to be compensated, the weight of each window is calculated. Based on the weight, the compensation results are weighted and fused to obtain the final compensation value of the trajectory point, thus generating a continuous and smooth compensated trajectory.

7. The robot teaching method based on visual pose capture as described in claim 6, characterized in that, For the trajectory points to be compensated, calculate the weight of each window, assuming it corresponds to the first window. The arc distance between the centers of the windows is Then the weights of each window are: in, For the weight distribution parameters, For the current trajectory point to be compensated to the th The arc length distance between the centers of the active windows.

8. A robot teaching system based on visual pose capture, characterized in that, include: The calibration module is configured to: acquire the original teaching trajectory of the teaching tool, perform probe calibration on the teaching tool to determine the fixed spatial relationship between the tip of the teaching tool and the rigid body feature set coordinate system, and perform TCP calibration and hand-eye calibration on the robotic arm to establish the spatial transformation relationship between the robotic arm base coordinate system and the motion capture coordinate system. The key point extraction module is configured to: calculate the cumulative arc length of the trajectory based on the original teaching trajectory using an equal arc length sampling method and perform linear interpolation between adjacent sampling points to obtain several key points; The closed-loop correction module is configured such that the robotic arm moves sequentially to the target pose corresponding to the key point, and captures the current pose of the teaching tool in real time at each key point; Based on the current pose and the recorded pose of the original teaching trajectory, the position error and posture error are calculated, and a correction amount is generated to iteratively update the pose of the robotic arm until the preset error threshold is met to obtain the corrected key point correspondence. The weighted affine compensation module is configured to: establish a local affine compensation model and generate a compensation trajectory based on the corrected key point correspondence; The robotic arm control module is configured to: enable the robotic arm end effector to reproduce the trajectory based on the compensation trajectory.

9. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the robot teaching method based on visual pose capture as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the robot teaching method based on visual pose capture as described in any one of claims 1-7.

Images