Vision-based long flexible robot arm end positioning method and system

Abstract

The application discloses a long flexible mechanical arm end positioning method and system based on vision, acquires a cloth operation point image pixel coordinate, and maps to a mechanical arm coordinate system. The trajectory of the mechanical arm end is planned, and each joint angle change is solved based on inverse kinematics. Considering the error caused by the gravity and flexible deformation of the long arm frame, a camera is installed at the end to collect the target area feature point image in real time, the expected image features are calculated combined with the camera internal and external parameters, and the image feature error is obtained by comparing the actual features. The compensation speed is solved by using the visual servo control method, and the joint speed is further obtained through the mechanical arm Jacobian matrix to realize the end precise positioning. When the error converges to the preset range, the mechanical arm automatically stops. The method can improve the positioning accuracy of the 13-meter cloth machine end cloth port to within 5 cm, effectively reduces the artificial dependence and operation strength, improves the automation and precision of the cloth operation, and has a wide application prospect.

Description

Technical Field

[0001] This invention relates to the field of engineering machinery and automation control, and in particular to a vision-based method and system for positioning the end effector of a long, flexible robotic arm. Background Technology

[0002] Long, flexible robotic arms are a type of multi-joint robotic arm system characterized by their relatively long structural dimensions, light weight, and significant flexibility. Common examples include robotic arms for aircraft cleaning, concrete pouring, and space operations. These robotic arms are widely used in construction, municipal engineering, and emergency rescue operations. Due to their large arm length and significant flexibility, controlling them to perform delicate tasks requires a high level of skill from the operators.

[0003] Furthermore, in most construction operation scenarios, the target object is usually far from the robotic arm base and the operator, resulting in insufficient precision in end-effector position control and easy deviation between the robotic arm end-effector and the target work point. To ensure that the end-effector can reach the target position, existing operation methods often require multiple personnel to coordinate, observe, direct, or guide the movement of the robotic arm, which is not only time-consuming but also carries high operational risks.

[0004] While vision technology can provide auxiliary information for end-effector positioning, traditional fixed or body-mounted vision sensors often face challenges such as limited field of view and target occlusion by the robot body in large fields of view and long distances. This makes it difficult to continuously and stably acquire relative positional information between the end-effector and the target, thus limiting the application of vision-based positioning technology in long, flexible robotic arm scenarios. Existing vision-based positioning methods generally rely on a fixed camera viewpoint or preset desired pixel coordinates, but these methods have several shortcomings in actual construction: the target easily deviates from the camera's field of view, making servo control impossible; secondly, the desired pixel coordinates are difficult to obtain accurately in advance, thus affecting positioning accuracy.

[0005] To address this issue, this invention introduces a drone and a gimbal camera installed at the end of a robotic arm. By combining visual recognition and image-based visual servoing technology, it achieves precise positioning of the end of a long, flexible robotic arm, significantly reducing the number of personnel and operational difficulty, while improving positioning accuracy and work efficiency. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by proposing a vision-based method and system for positioning the end effector of a long, flexible robotic arm. Before operation, a camera mounted on a drone is used to collect an overhead view of the construction area. Image recognition technology is used to automatically detect the target work area and target work point, and the three-dimensional coordinates of the target point are calculated through ranging technology and spatial mapping.

[0007] The objective of this invention is achieved through the following technical solution: a vision-based method for end-effector positioning of a long, flexible robotic arm, comprising the following steps:

[0008] Step 1: Obtain an image containing the target to be worked using a drone, and convert the pixel coordinates of the corner points of the target in the image into position coordinates in the robotic arm coordinate system based on the positional relationship between the drone and the robotic arm;

[0009] Step 2: Based on the obtained position of the corner point of the target to be worked, obtain the motion trajectory of the robotic arm and solve for the change value of the joint angle of the robotic arm;

[0010] Step 3: Install a gimbal camera at the end of the robotic arm to detect and identify the real-time pixel coordinates of the corner points of the target to be worked on;

[0011] Step 4: Based on the fixed assembly relationship between the gimbal camera and the end effector of the robotic arm, the desired position of the end effector of the robotic arm is transformed into the camera coordinate system, and the imaging projection is completed by combining the intrinsic and extrinsic parameters of the camera to obtain the desired pixel coordinates of the desired position in the image plane.

[0012] Step 5: Based on the pixel coordinates of the desired image features and the real-time pixel coordinates, calculate the image feature error;

[0013] Step 6: Obtain the depth value based on the binocular parallax principle and the camera intrinsic parameter matrix, and then calculate the image Jacobian matrix by combining the depth value;

[0014] Step 7: Image-based visual servoing method: Based on the image feature error and the image Jacobian matrix, the desired camera speed required to achieve the desired image features is obtained; Based on the positional relationship between the gimbal camera and the robotic arm end effector, the robotic arm end effector speed required to reach the desired position is calculated, and the desired joint angle speed is obtained by solving the robotic arm Jacobian matrix.

[0015] Step 8: Control the end effector movement of the robotic arm based on the expected velocity of the joint angle. When the real-time image feature error is reduced to an acceptable range, the robotic arm stops moving.

[0016] Furthermore, in step 1, before the fabric laying operation begins, the drone and its onboard camera are used to capture images of the target to be worked on in the construction scene. After obtaining the pixel information of the target to be worked on, the coordinates of the target point relative to the drone coordinate system are obtained based on the camera's focal length and the drone's flight altitude.

[0017] Furthermore, in step 1, the pixel coordinates of the corner points in the target image to be processed are obtained according to the contour detection method.

[0018] Furthermore, in step 2, the motion trajectory of the robotic arm is obtained using a fifth-order polynomial interpolation method.

[0019] Furthermore, in step 4, the specific process of obtaining the desired pixel coordinates by combining the camera's intrinsic and extrinsic parameter matrices with the relative relationship between the robotic arm's end effector is as follows:

[0020] (1) Calculate the transformation matrix between the end of the robotic arm and the gimbal camera based on the desired position of the end of the robotic arm, the angle of the last section of the robotic arm and the installation position of the gimbal camera;

[0021] (2) Based on the location of the target point obtained in step 1, calculate the coordinates of the corner point of the target point in the gimbal coordinate system;

[0022] (3) Based on the intrinsic parameter matrix of the binocular camera, the expected image features of the target corner point are obtained by combining the image Jacobian matrix.

[0023] Furthermore, in step 4, the field of view is adjusted in real time by the gimbal to ensure that the target remains within the effective imaging range, thereby continuously providing stable and reliable desired image features.

[0024] Furthermore, in step 5, the image feature error is the difference between the real-time pixel coordinates and the expected pixel coordinates.

[0025] Furthermore, based on the positional relationship between the gimbal camera and the robotic arm's end effector, the required end effector speed to reach the desired position is calculated.

[0026] Furthermore, in step 8, when the image feature error is less than a threshold, the robotic arm stops moving, where the threshold is a preset tolerance.

[0027] On the other hand, the present invention also provides a vision-based end-effector positioning system for a long, flexible robotic arm, the system comprising:

[0028] The target coordinate acquisition module is used to obtain an image containing the target to be worked through the UAV, and convert the pixel coordinates of the corner points of the target to be worked in the image into position coordinates in the robotic arm coordinate system based on the positional relationship between the UAV and the robotic arm.

[0029] The motion trajectory acquisition module is used to obtain the motion trajectory of the robotic arm based on the position of the corner point of the target to be worked, and to solve for the change value of the joint angle of the robotic arm.

[0030] The pixel coordinate recognition module is used to detect and identify the real-time pixel coordinates of the corner points of the target to be operated based on the gimbal camera installed at the end of the robotic arm.

[0031] The desired pixel coordinate solution module, based on the fixed assembly relationship between the gimbal camera and the end effector of the robotic arm, transforms the desired position of the end effector of the robotic arm into the camera coordinate system, and combines the camera's intrinsic and extrinsic parameters to complete the imaging projection, thereby obtaining the desired pixel coordinates of the desired position in the image plane.

[0032] The feature error solving module is used to solve for the image feature error based on the pixel coordinates of the desired image features and the real-time pixel coordinates.

[0033] The image Jacobian matrix module is used to obtain depth values ​​based on the binocular parallax principle and the camera intrinsic parameter matrix, and then to obtain the image Jacobian matrix by combining the depth values.

[0034] The joint angle expected velocity calculation module is used for image-based visual servoing methods. It obtains the camera expected velocity required to achieve the desired image features based on the image feature error and the image Jacobian matrix. Based on the positional relationship between the gimbal camera and the robotic arm end effector, it calculates the robotic arm end effector velocity required to reach the desired position and obtains the joint angle expected velocity based on the robotic arm Jacobian matrix.

[0035] The robotic arm end effector control module controls the movement of the robotic arm end effector based on the expected velocity of the joint angle. When the real-time image feature error is reduced to an acceptable range, the robotic arm stops moving.

[0036] Compared with existing technologies, this invention deeply integrates UAV remote sensing, image recognition, motion planning, visual servoing and flexible robotic arm control, significantly improving the automated positioning accuracy of long flexible robotic arms when performing construction tasks, reducing operational intensity and personnel risks, and has broad engineering application prospects.

[0037] The beneficial effects of this invention are as follows:

[0038] 1. This invention uses drones, gimbal cameras, and other equipment, combined with vision and trajectory planning technologies, to achieve automated high-precision positioning of the robotic arm's end effector. It eliminates the need for close-range manual command or traction, significantly reducing operational intensity and construction risks, and is suitable for complex construction environments.

[0039] 2. By utilizing UAV top-down imaging to identify the work area and target points, and combining spatial mapping and multi-coordinate system transformation, reliable three-dimensional target positions can be obtained in complex environments, providing high-quality data for precise positioning.

[0040] 3. Visual servo control is constructed based on the error between the actual and expected pixel coordinates to compensate for the end-positioning error caused by the flexible deformation of the arm and load disturbance in real time, so that the end position of the robotic arm always approaches the expected position during the movement, thereby improving the positioning accuracy of the long flexible robotic arm.

[0041] 4. Experiments conducted on a concrete placing boom approximately 13 meters long show that the positioning method proposed in this invention can reduce the end-positioning error to within 5 cm, verifying the significant effect of trajectory planning, desired image feature solving, and visual servoing technology in positioning operations.

[0042] 5. This method has good compatibility and scalability, and can be applied to various types of long flexible boom engineering equipment, such as concrete placing booms. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the positioning method of the present invention.

[0045] Figure 2 This is a schematic diagram of the system structure for a vision-based end-effector positioning method for a long, flexible robotic arm.

[0046] Figure 3 A flowchart for drone identification of columns and trajectory planning and motion control of robotic arm end effectors.

[0047] Figure 4 This diagram illustrates the solution for the desired image and the principle of visual servo closed-loop control.

[0048] Figure 5 This is a diagram showing the movement process of the robotic arm under actual working conditions.

[0049] Figure 6 This is a schematic diagram showing the change in the end position during actual operation. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are merely illustrative and not intended to limit the invention.

[0051] This invention system combines the spatial positional relationships of the drone, the robotic arm base, and each joint to calculate the spatial position of the target point in the robotic arm coordinate system. It employs trajectory planning algorithms such as fifth-order polynomial interpolation to generate a smooth motion trajectory for the robotic arm from its current state to the target point. Furthermore, it solves for the angles of each joint through inverse kinematics, achieving automatic control of the long, flexible robotic arm's end effector.

[0052] Due to the high flexibility of the long, flexible robotic arm structure, structural deformation can occur during actual operation due to its own weight and dynamic loads, causing deviations between the end effector position and the set target. This invention installs a gimbal-mounted vision sensor at the end effector of the robotic arm to acquire images of the work point in real time. Based on a visual servo control algorithm, the robotic arm's posture is dynamically adjusted according to image feature errors to achieve high-precision closed-loop control. Movement automatically stops when the image error converges to a set threshold, thus completing high-precision end effector positioning.

[0053] like Figure 1 and Figure 2As shown, this invention proposes a vision-based end-effector positioning method for long, flexible robotic arms, targeting large-sized robotic arms with multi-degree-of-freedom boom structures, and is particularly suitable for remote, automated, and intelligent concrete placing operations in engineering machinery such as large flexible boom concrete placing machines. The main implementation process is as follows:

[0054] 1. Work site identification and spatial positioning

[0055] Before the robotic arm's end-effector positioning operation begins, the operator uses a handle to control the drone to take off and ascend to a certain height h. The drone then uses an onboard high-resolution camera to acquire a top-down view of the work area. The system uses visual algorithms such as contour detection and region segmentation to identify the area and points to be worked on, obtaining the pixel coordinates of the columns to be poured in the image. .

[0056] Based on the known UAV flight altitude h and camera focal length f, the pixel coordinates of the task point are converted into physical coordinates (x, y, z) in the UAV coordinate system using the principle of similar triangle imaging. The calculation formula is as follows:

[0057]

[0058] By combining the spatial relative positions of the UAV takeoff point and the long flexible robotic arm base, rigid body transformation is used to map the target point position to the robotic arm's own coordinate system, providing input for subsequent motion planning.

[0059] 2. Robotic arm motion trajectory planning

[0060] like Figure 3 As shown, after obtaining the spatial coordinates of the target work point, the system generates the end effector trajectory based on the current state of the robotic arm and using a fifth-order polynomial interpolation method. The trajectory is expressed as:

[0061]

[0062] in This represents the spatial path of the end effector as time t changes. The trajectory starts at the current actual position of the robotic arm's end effector and ends above the mapped target placement point (e.g., 1m above the target point). By constraining the trajectory's start and end velocities and acceleration, the smoothness and continuity of the boom's movement are achieved, avoiding large swaying and impacts.

[0063] 3. Inverse kinematics solution and joint command generation

[0064] Based on the position of the trajectory point in the robot arm coordinate system, the system uses inverse kinematics to solve the joint angles at each moment in real time, based on the arm structure parameters and constraints. :

[0065]

[0066] IK is the inverse kinematics operator. The resulting joint angle sequence is sent as control commands to the robotic arm controller, driving the robotic arm to move along the planned trajectory.

[0067] 4. Perception and Compensation of Deformation Error in Flexible Arm

[0068] Because the robotic arm has a long boom and a flexible structure, it will undergo elastic deformation due to its own weight and dynamic load during deployment and movement, resulting in an error between the actual position of the robotic arm's end and the expected trajectory.

[0069] Therefore, such as Figure 4 As shown, this invention has a gimbal camera installed at the material placement port of the robotic arm. During operation, the end-effector camera collects the pixel coordinates of the four corners of the column to be poured in real time. Combining the assembly parameters of the robotic arm end-effector and the camera, and the camera's intrinsic and extrinsic parameter matrix K, the desired spatial position of the target point is determined using the space-pixel transformation relationship. Projecting onto the camera image plane yields the desired pixel coordinates. :

[0070]

[0071] in, This is a transformation matrix that converts the absolute spatial coordinates of the target point into coordinates in the camera coordinate system.

[0072] 5. Real-time target detection and closed-loop servo correction

[0073] The system is based on target detection and key point extraction algorithms to identify the pixel coordinates s of feature points of the target structure (pillar) in the current image in real time.

[0074] With desired pixel coordinates Using this as a baseline, calculate the current image feature error e:

[0075]

[0076] A visual servoing (IBVS)-based control method is employed to construct the image Jacobian matrix. This establishes a mapping relationship between error and the end effector speed of the robotic arm.

[0077]

[0078] in, This represents the proportional gain coefficient in visual servo control. It determines the speed and intensity of the robotic arm's error correction.

[0079] Based on the aforementioned compensation speed, the main control unit uses the Jacobian matrix to calculate the end velocity. Compensation velocity mapped to joint space The robotic arm's joints are adjusted in real time to gradually converge the end-effector of the fabric to the target point in pixel space. ( When setting a tolerance, the system determines that the end effector is precisely aligned with the target point and automatically stops moving.

[0080] 6. Verification through practical experiments

[0081] To further verify the effectiveness of the proposed method on actual equipment, an end-effector positioning experiment was conducted on a 13m long three-joint fabric placing machine. Based on the position of the uprights identified by the drone, a corresponding end-effector trajectory was planned. The specific positioning operation is as follows: Figure 5 As shown, during the 0-30s interval, the end effector completes its movement along a pre-planned trajectory. Due to the long boom and hollow structure, the end effector position is affected by deformation. For example... Figure 6 As shown, at the end of the movement, there is still a positioning error of about 60cm at the end of the robotic arm. At this time, the error between the end position and the desired point is large, which affects the operation accuracy.

[0082] At this point, the desired image features are calculated and combined with an image-based visual servoing method to achieve precise end-effector localization. For example... Figure 5 and Figure 6 As shown, during the 40s-60s period, as the robotic arm moves, the end-effector positioning error is reduced from 60cm to about 5cm, which greatly improves the end-effector positioning accuracy.

[0083] On the other hand, corresponding to the aforementioned embodiment of a vision-based long flexible robotic arm end-effector positioning method, the present invention also provides a vision-based long flexible robotic arm end-effector positioning system, the system comprising:

[0084] The target coordinate acquisition module is used to obtain an image containing the target to be worked through the UAV, and convert the pixel coordinates of the corner points of the target to be worked in the image into position coordinates in the robotic arm coordinate system based on the positional relationship between the UAV and the robotic arm.

[0085] The motion trajectory acquisition module is used to obtain the motion trajectory of the robotic arm based on the position of the corner point of the target to be worked, and to solve for the change value of the joint angle of the robotic arm.

[0086] The pixel coordinate recognition module is used to detect and identify the real-time pixel coordinates of the corner points of the target to be operated based on the gimbal camera installed at the end of the robotic arm.

[0087] The desired pixel coordinate solution module, based on the fixed assembly relationship between the gimbal camera and the end effector of the robotic arm, transforms the desired position of the end effector of the robotic arm into the camera coordinate system, and combines the camera's intrinsic and extrinsic parameters to complete the imaging projection, thereby obtaining the desired pixel coordinates of the desired position in the image plane.

[0088] The feature error solving module is used to solve for the image feature error based on the pixel coordinates of the desired image features and the real-time pixel coordinates.

[0089] The image Jacobian matrix module is used to obtain depth values ​​based on the binocular parallax principle and the camera intrinsic parameter matrix, and then to obtain the image Jacobian matrix by combining the depth values.

[0090] The joint angle expected velocity calculation module is used for image-based visual servoing methods. It obtains the camera expected velocity required to achieve the desired image features based on the image feature error and the image Jacobian matrix. Based on the positional relationship between the gimbal camera and the robotic arm end effector, it calculates the robotic arm end effector velocity required to reach the desired position and obtains the joint angle expected velocity based on the robotic arm Jacobian matrix.

[0091] The robotic arm end effector control module controls the movement of the robotic arm end effector based on the expected velocity of the joint angle. When the real-time image feature error is reduced to an acceptable range, the robotic arm stops moving.

[0092] The above embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A vision-based method for end-effector positioning of a long, flexible robotic arm, characterized in that: Includes the following steps: Step 1: Obtain an image containing the target to be worked using a drone, and convert the pixel coordinates of the corner points of the target in the image into position coordinates in the robotic arm coordinate system based on the positional relationship between the drone and the robotic arm; Step 2: Based on the obtained position of the corner point of the target to be worked, obtain the motion trajectory of the robotic arm and solve for the change value of the joint angle of the robotic arm; Step 3: Install a gimbal camera at the end of the robotic arm to detect and identify the real-time pixel coordinates of the corner points of the target to be worked on; Step 4: Based on the fixed assembly relationship between the gimbal camera and the end effector of the robotic arm, the desired position of the end effector of the robotic arm is transformed into the camera coordinate system, and the imaging projection is completed by combining the intrinsic and extrinsic parameters of the camera to obtain the desired pixel coordinates of the desired position in the image plane. Step 5: Based on the pixel coordinates of the desired image features and the real-time pixel coordinates, calculate the image feature error; Step 6: Obtain the depth value based on the binocular parallax principle and the camera intrinsic parameter matrix, and then calculate the image Jacobian matrix by combining the depth value; Step 7: Image-based visual servoing method: Based on the image feature error and the image Jacobian matrix, the desired camera speed required to achieve the desired image features is obtained; Based on the positional relationship between the gimbal camera and the robotic arm end effector, the robotic arm end effector speed required to reach the desired position is calculated, and the desired joint angle speed is obtained by solving the robotic arm Jacobian matrix. Step 8: Control the end effector movement of the robotic arm based on the expected velocity of the joint angle. When the real-time image feature error is reduced to an acceptable range, the robotic arm stops moving.

2. The vision-based end-effector positioning method for a long flexible robotic arm according to claim 1, characterized in that, In step 1, before the fabric laying operation begins, the drone and its onboard camera are used to capture images of the target in the construction scene. After obtaining the pixel information of the target, the coordinates of the target point relative to the drone coordinate system are obtained based on the camera's focal length and the drone's flight altitude.

3. The vision-based end-effector positioning method for a long, flexible robotic arm according to claim 1, characterized in that, In step 1, the pixel coordinates of the corner points in the target image to be processed are obtained according to the contour detection method.

4. The vision-based end-effector positioning method for a long flexible robotic arm according to claim 1, characterized in that, In step 2, the motion trajectory of the robotic arm is obtained using a fifth-order polynomial interpolation method.

5. The vision-based end-effector positioning method for a long, flexible robotic arm according to claim 1, characterized in that, In step 4, the specific process of obtaining the desired pixel coordinates by combining the camera's intrinsic and extrinsic parameter matrices with the relative relationship between the robotic arm's end effector is as follows: (1) Calculate the transformation matrix between the end of the robotic arm and the gimbal camera based on the desired position of the end of the robotic arm, the angle of the last section of the robotic arm and the installation position of the gimbal camera; (2) Based on the location of the target point obtained in step 1, calculate the coordinates of the corner point of the target point in the gimbal coordinate system; (3) Based on the intrinsic parameter matrix of the binocular camera, the expected image features of the target corner point are obtained by combining the image Jacobian matrix.

6. The vision-based end-effector positioning method for a long, flexible robotic arm according to claim 1, characterized in that, In step 4, the field of view is adjusted in real time by the gimbal to ensure that the target remains within the effective imaging range, thereby continuously providing stable and reliable desired image features.

7. The vision-based end-effector positioning method for a long flexible robotic arm according to claim 1, characterized in that, In step 5, the image feature error is the difference between the real-time pixel coordinates and the expected pixel coordinates.

8. The vision-based end-effector positioning method for a long flexible robotic arm according to claim 3, characterized in that, Based on the positional relationship between the gimbal camera and the robotic arm's end effector, the required end effector speed to reach the desired position is calculated.

9. The vision-based end-effector positioning method for a long flexible robotic arm according to claim 1, characterized in that, Step 8: When the image feature error is less than the threshold, stop the movement of the robotic arm. The threshold is a preset tolerance.

10. A vision-based end-effector positioning system for a long, flexible robotic arm that implements the method of any one of claims 1-9, characterized in that, The system includes: The target coordinate acquisition module is used to obtain an image containing the target to be worked through the UAV, and convert the pixel coordinates of the corner points of the target to be worked in the image into position coordinates in the robotic arm coordinate system based on the positional relationship between the UAV and the robotic arm. The motion trajectory acquisition module is used to obtain the motion trajectory of the robotic arm based on the position of the corner point of the target to be worked, and to solve for the change value of the joint angle of the robotic arm. The pixel coordinate recognition module is used to detect and identify the real-time pixel coordinates of the corner points of the target to be operated based on the gimbal camera installed at the end of the robotic arm. The desired pixel coordinate solution module, based on the fixed assembly relationship between the gimbal camera and the end effector of the robotic arm, transforms the desired position of the end effector of the robotic arm into the camera coordinate system, and combines the camera's intrinsic and extrinsic parameters to complete the imaging projection, thereby obtaining the desired pixel coordinates of the desired position in the image plane. The feature error solving module is used to solve for the image feature error based on the pixel coordinates of the desired image features and the real-time pixel coordinates. The image Jacobian matrix module is used to obtain depth values ​​based on the binocular parallax principle and the camera intrinsic parameter matrix, and then to obtain the image Jacobian matrix by combining the depth values. The joint angle expected velocity calculation module is used for image-based visual servoing methods. It obtains the camera expected velocity required to achieve the desired image features based on the image feature error and the image Jacobian matrix. Based on the positional relationship between the gimbal camera and the robotic arm end effector, it calculates the robotic arm end effector velocity required to reach the desired position and obtains the joint angle expected velocity based on the robotic arm Jacobian matrix. The robotic arm end effector control module controls the movement of the robotic arm end effector based on the expected velocity of the joint angle. When the real-time image feature error is reduced to an acceptable range, the robotic arm stops moving.

Images