A robotic arm servo control method based on image feature matching

Abstract

This invention discloses a robotic arm servo control method based on image feature matching, comprising the following steps: S1: determining the transformation matrix between the camera and the robotic arm end effector; S2: training an object detection model using a constructed object detection dataset; S3: obtaining the relative pose relationship... c M o The desired image and depth information of the target object: S4: Obtain the rectangular bounding box information of the target object in the current image; S5: Obtain the set of pixel coordinates p, p' of the image feature points that match the current image and the desired image. * And depth set Z; S6: Calculate the control quantity required for the robot arm's movement; then control the robot arm according to the control quantity; S7: Repeat steps S4-S6 to make the robot arm's end effector continuously approach the target until the robot arm's end effector and the target object maintain a relative pose relationship. c M o This invention achieves precise control of the robot's end effector by extracting and matching feature points in an image and combining this with an IBVS servo controller.

Description

Technical Field

[0001] This invention relates to the field of robotic arm servo control, and more specifically to a robotic arm servo control method based on image feature matching. Background Technology

[0002] Visual servoing is a real-time control system that uses vision as a feedback signal. The robotic arm adjusts the position of its end effector based on the position of the target object identified by vision, so as to always maintain a certain distance from the target object.

[0003] With the widespread use of robots in various application scenarios, the demand for visual servoing control systems is increasing. However, traditional visual servoing methods usually require accurate camera calibration and high-quality feature extraction to ensure control accuracy. Currently, the commonly used IBVS method based on feature point tracking suffers from insufficient accuracy and tracking failure when dealing with complex scenes, especially in the presence of occlusion or dynamic environments, where the problems are more pronounced.

[0004] Therefore, given the current problems of insufficient accuracy and tracking failure in visual servo control when dealing with complex scenarios, developing a robotic arm servo control method that can improve the robustness and accuracy of visual servo systems is an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a robotic arm servo control method based on image feature matching, so as to solve the problems of insufficient accuracy and tracking failure in existing visual servo control when dealing with complex scenes.

[0006] To address the aforementioned technical problems, this invention provides a robotic arm servo control method based on image feature matching, comprising the following steps:

[0007] S1: Determine the transformation matrix between the camera and the end effector of the robotic arm using a hand-eye calibration method;

[0008] S2: Construct a target detection dataset and target detection model for the target object to be tracked, and train the target detection model using the target detection dataset;

[0009] S3: Adjust the relative pose between the camera and the robotic arm's end effector in the virtual environment. c M o And obtain the relative pose relationship c M o The desired image of the target object and the depth information of the desired image:

[0010] S4: Capture the current image of the target object at the current moment through the camera, and input the current image into the trained target detection model to detect the rectangular bounding box information of the target object in the current image;

[0011] S5: Combine the rectangular bounding box information to extract and match feature points of the current image and the desired image respectively, to obtain the set p of pixel coordinates of feature points of the current image and the set p of pixel coordinates of feature points of the desired image. * Then, obtain the depth set Z of the set of pixel coordinates p of the feature points in the current image;

[0012] S6: Based on the current image feature point pixel coordinate set p, the corresponding depth set Z, and the desired image feature point pixel coordinate set p * and the corresponding depth set Z * Calculate the control quantities required for the robotic arm's movement; then control the robotic arm based on these control quantities.

[0013] S7: Repeat steps S4-S6 to make the robotic arm end effector continuously approach the target until the robotic arm end effector and the target object maintain a relative pose relationship. c M o .

[0014] Further, in step S1, the camera is mounted on the end effector of the robotic arm using an eye-to-hand calibration method. Then, the transformation relationship between the camera and the end effector is determined using an eye-to-hand calibration method. Finally, the transformation matrix between the camera coordinate system and the robotic arm coordinate system is determined based on the transformation relationship. g T c .

[0015] Furthermore, step S2 specifically includes:

[0016] S21: Acquire images of the target objects to be tracked, and then annotate the rectangular bounding boxes and category information of the multi-class targets in the acquired images to obtain the labeled target detection dataset;

[0017] S22: Divide the labeled target detection dataset into training set, validation set and test set according to the set ratio;

[0018] S23: Construct an object detection model, and use the divided object detection dataset to train, verify, and test the object detection model to obtain a trained object detection model.

[0019] Furthermore, step S3 specifically includes:

[0020] S31: Import the target object into the virtual environment, and then adjust the camera parameters in the virtual environment;

[0021] S32: Adjust the relative position between the camera and the target object in the virtual environment so that the target object is within the camera's field of view and maintains a relative pose relationship between the target object and the camera. c M o ;

[0022] S33: Capture the relative pose of the target object using a camera in a virtual environment. c M o The desired image and depth information under the given conditions.

[0023] Furthermore, step S5 specifically includes:

[0024] S51: Extract feature points from the current image using a feature point extraction network to obtain the set of pixel coordinates of the current image feature points p. o =(u o1 ,v o1 ,...,u ok ,v ok )∈R 2k and the current image feature point description information set D o ={d o1 ,…,d ok}∈R k The feature points of the desired image are extracted using a feature point extraction network, resulting in a set of pixel coordinates for these feature points. and the set of descriptive information of image feature points

[0025] S52: Set the pixel coordinates of the current image feature points p o The current image feature point description information set D o , Expected set of pixel coordinates of image feature points and the set of descriptive information of image feature points The input is fed into a feature matching network to obtain a set of matching pixel coordinates of current image feature points, p = (u1, v1, ..., u...). k ,v k )∈R 2k and the set of pixel coordinates of the desired image feature points

[0026] S53: Extract a subset of current image feature point pixel coordinates p1 = (u1, v1, ..., u1) within the rectangular bounding box of the target object's location from the current image feature point pixel coordinate set p. n ,v n )∈R 2n n≤k; Assign the subset p1 of the current image feature point pixel coordinates to the current image feature point pixel coordinate set p = p1 = (u1, v1, ..., un ,v n )∈R 2n , n≤k, and obtain the depth value set Z=(z1,…,z1) corresponding to the pixel coordinate set p=p1 of the feature points in the current image. n )∈R n .

[0027] Furthermore, step S5 also includes:

[0028] S54: Determine the number of elements in the current image feature point pixel coordinate subset p. If the number of elements is less than 3, use the set of corner pixel coordinates of the bounding box of the target object's location as the current image feature point pixel coordinate set p = p. c ={p c1 ,p c2 ,p c3 ,p c4}

[0029] Furthermore, in step S6, calculating the control quantities required for the robotic arm's movement specifically includes:

[0030] S61: Obtain the visual feature vector s(p(t),ξ) of the current position based on the set of pixel coordinates of image feature points p, and based on the desired set of pixel coordinates of image feature points p * Obtain the visual feature vector s of the desired location * (p * ,ξ), where ξ represents the camera intrinsic parameters;

[0031] S62: Based on the visual feature vector s(p(t),ξ) of the current position and the visual feature vector s of the desired position * (p * ,ξ), calculate the visual feature vector error e(t)=s(p(t),ξ)-s * (p * ,ξ);

[0032] S63: Calculate the desired motion speed required for the robotic arm's end effector to track the target at the next moment based on the classical IBVS control law;

[0033] S64: The desired motion velocity is processed by a Kalman filter, and then the next state is estimated using the system's dynamic model to obtain the control quantity.

[0034] Furthermore, step S63 specifically includes:

[0035] S631: Using the camera intrinsic parameter ξ, the pixel values ​​are converted into Cartesian coordinates on the image plane: s = s(p(t), ξ) = (x1, y1, ..., x k ,y k)∈R 2k ;

[0036] S632: Spatial velocity V of the robot's end effector is expressed as camera velocity. c =(v x ,v y ,v z ,ω x ,ω y ,ω z )∈R 6 , where v x ,v y ,v z These are the camera's translational speeds, ω and ω', respectively. x ,ω y ,ω z These are the camera's rotation speeds;

[0037] S633: Combine the visual feature vector s(p(t),ξ) at the current position with the visual feature vector s(t) at the desired position. * (p * The input of ξ) into the IBVS servo control system is expressed by the following formula: c Related to s:

[0038]

[0039] Where L(s,Z)∈R 2k×6 This is the interaction matrix, which corresponds to the image Jacobian matrix;

[0040] S634: The desired motion velocity required for the robotic arm's end effector to track the target at the next moment is calculated using the classical IBVS control law. The calculation formula is as follows:

[0041]

[0042] in, Let λ be an approximation of L, and λ be the proportionality coefficient.

[0043] Furthermore, in step S6, controlling the robotic arm according to the control quantity specifically includes:

[0044] S65: Calculate the Jacobian matrix J of the end effector based on the robot's kinematic model;

[0045] S66: Based on the control quantity V c By using the Jacobian matrix J and inverse kinematics of the robot, the required rotational speed of each joint can be calculated. Or the position of rotation q;

[0046] S67: Based on speed Alternatively, position q can be used to control the speed or angle of each joint motor, thereby achieving precise control of the end effector speed of the robotic arm at the next moment.

[0047] Furthermore, step S7 specifically includes:

[0048] S71: Set the error threshold e * ;

[0049] S72: Repeat steps S4-S6 until the visual feature vector error e(t) is reached. <e * Then the output speed V of the robotic arm end effector is... c =0, the robotic arm stops moving.

[0050] The beneficial effects of this invention are as follows:

[0051] 1. By using a deep learning-based image feature extraction and matching method, the problem of obtaining visual feature input for servo controllers caused by factors such as occlusion and lighting in complex scenes is solved. At the same time, the rapid extraction and matching of feature points meets the real-time control requirements of the robotic arm, and can more accurately determine the relative pose of the target object and the end effector of the robotic arm, reducing the positioning error of the robotic arm in visual servo control.

[0052] 2. The method of extracting the bounding box information of target objects by training the target detection model emphasizes the local features of the area where the target object is located during the image feature point matching process, which solves the problem of servo controller input errors caused by incorrect matching of feature points outside the target object within the camera's field of view; at the same time, the bounding box information of the target object can be used as image features, which solves the problem of no solution for the control quantity when the number of image feature points is too small, effectively enhancing the robustness of the system and reducing the possibility of tracking failure.

[0053] 3. By using a Kalman filter to process the desired motion speed, the problem of robotic arm jitter caused by sudden changes in control quantity due to calculation errors is solved, making the robotic arm movement smoother; at the same time, the tracking ability of dynamic targets is improved, enabling the robotic arm to respond quickly to the movement of target objects and track them effectively. Attached Figure Description

[0054] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, use the same reference numerals to denote the same or similar parts. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0055] Figure 1 This is a flowchart of the visual feature extraction algorithm of the present invention;

[0056] Figure 2 This is a flowchart illustrating the implementation of the dynamic tracking method of the present invention. Detailed Implementation

[0057] like Figure 1 The robotic arm servo control method based on image feature matching shown includes the following steps:

[0058] S1: Determine the transformation matrix between the camera and the end effector of the robotic arm using a hand-eye calibration method;

[0059] S2: Construct a target detection dataset and target detection model for the target object to be tracked, and train the target detection model using the target detection dataset;

[0060] S3: Adjust the relative pose between the camera and the robotic arm's end effector in the virtual environment. c M o And obtain the relative pose relationship c M o The desired image of the target object and the depth information of the desired image:

[0061] S4: The camera captures the current image of the target object and inputs it into a trained target detection model to detect the rectangular bounding box information of the target object in the current image. By extracting the bounding box information of the target object through the training of the target detection model, the local features of the area where the target object is located are emphasized during the image feature point matching process. This solves the problem of servo controller input errors caused by incorrect matching of feature points outside the target object within the camera's field of view. At the same time, the bounding box information of the target object can be used as image features, solving the problem of no solution for the control quantity when the number of image feature points is too small. This effectively enhances the robustness of the system and reduces the possibility of tracking failure.

[0062] S5: Combine the rectangular bounding box information to extract and match feature points of the current image and the desired image respectively, to obtain the set p of pixel coordinates of feature points of the current image and the set p of pixel coordinates of feature points of the desired image. * Then, the depth set Z of the current image feature point pixel coordinate set p is obtained from the current image. Through the image feature extraction and matching method based on deep learning, the problem of difficulty in obtaining visual feature input of servo controller caused by factors such as occlusion and lighting in complex scenes is solved. At the same time, the rapid extraction and matching of feature points meets the real-time control requirements of the robotic arm, and can more accurately determine the relative pose of the target object and the end of the robotic arm, reducing the positioning error of the robotic arm in visual servo control.

[0063] S6: Based on the current image feature point pixel coordinate set p, the corresponding depth set Z, and the desired image feature point pixel coordinate set p * and the corresponding depth set Z *Calculate the control quantities required for the robotic arm's movement; then control the robotic arm based on these control quantities.

[0064] S7: Repeat steps S4-S6 to make the robotic arm end effector continuously approach the target until the robotic arm end effector and the target object maintain a relative pose relationship. c M o .

[0065] This invention extracts and matches feature points in images and combines them with an IBVS servo controller to control the robot's end effector, enabling it to better handle complex scenes and large objects, and exhibiting higher robustness and accuracy.

[0066] According to one embodiment of this application, in step S1, a camera (optionally an RGB-D depth camera) is mounted on the end effector of the robotic arm using an eye-in-hand method. Then, the transformation relationship between the camera and the end effector is determined using a hand-eye calibration method. Finally, the transformation matrix between the camera coordinate system and the robotic arm coordinate system is determined based on the transformation relationship. g T c .

[0067] According to one embodiment of this application, step S2 specifically includes:

[0068] S21: Collect RGB images of the target objects to be tracked (the target objects are mainly industrial parts of various types), and then use the annotation software X-Anylabeling to automatically annotate the rectangular bounding boxes and their category information of the multi-class targets in the collected images to obtain the labeled target detection dataset.

[0069] S22: Divide the labeled target detection dataset into training set, validation set and test set in sequence according to the set ratio (the set ratio can be set to 7:2:1);

[0070] S23: Construct an object detection model (YOLOv8m model can be used), train, validate, and test the object detection model using the pre-defined object detection dataset to obtain a trained object detection model. During training, when a significant decrease in training error and the mAP (mean accuracy) reach the expected level are observed after 500 training rounds, the object detection model is considered successfully trained.

[0071] According to one embodiment of this application, step S3 specifically includes:

[0072] S31: Import the target object into the virtual environment of the software Blender, and then adjust the camera's resolution, focal length, size and other parameters in the virtual environment to make them consistent with the parameters of the camera in the real environment.

[0073] S32: Adjust the relative position between the camera and the target object in the virtual environment so that the target object is within the camera's field of view and that the target object and the camera have a certain relative pose relationship. c M o ;

[0074] S33: Capture the relative pose of the target object using a camera in a virtual environment. c M o The desired image and depth information under the given conditions.

[0075] According to one embodiment of this application, step S4 specifically includes:

[0076] S4: Acquire the current image of the target object using a camera at the end of the robotic arm, and input the current image into the target detection model trained based on the YOLOv8m model to obtain the set of pixel coordinates P of the corner points of the rectangular bounding box of the target object in the current image. c ={p c1 ,p c2 ,p c3 ,p c4}

[0077] According to one embodiment of this application, step S5 specifically includes:

[0078] S51: Extract feature points of the current image using the SuperPoint feature point extraction network to obtain the set of pixel coordinates of the current image feature points p. o =(u o1 ,v o1 ,...,u ok ,v ok )∈R 2k and the current image feature point description information set D o ={d o1 ,…,d ok}∈R k The SuperPoint feature point extraction network extracts feature points from the desired image, resulting in a set of pixel coordinates for these feature points. and the set of descriptive information of image feature points

[0079] S52: Set the pixel coordinates of the current image feature points p o The current image feature point description information set D o , Expected set of pixel coordinates of image feature points and the set of descriptive information of image feature points The input is fed into the LightGlue feature matching network to obtain the set of pixel coordinates of the current image feature points, p = (u1, v1, ..., u...). k ,v k )∈R 2k and the set of pixel coordinates of the desired image feature points

[0080] S53: Extract a subset of current image feature point pixel coordinates p1 = (u1, v1, ..., u1) within the rectangular bounding box of the target object's location from the current image feature point pixel coordinate set p. n ,v n )∈R 2n n≤k; Assign the subset p1 of the current image feature point pixel coordinates to the current image feature point pixel coordinate set p = p1 = (u1, v1, ..., u n ,v n )∈R 2n , n≤k, and obtain the depth value set Z=(z1,…,z1) corresponding to the pixel coordinate set p=p1 of the feature points in the current image. n )∈R n .

[0081] According to one embodiment of this application, step S5 further includes:

[0082] S54: Determine the number of elements in the current image feature point pixel coordinate subset p. If the number of elements is less than 3, use the set of corner pixel coordinates of the bounding box of the target object's location as the current image feature point pixel coordinate set p = p. c ={p c1 ,p c2 ,p c3 ,p c4}

[0083] According to one embodiment of this application, step S6, calculating the control quantities required for the movement of the robotic arm, specifically includes:

[0084] S61: Obtain the visual feature vector s(p(t),ξ) of the current position based on the set of pixel coordinates of image feature points p, and based on the desired set of pixel coordinates of image feature points p * Obtain the visual feature vector s of the desired location * (p * ,ξ), where ξ represents the camera intrinsic parameters;

[0085] S62: Based on the visual feature vector s(p(t),ξ) of the current position and the visual feature vector s of the desired position * (p *,ξ), calculate the visual feature vector error e(t)=s(p(t),ξ)-s * (p * ,ξ);

[0086] S63: Calculate the desired motion speed required for the robotic arm's end effector to track the target at the next moment based on the classical IBVS control law;

[0087] S64: The desired motion velocity is processed by a Kalman filter, and then the next state is estimated using the system's dynamic model to obtain the control quantity.

[0088] According to one embodiment of this application, step S63 specifically includes:

[0089] S631: Using the camera intrinsic parameter ξ, the pixel values ​​are converted into Cartesian coordinates on the image plane: s = s(p(t), ξ) = (x1, y1, ..., x k ,y k )∈R 2k Similarly, p * and s * It is a camera image I from the target location. * The measured values ​​obtained;

[0090] S632: Spatial velocity V of the robot's end effector is expressed as camera velocity. c =(v x ,v y ,v z ,ω x ,ω y ,ω z )∈R 6 , where v x ,v y ,v z These are the camera's translational speeds, ω and ω', respectively. x ,ω y ,ω z These are the camera's rotation speeds;

[0091] S633: Combine the visual feature vector s(p(t),ξ) at the current position with the visual feature vector s(t) at the desired position. * (p * The input of ξ) into the IBVS servo control system is expressed by the following formula: c Related to s:

[0092]

[0093] Where L(s,Z)∈R 2k×6 This is the interaction matrix, which corresponds to the image Jacobian matrix;

[0094] S634: The desired motion velocity required for the robotic arm's end effector to track the target at the next moment is calculated using the classical IBVS control law. The calculation formula is as follows:

[0095]

[0096] in, λ is an approximation of L, because L is usually not accurately obtained in IBVS experiments, and λ is the proportionality coefficient.

[0097] According to one embodiment of this application, due to the influence of noise and errors, the expected motion speed is not directly used for robot control. Instead, the calculation result is processed by a Kalman filter to suppress the influence of outliers on the accuracy of the servo control system. The predicted state estimation formula is as follows:

[0098]

[0099] The formula for predicting state covariance is:

[0100] P k =AP k-1 A T +Q

[0101] in, This is the state estimation vector for time step k; P is the state estimation vector for time step k-1; k P is the state covariance matrix at time step k; k-1 Let A be the state covariance matrix at time step k-1; let A be the state transition matrix, used to describe the dynamic behavior of the system; and let Q be the process noise covariance matrix, representing the uncertainty in the prediction process.

[0102] Then, the next state is estimated using the system's dynamic model. The Kalman filter receives the measurement results and updates the state estimation vector and state covariance matrix. The Kalman gain is calculated as follows:

[0103] K = P k H T HP k H T +R) -1

[0104] The formula for updating the state estimate is as follows:

[0105]

[0106] The formula for updating the state covariance is as follows:

[0107] P k =(I-KH)P k

[0108] Where K is the Kalman gain, which measures the degree of influence of the measurement update on the state estimate; z k H is the measurement result; H is the observation matrix, used to map the state estimation vector to the measurement space; R is the measurement noise covariance matrix, representing the uncertainty in the measurement process; I is the identity matrix.

[0109] The control quantity V is finally obtained through the above method. c =(v x ,v y ,v z ,ω x ,ω y ,ω z )∈R 6 This refers to the speed of motion required by the end effector of the robotic arm when tracking the target.

[0110] According to one embodiment of this application, step S6, controlling the robotic arm according to the control quantity, specifically includes:

[0111] S65: Calculate the Jacobian matrix J of the end effector based on the robot's kinematic model;

[0112] S66: Based on the control quantity V c By using the Jacobian matrix J and inverse kinematics of the robot, the required rotational speed of each joint can be calculated. Or the position of rotation q; for example, according to the properties of the Jacobian matrix, the end effector velocity V of the robotic arm is related to the joint velocity. The relationship between them can be expressed by the following formula:

[0113]

[0114] S67: Based on speed Alternatively, position q can be used to control the speed or angle of each joint motor, thereby achieving precise control of the end effector speed of the robotic arm at the next moment.

[0115] According to one embodiment of this application, step S7 specifically includes:

[0116] S71: Set the error threshold e * ;

[0117] S72: Repeat steps S4-S6 until the visual feature vector error e(t) is reached. <e * Then the output speed V of the robotic arm end effector is... c =0, the robotic arm stops moving, and at this time the end effector of the robotic arm and the target object are basically in a relative pose. c M o That is, to complete control.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A servo control method for a robotic arm based on image feature matching, characterized in that, Including the following steps: S1: Determine the transformation matrix between the camera and the end effector of the robotic arm using a hand-eye calibration method; S2: Construct a target detection dataset and a target detection model for the target object to be tracked, and train the target detection model using the target detection dataset; S3: Adjust the relative pose between the camera and the robotic arm's end effector in the virtual environment. And obtain the relative pose relationship. The desired image of the target object and the depth information of the desired image; S4: Acquire the current image of the target object at the current moment through the camera, and input the current image into the trained target detection model to detect the rectangular bounding box information of the target object in the current image; S5: Combine the rectangular bounding box information to extract and match the feature points of the current image and the desired image respectively, to obtain the set of pixel coordinates of the current image feature points of the matched current image. and the set of pixel coordinates of the desired image feature points Then, obtain the set of pixel coordinates of feature points in the current image from the current image. depth set ; S6: Based on the current image feature point pixel coordinate set and the corresponding depth set and the set of pixel coordinates of the desired image feature points and the corresponding depth set Calculate the control quantities required for the movement of the robotic arm; then control the robotic arm according to the control quantities. S7: Repeat steps S4-S6, causing the robotic arm end effector to continuously approach the target until the robotic arm end effector and the target object maintain the aforementioned relative pose relationship. .

2. The robotic arm servo control method based on image feature matching according to claim 1, characterized in that, In step S1, the camera is mounted on the end effector of the robotic arm using an eye-to-hand calibration method. Then, the transformation relationship between the camera and the end effector is determined using an eye-to-hand calibration method. Finally, the transformation matrix between the camera coordinate system and the robotic arm coordinate system is determined based on the transformation relationship. .

3. The robotic arm servo control method based on image feature matching according to claim 2, characterized in that, Step S2 specifically includes: S21: Acquire images of the target objects to be tracked, and then annotate the rectangular bounding boxes and category information of the multi-class targets in the acquired images to obtain the labeled target detection dataset; S22: Divide the labeled target detection dataset into a training set, a validation set, and a test set according to a set ratio; S23: Construct an object detection model, and use the divided object detection dataset to train, verify, and test the object detection model to obtain a trained object detection model.

4. The robotic arm servo control method based on image feature matching according to claim 3, characterized in that, Step S3 specifically includes: S31: Import the target object into the virtual environment, and then adjust the camera parameters in the virtual environment; S32: Adjust the relative position between the camera and the target object in the virtual environment so that the target object is within the camera's field of view and maintains a relative pose relationship between the target object and the camera. ; S33: Capture the relative pose of the target object using a camera in a virtual environment. The desired image and depth information under the given conditions.

5. The robotic arm servo control method based on image feature matching according to claim 4, characterized in that, Step S5 specifically includes: S51: Extract feature points of the current image using a feature point extraction network to obtain a set of pixel coordinates of the current image feature points. and the current image feature point description information set The feature points of the desired image are extracted using a feature point extraction network to obtain a set of pixel coordinates of the feature points in the desired image. and the set of descriptive information of image feature points ; S52: Set the pixel coordinates of the current image feature points The current image feature point description information set , Expected set of pixel coordinates of image feature points and the set of descriptive information of image feature points The input is fed into a feature matching network to obtain a set of pixel coordinates of the current image feature points that match. and the set of pixel coordinates of the desired image feature points ; S53: From the current image feature point pixel coordinate set Extract the subset of pixel coordinates of current image feature points within the rectangular bounding box containing the location of the target object. , ; Subset of pixel coordinates of current image feature points Assigned to the set of pixel coordinates of the current image feature points , And obtain the set of pixel coordinates of the feature points in the current image. Corresponding depth value set .

6. The robotic arm servo control method based on image feature matching according to claim 5, characterized in that, Step S5 further includes: S54: Determine the subset of pixel coordinates of the current image feature points The number of elements in the image is such that when the number of elements is less than 3, the set of pixel coordinates of the corner points of the rectangular bounding box where the target object is located is used as the set of pixel coordinates of the image feature points of the current image. .

7. The robotic arm servo control method based on image feature matching according to claim 5 or 6, characterized in that, In step S6, calculating the control quantities required for the robotic arm's movement specifically includes: S61: Based on the set of pixel coordinates of image feature points Obtain the visual feature vector of the current position. According to the set of pixel coordinates of the desired image feature points Obtain the visual feature vector of the desired location ,in, Indicates camera intrinsic parameters; S62: Based on the visual feature vector of the current position Visual feature vectors of the desired location Calculate the error of visual feature vectors ; S63: Calculate the desired motion velocity required by the robotic arm's end effector to track the target at the next moment based on the classical IBVS control law; S64: The desired motion speed is processed by a Kalman filter, and then the next state is estimated using the dynamic model of the system to obtain the control quantity.

8. The robotic arm servo control method based on image feature matching according to claim 7, characterized in that, Step S63 specifically includes: S631: Utilizing camera intrinsic parameters The pixel values ​​are converted to Cartesian coordinates in the image plane. ; S632: Spatial velocity of the robot's end effector expressed as camera velocity. ,in, These represent the camera's translation speed, These are the camera's rotation speeds; S633: Transfer the visual feature vector of the current position. Visual feature vectors of the desired location Input into the IBVS servo control system, using the following formula and Related: in, This is the interaction matrix, which corresponds to the image Jacobian matrix; S634: The desired motion velocity required for the robotic arm's end effector to track the target at the next moment is calculated using the classical IBVS control law. The calculation formula is as follows: in, for Approximate value, This is the proportionality coefficient.

9. The robotic arm servo control method based on image feature matching according to claim 8, characterized in that, In step S6, controlling the robotic arm according to the control quantity specifically includes: S65: Calculate the Jacobian matrix of the end effector based on the robot's kinematic model. ; S66: According to the control quantity Using the Jacobian matrix The required rotational speed of each joint is calculated using robot inverse kinematics. or the position of rotation ; S67: According to the stated speed or location It controls the speed or angle of each joint motor to achieve precise control of the end effector speed of the robotic arm at the next moment.

10. The robotic arm servo control method based on image feature matching according to claim 7, characterized in that, Step S7 specifically includes: S71: Set error threshold ; S72: Repeat steps S4-S6 until the visual feature vector error is reached. Then output the end effector speed of the robotic arm. The robotic arm stopped moving.

Images