An object grasping method based on visual feedback control

Abstract

This invention discloses an object grasping method based on visual feedback control, belonging to the field of robot control technology, and includes the following steps: S1: acquiring the position and posture information of the target object; S2: robotic arm path planning; S3: grasping the target object. This invention uses a Kinect visual sensor to acquire the position and posture information of the target object in real time, and feeds this information back to the controller, which then controls the robot's robotic arm to achieve stable grasping of the target object and transfer it to its destination, thus improving the robot's grasping ability and intelligence in complex environments.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, specifically to an object grasping method based on visual feedback control. Background Technology

[0002] In the past decade, with the continuous development of automation technology in industry, various types of robots have been widely used in various fields, such as production services and manufacturing. In the field of robotic arms, object grasping and visual recognition and detection are key research objects. However, in practical applications, there are still some problems with robot visual recognition and path planning, such as the grasping strategy of the robotic arm to grasp different objects and the accuracy of visual target recognition. These problems urgently need to be solved. To address this, a visual feedback control-based object grasping method is proposed. Summary of the Invention

[0003] The technical problem to be solved by this invention is: how to achieve accurate identification of the position and posture of a target object, thereby achieving accurate path planning, improving the robot's grasping ability and intelligence in complex environments, and providing an object grasping method based on visual feedback control.

[0004] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0005] S1: Acquisition of target object position and orientation information

[0006] A vision camera is installed on the robot to acquire point cloud data using a vision sensor. The point cloud data is preprocessed, and the PCL algorithm is used to find a plane based on the point cloud data. Given point cloud data to the plane, the point cloud data in three-dimensional space is transformed into point cloud data on the plane. The normal vector after plane fitting and the points on the plane are used to identify the position of the target object and obtain the coordinates of the center point of the target object, i.e. the position information of the target object. Then, the pose information of the target object is calculated based on the coordinates of the center point of the target object.

[0007] S2: Robotic Arm Path Planning

[0008] Cartesian path planning is performed on the robot's robotic arm. Inverse kinematics is used to calculate the joint angles of the robotic arm so that the end effector of the robotic arm can reach the target object. Then, a PID controller is used to control the movement of the end effector of the robotic arm so that the end effector of the robotic arm moves to the target object.

[0009] S3: Target Object Grabbing

[0010] After completing the path planning of the robotic arm, the robot uses its robotic arm to grasp the target object. First, the posture of the robotic arm's end effector is determined, i.e., the end effector is opened. Using the target object position information obtained in step S1, the end effector is moved to the grasping position of the target object. When the end effector reaches the grasping position, based on the posture of the end effector and the posture information of the target object, the robot controls the end effector to grasp the target object. Then, the end effector is moved to the target position. Finally, the target object is released, i.e., the end effector is opened to release the target object from the end effector, and then the robotic arm returns to its initial posture, completing the target object grasping task.

[0011] Furthermore, in step S1, the robot includes two symmetrical robotic arms, each with seven independently rotating motion joints, the rotation angle, rotation speed and torque parameters of each joint are known.

[0012] Furthermore, in step S1, the preprocessing of the point cloud data includes downsampling, filtering, and outlier removal.

[0013] Furthermore, in step S1, the plane is represented as a normal vector. And a point P0, where the normal vector Perpendicular to the plane, P0 is any point on the plane.

[0014] Furthermore, the normal vector The specific solution process is as follows:

[0015] S101: Suppose we have a point cloud dataset S = {P1, P2, ..., P...} containing N points. N}, where each point P i All can be represented as (x i ,y i ,z i The points in the point cloud dataset S are treated as vectors in three-dimensional space in the form of ) , i.e.

[0016] S102: Calculate the covariance matrix C of the point cloud dataset S, using the following formula:

[0017]

[0018] in, It is the average of all points, that is:

[0019]

[0020] S103: Find the eigenvectors and corresponding eigenvalues ​​of the covariance matrix C. Sort the eigenvectors according to their corresponding eigenvalues ​​from largest to smallest. Then select the k smallest eigenvectors and use them to calculate the normal vector. normal vector as follows:

[0021]

[0022] Furthermore, in step S1, the specific process of converting point cloud data in three-dimensional space into point cloud data on a plane is as follows:

[0023] S111: Select any point in the point cloud dataset S. Let point P1 be selected. Then P1 can be represented as (x1, y1, z1). The normal vector of the plane containing the target object... A point P1 = (x1, y1, z1) on a plane can be represented in homogeneous coordinate form. And P1 = [x1, y1, z1, 1] T ;

[0024] S112: For each point in the point cloud data, represent it in homogeneous coordinate form q = [x, y, z, 1]. T Move point q along the normal vector Projecting this onto the plane, we obtain the coordinates of the projection of point q' onto the plane:

[0025]

[0026] in,

[0027] S113: Obtain the non-homogeneous coordinate form q'=[x',y',z'] T Then, the non-homogeneous coordinates of all projected points are combined into a point cloud dataset on a plane, thereby transforming the point cloud data in three-dimensional space into point cloud data on a plane.

[0028] Furthermore, in step S1, the specific process of calculating the coordinates of the center point of the target object is as follows:

[0029] S121: Extract the point coordinates (x, y) of the target object through point cloud data processing. i ,y i ,z i Initialize cluster center C k Randomly select k points as the initial cluster centers;

[0030] S122: Calculate each point P i To each cluster center C k distance d ikThe formula is as follows:

[0031]

[0032] S122: The calculated distance d ik Point P i Assign the child to the cluster containing the nearest cluster centroid, calculate the average of all coordinates within that cluster, and then determine the center coordinates of that cluster.

[0033]

[0034]

[0035]

[0036] Where, n k This represents the number of points in cluster k;

[0037] S123: Repeatedly calculate d ik , The value is returned until it no longer changes, and the center point coordinates of the cluster are returned as the center point coordinates of the target object.

[0038] Furthermore, in step S1, the specific process of calculating the pose information of the target object based on the center point coordinates is as follows:

[0039] S131: Perform eigenvalue decomposition on the covariance matrix C to obtain eigenvalues ​​λ1, λ2, λ3 and corresponding eigenvectors v1, v2, v3;

[0040] S132: Move point P i Projecting these features onto the plane formed by feature vectors v1 and v2 yields a two-dimensional point cloud data Q = {q1, q2, ..., q...}. n},in

[0041] S133: According to the properties of the PCA algorithm, the principal direction vector v1 and the normal vector on the plane... Orthogonal, taking another vector v2 on the plane as one component of this vector, while the other dimension is 0, thus obtaining the vector in three-dimensional space:

[0042] v3 = [v1[0], v1[1], 0]

[0043] Where v1 is the principal direction vector calculated by the PCA algorithm;

[0044] S134: Rotate vector v1 back to the original coordinate system, where the rotation matrix is ​​R, which rotates the basis vectors [v2,v3] on the plane back to the basis vectors [x,y,z] in the original coordinate system. Then R is:

[0045]

[0046] S135: Calculate the pose information of the target object based on the center point coordinates and the v1 vector. The calculation method is as follows:

[0047] S1351: Let the coordinates of the center point of the target object be c, and the attitude vector be v, then we have:

[0048] v = [v1[0], v1[1], 0] * R

[0049] Where * denotes the product of a vector and a matrix;

[0050] S1352: Based on the properties of vectors, the orientation vector of the target object is represented as follows:

[0051] o=[c[0]+v[0],c[1]+v[1],c[2]+v[2]-c

[0052] Where o is the orientation vector of the target object, and c is the coordinate of the center point of the target object;

[0053] This allows us to obtain the orientation information of the target object.

[0054] Furthermore, in step S2, the specific processing procedure is as follows:

[0055] S21: Use cubic spline interpolation to fit the position, velocity, and acceleration of the robotic arm, thereby generating a continuous second-order polynomial between adjacent data points;

[0056] S22: A PID controller is used to control the movement of the robotic arm. The PID controller calculates the control output by comparing the actual output with the desired output. The position, speed and acceleration of the robotic arm are taken as the desired output, and the actual output is set as the current state of the robotic arm. Then the error signal is calculated and the PID controller is used to calculate the control output.

[0057] S23: Send the control output to the PID controller of the robotic arm to execute the desired motion, causing the end effector of the robotic arm to move to the target object.

[0058] Furthermore, in step S21, the spline interpolation function is calculated using the following formula:

[0059] S i (x)=a i +bi (xx i )+c i (xx i ) 2 +d i (xx i ) 3

[0060] Among them, S i (x) is in x i and x i+1 spline interpolation function between, a i It is the position of the i-th data point, b i c is the velocity of the i-th data point. i d is the acceleration of the i-th data point. i It is the rate of change of velocity at the i-th data point, and the position, velocity, and acceleration of the robotic arm at any time point are calculated using a spline interpolation function;

[0061] Furthermore, in step S22, the control output is calculated using the following formula:

[0062]

[0063] Where u(t) is the control output, K p K i and K d These are the parameters of the PID controller, and e(t) is the error signal. It is the rate of change of the error signal. It is the integral of the error signal.

[0064] Furthermore, in step S3, the specific processing procedure is as follows:

[0065] S31: Determine the posture of the robotic arm end effector, that is, control the opening of the robotic arm end effector, which is achieved by controlling the gripper of the robotic arm end effector;

[0066] S32: Use the target object position information obtained in step S1 to determine the position of the target object and calculate the gripping position that the robotic arm end effector needs to move to.

[0067] S33: When the end effector of the robotic arm reaches the gripping position, control the end effector of the robotic arm to grip the target object according to the posture of the end effector; control the gripper to close and clamp the target object;

[0068] S34: Confirm that the robotic arm has stably grasped the target object. This step uses a built-in force sensor to detect whether the force of the gripper has reached a preset threshold to determine whether the target object has been successfully grasped.

[0069] S35: Then move the robotic arm end effector to the target position, calculate the path planning of the robotic arm end effector again using the path planning method in step S2, and control it.

[0070] S36: Finally, release the target object, that is, control the end effector of the robotic arm to open, so that the target object is released from the end effector of the robotic arm; after completing the target object grasping task, the robotic arm needs to return to the initial pose in order to perform the next operation.

[0071] Compared with the prior art, the present invention has the following advantages: This object grasping method based on visual feedback control acquires the position and posture information of the target object in real time through the Kinect visual sensor, feeds this information back to the controller, and then controls the robot's robotic arm to achieve stable grasping of the target object and transfer it to the destination, thereby improving the robot's grasping ability and intelligence in complex environments. Attached Figure Description

[0072] Figure 1 This is a flowchart illustrating the object grasping method based on visual feedback control according to the present invention.

[0073] Figure 2 This is a schematic diagram showing the position of the end effector of the right robotic arm of the robot in an embodiment of the present invention;

[0074] Figure 3 This is a schematic diagram illustrating the change in the value function of the robotic arm. Detailed Implementation

[0075] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0076] like Figure 1 As shown, this embodiment provides a technical solution: a method for object grasping based on visual feedback control, including the following steps:

[0077] Step 1: Understanding the Robot's Parameters. This is an intelligent collaborative robot with a humanoid design. Each of its left and right robotic arms has seven independently rotating joints. The robot's joint angles, rotation speeds, and torque parameters are crucial inputs to the robot's control system. They are used to calculate the motion trajectory of the end effector and control the force applied to the robotic arm. The robot can use these parameters to achieve high-precision motion and force control. During object grasping, the robot can adjust the joint angles and rotation speeds to control the robotic arm's trajectory, ensuring it accurately grasps the target object. Furthermore, the robot can use torque parameters for force control, ensuring that the robotic arm does not apply excessive force while grasping the target object, thus avoiding damage to the object or the robot itself.

[0078] Step Two: Target Object Position Recognition. A Kinect camera (Kinect vision sensor) is installed on the robot to enable real-time visual capture of the target object, acquiring its image and motion data. This data can be used by autonomous motion planning algorithms to determine the target object's position and orientation, thereby assisting the robotic arm in performing precise grasping operations.

[0079] First, point cloud data from the Kinect camera is acquired. The input point cloud data is then preprocessed, including downsampling, filtering, and outlier removal, to improve the efficiency and accuracy of subsequent calculations.

[0080] The PCL algorithm is used to find a plane based on point cloud data. Given point cloud data onto a "desktop," which refers to the plane fitted to the desktop in the Rviz 3D visualization platform, plane fitting is performed on the point cloud data to find the set of points in the point cloud that best fits the planar model, and this set is treated as a plane. The desktop can be represented as a normal vector. And a point P0, where the normal vector The plane is perpendicular to the plane, and P0 is any point on the plane; thus, a two-dimensional coordinate system can be obtained, and the point cloud data in three-dimensional space can be projected onto this plane to obtain the point cloud data on the plane.

[0081] Therefore, it is necessary to first solve for this normal vector. And point P0.

[0082] normal vector The specific solution process is as follows:

[0083] Suppose we have a point cloud dataset S = {P1, P2, ..., Pn} containing N points. N}, where each point P i All can be represented as (x i ,y i ,z iThese points can be viewed as vectors in three-dimensional space, i.e. To find the normal vector We need to first calculate the covariance matrix C of the point cloud dataset S. The covariance matrix C is calculated using the following formula:

[0084]

[0085] in, It is the average of all points, that is:

[0086]

[0087] Next, we need to find the eigenvectors and corresponding eigenvalues ​​of the covariance matrix C. Since the covariance matrix C is a symmetric matrix, the eigenvectors are orthogonal. We sort the eigenvectors according to their corresponding eigenvalues ​​from largest to smallest, and then select the k smallest eigenvectors (in this embodiment, k = 2). Finally, we can use these k eigenvectors to calculate the normal vector. The specific method involves averaging these feature vectors and then normalizing the result, i.e.:

[0088]

[0089] The specific solution process for point P0 is as follows:

[0090] We can choose any point in the point cloud dataset S. Let's say we choose point P1, which can be represented as (x1, y1, z1). Then, we can calculate the normal vector of the plane containing the target object. A point P1 = (x1, y1, z1) on a plane can be represented in homogeneous coordinate form. And P1 = [x1, y1, z1, 1] T For each point in the point cloud data, it is represented in homogeneous coordinate form q = [x, y, z, 1]. T Move point q along the normal vector Projecting this onto the plane, we obtain the coordinates of the projection of point q' onto the plane:

[0091]

[0092] in,

[0093] We obtain the non-homogeneous coordinate form q'=[x',y',z'] T Then, the non-homogeneous coordinates of all projected points are combined into a point cloud dataset on a plane, thereby transforming the point cloud data in three-dimensional space into point cloud data on a plane.

[0094] Next, the location of the target object can be identified using the normal vector obtained from the plane fitting and the points on the plane. Object detection algorithms provided by OpenCV are used for object detection, and the center point coordinates of the target object are calculated using the Euclidean clustering algorithm. The point coordinates (x, y, y) of the target object are then extracted through point cloud data processing. i ,y i ,z i Initialize cluster center C k We can randomly select k points as the initial cluster centers, and then calculate P for each point. i To each cluster center C k distance d ik The formula is as follows:

[0095]

[0096] Then, the calculated distance d ik Point P i Assign the child to the cluster containing the nearest cluster centroid, calculate the average of all coordinates within that cluster, and then determine the center coordinates of that cluster.

[0097]

[0098]

[0099]

[0100] Where, n k This represents the number of points in cluster k.

[0101] Repeatedly calculate d ik , The value is returned until it no longer changes, and the center point coordinates of the cluster are returned as the center point coordinates of the target object.

[0102] The specific process for calculating the pose of the target object is as follows:

[0103] Perform eigenvalue decomposition on the covariance matrix C to obtain eigenvalues ​​λ1, λ2, λ3 and corresponding eigenvectors v1, v2, v3. Then, point P... i Projecting these features onto the plane formed by feature vectors v1 and v2 yields a two-dimensional point cloud data Q = {q1, q2, ..., q...}. n},in:

[0104]

[0105] Based on the properties of the PCA algorithm, the principal direction vector v1 and the normal vector on the plane... Since they are orthogonal, we can take another vector v2 on the plane as one component of this vector, while the other dimension is 0, thus obtaining the vector in three-dimensional space:

[0106] v3 = [v1[0], v1[1], 0] (10)

[0107] Where v1 is the principal direction vector calculated by the PCA algorithm.

[0108] Next, the vector v1 is rotated back to the original coordinate system. Let the rotation matrix be R, which rotates the basis vectors [v2, v3] on the plane back to the basis vectors [x, y, z] in the original coordinate system. Then, R is:

[0109]

[0110] The pose information of the target object is calculated based on the center point coordinates and the v1 vector.

[0111] Specifically, let the coordinates of the center point of the target object be c, and the attitude vector be v, then we have:

[0112] v=[v1[0],v1[1],0]*R (12)

[0113] Where * denotes the product of a vector and a matrix;

[0114] Based on the properties of vectors, the orientation vector of the target object is represented as:

[0115] o=[c[0]+v[0],c[1]+v[1],c[2]+v[2]]-c (13)

[0116] Where o is the orientation vector of the target object, and c is the coordinate of the center point of the target object; in this way, the pose information of the target object can be calculated from the point cloud data.

[0117] Step 3: Perform Cartesian path planning for the robot's robotic arm. Cartesian motion is achieved through interpolation and a PID controller. Interpolation is the process of fitting data points in a dataset to a function, thereby predicting the future values ​​of the data. Here, the position, velocity, and acceleration of the robotic arm are first treated as a dataset, and they are fitted to a function so that the robot's robotic arm can perform continuous motion in Cartesian space. First, cubic spline interpolation is used to fit the position, velocity, and acceleration of the robot's robotic arm, thereby generating a continuous second-order polynomial between adjacent data points.

[0118] Use the following formula to calculate the spline interpolation function:

[0119] Si (x)=a i +b i (xx i )+c i (xx i ) 2 +d i (xx i ) 3 (14)

[0120] Among them, S i (x) is in x i and x i+1 spline interpolation function between, a i It is the position of the i-th data point, b i c is the velocity of the i-th data point. i d is the acceleration of the i-th data point. i It is the rate of change of velocity at the i-th data point. These parameters can be calculated using the values ​​of adjacent data points and the derivative. Using the spline interpolation function, we can calculate the position, velocity, and acceleration of the robotic arm at any time point.

[0121] Then, a PID controller is used to control the movement of the robotic arm. The PID controller calculates the control output by comparing the actual output with the desired output, using the following formula:

[0122]

[0123] Where u(t) is the control output, K p K i and K d These are the parameters of the PID controller, and e(t) is the error signal. It is the rate of change of the error signal. It is the integral of the error signal. The position, velocity, and acceleration of the robotic arm are taken as the desired output, and the actual output is set as the current state of the robotic arm. Then, the error signal is calculated, and a PID controller is used to calculate the control output.

[0124] Finally, the control output is sent to the PID controller of the robotic arm to execute the desired motion. The motion execution process is as follows: 1. Obtain the target point on the path from the request; 2. Calculate the intermediate points on the path using an interpolation method so that the robotic arm's end effector can move smoothly; 3. Execute the PID controller at each intermediate point on the path to control the speed and acceleration of the robotic arm, making it move along the path;

[0125] Step 4: Grasp the target object

[0126] After completing the path planning of the robotic arm, the robot uses its robotic arm to grasp the target object. First, the posture of the robotic arm's end effector is determined, i.e., the end effector is opened. Using the target object position information obtained in step S1, the end effector is moved to the grasping position of the target object. When the end effector reaches the grasping position, based on the posture of the end effector and the posture information of the target object, the robot controls the end effector to grasp the target object. This process also needs to consider factors such as grasping force and grasping speed to ensure that the robotic arm can successfully grasp the target object, and a grasping stability test is performed to ensure that the target object has been firmly grasped by the robotic arm. Then, the end effector is moved to the target position. Finally, the target object is released, i.e., the end effector is opened to release the target object from the end effector, and then the robotic arm returns to its initial posture, completing the target object grasping task.

[0127] In this embodiment, the end effector of the robot's right robotic arm, as shown in the MatPlot diagram, progresses from its initial pose, grasping pose, pose during the grasping of the target object, to its position after placing the target object at its destination. Figure 2 As shown.

[0128] It should be noted that, as Figure 3 The diagram shown illustrates the change in the robotic arm's value function in this embodiment. The robotic arm's value function design in this embodiment includes task completion rate (measuring the success rate of the robotic arm in performing the object grasping task) and time efficiency (the time required for the robotic arm to perform the object grasping task), thereby evaluating the effectiveness of the robotic arm's actions. The value function is defined as follows:

[0129] V(s)=α*P_success(s)+β*P_efficiency(s) (16)

[0130] Where V(s) represents the value of state s, P_success(s) represents the probability of completing the object grasping task in state s, P_success(s) can be calculated based on visual detection algorithms, grasping strategies, etc., and P_efficiency(s) represents the time efficiency of the robotic arm in performing the grasping task in state s. Assuming the time required to complete the task is T, it can be defined as follows:

[0131] P_efficiency(s)=1 / T (17)

[0132] Among them, T can be measured and calculated based on the actual execution time, while α and β are parameters that balance task completion and time efficiency.

[0133] In summary, the object grasping method based on visual feedback control described in the above embodiments acquires the position and posture information of the target object in real time through the Kinect visual sensor, feeds this information back to the controller, and then controls the robot's robotic arm to achieve stable grasping of the target object and transfer it to the destination, thereby improving the robot's grasping ability and intelligence in complex environments.

[0134] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for object grasping based on visual feedback control, characterized in that, Includes the following steps: S1: Acquisition of target object position and orientation information A vision camera is installed on the robot to acquire point cloud data using a vision sensor. The point cloud data is preprocessed, and the PCL algorithm is used to find a plane based on the point cloud data. Given point cloud data to the plane, the point cloud data in three-dimensional space is transformed into point cloud data on the plane. The normal vector after plane fitting and the points on the plane are used to identify the position of the target object and obtain the coordinates of the center point of the target object, i.e. the position information of the target object. Then, the pose information of the target object is calculated based on the coordinates of the center point of the target object. S2: Robotic Arm Path Planning Cartesian path planning is performed on the robot's robotic arm. Inverse kinematics is used to calculate the joint angles of the robotic arm so that the end effector of the robotic arm can reach the target object. Then, a PID controller is used to control the movement of the end effector of the robotic arm so that the end effector of the robotic arm moves to the target object. S3: Target Object Grabbing After completing the path planning of the robotic arm, the robot uses its robotic arm to grasp the target object. First, the posture of the robotic arm's end effector is determined, i.e., the end effector is opened. Using the target object position information obtained in step S1, the end effector is moved to the grasping position of the target object. When the end effector reaches the grasping position, based on the posture of the end effector and the posture information of the target object, the robot controls the end effector to grasp the target object. Then, the end effector is moved to the target position. Finally, the target object is released, i.e., the end effector is opened to release the target object from the end effector, and then the robotic arm returns to its initial posture, completing the target object grasping task. In step S1, the specific process of converting point cloud data in three-dimensional space into point cloud data on a plane is as follows: S111: Select any point in the point cloud dataset S. Let the selected point be... ,but It can be represented as The normal vector of the plane containing the target object A point on the flat surface Represented in homogeneous coordinate form and ; S112: For each point in the point cloud data, represent it in homogeneous coordinate form. , will point Along the normal vector Projecting onto a plane, thus obtaining a point. Coordinates of the projection point on the plane: ； in, , ; S113: Obtain the nonhomogeneous coordinate form Then, the non-homogeneous coordinates of all the projected points are combined into a point cloud dataset on the plane, thereby transforming the point cloud data in three-dimensional space into point cloud data on the plane. In step S1, the specific process of calculating the coordinates of the center point of the target object is as follows: S121: Extract the point coordinates of the target object through point cloud data processing. Initialize cluster centers Random selection These points are used as the initial cluster centers; S122: Calculate each point To each cluster center distance The formula is as follows: ； S122: Distance calculated , will point Assign the child to the cluster containing the nearest cluster centroid, calculate the average of all coordinates within that cluster, and then determine the center coordinates of that cluster. : ； ； ； in, Cluster The number of points in; S123: Repeated calculations , The value is returned until it no longer changes, and the center point coordinates of the cluster are returned as the center point coordinates of the target object. In step S1, the specific process of calculating the pose information of the target object based on the center point coordinates is as follows: S131: Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalues. and the corresponding feature vectors are ; S132: Point Projected onto feature vector A two-dimensional point cloud data is obtained on the resulting plane. ,in ; S133: According to the properties of the PCA algorithm, the principal direction vector on the plane... and normal vector Orthogonal, take another vector on the plane As one component of this vector, and the other dimension being 0, we obtain the vector in three-dimensional space: ； in, It is the principal direction vector calculated by the PCA algorithm; S134: Will The vectors are rotated back to their original coordinate system, where the rotation matrix is ​​R, which transforms the basis vectors on the plane... Rotate back to the basis vectors in the original coordinate system Then R is obtained as: ； S135: Based on the center point coordinates of the target object and Vector calculations yield the pose information of the target object, and the calculation method is as follows: S1351: Let the coordinates of the center point of the target object be... The attitude vector is Then we have: ； Where * denotes the product of a vector and a matrix; S1352: Based on the properties of vectors, the orientation vector of the target object is represented as follows: ； in, Let be the orientation vector of the target object. The coordinates of the center point of the target object; This allows us to obtain the orientation information of the target object.

2. The object grasping method based on visual feedback control according to claim 1, characterized in that: In step S1, the preprocessing methods for point cloud data include downsampling, filtering, and outlier removal.

3. The object grasping method based on visual feedback control according to claim 1, characterized in that: In step S1, the plane is represented by a normal vector. and a point , where the normal vector Perpendicular to the plane It is any point on the plane.

4. The object grasping method based on visual feedback control according to claim 3, characterized in that: normal vector The specific solution process is as follows: S101: Suppose we have a point cloud dataset S={ containing N points. }, where each point All can be represented as In this form, points in the point cloud dataset S are considered as vectors in three-dimensional space, i.e. ; S102: Calculate the covariance matrix C of the point cloud dataset S, using the following formula: ； in, It is the average of all points, that is: ； S103: Find the eigenvectors and corresponding eigenvalues ​​of the covariance matrix C, sort the eigenvectors in descending order of their corresponding eigenvalues, and then select the smallest one. 1 feature vector, using eigenvectors are used to calculate the normal vector. normal vector as follows: 。 5. The object grasping method based on visual feedback control according to claim 4, characterized in that: In step S2, the specific processing procedure is as follows: S21: Use cubic spline interpolation to fit the position, velocity, and acceleration of the robotic arm, thereby generating a continuous second-order polynomial between adjacent data points; S22: A PID controller is used to control the movement of the robotic arm. The PID controller calculates the control output by comparing the actual output with the desired output. The position, speed and acceleration of the robotic arm are taken as the desired output, and the actual output is set as the current state of the robotic arm. Then the error signal is calculated and the PID controller is used to calculate the control output. S23: Send the control output to the PID controller of the robotic arm to execute the desired motion, causing the end effector of the robotic arm to move to the target object.

6. The object grasping method based on visual feedback control according to claim 5, characterized in that: In step S21, the spline interpolation function is calculated using the following formula: ； in, Is and spline interpolation function between them It is the first The location of each data point It is the first The speed of each data point It is the first The acceleration of each data point It is the first The rate of change of velocity at each data point is used to calculate the position, velocity, and acceleration of the robotic arm at any given time point using a spline interpolation function.

7. The object grasping method based on visual feedback control according to claim 5, characterized in that: In step S22, the control output is calculated using the following formula: ； in, It controls the output. , and These are parameters of the PID controller. It is an error signal. It is the rate of change of the error signal. It is the integral of the error signal.

Images