A multi-degree-of-freedom mechanical arm vision recognition and grabbing method

Abstract

The application relates to the technical field of mechanical arm control, and discloses a visual identification and grabbing method based on a multi-degree-of-freedom mechanical arm, which comprises the following steps: acquiring point cloud data of a target object by using a depth camera, screening the point cloud data to determine target feature points, converting the target feature points into mechanical arm base coordinate system pose data, solving optimal joint angles, acquiring end pose data fed back by the depth camera in real time, calculating joint angle correction amounts based on a preset formula and the end pose data, outputting control signals through a PID controller to control a seven-degree-of-freedom carbon fiber mechanical arm to grab the target object, and verifying whether the seven-degree-of-freedom carbon fiber mechanical arm successfully grabs the target object through point cloud collision detection. Through implementation of the application, the problem that the visual grabbing of the mechanical arm has significant deficiencies in flexibility, visual perception robustness, control coordination precision and grabbing verification reliability and is difficult to adapt to high-precision and high-stability grabbing requirements in a complex environment is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mechanical arm control, and particularly relates to a visual recognition and grabbing method based on a multi-degree-of-freedom mechanical arm. BACKGROUND

[0002] With the rapid development of industrial automation and intelligent manufacturing, the mechanical arm, as the core equipment of flexible production, has evolved from traditional fixed trajectory operation to integrated intelligent grabbing. Among them, the introduction of visual guidance technology is the key to promoting this change. Through the acquisition of three-dimensional information of the target object by a depth camera, the mechanical arm can break through the limitations of the preset trajectory and adapt to the dynamically changing grabbing scene.

[0003] Traditional mechanical arms are mostly designed with four or six degrees of freedom, and their kinematic models are simple and their inverse solutions are unique. However, the lack of redundant degrees of freedom leads to insufficient obstacle avoidance capability and posture flexibility. In a cluttered environment, if there are obstacles around the target object, such as supports beside the assembly line or stacked parts, such mechanical arms often have to give up grabbing because they cannot adjust the posture of the intermediate joints. At the same time, the connecting rods of such mechanical arms are mostly made of metal, and the high motion inertia not only limits the dynamic response speed, but also exacerbates joint wear and reduces the service life of the equipment. In terms of visual recognition, most rely on the target features of "specific color + regular shape", and achieve positioning through simple color threshold segmentation and edge detection. However, when facing transparent objects, shiny surfaces or irregular shapes, the recognition accuracy decreases significantly, and even the target is lost. This defect is particularly prominent in scenes involving diversified materials such as consumer electronics and food processing. In addition, most systems use an open-loop control mode of "one-time coordinate transformation", that is, visual data is only used for initial positioning, and the end pose drift caused by connecting rod deformation and load changes during the motion of the mechanical arm is not considered. The final grabbing error often exceeds ±5mm, which is difficult to meet the high-precision requirements of precision assembly and other high-precision requirements.

[0004] In addition, the conversion of the coordinate system of the ordinary depth camera and the base coordinate system of the mechanical arm mostly relies on static calibration. However, in long-term operation, the slight loosening of the camera support and the vibration of the mechanical arm base will cause the calibration parameters to drift, causing the coordinate transformation error to accumulate. At the same time, the motion planning algorithm fails to fully utilize the real-time nature of visual feedback. The inverse kinematics solution of the four-degree-of-freedom mechanical arm only focuses on whether the end pose meets the requirements, while ignoring the optimization space that can be excavated by the redundant degrees of freedom. Even if some six-degree-of-freedom mechanical arms introduce redundant control, their optimization targets are mostly limited to a single indicator, and a multi-objective balance of "pose accuracy-energy consumption-joint protection" has not been formed. In actual application, this will cause the mechanical arm to excessively rotate the joints to avoid obstacles when grabbing, which not only increases energy consumption, but also may cause the joints to be in extreme positions for a long time, shortening the service life.

[0005] In summary, the related mechanical arm visual grasping technology has significant deficiencies in flexibility of degrees of freedom, robustness of visual perception, control coordination accuracy, and reliability of grasping verification, and is difficult to adapt to high-precision and high-stability grasping requirements in complex environments. SUMMARY

[0006] Therefore, the application provides a multi-degree-of-freedom mechanical arm visual recognition grasping method to solve the problem that the related mechanical arm visual grasping technology has significant deficiencies in flexibility of degrees of freedom, robustness of visual perception, control coordination accuracy, and reliability of grasping verification, and is difficult to adapt to high-precision and high-stability grasping requirements in complex environments.

[0007] The application provides a multi-degree-of-freedom mechanical arm visual recognition grasping method, which is applied to a multi-degree-of-freedom mechanical arm, and the multi-degree-of-freedom mechanical arm comprises a seven-degree-of-freedom carbon fiber mechanical arm, a depth camera, and a controller. The depth camera is installed on a base of the seven-degree-of-freedom carbon fiber mechanical arm, and the method comprises the following steps: acquiring point cloud data of a target object by using the depth camera, screening the point cloud data to determine target feature points, converting the target feature points into base coordinate system pose data of the mechanical arm, solving optimal joint angles based on the coordinate system pose data, a preset multi-target optimization model, and an obstacle avoidance constraint condition, acquiring end pose data fed back by the depth camera in real time, calculating a joint angle correction amount based on a preset formula and the end pose data, outputting a control signal by using a PID controller based on the optimal joint angles and the joint angle correction amount, controlling the seven-degree-of-freedom carbon fiber mechanical arm to grasp the target object, and verifying whether the seven-degree-of-freedom carbon fiber mechanical arm successfully grasps the target object by point cloud collision detection.

[0008] The embodiment provides a multi-degree-of-freedom mechanical arm vision recognition and grabbing method, first, point cloud data of a target object is acquired by using a depth camera, target feature points are determined by screening the point cloud data, the semantic type of each point is accurately judged by combining a semantic segmentation model, and through preset confidence threshold and mechanical arm workspace depth range constraint, the effective information truly related to the grabbing task is stripped from the mass point cloud of the complex scene. Breakthrough the limitation of traditional visual recognition which only relies on color or simple shape features, even if facing transparent objects, shiny surfaces or irregular shaped target objects, the target can also be locked through the spatial distribution and reflection intensity characteristics of three-dimensional point cloud. Secondly, the target feature points are converted into mechanical arm base coordinate system pose data, realizing accurate mapping from visual perception space to mechanical arm motion space. The conversion matrix obtained based on the calibration board strictly maps the camera coordinate system coordinates output by the depth camera to the mechanical arm base coordinate system, eliminating the spatial deviation between different sensors. Through solving the optimal joint angle based on the coordinate system pose data, the preset multi-target optimization model and the obstacle avoidance constraint condition, the redundancy characteristics of the seven-degree-of-freedom mechanical arm are fully utilized. The multi-target optimization model does not pursue single end pose accuracy, but considers pose accuracy, motion energy consumption and joint limit protection: through the weight coefficient, the priority of different targets is balanced, while ensuring that the end can accurately reach the target position, the intermediate joint attitude is adjusted to avoid obstacles by using the redundant freedom, and the joint motion energy consumption is minimized, and the joint is prevented from being in the limit position for a long time to avoid aggravating wear. Compared with the simple trajectory generation of traditional few-degree-of-freedom mechanical arms, it can adapt to more complex environments and prolong the service life of the mechanical arm, thereby realizing the coordinated optimization of flexibility, safety and economy. Through acquiring the end pose data fed back by the depth camera in real time, the joint angle correction amount is calculated based on a preset formula, and a dynamic closed-loop control mechanism is constructed. In the process of mechanical arm movement, the actual pose of the end is continuously monitored by using the depth camera, the deviation matrix is generated by comparing the theoretical planning pose, and the pose deviation is converted into the joint angle correction amount through the Jacobian matrix. Real-time correction can dynamically compensate the pose drift of the mechanical arm caused by link deformation, load change or environmental interference, so as to ensure that the end effector can accurately track the target position, and the problem of error accumulation in traditional open-loop control is avoided. Then, the PID controller outputs the control signal based on the optimal joint angle and the joint angle correction amount, realizing the smoothness and stability of the mechanical arm movement. Through the synergistic effect of proportional, integral and differential coefficients, the PID control quickly responds to the joint angle correction demand: the proportional term adjusts the deviation immediately, the integral term eliminates the accumulated error, and the differential term suppresses the action overshoot. It ensures that the mechanical arm moves smoothly when adjusting the attitude or approaching the target object, avoids the target object from sliding or the mechanical arm from vibrating caused by sudden movement, and improves the reliability of the grabbing execution.The success of grabbing the target object is verified through point cloud collision detection, objective and quantitative grabbing state judgment criteria are established, the point clouds of the end effector and the target object are segmented, the minimum contact distance and the contact area ratio of the two are calculated, and whether a stable clamping is formed is accurately judged from the three-dimensional space angle. When the contact distance is less than the set threshold and the contact area ratio meets the requirement, it can be determined that the grabbing is successful, otherwise the retry mechanism is triggered. The "virtual covering" and "actual clamping" are effectively distinguished, the misjudgment rate is reduced, and the overall grabbing success rate is further improved through the closed-loop verification and retry mechanism, and the demand for manual intervention is reduced. Through the implementation of the application, the problems that the related mechanical arm visual grabbing technology has significant deficiencies in flexibility, visual perception robustness, control coordination accuracy and grabbing verification reliability, and is difficult to adapt to the high-precision and high-stability grabbing requirements in complex environments are solved.

[0009] In an optional implementation, point cloud data of the target object is acquired by using a depth camera, and target feature points are determined by screening the point cloud data, including:

[0010] The depth camera collects point cloud data P of the target object, , Pi is the i-th point, is a three-dimensional coordinate, is a reflection intensity,

[0011] The point cloud data is input into a semantic segmentation model to obtain a semantic probability of each point The semantic type corresponding to the semantic probability includes: target object type, background type and obstacle type.

[0012] The target feature points are determined by screening the point cloud data based on a screening formula, and the screening formula is as follows:

[0013] (1),

[0014] Wherein, represents the target feature point, represents a confidence threshold, and the value range is [0.6, 0.8], represents the depth range of the working space of the seven-degree-of-freedom carbon fiber mechanical arm.

[0015] In an optional implementation, the target feature points are converted into mechanical arm base coordinate system pose data, and the conversion formula is as follows:

[0016] ,

[0017] Wherein, represents the coordinates of the target feature point in the camera coordinate system, which is directly output by the depth camera, represents the coordinates of the converted target feature point in the mechanical arm base coordinate system, The transformation matrix is characterized by a rotation matrix R and a translation vector t obtained by calibrating the calibration plate.

[0018] In an optional embodiment, in the preset multi-objective optimization model,

[0019] The optimization objective function is as follows:

[0020] ,

[0021] wherein the weight coefficient ;

[0022] The sub-objective is defined as follows:

[0023] The pose accuracy , wherein is the expected end pose matrix, is the actual pose matrix, is the F-norm;

[0024] The energy consumption , wherein is the joint energy consumption coefficient of the kth joint, which is positively correlated with the joint inertia, is the joint angular velocity of the kth joint;

[0025] The joint limit penalty , wherein is the safety threshold, is the maximum limit angle of the kth joint, is the attenuation factor, and the penalty is triggered when .

[0026] In an optional embodiment, the obstacle avoidance constraint condition is as follows:

[0027] (4)

[0028] wherein is the shortest distance between the link of the seven-degree-of-freedom carbon fiber robot arm and the obstacle, is the safety distance.

[0029] In an optional embodiment, based on the coordinate system pose data, the preset multi-objective optimization model and the obstacle avoidance constraint condition, the optimal joint angle is solved, comprising:

[0030] The gradient descent method is used to iteratively solve the optimal joint angle , and the formula is as follows:

[0031] (5)

[0032] wherein is the learning rate, To optimize the gradient of the objective function at , iterate to convergence.

[0033] In an optional embodiment, the end pose data fed back by the depth camera in real time is acquired, and the joint angle correction amount is calculated based on a preset formula and the end pose data, comprising:

[0034] The end pose data fed back by the depth camera in real time is acquired, and it is assumed that the actual pose of the end detected by the depth camera at is , is a 4x4 matrix, and the theoretical pose of inverse kinematics planning is , and the deviation matrix is established as follows:

[0035] (6)

[0036] wherein, is the inverse matrix of the theoretical pose;

[0037] The mapping relationship between the joint angle velocity and the end velocity is described by the Jacobian matrix of the robot arm , the pose deviation is converted into the joint angle correction amount, and the conversion formula is as follows:

[0038] (7)

[0039] wherein, is a 6x7 matrix, is the pseudo-inverse of the Jacobian, is a regularization parameter for avoiding matrix singularity, is the vectorization operation of the deviation matrix, which is used to convert the 4x4 matrix into a 16x1 vector, and the first 6 elements correspond to the translation and rotation deviation.

[0040] In an optional embodiment, based on the optimal joint angle and the joint angle correction amount, a control signal is output by a PID controller to control the seven-degree-of-freedom carbon fiber robot arm to grasp the target object, comprising:

[0041] The corrected joint angle is , and the control signal output by the PID controller is as follows:

[0042]

[0043] wherein, is a proportional coefficient, is an integral coefficient, is a differential coefficient, , The integral operation is performed on the joint angle deviation.

[0044] In an optional embodiment, whether the target object is successfully grabbed is verified by point cloud collision detection, including:

[0045] After the grabbing action is completed, point cloud data of the current scene is collected ;

[0046] Wherein, P grasp is a set of point cloud data of the current scene collected by the depth camera, is an index of each point in the point cloud data, is a three-dimensional coordinate of the mth point in the camera coordinate system is the reflection intensity of the mth point;

[0047] The point cloud data P is denoised and ROI cropped, and the end effector point cloud P is segmented ;

[0048] The minimum contact distance d between the end effector point cloud P and the target object point cloud P is calculated, and the formula is as follows:

[0049] (9)

[0050] Wherein, is any point in P , is any point in P , is the Euclidean distance calculation function;

[0051] The contact area ratio a between the end effector point cloud P and the target object point cloud P is calculated, and the formula is as follows:

[0052]

[0053] Wherein, is a contact determination threshold;

[0054] If a , it is determined that the grabbing is successful; otherwise, it is determined that the grabbing fails.

[0055] In an optional embodiment, the method further includes: if it is determined that the grabbing fails, triggering a retry mechanism to re-plan the grabbing point and the end effector pose. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the specific embodiments or the prior art of the present application, the drawings required to be used in the specific embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0057] Figure 1 is a flowchart of a multi-degree-of-freedom mechanical arm vision recognition and grabbing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] With the rapid development of industrial automation and intelligent manufacturing, the mechanical arm as the core equipment of flexible production has evolved from traditional fixed trajectory operation to integrated intelligent grabbing. Among them, the introduction of vision guided technology is the key to promote this change. Through the acquisition of three-dimensional information of the target object by the depth camera, the mechanical arm can break through the limitation of the preset trajectory and adapt to the dynamic change of the grabbing scene.

[0059] Traditional mechanical arms are mostly designed with four or six degrees of freedom, with simple kinematic model and unique inverse solution, but the lack of redundant freedom leads to insufficient obstacle avoidance ability and posture flexibility. In a cluttered environment, if there are obstacles around the target object, such as supports beside the assembly line and stacked parts, such mechanical arms often have to give up grabbing because they cannot adjust the posture of the intermediate joints; at the same time, the connecting rods of the mechanical arms are mostly made of metal, and the high motion inertia not only limits the dynamic response speed, but also aggravates the joint wear and reduces the service life of the equipment. In terms of visual recognition, it mostly depends on the target features of "specific color + regular shape", and realizes positioning through simple color threshold segmentation and edge detection, but when facing transparent objects, shiny surfaces or irregular shapes, the recognition accuracy decreases significantly, and even the target is lost, which is particularly prominent in scenes involving diversified materials such as consumer electronics and food processing. In addition, most systems use an open-loop control mode of "one-time coordinate transformation", that is, the vision data is only used for initial positioning, and the end pose drift caused by connecting rod deformation and load change during the movement of the mechanical arm is not considered, and the final grabbing error often exceeds ±5mm, which is difficult to meet the high-precision requirements of precision assembly and other high-precision requirements.

[0060] Moreover, the conversion between the common depth camera coordinate system and the robot base coordinate system relies on static calibration, but in long-term operation, the slight loosening of the camera support and the vibration of the robot base will cause the calibration parameters to drift, resulting in the accumulation of coordinate conversion errors. At the same time, the motion planning algorithm fails to fully utilize the real-time nature of visual feedback, and the inverse kinematics solution of the four-degree-of-freedom robot only focuses on whether the end position meets the standard, while ignoring the optimization space that can be excavated from the redundant degrees of freedom. Even if some six-degree-of-freedom robots introduce redundant control, their optimization objectives are mostly limited to a single indicator, and a multi-objective balance of "position accuracy-energy consumption-joint protection" has not been formed. In practical applications, this can cause the robot to excessively rotate the joints to avoid obstacles when grasping, not only increasing energy consumption, but also possibly causing the joints to be in extreme positions for a long time, shortening the service life.

[0061] In summary, the related robot visual grasping technology has significant shortcomings in flexibility, visual perception robustness, control coordination accuracy, and grasping verification reliability, making it difficult to meet the high-precision and high-stability grasping requirements in complex environments.

[0062] The embodiment provides a multi-degree-of-freedom mechanical arm vision recognition and grabbing method, first, point cloud data of a target object is acquired by using a depth camera, target feature points are determined by screening the point cloud data, the semantic type of each point is accurately judged by combining a semantic segmentation model, and through preset confidence threshold and mechanical arm workspace depth range constraint, the effective information truly related to the grabbing task is stripped from the mass point cloud of the complex scene. Breakthrough the limitation of traditional visual recognition which only relies on color or simple shape features, even if facing transparent objects, shiny surfaces or irregular shaped target objects, the target can also be locked through the spatial distribution and reflection intensity characteristics of three-dimensional point cloud. Secondly, the target feature points are converted into mechanical arm base coordinate system pose data, realizing accurate mapping from visual perception space to mechanical arm motion space. The conversion matrix obtained based on the calibration board strictly maps the camera coordinate system coordinates output by the depth camera to the mechanical arm base coordinate system, eliminating the spatial deviation between different sensors. Through solving the optimal joint angle based on the coordinate system pose data, the preset multi-target optimization model and the obstacle avoidance constraint condition, the redundancy characteristics of the seven-degree-of-freedom mechanical arm are fully utilized. The multi-target optimization model does not pursue single end pose accuracy, but considers pose accuracy, motion energy consumption and joint limit protection: through the weight coefficient, the priority of different targets is balanced, while ensuring that the end can accurately reach the target position, the intermediate joint attitude is adjusted to avoid obstacles by using the redundant freedom, and the joint motion energy consumption is minimized, and the joint is prevented from being in the limit position for a long time to avoid aggravating wear. Compared with the simple trajectory generation of traditional few-degree-of-freedom mechanical arms, it can adapt to more complex environments and prolong the service life of the mechanical arm, thereby realizing the coordinated optimization of flexibility, safety and economy. Through acquiring the end pose data fed back by the depth camera in real time, the joint angle correction amount is calculated based on a preset formula, and a dynamic closed-loop control mechanism is constructed. In the process of mechanical arm movement, the actual pose of the end is continuously monitored by using the depth camera, the deviation matrix is generated by comparing the theoretical planning pose, and the pose deviation is converted into the joint angle correction amount through the Jacobian matrix. Real-time correction can dynamically compensate the pose drift of the mechanical arm caused by link deformation, load change or environmental interference, so as to ensure that the end effector can accurately track the target position, and the problem of error accumulation in traditional open-loop control is avoided. Then, the PID controller outputs the control signal based on the optimal joint angle and the joint angle correction amount, realizing the smoothness and stability of the mechanical arm movement. Through the synergistic effect of proportional, integral and differential coefficients, the PID control quickly responds to the joint angle correction demand: the proportional term adjusts the deviation immediately, the integral term eliminates the accumulated error, and the differential term suppresses the action overshoot. It ensures that the mechanical arm moves smoothly when adjusting the attitude or approaching the target object, avoids the target object from sliding or the mechanical arm from vibrating caused by sudden movement, and improves the reliability of the grabbing execution.The success of grabbing the target object is verified through point cloud collision detection, objective and quantitative grabbing state judgment criteria are established, the point clouds of the end effector and the target object are segmented, the minimum contact distance and the contact area ratio of the two are calculated, and whether a stable clamping is formed is accurately judged from the three-dimensional space. When the contact distance is less than the set threshold and the contact area ratio meets the requirements, it can be determined that the grabbing is successful, otherwise the retry mechanism is triggered. The "virtual cover" and "actual clamping" are effectively distinguished, the misjudgment rate is reduced, and the overall grabbing success rate is further improved through the closed-loop verification and retry mechanism, and the demand for manual intervention is reduced. Through the implementation of the application, the problems that the related mechanical arm visual grabbing technology has significant deficiencies in flexibility, visual perception robustness, control coordination accuracy and grabbing verification reliability, and is difficult to adapt to the high-precision and high-stability grabbing requirements in complex environments are solved.

[0063] According to the embodiment of the application, a multi-degree-of-freedom mechanical arm visual recognition grabbing method is provided. It should be noted that the steps shown in the flowchart of the drawing can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0064] In this embodiment, a multi-degree-of-freedom mechanical arm visual recognition grabbing method is provided, which is applied to an intelligent low-voltage electrical system, the intelligent low-voltage electrical system includes an image sensor module, a communication module, an intelligent chip module and a low-voltage electrical module, Figure 1 is a flowchart of a multi-degree-of-freedom mechanical arm visual recognition grabbing method according to the embodiment of the application, as Figure 1 shown, the flowchart includes the following steps:

[0065] Step S101, acquiring point cloud data of the target object by using a depth camera, and filtering the point cloud data to determine target feature points.

[0066] Specifically, the above step S101 includes:

[0067] Step S1011, the depth camera collects point cloud data P of the target object, , is the i-th point, is the three-dimensional coordinate, is the reflection intensity.

[0068] Further, the depth camera emits a detection signal to the space where the target object is located based on active or passive optical principles and receives the reflected information, and then generates point cloud data P containing spatial position and surface characteristics. The point cloud data P here completely records the scene information in the form of discrete points: represents the i-th spatial sampling point, Precise description of the three-dimensional coordinates of the point under the camera coordinate system is used to locate the spatial position of the target object; As a reflection intensity parameter, it reflects the reflection characteristics of the target object surface to the detection signal. For example, the reflection intensity of a metal surface is usually higher than that of a non-metal, and there is a difference in reflection intensity between smooth and rough surfaces.

[0069] Step S1012, input the point cloud data into the semantic segmentation model to obtain the semantic probability of each point The semantic type corresponding to the semantic probability includes: target object type, background type and obstacle type.

[0070] Further, the semantic segmentation model relies on a deep learning algorithm to classify and infer each point Output semantic probability Map the point cloud data to semantic categories such as "target object type, background type, and obstacle type". Break through the dependence on single features such as color and shape in traditional visual recognition. Even if the target object is obscured, the surface is reflective, or the shape is irregular, the semantic segmentation model can capture the global features and local correlations of the point cloud by learning a large amount of labeled data to determine the probability of each point belonging to the target object.

[0071] Step S103, screening the point cloud data based on the screening formula to determine the target feature points, the screening formula is as follows:

[0072] (1)

[0073] Wherein, characterizes the target feature point, characterizes the confidence threshold, the value range is [0.6, 0.8], characterizes the depth range of the working space of the seven-degree-of-freedom carbon fiber mechanical arm.

[0074] Further, to accurately extract and grasp the effective information related to the task from the complex scene, the preliminary processed point cloud data needs to be screened again based on the screening formula. The screening formula A double constraint mechanism is constructed: on the one hand, the semantic probability constraint Ensure that the selected points have high confidence target object attributes, representing the maximum value of the semantic probability in the current point cloud data, multiplied by the confidence threshold , The value is 0.6-0.8, which balances the risk of missed detection and false detection, and is used as a semantic threshold for screening to filter out background, obstacles and other interference points; on the other hand, the depth range constraint By combining the physical workspace of the seven-DOF carbon fiber robotic arm, the depth range of the target feature points is defined, eliminating point clouds that the robotic arm cannot reach or that are outside its effective working range, thus ensuring the feasibility of subsequent motion planning. Through this filtering process, the raw point cloud data is refined into a set of points containing only the key features of the target object. .

[0075] Step S102: Convert the target feature points into pose data of the robot arm's base coordinate system.

[0076] Specifically, the target feature points are converted into pose data in the robot arm's base coordinate system. The conversion formula is as follows:

[0077]

[0078] in, The coordinates of the target feature points in the camera coordinate system are directly output by the depth camera. The coordinates of the target feature points in the robot arm's base coordinate system after the representation transformation. The transformation matrix is ​​characterized by being obtained through calibration using a calibration plate and includes the rotation matrix R and the translation vector t.

[0079] Furthermore, the core of establishing the spatial correlation between visual perception and robotic arm motion control is to achieve coordinate system unification. The coordinates of target feature points output by the depth camera are based on the camera's own coordinate system, while the robotic arm's motion planning needs to be performed in its base coordinate system. Due to differences in installation position and orientation, there is an inherent deviation in spatial pose between the two. To eliminate this deviation, a transformation matrix needs to be introduced to establish a mapping relationship from the camera coordinate system to the robotic arm's base coordinate system through homogeneous coordinate transformation. The construction of the transformation matrix includes two key dimensions: rotation and translation. The rotation matrix describes the pose difference between the camera coordinate system and the robotic arm's base coordinate system. For example, when the camera is installed at an angle, the rotation matrix needs to compensate for the angle deviation. The translation vector quantifies the spatial position difference between the origins of the two coordinate systems, reflecting the physical distance between the camera and the robotic arm base. The matrix is ​​obtained through calibration using a calibration board: a calibration board with known geometric dimensions and spatial position is placed in the working scene. The depth camera collects the camera coordinates of the feature points on the calibration board in the camera coordinate system. Combined with the theoretical coordinates in the robotic arm's base coordinate system, the matrix is ​​solved using algorithms such as the least squares method. The optimal parameters are determined to ensure the accuracy of coordinate transformation. In the actual transformation process, the mathematical properties of homogeneous coordinates are utilized to extend the three-dimensional coordinates to four-dimensional homogeneous coordinates, completing the rotation and translation transformations in one step through matrix multiplication. Essentially, this operation transforms the target feature points from a spatial description from the "camera's perspective" to pose data from the "robotic arm's perspective," enabling the robotic arm to understand the spatial position and orientation of the target object based on its own kinematic model, providing a unified spatial reference for subsequent inverse kinematics solutions and trajectory planning.

[0080] In step S103, the optimal joint angle is solved based on the coordinate system pose data, the preset multi-objective optimization model, and the obstacle avoidance constraint condition.

[0081] Specifically, in the preset multi-objective optimization model,

[0082] The optimization objective function is as follows:

[0083]

[0084] wherein the weight coefficient ;

[0085] The sub-objective is defined as follows:

[0086] The pose accuracy , wherein is an expected end pose matrix, is an actual pose matrix, is an F-norm;

[0087] The energy consumption , wherein is an energy consumption coefficient of the kth joint, which is positively correlated with the joint inertia, is an angular velocity of the kth joint;

[0088] The joint limit penalty , wherein is a safety threshold, is a maximum limit angle of the kth joint, is a decay factor, and the penalty is triggered when .

[0089] Further, the core of planning the motion of a seven-degree-of-freedom carbon fiber robot arm from the current state to the target pose is to solve the optimal joint angle that satisfies multiple constraints. The seven-degree-of-freedom robot arm has more joints than the six degrees of freedom required to complete the end pose, and has redundant degrees of freedom. The inverse kinematics solution is not unique, which provides space for multi-objective optimization. On the basis of meeting the end pose requirements, additional targets such as energy consumption and joint protection are considered to improve the rationality of the motion and the service life of the robot arm. The preset multi-objective optimization model constructs a comprehensive optimization objective function that includes pose accuracy, motion energy consumption, and joint limit protection. The priority of different targets is balanced by the weight coefficient: the pose accuracy sub-objective e quantifies the accuracy of the end effector reaching the target position and attitude by calculating the F-norm of the expected end pose matrix and the actual pose matrix. The F-norm comprehensively considers the overall deviation of the matrix elements, ensuring that the end pose approximates the target in both position and attitude, which is the basic constraint for the robot arm to complete the grasping task. The motion energy consumption sub-objective E: the motion energy consumption of the robot arm is positively correlated with the square of the joint angular velocity and the joint inertia, and is calculated by Modeling. Among them, is the energy consumption coefficient of the kth joint, the larger the joint inertia is, the larger the energy consumption coefficient is, is the joint angular velocity. This sub-target encourages the robot arm to move in a "low energy consumption" mode, avoiding energy waste and mechanical wear caused by high-speed rotation or frequent start-stop of the joint. The joint limit penalty sub-target is to prevent the joint from being in the limit position for a long time, and introduces . Among them, is a safety threshold, reserving a 5° buffer space, is a decay factor, when the joint angle , the penalty term grows exponentially, forcing the optimization algorithm to adjust the joint angle away from the limit position.

[0090] Specifically, the obstacle avoidance constraint condition is as follows:

[0091] (4),

[0092] Among them, is the shortest distance between the link of the seven-degree-of-freedom carbon fiber robot arm and the obstacle, is a safety distance.

[0093] Further, the obstacle avoidance constraint condition further guarantees the safety of the movement, by calculating the shortest distance between the link of the robot arm and the environmental obstacle, to ensure that it is greater than the safety distance, to avoid collision during movement.

[0094] Specifically, the above step S103 includes:

[0095] The gradient descent method is used to iteratively solve the optimal joint angle , and the formula is as follows:

[0096] (5)

[0097] Among them, is the learning rate, is the gradient of the optimization objective function at , and the iteration is converged to .

[0098] Further, to solve this multi-constraint optimization problem, the gradient descent method is used to iteratively search for the optimal joint angle: starting from the initial joint angle , the gradient of the objective function is calculated, and the joint angle is gradually updated in the negative direction of the gradient with a learning rate , until the norm of the gradient converges, and the optimal joint angle under the multiple constraints of "pose accuracy, reasonable energy consumption, joint safety, and obstacle avoidance" is finally obtained, providing direct instructions for the movement control of the robot arm.

[0099] Step S104, acquiring the end position data fed back by the depth camera in real time, calculating the joint angle correction amount based on a preset formula and the end position data.

[0100] Specifically, the above step S104 includes:

[0101] Step S1041, acquiring the end position data fed back by the depth camera in real time, assuming that the depth camera detects the actual end position as at time t, and the theoretical position as , is a 4×4 matrix, and the theoretical position calculated by inverse kinematics planning is , and the deviation matrix is established as follows:

[0102] (6)

[0103] wherein is the inverse matrix of the theoretical position.

[0104] Further, the depth camera continuously monitors the actual position of the end of the robot arm, and at any time , the actual end position matrix is acquired, which contains translation and rotation information. Meanwhile, according to the initial motion planning, the theoretical position matrix is output by the inverse kinematics algorithm. To quantify the deviation between the actual position and the theoretical position, the deviation matrix is constructed by matrix operation: wherein is the inverse matrix of the theoretical position matrix, which is used to “reverse convert” the theoretical position to the reference system of the actual position, so that the deviation matrix can reflect the translation error and rotation error at the same time. The “difference between the actual position and the theoretical position” is converted into a calculable matrix form, which provides a basis for subsequent correction of the joint angle.

[0105] Step S1042, the mapping relationship between the joint angle velocity and the end velocity is described by the Jacobian matrix of the robot arm, the position deviation is converted into the joint angle correction amount, and the conversion formula is as follows:

[0106] (7)

[0107] wherein is a 6×7 matrix, is the pseudo-inverse of the Jacobian, is a regularization parameter used to avoid matrix singularity, is the vectorization operation of the deviation matrix, which is used to convert the 4×4 matrix into a 16×1 vector, and the first 6 elements correspond to the translation and rotation deviation.

[0108] Further, the kinematic characteristics of the mechanical arm are described by the Jacobian matrix The mapping relationship between the joint angular velocity and the end line velocity, angular velocity is established, which reflects the influence of small joint movement on the end pose. To convert the pose deviation into the correction amount of joint angle, the pseudo-inverse of Jacobian matrix is calculated , wherein, is a regularization parameter, which is used to avoid matrix singularity. When the Jacobian matrix is full rank, direct inversion is susceptible to noise, and regularization can enhance the robustness of the algorithm. Then, through vectorization operation convert the 4x4 deviation matrix into a 16x1 vector, extract the first 6 elements, which correspond to the linear representation of translation and rotation deviation, and substitute them into the formula , so as to obtain the joint angle correction amount . By using the "inverse mapping" of the Jacobian matrix, the end pose deviation is decomposed into each joint, and the joint angle adjustment amount that minimizes the end pose deviation is calculated, so as to realize dynamic closed-loop correction. That is, even if the mechanical arm deviates from the theoretical trajectory due to external interference during movement, the joint angle can be quickly adjusted through real-time feedback to ensure that the end effector always approaches the target pose.

[0109] In step S105, based on the optimal joint angle and the joint angle correction amount, a control signal is output through a PID controller to control the seven-degree-of-freedom carbon fiber mechanical arm to grasp the target object.

[0110] Specifically, the above step S105 includes:

[0111] The corrected joint angle is , and the control signal output by the PID controller is as follows:

[0112] , wherein, is a proportional coefficient, is an integral coefficient, is a differential coefficient, , The integral operation is performed on the joint angle deviation.

[0113] Further, converting the joint angle command into a control signal for driving the mechanical arm to move relies on the PID controller to achieve precise and stable motion control, ensuring that the mechanical arm completes the grasping action according to the expected trajectory. First, the corrected joint angle command is determined. The movement of the mechanical arm is determined by the initial planned optimal joint angle and the real-time closed-loop correction amount, that is, . This superposition process integrates global optimal planning and local real-time correction: ensures the overall rationality of the mechanical arm movement, Then, for small deviations in the movement process, such as mechanical arm deformation, environmental interference, dynamic compensation is carried out to ensure that the end effector always approaches the target pose. Subsequently, the PID controller outputs the control signal for driving the joint movement according to the corrected joint angle command . The core logic of PID control is to realize precise regulation of joint movement through the synergistic effect of proportional P, integral I, and derivative D terms: the proportional term According to the current joint angle deviation Directly output the control quantity, quickly respond to the deviation, and make the joint angle approach the target value. The proportional coefficient determines the "sensitivity" of the response, and the larger the value, the faster the correction speed of the deviation, but too large may cause system oscillation. The integral term integrates the joint angle deviation, accumulates the historical deviation and outputs the control quantity, which is used to eliminate "static deviation", such as continuous small deviation caused by friction. The integral coefficient is moderately set, too small cannot effectively eliminate static deviation, and too large may cause integral saturation and overshoot. The derivative term outputs the control quantity based on the rate of change of the joint angle deviation, predicts the development trend of the deviation and adjusts in advance to suppress system oscillation and improve stability. The derivative coefficient can enhance the damping characteristics of the system to avoid motion overshoot caused by proportional and integral effects. Through the formula , the PID controller converts the joint angle deviation into a continuous control signal to drive the joint motor of the seven-degree-of-freedom carbon fiber robot to move. In this process, the lightweight characteristics of the carbon fiber robot, low inertia, high rigidity, and the dynamic adjustment capability of the PID control are combined to ensure the rapid response of the robot movement, adapt to the dynamic grabbing demand, and ensure the stability of the movement to avoid target object slipping due to vibration, and finally realize precise and stable grabbing of the target object.

[0114] Step S106, verify whether the seven-degree-of-freedom carbon fiber robot successfully grabs the target object through point cloud collision detection.

[0115] Specifically, the above step S106 includes:

[0116] Step S1061, after the grabbing action is completed, the point cloud data of the current scene is collected ,

[0117] where P grasp is the current scene point cloud data set collected by the depth camera, is the index of each point in the point cloud data, is the three-dimensional coordinates of the mth point in the camera coordinate system is the reflection intensity of the mth point.

[0118] Further, after a short delay of the completion of the grasping action, to avoid the inertial motion of the robotic arm affecting the quality of the point cloud, the depth camera again collects the point cloud data of the current scene. Unlike the initial point cloud collection, at this time the scene contains the interactive state of "end effector + target object": The spatial coordinates and reflection intensity of both are covered. The metal material of the end effector and the surface characteristics of the target object can be distinguished by , providing complete three-dimensional data of the "post-grasping state" for subsequent collision detection.

[0119] Step S1062, the point cloud data is denoised and ROI cropped, and the end effector point cloud and the target object point cloud are segmented.

[0120] Further, the collected is denoised and ROI cropped, and only the spatial range where the end effector and the target object are located is retained, reducing environmental point cloud interference. Through semantic segmentation or geometric feature matching, such as using the CAD model of the end effector and the pre-screened features of the target object, the preprocessed point cloud is segmented into end effector point cloud and target object point cloud . This segmentation process distinguishes the spatial features and material features of the two, for example, the regular geometric shape of the end effector, such as the parallel structure of the gripper, and the uniform reflection intensity, can be distinguished from the irregular shape and diverse reflection characteristics of the target object, providing clear point cloud subsets for subsequent contact analysis.

[0121] Step S1063, the minimum contact distance between the end effector point cloud and the target object point cloud is calculated, and the formula is as follows:

[0122] (9)

[0123] wherein is any point in , is any point in , is the Euclidean distance calculation function.

[0124] Further, the point pairs in are traversed , and the closest distance between the two is calculated by the Euclidean distance formula, reflecting the physical contact closeness between the end effector and the target object. If tends to 0, it indicates that there is direct contact; if the distance is too large, it may not be grasped or the grasp may be loose.

[0125] Step S1064: Calculate the end effector point cloud. Point cloud of target object Contact area ratio The formula is as follows:

[0126]

[0127] in, This is the contact determination threshold.

[0128] Furthermore, statistical end effector point cloud Point cloud of target object The number of points that satisfy the condition that distance is ≤ contact threshold, and The ratio of the total points, i.e. This quantifies the contact coverage area between the end effector and the target object. The higher the proportion, the more sufficient the contact and the more stable the grip. For example, the larger the contact area between the gripper and the target object, the stronger the grip.

[0129] Step S1065, if and If the data is successfully scraped, it is considered scraped successfully; otherwise, it is considered scraped unsuccessfully.

[0130] Furthermore, by combining the two quantitative indicators mentioned above, a dual judgment condition is set: successful grasping is determined, otherwise grasping is determined to have failed. This avoids both spurious contact, such as the end effector only slightly touching the target object, and partial contact, such as only the gripper tip touching, which could easily cause the target object to slip, ensuring that the judgment result is highly consistent with the actual grasping state.

[0131] In one alternative implementation, the method further includes: if the grasping is determined to have failed, triggering a retry mechanism to replan the grasping point and the end effector posture.

[0132] Furthermore, if the failure is due to a small contact area, the algorithm will prioritize "flat areas" or "areas near the geometric center" of the target object's surface, thereby increasing the contact range between the end effector and the target object and improving performance. If the failure is due to gripping point misalignment, the algorithm will combine the real-time changes in point cloud data to recalculate the centroid or feature point coordinates of the target object and adjust the spatial position of the gripping point to ensure accurate alignment of the end effector. If the target object is a planar object, the end effector's posture is constrained to be parallel to the surface normal vector of the target object to ensure that the gripper and the target object have a large area of ​​contact; if the target object is a cylindrical object, the end effector's posture is constrained to be perpendicular to the cylinder axis to improve gripping stability.

[0133] The embodiment provides a multi-degree-of-freedom mechanical arm vision recognition and grabbing method, first, point cloud data of a target object is acquired by using a depth camera, target feature points are determined by screening the point cloud data, the semantic type of each point is accurately judged by combining a semantic segmentation model, and through preset confidence threshold and mechanical arm workspace depth range constraint, the effective information truly related to the grabbing task is stripped from the mass point cloud of the complex scene. Breakthrough the limitation of traditional visual recognition which only relies on color or simple shape features, even if facing transparent objects, shiny surfaces or irregular shaped target objects, the target can also be locked through the spatial distribution and reflection intensity characteristics of three-dimensional point cloud. Secondly, the target feature points are converted into mechanical arm base coordinate system pose data, realizing accurate mapping from visual perception space to mechanical arm motion space. The conversion matrix obtained based on the calibration board strictly maps the camera coordinate system coordinates output by the depth camera to the mechanical arm base coordinate system, eliminating the spatial deviation between different sensors. Through solving the optimal joint angle based on the coordinate system pose data, the preset multi-target optimization model and the obstacle avoidance constraint condition, the redundancy characteristics of the seven-degree-of-freedom mechanical arm are fully utilized. The multi-target optimization model does not pursue single end pose accuracy, but considers pose accuracy, motion energy consumption and joint limit protection: through the weight coefficient, the priority of different targets is balanced, while ensuring that the end can accurately reach the target position, the intermediate joint attitude is adjusted to avoid obstacles by using the redundant freedom, and the joint motion energy consumption is minimized, and the joint is prevented from being in the limit position for a long time to avoid aggravating wear. Compared with the simple trajectory generation of traditional few-degree-of-freedom mechanical arms, it can adapt to more complex environments and prolong the service life of the mechanical arm, thereby realizing the coordinated optimization of flexibility, safety and economy. Through acquiring the end pose data fed back by the depth camera in real time, the joint angle correction amount is calculated based on a preset formula, and a dynamic closed-loop control mechanism is constructed. In the process of mechanical arm movement, the actual pose of the end is continuously monitored by using the depth camera, the deviation matrix is generated by comparing the theoretical planning pose, and the pose deviation is converted into the joint angle correction amount through the Jacobian matrix. Real-time correction can dynamically compensate the pose drift of the mechanical arm caused by link deformation, load change or environmental interference, so as to ensure that the end effector can accurately track the target position, and the problem of error accumulation in traditional open-loop control is avoided. Then, the PID controller outputs the control signal based on the optimal joint angle and the joint angle correction amount, realizing the smoothness and stability of the mechanical arm movement. Through the synergistic effect of proportional, integral and differential coefficients, the PID control quickly responds to the joint angle correction demand: the proportional term adjusts the deviation immediately, the integral term eliminates the accumulated error, and the differential term suppresses the action overshoot. It ensures that the mechanical arm moves smoothly when adjusting the attitude or approaching the target object, avoids the target object from sliding or the mechanical arm from vibrating caused by sudden movement, and improves the reliability of the grabbing execution.Whether the target object is successfully grabbed is verified through point cloud collision detection, an objective and quantitative judgment standard of the grabbing state is established, the point clouds of the end effector and the target object are segmented, the minimum contact distance and the contact area ratio of the two are calculated, and whether a stable clamping is formed is accurately judged from the perspective of three-dimensional space. When the contact distance is less than the set threshold and the contact area ratio meets the requirement, it can be determined that the grabbing is successful, otherwise the retry mechanism is triggered. The "virtual covering" and "actual clamping" are effectively distinguished, the misjudgment rate is reduced, and the overall success rate of grabbing is further improved through the closed-loop verification and retry mechanism, thereby reducing the need for manual intervention. Through the implementation of the present application, the problems that the related mechanical arm visual grabbing technology has significant deficiencies in flexibility, visual perception robustness, control coordination precision and grabbing verification reliability, and is difficult to adapt to the high-precision and high-stability grabbing requirements in complex environments are solved.

[0134] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A vision recognition and grasping method based on a multi-degree-of-freedom robotic arm, applied to a multi-degree-of-freedom robotic arm, the multi-degree-of-freedom robotic arm comprising a seven-degree-of-freedom carbon fiber robotic arm, a depth camera, and a controller, wherein the depth camera is mounted on the base of the seven-degree-of-freedom carbon fiber robotic arm, characterized in that, The method includes: Point cloud data of the target object is acquired using a depth camera, and the point cloud data is then filtered to determine the target feature points; The target feature points are converted into pose data in the robot arm's base coordinate system. Based on coordinate system pose data, a preset multi-objective optimization model, and obstacle avoidance constraints, the optimal joint angle is solved. Acquire the end-effector pose data in real time from the depth camera, and calculate the joint angle correction based on the preset formula and the end-effector pose data; Based on the optimal joint angle and joint angle correction, the PID controller outputs a control signal to control the seven-degree-of-freedom carbon fiber robotic arm to grasp the target object. Point cloud collision detection was used to verify whether the seven-DOF carbon fiber robotic arm successfully grasped the target object. The step of acquiring point cloud data of the target object using a depth camera and filtering the point cloud data to determine target feature points includes: The depth camera acquires point cloud data P of the target object. , For the i-th point, In three-dimensional coordinates, Reflection intensity, Input the point cloud data into the semantic segmentation model to obtain the semantic probability of each point. The semantic types corresponding to the semantic probabilities include: target type, background type, and obstacle type; The target feature points are determined by filtering the point cloud data based on a filtering formula, which is as follows: ， in, Characterize the target feature points, The confidence threshold is defined, with a value ranging from [0.6, 0.8]. Characterizes the working space depth range of the seven-degree-of-freedom carbon fiber robotic arm; The target feature points are converted into pose data in the robot arm's base coordinate system using the following formula: ， in, The coordinates of the target feature points in the camera coordinate system are directly output by the depth camera. The coordinates of the target feature points in the robot arm's base coordinate system after the representation transformation. The transformation matrix, obtained through calibration using a calibration plate, contains the rotation matrix R and the translation vector t. In the preset multi-objective optimization model The objective function is optimized as follows: ， Among them, the weighting coefficient ; The sub-goals are defined as follows: Pose accuracy ,in, Let be the desired end pose matrix. This is the actual pose matrix. It is the F-norm; Energy consumption ,in, Let be the joint energy dissipation coefficient of the k-th joint, which is positively correlated with the joint inertia. Let be the joint angular velocity of the k-th joint; Joint Limit Punishment ,in, As a safety threshold, Let be the maximum limiting angle of the k-th joint. As the attenuation factor, when The penalty will be triggered at that time.

2. The method according to claim 1, characterized in that, The obstacle avoidance constraints are as follows: ， in, This represents the shortest distance between the link of the seven-DOF carbon fiber robotic arm and the obstacle. For safe distance.

3. The method according to claim 2, characterized in that, The process of solving for the optimal joint angle based on coordinate system pose data, a preset multi-objective optimization model, and obstacle avoidance constraints includes: The optimal joint angle is solved iteratively using the gradient descent method. The formula is as follows: ， in, For learning rate, To optimize the objective function in The gradient at point , iterated to convergence.

4. The method according to claim 3, characterized in that, The process of acquiring the end-effector pose data from the depth camera in real time, and calculating the joint angle correction based on a preset formula and the end-effector pose data, includes: To acquire the end-effector pose data from the depth camera in real time, let the depth camera be in... The actual pose of the end effector is detected at all times. , Given a 4×4 matrix, the theoretical pose for inverse kinematics programming is: The deviation matrix is ​​established as follows: ， in, The inverse matrix of the theoretical pose; Using the Jacobi matrix of the robotic arm The mapping relationship between joint angular velocity and end-effector velocity is described, and pose deviation is converted into joint angle correction. The conversion formula is as follows: ， in, It is a 6×7 matrix. For Jacobi's pseudo-rebellion, This is a regularization parameter used to avoid matrix singularities. This is a vectorization operation for the deviation matrix, used to convert a 4×4 matrix into a 16×1 vector, taking the first 6 elements corresponding to the translation and rotation deviations.

5. The method according to claim 4, characterized in that, The process involves using an optimal joint angle and joint angle correction, along with a PID controller to output control signals to control the seven-degree-of-freedom carbon fiber robotic arm to grasp the target object. This includes: The corrected joint angle is The control signal output by the PID controller is as follows: ， in, For proportionality coefficient, For integral coefficients, The differential coefficients are... , Perform an integral calculation on the joint angle deviation.

6. The method according to claim 5, characterized in that, The verification of whether the target object was successfully captured through point cloud collision detection includes: After the grasping action is completed, collect the point cloud data of the current scene. ; Among them, P grasp This is a collection of point cloud data of the current scene acquired by a depth camera. , is the index of each point in the point cloud data. Let m be the three-dimensional coordinates of the m-th point in the camera coordinate system. Let m be the reflection intensity at the m-th point; Point cloud data After noise reduction and ROI cropping, the end effector point cloud was obtained. Point cloud of target object ; Computational end effector point cloud Point cloud of target object Minimum contact distance The formula is as follows: ， in, for any point in, for any point in, The function for calculating Euclidean distance; Computational end effector point cloud Point cloud of target object Contact area ratio The formula is as follows: ， in, The contact detection threshold; like If the data is successfully scraped, it is considered scraped successfully; otherwise, it is considered scraped unsuccessfully.

7. The method according to claim 6, characterized in that, The method further includes: If the grabbing fails, a retry mechanism is triggered to re-plan the grabbing point and the attitude of the end effector.

Images