Mechanical arm self-adaptive grading grabbing control method and system based on visual feedback

The adaptive hierarchical grasping control method for robotic arms using visual feedback dynamically allocates the weights of control components, solving the problem of control command jumps during the grasping process, achieving smooth transitions and efficient grasping, and improving system stability and compliance.

CN121989258AActive Publication Date: 2026-05-08ANHUI UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202610454778.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-04-08
Publication Date
2026-05-08
Estimated Expiration
2046-04-08

AI Technical Summary

Technical Problem

In existing technologies, robotic arms suffer from abrupt changes in control commands when switching from vision-dominant to force-dominant during the grasping process, resulting in unstable robotic arm movements.

Method used

An adaptive hierarchical grasping control method for robotic arms based on visual feedback is adopted. The target position information is obtained through a visual sensor and the contact state information is obtained through a force sensor. The weights of three control components are dynamically allocated, including a vision-dominated stage, a transition stage, and a force-dominated stage, to achieve the continuity and smooth transition of control commands.

Benefits of technology

It improves the stability and smoothness of the robotic arm's grasping process, reduces hardware costs, and eliminates the need for expensive depth cameras and high-resolution tactile sensors, thereby increasing the grasping success rate and compliance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121989258A_ABST
    Figure CN121989258A_ABST
Patent Text Reader

Abstract

The invention provides a mechanical arm self-adaptive grading grabbing control method and system based on visual feedback, and belongs to the technical field of robot control. The method comprises the steps that target position information is obtained through a visual sensor, and contact state information of the tail end of a mechanical arm and a target is obtained through a force sensor; according to the target position information and the contact state information, the current control stage of the mechanical arm is judged, and the control stage comprises the vision dominant stage that the mechanical arm approaches the target but does not make contact with the target, the force dominant stage that the mechanical arm grabs the target and the transition stage from the vision dominant stage to the force dominant stage; generating a first control component and a second control component based on the target position information, and generating a third control component based on the contact state information; according to the current control stage of the mechanical arm, the weights of the first control component, the second control component and the third control component are dynamically distributed and fused to generate a comprehensive control instruction; and the mechanical arm is driven to grab the target according to the comprehensive control instruction, and jumping of the control instruction is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to an adaptive hierarchical grasping control method and system for robotic arms based on visual feedback. Background Technology

[0002] Robotic arm grasping control is one of the core technologies for achieving automated operations. In practical applications, grasping control requires the comprehensive use of information from multiple sensors: vision sensors provide target position information to guide the robotic arm towards the target; force sensors provide contact state information to achieve compliant grasping. How to effectively integrate these two heterogeneous information systems is a key technical challenge for achieving intelligent grasping.

[0003] Existing technologies mainly employ the following two approaches to solve the above problems:

[0004] (1) Serial Usage Mode: This approach adopts a serial method of "vision first, then force control," that is, vision is used to guide the robotic arm to approach the target first, and then the force control mode is switched after the approach is completed. Although this serial method is simple to implement, it has obvious technical defects: there is an instantaneous switching between the two control modes, and the control command will change abruptly, resulting in unstable movement of the robotic arm and even impact vibration. Different control modes are regarded as mutually exclusive choices or either-or combinations, rather than as a cooperative relationship that can cooperate and dynamically adjust the degree of contribution. This concept limits the further improvement of control performance.

[0005] (2) Parallel usage mode: This approach attempts to use visual information and force information simultaneously for control decision-making, and generate comprehensive control commands through a certain fusion algorithm. However, visual information and force information have different physical meanings, different dimensions, and different update frequencies, making direct fusion difficult to achieve the desired effect. Furthermore, the tuning of fusion parameters is difficult, and the stability of the system is hard to guarantee. For example, in the Chinese invention patent application CN121157057A, entitled "A Dynamic Grasping Control Method for Robots Based on Visual-Tactile Deep Fusion," during the contact establishment phase, contact events are detected by tactile sensors. Visual and tactile data are fused to establish a coordinate system, a visual-tactile consistency score is calculated, and an initial grasping force is applied. During the stabilization phase, wavelet packet energy entropy and pressure gradient are used for slip detection, and adaptive impedance control is combined to dynamically adjust the grasping force and impedance parameters, achieving intelligent control throughout the entire process from approach and contact to stabilization. The core of this method lies in the deep fusion at the perception level. Through complex algorithms, the geometric features acquired visually are precisely registered with the pressure data acquired tactilely, solving the problem of "how to align the position seen by the eye and the force felt by the hand in the same coordinate system." To ensure accurate perception, this method requires expensive depth cameras and high-resolution tactile sensors, and has high computational complexity. Furthermore, to ensure the reliability of tracking, visual information is usually given a high or even constant weight throughout the process, which may lead to the inability to fully utilize force sensor information to achieve optimal compliant control during the close contact phase. Summary of the Invention

[0006] The technical problem to be solved by this invention is: how to solve the problem of command jump when the robotic arm switches from vision-driven to force-driven during the grasping process.

[0007] This invention solves the above-mentioned technical problems through the following technical solution: an adaptive hierarchical grasping control method for robotic arms based on visual feedback, comprising: The target position information is obtained through a vision sensor, and the contact state information between the end effector of the robotic arm and the target is obtained through a force sensor. Based on the target location information and contact status information, determine the current control stage of the robotic arm. The control stage includes the vision-dominated stage when the robotic arm approaches the target but does not make contact with the target, the force-dominated stage when the robotic arm grasps the target, and the transition stage from the vision-dominated stage to the force-dominated stage. The first control component and the second control component are generated based on the target position information, and the third control component is generated based on the contact state information. According to the current control stage of the robotic arm, the weights of the first control component, the second control component, and the third control component are dynamically allocated and fused to generate a comprehensive control command. The robotic arm is driven to grasp the target according to the integrated control commands.

[0008] This invention achieves continuous control commands and a smooth transition at the control level through gradual weighting rather than mode switching, avoiding abrupt changes in control commands and improving system stability and motion smoothness. It employs an architecture with three control components operating in parallel and dynamically varying weights. During the vision-dominated phase, vision contributes the most; during the transition phase, the visual weight gradually decreases while the force weight gradually increases; and during the force-dominated phase, the force weight is highest, but vision retains a low weight for monitoring. Control commands remain continuous without abrupt changes, resulting in smooth and stable robotic arm movement. Vision provides continuous monitoring throughout the entire process, allowing for timely intervention in case of anomalies.

[0009] Preferably, the method for determining the current control stage of the robotic arm based on target position information and contact state information is as follows: Based on the target location information, the distance between the target and the robotic arm is calculated; based on the contact state information, the contact force between the end effector of the robotic arm and the target is calculated. When the distance between the target and the robotic arm is greater than or equal to the distance threshold, the current control phase of the robotic arm is the vision-dominated phase; when the distance between the target and the robotic arm is less than the distance threshold and the contact force exceeds the first force threshold, the current control phase of the robotic arm changes from the vision-dominated phase to the transition phase; when the contact force exceeds the second force threshold, the current control phase of the robotic arm is the force-dominated phase, and the second force threshold is greater than the first force threshold.

[0010] Preferably, the process of generating a first control component and a second control component based on target location information, and generating a third control component based on contact state information, includes: An image-based visual servoing method is used to generate a first control component based on the distance between the target and the robotic arm and the deviation of the target distance. The target position information is converted into a spatial target position and a second control component is generated by solving inverse kinematics. An impedance control method is used to convert the deviation of the contact force between the robotic arm end and the target and the target force into a position correction amount, generating a third control component.

[0011] Preferably, the process of dynamically allocating the weights of the first control component, the second control component, and the third control component according to the current control stage of the robotic arm includes: When the current control phase of the robotic arm is vision-driven, the weight of the third control component is 0, and the weight of the first control component is... Weights of the second control component Keep constant, and ; When the current control phase of the robotic arm is a transition phase, the weight of the first control component... From weight Gradually reduce to weight The weight of the third control component Gradually increase the weight from 0. The weight of the second control component satisfy: , This refers to the time after entering the transition phase. , This represents the total duration of the transition phase. When the current control phase of the robotic arm is the force-dominated phase, the weight of the first control component... Weights of the second control component Weights of the third control component Keep constant, and .

[0012] Preferably, the weight of the first control component Weights of the second control component Satisfying Relationship: , Weight Weight satisfy: , .

[0013] Preferably, the weight of the first control component From weight Gradually decrease to the weight according to a linear or exponential relationship. The weight of the third control component The weight gradually increases from 0 according to a linear or exponential relationship. .

[0014] Preferably, when the current control phase of the robotic arm is vision-driven, the integrated control commands are... for:

[0015] When the current control phase of the robotic arm is a transition phase, the integrated control commands... for:

[0016] When the current control phase of the robotic arm is the force-dominated phase, the comprehensive control commands are... for:

[0017] in, , , These are the first control component, the second control component, and the third control component, respectively.

[0018] This invention also provides a vision-feedback-based adaptive hierarchical grasping control system for robotic arms, comprising: The perception module is used to acquire target position information through a vision sensor and to acquire contact state information between the end effector of the robotic arm and the target through a force sensor. The decision module is used to determine the current control stage of the robotic arm based on the target position information and contact status information. The control stages include the vision-dominated stage when the robotic arm approaches the target but does not make contact with the target, the force-dominated stage when the robotic arm grasps the target, and the transition stage from the vision-dominated stage to the force-dominated stage. The fusion control module is used to generate a first control component and a second control component based on the target position information, and a third control component based on the contact state information; according to the current control stage of the robotic arm, the weights of the first control component, the second control component, and the third control component are dynamically allocated and fused to generate a comprehensive control command; The execution module is used to drive the robotic arm to grasp the target according to the integrated control commands.

[0019] Preferably, the fusion control module includes: A visual servo controller is used to generate a first control component based on the distance between the target and the robotic arm and the deviation of the target distance, using an image-based visual servoing method. The position controller is used to convert the target position information into the spatial target position and generate the second control component by solving inverse kinematics. The force controller is used to convert the deviation between the contact force between the robotic arm end and the target and the target force into a position correction amount using an impedance control method, thereby generating a third control component.

[0020] Preferably, the execution module includes a servo driver, a motor, a robotic arm, and joints located on the robotic arm. The servo driver receives integrated control commands and converts them into current commands, which are then input to the motor. The motor transmits torque through the joint reducer to drive each joint to rotate, thereby driving the end effector of the robotic arm to grasp the target.

[0021] The advantages provided by this invention are: 1. This invention divides the entire grasping process into three stages: a vision-dominated stage, a transition stage, and a force-dominated stage. Based on the robotic arm's tasks in these three stages, the weights of the first, second, and third control components are dynamically allocated. The vision-dominated stage ensures target tracking capability, the force-dominated stage ensures contact compliance, and the transition stage balances both aspects, resulting in excellent overall dynamic performance and improved grasping success rate and compliance.

[0022] 2. Compared with the fusion method of perception layer in the existing technology, the present invention only requires 2D vision and ordinary force sensor, which significantly reduces hardware cost, simplifies calculation, and does not require expensive depth camera and high-resolution tactile sensor.

[0023] 3. Compared to existing technologies that assign high weight to visual information to ensure tracking reliability, this invention uses a lower visual weight during the force-dominated stage without reducing tracking reliability. This is because the source of "reliability" has fundamentally changed. The robotic arm has established physical contact with the target, and the force control closed loop itself has the ability to suppress external disturbances. As long as contact is not lost, the stability of the grip is guaranteed by force feedback. The retained lower visual weight is only used for anomaly monitoring and safety redundancy, such as detecting whether the target has abnormally shifted out of the field of view, or providing macroscopic guidance when it is necessary to move the object to another position. In other words, reliability no longer depends on "constantly staring," but on a dual mechanism of "tactile closed loop after physical contact + visual backup." The low visual weight is just the right amount of safety margin in this mechanism. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] Figure 1 A flowchart of the visual feedback-based adaptive hierarchical grasping control method for robotic arms provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram showing the changes in the weights of the three control components at each stage in the visual feedback-based adaptive hierarchical grasping control method for robotic arms provided in Embodiment 1 of the present invention. Figure 3 This is a schematic diagram of the adaptive hierarchical grasping control system for a robotic arm based on visual feedback provided in Embodiment 2 of the present invention; Figure 4 The control principle diagram of the adaptive hierarchical grasping control system for a robotic arm based on visual feedback provided in Embodiment 2 of the present invention is shown.

[0026] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Example 1 like Figure 1 As shown, this embodiment provides an adaptive hierarchical grasping control method for a robotic arm based on visual feedback, including the following steps: S1. Obtain target position information through a vision sensor and obtain contact state information between the end effector of the robotic arm and the target through a force sensor.

[0029] Among them, the vision sensor can provide two-dimensional position information of the target in the image plane, and the force sensor can provide force or torque information when the end of the robotic arm comes into contact with the target.

[0030] S2. Based on the target position information and contact status information, determine the current control stage of the robotic arm. The control stages include the vision-dominated stage when the robotic arm approaches the target but does not make contact with the target, the force-dominated stage when the robotic arm grasps the target, and the transition stage from the vision-dominated stage to the force-dominated stage.

[0031] The method for determining the current control stage of the robotic arm based on target location information and contact state information is as follows: Based on the target location information, the distance between the target and the robotic arm is calculated; based on the contact state information, the contact force between the end effector of the robotic arm and the target is calculated. When the distance between the target and the robotic arm is greater than or equal to the distance threshold, the current control phase of the robotic arm is the vision-dominated phase; when the distance between the target and the robotic arm is less than the distance threshold and the contact force exceeds the first force threshold, the current control phase of the robotic arm changes from the vision-dominated phase to the transition phase; when the contact force exceeds the second force threshold, the current control phase of the robotic arm is the force-dominated phase, and the second force threshold is greater than the first force threshold.

[0032] Using a multi-dimensional criterion consisting of the distance between the target and the robotic arm and the contact force between the robotic arm's end effector and the target, it can adaptively adjust according to the target characteristics, making it suitable for grasping targets of different types, sizes, and materials, thus improving environmental adaptability.

[0033] S3. Generate a first control component and a second control component based on the target position information, and generate a third control component based on the contact state information; dynamically allocate the weights of the first control component, the second control component, and the third control component according to the current control stage of the robotic arm, and fuse them to generate a comprehensive control command.

[0034] The process of generating the first and second control components based on the target location information, and generating the third control component based on the contact state information, includes: An image-based visual servoing method is used to generate a first control component based on the distance between the target and the robotic arm and the deviation of the target distance. The target position information is converted into a spatial target position and a second control component is generated by solving inverse kinematics. An impedance control method is used to convert the deviation of the contact force between the robotic arm end and the target and the target force into a position correction amount, generating a third control component.

[0035] See Figure 2 The process of dynamically allocating the weights of the first control component, the second control component, and the third control component based on the current control stage of the robotic arm includes: When the current control phase of the robotic arm is vision-driven, the weight of the third control component is 0, and the weight of the first control component is... Weights of the second control component Keep constant, and In this embodiment, the weight of the first control component Weights of the second control component Satisfying Relationship: , In the vision-driven phase, integrated control commands are implemented. for:

[0036] in, , , These are the first control component, the second control component, and the third control component, respectively.

[0037] When the current control phase of the robotic arm is a transition phase, the weight of the first control component... From weight Gradually reduce to weight The weight of the third control component Gradually increase the weight from 0. The weight of the second control component satisfy: , This refers to the time after entering the transition phase. , This is the total duration of the transition phase; during the transition phase, integrated control commands are issued. for:

[0038] In this embodiment, the weight of the first control component From weight Gradually decrease to the weight according to a linear or exponential relationship. The weight of the third control component The weight gradually increases from 0 according to a linear or exponential relationship. Linear gradients are simple, intuitive, and easy to implement; exponential gradients can be achieved by setting parameters. , Adjust the rate of change accordingly. Besides linear gradual change and exponential decay, other smoothing functions can be used to weight the first control component. Weights of the third control component Dynamic adjustments can be made, as long as the changes do not involve abrupt changes.

[0039] When the weight of the first control component From weight Gradually decrease to the weight according to the linear relationship. The weight of the third control component The weight is gradually increased from 0 according to a linear relationship. hour:

[0040]

[0041] When the weight of the first control component From weight Gradually decreasing to weight according to the exponential relationship. The weight of the third control component The weight gradually increases from 0 according to an exponential relationship. hour:

[0042]

[0043] in, , All are variable parameters, among which, Control the descent speed. Control the ascent speed, , .

[0044] When the current control phase of the robotic arm is the force-dominated phase, the weight of the first control component... Weights of the second control component Weights of the third control component Keep constant, and In this embodiment, the weights Weight satisfy: , During the force-dominated phase, integrated control commands are implemented. for:

[0045] S4. Drive the robotic arm to grasp the target according to the integrated control command.

[0046] In this invention, the first control component is generated based on the distance between the target and the robotic arm and the deviation of the target distance. Therefore, the first control component corresponds to the weight of vision control. The second control component is obtained by solving inverse kinematics. Therefore, the second control component corresponds to the weight of position control. The third control component is calculated based on the deviation of the contact force between the robotic arm end and the target and the target force. Therefore, the third control component corresponds to the weight of stress control.

[0047] This invention divides the entire grasping process into three stages: a vision-dominated stage, a transition stage, and a force-dominated stage. Based on the robotic arm's tasks in each stage, the weights of the first, second, and third control components are dynamically allocated. In the vision-dominated stage, the primary task is to quickly approach and track the target; therefore, in this stage, the first control component (visual control) has the highest weight, the second control component (position control) has a lower weight, and the third control component (force control) has zero weight. The robotic arm primarily adjusts its movement based on visual feedback to ensure the target remains within its field of vision.

[0048] When the robotic arm's current control phase transitions from a vision-dominated phase to a transition phase, the first, second, and third control components simultaneously participate in the control of the robotic arm, but their respective weights change dynamically. Specifically, the weight of the first control component (vision control) gradually decreases, the weight of the third control component (force control) begins to increase after the robotic arm's end effector contacts the target (i.e., entering the transition phase), and the weight of the second control component (position control) gradually increases to a certain value before gradually decreasing. By dynamically adjusting the weights of the three control components, the integrated control command gradually transitions from vision-dominated to force-dominated. Throughout this process, the integrated control command changes continuously without any abrupt transition points, thus avoiding the instantaneous command jumps when switching from the vision-dominated to the force-dominated phase and ensuring smooth and stable robotic arm movement.

[0049] Once the transition is complete, the robotic arm enters the force-dominated phase. Since the main task of the force-dominated phase is to achieve compliant contact and stable gripping, the third control component (force control) has the highest weight, the first control component (vision control) has a lower weight, and the second control component (position control) has a moderate weight. In this phase, the robotic arm mainly adjusts its movement based on force feedback to achieve a soft landing effect.

[0050] The vision-driven phase ensures target tracking capability, the force-driven phase ensures contact compliance, and the transition phase achieves a balance between the two, resulting in excellent overall dynamic performance and improved grasping success rate and compliance.

[0051] As can be seen, this invention does not select which control mode to use, but rather determines the contribution of each control mode. That is, it adopts a progressive control weight transfer strategy. At different stages of grasping, visual control (first control component), position control (second control component), and force control (third control component) coexist, but their respective weights are different. As the task progresses, the weight gradually shifts from visual dominance to force dominance, achieving seamless integration of multiple control modes.

[0052] This invention achieves continuity of control commands and a smooth transition at the control level through gradual weighting rather than mode switching, avoiding abrupt changes in control commands and improving system stability and motion smoothness. It employs an architecture with three control components operating in parallel and dynamically varying weights. During the vision-dominated phase, vision contributes the most; during the transition phase, the visual weight gradually decreases while the force weight gradually increases; and during the force-dominated phase, the force weight is highest, but vision retains a low weight for monitoring. Control commands remain continuous without abrupt changes, resulting in smooth and stable robotic arm movement. Vision provides continuous monitoring throughout the entire process, allowing for timely intervention in case of anomalies. Compared to existing fusion methods at the perception level, this invention requires only 2D vision and a common force sensor, significantly reducing hardware costs, simplifying computation, and eliminating the need for expensive depth cameras and high-resolution tactile sensors.

[0053] Furthermore, this invention can be implemented by modifying existing robotic arm platforms at the software level, without requiring special hardware customization, and is easy to promote and apply.

[0054] Example 2 See Figure 3 This embodiment provides a visual feedback-based adaptive hierarchical grasping control system for robotic arms, including: The perception module is used to acquire target position information through a vision sensor and to acquire contact state information between the robotic arm end effector and the target through a force sensor.

[0055] The decision module is used to determine the current control stage of the robotic arm based on the target position information and contact status information. The control stages include the vision-dominated stage when the robotic arm approaches the target but does not make contact with the target, the force-dominated stage when the robotic arm grasps the target, and the transition stage from the vision-dominated stage to the force-dominated stage.

[0056] The method for determining the current control stage of the robotic arm based on target location information and contact state information is as follows: Based on the target location information, the distance between the target and the robotic arm is calculated; based on the contact state information, the contact force between the end effector of the robotic arm and the target is calculated. When the distance between the target and the robotic arm is greater than or equal to the distance threshold, the current control phase of the robotic arm is the vision-dominated phase; when the distance between the target and the robotic arm is less than the distance threshold and the contact force exceeds the first force threshold, the current control phase of the robotic arm changes from the vision-dominated phase to the transition phase; when the contact force exceeds the second force threshold, the current control phase of the robotic arm is the force-dominated phase, and the second force threshold is greater than the first force threshold.

[0057] The fusion control module generates a first control component and a second control component based on the target position information, and a third control component based on the contact state information. It dynamically allocates weights to the first, second, and third control components according to the current control stage of the robotic arm, and fuses them to generate a comprehensive control command. (See also...) Figure 4 The fusion control module includes: A visual servo controller is used to generate a first control component based on the distance between the target and the robotic arm and the deviation of the target distance, thereby mapping the target position deviation in the image plane into motion commands for the robotic arm.

[0058] The position controller is used to convert the target position information into a spatial target position, and generate a second control component through inverse kinematics solution or trajectory planning method to generate a smooth motion trajectory from the current position to the target position.

[0059] A force controller is used to convert the deviation between the contact force between the end effector of the robotic arm and the target and the target force into a position correction quantity using impedance control or admittance control methods. This generates a third control component, which adjusts the movement of the robotic arm to achieve compliant contact.

[0060] The fusion control module also includes a weight scheduler, which dynamically allocates the weights of the first control component, the second control component, and the third control component according to the current control stage of the robotic arm. The specific process includes: When the current control phase of the robotic arm is vision-driven, the weight of the third control component is 0, and the weight of the first control component is... Weights of the second control component Keep constant, and ; When the current control phase of the robotic arm is a transition phase, the weight of the first control component... From weight Gradually reduce to weight The weight of the third control component Gradually increase the weight from 0. The weight of the second control component satisfy: , This refers to the time after entering the transition phase. , This represents the total duration of the transition phase. When the current control phase of the robotic arm is the force-dominated phase, the weight of the first control component... Weights of the second control component Weights of the third control component Keep constant, and .

[0061] Among them, the weight of the first control component Weights of the second control component Satisfying Relationship: , Weight Weight satisfy: , The weight of the first control component From weight Gradually decrease to the weight according to a linear or exponential relationship. The weight of the third control component The weight gradually increases from 0 according to a linear or exponential relationship. .

[0062] When the current control phase of the robotic arm is vision-driven, the integrated control commands are... for:

[0063] When the current control phase of the robotic arm is a transition phase, the integrated control commands... for:

[0064] When the current control phase of the robotic arm is the force-dominated phase, the comprehensive control commands are... for:

[0065] in, , , These are the first control component, the second control component, and the third control component, respectively.

[0066] The execution module is used to drive the robotic arm to grasp the target according to the integrated control commands. The execution module includes a servo driver, a motor, a robotic arm, and joints located on the robotic arm. The servo driver receives the integrated control commands and converts them into current commands, which are then input into the motor. The torque is transmitted through the joint reducers to drive the joints to rotate, thereby driving the end effector of the robotic arm to grasp the target.

[0067] Existing technologies use a threshold switching method. When the switching conditions are met, the control command instantly jumps from the output of one control mode to the output of another control mode, resulting in obvious command jumps and causing the robotic arm to move unsteadily.

[0068] This invention employs a weighted gradual change method, where control commands gradually transition from vision-driven to force-driven. The commands change continuously throughout the process without any abrupt transitions, resulting in smooth and stable movement of the robotic arm.

[0069] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visual feedback-based adaptive hierarchical grasping control method for robotic arms, characterized in that: include: The target position information is obtained through a vision sensor, and the contact state information between the end effector of the robotic arm and the target is obtained through a force sensor. Based on the target location information and contact status information, determine the current control stage of the robotic arm. The control stage includes the vision-dominated stage when the robotic arm approaches the target but does not make contact with the target, the force-dominated stage when the robotic arm grasps the target, and the transition stage from the vision-dominated stage to the force-dominated stage. The first control component and the second control component are generated based on the target location information, and the third control component is generated based on the contact state information. Based on the current control stage of the robotic arm, the weights of the first control component, the second control component, and the third control component are dynamically allocated and fused to generate a comprehensive control command; The robotic arm is driven to grasp the target according to the integrated control commands.

2. The adaptive hierarchical grasping control method for a robotic arm based on visual feedback according to claim 1, characterized in that: The method for determining the current control stage of the robotic arm based on target position information and contact status information is as follows: Based on the target location information, the distance between the target and the robotic arm is calculated; based on the contact state information, the contact force between the end effector of the robotic arm and the target is calculated. When the distance between the target and the robotic arm is greater than or equal to the distance threshold, the current control phase of the robotic arm is the vision-driven phase. When the distance between the target and the robotic arm is less than the distance threshold and the contact force exceeds the first force threshold, the current control phase of the robotic arm changes from the vision-dominated phase to the transition phase. When the contact force exceeds the second force threshold, the current control phase of the robotic arm becomes the force-dominated phase, and the second force threshold is greater than the first force threshold.

3. The adaptive hierarchical grasping control method for a robotic arm based on visual feedback according to claim 2, characterized in that: The process of generating a first control component and a second control component based on target location information, and generating a third control component based on contact state information, includes: An image-based visual servoing method is used to generate a first control component based on the distance between the target and the robotic arm and the deviation of the target distance. The target position information is converted into a spatial target position and a second control component is generated by solving inverse kinematics. An impedance control method is used to convert the deviation of the contact force between the robotic arm end and the target and the target force into a position correction amount, generating a third control component.

4. The adaptive hierarchical grasping control method for a robotic arm based on visual feedback according to claim 1, characterized in that: The process of dynamically allocating the weights of the first control component, the second control component, and the third control component based on the current control stage of the robotic arm includes: When the current control phase of the robotic arm is vision-driven, the weight of the third control component is 0, and the weight of the first control component is... Weights of the second control component Keep constant, and ; When the current control phase of the robotic arm is a transition phase, the weight of the first control component... From weight Gradually reduce to weight The weight of the third control component Gradually increase the weight from 0. The weight of the second control component satisfy: , This refers to the time after entering the transition phase. , This represents the total duration of the transition phase. When the current control phase of the robotic arm is the force-dominated phase, the weight of the first control component... Weights of the second control component Weights of the third control component Keep constant, and .

5. The adaptive hierarchical grasping control method for a robotic arm based on visual feedback according to claim 4, characterized in that: Weight of the first control component Weights of the second control component Satisfying Relationship: , ; Weight Weight satisfy: , .

6. The adaptive hierarchical grasping control method for a robotic arm based on visual feedback according to claim 4, characterized in that: Weight of the first control component From weight Gradually decrease to the weight according to a linear or exponential relationship. The weight of the third control component The weight gradually increases from 0 according to a linear or exponential relationship. .

7. The adaptive hierarchical grasping control method for a robotic arm based on visual feedback according to claim 4, characterized in that: When the current control phase of the robotic arm is vision-driven, the integrated control commands are... for: When the current control phase of the robotic arm is a transition phase, the integrated control commands... for: When the current control phase of the robotic arm is the force-dominated phase, the comprehensive control commands are... for: in, , , These are the first control component, the second control component, and the third control component, respectively.

8. A visual feedback-based adaptive hierarchical grasping control system for robotic arms, characterized in that: include: The perception module is used to acquire target position information through a vision sensor and to acquire contact state information between the end effector of the robotic arm and the target through a force sensor. The decision module is used to determine the current control stage of the robotic arm based on the target position information and contact status information. The control stages include the vision-dominated stage when the robotic arm approaches the target but does not make contact with the target, the force-dominated stage when the robotic arm grasps the target, and the transition stage from the vision-dominated stage to the force-dominated stage. The fusion control module is used to generate a first control component and a second control component based on the target position information, and to generate a third control component based on the contact state information. Based on the current control stage of the robotic arm, the weights of the first control component, the second control component, and the third control component are dynamically allocated and fused to generate a comprehensive control command; The execution module is used to drive the robotic arm to grasp the target according to the integrated control commands.

9. The adaptive hierarchical grasping control system for a robotic arm based on visual feedback according to claim 8, characterized in that: The fusion control module includes: A visual servo controller is used to generate a first control component based on the distance between the target and the robotic arm and the deviation of the target distance, using an image-based visual servoing method. The position controller is used to convert the target position information into the spatial target position and generate the second control component by solving inverse kinematics. The force controller is used to convert the deviation between the contact force between the robotic arm end and the target and the target force into a position correction amount using an impedance control method, thereby generating a third control component.

10. The adaptive hierarchical grasping control system for a robotic arm based on visual feedback according to claim 8, characterized in that: The execution module includes a servo driver, a motor, a robotic arm, and joints located on the robotic arm. The servo driver receives integrated control commands and converts them into current commands, which are then input to the motor. The motor transmits torque through the joint reducer to drive each joint to rotate, thereby driving the end effector of the robotic arm to grasp the target.

Citation Information

Patent Citations

  • Robot dynamic grabbing control method based on visual sense and tactile sense depth fusion

    CN121157057A

  • Mechanical arm control method and system with visual sense and force sense fused perception

    CN119772899A

  • Autonomous navigation and positioning method for semiconductor mechanical arm based on AI vision

    CN120606403A

  • Robot shaft hole assembly rotating axial decoupling and six-degree-of-freedom pose correction method, system and equipment and medium

    CN121374569A

  • Robot control device, robot system, robot, robot control method and program

    JP2015074058A