Manipulator grabbing control method and device based on position and posture recognition

Abstract

The invention discloses a manipulator grabbing control method and equipment based on position and posture recognition. The manipulator grabbing control method comprises the following steps that S1, initial image information of a target object is acquired through image acquisition equipment; s2, performing preprocessing and feature extraction on the initial image information, and obtaining three-dimensional position coordinates and attitude parameters of the target object through a preset position and attitude recognition algorithm; and S3, according to the three-dimensional position coordinates and the posture parameters, a grabbing path of the manipulator is planned by combining a kinematic model of the manipulator. By introducing technical means such as a deep learning recognition algorithm, forward / inverse kinematics model collaborative planning, real-time posture dynamic adjustment and force sensing feedback control, the problems that a traditional mechanical arm is low in grabbing precision, poor in adaptability and insufficient in operation stability are solved, intelligent and high-precision grabbing control in a complex scene is achieved, and the grabbing precision of the mechanical arm is improved. And the requirements of the modern industry on high efficiency, reliability and flexibility of automatic equipment are met.

Description

Technical Field

[0001] This invention relates to the field of mechanical control, specifically to a robotic gripper grasping control method and device based on position and posture recognition. Background Technology

[0002] In fields such as industrial automation and intelligent manufacturing, robotic arms, as crucial execution devices, directly impact production quality and operational efficiency through the precision, efficiency, and adaptability of their gripping control. Traditional robotic arm gripping control technologies generally suffer from the following limitations:

[0003] Insufficient target recognition accuracy: Traditional position and posture recognition often relies on a single sensor or simple algorithm. In scenarios with complex backgrounds, diverse object postures, or indistinct surface features, recognition errors are prone to occur, leading to inaccurate grasping and positioning.

[0004] Limited attitude adaptation capability: Most traditional solutions use preset paths or static attitude planning, which makes it difficult to respond to the small displacements and attitude changes of the target object in real time, resulting in a mismatch between the end effector and the object's attitude and a high failure rate in grasping.

[0005] Poor coordination between path planning and motion control: Some solutions do not fully integrate the kinematic characteristics of the robot for path optimization, which can easily lead to problems such as unreasonable motion trajectory, joint jamming, or excessive energy consumption, affecting operating efficiency and equipment life.

[0006] The gripping force control lacks precise feedback: Traditional gripping actions often rely on fixed driving force control, which cannot dynamically adjust the force according to the material and shape of the object. This can easily lead to damage to fragile objects or the loss of lightweight objects, resulting in poor adaptability.

[0007] The system has a low level of automation and intelligence: most solutions require manual intervention for path calibration or parameter setting, which is highly dependent on the experience of operators and is difficult to adapt to the flexible manufacturing needs of mass production and multi-product switching. Therefore, a robotic arm grasping control method and equipment based on position and posture recognition is proposed. Summary of the Invention

[0008] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0009] Step S1: Acquire initial image information of the target object using an image acquisition device;

[0010] Step S2: Preprocess and extract features from the initial image information, and obtain the three-dimensional position coordinates and pose parameters of the target object through a preset position and pose recognition algorithm;

[0011] Step S3: Based on the three-dimensional position coordinates and attitude parameters, and combined with the kinematic model of the robot, plan the grasping path of the robot.

[0012] Step S4: Control the robotic arm to move to the target position according to the grasping path;

[0013] Step S5: Adjust the attitude of the robot's end effector according to the attitude parameters to match the attitude of the target object;

[0014] Step S6: Control the end effector of the robotic arm to perform a grasping action and grasp the target object;

[0015] Step S7: After the grasping is completed, control the robotic arm to move the target object to the preset position and reset it.

[0016] Furthermore, the preprocessing in step S2 includes image denoising, image enhancement, and image segmentation, while feature extraction includes extracting the contour features, texture features, and geometric features of the target object.

[0017] Furthermore, the image segmentation employs a threshold-based segmentation algorithm or an edge detection segmentation algorithm to separate the target object from the background region.

[0018] Furthermore, the preset position and pose recognition algorithm in step S2 is a deep learning-based recognition algorithm, including a pre-trained convolutional neural network model or a Transformer model. The pre-trained model performs feature mapping on the pre-processed image information to output the three-dimensional position coordinates and pose parameters of the target object.

[0019] Furthermore, in step S3, the kinematic model includes a forward kinematic model and an inverse kinematic model of the robot. The forward kinematic model is used to calculate the position of the end effector based on the joint parameters, and the inverse kinematic model is used to calculate the joint motion parameters based on the target position. The grasping path is planned through the collaborative calculation of the forward kinematic model and the inverse kinematic model.

[0020] Furthermore, in step S5, when adjusting the attitude of the end effector, the dynamic attitude data of the target object is collected in real time, the attitude deviation value is calculated, and the rotation angle and tilt angle of the end effector are adjusted according to the attitude deviation value through a PID control algorithm.

[0021] Furthermore, when performing the grasping action in step S6, it includes:

[0022] Step S61: Determine the grasping point of the target object based on the geometric features obtained in step S2;

[0023] Step S62: Control the end effector to move to the gripping point;

[0024] Step S63: Collect the contact force through the force sensor, and stop applying force when the contact force reaches the preset threshold.

[0025] A robotic gripping control device based on position and orientation recognition, comprising:

[0026] The image acquisition unit is used to perform the image acquisition operation in step S1;

[0027] The data processing unit, connected to the image acquisition unit, is used to perform the preprocessing, feature extraction and recognition operations in step S2, and output the three-dimensional position coordinates and attitude parameters.

[0028] The path planning unit is connected to the data processing unit and is used for the path planning operation in step S3.

[0029] The motion control unit is connected to the path planning unit and the robot arm respectively, and is used to perform the motion control operations in steps S4-S7, including position movement control, posture adjustment control, grasping action control and reset control;

[0030] An end effector assembly, connected to a motion control unit, is used to physically grasp a target object;

[0031] A force sensing unit is disposed on the end effector assembly and is used to perform the contact force acquisition operation in step S63.

[0032] The storage unit is used to store the preset position and posture recognition algorithm, kinematic model and preset threshold.

[0033] Compared with existing technologies, this invention has the following advantages: The robotic arm grasping control method and device based on position and posture recognition uses deep learning algorithms to identify the three-dimensional position coordinates and posture parameters of the target object. Combined with image preprocessing and feature extraction, it significantly improves the accuracy of target recognition, laying the foundation for precise grasping. It employs a PID control algorithm to adjust the end effector's posture in real time. By dynamically collecting object posture data and calculating deviation values, the end effector can adaptively match changes in object posture, solving the problem that static planning is difficult to handle small displacements or posture changes, and improving the grasping adaptability in complex scenarios. It optimizes the grasping path and motion efficiency based on forward / inverse kinematics models, comprehensively considering joint range of motion, accuracy, and speed limitations to ensure a more reasonable robotic arm trajectory, reduce ineffective movements, improve motion efficiency and operational safety, and guarantee the safety and stability of the grasping process. Real-time detection of the grasping action is achieved through force sensors. By controlling contact force and threshold parameters, the system can precisely adjust the end effector's driving force, preventing damage from excessive force while ensuring a firm grip. This is particularly suitable for grasping fragile or irregularly shaped objects. It enhances the system's automation and intelligence, with fully automated control covering image acquisition, recognition, path planning, posture adjustment, grasping, and resetting, reducing manual intervention and operational complexity, and adapting to industrial batch operations. The modular structure facilitates implementation and maintenance. The control equipment utilizes modular design for image acquisition, data processing, path planning, and motion control, with clearly defined and collaborative functions for each unit. This not only facilitates system integration and debugging but also enables future maintenance and upgrades, improving the equipment's practicality and scalability. Real-time monitoring of operating status and posture changes allows it to handle object position / posture changes in dynamic scenarios. Combined with multi-algorithm fusion recognition and control strategies, the robotic arm can adapt to target objects of different materials and shapes, expanding its application range. This makes the system even more worthy of widespread adoption. Attached Figure Description

[0034] Figure 1 This is a structural block diagram of the present invention. Detailed Implementation

[0035] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0036] like Figure 1 As shown, this embodiment provides a technical solution: a robotic arm grasping control method based on position and posture recognition, comprising the following steps:

[0037] Step S1: Acquire initial image information of the target object using an image acquisition device;

[0038] Step S2: Preprocess and extract features from the initial image information, and obtain the three-dimensional position coordinates and pose parameters of the target object through a preset position and pose recognition algorithm;

[0039] Step S3: Based on the three-dimensional position coordinates and attitude parameters, and combined with the kinematic model of the robot, plan the grasping path of the robot.

[0040] Step S4: Control the robotic arm to move to the target position according to the grasping path;

[0041] Step S5: Adjust the attitude of the robot's end effector according to the attitude parameters to match the attitude of the target object;

[0042] Step S6: Control the end effector of the robotic arm to perform a grasping action and grasp the target object;

[0043] Step S7: After the grasping is completed, control the robotic arm to move the target object to the preset position and reset it.

[0044] Preprocessing in step S2 includes image denoising, image enhancement, and image segmentation; feature extraction includes extracting the contour features, texture features, and geometric features of the target object.

[0045] Image denoising in preprocessing effectively eliminates interference factors such as light noise and equipment noise in the shooting environment, avoiding interference from noisy data to subsequent analysis. Image enhancement highlights key visual information of the target object (such as edges and contrast), making the features of objects that were originally blurry or low in contrast clearer and providing high-quality input data for subsequent processing. Image segmentation can clearly separate the target object from the background region, eliminating interference from irrelevant background information, allowing feature extraction to focus on the target object itself, reducing unnecessary computation, and improving processing efficiency. Extracting the contour features (such as shape boundaries), texture features (such as surface texture), and geometric features (such as size and proportion) of the target object provides multi-dimensional and multi-type feature data support for subsequent position and posture recognition algorithms, enabling the algorithm to more comprehensively understand the attributes of the target object, thereby improving the accuracy and robustness of 3D position coordinates and posture parameter recognition. High-quality preprocessing results and accurate feature extraction directly affect the accuracy of subsequent path planning, posture adjustment, and other steps, reducing grasping errors caused by image quality issues and ensuring the reliability of the entire robotic arm grasping control process from the source.

[0046] The image segmentation uses a threshold-based segmentation algorithm or an edge detection segmentation algorithm to separate the target object from the background region;

[0047] By explicitly employing either threshold segmentation or edge detection algorithms, the appropriate segmentation method can be selected for different types of target objects and scene features: threshold segmentation is suitable for scenes with significant differences in grayscale between the target and the background, enabling rapid pixel-level separation; edge detection segmentation is suitable for objects with clear contour features, accurately capturing object boundaries. The targeted application of these two algorithms effectively solves the problem of blurred or mis-segmented targets in complex backgrounds, improving the reliability of the segmentation results.

[0048] By accurately separating the target object from the background region, the interference of irrelevant pixels in the background (such as stray light, shadows, and other objects) on feature extraction can be completely eliminated, so that subsequent contour, texture, and geometric feature extraction can focus only on the target object itself, reducing the amount of invalid computation and significantly improving the purity and effectiveness of feature data.

[0049] Clear target-background segmentation results provide more easily parsed image data for subsequent deep learning recognition algorithms (convolutional neural networks, Transformer models), avoiding feature misjudgment caused by background noise interference, thereby indirectly improving the accuracy of 3D position coordinates and posture parameter recognition, and providing more reliable preliminary data support for precise grasping by robotic arms.

[0050] Thresholding segmentation and edge detection are both mature and lightweight image segmentation algorithms that can be implemented quickly without complex computing power, making them easy to deploy in embedded devices or industrial control terminals. While ensuring segmentation results, they reduce the hardware cost and real-time pressure of the entire control system.

[0051] The preset position and pose recognition algorithm in step S2 is a deep learning-based recognition algorithm, including a pre-trained convolutional neural network model or a Transformer model. The pre-trained model performs feature mapping on the pre-processed image information to output the three-dimensional position coordinates and pose parameters of the target object. The deep learning model has a powerful feature learning capability and can capture the subtle features of the target object (such as surface texture and local contour changes) from the pre-processed image. Compared with traditional algorithms, it is more adaptable to complex scenes such as lighting changes and background interference, and significantly improves the recognition accuracy of three-dimensional position coordinates and pose parameters.

[0052] Pre-trained neural network models, trained on large-scale data, have learned general visual feature patterns. When faced with target objects of different types, shapes, and materials, they can achieve efficient recognition without redesigning feature extraction rules, significantly improving the robot's adaptability to diverse grasping tasks. Through the feature mapping process of the deep learning model, an end-to-end optimized link can be formed directly from the image input to the position / pose parameter output, reducing the error accumulation caused by multi-step manual feature design in traditional algorithms and improving the consistency and reliability of parameter calculation. Traditional recognition algorithms rely on manually designed feature extraction rules, while deep learning algorithms can automatically learn effective features, reducing reliance on professional experience, lowering the complexity of system development and debugging, and facilitating continuous optimization of recognition performance through model iteration. Precise 3D position coordinates and pose parameter outputs provide a reliable data foundation for subsequent path planning, enabling the robot to plan motion trajectories based on more accurate target information, further improving the accuracy and stability of grasping actions.

[0053] In step S3, the kinematic model includes the forward kinematic model and the inverse kinematic model of the robot. The forward kinematic model is used to calculate the position of the end effector based on the joint parameters, and the inverse kinematic model is used to calculate the joint motion parameters based on the target position. The grasping path is planned through the collaborative calculation of the forward kinematic model and the inverse kinematic model.

[0054] The forward kinematics model can accurately calculate the actual position of the end effector based on the joint parameters, while the inverse kinematics model can infer the required motion parameters of each joint based on the target position. The collaborative calculation of the two can realize closed-loop verification from the target position to the joint motion and then to the end position, ensuring that the planned grasping path is feasible within the physical motion range of the robot and avoiding the problem of the theoretical path being disconnected from the actual motion. By combining forward and inverse kinematics models, the range of motion, accuracy, and speed limitations of each joint of the robotic arm can be comprehensively considered to plan the shortest path or the lowest energy-consuming motion trajectory. This reduces unnecessary joint rotation and wasted travel, significantly improving the robotic arm's motion efficiency and reducing equipment operating energy consumption. The model calculation process fully incorporates joint motion limit parameters, effectively avoiding problems such as joint overtravel, collisions, or excessive mechanical stress caused by improper path planning. This ensures smooth movement of each component during the grasping process, reduces the risk of equipment damage, and extends service life. For different target objects with varying 3D position coordinates and attitude parameters, the forward and inverse kinematics models can flexibly adjust joint motion parameters to generate personalized paths adapted to specific grasping angles and positions. This is particularly suitable for precise operations in complex scenarios such as narrow spaces and multiple obstacles. The precisely planned grasping path provides a stable motion foundation for subsequent steps, reducing the amount of attitude adjustment compensation caused by path deviations and improving the coherence and control accuracy of the entire grasping process.

[0055] When adjusting the attitude of the end effector in step S5, the process includes real-time acquisition of dynamic attitude data of the target object, calculation of attitude deviation value, and adjustment of the rotation angle and tilt angle of the end effector through a PID control algorithm based on the attitude deviation value.

[0056] By acquiring dynamic attitude data of the target object in real time, it can promptly capture attitude changes caused by external interference or its own minute displacements, solving the problem that traditional static attitude planning cannot handle dynamic scenarios. This ensures that the end effector and the object's attitude remain accurately matched, significantly improving the success rate of grasping in complex environments. A PID control algorithm is used to calculate attitude deviation values ​​and adjust the rotation and tilt angles of the end effector. Utilizing the synergistic effect of proportional, integral, and derivative components, attitude deviations can be quickly eliminated, reducing overshoot and oscillations, resulting in a smoother attitude adjustment process and improved response speed. This is particularly suitable for attitude control applications. In scenarios involving the grasping of precision components with high attitude sensitivity, the robot dynamically calculates attitude deviations and adjusts the end effector angle accordingly for target objects of different shapes and structures. This allows for flexible adaptation to the unique attitude characteristics of the object, avoiding grasping offsets or slippage caused by attitude mismatches, thus expanding the robot's applicability for grasping irregular and shaped objects. Real-time attitude feedback and closed-loop control mechanisms form a dynamic correction link, which can effectively compensate for accumulated deviations such as mechanical errors and installation errors during the robot's movement, ensuring that the deviation between the final attitude of the end effector and the target attitude is kept within a minimum range, providing a stable attitude foundation for subsequent grasping actions.

[0057] When performing the grabbing action in step S6, it includes:

[0058] Step S61: Determine the grasping point of the target object based on the geometric features obtained in step S2;

[0059] Step S62: Control the end effector to move to the gripping point;

[0060] Step S63: Collect the grasping contact force through the force sensor, and stop applying force when the contact force reaches the preset threshold;

[0061] Based on the geometric features extracted in step S2, the gripping point is determined. Areas with stable object structure and balanced force can be selected to avoid object falling or posture deviation due to unreasonable gripping points, thereby improving gripping stability from the source.

[0062] By defining the steps from the grasping point to moving to the grasping point and then stopping the force application (steps S61-S63), the grasping action is standardized, reducing the randomness of manual operation, ensuring the consistency of grasping logic in different scenarios, and improving the standardization of system operation.

[0063] By using force sensors to collect contact force in real time and comparing it with a preset threshold, the driving force of the end effector can be dynamically adjusted: when the contact force reaches the threshold, the force application automatically stops, which can avoid the deformation and damage of objects due to excessive torque (especially suitable for fragile items and precision parts), and ensure the gripping firmness, thus resolving the contradiction between looseness and fall-off and excessive tightness and damage. The gripping point is determined based on geometric features, which can adapt to irregular and non-standard shaped objects, while the force control feedback mechanism can adapt to the force characteristics of objects of different materials (e.g., soft objects require less force, and hard objects require appropriate force), which significantly improves the robot's ability to grip diverse targets. The real-time force sensing monitoring and threshold control form a closed-loop feedback, which can respond promptly to abnormal forces during the gripping process (e.g., sudden collisions, object slippage), and avoid mechanical overload by dynamically adjusting the force output, thereby reducing the risk of equipment failure and extending the service life of the robot.

[0064] A robotic gripping control device based on position and orientation recognition, comprising:

[0065] The image acquisition unit is used to perform the image acquisition operation in step S1;

[0066] The data processing unit, connected to the image acquisition unit, is used to perform the preprocessing, feature extraction and recognition operations in step S2, and output the three-dimensional position coordinates and attitude parameters.

[0067] The path planning unit is connected to the data processing unit and is used for the path planning operation in step S3.

[0068] The motion control unit is connected to the path planning unit and the robot arm respectively, and is used to perform the motion control operations in steps S4-S7, including position movement control, posture adjustment control, grasping action control and reset control;

[0069] An end effector assembly, connected to a motion control unit, is used to physically grasp a target object;

[0070] A force sensing unit is disposed on the end effector assembly and is used to perform the contact force acquisition operation in step S63.

[0071] The storage unit is used to store the preset position and posture recognition algorithm, kinematic model and preset threshold.

[0072] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0073] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0074] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A robotic gripping control method based on position and posture recognition, characterized in that, Includes the following steps: Step S1: Acquire initial image information of the target object using an image acquisition device; Step S2: Preprocess and extract features from the initial image information, and obtain the three-dimensional position coordinates and pose parameters of the target object through a preset position and pose recognition algorithm; Step S3: Based on the three-dimensional position coordinates and attitude parameters, and combined with the kinematic model of the robot, plan the grasping path of the robot. Step S4: Control the robotic arm to move to the target position according to the grasping path; Step S5: Adjust the attitude of the robot's end effector according to the attitude parameters to match the attitude of the target object; Step S6: Control the end effector of the robotic arm to perform a grasping action and grasp the target object; Step S7: After the grasping is completed, control the robotic arm to move the target object to the preset position and reset it.

2. The robotic gripping control method based on position and posture recognition according to claim 1, characterized in that: The preprocessing in step S2 includes image denoising, image enhancement and image segmentation, and feature extraction includes extracting the contour features, texture features and geometric features of the target object.

3. The robotic gripping control method based on position and posture recognition according to claim 1, characterized in that: The image segmentation uses a threshold-based segmentation algorithm or an edge detection segmentation algorithm to separate the target object from the background region.

4. The robotic gripping control method based on position and posture recognition according to claim 1, characterized in that: The preset position and pose recognition algorithm in step S2 is a deep learning-based recognition algorithm, including a pre-trained convolutional neural network model or a Transformer model. The pre-trained model performs feature mapping on the pre-processed image information to output the three-dimensional position coordinates and pose parameters of the target object.

5. The robotic gripping control method based on position and posture recognition according to claim 1, characterized in that: In step S3, the kinematic model includes the forward kinematic model and the inverse kinematic model of the robot. The forward kinematic model is used to calculate the position of the end effector based on the joint parameters, and the inverse kinematic model is used to calculate the joint motion parameters based on the target position. The grasping path is planned through the collaborative calculation of the forward kinematic model and the inverse kinematic model.

6. The robotic gripping control method based on position and posture recognition according to claim 1, characterized in that: In step S5, when adjusting the attitude of the end effector, the dynamic attitude data of the target object is collected in real time, the attitude deviation value is calculated, and the rotation angle and tilt angle of the end effector are adjusted according to the attitude deviation value through a PID control algorithm.

7. The robotic gripping control method based on position and posture recognition according to claim 1, characterized in that: When performing the grabbing action in step S6, it includes: Step S61: Determine the grasping point of the target object based on the geometric features obtained in step S2; Step S62: Control the end effector to move to the gripping point; Step S63: Collect the contact force through the force sensor, and stop applying force when the contact force reaches the preset threshold.

8. A robotic gripping control device based on position and posture recognition, the device being based on the control method according to any one of claims 1-7, characterized in that: include: The image acquisition unit is used to perform the image acquisition operation in step S1; The data processing unit, connected to the image acquisition unit, is used to perform the preprocessing, feature extraction and recognition operations in step S2, and output the three-dimensional position coordinates and attitude parameters. The path planning unit is connected to the data processing unit and is used for the path planning operation in step S3. The motion control unit is connected to the path planning unit and the robot arm respectively, and is used to perform the motion control operations in steps S4-S7, including position movement control, posture adjustment control, grasping action control and reset control; An end effector assembly, connected to a motion control unit, is used to physically grasp a target object; A force sensing unit is disposed on the end effector assembly and is used to perform the contact force acquisition operation in step S63. The storage unit is used to store the preset position and posture recognition algorithm, kinematic model and preset threshold.

Images