Self-adaptive mechanical arm clamping control method and system based on image recognition

Abstract

The invention is suitable for the technical field of mechanical arm control, and provides a self-adaptive mechanical arm clamping control method and system based on image recognition, the system comprises a mechanical arm, and the mechanical arm comprises a mechanical arm body and a mechanical clamping jaw; the image acquisition module is used for acquiring a 2D image and depth information of an object; the image processing module is used for carrying out preprocessing, contour extraction and pose coordinate determination on the acquired image; the data transmission module is used for transmitting the contour, size and pose coordinate information of the object processed by the image processing module to the controller; the controller is used for receiving information sent by the data transmission module and controlling movement and clamping force of the mechanical arm; according to the mechanical arm, the path can be automatically planned and proper force can be applied according to the characteristics of different objects, the flexibility, the accuracy and the safety of the clamping operation of the mechanical arm are improved, and the mechanical arm is suitable for various automatic scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arm control, and particularly to an adaptive force robotic arm grasping control method and system based on image recognition. Background Art

[0002] In industrial automation production, the grasping operation of a robotic arm is an important link. Traditional robotic arm grasping methods usually require pre-setting information such as the position and shape of an object, and complex parameter adjustments are needed for different objects, resulting in poor flexibility and adaptability. In the prior art, although there are some vision-based robotic arm grasping systems, most of them use binocular cameras or complex sensor combinations, which are costly and have a large system complexity. At the same time, in terms of clamping force control, there is a lack of an effective method to dynamically adjust the force size according to the real-time recognition result of the object, which easily leads to object damage or grasping failure.

[0003] Therefore, in view of the above situation, there is an urgent need to provide an adaptive force robotic arm grasping control method and system based on image recognition to overcome the deficiencies in current practical applications. Summary of the Invention

[0004] The purpose of the present invention is to provide an adaptive force robotic arm grasping control method and system based on image recognition, aiming to solve the problems in the above background art.

[0005] The present invention is implemented as follows. An adaptive force robotic arm grasping control method and system based on image recognition includes: A robotic arm, which includes a robotic arm body and a robotic gripper; An image acquisition module, which is used to acquire the 2D image and depth information of an object; An image processing module, connected to the image acquisition module, which is used to preprocess the acquired image, extract the contour, and determine the pose coordinates; A data transmission module, which is used to transmit the object contour, size, and pose coordinate information processed by the image processing module to the controller; A controller, which is used to receive the information sent by the data transmission module and control the movement and clamping force of the robotic arm; A thin film pressure sensor installed on the robotic gripper, which is used to monitor the clamping force in real time and feedback it to the controller.

[0006] As a further aspect of the present invention: The image acquisition module is a monocular 3D camera, and the image acquisition module is installed directly above the working space of the robotic arm, and the optical axis of the lens is perpendicular to the plane where the object is located, ensuring that the field of view covers the entire working area of the robotic arm.

[0007] As a further aspect of the present invention: The image processing module includes: An image preprocessing unit for denoising, grayscaling, and edge detection of an image, where Gaussian filtering algorithm is used for denoising, and the size and standard deviation of the filter kernel are estimated according to the estimated value of the image noise variance Dynamic adjustment: When is less than the set threshold then, a filter kernel is used, and the standard deviation is set to 1; when is between and then, a filter kernel is used, and the standard deviation is set to 1.5; when is greater than then, a filter kernel is used, and the standard deviation is set to 2; A contour extraction unit that uses a gradient-based contour detection algorithm and a MaskR-CNN algorithm to extract the object contour; A pose coordinate determination unit that uses the PnP algorithm in combination with the monocular 3D camera calibration parameters to calculate the three-dimensional position and rotation angle of the object, and improves the accuracy through an iterative optimization method. The optimization objective function is: ； where are the actual coordinates of the feature points in the image, are the coordinates obtained by reprojection according to the current pose estimate, and n is the number of feature points.

[0008] As a further solution of the present invention: The controller includes: A path planning unit that uses an improved algorithm to plan the grasping path according to the object pose and the manipulator kinematic constraints. The cost function of the path planning includes a joint constraint function: ； where is the angle of the j-th joint of the manipulator, and are the upper and lower limits of the angle of this joint, and m is the number of joints; An initial force determination unit that uses a support vector regression (SVR) model to predict the magnitude of the initial force for grasping the object; A force closed-loop control unit that uses an adaptive fuzzy PID control algorithm to dynamically adjust the grasping force and adjusts the PID parameters according to the deviation and the rate of change of the deviation feedback by the force sensor .

[0009] As a further solution of the present invention: The improved The algorithm introduces a dynamic weight mechanism, which adjusts the weight of the heuristic function according to the distribution density of obstacles. When the obstacle density is high, the weight of the target point distance is increased, and when the obstacle density is low, the weight is decreased to improve the path smoothness.

[0010] As a further solution of the present invention: The force closed-loop control unit adjusts the parameters of the PID controller in real time according to the feedback signal of the thin film pressure sensor to ensure the stability and safety of the clamping force.

[0011] As a further solution of the present invention: The robotic arm is a Fanuc robotic arm.

[0012] As a further solution of the present invention: The image processing module is an embedded processor with image processing functions.

[0013] An adaptive force robotic arm clamping control method based on image recognition, applicable to the adaptive force robotic arm clamping control system as described above. This method includes the following steps: Collect the 2D image and depth information of the object through a monocular 3D camera; Use the image processing module to preprocess the image, extract the contour, and determine the pose coordinates to obtain the contour, size, and pose information of the object; Use the data transmission module to transmit the object contour, size, and pose coordinate information processed by the image processing module to the controller; The controller determines the initial clamping force according to the object attributes and plans the movement path of the robotic arm; Use the controller to control the robotic arm to move along the planned path and clamp the object with the initial clamping force; Use the thin film pressure sensor to monitor the clamping force in real time and adopt an adaptive fuzzy PID control algorithm to dynamically adjust the clamping force.

[0014] As a further solution of the present invention: The contour extraction step includes: Preprocess the image using Gaussian filtering, grayscale conversion, and Canny edge detection, and the filter kernel parameters are dynamically adjusted according to the noise level; Use the MaskR-CNN algorithm to assist in extracting the object contour and calculate the actual size of the object in combination with the depth information.

[0015] Compared with the prior art, the beneficial effects of the present invention are: First, in the present invention, a monocular 3D camera is used, which reduces the cost and simplifies the system complexity compared with the traditional binocular camera and multi-sensor combination scheme; Second, the present invention uses an image processing module to accurately obtain the contour, size, and pose coordinate information of an object in real time. In combination with a data transmission module and a controller, an improved algorithm is used to achieve precise grasping of the object by the robot, reducing the probability of object damage and grasping failure. Specifically, the improved image preprocessing algorithm improves the image quality, the MaskR-CNN algorithm assists in contour extraction to enhance the accuracy of contour extraction, the iteratively optimized PnP algorithm improves the pose calculation accuracy, and the improved algorithm makes the path planning more efficient and compliant with the motion constraints of the robotic arm. The SVR algorithm and the adaptive fuzzy PID control algorithm respectively improve the accuracy of initial force determination and the stability of force closed-loop control; Third, the present invention can automatically plan paths and apply appropriate forces according to the characteristics of different objects, improving the flexibility, accuracy, and safety of the robotic arm grasping operation, and is applicable to a variety of automation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 It is a schematic diagram of the system structure of the present invention.

[0018] Figure 2 It is a schematic diagram of the image processing flow in the present invention.

[0019] Figure 3 It is a flowchart of the robotic arm control implementation in the present invention.

[0020] Figure 4 It is a flowchart of the force closed-loop control implementation in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The following will clearly and completely describe the technical solutions of the present invention with reference to the drawings. Obviously, the described embodiments are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0022] The following further explains the present invention in combination with specific embodiments.

[0023] Please refer to Figures 1 - 4, an adaptive force robotic arm clamping control system provided by an embodiment of the present invention, the system includes an image acquisition module, an image processing module, a data transmission module, a controller, a Fanuc robotic arm, and a thin film pressure sensor; Image acquisition module: The image acquisition module is used to acquire the image information of an object, including the contour, size, and pose characteristics of the object. A monocular 3D camera, model [EZVIZ C8C], is selected. This camera can also obtain the 2D image and depth information of the object. The monocular 3D camera is connected to the image processing module through a USB interface and is installed at a fixed position directly above the working space of the robotic arm. The optical axis of the lens is perpendicular to the plane where the object is located to ensure that the lens field of view covers the entire working area of the robotic arm.

[0024] Image processing module: This module is an embedded processor with image processing functions (NVIDIA Jetson AGX Orin). This module is connected to the image acquisition module and is used to process the acquired image, extract the contour of the object, calculate the size of the object, and determine the pose coordinate information of the object. The specific processing steps are as follows: Image preprocessing: Perform preprocessing operations such as denoising, grayscale conversion, and edge detection on the acquired image to remove noise and interference information in the image and highlight the edges and contours of the object. The Gaussian filtering algorithm is used for denoising. The color image is converted to a grayscale image through grayscale conversion for subsequent processing. The Canny operator is used for edge detection, which can accurately detect the edges of the object. In the Gaussian filtering algorithm, the size and standard deviation of the filter kernel are dynamically adjusted according to the noise level of the image. If the estimated value of the image noise variance is , when is less than the set threshold , a filter kernel is used, and the standard deviation is set to 1; when is between and , a filter kernel is used, and the standard deviation is set to 1.5; when is greater than , a filter kernel is used, and the standard deviation is set to 2, so as to improve the denoising effect while retaining the image details to the greatest extent. Among them, the image processing module selects an embedded processing platform and installs image processing software (OpenCV library) to realize real-time processing of images.

[0025] Contour extraction: Use a gradient-based contour detection algorithm to extract the contour information of the object from the preprocessed image. On this basis, introduce the MaskR-CNN algorithm in deep learning to assist in contour extraction. First, input the preprocessed image into the pre-trained MaskR-CNN model. The model extracts the features of the image through the feature extraction network (ResNet), then uses the Region Proposal Network (RPN) to generate candidate regions that may contain the object, and then classifies and regresses the bounding boxes of these candidate regions to finally obtain the precise contour of the object. According to the extracted contour, calculate the circumscribed rectangle of the object to obtain the length and width of the object on the image plane. Then, combined with the depth information of the monocular 3D camera, use the principle of triangulation or depth image parsing to convert the size on the image plane into the size in the actual physical space, that is, the length, width, and height dimensions of the object.

[0026] Pose coordinate determination: Through the monocular vision pose estimation method (PnP algorithm), with the help of the internal parameters of the camera obtained by calibrating the checkerboard calibration plate using the Zhang's calibration method, and the feature points detected from the object contour through the Harris corner detection algorithm, calculate the pose coordinate information of the object in the world coordinate system. This includes three-dimensional position coordinates (X, Y, Z), as well as the rotation angles around the three coordinate axes (α, β, γ). To improve the accuracy of pose calculation, the PnP algorithm is improved by using an iterative optimization method. Based on the initially calculated pose, construct an objective function according to the reprojection error , where is the actual coordinate of the feature point in the image, is the coordinate reprojected according to the current pose estimation, and n is the number of feature points. Use the Levenberg-Marquardt algorithm to iteratively optimize the objective function, continuously adjust the pose parameters until the reprojection error converges to less than the set threshold, and obtain more accurate object pose coordinates.

[0027] Data transmission module: Select the USB interface to achieve data transmission between the image processing module and the controller. Transmit the object contour, size, and pose coordinate information obtained by the image processing module to the controller in real time to ensure the real-time and accuracy of data transmission.

[0028] Controller: The controller is connected to the data transmission module and is used to receive the object pose coordinate information and the magnitude information of the required force, control the Fanuc robotic arm to plan the path and apply an appropriate force magnitude to grasp the object. The controller uses a PLC controller, installs the Fanuc robotic arm control software, and is connected to the Fanuc robotic arm through a dedicated communication interface, with powerful data processing and control capabilities: Path planning: According to the object pose coordinate information and the current position of the robotic arm, use the improved The algorithm is used for path planning. Based on the traditional A algorithm, a dynamic weight mechanism is introduced. The weight of the heuristic function is dynamically adjusted according to the distribution density of obstacles in the workspace of the robotic arm. When the distribution density of obstacles is high, the weight of the distance to the target point in the heuristic function is increased, so that the robotic arm approaches the target point faster and the search range is reduced; when the distribution density of obstacles is low, this weight is reduced, and more attention is paid to the smoothness of the path. At the same time, considering the joint limitations and kinematic constraints of the robotic arm, a joint constraint function is constructed , where is the angle of the j-th joint of the robotic arm,[ and are the upper and lower limits of the angle of this joint, and m is the number of joints. During the path planning process, the joint constraint function is added as a penalty term to the cost function of the algorithm to ensure the feasibility of the path, plan the motion path of the robotic arm, avoid obstacles, and ensure that the robotic arm can reach the target position safely and efficiently; the robotic arm control implementation process is as follows: The robotic arm starts from the starting position, based on the improved algorithm and according to the object pose and the kinematic constraints of the robotic arm, calculates the optimal path to avoid obstacles, passes through the intermediate nodes of path planning (intermediate node 1 of path planning algorithm, intermediate node 2 of path planning algorithm), and finally reaches the object grasping position. In the obstacle area detection stage, the search strategy is adjusted through the dynamic weight mechanism to ensure that the motion trajectory of the robotic arm avoids obstacles and meets the joint constraint conditions.

[0029] Determination of the initial force magnitude: According to the information such as the size, weight, and material of the object obtained by the image processing module, determine the magnitude of the force required to grasp the object. A relationship model between object attributes and the initial grasping force is established using the support vector regression (SVR) algorithm. A large number of object samples with different sizes, weights, and materials are collected, and a grasping experiment is conducted on each sample, and the minimum stable grasping force when the grasping is successful is recorded as the label data. The size, weight, and material characteristics of the object are encoded and used as the input of the SVR model, and a prediction model is obtained through training. In practical applications, the currently recognized object attributes are input into the trained SVR model to predict the initial grasping force. The controller controls the robotic gripper of the robotic arm to apply an appropriate force to grasp the object.

[0030] Closed-loop control of force: The force sensor is installed on the robotic gripper of the robotic arm. During the grasping process, the feedback signal of the force sensor is monitored in real time, and the adaptive fuzzy PID control algorithm is used to dynamically adjust the magnitude of the force of the robotic gripper of the robotic arm. According to the deviation between the actual force value and the target force value feedback by the force sensor and the deviation change rate Perform online adjustment. For example, when the deviation is large and the deviation change rate is also large, increase the proportional coefficient to accelerate the system response speed; when the deviation e is small and the deviation change rate is small, decrease the integral coefficient to avoid integral saturation. By continuously adjusting the PID parameters, ensure the stability and safety of the clamping process.

[0031] Fanuc robotic arm: It includes the robotic arm body and robotic gripper, and is connected to the controller. It can move along the planned path according to the control signal sent by the controller and clamp the object with an appropriate force magnitude.

[0032] Thin-film pressure sensor: A high-precision and high-sensitivity thin-film pressure sensor is adopted and installed on the robotic gripper of the robotic arm to measure the force exerted when clamping the object in real time and convert the force signal into an electrical signal for transmission to the controller. Among them, the thin-film pressure sensor is installed on the robotic gripper of the robotic arm, and the force sensor is connected to the analog input interface of the controller through a signal line.

[0033] In the embodiment of the present invention, a monocular 3D camera is adopted, which reduces the cost and simplifies the system complexity compared with the traditional binocular camera and multi-sensor combination scheme; the image processing module is used to accurately obtain the contour, size and pose coordinate information of the object in real time. Combined with the data transmission module and the controller, an improved algorithm is used to achieve precise clamping of the object by the robot, reducing the probability of object damage and clamping failure. Specifically, the improved image preprocessing algorithm improves the image quality, the MaskR-CNN algorithm assists in contour extraction to improve the accuracy of contour extraction, the iteratively optimized PnP algorithm improves the pose calculation accuracy, and the improved algorithm makes the path planning more efficient and conforms to the motion constraints of the robotic arm. The SVR algorithm and the adaptive fuzzy PID control algorithm respectively improve the accuracy of initial force determination and the stability of force closed-loop control; it can automatically plan the path and apply an appropriate force according to the characteristics of different objects, improving the flexibility, accuracy and safety of the robotic arm clamping operation, and is applicable to various automation scenarios; the present invention effectively solves the problems of poor flexibility of the traditional robotic arm clamping method, inaccuracy in image recognition and control, high cost, and complex system in the prior art.

[0034] Please refer to Figures 1 - 4 , a method for controlling the clamping of an adaptive force robotic arm based on image recognition provided by the embodiment of the present invention is applicable to the control system for clamping an adaptive force robotic arm based on image recognition as described above. The method includes the following steps: Collect the 2D image and depth information of the object through a monocular 3D camera; The image processing module preprocesses the image, extracts the contour, and determines the pose coordinates to obtain the contour, size, and pose information of the object; The data transmission module transmits the contour, size, and pose coordinate information of the object processed by the image processing module to the controller; The controller determines the initial clamping force according to the object attributes and plans the motion path of the robotic arm; The controller controls the robotic arm to move along the planned path and clamp the object with the initial clamping force; The thin-film pressure sensor is used to monitor the clamping force in real time, and the adaptive fuzzy PID control algorithm is adopted to dynamically adjust the clamping force; In the embodiments of the present invention: The image acquisition module real-time acquires images of the area where the target object is located, including 2D images and depth images; The image processing module preprocesses the acquired images, extracts the contour, and determines the pose coordinates to obtain the contour, size, and pose coordinate information of the object, and then determines the magnitude of the force required during clamping using the SVR model according to the attributes of the object; The data transmission module transmits the pose coordinate information of the object and the magnitude information of the required force to the control; The controller sends the processed data of the image processing to the Fanuc robotic arm, controls the robotic arm to move along the path planned by the improved algorithm, and the mechanical gripper clamps the object with an appropriate force magnitude; Force closed-loop control: The controller first sets the target force value. When the robotic arm clamps the object, the thin-film pressure sensor converts the real-time force signal into an electrical signal and feeds it back to the controller. The adaptive fuzzy PID control algorithm is used to dynamically adjust the magnitude of the force of the mechanical gripper of the robotic arm to ensure the stability and safety of the clamping process. Specifically, the controller compares the actual force value with the target force value: when the actual force value > the target force value, the intensity of the control signal output to the mechanical gripper is reduced to weaken the clamping force; when the actual force value < the target force value, the intensity of the control signal is increased to enhance the clamping force, and further dynamic real-time monitoring is realized, so as to achieve dynamic adjustment. The increased or decreased intensity of the control signal refers to the intensity of the drive signal output by the controller to the mechanical gripper of the robotic arm. This signal is used to control the clamping force magnitude of the gripper, and its essence is an adjustable electrical signal parameter. Its essence is to use the fuzzy PID control algorithm to perform real-time adjustment on the parameters of the PID controller to ensure the stability and safety of the clamping process.

[0035] In addition, the software implementation of the present invention is as follows: Camera calibration: The Zhang's calibration method is used to calibrate the monocular 3D camera to obtain the internal parameters (focal length, principal point coordinates, etc.) and external parameters (rotation matrix, translation vector) of the camera, providing basic data for subsequent pose estimation.

[0036] Image processing algorithm: Implement functions such as image preprocessing, edge detection, contour extraction, and corner detection based on the OpenCV library. Use the PnP algorithm to calculate the object pose coordinates in combination with calibration parameters and object feature points, and use the iterative optimization method to improve the accuracy. At the same time, use the MaskR-CNN algorithm to assist in contour extraction.

[0037] Path planning algorithm: Implement the algorithm in the controller to plan the grasping path according to the manipulator workspace and object pose coordinates, considering the obstacle distribution and manipulator joint limitations.

[0038] Force control algorithm: Establish a clamping force database, collect object attributes and clamping force data to train the SVR model for determining the initial clamping force. Use the adaptive fuzzy PID control algorithm to achieve force closed-loop control and adjust the clamping force according to the feedback of the force sensor.

[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive force robotic arm gripping control system based on image recognition, comprising a robotic arm, the robotic arm including a robotic arm body and a mechanical gripper, characterized in that, It further includes: An image acquisition module for acquiring 2D images and depth information of an object; An image processing module connected to the image acquisition module for preprocessing the acquired images, extracting contours, and determining pose coordinates; A data transmission module for transmitting the object contour, size, and pose coordinate information processed by the image processing module to the controller; A controller for receiving the information sent by the data transmission module and controlling the movement and gripping force of the robotic arm; A thin-film pressure sensor installed on the robotic gripper for real-time monitoring of the gripping force and feedback to the controller.

2. The adaptive force robotic arm clamping control system based on image recognition according to claim 1, wherein The image acquisition module is a monocular 3D camera, and the image acquisition module is installed directly above the working space of the robotic arm, with the optical axis of the lens perpendicular to the plane where the object is located, ensuring that the field of view covers the entire working area of the robotic arm.

3. The adaptive force robotic arm clamping control system based on image recognition according to claim 2, characterized in that, The image processing module includes: An image preprocessing unit for denoising, grayscaling, and edge detection of an image, where Gaussian filtering algorithm is used for denoising, and the size and standard deviation of the filter kernel are estimated according to the estimated value of the image noise variance Dynamic adjustment: When is less than the set threshold , use the filter kernel with a standard deviation set to 1; when is between and , use the filter kernel with a standard deviation set to 1.5; when is greater than , use the filter kernel with a standard deviation set to 2; A contour extraction unit that extracts the object contour using a gradient-based contour detection algorithm and the Mask R-CNN algorithm; A pose coordinate determination unit that calculates the three-dimensional position and rotation angle of the object using the PnP algorithm in combination with the calibration parameters of the monocular 3D camera, and improves the accuracy through an iterative optimization method. The optimization objective function is: ； where are the actual coordinates of the feature points in the image, are the coordinates obtained by reprojection according to the current pose estimation, and n is the number of feature points.

4. The adaptive force robotic arm clamping control system based on image recognition according to claim 3, wherein The controller includes: Path planning unit, using an improved algorithm to plan the grasping path according to the object pose and the kinematic constraints of the robotic arm, where the cost function of path planning includes a joint constraint function: ； where is the angle of the j-th joint of the robotic arm, and are the upper and lower limits of the angle of this joint, and m is the number of joints; An initial force determination unit that predicts the magnitude of the initial force for gripping the object using a support vector regression (SVR) model; The force closed-loop control unit uses an adaptive fuzzy PID control algorithm to dynamically adjust the clamping force and adjusts the PID parameters according to the deviation and the rate of change of the deviation feedback by the force sensor. .

5. The adaptive force robotic arm clamping control system based on image recognition according to claim 4, characterized in that, The improved algorithm introduces a dynamic weight mechanism to adjust the weight of the heuristic function according to the obstacle distribution density. When the obstacle density is high, the weight of the target point distance is increased, and when the obstacle density is low, the weight is decreased to improve the path smoothness.

6. The adaptive force robotic arm clamping control system based on image recognition according to claim 5, characterized in that The force closed-loop control unit adjusts the parameters of the PID controller in real time according to the feedback signal of the thin-film pressure sensor to ensure the stability and safety of the gripping force.

7. The adaptive force robotic arm clamping control system based on image recognition according to claim 1, wherein The robotic arm is a Fanuc robotic arm.

8. The adaptive force robotic arm clamping control system based on image recognition according to claim 1, wherein The image processing module is an embedded processor with image processing capabilities.

9. An adaptive force manipulator clamping control method based on image recognition, characterized in that, Applicable to the adaptive force robotic arm gripping control system based on image recognition as described in claim 6, the method includes the following steps: Acquire 2D images and depth information of the object through a monocular 3D camera; Use the image processing module to preprocess the images, extract contours, and determine pose coordinates to obtain the object's contour, size, and pose information; Use the data transmission module to transmit the object contour, size, and pose coordinate information processed by the image processing module to the controller; The controller determines the initial gripping force according to the object attributes and plans the movement path of the robotic arm; Use the controller to control the robotic arm to move along the planned path and grip the object with the initial gripping force; Use the thin-film pressure sensor to monitor the gripping force in real time and dynamically adjust the gripping force using an adaptive fuzzy PID control algorithm.

10. The adaptive force robotic arm gripping control method based on image recognition according to claim 9, characterized in that, The contour extraction step includes: Preprocess the images using Gaussian filtering, grayscale conversion, and Canny edge detection, and dynamically adjust the filter kernel parameters according to the noise level; Use the Mask R-CNN algorithm to assist in extracting the object contour and calculate the actual size of the object in combination with the depth information.

Images