An adaptive force manipulator gripping control method and system based on image recognition

Abstract

The present invention is applicable to the field of robotic arm control technology, and provides an adaptive force robotic arm clamping control method and system based on image recognition. The system includes a robotic arm, and the robotic arm includes a robotic arm body and a mechanical gripper; an image acquisition module, which is used to acquire 2D images and depth information of objects; an image processing module, which is used to preprocess, extract contours and determine posture coordinates of the acquired images; a data transmission module, which is used to transmit the object contour, size and posture coordinate information processed by the image processing module to a controller; a controller, which is used to receive information sent by the data transmission module and control the movement and clamping force of the robotic arm; a thin film pressure sensor installed on the mechanical gripper, which is used to monitor the clamping force in real time and feed it back to the controller. The present invention can automatically plan paths and apply appropriate forces according to the characteristics of different objects, thereby improving the flexibility, accuracy and safety of the robotic arm clamping operation, and is suitable for a variety of automation scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arm control, and in particular to an image recognition-based adaptive force robotic arm gripping control method and system. Background Art

[0002] In industrial automation production, the gripping operation of a robotic arm is a crucial step. Traditional robotic arm gripping methods usually require pre-setting information such as the position and shape of the object, requiring complex parameter adjustments for different objects, resulting in poor flexibility and adaptability. In the existing technology, although there are some vision-based robotic arm gripping systems, most of them use binocular cameras or complex sensor combinations, which are costly and complex. At the same time, in terms of gripping force control, there is a lack of effective methods to dynamically adjust the force based on the real-time object recognition results, which can easily lead to object damage or gripping failure.

[0003] Therefore, in view of the above situation, there is an urgent need to provide an adaptive force manipulator gripping control method and system based on image recognition to overcome the shortcomings in current practical applications. Summary of the Invention

[0004] The purpose of the present invention is to provide an adaptive force manipulator gripping control method and system based on image recognition, aiming to solve the problems in the above-mentioned background technology.

[0005] The present invention is implemented as follows: an adaptive force manipulator gripping control method and system based on image recognition, comprising:

[0006] A robotic arm, comprising a robotic arm body and a robotic gripper;

[0007] Image acquisition module, used to collect 2D images and depth information of objects;

[0008] An image processing module, connected to the image acquisition module, for performing preprocessing, contour extraction, and position coordinate determination on the acquired image;

[0009] A data transmission module is used to transmit the object contour, size and posture coordinate information processed by the image processing module to the controller;

[0010] A controller, configured to receive information sent by the data transmission module and control the movement and gripping force of the robotic arm;

[0011] The thin film pressure sensor installed on the mechanical gripper is used to monitor the gripping force in real time and provide feedback to the controller.

[0012] As a further solution of the present invention: the image acquisition module is a monocular 3D camera, and the image acquisition module is installed directly above the working space of the robotic arm, and the optical axis of the lens is perpendicular to the plane where the object is located, ensuring that the field of view covers the entire working area of the robotic arm.

[0013] As a further solution of the present invention: the image processing module includes:

[0014] Image preprocessing unit, used for denoising, grayscale and edge detection of images. The denoising adopts Gaussian filtering algorithm, and the size and standard deviation of the filter kernel are estimated according to the image noise variance. Dynamic Adjustment:

[0015] when Less than the set threshold When using The filter kernel of , the standard deviation is set to 1; when Between and When between The filter kernel has a standard deviation of 1.5; when Greater than When using The filter kernel has a standard deviation of 2;

[0016] Contour extraction unit, which uses gradient-based contour detection algorithm and Mask R-CNN algorithm to extract object contours;

[0017] The pose coordinate determination unit uses the PnP algorithm combined with the monocular 3D camera calibration parameters to calculate the three-dimensional position and rotation angle of the object, and improves the accuracy through iterative optimization method. The optimization objective function is:

[0018] ；

[0019] in is the actual coordinate of the feature point in the image, is the coordinate obtained by reprojection based on the current pose estimate, and n is the number of feature points.

[0020] As a further solution of the present invention: the controller includes:

[0021] Path planning unit, using improved The algorithm plans the gripping path based on the object pose and the kinematic constraints of the robot arm. The cost function of the path planning includes the joint constraint function:

[0022] ；

[0023] in is the angle of the j-th joint of the robot arm, and are the upper and lower limits of the joint angle, and m is the number of joints;

[0024] The initial force determination unit uses the support vector regression (SVR) model to predict the initial force of the gripping object;

[0025] The force closed-loop control unit uses an adaptive fuzzy PID control algorithm to dynamically adjust the gripping force and adjust the PID parameters based on the deviation and deviation change rate of the force sensor feedback. .

[0026] As a further solution of the present invention: the improved The algorithm introduces a dynamic weight mechanism to adjust the heuristic function weight according to the obstacle distribution density. When the obstacle density is high, the target point distance weight is increased, and when the obstacle density is low, the weight is reduced to improve the path smoothness.

[0027] As a further solution of the present invention: the force closed-loop control unit adjusts the parameters of the PID controller in real time according to the feedback signal of the thin film pressure sensor to ensure the stability and safety of the clamping force.

[0028] As a further solution of the present invention: the robotic arm is a FANUC robotic arm.

[0029] As a further solution of the present invention: the image processing module is an embedded processor with image processing function.

[0030] An adaptive force manipulator gripping control method based on image recognition is applicable to the adaptive force manipulator gripping control system based on image recognition as described above. The method comprises the following steps:

[0031] Collect 2D images and depth information of objects through a monocular 3D camera;

[0032] Use the image processing module to preprocess the image, extract the contour and determine the pose coordinates to obtain the contour, size and pose information of the object;

[0033] The data transmission module transmits the object contour, size and posture coordinate information processed by the image processing module to the controller;

[0034] The controller determines the initial gripping force based on the object's properties and plans the robot's motion path;

[0035] Use the controller to control the robot arm to move along the planned path and grasp the object with the initial gripping force;

[0036] A thin film pressure sensor is used to monitor the clamping force in real time, and an adaptive fuzzy PID control algorithm is used to dynamically adjust the clamping force.

[0037] As a further solution of the present invention: the contour extraction step includes:

[0038] Gaussian filtering, grayscale conversion, and Canny edge detection are used to preprocess the image, and the filter kernel parameters are dynamically adjusted according to the noise level;

[0039] The Mask R-CNN algorithm is used to assist in extracting the object contour and calculate the actual size of the object in combination with the depth information.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] First, the present invention uses a monocular 3D camera, which reduces costs and simplifies system complexity compared to traditional binocular cameras and multi-sensor combination solutions;

[0042] Secondly, the present invention uses an image processing module to accurately obtain the contour, size, and pose coordinate information of an object in real time. In combination with a data transmission module and a controller, an improved algorithm is used to achieve precise gripping of objects by the robot, reducing the probability of object damage and gripping failure. Specifically, the improved image preprocessing algorithm improves image quality, the Mask R-CNN algorithm assists contour extraction to improve the accuracy of contour extraction, the iteratively optimized PnP algorithm improves the accuracy of pose calculation, and the improved The algorithm makes path planning more efficient and complies with the robot's motion constraints. The SVR algorithm and adaptive fuzzy PID control algorithm improve the accuracy of initial force determination and the stability of force closed-loop control, respectively.

[0043] Third, the present invention can automatically plan paths and apply appropriate forces according to the characteristics of different objects, thereby improving the flexibility, accuracy and safety of the robot arm's gripping operations and being suitable for a variety of automation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 Schematic diagram of the system structure of the present invention.

[0046] Figure 2 Schematic diagram of the image processing flow in the present invention.

[0047] Figure 3 This is a flowchart for implementing the robotic arm control in the present invention.

[0048] Figure 4 This is a flow chart for implementing the force closed-loop control in the present invention. DETAILED DESCRIPTION

[0049] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] The present invention will be further explained below with reference to specific embodiments.

[0051] See also Figure 1-Figure 4 , an embodiment of the present invention provides an adaptive force manipulator gripping control system based on image recognition, the system comprising an image acquisition module, an image processing module, a data transmission module, a controller, a FANUC manipulator arm, and a thin film pressure sensor;

[0052] Image Acquisition Module: This module captures image information about objects, including their outline, size, and position. A monocular 3D camera, model [Ezviz C8C], is used. This camera can also capture 2D images and depth information about objects. The monocular 3D camera is connected to the image processing module via a USB port and mounted in a fixed position directly above the robotic arm's workspace, with the lens' optical axis perpendicular to the object's plane to ensure that the lens' field of view covers the entire robotic arm's working area.

[0053] Image processing module: This module is an embedded processor (NVIDIA Jetson AGX Orin) with image processing capabilities. This module is connected to the image acquisition module and is used to process the acquired images, extract the outline of the object, calculate the object's size, and determine the object's position coordinate information. The specific processing steps are as follows:

[0054] Image preprocessing: Perform preprocessing operations such as denoising, grayscale, and edge detection on the collected images to remove noise and interference information in the image and highlight the edges and contours of the objects. The Gaussian filter algorithm is used for denoising, and the color image is converted into a grayscale image through grayscale to facilitate subsequent processing. The Canny operator is used for edge detection to accurately detect the edges of objects. In the Gaussian filter algorithm, the size and standard deviation of the filter kernel are dynamically adjusted according to the noise level of the image. If the image noise variance estimate is ,when Less than the set threshold When using The filter kernel of , the standard deviation is set to 1; when Between and When between The filter kernel has a standard deviation of 1.5; when Greater than When using The filter kernel is set to 2 with a standard deviation of 2 to improve the denoising effect while preserving image details to the greatest extent. The image processing module uses an embedded processing platform and installs image processing software (OpenCV library) to achieve real-time image processing.

[0055] Contour extraction: A gradient-based contour detection algorithm is used to extract object contour information from the preprocessed image. Based on this, the Mask R-CNN algorithm from deep learning is introduced to assist in contour extraction. The preprocessed image is first fed into a pretrained Mask R-CNN model. The model extracts image features using a feature extraction network (ResNet). The model then uses a region proposal network (RPN) to generate candidate regions that may contain objects. These candidate regions are then classified and subjected to bounding box regression to ultimately obtain the object's precise contour. Based on the extracted contour, the object's bounding rectangle is calculated to determine its length and width on the image plane. Combined with the depth information from the monocular 3D camera, triangulation or depth image analysis is used to convert the image dimensions to their actual physical dimensions, namely, the object's length, width, and height.

[0056] Pose coordinate determination: Through the monocular vision pose estimation method (PnP algorithm), with the help of the camera internal parameters obtained by calibration using the checkerboard calibration plate through the Zhang calibration method, and the feature points obtained from the object contour detection through the Harris corner detection algorithm, the pose coordinate information of the object in the world coordinate system is calculated. This includes the three-dimensional position coordinates (X, Y, Z), and the rotation angles around the three coordinate axes (α, β, γ). In order to improve the accuracy of pose calculation, the PnP algorithm is improved by iterative optimization. Based on the pose obtained by the initial calculation, the objective function is constructed according to the reprojection error. ,in is the actual coordinate of the feature point in the image, is the coordinate obtained by reprojecting the current pose estimate, and n is the number of feature points. The objective function is iteratively optimized using the Levenberg-Marquardt algorithm, which continuously adjusts the pose parameters until the reprojection error converges to a value less than the set threshold, resulting in more accurate object pose coordinates.

[0057] Data transmission module: A USB interface is used to realize data transmission between the image processing module and the controller. The object outline, size, and position coordinate information obtained by the image processing module are transmitted to the controller in real time to ensure the real-time and accurate data transmission.

[0058] Controller: The controller is connected to the data transmission module and is used to receive the object's position coordinate information and the required force information, control the FANUC robot arm to plan the path, and apply the appropriate force to grasp the object. The controller uses a PLC controller, installs FANUC robot arm control software, and connects to the FANUC robot arm through a dedicated communication interface. It has powerful data processing and control capabilities:

[0059] Path planning: Based on the object's pose coordinate information and the current position of the robot arm, the improved The algorithm is used for path planning. Based on the traditional A algorithm, a dynamic weight mechanism is introduced. The weight of the heuristic function is dynamically adjusted according to the distribution density of obstacles in the robot's workspace. When the obstacle distribution density is high, the weight of the distance to the target point in the heuristic function is increased, so that the robot moves closer to the target point faster and reduces the search range; when the obstacle distribution density is low, the weight is reduced, and more emphasis is placed on the smoothness of the path. At the same time, considering the joint limitations and kinematic constraints of the robot, by constructing the joint constraint function ,in is the angle of the j-th joint of the robot arm, and is the upper and lower limit of the angle of the joint, and m is the number of joints. In the path planning process, the joint constraint function is added as a penalty term to In the algorithm's cost function, the feasibility of the path is guaranteed, the robot's motion path is planned, obstacles are avoided, and the robot reaches the target position safely and efficiently. The robot control implementation process is as follows:

[0060] The robotic arm starts from the starting position and based on the improved The algorithm then calculates the optimal path to avoid obstacles based on the object's pose and the robot's kinematic constraints, passing through the path planning intermediate nodes (path planning algorithm intermediate node 1 and path planning algorithm intermediate node 2) and ultimately reaching the object's gripping position. During the obstacle area detection phase, a dynamic weighting mechanism adjusts the search strategy to ensure that the robot's trajectory avoids obstacles and meets joint constraints.

[0061] Determining the size of the initial force: Based on the object's size, weight, and material information obtained by the image processing module, the amount of force required to grip the object is determined. A support vector regression (SVR) algorithm is used to establish a relationship model between object attributes and initial gripping force. A large number of object samples of different sizes, weights, and materials are collected, and gripping experiments are conducted on each sample. The minimum stable gripping force when the gripping is successful is recorded as label data. The size, weight, and material characteristics of the object are encoded as input to the SVR model, and a prediction model is obtained through training. In actual applications, the currently recognized object attributes are input into the trained SVR model to predict the initial gripping force. The controller applies the appropriate force to grip the object by controlling the mechanical gripper of the robotic arm.

[0062] Closed-loop control of force: The force sensor is installed on the mechanical gripper of the robot arm. During the gripping process, the feedback signal of the force sensor is monitored in real time, and the adaptive fuzzy PID control algorithm is used to dynamically adjust the force of the mechanical gripper of the robot arm. The deviation between the actual force value fed back by the force sensor and the target force value is and the rate of change of deviation , using fuzzy control rules to adjust the parameters of PID controller For example, when the deviation Large and deviation change rate When is also large, increase the proportional coefficient , speed up the system response speed; when the deviation e is small and the deviation change rate When it is small, reduce the integral coefficient , avoid integral saturation. By continuously adjusting PID parameters, the stability and safety of the clamping process are ensured.

[0063] FANUC robotic arm: It includes a robotic arm body and a mechanical gripper, and is connected to a controller. It can move along a planned path based on the control signals sent by the controller and grip objects with appropriate force.

[0064] Thin-film pressure sensor: A high-precision, high-sensitivity thin-film pressure sensor is installed on the gripper of the robotic arm. It measures the force applied when gripping an object in real time and converts the force signal into an electrical signal that is transmitted to the controller. The thin-film pressure sensor is installed on the gripper of the robotic arm, and a signal line connects the force sensor to the analog input interface of the controller.

[0065] In the embodiments of the present invention, a monocular 3D camera is used, which reduces costs and simplifies system complexity compared to the traditional binocular camera and multi-sensor combination solution; an image processing module is used to accurately obtain the outline, size and pose coordinate information of the object in real time, and combined with a data transmission module and controller, an improved algorithm is used to achieve precise gripping of the object by the robot, reducing the probability of object damage and gripping failure. Specifically, the improved image preprocessing algorithm improves image quality, the Mask R-CNN algorithm assists contour extraction to improve the accuracy of contour extraction, the iteratively optimized PnP algorithm improves the accuracy of pose calculation, and the improved The algorithm makes path planning more efficient and complies with the motion constraints of the robotic arm. The SVR algorithm and adaptive fuzzy PID control algorithm respectively improve the accuracy of initial force determination and the stability of force closed-loop control. It can automatically plan paths and apply appropriate forces according to the characteristics of different objects, thereby improving the flexibility, accuracy and safety of the robotic arm's gripping operations and being suitable for a variety of automation scenarios. The present invention effectively solves the problems of poor flexibility of traditional robotic arm gripping methods, inaccurate image recognition and control of existing technologies, high costs, and complex systems.

[0066] See also Figure 1-Figure 4 An embodiment of the present invention provides an adaptive force manipulator gripping control method based on image recognition, which is applicable to the adaptive force manipulator gripping control system based on image recognition as described above. The method comprises the following steps:

[0067] Collect 2D images and depth information of objects through a monocular 3D camera;

[0068] Use the image processing module to preprocess the image, extract the contour and determine the pose coordinates to obtain the contour, size and pose information of the object;

[0069] The data transmission module transmits the object contour, size and posture coordinate information processed by the image processing module to the controller;

[0070] The controller determines the initial gripping force based on the object's properties and plans the robot's motion path;

[0071] Use the controller to control the robot arm to move along the planned path and grasp the object with the initial gripping force;

[0072] The clamping force is monitored in real time using a thin film pressure sensor and dynamically adjusted using an adaptive fuzzy PID control algorithm;

[0073] In an embodiment of the present invention:

[0074] The image acquisition module collects images of the target object area in real time, including 2D images and depth images;

[0075] The image processing module pre-processes the collected images, extracts contours, and determines the pose coordinates to obtain the object's contour, size, and pose coordinate information. It then uses the SVR model to determine the force required for gripping based on the object's properties.

[0076] The data transmission module transmits the object's position coordinate information and the required force information to the control;

[0077] The controller sends the processed image data to the FANUC robotic arm, controlling the arm to move along the path planned by the improved algorithm, and the gripper to grasp the object with appropriate force.

[0078] Force closed-loop control: The controller first sets the target force value. When the robotic arm grips an object, the thin film pressure sensor converts the real-time force signal into an electrical signal and feeds it back to the controller. An adaptive fuzzy PID control algorithm is used to dynamically adjust the force of the robotic arm's mechanical gripper to ensure the stability and safety of the gripping process. Specifically, the controller compares the actual force value with the target force value: when the actual force value is greater than the target force value, the control signal strength output to the mechanical gripper is reduced to weaken the clamping force; when the actual force value is less than the target force value, the control signal strength is increased to enhance the clamping force, further realizing dynamic real-time monitoring and thus dynamic adjustment. The increase or decrease in the control signal strength refers to the drive signal strength output by the controller to the robotic arm's mechanical gripper. This signal is used to control the gripping force of the gripper, and its essence is an adjustable electrical signal parameter. Its essence is to use the fuzzy PID control algorithm to adjust the parameters of the PID controller. Make real-time adjustments to ensure the stability and safety of the clamping process.

[0079] In addition, the software implementation of the present invention is as follows:

[0080] Camera calibration: Use Zhang's calibration method to calibrate the monocular 3D camera to obtain the camera's intrinsic parameters (focal length, principal point coordinates, etc.) and extrinsic parameters (rotation matrix, translation vector), providing basic data for subsequent pose estimation.

[0081] Image processing algorithms: This algorithm uses the OpenCV library to implement image preprocessing, edge detection, contour extraction, and corner detection. It uses the PnP algorithm, combined with calibration parameters and object feature points, to calculate object pose coordinates, and employs iterative optimization to improve accuracy. Furthermore, the Mask R-CNN algorithm is used to assist with contour extraction.

[0082] Path planning algorithm: implemented in the controller The algorithm plans the gripping path based on the robot workspace and object pose coordinates, taking into account the obstacle distribution and robot joint limitations.

[0083] Force Control Algorithm: A gripping force database is established, collecting object attributes and gripping force data to train an SVR model to determine the initial gripping force. An adaptive fuzzy PID control algorithm is used to implement closed-loop force control, adjusting the gripping force based on force sensor feedback.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive force manipulator gripping control system based on image recognition, comprising a manipulator, wherein the manipulator comprises a manipulator body and a manipulator gripper, characterized in that: Also includes: Image acquisition module, used to collect 2D images and depth information of objects; An image processing module, connected to the image acquisition module, for performing preprocessing, contour extraction, and position coordinate determination on the acquired image; A data transmission module is used to transmit the object contour, size and posture coordinate information processed by the image processing module to the controller; A controller, configured to receive information sent by the data transmission module and control the movement and gripping force of the robotic arm; A thin film pressure sensor installed on the mechanical gripper is used to monitor the gripping force in real time and provide feedback to the controller; The image acquisition module is a monocular 3D camera, and the image acquisition module is installed directly above the working space of the robotic arm, with the optical axis of the lens perpendicular to the plane where the object is located, ensuring that the field of view covers the entire working area of the robotic arm; The image processing module includes: Image preprocessing unit, used for denoising, grayscale and edge detection of images. The denoising adopts Gaussian filtering algorithm, and the size and standard deviation of the filter kernel are estimated according to the image noise variance. Dynamic Adjustment: when Less than the set threshold When using The filter kernel of , the standard deviation is set to 1; when Between and When between The filter kernel has a standard deviation of 1.5; when Greater than When using The filter kernel has a standard deviation of 2; Contour extraction unit, which uses gradient-based contour detection algorithm and Mask R-CNN algorithm to extract object contours; The pose coordinate determination unit uses the PnP algorithm combined with the monocular 3D camera calibration parameters to calculate the three-dimensional position and rotation angle of the object, and improves the accuracy through iterative optimization method. The optimization objective function is: ； in is the actual coordinate of the feature point in the image, is the coordinate obtained by reprojection based on the current pose estimation, and n is the number of feature points; The controller includes: Path planning unit, using improved The algorithm plans the gripping path based on the object pose and the kinematic constraints of the robot arm. The cost function of the path planning includes the joint constraint function: ； in is the angle of the j-th joint of the robot arm, and are the upper and lower limits of the joint angle, and m is the number of joints; The initial force determination unit uses the support vector regression (SVR) model to predict the initial force of the gripping object; The force closed-loop control unit uses an adaptive fuzzy PID control algorithm to dynamically adjust the gripping force and adjust the PID parameters based on the deviation and deviation change rate of the force sensor feedback. ; The improved The algorithm introduces a dynamic weight mechanism to adjust the heuristic function weight according to the obstacle distribution density. When the obstacle density is high, the target point distance weight is increased, and when the obstacle density is low, the weight is reduced to improve the path smoothness.

2. The adaptive force manipulator gripping control system based on image recognition according to claim 1 is characterized in that: The force closed-loop control unit adjusts the parameters of the PID controller in real time according to the feedback signal of the thin film pressure sensor to ensure the stability and safety of the clamping force.

3. The adaptive force manipulator gripping control system based on image recognition according to claim 1 is characterized in that: The robotic arm is a FANUC robotic arm.

4. The adaptive force manipulator gripping control system based on image recognition according to claim 1 is characterized in that: The image processing module is an embedded processor with image processing function.

5. An adaptive force manipulator gripping control method based on image recognition, characterized in that: Applicable to the adaptive force manipulator gripping control system based on image recognition as claimed in claim 4, the method comprising the following steps: Collect 2D images and depth information of objects through a monocular 3D camera; Use the image processing module to preprocess the image, extract the contour and determine the pose coordinates to obtain the contour, size and pose information of the object; The data transmission module transmits the object contour, size and posture coordinate information processed by the image processing module to the controller; The controller determines the initial gripping force based on the object's properties and plans the robot's motion path; Use the controller to control the robot arm to move along the planned path and grasp the object with the initial gripping force; A thin film pressure sensor is used to monitor the clamping force in real time, and an adaptive fuzzy PID control algorithm is used to dynamically adjust the clamping force.

6. The adaptive force manipulator gripping control method based on image recognition according to claim 5 is characterized in that: The contour extraction step comprises: Gaussian filtering, grayscale conversion, and Canny edge detection are used to preprocess the image, and the filter kernel parameters are dynamically adjusted according to the noise level; The Mask R-CNN algorithm is used to assist in extracting the object contour and calculate the actual size of the object in combination with the depth information.

Images