Local pose detection and correction method for intelligent transfer robot

Abstract

The invention discloses a local pose detection and correction method for an intelligent transfer robot. The method comprises the following steps: 1) circular ring target detection; 2) reading multi-ring circle center coordinates; 3) calculating the offset and the offset angle; and 4) correcting local poses. According to the method, the circular ring is used as a visual calibration object, and circular ring model training is performed based on YOLOv8, so that the visual calibration method of the circular ring has relatively high robustness when facing background interference, and the local positioning precision of the robot in a complex environment can be remarkably improved; the circle center is detected based on sub-pixel optimization and an RANSAC-K-means method, and the method has high circle center reading precision and robustness in the aspect of circle center recognition, especially in a multi-circular-ring complex scene; according to the geometric calculation model based on double-ring calibration, the model can accurately describe the position relation of the robot relative to the rings, and a reliable positioning basis is provided.

Description

Technical Field

[0001] The present invention relates to the field of control technology, and in particular to a method for detecting and correcting the local posture of an intelligent handling robot. Background Art

[0002] With the continuous development of smart manufacturing in China and the advancement of the "Industry 4.0" concept, the importance and application prospects of intelligent material handling robots in the production processes of high-end manufacturing will increase significantly. In modern logistics and warehousing, handling robots are widely used for cargo stacking and sorting. Traditional palletizing robots rely on fixed operating procedures and mechanical control, but accurate local positioning and posture correction in complex environments are a major challenge. Therefore, an efficient and accurate local posture detection and correction method is crucial.

[0003] The invention patent document with application number CN202211488204.5 discloses a precision control algorithm for an intelligent handling robot, and publishes the specific structure and working principle, but does not mention the specific control circuit design.

[0004] The invention patent document with application number CN202111447671.9 discloses a material placement correction method for a palletizing robot, announces the structure and correction principle of the palletizing robot, involves the robot structure and correction method, but does not involve specific circuit design and control process.

[0005] Existing patent literature indicates that existing intelligent handling robots primarily focus on their composition and structure, but lack specific circuit design, control processes, and control methods. Furthermore, no method for calibrating the center of a ring based on the robot's station has been discovered. Therefore, it is crucial to design appropriate systems and local posture control methods for developing intelligent handling robots. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for detecting and correcting the local posture of an intelligent handling robot.

[0007] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0008] A method for detecting and correcting the local posture of an intelligent handling robot is disclosed. The robot includes a robotic arm, a visual detection system, and a control system. The robotic arm includes a robotic arm gripper, a steering gear, a lifting platform, a servo motor, a Mecanum wheel, and a robot chassis. The robotic arm gripper, steering gear, lifting platform, servo motor, and Mecanum wheel are mounted on the robot chassis. The visual detection system includes a monocular camera, which is mounted on the lifting platform. When the monocular camera detects a circular ring calibration object at the robot station, the control system controls the servo motor to drive the Mecanum wheel to control the robotic arm for posture correction. The posture calculation of the robot is based on the center coordinates of two circular rings in the visual image detected by the monocular camera. The steps of detecting and correcting the local posture of the robot include:

[0009] 1) Ring target detection

[0010] 10) Dataset Collection: A monocular camera is mounted on a lifting platform and shone vertically downward on the surface of the object to be grasped to collect an image of the object. The collected object image is transmitted to the control system to form a data set;

[0011] 11) Dataset preprocessing: Dataset preprocessing methods include grayscale conversion, Gaussian filtering, edge detection, rotation and scaling, color space conversion, rotation and scaling, background removal, super-resolution reconstruction, multi-scale transformation, and variational denoising;

[0012] 12) Label classification of the dataset: Use the K-means algorithm to divide the edge points into two groups of labels, corresponding to the two circles of different colors in the image;

[0013] 13) Train the YOLOv8 model on the label-split images;

[0014] 14) Using neural networks for ring detection;

[0015] 2) Reading the center coordinates of multiple rings

[0016] 20) Calibrate the monocular camera;

[0017] 21) Obtain the camera's intrinsic parameter matrix and distortion coefficients and process the image distortion;

[0018] 22) binarizing the image;

[0019] 23) Perform Gaussian blur denoising on the image;

[0020] 24) Use the Canny edge detection method to preliminarily extract the target edge points;

[0021] 25) Use sub-pixel edge optimization technology to further improve the accuracy of edge points;

[0022] 26) Use K-means clustering algorithm to divide multiple groups of ring edge points into multiple groups;

[0023] 27) Use the RANSAC algorithm to perform robust fitting on the edge point set of each ring;

[0024] 28) Filter inliers based on distance threshold;

[0025] 29) Refit the circle using the least squares method and read the coordinates of the circle center;

[0026] 3) Calculation of offset and offset angle;

[0027] 4) Local pose correction.

[0028] As a specific implementation, in step 3), the posture calculation of the robot is based on the center coordinates of the two rings in the visual image detected by the monocular camera. Specifically,

[0029] The coordinates of the centers of the two circles in the image are (u ai ,v ai ) and (u bi ,v bi ), the distance between the two rings in the image is d, and the proportional factor between d and the actual center distance D of the two rings is α; the coordinates of the two center points in the image (u ai ,v ai ) and (u bi ,v bi ) is the horizontal distance d ix , vertical distance d iy ; The angle between the two circle centers in the image is θ i ';

[0030] The coordinates of the centers of the two rings in real space are (x ai ,y ai ) and (x bi ,y bi ), the horizontal distance between the centers of the two rings in the actual coordinate system is d axi , the vertical distance is d ayi ; The geometric angle between the centers of the two rings in the actual coordinates is θ i ;

[0031] The angle difference between the robot's target pose and actual pose is θ i ';

[0032] The calculation steps for the offset and offset angle are as follows:

[0033] 31) Calculate the target position and posture of the robot;

[0034]

[0035] 32) Calculate the current position and posture of the robot using the following formula:

[0036]

[0037] 33) The calculation formula of the proportional factor α is:

[0038]

[0039] 34) The robot's offset Δx is calculated by the relative position difference of the two circle center coordinates detected in the image. i and Δy i and attitude error Δθ i， The formula is:

[0040]

[0041] 35) The position error Δx i and Δy i and attitude error Δθ i Input into the control system;

[0042] 36) The control system outputs the control signal u x (t),u y (t) and u θ (t), the formula is:

[0043]

[0044] 37) Convert the coordinates of the center of the ring in the two image coordinate systems to the camera coordinate system;

[0045] 38) Calculate the current position and posture of the robot;

[0046] 39) Calculate the offset and the offset angle.

[0047] As a specific implementation method, the specific process of step 4) is as follows:

[0048] 41) Set the error Δx i and Δy i and attitude error Δθ i Input into the control system;

[0049] 42) The control system outputs the control signal u x (t),u y (t) and u θ (t), the formula is:

[0050]

[0051] 43) The controller controls the robot chassis to perform posture correction;

[0052] 44) The controller controls the robot chassis to perform position correction;

[0053] 45) Robot calibration is completed.

[0054] As a specific implementation manner, the control system includes a microprocessor, a power module, a reset module, an RTC clock crystal oscillator, a main clock crystal oscillator, a backup power supply, an EEPROM module, a Linux system board, a closed-loop drive board, a power module, a reset module, an RTC clock crystal oscillator, a main clock crystal oscillator, a backup power supply, and an EEPROM module. The microprocessor is connected to the microprocessor respectively, the Linux system board and the closed-loop drive board are connected to the microprocessor by wire, the Linux system board is connected to the monocular camera by wire, the closed-loop drive board is connected to the servo motor by wire, and the robot also includes a servo installed on the robotic arm, the servo is connected to the microprocessor by wire, and the servo is connected to the robotic arm gripper.

[0055] As a specific implementation method, the specific process of calibrating a monocular camera is as follows: use a standard checkerboard calibration plate with a 6×9 layout and a grid size of 27mm×27mm to calibrate the camera. After obtaining the camera's intrinsic parameter matrix and distortion coefficient, the image is distortion corrected based on the Zhang Zhengyou calibration method.

[0056] Compared with the prior art, the present invention is beneficial in that:

[0057] 1) This paper uses a ring as a visual calibration object and trains a ring model based on YOLOv8. This makes the ring visual calibration method more robust in the face of background interference and can significantly improve the local positioning accuracy of the robot in complex environments.

[0058] 2) The present invention detects the center of a circle by using sub-pixel optimization and the RANSAC-K-means method. This method has high circle center reading accuracy and robustness in circle center recognition, especially in complex scenes with multiple circles.

[0059] 3) The present invention adopts a geometric inference model based on double-ring calibration, which can accurately describe the position relationship of the robot relative to the ring and provide a reliable positioning basis. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive efforts.

[0061] Figure 1 This is a schematic structural diagram of the intelligent handling robot according to the present invention;

[0062] Figure 2 The inverse kinematics model of the intelligent handling robot described in the present invention;

[0063] Figure 3 This is a block diagram of the control system of the intelligent handling robot described in the present invention;

[0064] Figure 4 This is a circuit diagram of the control system of the intelligent handling robot described in the present invention;

[0065] Figure 5 Schematic diagram of the local pose detection and correction method;

[0066] Figure 6 Beacon diagram for robot sites;

[0067] Figure 7 It is a circle center detection process that integrates YOLOv8 and K-means;

[0068] Figure 8 The process of calculating offset and offset angle and correcting local posture;

[0069] Figure 9 It is the process of detecting and correcting the local posture of the robot;

[0070] Among them: 101, robotic arm gripper; 102, servo; 103, monocular camera; 104, lifting platform; 105, step servo A; 106, Mecanum wheel A; 107, step servo B; 108, storage tray; 109, Mecanum wheel B; 110, step servo C; 111, step servo D; 112, Mecanum wheel C; 113, step servo E; 114, step servo F; 115, Mecanum wheel D; 116, bearing; 117, belt; 118, Single-column robotic arm; 119. Robot chassis; 201. Closed-loop drive board A; 202. Closed-loop drive board B; 203. Closed-loop drive board C; 204. Closed-loop drive board D; 205. Microprocessor; 206. Linux system board; 207. Power supply module; 208. Closed-loop drive board E; 209. Closed-loop drive board F; 213. EEPROM module; 214. Backup power supply; 215. RTC clock crystal oscillator; 216. Main clock crystal oscillator; 217. Reset module. DETAILED DESCRIPTION

[0071] The following is a clear and complete description of the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention:

[0072] A method for detecting and correcting the local posture of an intelligent handling robot. The robot includes a mechanical arm 118, a visual detection system, and a control system.

[0073] See also Figure 1 As shown, the manipulator 118 is a single-column manipulator, including a manipulator gripper 101, a steering gear 102, a lifting platform 104, a servo motor, a Mecanum wheel D115, a storage tray 108, a bearing 116, a belt 117, a single-column manipulator 118 and a robot chassis 119; the manipulator gripper 101, the steering gear 102, the lifting platform 104, the servo motor, the Mecanum wheel D115, the storage tray 108, the bearing 116, the belt 117, the single-column manipulator The robotic arm 118 is mounted on the robot chassis 119. The visual inspection system includes a monocular camera 103, which is mounted on a lifting platform 104. When the monocular camera 103 detects the circular calibration object at the robot station, the control system controls the servo motor to drive the Mecanum wheel D115 to control the robotic arm 118 for posture correction. The posture calculation of the robot is based on the center coordinates of the two circular rings in the visual image detected by the monocular camera 103.

[0074] There are 6 servo motors in total, including stepper servo A105, stepper servo B107, stepper servo C110, stepper servo D111, stepper motor E113, and stepper motor F114. There are 4 Mecanum wheels, including Mecanum wheel A106, Mecanum wheel B109, Mecanum wheel C112, and Mecanum wheel 115D.

[0075] The lifting platform 104 is installed on the robotic arm 118 so as to be movable in the up and down directions. The monocular camera 103, the servo 102 and the robotic arm gripper 101 are installed on the lifting platform 104 in sequence in the up and down directions. The belt 117 is installed on the robotic arm 118 in the vertical direction and is used to drive the lifting platform 104 to move up and down. Specifically, the robotic arm 118 is provided with upper and lower rollers. The belt 117 is tensioned on the two rollers. The two rollers are fixed on the lifting platform 104. The lower roller is installed on the output end of the stepping servo A105. The belt 117 is driven by the stepping servo A105 to drive the lifting platform 104 to move up and down.

[0076] Step servo A105, step servo B107, step servo C110, and step servo D111 are used to drive Mecanum wheel A106, Mecanum wheel B109, Mecanum wheel C112, and Mecanum wheel D115 to rotate, respectively.

[0077] A storage tray 108 is installed above the robot base plate, a step servo E113 is installed below the robot base plate, a bearing 116 is installed above the step servo E113, the bearing 116 is connected to the single-column robotic arm 118, and the step servo E113 drives the bearing 116 to drive the single-column robotic arm 118 to rotate.

[0078] After monocular camera 103 detects the circular calibration object at the robot's station, stepper servos A105, B107, C110, and D111 respectively drive Mecanum wheels A106, B109, C112, and D115, thereby controlling the robot's posture correction. After posture correction, stepper motor F114 drives single-column robotic arm 118 to rotate above storage tray 108. Stepper motor E113 drives belt 117 to lower lifting platform 104 to the storage tray 108. Servo 102 drives robotic arm gripper 101 to grasp the material. If the material is successfully grasped, stepper servo E113 drives single-column robotic arm 118 to rotate above the material placement area at the robot's station and then place the material.

[0079] like Figure 2 As shown, the inverse kinematics model equation of the robot is as follows:

[0080]

[0081] Where: v x and v y is the robot's forward and lateral speed, ω is the robot's angular velocity around its center, R is the wheel radius, W and H are half of the robot's chassis wheelbase and front and rear wheelbase, respectively, ω i (i=1,2,3,4) are the angular velocities of the four Mecanum wheels respectively.

[0082] like Figure 3As shown, the robot control system includes a microprocessor 205, a power module 207, a reset module 217, an RTC clock crystal oscillator 215, a main clock crystal oscillator 216, a backup power supply 214, an EEPROM module 213, a Linux system board 206, a closed-loop drive board, a power module 207, a reset module 217, an RTC clock crystal oscillator 215, a main clock crystal oscillator 216, a backup power supply 214, and an EEPROM module 213. The microprocessor is connected to the microprocessor respectively, the Linux system board and the closed-loop drive board are connected to the microprocessor by wire, the Linux system board is connected to the monocular camera by wire, the closed-loop drive board is connected to the servo motor by wire, and the robot further includes a servo installed on the robotic arm, the servo is connected to the microprocessor by wire, and the servo is connected to the robotic arm gripper.

[0083] Here, the closed-loop drive board includes closed-loop drive board A, closed-loop drive board B, closed-loop drive board C, closed-loop drive board D, closed-loop drive board E, and closed-loop drive board F, which correspond to step servo A, step servo B, step servo C and step servo D, step servo E, and step servo F respectively.

[0084] Specifically, the microprocessor 205 is connected to the Linux system board 206 by wire, and the Linux system board 206 is connected to the monocular camera 103 by wire; the microprocessor 205 is connected to the closed-loop drive board A201 by wire, and the closed-loop drive board A201 is connected to the step servo A105 by wire, and the step servo A105 is connected to the Mecanum wheel A106; the microprocessor 205 is connected to the closed-loop drive board A201 by wire, and the closed-loop drive board B202 is connected to the step servo B107 by wire, and the step servo B107 is connected to the Mecanum wheel B109; the microprocessor 205 is connected to the closed-loop drive board C203 is connected by wire, the closed-loop drive board C201 is connected by wire to the step servo C110, and the step servo C110 is connected to a Mecanum wheel C112; the microprocessor 205 is connected by wire to the closed-loop drive board D204, and the closed-loop drive board D204 is connected by wire to the step servo D111, and the step servo D111 is connected to a Mecanum wheel D115; the closed-loop drive board E208 is connected by wire to the step servo E113; the closed-loop drive board F209 is connected by wire to the step servo F114; the microprocessor 205 is connected by wire to the servo 102, and the servo 102 is connected to the robotic arm gripper 101.

[0085] like Figure 4 As shown in Figure 1, the microprocessor reset circuit includes a 10kΩ pull-up resistor R29 and a 10μF filter capacitor C22. One end of the pull-up resistor R29 is connected to the power supply VCC3.3 terminal and the other end is connected to the microcontroller's pin 7. One end of the filter capacitor C22 is connected to the microcontroller's pin 7 and the other end is connected to the power supply GND terminal.

[0086] The microcontroller's clock circuit consists of a main clock circuit and an RTC clock circuit. The main clock circuit uses a 12MHz crystal oscillator Y1. Its terminals are connected to the power supply GND terminal via 22pF capacitors C20 and C21, respectively, and are also connected to pins 5 and 6 of the microcontroller. The RTC clock circuit uses a 32.768kHz crystal oscillator Y2. Its terminals are connected to the power supply GND terminal via 12pF capacitors C23 and C24, respectively, and are also connected to pins 3 and 4 of the microcontroller.

[0087] The microcontroller's power management circuit includes an independent power supply design and a two-stage voltage regulation design. The TPS5430 voltage regulator IC3 provides an independent power supply design for the monocular camera 103. Pin 7 of the TPS5430 is connected to the power supply VCC12, and pin 6 of the TPS5430 is connected to the power supply GND terminal. Pin 4 of the TPS5430 is connected to the vision module power input terminal through a voltage divider by resistors R5 and R6. Pin 1 of the TPS5430 is connected to inductor L3, the other end of which is connected to the vision module power input terminal. Pin 6 of the TPS5430 is connected to capacitor C3, the other end of which is connected to the power supply GND terminal. The TPS5430 voltage regulator IC2 provides an independent power supply design for the servo 103. The 5th pin of the TPS5430 is connected to the power supply VCC12, the 6th pin of the TPS5430 is connected to the power supply GND terminal, and the 4th pin of the TPS5430 is connected to the power input terminal of the servo 103 through the voltage divider of resistors R3 and R4. The 1st pin of the TPS5430 is connected to the inductor L2, and the other end of the inductor L2 is connected to the power input terminal of the servo driver module.

[0088] The microcontroller power supply uses a two-stage DC-DC and LDO voltage regulator design. Pin 7 of the RY3420U3 is connected to the power supply VCC12, pin 6 of the RY3420IC1 is connected to the power supply GND, pin 1 of the RY3420 is connected to inductor L1, the other end of inductor L1 is connected to capacitor C1, and the other end of capacitor C1 is connected to the power supply GND. Pin 4 of the RY3420 is connected to pin 1 of the TPS5430. Pin 1 of the TPS5430 is connected to pin 4 of the RY3420, pin 2 of the TPS5430 is connected to the power supply GND, pin 1 of the TPS5430 is connected to pin 4 of the RY3420 through resistor R7, pin 4 of the TPS5430 is connected to the microcontroller power input through a voltage divider between resistors R1 and R2, and pin 5 of the TPS5430 is connected to inductor L1, the other end of inductor L1 is connected to the microcontroller power input.

[0089] The microcontroller's storage circuit includes a backup power supply circuit and an EEPROM storage circuit. The backup power supply circuit uses a CR2032 button battery BT1, the positive electrode of which is connected to the VCC3.3 terminal through a diode D7, and the negative electrode is connected to the power supply GND terminal. The EEPROM storage circuit uses an AT24C02 memory chip U4, with pins 1 to 3 of the memory chip U4 connected to the power supply GND terminal, pin 4 connected to the power supply GND terminal, pin 5 connected to pin 24 of the microcontroller, and pin 6 connected to pin 25 of the microcontroller. The robot chassis drive module 210 includes four UART-driven closed-loop drive boards A201, B202, C203, and D204 connected to pins 68 and 69 of the microcontroller. The robot arm drive module 212 includes closed-loop drive boards E208 and F209 driven by UART communication, and a servo 102.

[0090] The steps of robot local posture detection and correction are shown in Figure 9 Shown, including:

[0091] 1) Ring target detection, such as Figure 7 As shown,

[0092] 10) Dataset Collection: A monocular camera is mounted on a lifting platform and shone vertically downward on the surface of the object to be grasped to collect an image of the object. The collected object image is transmitted to the control system to form a data set;

[0093] 11) Preprocessing the dataset: grayscale, Gaussian filtering, edge detection, rotation and scaling, color space conversion, rotation and scaling, background removal, super-resolution reconstruction, multi-scale transformation, variational denoising, etc. are used to preprocess the dataset.

[0094] 12) Label classification of the dataset: Use the K-means algorithm to divide the edge points into two groups of labels, corresponding to the two circles of different colors in the image;

[0095] 13) Train the YOLOv8 model on the label-split images;

[0096] 14) Using neural networks for ring detection;

[0097] 2) Read the coordinates of the center of multiple rings. The specific steps are as follows: Figure 6 As shown,

[0098] 20) Calibrate the monocular camera;

[0099] 21) Obtain the camera's intrinsic parameter matrix and distortion coefficients and process the image distortion;

[0100] 22) binarizing the image;

[0101] 23) Performing Gaussian blur denoising on the image, specifically smoothing the image by Gaussian blurring to reduce noise interference;

[0102] 24) Use the Canny edge detection method to preliminarily extract the target edge points;

[0103] 25) Use sub-pixel edge optimization technology to further improve the accuracy of edge points;

[0104] 26) Use K-means clustering algorithm to divide multiple groups of ring edge points into multiple groups;

[0105] 27) Use the RANSAC algorithm to perform robust fitting on the edge point set of each ring;

[0106] 28) Filter inliers based on distance threshold;

[0107] 29) Refit the circle using the least squares method and read the coordinates of the circle center;

[0108] 3) Calculation of offset and offset angle;

[0109] 4) Local posture correction.

[0110] Here, in step 2), the detection accuracy and robustness are significantly improved by combining sub-pixel optimization technology with the RANSAC-K-Means algorithm.

[0111] In step 3), the robot's posture calculation is based on the coordinates of the centers of the two circles in the visual image detected by the monocular camera. Specifically, Figure 5 As shown:

[0112] The coordinates of the centers of the two circles in the image are (u ai ,v ai ) and (u bi ,v bi ), the distance between the two rings in the image is d, and the proportional factor between d and the actual center distance D of the two rings is α; the coordinates of the two center points in the image (u ai ,v ai ) and (u bi ,v bi ) is the horizontal distance d ix , vertical distance d iy ; The angle between the two circle centers in the image is θ i ';

[0113] The coordinates of the centers of the two rings in real space are (x ai ,y ai ) and (x bi,y bi ), the horizontal distance between the centers of the two rings in the actual coordinate system is d axi , the vertical distance is d ayi ; The geometric angle between the centers of the two rings in the actual coordinates is θ i ;

[0114] The angle difference between the robot's target pose and actual pose is θ i ';

[0115] The calculation steps for offset and offset angle are as follows, see Figure 8 As shown:

[0116] 31) Calculate the target position and posture of the robot;

[0117]

[0118] 32) Calculate the current position and posture of the robot using the following formula:

[0119]

[0120] 33) The calculation formula of the proportional factor α is:

[0121]

[0122] 34) The robot's offset Δx is calculated by the relative position difference of the two circle center coordinates detected in the image. i and Δy i and attitude error Δθ i， The formula is:

[0123]

[0124] 35) The position error Δx i and Δy i and attitude error Δθ i Input into the control system;

[0125] 36) The control system outputs the control signal u x (t),u y (t) and u θ (t), the formula is:

[0126]

[0127] 37) Convert the coordinates of the center of the ring in the two image coordinate systems to the camera coordinate system;

[0128] 38) Calculate the current position and posture of the robot;

[0129] 39) Calculate the offset and the offset angle.

[0130] The specific process of step 4) is as follows:

[0131] 41) Set the error Δx i and Δy i and attitude error Δθ i Input into the control system;

[0132] 42) The control system outputs the control signal u x (t),u y (t) and u θ (t), the formula is:

[0133]

[0134] 43) The controller controls the robot chassis to perform posture correction;

[0135] 44) The controller controls the robot chassis to perform position correction;

[0136] 45) Robot calibration is completed.

[0137] It should be emphasized that the above are only preferred embodiments of the present invention and do not constitute any form of limitation to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of this case.

Claims

1. A method for detecting and correcting the local posture of an intelligent handling robot, characterized in that: The robot includes a robotic arm, a visual detection system, and a control system. The robotic arm includes a robotic arm gripper, a steering gear, a lifting platform, a servo motor, a Mecanum wheel, and a robot chassis. The robotic arm gripper, the steering gear, the lifting platform, the servo motor, and the Mecanum wheel are installed on the robot chassis. The visual detection system includes a monocular camera, which is installed on the lifting platform. When the monocular camera detects a circular calibration object at the robot station, the control system controls the servo motor to drive the Mecanum wheel to control the robotic arm for posture correction. The posture calculation of the robot is based on the center coordinates of the two circular rings in the visual image detected by the monocular camera. The steps of detecting and correcting the local posture of the robot include: 1) Ring target detection 10) Dataset Collection: A monocular camera is mounted on a lifting platform and shone vertically downward on the surface of the object to be grasped to collect an image of the object. The collected object image is transmitted to the control system to form a data set; 11) Dataset preprocessing: Dataset preprocessing methods include grayscale conversion, Gaussian filtering, edge detection, rotation and scaling, color space conversion, rotation and scaling, background removal, super-resolution reconstruction, multi-scale transformation, and variational denoising; 12) Label classification of the dataset: Use the K-means algorithm to divide the edge points into two groups of labels, corresponding to the two circles of different colors in the image; 13) Train the YOLOv8 model on the label-split images; 14) Using neural networks for ring detection; 2) Reading the center coordinates of multiple rings 20) Calibrate the monocular camera; 21) Obtain the camera's intrinsic parameter matrix and distortion coefficients and process the image distortion; 22) binarizing the image; 23) Perform Gaussian blur denoising on the image; 24) Use the Canny edge detection method to preliminarily extract the target edge points; 25) Use sub-pixel edge optimization technology to further improve the accuracy of edge points; 26) Use K-means clustering algorithm to divide multiple groups of ring edge points into multiple groups; 27) Use the RANSAC algorithm to perform robust fitting on the edge point set of each ring; 28) Filter inliers based on distance threshold; 29) Refit the circle using the least squares method and read the coordinates of the circle center; 3) Calculation of offset and offset angle; 4) Local pose correction.

2. The method for detecting and correcting the local posture of an intelligent handling robot according to claim 1, characterized in that: In step 3), the robot's posture calculation is based on the coordinates of the centers of the two circles in the visual image detected by the monocular camera. Specifically, The coordinates of the centers of the two circles in the image are (u ai ,v ai ) and (u bi ,v bi ), the distance between the two rings in the image is d, and the proportional factor between d and the actual center distance D of the two rings is α; the coordinates of the two center points in the image (u ai ,v ai ) and (u bi ,v bi ) is the horizontal distance d ix , vertical distance d iy ; The angle between the two circle centers in the image is θ i '; The coordinates of the centers of the two rings in real space are (x ai ,y ai ) and (x bi ,y bi ), the horizontal distance between the centers of the two rings in the actual coordinate system is d axi , the vertical distance is d ayi ; The geometric angle between the centers of the two rings in the actual coordinates is θ i ; The angle difference between the robot's target pose and actual pose is θ i '; The calculation steps for the offset and offset angle are as follows: 31) Calculate the target position and posture of the robot; 32) Calculate the current position and posture of the robot using the following formula: 33) The calculation formula of the proportional factor α is: 34) The robot's offset Δx is calculated by the relative position difference of the two circle center coordinates detected in the image. i and Δy i and attitude error Δθ i， The formula is: 35) The position error Δx i and Δy i and attitude error Δθ i Input into the control system; 36) The control system outputs the control signal u x (t),u y (t) and u θ (t), the formula is: 37) Convert the coordinates of the center of the ring in the two image coordinate systems to the camera coordinate system; 38) Calculate the current position and posture of the robot; 39) Calculate the offset and the offset angle.

3. The method for detecting and correcting the local posture of an intelligent handling robot according to claim 2, characterized in that: The specific process of step 4) is as follows: 41) Set the error Δx i and Δy i and attitude error Δθ i Input into the control system; 42) The control system outputs the control signal u x (t),u y (t) and u θ (t), the formula is: 43) The controller controls the robot chassis to perform posture correction; 44) The controller controls the robot chassis to perform position correction; 45) Robot calibration is completed.

4. The method for detecting and correcting the local posture of an intelligent handling robot according to claim 1, wherein: The control system includes a microprocessor, a power module, a reset module, an RTC clock crystal oscillator, a main clock crystal oscillator, a backup power supply, an EEPROM module, a Linux system board, a closed-loop drive board, a power module, a reset module, an RTC clock crystal oscillator, a main clock crystal oscillator, a backup power supply, and an EEPROM module. The microprocessors are respectively connected to the microprocessors, the Linux system board and the closed-loop drive board are respectively connected to the microprocessors by wire, the Linux system board is connected to the monocular camera by wire, and the closed-loop drive board is connected to the servo motor by wire. The robot also includes a servo installed on the robotic arm, the servo is connected to the microprocessor by wire, and the servo is connected to the robotic arm gripper.

5. The method for detecting and correcting the local posture of an intelligent handling robot according to claim 1, wherein: The specific process of calibrating a monocular camera is as follows: a standard checkerboard calibration plate with a 6×9 layout and a grid size of 27mm×27mm is used to calibrate the camera. After obtaining the camera's intrinsic parameter matrix and distortion coefficient, the image is corrected for distortion based on the Zhang Zhengyou calibration method.

Images