A hand-eye calibration method and system for hair transplantation robot

Abstract

The application discloses a kind of hand-eye calibration method and system for hair transplant robot, comprising: obtaining the real-time coordinate value of cutter head touching the bottom circle center of calibration board center groove in different height plane, the bottom circle center is under the calibration point pair set of camera coordinate system and the initial point pair set of corresponding cutter head under the arm coordinate, the difference value of initial point pair set and real-time coordinate value is calculated, the total set of difference point pair and calibration point pair total set are obtained, and the optimal hand-eye conversion relationship is constructed.According to the coordinate value of follicle center point under the camera coordinate system and the initial point coordinate value of corresponding cutter head under the arm base coordinate system, the real-time coordinate value of cutter head touching follicle center point is calculated by optimal hand-eye conversion relationship, and the real-time coordinate value is used to control the movement of actuator by the arm.The calibration problem of eye on hand is converted into optimization problem based on difference value, improve the accuracy of calibration calculation, can realize the accurate positioning of follicle in operation for hair transplant robot, improve positioning accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hand-eye calibration, and particularly relates to a hand-eye calibration method and system for a hair transplant robot. BACKGROUND

[0002] In a traditional hair transplant surgery, a doctor needs to hold a surgical tool for a long time to extract a hair follicle and maintain a high level of tension, and sometimes the surgery fails due to physical exhaustion. Nowadays, high-precision positioning of a hair transplant robot can make hair follicle extraction more accurate, which is conducive to wound healing of a patient, reduces a trauma surface and a blood loss amount, and reduces pain. Moreover, through a hand-eye calibration method, the hair transplant robot can be accurately positioned at a target position under visual guidance, and the surgery quality is improved.

[0003] A traditional calibration method relies on manual experience to manually operate a calibration plate and control a mechanical arm and a camera to calibrate. In the operation process, it is relatively cumbersome, time-consuming and labor-consuming. A center point of the calibration plate is manually taken by a traditional manual method, a center point coordinate is judged by a naked eye, and a manual selection process is time-consuming and prone to errors. A robot often adopts an eye-on-hand form, that is, a camera is fixed on a mechanical arm, a hair follicle has a small structure, and a scanning imaging distance of a high-precision structured light camera is limited, which easily leads to poor calibration accuracy. SUMMARY

[0004] The purpose of the present application is to provide a hand-eye calibration method and system for a hair transplant robot to solve one or more technical problems in the prior art and at least provide a beneficial choice or create conditions.

[0005] The solution to the technical problem of the present application is to provide a hand-eye calibration method and system for a hair transplant robot.

[0006] According to the embodiment of the first aspect of the present application, a hand-eye calibration method for a hair transplant robot is provided, comprising:

[0007] A mechanical arm base coordinate system corresponding to a mechanical arm and a camera coordinate system corresponding to a structured light camera are established, a tool center point is set at a tool bit tip of a tool body, and the structured light camera and the tool body are arranged on an executor at an end of the mechanical arm;

[0008] In the same height plane, a real-time coordinate value of a bottom circle center of a center groove touched by the tool bit to the calibration plate is obtained under the mechanical arm base coordinate system, the mechanical arm is controlled to move, a set of calibration points of the bottom circle center under the camera coordinate system is obtained, and a set of initial points of the tool bit under the mechanical arm base coordinate system is obtained, a difference value point set is obtained by difference calculation on the set of initial points and the real-time coordinate value;

[0009] Adjust the height of the calibration board, determine whether the adjustment frequency meets the set height variation frequency, if yes, output the difference point pair set and the calibration point pair set, and construct the optimal hand-eye conversion relationship according to the difference point pair set and the calibration point pair set;

[0010] If not, reacquire data to calculate the difference point pair set of the next height plane;

[0011] Obtain the coordinate value of the hair follicle center point in the camera coordinate system and the initial point coordinate value of the corresponding tool bit in the mechanical arm base coordinate system, and calculate the real-time coordinate value of the tool bit touching the hair follicle center point in the mechanical arm base coordinate system through the optimal hand-eye conversion relationship;

[0012] According to the real-time coordinate value of the tool bit touching the hair follicle center point, control the motion of the actuator at the end of the mechanical arm.

[0013] Further, the real-time coordinate value of the tool bit touching the center groove bottom circle center of the calibration board in the mechanical arm base coordinate system specifically includes:

[0014] The actuator is provided with a pressure sensor, and the pressure sensor is used to detect the pressure feedback by the tool bit and read the feedback pressure value;

[0015] The control mechanical arm moves the tool bit to the center groove bottom circle center of the calibration board;

[0016] When the pressure sensor reads the feedback pressure value, it is considered that the tool bit has touched the center groove bottom circle center of the calibration board, and the mechanical arm stops moving, at which time the coordinate value of the tool bit is recorded to obtain the real-time coordinate value.

[0017] Further, the real-time coordinate value of the tool bit touching the center groove bottom circle center of the calibration board in the mechanical arm base coordinate system further includes:

[0018] After obtaining the real-time coordinate value, the mechanical arm controls the actuator to move and lifts the tool bit upward by a set imaging distance, so that the tool bit reaches the imaging initial position.

[0019] Further, the control mechanical arm motion, acquiring the calibration point pair set of the bottom circle center in the camera coordinate system and acquiring the initial point pair set of the corresponding tool bit in the mechanical arm coordinate specifically includes:

[0020] Plan the motion step of the mechanical arm, control the motion of the mechanical arm according to the motion step, and the structured light camera collects the bottom circle image of the center groove;

[0021] Based on the point cloud template matching algorithm, the coordinate value of the bottom circle center in the camera coordinate system is obtained to form the calibration point pair set A={a1, a2, …, a n};

[0022] When the structured light camera collects the bottom circle image of the central groove, the initial point coordinate value of the tool head in the mechanical arm coordinate is obtained, and a corresponding initial point pair set D = {d1, d2, …, d n} in the mechanical arm coordinate of the tool head is formed.

[0023] Further, the difference value calculation between the initial point pair set and the real-time coordinate value is performed to obtain a difference point pair set, which specifically includes:

[0024] Each coordinate value in the initial point pair set D = {d1, d2, …, d n} is subtracted by the real-time coordinate value c to obtain a difference point pair set, wherein the difference point pair set is B = {d1-c, d2-c, …, d n -c}.

[0025] Further, the height of the calibration plate is adjusted, and it is judged whether the adjustment times meet the set height change times, and if yes, the difference point pair total set and the calibration point pair total set are outputted, and an optimal hand-eye conversion relationship is constructed according to the difference point pair total set and the calibration point pair total set, which specifically includes:

[0026] The height of the calibration plate is adjusted, and the adjustment times are recorded, and it is judged whether the adjustment times are the same as the set height change times;

[0027] If yes, the data acquisition is completed, and a difference point pair total set B m ={B1, B2, …, B m} and a calibration point pair total set A m ={A1, A2, …, A m} of several height planes are outputted.

[0028] Based on the SVD algorithm, the difference point pair total set B m ={B1, B2, …, B m} and the calibration point pair total set A m ={A1, A2, …, A m} are used to construct an optimal hand-eye conversion relationship.

[0029] According to the embodiment of the second aspect of the application, a hand-eye calibration system for a hair transplant robot is provided, which comprises a mechanical arm, an executor, a structured light camera, a tool body, a calibration plate, a coordinate system construction module, a data acquisition module, a difference value calculation module, an adjustment times judgment module and a hand-eye calibration calculation module.

[0030] The executor is mounted at the end of the mechanical arm, the structured light camera and the tool body are arranged on the executor, the calibration plate is located in the field of view of the structured light camera, the center of the calibration plate is provided with a central groove, and the central groove is a circular groove.

[0031] The coordinate system construction module is configured to establish a mechanical arm base coordinate system corresponding to the mechanical arm and a camera coordinate system corresponding to the structured light camera, and set a tool center point at a tip of a tool head of the tool body;

[0032] The data acquisition module is configured to acquire real-time coordinate values of the tool head touching a center of a bottom circle of a center groove of the calibration plate in the mechanical arm base coordinate system, a set of calibration point pairs of the center of the bottom circle in the camera coordinate system, and a set of initial point pairs of the corresponding tool head in the mechanical arm base coordinate system, and acquire coordinate values of the follicle center point in the camera coordinate system and initial point coordinate values of the corresponding tool head in the mechanical arm base coordinate system.

[0033] The difference calculation module is configured to calculate differences between the set of initial point pairs and the real-time coordinate values to obtain a set of difference point pairs.

[0034] The adjustment frequency judgment module is configured to judge whether an adjustment frequency of the calibration plate satisfies a set height variation frequency, and if yes, output a total set of difference point pairs and a total set of calibration point pairs, and if not, the data acquisition module reacquires data and the difference calculation module calculates a set of difference point pairs of a next height plane.

[0035] The hand-eye calibration calculation module is configured to construct an optimal hand-eye conversion relationship according to the total set of difference point pairs and the total set of calibration point pairs, and calculate real-time coordinate values of the tool head touching the follicle center point in the mechanical arm base coordinate system according to the coordinate values of the follicle center point in the camera coordinate system and the initial point coordinate values of the corresponding tool head in the mechanical arm base coordinate system through the optimal hand-eye conversion relationship.

[0036] Further, the hand-eye calibration system for the hair transplant robot further comprises a pressure sensor.

[0037] The pressure sensor is arranged in an interior of the effector, and is located at a tool tail of the tool body, configured to detect a feedback pressure of the tool head and read a feedback pressure value.

[0038] When the pressure sensor reads the feedback pressure value, it is considered that the tool head has touched the center of the bottom circle of the center groove of the calibration plate, and the mechanical arm stops moving.

[0039] Further, the hand-eye calibration system for the hair transplant robot further comprises a lifting platform.

[0040] The calibration plate is mounted on the lifting platform, the lifting platform is in wireless communication with the mechanical arm, and the lifting platform is configured to adjust a height of the calibration plate.

[0041] The beneficial effects of the present application are: adjusting the height of the calibration board, obtaining the real-time coordinate value of the cutter head touching the center of the bottom circle of the center groove of the calibration board on different height planes, the calibration point pair set of the center of the bottom circle under the camera coordinate system and the initial point pair set of the corresponding cutter head under the mechanical arm coordinate, difference calculation of the initial point pair set and the real-time coordinate value, difference point pair total set of different height planes and calibration point pair total set, and construction of the optimal hand-eye conversion relationship. According to the coordinate value of the hair follicle center point in the camera coordinate system and the initial point coordinate value of the corresponding cutter head in the mechanical arm base coordinate system, the real-time coordinate value of the cutter head touching the hair follicle center point is calculated through the optimal hand-eye conversion relationship, and the real-time coordinate value is used to control the movement of the actuator by the mechanical arm. The eye-in-hand calibration problem is converted into an optimization problem based on the difference, the calibration calculation accuracy is improved, the accurate positioning of the hair follicle in the operation of the hair transplantation robot can be realized, and the positioning accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a schematic flow chart of a hand-eye calibration method for a hair transplantation robot provided by an embodiment of the present application;

[0043] Figure 2 is a structural schematic diagram of a hand-eye calibration system for a hair transplantation robot provided by an embodiment of the present application;

[0044] Figure 3 is a partial structural schematic diagram of a hand-eye calibration system for a hair transplantation robot provided by another embodiment of the present application.

[0045] The drawings show that: 100, mechanical arm, 200, actuator, 210, structured light camera, 220, cutter body, 300, calibration board, 310, center groove, 400, lifting platform. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0047] It should be noted that although the functional modules are divided in the system schematic diagram, in some cases, the steps shown or described in the module division in the system or the order of execution in the flow chart can be different. The terms "first", "second", etc. in the description, claims and above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0048] In the description of the present application, it should be noted that, unless otherwise explicitly defined, the words such as setting, installing, connecting and the like should be understood in a broad sense, and the person skilled in the art can determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0049] The English name of the tool center point is Tool Central Point, abbreviated as TCP. The tool center point (TCP) in the initial state is the origin of the tool coordinate system.

[0050] Referring to Figure 1 According to the first aspect of the embodiment of the present application, a hand-eye calibration method for a hair transplant robot comprises the following steps:

[0051] S100, a mechanical arm corresponding mechanical arm base coordinate system and a structured light camera corresponding camera coordinate system are established, and the tool center point is set at the tool bit tip of the tool body. The structured light camera and the tool body are arranged on the actuator at the end of the mechanical arm.

[0052] In this embodiment, the structured light camera and the tool body are installed on the actuator at the end of the mechanical arm, and the tool center point TCP coordinates are defined to the tool bit tip of the tool body. Compared with the existing adjustment according to the actuator for solving the conversion relationship by using the calibration tool, the embodiment integrates the actuator and the calibration tool for calibration, effectively improving the positioning accuracy of the final machine. By setting the tool center point TCP at the tool bit, the coordinate value of the tool bit in the mechanical arm base coordinate system can be obtained in real time, and the data is substituted into the calibration calculation, effectively improving the positioning accuracy of the final machine.

[0053] S200, when in the same height plane, the real-time coordinate value of the tool bit touching the center of the bottom circle of the center groove of the calibration plate in the mechanical arm base coordinate system is obtained, the movement of the mechanical arm is controlled, the set of calibration points of the bottom circle center in the camera coordinate system and the set of initial points of the tool bit in the mechanical arm base coordinate system are obtained, and the difference between the initial point set and the real-time coordinate value is calculated to obtain the set of difference points.

[0054] The specific implementation process of S200 comprises the following:

[0055] S210, the actuator is provided with a pressure sensor, and the pressure sensor is used for detecting the feedback pressure of the tool bit and reading the feedback pressure value.

[0056] S220, the movement of the mechanical arm is controlled, and the tool bit is moved to the bottom circle center of the center groove of the calibration plate.

[0057] S230, when the pressure sensor reads the feedback pressure value, it is considered that the tool bit has touched the bottom circle center of the center groove of the calibration plate, and the movement of the mechanical arm is stopped. At this time, the coordinate value of the tool bit is recorded to obtain the real-time coordinate value.

[0058] In this embodiment, in the same height plane, the movement of the mechanical arm is controlled to drive the implementer to move so that the cutter head moves to the center of the bottom circle of the center groove of the calibration plate, the pressure sensor detects the feedback pressure of the cutter head, and when the pressure sensor reads the feedback pressure value of the cutter head, that is, the pressure value is not 0, it can be considered that the cutter head has touched the center of the bottom circle of the center groove of the calibration plate, so the mechanical arm stops moving, and at this time, the coordinate value of the cutter head is recorded to obtain the real-time coordinate value c of the cutter head touching the center of the bottom circle of the center groove of the calibration plate in the mechanical arm base coordinate system.

[0059] It should be noted that the first acquisition of the coordinate value of the cutter head touching the center of the bottom circle of the center groove of the calibration plate in the mechanical arm base coordinate system requires the calibration personnel to control the implementer at the end of the mechanical arm to touch the bottom circle of the center groove of the calibration plate, and record the coordinate value of the point in the mechanical arm base coordinate, and then no manual operation is required, reducing the limitation of obtaining data by human experience.

[0060] S240, after obtaining the real-time coordinate value, the mechanical arm controls the implementer to move to lift the cutter head upward by a set imaging distance, so that the cutter head reaches the imaging initial position.

[0061] In this embodiment, in the same height plane, the real-time coordinate value c of the cutter head touching the center of the bottom circle of the center groove of the calibration plate in the mechanical arm base coordinate system is obtained, the mechanical arm controls the implementer to move to lift the cutter head upward by a set imaging distance, so that the cutter head can reach the imaging initial position. Since the bottom circle can be imaged at a certain distance of the structured light camera, the bottom circle of the center groove of the calibration plate can be collected by the structured light camera.

[0062] S250, the movement step of the mechanical arm is planned, and according to the movement step, the movement of the mechanical arm is controlled, and the structured light camera collects the bottom circle image of the center groove.

[0063] In this embodiment, in the same height plane, the movement step of the mechanical arm is planned, and the mechanical arm moves along the x, y and z axes according to the set movement step. When the mechanical arm moves by one step, the structured light camera collects the bottom circle image of the center groove of the calibration plate in real time, obtains point cloud data once, and sends a feedback signal to the controller in the mechanical arm. According to the feedback signal, the mechanical arm executes the movement by the next movement step. Wherein, the bottom circle of the center groove is always within the field of view of the structured light camera, and after the structured light camera completes the collection, a feedback signal will be sent to the mechanical arm, and the mechanical arm continues to execute the movement by the next step.

[0064] It should be noted that when the structured light camera cannot collect the image, a feedback signal of being unable to collect will be sent to the mechanical arm, and the mechanical arm automatically changes the movement direction to quickly adjust to meet the requirements of collecting the image.

[0065] S260, based on the point cloud template matching algorithm, the coordinate value of the bottom circle center in the camera coordinate system is obtained, and a set of calibration point pairs A={a1, a2,..., a n} is formed.

[0066] In this embodiment, in the same height plane, the structured light camera collects the bottom circle image of the center groove of the calibration plate, obtains the point cloud data, and based on the point cloud template matching algorithm, the coordinate value of the bottom circle center in the camera coordinate system is output in real time. As the mechanical arm moves according to the set motion step, the structured light camera and the cutter body on the executor move continuously, and the structured light camera continuously collects images, thereby forming a set of calibration point pairs A={a1, a2,..., a n} compared with the prior art, the center coordinates of the bottom circle of the center groove of the calibration plate are avoided to be obtained artificially, and the efficiency of hand-eye calibration is improved.

[0067] It should be noted that after the structured light camera collects the calibration plate to obtain the point cloud data, a three-dimensional point cloud template of the calibration plate is established, and the center coordinates of the bottom circle of the center groove are segmented. When the three-dimensional point cloud template and the point cloud data of the real-time collected marker plate are matched by the point cloud matching algorithm, as long as the calibration plate is in the field of view of the structured light camera, the coordinate value of the bottom circle center of the center groove in the camera coordinate system can be output in real time, the center point coordinates of the calibration plate are avoided to be obtained artificially, and the efficiency of hand-eye calibration is improved.

[0068] S270, when the structured light camera collects the bottom circle image of the center groove, the initial point coordinate value of the cutter head in the mechanical arm coordinate is obtained, and a corresponding initial point pair set D={d1, d2,..., d n} in the mechanical arm coordinate of the cutter head is formed.

[0069] In this embodiment, in the same height plane, when the structured light camera collects the bottom circle image of the center groove, the initial point coordinate value of the cutter head in the mechanical arm coordinate at this time is obtained, and as the mechanical arm moves according to the set motion step, the structured light camera and the cutter body on the executor move continuously, and a corresponding initial point pair set D={d1, d2,..., d n} in the mechanical arm coordinate of the cutter head is formed.

[0070] S280, subtracting the real-time coordinate value c from the initial point pair set D={d1, d2,..., d n}, a difference point pair set B={d1-c, d2-c,..., d n -c} is obtained.

[0071] In this embodiment, in the same height plane, subtracting the real-time coordinate value c from the obtained initial point pair set D={d1, d2,..., d n}, a difference point pair set B={d1-c, d2-c,..., d n-c}.

[0072] For example, in the first height plane, the real-time coordinate value c1 of the center of the bottom circle of the center groove of the calibration plate touched by the tool head in the mechanical arm coordinate system is obtained, the calibration point pair set A1 = {a1, a2, …, a n} of the center of the bottom circle in the camera coordinate system is obtained, the initial point pair set D1 = {d1, d2, …, d n} of the corresponding tool head in the mechanical arm coordinate is obtained, the initial point pair set D1 = {d1, d2, …, d n} is subtracted from the real-time coordinate value c1 to obtain B1 = {d1-c1, d2-c1, …, d n -c1}.

[0073] In the second height plane, the real-time coordinate value c2 of the center of the bottom circle of the center groove of the calibration plate touched by the tool head in the mechanical arm coordinate system is obtained, the calibration point pair set A2 = {a1, a2, …, a n} of the center of the bottom circle in the camera coordinate system is obtained, the initial point pair set D2 = {d1, d2, …, d n} of the corresponding tool head in the mechanical arm coordinate is obtained, the initial point pair set D2 = {d1, d2, …, d n} is subtracted from the real-time coordinate value c2 to obtain B2 = {d1-c2, d2-c2, …, d n -c2}.

[0074] S300, whether the adjustment number of the calibration plate meets the set height change number, if yes, output the difference point pair total set and the calibration point pair total set, and construct the optimal hand-eye conversion relationship according to the difference point pair total set and the calibration point pair total set.

[0075] The specific implementation process of S300 includes the following:

[0076] S310, adjusting the height of the calibration plate, recording the adjustment number, and judging whether the adjustment number is the same as the set height change number;

[0077] In this embodiment, when the real-time coordinate value c of the center of the bottom circle of the center groove of the calibration plate touched by the tool head in the mechanical arm coordinate system is obtained in one height plane, the calibration point pair set A = {a1, a2, …, a n} of the center of the bottom circle in the camera coordinate system is obtained, the initial point pair set D = {d1, d2, …, d n} of the corresponding tool head in the mechanical arm coordinate is obtained, and the difference point pair set B = {d1-c, d2-c, …, d nAfter the step of -c}, the mechanical arm stops moving, the height of the calibration board is adjusted, the number of adjustments of the calibration board is recorded, and it is determined whether the number of adjustments of the calibration board is the same as the set number of height changes. That is, after all the data and data sets in one height plane are collected, it is confirmed whether the data and data sets of several height planes have been collected.

[0078] If yes, the data collection is completed, and the difference point pair total set B m ={B1, B2, …, B m} and the calibration point pair total set A m ={A1, A2, …, A m} of several height planes are outputted.

[0079] In this embodiment, when the number of adjustments of the calibration board is the same as the set number of height changes, the data collection is completed, and the difference point pair total set B m ={B1, B2, …, B m} and the calibration point pair total set A m ={A1, A2, …, A m} collected in several height planes are outputted, wherein the several height planes can be m height planes.

[0080] For example, in the first height plane, the real-time coordinate value c1 of the center of the bottom circle of the center groove of the calibration board touched by the tool head in the mechanical arm base coordinate system is obtained, the calibration point pair set A1={a1, a2, …, a n} of the center of the bottom circle in the camera coordinate system is obtained, and the initial point pair set D1={d1, d2, …, d n} of the tool head in the mechanical arm coordinate is obtained. The initial point pair set D1={d1, d2, …, d n} and the real-time coordinate value c1 are calculated to obtain B1={d1-c1, d2-c1, …, d n -c1}.

[0081] In the second height plane, the real-time coordinate value c2 of the center of the bottom circle of the center groove of the calibration board touched by the tool head in the mechanical arm base coordinate system is obtained, the calibration point pair set A2={a1, a2, …, a n} of the center of the bottom circle in the camera coordinate system is obtained, and the initial point pair set D2={d1, d2, …, d n} of the tool head in the mechanical arm coordinate is obtained. The initial point pair set D2={d1, d2, …, d n} and the real-time coordinate value c2 are calculated to obtain B2={d1-c2, d2-c2, …, d n -c2}.

[0082] This process continues until the number of adjustments to the calibration plate matches the number of changes in the set height, outputting the total set B of difference points obtained from m height planes. m ={B1,B2,…,B m} and the calibration point pair of the set A m ={A1,A2,…,A m In addition, it will also output the initial point pair of the set D. m ={D1,D2,…,D m} and the real-time coordinate point pair set C = {c1, c2, ..., c n}

[0083] S330, based on the SVD algorithm, uses the difference point pairs to determine the total set B. m ={B1,B2,…,B m} and the calibration point pair of the set A m ={A1,A2,…,A m}, thus constructing the optimal hand-eye conversion relationship.

[0084] In this embodiment, based on the difference point pair set B m ={B1,B2,…,B m} and the calibration point pair of the set A m ={A1,A2,…,A m Establish the hand-eye transformation relationship AX = B, and calculate the difference point pairs B based on the SVD algorithm. m ={B1,B2,…,B m} and the calibration point pair of the set A m ={A1,A2,…,A m The optimal hand-eye conversion matrix is ​​output by processing the rotation matrix R and the translation matrix T. The optimal hand-eye conversion relationship B = A × R + T is constructed based on the rotation matrix R and the translation matrix T.

[0085] By obtaining the coordinates of the center of the bottom circle where the cutter head touches the robot arm in the base coordinate system, and calculating the difference between the coordinates of the center of the bottom circle and the initial point set of the cutter head in the robot arm base coordinate system when the structured light camera captures the image of the bottom circle, an optimization relationship is established between the coordinates of the center of the bottom circle in the camera coordinate system when the camera captures the image of the bottom circle and the difference, and the optimal hand-eye conversion relationship is obtained.

[0086] S400, if not, then re-acquire the data to calculate the difference point pair set for the next height plane.

[0087] In this embodiment, when the number of adjustments of the calibration plate is not equal to the number of changes in the set height, the calibration plate has been adjusted to the next height plane, the real-time coordinate value of the center of the bottom circle of the center groove of the calibration plate touched by the tool tip in the mechanical arm base coordinate system in the next height plane is re-acquired, the set of calibration points of the center of the bottom circle in the camera coordinate system and the initial point set of the corresponding tool tip in the mechanical arm coordinate are acquired, and the initial point set is subtracted from the real-time coordinate value to obtain a difference point set.

[0088] For example, in the second height plane, after obtaining the difference point set B2={d1-c2,d2-c2,…,d n -c2}, the height of the calibration plate is adjusted so that the calibration plate is in the third height plane, it is judged whether the number of adjustments of the calibration plate meets the number of changes in the set height, when the number of adjustments of the calibration plate is not equal to the number of changes in the set height, the mechanical arm is controlled to move the tool tip to the center of the bottom circle of the center groove of the calibration plate, when the pressure sensor reads the feedback pressure value, it is considered that the tool tip has touched the center of the bottom circle of the center groove of the calibration plate, the mechanical arm stops moving, at this time, the coordinate value of the tool tip is recorded to obtain the real-time coordinate value c3 of the center of the bottom circle of the center groove of the calibration plate touched by the tool tip in the mechanical arm base coordinate system in the third height plane, then the set of calibration points of the center of the bottom circle in the camera coordinate system and the initial point set of the corresponding tool tip in the mechanical arm coordinate are acquired according to steps S240 to S280, the initial point set is subtracted from the real-time coordinate value to obtain the difference point set B3={d1-c3,d2-c3,…,d n -c3} in the third height plane, the height of the calibration plate is adjusted again so that the calibration plate is in the fourth height plane, it is judged again whether the number of adjustments of the calibration plate meets the number of changes in the set height, and the cycle is repeated until the number of adjustments of the calibration plate meets the number of changes in the set height.

[0089] S500, the coordinate value of the hair follicle center point in the camera coordinate system and the initial point coordinate value of the corresponding tool tip in the mechanical arm base coordinate system are acquired, and the coordinate value of the tool tip touching the hair follicle center point in the mechanical arm base coordinate system is calculated through the optimal hand-eye conversion relationship.

[0090] In this embodiment, when the structure light camera collects the hair follicle, the coordinate value of the hair follicle center point in the camera coordinate system and the initial point coordinate value of the corresponding tool tip in the mechanical arm base coordinate system are acquired. Through the coordinate value of the hair follicle center point in the camera coordinate system and the constructed optimal hand-eye conversion relationship B=A×R+T, the coordinate difference value of the tool tip in the mechanical arm base coordinate system is obtained, and the coordinate value of the tool tip touching the hair follicle center point in the mechanical arm base coordinate system is calculated according to the coordinate difference value of the tool tip and the initial point coordinate value of the corresponding tool tip in the mechanical arm base coordinate system. The marker data collected by the hand-eye is processed as an optimization problem based on difference, greatly simplifying the hand-eye calibration step of the eye on the hand, simplifying the process and improving the accuracy.

[0091] S600, according to the coordinate value of the tool head touching the hair follicle center point, the motion of the end effector of the mechanical arm is controlled.

[0092] In this embodiment, according to the coordinate value of the tool head touching the hair follicle center point, the motion of the end effector of the mechanical arm is controlled, so that the tool head touches the hair follicle center point and extracts the hair follicle.

[0093] Referring to Figure 2 According to the second aspect of the embodiment of the application, a hand-eye calibration system for a hair transplant robot comprises a mechanical arm 100, an end effector 200, a structured light camera 210, a tool body 220, a calibration board 300, a coordinate system construction module, a data acquisition module, a difference calculation module, an adjustment frequency judgment module, and a hand-eye calibration calculation module. The end of the mechanical arm 100 is provided with the end effector 200, the tool body 220 is arranged on the end effector 200, and the structured light camera 210 is also arranged on the end effector 200. The calibration board 300 is arranged within the field of view of the structured light camera 210, the center of the calibration board 300 is provided with a center groove 310, and the center groove 310 is a circular groove.

[0094] The coordinate system construction module is used to establish a mechanical arm base coordinate system corresponding to the mechanical arm 100 and a camera coordinate system corresponding to the structured light camera 210, and the tool center point TCP is set at the tip of the tool head of the tool body 220.

[0095] The tool center point TCP is set at the tool head, the coordinate value of the tool head in the mechanical arm base coordinate system can be obtained in real time, the data is substituted into the calibration calculation, and the positioning accuracy of the final machine is effectively improved.

[0096] The data acquisition module is used to acquire the real-time coordinate value of the tool head of the tool body 220 touching the bottom center of the center groove 310 of the calibration board 300 in the mechanical arm base coordinate system, acquire the set of calibration points of the bottom center in the camera coordinate system and the set of initial points of the tool head in the mechanical arm coordinate system corresponding to the set of calibration points, and acquire the coordinate value of the hair follicle center point in the camera coordinate system and the initial point coordinate value of the tool head in the mechanical arm base coordinate system corresponding to the coordinate value.

[0097] The difference calculation module is used to calculate the difference between the set of initial points and the real-time coordinate value obtained by the data acquisition module to obtain a set of difference points.

[0098] The adjustment frequency judgment module is used to judge whether the adjustment frequency of the calibration board 300 meets the set height variation frequency. If the adjustment frequency of the calibration board 300 meets the set height variation frequency, the difference calculation module outputs the total set of difference points and the data acquisition module outputs the total set of calibration points.

[0099] If the number of adjustments of the calibration plate 300 does not satisfy the set number of height changes, the data acquisition module reacquires data and the difference calculation module calculates the next set of difference point pairs of the height plane.

[0100] The hand-eye calibration calculation module is configured to construct an optimal hand-eye conversion relationship based on the total set of difference point pairs output by the difference calculation module and the total set of calibration point pairs output by the data acquisition module. Based on the coordinate value of the hair follicle center point in the camera coordinate system and the initial point coordinate value of the tool bit of the tool body 220 in the robot base coordinate system, the real-time coordinate value of the tool bit of the tool body 220 touching the hair follicle center point in the robot base coordinate system is calculated through the optimal hand-eye conversion relationship.

[0101] The tool bit tip of the tool body 220 of the executor 200 is used to avoid the situation that the final calibration result is not ideal due to installation errors of the tool body 220 even after traditional calibration. The tool bit is set behind the tool center point TCP, and the coordinate of the tool bit in the robot base coordinate system is acquired in real time. The data is substituted into the calibration calculation to improve the positioning accuracy of the final machine. The coordinate value of the tool bit touching the bottom circle center in the robot base coordinate system is obtained, and the initial point set of the tool bit in the robot base coordinate system when the bottom circle image is collected by the structured light camera 210 is subtracted to obtain the difference value. The optimal hand-eye conversion relationship is obtained by establishing an optimization relationship between the coordinate value of the center in the camera coordinate system when the bottom circle is collected by the camera and the difference value.

[0102] A hand-eye calibration system for a hair transplant robot further comprises a pressure sensor. The pressure sensor is arranged inside the executor 200, and the pressure sensor is located at the tool tail of the tool body 220 and is used to detect the feedback pressure of the tool bit and read the feedback pressure value. When the pressure sensor reads the feedback pressure value, it is considered that the tool bit of the tool body 220 has touched the bottom circle center of the center groove 310 of the calibration plate 300, and the robot 100 stops moving. The data acquisition module records the coordinate value at this time to obtain the real-time coordinate value of the tool bit of the tool body 220 touching the bottom circle center of the center groove 310 of the calibration plate 300 in the robot base coordinate system in the height plane.

[0103] Reference Figure 3The hand-eye calibration system for the hair transplant robot further comprises a lifting platform 400. The calibration plate 300 is installed on the lifting platform 400, the lifting platform 400 is used to adjust the height of the calibration plate 300, and the lifting platform 400 is in wireless communication with the mechanical arm 100. When the data acquisition module acquires all the data and data sets, the mechanical arm 100 stops moving, the lifting platform 400 receives the feedback signal of the mechanical arm 100, adjusts the height of the calibration plate 300, so that the calibration plate 300 is in the next height plane, and when the adjustment frequency does not satisfy the set height change frequency, the mechanical arm 100 is controlled to move autonomously, the cutter head of the cutter body 220 is moved to the center of the bottom circle of the center groove 310 of the calibration plate 300, and the data acquisition module starts to acquire data. By moving the mechanical arm 100 according to the set motion step length and moving the lifting platform 400 up and down, the calibration diversity of the data is expanded, the problems of manual movement and the need to constantly adjust the structured light camera 210 are reduced, and the accuracy of the hand-eye calibration is improved.

[0104] The preferred embodiments of the application are specifically described above, but the application is not limited to the embodiments described, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the application. These equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A hand-eye calibration method for a hair transplant robot, characterized in that, include: Establish the robot arm base coordinate system corresponding to the robot arm and the camera coordinate system corresponding to the structured light camera. Set the tool center point at the tip of the blade. The structured light camera and the blade are both set on the actuator at the end of the robot arm. When on the same height plane, obtain the real-time coordinate value of the bottom circle center of the cutter head touching the center groove of the calibration plate in the robot arm base coordinate system, control the robot arm to move, obtain the calibration point pair set of the bottom circle center in the camera coordinate system and obtain the corresponding initial point pair set of the cutter head in the robot arm base coordinate system, calculate the difference between the initial point pair set and the real-time coordinate value to obtain the difference point pair set. Among them, the initial point pair set Subtract the real-time coordinate value from each coordinate value in the table. This yields a set of difference point pairs, wherein the set of difference point pairs is... ; Adjust the height of the calibration plate and determine whether the number of adjustments meets the set number of height changes. If so, output the total set of difference point pairs and the total set of calibration point pairs. Based on the total set of difference point pairs and the total set of calibration point pairs, construct the optimal hand-eye conversion relationship. If not, re-acquire the data and calculate the difference point pair set for the next height plane; The coordinates of the hair follicle center point in the camera coordinate system and the initial point coordinates of the corresponding blade head in the robotic arm base coordinate system are obtained. Through the optimal hand-eye conversion relationship, the real-time coordinates of the blade head touching the hair follicle center point in the robotic arm base coordinate system are calculated. The actuator at the end of the robotic arm is controlled to move based on the real-time coordinates of the point where the blade touches the center of the hair follicle. The acquisition of the real-time coordinate values ​​includes The actuator is equipped with a pressure sensor, which is used to detect the pressure fed back by the cutter head and read the feedback pressure value. The control robot arm moves the cutter head to the center of the bottom circle of the central groove on the calibration plate; When the pressure sensor reads the feedback pressure value, it is considered that the cutter head has touched the center of the bottom circle of the central groove of the calibration plate, and the robotic arm stops moving. At this time, the coordinate value of the cutter head is recorded to obtain the real-time coordinate value.

2. The hand-eye calibration method for a hair transplant robot according to claim 1, characterized in that, The method of obtaining the real-time coordinate value of the bottom circle center of the central groove of the calibration plate when the cutter head touches the center groove of the calibration plate in the robot arm base coordinate system also includes: After obtaining the real-time coordinate values, the robotic arm controls the actuator to move, raising the cutter head upwards by the set imaging distance, so that the cutter head reaches the initial imaging position.

3. The hand-eye calibration method for a hair transplant robot according to claim 1, characterized in that, The process of controlling the movement of the robotic arm and obtaining the set of calibration points of the bottom circle center in the camera coordinate system and the set of initial points of the corresponding cutter head in the robotic arm coordinate system specifically includes: The movement step length of the robotic arm is planned, and the movement of the robotic arm is controlled according to the movement step length. The structured light camera acquires the bottom circle image of the central groove. Based on the point cloud template matching algorithm, the coordinates of the bottom circle center in the camera coordinate system are obtained, forming a calibration point pair set. ; When the structured light camera acquires the bottom circle image of the central slot, it obtains the initial point coordinates of the cutter head in the robot arm coordinate system, forming a corresponding set of initial point pairs of the cutter head in the robot arm coordinate system. .

4. The hand-eye calibration method for a hair transplant robot according to claim 1, characterized in that, The process of adjusting the height of the calibration plate, determining whether the number of adjustments meets the set number of height changes, and if so, outputting the total set of difference point pairs and the total set of calibration point pairs, and constructing the optimal hand-eye conversion relationship based on the total set of difference point pairs and the total set of calibration point pairs, specifically includes: Adjust the height of the calibration plate, record the number of adjustments, and determine whether the number of adjustments is the same as the set number of height changes. If so, then complete the data acquisition and output a set of difference point pairs for several height planes. and calibration point set ; Based on the SVD algorithm, according to the difference point pair set and calibration point set The optimal hand-eye conversion relationship is constructed.

5. A hand-eye calibration system for a hair transplant robot, characterized in that, Includes: robotic arm, actuator, structured light camera, cutter body, calibration plate, coordinate system construction module, data acquisition module, difference calculation module, adjustment number judgment module, and hand-eye calibration calculation module; The actuator is installed at the end of the robotic arm, the structured light camera and the blade are both mounted on the actuator, the calibration plate is located within the field of view of the structured light camera, and the calibration plate has a central groove in the center, which is a circular groove. The coordinate system construction module is used to establish the robot arm base coordinate system corresponding to the robot arm and the camera coordinate system corresponding to the structured light camera, and to set the tool center point at the tip of the blade body. The data acquisition module is used to acquire the real-time coordinates of the bottom circle center of the cutter head touching the center groove of the calibration plate in the robot arm base coordinate system when the cutter head touches the center groove of the calibration plate in the same height plane, the calibration point pair set of the bottom circle center in the camera coordinate system and the corresponding initial point pair set of the cutter head in the robot arm base coordinate system, as well as the coordinates of the hair follicle center point in the camera coordinate system and the corresponding initial point coordinates of the cutter head in the robot arm base coordinate system. The difference calculation module is used to calculate the difference between the initial point pair set and the real-time coordinate values ​​to obtain the difference point pair set; The adjustment count judgment module is used to determine whether the adjustment count of the calibration plate meets the set height change count. If yes, it outputs the total set of difference point pairs and the total set of calibration point pairs. If no, the data acquisition module re-acquires the data and the difference calculation module calculates the difference point pair set of the next height plane. The hand-eye calibration calculation module is used to construct the optimal hand-eye conversion relationship based on the set of difference point pairs and the set of calibration point pairs. Based on the coordinates of the hair follicle center point in the camera coordinate system and the initial point coordinates of the corresponding blade in the robot arm base coordinate system, the module calculates the real-time coordinates of the blade touching the hair follicle center point in the robot arm base coordinate system through the optimal hand-eye conversion relationship.

6. The hand-eye calibration system for a hair transplant robot according to claim 5, characterized in that, Also includes: Pressure sensor; The pressure sensor is installed inside the actuator and is located at the tail of the cutter body. It is used to detect the pressure fed back by the cutter head and read the feedback pressure value. When the pressure sensor reads the feedback pressure value, it is considered that the cutter head has touched the bottom circle center of the central groove of the calibration plate, and the robotic arm stops moving.

7. The hand-eye calibration system for a hair transplant robot according to claim 5, characterized in that, Also includes: Lifting platform; The calibration plate is mounted on the lifting platform, which is wirelessly connected to the robotic arm. The lifting platform is used to adjust the height of the calibration plate.

Images