Robotic arm registration method, apparatus, system, and electronic device

Abstract

Embodiments of the present application provide a kind of mechanical arm registration method, device, system and electronic equipment, applied to mechanical arm registration system, system includes: mechanical arm, physical probe, calibration block, optical camera and optical probe;Calibration block includes multiple recess points;Method includes: determining the first coordinate value of recess point under optical camera coordinate system based on optical probe;Determine the second coordinate value of recess point under the base coordinate system of mechanical arm based on physical probe;Determine the affine transformation matrix of optical camera coordinate system and the base coordinate system of mechanical arm based on the first coordinate value and the second coordinate value;By setting fixed multiple recess points, and respectively determining the coordinate value of the recess point under optical camera coordinate system and the base coordinate system of mechanical arm, the affine transformation matrix is determined by fitting to the corresponding coordinate value, to realize the registration of mechanical arm, the error caused by optical reference array movement is reduced, the registration precision is improved, and user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of robotic arm positioning technology, and in particular to a robotic arm registration method, apparatus, system and electronic device. Background Technology

[0002] Surgical robotic arms, with their advantages of high positioning accuracy and repeatability, are widely used in image-guided orthopedic surgeries. A commonly used surgical robotic arm navigation and positioning method is briefly described below: An optical reference array, which can be positioned by an optical positioning camera (binocular vision positioning camera), is rigidly fixed to the end effector of the robotic arm; the end effector is controlled to move to several postures, and the conversion relationships between the robotic arm base and the end effector flange, and between the optical positioning camera and the end effector reference array, are collected under these postures; the conversion relationship between the optical positioning camera and the robotic arm base is calculated using these two sets of conversion relationships, thus completing the registration of the robotic arm.

[0003] However, in existing robotic arm registration systems, an optical reference array is fixed at the end of the robotic arm. During the registration process, the end of the robotic arm moves along with the optical reference array, and a positioning camera is used to collect the pose of the reference array. Because the end of the robotic arm rotates during movement, it inevitably causes a large error in the pose data collected by the positioning camera, affecting the registration accuracy of the robotic arm. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a robotic arm registration method, device, system and electronic device. By setting a plurality of fixed groove points and determining the coordinate values ​​of the groove points in the optical camera coordinate system and the robotic arm base coordinate system respectively, and then fitting the corresponding coordinate values ​​to determine the affine transformation matrix, the registration of the robotic arm is achieved. This method reduces the error caused by the movement of the optical reference array, improves the registration accuracy and enhances the user experience.

[0005] In a first aspect, embodiments of the present invention provide a robotic arm registration method applied to a robotic arm registration system. The robotic arm registration system includes: a robotic arm, a physical probe, a calibration block, an optical camera, and an optical probe; the calibration block includes multiple groove points; the robotic arm registration method includes: determining a first coordinate value of the groove point in the optical camera coordinate system based on the optical probe; determining a second coordinate value of the groove point in the robotic arm base coordinate system based on the physical probe; and determining an affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first and second coordinate values.

[0006] In some preferred embodiments of the present invention, the calibration block includes a plurality of grooves that are not located on the same straight line; each groove is in close contact with the tip of a physical probe; each groove is also in close contact with the tip of an optical probe.

[0007] In some preferred embodiments of the present invention, the optical probe includes: a plurality of reflective spheres; an optical camera pre-stores information about the optical probe, including: an optical probe coordinate system and coordinate values ​​of the reflective spheres in the optical probe coordinate system; the step of determining the first coordinate value of the groove point in the optical camera coordinate system based on the optical probe includes: placing the tip of the optical probe into the groove point and determining the coordinate value of the reflective spheres in the optical camera coordinate system; and determining the first coordinate value of the groove point in the optical camera coordinate system based on the coordinate value of the reflective spheres in the optical camera coordinate system and the information of the optical probe.

[0008] In some preferred embodiments of the present invention, the groove point includes: a reference groove point; the step of determining the second coordinate value of the groove point in the robot arm base coordinate system based on a physical probe includes: moving the robot arm and placing the tip of the physical probe at the reference groove point; keeping the tip of the physical probe at the reference groove point and continuing to move the robot arm to maintain multiple postures, recording the positional relationship between the robot arm end flange coordinate system and the robot arm base coordinate system in each posture, the positional relationship including: the rotation matrix and translation vector from the robot arm end flange coordinate system to the robot arm base coordinate system; and determining the second coordinate value of the groove point in the robot arm base coordinate system based on the rotation matrix and translation vector.

[0009] In some preferred embodiments of the present invention, the step of determining the second coordinate value of the groove point in the robot arm base coordinate system based on the rotation matrix and translation vector includes: determining the second coordinate value of the reference groove point in the robot arm base coordinate system based on the translation vector; determining the coordinate value of the reference groove point in the robot arm end flange coordinate system based on the second coordinate value of the reference groove point in the robot arm base coordinate system, the rotation matrix, and the translation vector; and determining the second coordinate values ​​of the other groove points (excluding the reference groove point) in the robot arm base coordinate system based on the coordinate value of the reference groove point in the robot arm end flange coordinate system, the rotation matrix, and the translation vector.

[0010] In some preferred embodiments of the present invention, the step of determining the second coordinate value of the reference groove point in the coordinate system of the robot arm base based on the translation vector includes: obtaining translation vectors under various postures as a set of sphere center fitting points; determining the sphere center coordinate value based on the least squares method and the sphere center fitting point set; and using the sphere center coordinate value as the second coordinate value of the reference groove point in the coordinate system of the robot arm base.

[0011] In some preferred embodiments of the present invention, the step of determining the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first coordinate value and the second coordinate value to achieve robotic arm registration includes: determining a starting point set based on multiple first coordinate values; determining a target point set based on multiple second coordinate values; wherein both the starting point set and the target point set are ordered point sets, and the number of elements in the starting point set and the target point set are equal; determining the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the starting point set and the target point set to achieve robotic arm registration.

[0012] Secondly, embodiments of the present invention provide a robotic arm registration device applied to a robotic arm registration system. The robotic arm registration system includes: a robotic arm, a physical probe, a calibration block, an optical camera, and an optical probe; the calibration block includes multiple groove points; the robotic arm registration device includes: an optical coordinate determination module, used to determine the first coordinate value of the groove point in the optical camera coordinate system based on the optical probe; a mechanical coordinate determination module, used to determine the second coordinate value of the groove point in the robotic arm base coordinate system based on the physical probe; and a robotic arm registration module, used to determine the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first and second coordinate values, thereby achieving robotic arm registration.

[0013] Thirdly, embodiments of the present invention provide a robotic arm registration system for performing the above-described robotic arm registration method.

[0014] Fourthly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the robotic arm registration method described above.

[0015] The embodiments of the present invention bring the following beneficial effects: This invention provides a robotic arm registration method, apparatus, system, and electronic device, applied to a robotic arm registration system. The robotic arm registration system includes: a robotic arm, a physical probe, a calibration block, an optical camera, and an optical probe; the calibration block includes multiple groove points; the robotic arm registration method includes: determining a first coordinate value of the groove point in the optical camera coordinate system based on the optical probe; determining a second coordinate value of the groove point in the robotic arm base coordinate system based on the physical probe; determining an affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first and second coordinate values; by setting multiple fixed groove points and determining the coordinate values ​​of the groove points in the optical camera coordinate system and the robotic arm base coordinate system respectively, and then fitting the corresponding coordinate values ​​to determine the affine transformation matrix, the registration of the robotic arm is achieved. This method reduces the error caused by the movement of the optical reference array, improves the registration accuracy, and enhances the user experience. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the positional relationship between an optical reference array and an optical positioning camera in a conventional robotic arm registration method provided by an embodiment of the present invention. Figure 2 A flowchart of a robotic arm registration method provided in an embodiment of the present invention; Figure 3 A schematic diagram of an optical probe provided in an embodiment of the present invention; Figure 4 A schematic diagram of a robotic arm coordinate system provided in an embodiment of the present invention; Figure 5 A schematic diagram of a calibration block provided in an embodiment of the present invention; Figure 6 A schematic diagram illustrating an optical probe-based calibration position according to an embodiment of the present invention; Figure 7 A schematic diagram illustrating a physical probe-based position calibration method provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of a robotic arm registration device provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0018] Icons: 310-First reflective sphere; 410-Physical probe; 420-Robotic arm; 510-Calibration block; 610-Optical positioning camera; 810-Optical coordinate determination module; 820-Mechanical coordinate determination module; 830-Robotic arm registration module; 900-Memory; 901-Processor; 902-Bus; 903-Communication interface. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0021] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0022] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] Furthermore, terms such as "horizontal," "vertical," and "sag" do not imply that components must be absolutely horizontal or suspended, but rather that they can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal relative to "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0024] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0025] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0026] Surgical robotic arms, with their advantages of high positioning accuracy and repeatability, are widely used in image-guided orthopedic surgeries. A commonly used surgical robotic arm navigation and positioning method is briefly described below: An optical reference array, which can be positioned by an optical positioning camera (binocular vision positioning camera), is rigidly fixed to the end effector of the robotic arm; the end effector is controlled to move to several postures, and the conversion relationships between the robotic arm base and the end effector flange, and between the optical positioning camera and the end effector reference array, are collected under these postures; the conversion relationship between the optical positioning camera and the robotic arm base is calculated using these two sets of conversion relationships, thus completing the registration of the robotic arm.

[0027] Typically, see Figure 1 The diagram illustrates the positional relationship between an optical reference array and an optical positioning camera in a conventional robotic arm registration method provided by an embodiment of the present invention. The optical reference array has several (at least three) reflective spheres that can be identified by the optical positioning camera. The centers of the reflective spheres belonging to the same reference array are located on the same plane. The positioning camera system determines the overall spatial pose (translation + rotation) of the optical reference array by identifying the spatial positions of each reflective sphere on the reference array. The difference between the actual spatial pose of the optical reference array and the array pose output by the positioning camera system is the error of the positioning camera system. In practical use, it has been found that the error of the positioning camera system is affected by the attitude (rotation) of the reference array in the positioning camera space: keeping the position (translation) of the reference array under the positioning camera constant, the positioning error is small when the plane normal of the reference array is parallel to the central axis of the positioning camera's field of view; when the angle between the plane normal and the camera's field of view normal increases, the positioning error increases, and when the angle exceeds a certain range, the positioning error increases rapidly.

[0028] Based on this, embodiments of the present invention provide a robotic arm registration method, apparatus, system, and electronic device, which are described below through embodiments.

[0029] Example 1 This invention provides a robotic arm registration method applied to a robotic arm registration system. The robotic arm registration system includes: a robotic arm 420, a physical probe 410, a calibration block 510, an optical camera, and an optical probe. The calibration block 510 includes multiple grooves; the surface of the calibration block 510 includes at least three grooves, and all grooves are not on the same straight line. Each groove closely fits the tip of the physical probe 410, and each groove also closely fits the tip of the optical probe. See also... Figure 2 The flowchart shown in this embodiment of the invention provides a robotic arm registration method, which includes the following steps: Step S102: Determine the first coordinate value of the groove point in the optical camera coordinate system based on the optical probe; Specifically, the optical probe includes multiple reflective spheres, specifically at least three that are not on the same straight line. However, if more than three reflective spheres are used, all reflective spheres must be on the same plane. Since the relative positions of the multiple reflective spheres and the tip of the optical probe are fixed, the position of the optical probe tip can be calculated after the optical camera captures the position of the reflective spheres. Furthermore, when the tip of the optical probe is in close contact with the groove on the calibration block 510, the position of the optical probe tip is the position of the groove.

[0030] For example, see Figure 3 The schematic diagram shown in this embodiment of the invention provides an optical probe comprising four reflective spheres. The reflective sphere closest to the tip of the optical probe is designated as the first reflective sphere 310, wherein the distance between the first reflective sphere 310 and the reference array is 10 units. The unit distance can be specified; one unit distance can be 1 cm or any arbitrary length defined by the user. Ideally, the normal of the reference array composed of the reflective spheres is kept parallel to the central axis of the field of view of the optical positioning camera 610. The optical positioning camera 610 captures the position information of the reflective spheres and pre-establishes an optical camera coordinate system F. camera This embodiment does not impose specific restrictions on the optical camera coordinate system; typically, F... camera It's set up at the factory by the optical camera; users can make certain adjustments, for example... Figure 3 The coordinates of the first reflective sphere 310 of the optical probe shown are (0, 0, 20). Therefore, the tip of the optical probe at F... camera The coordinates below are (0, 0, 10), meaning that the groove point currently closely fitted with the optical probe tip is at F. camera The first coordinate value is (0, 0, 10). Furthermore, multiple groove points can be obtained at F through multiple measurements. camera The first coordinate value below.

[0031] Step S104: Determine the second coordinate value of the groove point in the coordinate system of the robot arm base based on the physical probe 410; Specifically, based on the robotic arm 420, a pre-established coordinate system F for the robotic arm base is constructed. base and the coordinate system F of the end flange of the robotic arm flange The physical probe 410 is rigidly fixed to the end flange of the robotic arm 420, and the position of the tip of the physical probe 410 in the coordinate system F of the end flange of the robotic arm is determined. flange The coordinate values ​​below, due to the F coordinates of the robotic arm in any orientation. base To F flange The transformation relationship can be directly obtained from the robotic arm system, so the tip of the physical probe 410 in the robotic arm base coordinate system F baseThe coordinate values ​​below also indicate that the groove point that closely matches the tip of the physical probe 410 is located in the coordinate system F of the robot arm base. base The second coordinate value below.

[0032] For example, the distance between the tip and the bottom of the physical probe 410 is 20 units, see [link to documentation]. Figure 4 The diagram shown is a schematic of a robotic arm coordinate system provided by an embodiment of the present invention. The bottom end of the physical probe 410 is rigidly fixed to the end flange of the robotic arm 420, and a coordinate system F is established based on the end flange of the robotic arm 420. flange F flange The origin and coordinate axis directions are not specifically defined. The origin of the coordinate system shown in the figure is the center point of the end flange. In this coordinate system, the coordinates of the tip of the physical probe 410 are (0, 2, -20). Due to the arbitrary posture of the robotic arm 420, F base To F flange The conversion relationship can be directly obtained from the robotic arm 420 system, then the tip of the physical probe 410 in the robotic arm base coordinate system F base The second coordinate value below is (X) b Y b Z b In other words, the groove point that closely matches the tip of the physical probe 410 is in the coordinate system F of the robotic arm base. base The second coordinate value below is (X) b Y b Z b Furthermore, multiple groove points can be obtained at F. base The second coordinate value below.

[0033] Step S106: Determine the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first coordinate value and the second coordinate value; Specifically, the first and second coordinate values ​​of each groove point are determined, and these values ​​are arranged in an ordered manner to form ordered point sets Pset1 and Pset2, respectively. Pset1 consists of the first coordinate values ​​of the groove points, and Pset2 consists of the second coordinate values ​​of the groove points. Furthermore, the coordinate values ​​of multiple groove points are in the same order within the ordered point sets Pset1 and Pset2. Pset1 is designated as the target point set (Target Landmarks), and Pset2 is designated as the source point set (SourceLandmarks). The rigid space transformation between the two point sets is calculated using the feature point transformation method (vtk Landmark Transform), yielding the spatial transformation matrix, which is F. camera To F base The affine transformation matrix was used to achieve registration of the robotic arm.

[0034] For example, see Figure 5 The diagram shown is a schematic representation of a calibration block according to an embodiment of the present invention. The calibration block 510 includes four recessed points. Figure 4 From left to right, the four groove points are: the first groove point, the second groove point, the third groove point, and the fourth groove point. Steps S102 and S104 determine the location of these four groove points at F. camera and F base The coordinate values ​​below are used to sequentially place the first to fourth groove points at F. camera The first coordinate values ​​in the coordinate system form Pset1, and the first to fourth groove points are sequentially placed in F. base The second coordinate values ​​in the coordinate system form Pset2. The rigid body space transformation between the two point sets is calculated using the feature point transformation method (vtk LandmarkTransform), yielding the spatial transformation matrix, which is F. camera To F base The affine transformation matrix was used to achieve registration of the robotic arm.

[0035] This invention provides a robotic arm registration method applied to a robotic arm registration system. The robotic arm registration system includes a robotic arm 420, a physical probe 410, a calibration block 510, an optical camera, and an optical probe. The calibration block 510 includes multiple groove points. The robotic arm registration method includes: determining the first coordinate value of the groove point in the optical camera coordinate system based on the optical probe; determining the second coordinate value of the groove point in the robotic arm base coordinate system based on the physical probe 410; determining the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first and second coordinate values; by setting multiple fixed groove points and determining the coordinate values ​​of the groove points in the optical camera coordinate system and the robotic arm base coordinate system respectively, and then fitting the corresponding coordinate values ​​to determine the affine transformation matrix, the registration of the robotic arm is achieved. This method reduces the error caused by the movement of the optical reference array, improves the registration accuracy, and enhances the user experience.

[0036] Example 2 Based on the above embodiments, this embodiment provides another robotic arm registration method. The optical camera pre-stores information about the optical probe, including: the optical probe coordinate system and the coordinate values ​​of the reflector in the optical probe coordinate system; the step of determining the first coordinate value of the groove point in the optical camera coordinate system based on the optical probe includes: placing the tip of the optical probe in the groove point and determining the coordinate value of the reflector in the optical camera coordinate system; and determining the first coordinate value of the groove point in the optical camera coordinate system based on the coordinate value of the reflector in the optical camera coordinate system and the information of the optical probe.

[0037] Specifically, the optical camera pre-stores information on multiple optical probes, each probe being unique. After detecting an optical reference array composed of reflective spheres, the optical positioning camera 610 can identify the optical probes and determine the position of the reflective spheres at F... camera The coordinates below, and the relative position of the optical probe tip and the reflective sphere, determine the reflective sphere's position at F. camera The coordinates below are used to determine the location of the groove point at F. camera The first coordinate value below; For example, see [link to example]. Figure 3 A coordinate system F is established based on the tip of the optical probe. p The optical probe is pre-calibrated, and its coordinate system F p The origin of the optical probe coincides with the tip of the probe. When the optical probe is within the field of view of the optical positioning camera 610, F camera To F p The conversion relationship can be directly obtained from the positioning camera, therefore the position of the probe tip (F) p The origin is at F camera The coordinates (as shown below) can be directly obtained from the positioning camera, which is the coordinate value of the groove point at F. camera The first coordinate value below.

[0038] Further, see Figure 6 The illustrated embodiment of the present invention provides a schematic diagram of an optical probe calibration position. Holding the optical probe, the tip of the probe is sequentially placed at the first to fourth groove points from left to right, and the position of the optical probe tip at F is obtained respectively. camera The coordinates of the points below are denoted as p1, p2, p3, and p4, which represent the coordinates of the points from the first to the fourth groove. Fcamera The first coordinate value below.

[0039] Furthermore, the groove point includes: a reference groove point; the step of determining the second coordinate value of the groove point in the robot arm base coordinate system based on the physical probe 410 includes: moving the robot arm 420 and placing the tip of the physical probe 410 at the reference groove point; keeping the tip of the physical probe 410 at the reference groove point and continuing to move the robot arm 420 to maintain multiple postures, recording the positional relationship between the robot arm end flange coordinate system and the robot arm base coordinate system in each posture, the positional relationship including: the rotation matrix and translation vector from the robot arm end flange coordinate system to the robot arm base coordinate system; and determining the second coordinate value of the groove point in the robot arm base coordinate system based on the rotation matrix and translation vector.

[0040] For details, see Figure 7 The diagram shown is a schematic representation of a physical probe-based calibration position provided by an embodiment of the present invention. The coordinate system of the robotic arm base is denoted as F. base The coordinate system of the end flange of the robotic arm is denoted as F.flange ;The robotic arm 420, in any posture, F base To F flange The conversion relationship can be directly obtained from the robotic arm 420 system. A physical probe 410 is rigidly fixed to the end flange of the robotic arm 420, and the probe tip is at F flange Let p be the coordinates of the point below. t .

[0041] Furthermore, the step of determining the second coordinate value of the groove point in the robot arm base coordinate system based on the rotation matrix and translation vector includes: determining the second coordinate value of the reference groove point in the robot arm base coordinate system based on the translation vector; determining the coordinate value of the reference groove point in the robot arm end flange coordinate system based on the second coordinate value of the reference groove point in the robot arm base coordinate system, the rotation matrix, and the translation vector; determining the second coordinate values ​​of the other groove points (excluding the reference groove point) in the robot arm base coordinate system based on the coordinate value of the reference groove point in the robot arm end flange coordinate system, the rotation matrix, and the translation vector, wherein the step of determining the second coordinate value of the reference groove point in the robot arm base coordinate system based on the translation vector includes: obtaining translation vectors under various postures as a set of sphere center fitting points; determining the sphere center coordinate value based on the least squares method and the sphere center fitting point set; and using the sphere center coordinate value as the second coordinate value of the reference groove point in the robot arm base coordinate system.

[0042] Specifically, using the first groove point as the reference groove point, the robotic arm 420 is set to manual drag mode, and the end of the robotic arm 420 is manually dragged to posture P. i (i=1, 2, 3…6, where each posture is different), so that the tip of the physical probe 410 at the end of the robotic arm 420 contacts the reference groove point, and record the posture P. i F base To F flange The rotation matrix is ​​R i The translation vector is t i Continue manually dragging the end of the robotic arm 420 to any position, so that the tip of the physical probe 410 at the end of the robotic arm 420 contacts the second groove point, and record F at this time. base To F flange The rotation matrix is ​​R7, and the translation vector is t7. Continue to manually drag the end of the robotic arm 420 to any posture, so that the tip of the physical probe 410 at the end of the robotic arm 420 contacts the third groove point, and record F at this time. base To F flange The rotation matrix is ​​R8, and the translation vector is t8. Continue to manually drag the end of the robotic arm 420 to any posture, so that the tip of the physical probe 410 at the end of the robotic arm 420 contacts the fourth groove point, and record F at this time. base To F flange The rotation matrix is ​​R9, and the translation vector is t9.

[0043] Furthermore, due to t i For F flange To F base The translation vector, which also represents F flange The origin is at F base The coordinates of the point below; using the least squares method to calculate t i A point set (i = 1, 2, 3…6) is fitted to a sphere to obtain the coordinates of its center, denoted as p1'. p1' is the coordinate of the reference concave point at F. base The second coordinate value below.

[0044] Specifically, when the end effector of the robotic arm 420 is in posture P i When i=1, 2, 3…6, the reference groove point coincides with the tip of the physical probe 410 at the end of the robotic arm 420. Therefore, the reference groove point at F is obtained. flange The coordinates of the point below are the values ​​of p. t Because the coordinates of the reference groove point are in F base Given that F base To F flange The transformation relationship is {R} i , t i Given that, substituting any set of {R} i , t i The reference groove point at F can be obtained by using (i=1, 2, 3…6). flange The coordinates of the point below (taking {R1, t1} as an example) are: That is, .

[0045] Furthermore, the second, third, and fourth groove points are calculated at F. base The coordinates of the points p2', p3', and p4' are given below. Specifically, when the tip of the physical probe 410 at the end of the robotic arm 420 is placed at the second groove point, the tip of the physical probe 410 coincides with the second groove point. The coordinates of the tip of the physical probe 410 at F are then calculated. base The coordinates of the point below are the obtained p2'; at this time, the tip of the physical probe 410 is at F flange The coordinates of the point p below t F has been obtained. base To F flange Converting to {R7, t7}, we can determine the position of the tip of the physical probe 410 at F. base The coordinates (p2') below are as follows: ; That is ; Similarly, we can conclude that: , .

[0046] Furthermore, the step of determining the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first and second coordinate values ​​to achieve robotic arm registration includes: determining a starting point set based on multiple first coordinate values; determining a target point set based on multiple second coordinate values; wherein both the starting point set and the target point set are ordered point sets, and the number of elements in the starting point set and the target point set are equal; and determining the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the starting point set and the target point set to achieve robotic arm registration.

[0047] Specifically, let P be the ordered set of points consisting of p1, p2, p3, and p4. set1 Let P be the ordered set of points consisting of p1', p2', p3', and p4'. set2 Using the feature point transformation method (VTK Landmark Transform (a matching algorithm)) of the VTK (visualization toolkit is an open-source and free software system mainly used for 3D computer graphics, image processing and visualization) function library, Pset1 is set as the target point set (Target Landmarks), and P... set2 Let the starting point set be SourceLandmarks. Calculate the rigid body space transformation between the two point sets (calculation mode set to VTK_LANDMARK_RIGIDBODY) to obtain a 4×4 space transformation matrix T, where T is F. camera To F base The affine transformation matrix, where the 3x3 sub-part in the upper left corner of T represents F. camera To F base The first three elements of the last column of the rotation matrix R and T constitute F. camera To F base The translation vector t, that is, The robotic arm registration is now complete.

[0048] It is important to emphasize that throughout the entire registration process of the robotic arm, the coordinate system F of the calibration block 510 and the optical positioning camera 610 remains constant. camera All must remain still.

[0049] This invention provides a robotic arm registration method. During the registration process, the robotic arm registration and calibration does not require the installation of an optical reference array at the end of the robotic arm 420, thereby avoiding positioning errors caused by the rotation of the reference array due to the rotation of the end of the robotic arm 420. Furthermore, by using a handheld optical probe to collect the coordinates of the groove points on the calibration block 510, it can be artificially ensured that the normal of the reference array plane of the optical probe is approximately parallel to the normal of the field of view of the positioning camera (when the angle between the two is small, the positioning error is within an acceptable range), thereby reducing the possibility of introducing significant camera positioning errors.

[0050] Example 3 Based on the above embodiments, this invention provides a robotic arm registration device applied to a robotic arm registration system. The robotic arm registration system includes: a robotic arm 420, a physical probe 410, a calibration block 510, an optical camera, and an optical probe; the calibration block 510 includes multiple recessed points; see also... Figure 8 The diagram shown is a schematic of a robotic arm registration device provided in an embodiment of the present invention. The robotic arm registration device includes: The optical coordinate determination module 810 is used to determine the first coordinate value of the groove point in the optical camera coordinate system based on the optical probe. The mechanical coordinate determination module 820 is used to determine the second coordinate value of the groove point in the coordinate system of the robot arm base based on the physical probe 410; The robotic arm registration module 830 is used to determine the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first coordinate value and the second coordinate value, so as to realize the registration of the robotic arm.

[0051] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-mentioned robotic arm registration device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0052] The robotic arm registration device provided in this embodiment of the invention has the same technical features as the robotic arm registration method provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0053] Example 4 Based on the above embodiments, this invention provides a robotic arm registration system, which includes: a robotic arm 420, a physical probe 410, a calibration block 510, an optical camera, an optical probe, and a robotic arm registration device; the robotic arm registration system is used to perform the above-described robotic arm registration method.

[0054] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-mentioned robotic arm registration system can be referred to the corresponding process in the aforementioned method embodiments, and will not be repeated here.

[0055] The robotic arm registration system provided in this embodiment of the invention has the same technical features as the robotic arm registration method provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0056] Example 5 This invention also provides an electronic device for running the above-described robotic arm registration method; see [link to related documentation]. Figure 9The schematic diagram of an electronic device provided by the embodiment of the present invention shown below includes a memory 900 and a processor 901. The memory 900 is used to store one or more computer instructions, which are executed by the processor 901 to implement the above-mentioned robotic arm registration method.

[0057] Furthermore, Figure 9 The electronic device shown also includes a bus 902 and a communication interface 903. The processor 901, the communication interface 903, and the memory 900 are connected via the bus 902.

[0058] The memory 900 may include high-speed random access memory (RAM) 900, and may also include non-volatile memory 900, such as at least one disk storage device 900. Communication between this system network element and at least one other network element is achieved through at least one communication interface 903 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 902 can be an ISA bus 902, a PCI bus 902, or an EISA bus 902, etc. The bus 902 can be divided into an address bus 902, a data bus 902, a control bus 902, etc. For ease of representation, Figure 9 The symbol is represented by only one double-headed arrow, but this does not mean that there is only one bus 902 or one type of bus 902.

[0059] The processor 901 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 901 or by instructions in software form. The processor 901 can be a general-purpose processor 901, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor 901 can be a microprocessor 901, or any conventional processor 901. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by the hardware decoding processor 901, or execution by a combination of hardware and software modules in the decoding processor 901. The software module can reside in a random access memory 900, flash memory, read-only memory 900, programmable read-only memory 900, electrically erasable programmable memory 900, registers, or other mature storage media in the art. This storage medium is located in memory 900, and processor 901 reads information from memory 900 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0060] The computer program product for registering a robotic arm provided in this embodiment of the invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0061] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interface 903. Indirect couplings or communication connections between apparatuses or units may be electrical, mechanical, or other forms.

[0062] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0063] In addition, the functional units in the embodiments provided by the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0064] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robotic arm registration method, characterized in that, An application is made in a robotic arm registration system, the robotic arm registration system comprising: a robotic arm, a physical probe, a calibration block, an optical camera, and an optical probe; the calibration block includes multiple groove points; the robotic arm registration method includes: The first coordinate value of the groove point in the optical camera coordinate system is determined based on the optical probe. The second coordinate value of the groove point in the coordinate system of the robotic arm base is determined based on the physical probe. The affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system is determined based on the first coordinate value and the second coordinate value. The groove point includes: a reference groove point; the step of determining the second coordinate value of the groove point in the coordinate system of the robotic arm base based on the physical probe includes: Move the robotic arm to place the tip of the physical probe at the reference groove point; Keep the tip of the physical probe at the reference groove point, continue to move the robotic arm, keep the robotic arm in multiple postures, and record the positional relationship between the robotic arm end flange coordinate system and the robotic arm base coordinate system under each posture. The positional relationship includes: the rotation matrix and translation vector from the robotic arm end flange coordinate system to the robotic arm base coordinate system. The second coordinate value of the groove point in the coordinate system of the robotic arm base is determined based on the rotation matrix and the translation vector.

2. The robotic arm registration method according to claim 1, characterized in that, The calibration block includes multiple grooves that are not located on the same straight line; each groove is in close contact with the tip of the physical probe; each groove is also in close contact with the tip of the optical probe.

3. The robotic arm registration method according to claim 1, characterized in that, The optical probe includes: a plurality of reflective spheres; the optical camera pre-stores information of the optical probe, the information including: an optical probe coordinate system and the coordinate values ​​of the reflective spheres in the optical probe coordinate system; The step of determining the first coordinate value of the notch point in the optical camera coordinate system based on the optical probe includes: The tip of the optical probe is placed in the groove to determine the coordinates of the reflective sphere in the optical camera coordinate system. The first coordinate value of the groove point in the optical camera coordinate system is determined based on the coordinate values ​​of the reflective sphere in the optical camera coordinate system and the information of the optical probe.

4. The robotic arm registration method according to claim 1, characterized in that, The step of determining the second coordinate value of the groove point in the coordinate system of the robotic arm base based on the rotation matrix and the translation vector includes: The second coordinate value of the reference groove point in the coordinate system of the robotic arm base is determined based on the translation vector. Based on the second coordinate value of the reference groove point in the coordinate system of the robot arm base, the rotation matrix, and the translation vector, the coordinate value of the reference groove point in the coordinate system of the robot arm end flange is determined. Based on the coordinates of the reference groove point in the coordinate system of the end flange of the robotic arm, the rotation matrix, and the translation vector, the second coordinates of the remaining groove points (excluding the reference groove point) in the coordinate system of the robotic arm base are determined.

5. The robotic arm registration method according to claim 4, characterized in that, The step of determining the second coordinate value of the reference groove point in the coordinate system of the robotic arm base based on the translation vector includes: The translation vectors under various postures are obtained as a set of sphere center fitting points; The coordinates of the sphere's center are determined based on the least squares method and the set of points fitted to the sphere's center. The coordinates of the sphere's center are used as the second coordinates of the reference groove point in the coordinate system of the robotic arm base.

6. The robotic arm registration method according to claim 1, characterized in that, The steps for determining the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first coordinate value and the second coordinate value, and realizing robotic arm registration, include: A set of starting points is determined based on multiple first coordinate values; A target point set is determined based on multiple second coordinate values; wherein both the starting point set and the target point set are ordered point sets, and the number of elements in the starting point set and the target point set are equal; Based on the starting point set and the target point set, the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system is determined to achieve robotic arm registration.

7. A robotic arm registration device, characterized in that, This is applied to a robotic arm registration system, which includes: a robotic arm, a physical probe, a calibration block, an optical camera, and an optical probe; the calibration block includes multiple groove points. The robotic arm registration device includes: An optical coordinate determination module is used to determine the first coordinate value of the groove point in the optical camera coordinate system based on the optical probe. A mechanical coordinate determination module is used to determine the second coordinate value of the groove point in the coordinate system of the robot arm base based on the physical probe; The robotic arm registration module is used to determine the affine transformation matrix between the optical camera coordinate system and the robotic arm base coordinate system based on the first coordinate value and the second coordinate value, so as to realize the robotic arm registration. The groove point includes: a reference groove point; the mechanical coordinate determination module is used to move the robotic arm and place the tip of the physical probe at the reference groove point; keep the tip of the physical probe at the reference groove point, continue to move the robotic arm, so that the robotic arm maintains multiple postures, and record the positional relationship between the robotic arm end flange coordinate system and the robotic arm base coordinate system under each posture, the positional relationship including: the rotation matrix and translation vector from the robotic arm end flange coordinate system to the robotic arm base coordinate system; and determine the second coordinate value of the groove point in the robotic arm base coordinate system based on the rotation matrix and the translation vector.

8. A robotic arm registration system, characterized in that, The robotic arm registration system is used to perform the robotic arm registration method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the robotic arm registration method according to any one of claims 1 to 6.

Images