Coordinate calibration method, device and equipment after fixed connection of multi-view equipment and robot

Abstract

The invention provides a coordinate calibration method, device and equipment after fixed connection of multi-view equipment and a robot, and relates to the technical field of robots. The method comprises the steps that an end effector of the robot is controlled to move, and the tip of the end effector passes through at least two positions; images of the tip of the end effector at the at least two positions are shot through the multi-view device; shooting images of the tip of the end effector at at least two positions based on the multi-view equipment, and determining a first coordinate of each position under a coordinate system of at least one camera device of the multi-view equipment; determining a second coordinate of the tip of the end effector in the coordinate system of the robot by utilizing a transformation relation between the tip of the end effector and the coordinate system of the end effector and the motion model of the robot; and based on each first coordinate and each second coordinate, calibrating coordinates between the multi-view equipment and an end effector of the robot. By adopting the method provided by the embodiment of the invention, the position coordinate calibration between the multi-view equipment and the robot can be realized.

Description

Technical Field

[0001] The present disclosure relates to the field of robotics, and in particular to a method, device, and apparatus for calibrating coordinates of a multi-camera device and a robot after they are fixedly connected. Background Art

[0002] In related technologies, after the multi-camera device is fixedly connected to the robot, it usually starts working from the fixed connection point. At this time, the coordinate system of the multi-camera device and the coordinate system of the robot are not yet associated and cannot be converted to each other. Summary of the Invention

[0003] The present disclosure provides a method, device, and apparatus for calibrating the coordinates of a multi-camera device and a robot after they are fixed together. The technical solution of the present disclosure is as follows:

[0004] In a first aspect, the present disclosure provides a coordinate calibration method after a multi-eye device and a robot are fixedly connected, comprising:

[0005] controlling the movement of an end effector of the robot so that the tip of the end effector passes through at least two positions;

[0006] capturing images of the tip of the end effector at the at least two positions using a multi-camera device;

[0007] determining a first coordinate of each of the at least two positions in a coordinate system of at least one camera device of the multi-camera device based on images of the tip of the end effector at the at least two positions captured by the multi-camera device;

[0008] Determining a second coordinate of the tip of the end effector in the robot coordinate system by using a transformation relationship between the tip of the end effector and the coordinate system of the end effector and a motion model of the robot;

[0009] Based on each of the first coordinates and the second coordinates, the coordinates between the multi-eye device and the end effector of the robot are calibrated.

[0010] In a possible implementation, calibrating the coordinates between the multi-eye device and the end effector of the robot based on each of the first coordinates and the second coordinates includes:

[0011] Calculating a rotation matrix estimate based on each of the first coordinate and the second coordinate;

[0012] Projecting the estimated rotation matrix onto the SO3 manifold to obtain the target rotation matrix;

[0013] Determining a target translation vector based on the target rotation matrix;

[0014] The coordinates between the multi-eye device and the end effector of the robot are calibrated based on the target translation vector.

[0015] In a possible implementation, calculating a rotation matrix estimate based on each of the first coordinates and the second coordinates includes:

[0016] Establishing an equation relationship between each of the first coordinates and the second coordinates, and calculating a rotation matrix estimate;

[0017] The projecting of the rotation matrix estimate onto the SO3 manifold to obtain a target rotation matrix includes:

[0018] The rotation matrix estimate is projected onto the SO3 manifold by singular value decomposition (SVD) to obtain the target rotation matrix.

[0019] In a possible implementation, determining the target translation vector based on the target rotation matrix includes:

[0020] Obtain at least two translation vector estimates based on the target rotation matrix and the equation relationship;

[0021] The target translation vector is determined based on the at least two translation vector estimates.

[0022] In a possible implementation, determining the target translation vector based on the at least two translation vector estimates includes:

[0023] The target translation vector is calculated based on the at least two translation vector estimation values through a preset algorithm; wherein the preset algorithm includes at least one of a averaging algorithm, a weighted average algorithm, a median algorithm, a least squares method, a maximum likelihood estimation, and a Bayesian estimation.

[0024] In a possible implementation, the multi-eye device and the robot are connected via a fixed connection point.

[0025] In a second aspect, the present disclosure provides a coordinate calibration device after a multi-eye device and a robot are fixedly connected, comprising:

[0026] a control module, configured to control the movement of an end effector of the robot so that a tip of the end effector passes through at least two positions;

[0027] A shooting module, configured to shoot images of the tip of the end effector at the at least two positions through a multi-camera device;

[0028] a first coordinate determination module, configured to determine, based on images of the tip of the end effector at the at least two positions captured by the multi-camera device, a first coordinate of each of the positions in a coordinate system of at least one camera device of the multi-camera device;

[0029] a second coordinate determination module, configured to determine a second coordinate of the tip of the end effector in the robot coordinate system by using a transformation relationship between the tip of the end effector and the coordinate system of the end effector and a motion model of the robot;

[0030] A calibration module is used to calibrate the coordinates between the multi-eye device and the end effector of the robot based on each of the first coordinates and the second coordinates.

[0031] In a third aspect, the present disclosure provides an electronic device, comprising:

[0032] processor;

[0033] a memory for storing instructions executable by the processor;

[0034] The processor is configured to execute the instructions to implement the coordinate calibration method after the multi-eye device and the robot are fixedly connected as described in the first aspect.

[0035] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the coordinate calibration method after the multi-eye device and the robot are fixedly connected as described in the first aspect.

[0036] In a fifth aspect, the present disclosure provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the coordinate calibration method after the multi-eye device and the robot are fixedly connected as described in the first aspect.

[0037] The technical solution disclosed herein brings at least the following beneficial effects:

[0038] In an embodiment of the present disclosure, the end effector of a robot is controlled to move so that the tip of the end effector passes through at least two positions; a multi-camera device captures images of the tip of the end effector at the at least two positions; based on the images of the tip of the end effector at the at least two positions captured by the multi-camera device, a first coordinate of each position in the coordinate system of at least one camera of the multi-camera device is determined; a transformation relationship between the tip of the end effector and the coordinate system of the end effector and the robot's motion model is used to determine a second coordinate of the tip of the end effector in the robot coordinate system; and based on each of the first and second coordinates, the coordinates between the multi-camera device and the robot's end effector are calibrated. In this way, position coordinate calibration between the multi-camera device and the robot can be achieved, allowing the multi-camera device and the robot to accurately know each other's positions in their own coordinate systems. This not only determines the relative positional relationship between the multi-camera device and the robot's end effector, improving the robot's positioning accuracy in complex tasks, but also facilitates collaborative operation between the multi-camera device and the robot, providing a basis for automated operation and improving the system's usability and reliability.

[0039] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0041] Figure 1 A schematic flow chart of a coordinate calibration method after a multi-eye device and a robot are fixedly connected, provided in an embodiment of the present disclosure;

[0042] Figure 2 This is a schematic structural diagram of a coordinate calibration device after a multi-eye device and a robot are fixedly connected, provided by an embodiment of the present disclosure;

[0043] Figure 3 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0045] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0046] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0047] The acquisition, storage, use, and processing of data in this disclosed technical solution comply with the relevant provisions of national laws and regulations.

[0048] It should be noted that in the embodiments of the present disclosure, there may be certain software, components, models, etc. that already exist in the industry. They should be considered as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present disclosure, but it does not mean that the applicant has or will necessarily use the solution.

[0049] The technical solutions provided by various embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0050] Figure 1 This is a flowchart of a coordinate calibration method after a multi-eye device and a robot are fixedly connected, provided in an embodiment of the present disclosure.

[0051] like Figure 1 As shown, the coordinate calibration method after the multi-eye device and the robot are fixedly connected may include the following steps:

[0052] S101 , controlling the movement of the end effector of the robot so that the tip of the end effector passes through at least two positions.

[0053] In an embodiment of the present disclosure, during coordinate calibration after the multi-camera device and the robot are securely connected, the robot's end effector can be controlled to move. During this movement, the tip of the robot's end effector must be controlled to pass through at least two positions. For example, these positions can be pre-set or dynamically set based on task requirements. The position information can be in the form of coordinates, which are sent or input to the robot in this form to generate a motion instruction for the tip of the robot's end effector to move to the set position. When the tip of the robot's end effector reaches the set position, a sensor or vision system can be used to determine whether the tip of the robot's end effector has accurately reached the set position.

[0054] S102 , capturing images of the tip of the end effector at at least two positions using a multi-camera device.

[0055] In one possible implementation, the multi-camera device and the robot may be connected via fixed connection points. For example, when the multi-camera device and the robot are connected via fixed connection points, it is necessary to ensure that the multi-camera device and the robot are properly installed and connected to the robot system. The multi-camera device may have multiple imaging devices, such as cameras.

[0056] In an embodiment of the present disclosure, when the tip of the end effector of the robot reaches each set position, an image can be captured by a multi-eye device, and the image needs to contain clear features of the tip of the end effector of the robot. Exemplarily, the captured images can also be stored for subsequent processing and analysis, for example, they can be stored in a database or system storage unit. As an example, assuming that the set positions are P1 and P2, the movement of the robot end effector can be controlled so that the tip of the end effector moves to P1 and P2 in sequence. When the tip of the end effector reaches P1, image 1 is captured by the multi-eye device, and when the tip of the end effector reaches P2, image 2 is captured by the multi-eye device, and image 1 and image 2 are stored.

[0057] S103 : Based on images of the tip of the end effector at at least two positions captured by the multi-camera device, determine a first coordinate of each position in a coordinate system of at least one camera device of the multi-camera device.

[0058] In an embodiment of the present disclosure, after capturing images of the tip of the end effector at at least two positions using a multi-camera device, the position of the tip of the end effector in each image, such as pixel coordinates, can be determined using an image recognition algorithm (e.g., feature point detection, template matching algorithm, etc.). The pixel coordinates of the tip of the end effector in each image can then be converted into three-dimensional coordinates under the camera system, i.e., first coordinates, using the calibration parameters of the multi-camera device (e.g., intrinsic parameters and distortion parameters of the camera device). In this way, the first coordinates of the tip of the end effector of the robot in the coordinate system of the multi-camera device can be determined. The camera device can be any camera device of the multi-camera device.

[0059] It is understandable that before determining the first coordinate of each position in the coordinate system of at least one camera device of the multi-camera apparatus, the captured image may be preprocessed, such as denoising, contrast enhancement, etc., to improve image quality.

[0060] S104 , using the transformation relationship between the tip of the end effector and the coordinate system of the end effector and the motion model of the robot, determine the second coordinate of the tip of the end effector in the robot coordinate system.

[0061] In an embodiment of the present disclosure, the coordinates of the end effector tip in the robot coordinate system, i.e., the second coordinates, can also be determined when the end effector tip of the robot passes through each set position. Exemplarily, the position coordinates of the end effector in the robot coordinate system can be calculated based on the robot's motion model, and then the position coordinates of the end effector tip in the robot coordinate system can be calculated based on the transformation relationship between the end effector tip and the end effector coordinate system as the second coordinates. It is understood that the transformation relationship between the end effector tip and the end effector coordinate system can be determined based on the design and geometric characteristics of the end effector, and the transformation relationship can include, for example, rotation and translation information; the robot's motion model can be used to describe the motion relationship of each joint of the robot and the position of the end effector, and can be provided by the robot manufacturer, for example, or obtained through robot calibration.

[0062] S105 , calibrating the coordinates between the multi-eye device and the end effector of the robot based on each first coordinate and the second coordinate.

[0063] In an embodiment of the present disclosure, after determining the first coordinate of each position in the coordinate system of at least one camera of the multi-camera device and the second coordinate of the tip of the end effector in the coordinate system of the robot, the multi-camera device and the end effector of the robot can be calibrated. For example, an equation relationship can be established based on each first coordinate and the second coordinate, and a coordinate transformation relationship between the first coordinate and the second coordinate can be calculated based on the equation relationship, so that the coordinates between the multi-camera device and the end effector of the robot can be calibrated based on the coordinate transformation relationship.

[0064] In an embodiment of the present disclosure, the end effector of a robot is controlled to move so that the tip of the end effector passes through at least two positions; a multi-camera device captures images of the tip of the end effector at the at least two positions; based on the images of the tip of the end effector at the at least two positions captured by the multi-camera device, a first coordinate of each position in the coordinate system of at least one camera of the multi-camera device is determined; a transformation relationship between the tip of the end effector and the coordinate system of the end effector and the robot's motion model is used to determine a second coordinate of the tip of the end effector in the robot coordinate system; and based on each of the first and second coordinates, the coordinates between the multi-camera device and the robot's end effector are calibrated. In this way, position coordinate calibration between the multi-camera device and the robot can be achieved, allowing the multi-camera device and the robot to accurately know each other's positions in their own coordinate systems. This not only determines the relative positional relationship between the multi-camera device and the robot's end effector, improving the robot's positioning accuracy in complex tasks, but also facilitates collaborative operation between the multi-camera device and the robot, providing a basis for automated operation and improving the system's usability and reliability.

[0065] In some possible implementations, calibrating the coordinates between the multi-eye device and the end effector of the robot based on each first coordinate and the second coordinate includes:

[0066] Computing a rotation matrix estimate based on each of the first and second coordinates;

[0067] Project the estimated rotation matrix onto the SO3 manifold to obtain the target rotation matrix;

[0068] Based on the target rotation matrix, determine the target translation vector;

[0069] The coordinates between the multi-camera device and the robot's end effector are calibrated based on the target translation vector.

[0070] In an embodiment of the present disclosure, a rotation matrix estimation value may be calculated based on each first coordinate and second coordinate, and the rotation matrix estimation value may be used to describe the rotation relationship between the two coordinate systems.

[0071] In a further possible embodiment, an equation relationship can be established based on each first coordinate and the second coordinate to calculate the rotation matrix estimate. For example, for each position, an equation relationship can be established between the first coordinate and the second coordinate, as shown in the following formula (1). Using the equation relationship established between the first coordinate and the second coordinate of at least two positions, the rotation matrix estimate can be solved by mathematical methods (e.g., subtracting two equations from each other).

[0072]

[0073] in, represents the first coordinate, i represents the number of the first coordinate, represents the rotation matrix, represents the second coordinate, Represents the translation vector.

[0074] The rotation matrix estimate can then be projected onto the SO3 manifold, also known as the special orthogonal group, to obtain the target rotation matrix. The SO3 manifold, also known as the special orthogonal group, can be used to describe possible rotations in three-dimensional space.

[0075] In a further possible implementation, the rotation matrix estimate can be projected onto the SO3 manifold by singular value decomposition (SVD) to obtain a target rotation matrix. As an example, the rotation matrix estimate can be subjected to singular value decomposition (SVD) and projected onto the SO3 manifold. The matrix obtained by projecting the rotation matrix estimate onto the SO3 manifold is the target rotation matrix. It is understandable that the implementation method of this projection process is similar to that of the related art and will not be repeated here.

[0076] Then, the target translation vector can be determined based on the target rotation matrix. For example, the target rotation matrix can be substituted into the above equation to solve based on The final required translation vector, also known as the target translation vector, can be determined.

[0077] In a further possible implementation, determining the target translation vector based on the target rotation matrix includes:

[0078] Based on the target rotation matrix and the equation relationship, at least two translation vector estimates are obtained;

[0079] Based on the at least two translation vector estimates, a target translation vector is determined.

[0080] In the embodiments of the present application, considering that there are at least two equations established based on the first coordinate and the second coordinate, at least two translation vector estimates are also obtained. Therefore, when determining the target translation vector based on the target rotation matrix, at least two translation vector estimates can be first obtained based on the target rotation matrix and the equations. For example, the target rotation matrix is substituted into the equations to solve for at least two translation vector estimates. Then, the target translation vector can be calculated based on the at least two translation vector estimates.

[0081] Furthermore, a target translation vector may be calculated based on the at least two translation vector estimates using a preset algorithm. Exemplarily, the preset algorithm may include at least one of a mean algorithm, a weighted average algorithm, a median algorithm, a least squares method, a maximum likelihood estimation, and a Bayesian estimation. That is, the mean, weighted average, median, maximum likelihood estimation, or Bayesian estimation of the at least two translation vector estimates may be calculated as the target translation vector.

[0082] To clarify the coordinate calibration method for the multi-camera device and robot after they are connected, the following is a detailed example. In this example, the robot's end effector has a tip that can be identified by a camera, and depth information can be recovered using binocular vision. The coordinate transformation between the tip and the robot's end effector is known (derived from TCP (Tool Center Point) calibration). The calibration process is as follows:

[0083] The robot movement is controlled so that the tip of the end effector is at different set positions. The tip of the end effector is captured by multiple (or more than two) cameras at each moment when it is at different set positions, and the three-dimensional coordinates (i.e., the first coordinates) of the tip of the end effector in a certain camera coordinate system in the image are restored. At this time, the coordinates of the camera tip in the robot coordinate system (i.e., the second coordinates) can be simply calculated through the transformation of the tip and end coordinate systems and the robot's motion model, which is The following equation (1) can be derived based on the first coordinate and the second coordinate.

[0084]

[0085] in, represents the first coordinate, i represents the number of the first coordinate, represents the rotation matrix, represents the second coordinate, Represents the translation vector.

[0086] Estimate first Subtracting the above equations (1) from each other, we get:

[0087]

[0088] Based on equation (2), the following equation can be formed:

[0089]

[0090] In this way, we can solve The estimated value of Projecting it onto the so3 manifold gives the target rotation matrix. Substitute into equation (1) to find multiple estimated value.

[0091]

[0092] Afterwards, find multiple The mean of the estimated values is the target translation vector.

[0093] The specific implementation and technical effects of each step of this embodiment are similar to those of the above method embodiment and will not be repeated here.

[0094] Based on the same inventive concept, the embodiment of the present disclosure also provides a coordinate calibration device after a multi-eye device and a robot are fixedly connected. Figure 2 As shown, the coordinate calibration device 200 after the multi-eye device and the robot are fixedly connected includes:

[0095] a control module 210 for controlling the movement of the end effector of the robot so that the tip of the end effector passes through at least two positions;

[0096] A shooting module 220 is configured to shoot images of the tip of the end effector at the at least two positions using a multi-camera device;

[0097] A first coordinate determination module 230 is configured to determine, based on images of the tip of the end effector at the at least two positions captured by the multi-camera device, a first coordinate of each position in a coordinate system of at least one camera device of the multi-camera device;

[0098] a second coordinate determining module 240 for determining a second coordinate of the tip of the end effector in the robot coordinate system by using a transformation relationship between the tip of the end effector and the coordinate system of the end effector and a motion model of the robot;

[0099] The calibration module 250 is configured to calibrate the coordinates between the multi-eye device and the end effector of the robot based on each of the first coordinates and the second coordinates.

[0100] In a possible implementation, the calibration module 250 includes:

[0101] a calculation unit, configured to calculate a rotation matrix estimate based on each of the first coordinates and the second coordinates;

[0102] A projection unit, configured to project the estimated rotation matrix onto the SO3 manifold to obtain a target rotation matrix;

[0103] a determining unit, configured to determine a target translation vector based on the target rotation matrix;

[0104] A calibration unit is used to calibrate the coordinates between the multi-eye device and the end effector of the robot based on the target translation vector.

[0105] In a possible implementation, the computing unit is configured to:

[0106] Establishing an equation relationship between each of the first coordinates and the second coordinates, and calculating a rotation matrix estimate;

[0107] The projection unit is configured to:

[0108] The rotation matrix estimate is projected onto the SO3 manifold by singular value decomposition (SVD) to obtain the target rotation matrix.

[0109] In a possible implementation, the determining unit is configured to:

[0110] Obtain at least two translation vector estimates based on the target rotation matrix and the equation relationship;

[0111] The target translation vector is determined based on the at least two translation vector estimates.

[0112] In a possible implementation manner, the determining unit is further configured to:

[0113] The target translation vector is calculated based on the at least two translation vector estimation values through a preset algorithm; wherein the preset algorithm includes at least one of a averaging algorithm, a weighted average algorithm, a median algorithm, a least squares method, a maximum likelihood estimation, and a Bayesian estimation.

[0114] In a possible implementation, the multi-eye device and the robot are connected via a fixed connection point.

[0115] The specific implementation method and technical effects of the device provided by the embodiment of the present disclosure are similar to those of the above-mentioned method embodiment, and will not be repeated here.

[0116] According to an embodiment of the present disclosure, the present disclosure also discloses an electronic device, a computer-readable storage medium, and a computer program product.

[0117] Figure 3A schematic block diagram of an electronic device 300 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 300 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0118] like Figure 3 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 can also be stored in the RAM 303. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0119] Multiple components in the electronic device 300 are connected to the I / O interface 305, including an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a magnetic disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0120] The computing unit 301 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the coordinate calibration method for a multi-eye device and a robot after being connected. For example, in some embodiments, the coordinate calibration method for a multi-eye device and a robot after being connected can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the coordinate calibration method for a multi-eye device and a robot after being connected can be performed. Alternatively, in other embodiments, the computing unit 301 may be configured in any other appropriate manner (for example, by means of firmware) to execute a coordinate calibration method after the multi-eye device and the robot are fixedly connected.

[0121] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] The program code of the computer program product for implementing the method of the present disclosure can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0123] In the context of the present disclosure, a computer-readable storage medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can be a machine-readable signal medium or a machine-readable storage medium. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0126] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of this disclosure can be achieved, and this document is not limited here.

[0128] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A coordinate calibration method after a multi-eye device and a robot are fixedly connected, characterized in that: include: controlling the movement of an end effector of the robot so that the tip of the end effector passes through at least two positions; capturing images of the tip of the end effector at the at least two positions using a multi-camera device; determining a first coordinate of each of the at least two positions in a coordinate system of at least one camera device of the multi-camera device based on images of the tip of the end effector at the at least two positions captured by the multi-camera device; Determining a second coordinate of the tip of the end effector in the robot coordinate system by using a transformation relationship between the tip of the end effector and the coordinate system of the end effector and a motion model of the robot; Based on each of the first coordinates and the second coordinates, the coordinates between the multi-eye device and the end effector of the robot are calibrated.

2. The coordinate calibration method after the multi-eye device and the robot are fixedly connected according to claim 1, characterized in that: The step of calibrating the coordinates between the multi-eye device and the end effector of the robot based on each of the first coordinates and the second coordinates includes: Calculating a rotation matrix estimate based on each of the first coordinate and the second coordinate; Projecting the estimated rotation matrix onto the SO3 manifold to obtain the target rotation matrix; Determining a target translation vector based on the target rotation matrix; The coordinates between the multi-eye device and the end effector of the robot are calibrated based on the target translation vector.

3. The coordinate calibration method after the multi-eye device and the robot are fixedly connected according to claim 2, characterized in that: The calculating a rotation matrix estimate based on each of the first coordinates and the second coordinates includes: Establishing an equation relationship between each of the first coordinates and the second coordinates, and calculating a rotation matrix estimate; The projecting of the rotation matrix estimate onto the SO3 manifold to obtain a target rotation matrix includes: The rotation matrix estimate is projected onto the SO3 manifold by singular value decomposition (SVD) to obtain the target rotation matrix.

4. The coordinate calibration method after the multi-eye device and the robot are fixedly connected according to claim 3, characterized in that: The determining of the target translation vector based on the target rotation matrix includes: Obtain at least two translation vector estimates based on the target rotation matrix and the equation relationship; The target translation vector is determined based on the at least two translation vector estimates.

5. The coordinate calibration method after the multi-eye device and the robot are fixedly connected according to claim 4, characterized in that: The determining the target translation vector based on the at least two translation vector estimation values includes: The target translation vector is calculated based on the at least two translation vector estimation values through a preset algorithm; wherein the preset algorithm includes at least one of a averaging algorithm, a weighted average algorithm, a median algorithm, a least squares method, a maximum likelihood estimation, and a Bayesian estimation.

6. The coordinate calibration method after the multi-eye device and the robot are fixedly connected according to any one of claims 1 to 5, characterized in that: The multi-eye device and the robot are connected via a fixed connection point.

7. A coordinate calibration device after a multi-eye device and a robot are fixedly connected, characterized in that: include: a control module, configured to control the movement of an end effector of the robot so that a tip of the end effector passes through at least two positions; A shooting module, configured to shoot images of the tip of the end effector at the at least two positions through a multi-camera device; a first coordinate determination module, configured to determine, based on images of the tip of the end effector at the at least two positions captured by the multi-camera device, a first coordinate of each of the positions in a coordinate system of at least one camera device of the multi-camera device; a second coordinate determination module, configured to determine a second coordinate of the tip of the end effector in the robot coordinate system by using a transformation relationship between the tip of the end effector and the coordinate system of the end effector and a motion model of the robot; A calibration module is used to calibrate the coordinates between the multi-eye device and the end effector of the robot based on each of the first coordinates and the second coordinates.

8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the coordinate calibration method after the multi-eye device and the robot are fixedly connected as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the coordinate calibration method after the multi-eye device and the robot are fixedly connected is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the coordinate calibration method after the multi-eye device and the robot are fixedly connected is implemented as described in any one of claims 1-6.

Images