Method, device and system for calibrating mounting position of camera on mechanical arm

Abstract

The invention relates to a calibration method, device and system for the installation position of a camera on a mechanical arm, and the method comprises the steps: obtaining a first relative pose relation between a mechanical arm base and the original point of the mechanical arm large arm after the mechanical arm large arm rotates by a set angle each time, and obtaining a first relative pose relation between the mechanical arm base and the original point of the mechanical arm large arm; identifying a calibration object on the calibration plate by using the camera to obtain a second relative pose relationship between the camera and the calibration plate; based on the multiple first relative pose relations, the multiple second relative pose relations and the third relative pose relation between the mechanical arm base and the calibration plate, the optimal fourth relative pose relation between the original point of the large arm of the mechanical arm and the camera is determined; and based on the optimal fourth relative pose relation, the mounting position of the camera is calibrated, and efficient and accurate calibration of the mounting position of the camera on the mechanical arm is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of camera calibration, and in particular to a method, device and system for calibrating the installation position of a camera on a robotic arm. Background Art

[0002] In the application of a four-degree-of-freedom robotic arm, how to ensure the correct installation position of the camera on the robotic arm is a technical challenge. An inappropriate installation position may lead to inaccurate positioning of the visual system, affect the operation of the robotic arm, and may even lead to operational errors, damage to equipment or cause safety accidents. In addition, once the configuration of the robotic arm or the working environment changes, the use of traditional manual calibration methods is not only time-consuming and labor-intensive, but also difficult to ensure accuracy.

[0003] With regard to the problem of low efficiency and accuracy in calibrating the installation position of the camera on the robotic arm in the related art, no effective solution has been proposed so far. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device and system for calibrating the installation position of a camera on a robotic arm in response to the above technical problems.

[0005] In a first aspect, an embodiment of the present application provides a method for calibrating the installation position of a camera on a robotic arm, wherein the robotic arm includes a robotic arm base and a robotic arm, the robotic arm rotates about the robotic arm origin (the robotic arm origin refers to the connection point between the robotic arm and the robotic arm base), the camera is installed on the robotic arm, and a calibration plate is placed within the visible range of the camera, the method comprising:

[0006] After the robot arm rotates by a set angle each time, a first relative posture relationship between the robot arm base and the origin of the robot arm is obtained, and a second relative posture relationship between the camera and the calibration plate is obtained by using the camera to recognize the calibration object on the calibration plate;

[0007] Determine an optimal fourth relative posture relationship between the origin of the upper arm of the robotic arm and the camera based on a plurality of the first relative posture relationships, a plurality of the second relative posture relationships, and a third relative posture relationship between the robotic arm base and the calibration plate;

[0008] Based on the optimal fourth relative posture relationship, the installation position of the camera is calibrated.

[0009] In some embodiments, each group of the first relative posture relationship, the second relative posture relationship, the third relative posture relationship, and the fourth relative posture relationship has the following relationship:

[0010] The third relative posture relationship=the first relative posture relationship*the second relative posture relationship*the fourth relative posture relationship.

[0011] In some embodiments, determining the optimal fourth relative pose relationship between the origin of the manipulator arm and the camera based on the plurality of the first relative pose relationships, the plurality of the second relative pose relationships, and the third relative pose relationship between the manipulator base and the calibration plate comprises:

[0012] constructing an objective function based on the relationship between the plurality of the first relative posture relationships, the plurality of the second relative posture relationships, and the corresponding plurality of the third relative posture relationships and the plurality of the fourth relative posture relationships;

[0013] By minimizing the objective function, the optimal third relative posture relationship and the optimal fourth relative posture relationship are determined.

[0014] In some embodiments, constructing the objective function based on the plurality of the first relative posture relationships, the plurality of the second relative posture relationships, and the corresponding plurality of the third relative posture relationships, and the plurality of the fourth relative posture relationships includes:

[0015] Based on the relationship between the first relative posture relationship, the second relative posture relationship, the third relative posture relationship, and the fourth relative posture relationship, a corresponding plurality of error terms are determined, and the error term is A i ×B×C i -D;

[0016] Based on the multiple error terms, the objective function is constructed, and the objective function is:

[0017]

[0018] Wherein, n represents the number of rotations of the robot arm, Ai represents the i-th first relative posture relationship, B represents the fourth relative posture relationship; Ci represents the i-th second relative posture relationship, D represents the third relative posture relationship, |.| F represents the Frobenius norm.

[0019] In some embodiments, the objective function is minimized by the following formula (1) to determine the optimal third relative posture relationship and the optimal fourth relative posture relationship:

[0020]

[0021] In some embodiments, the variable matrix corresponding to the third relative posture relationship and the fourth relative posture relationship in formula (1) is expanded into a vector form:

[0022] b=vec(B), d=vec(D)

[0023] Vectorize the corresponding error term into the residual vector ri:

[0024] r i =vec(A i BC i -D)

[0025] The objective function is optimized as:

[0026]

[0027] Among them, b represents the vector after the fourth relative posture relationship is expanded, and d represents the vector after the third relative posture relationship is expanded.

[0028] In some embodiments, calibrating the installation position of the camera based on the optimal fourth relative posture relationship includes:

[0029] Converting the optimal fourth relative posture relationship into a translation vector and a rotation Euler angle;

[0030] The installation position of the camera is calibrated based on the translation vector and the rotation Euler angle.

[0031] In a second aspect, an embodiment of the present application further provides a device for calibrating the installation position of a camera on a robotic arm, wherein the robotic arm comprises a robotic arm base and a robotic arm, the robotic arm rotates about the robotic arm origin, the camera is mounted on the robotic arm, a calibration plate is placed within the visible range of the camera, and the device comprises:

[0032] An acquisition module is used to obtain a first relative posture relationship between the base of the robotic arm and the origin of the robotic arm after the robotic arm rotates a set angle each time, and to obtain a second relative posture relationship between the camera and the calibration plate by using the camera to identify the calibration object on the calibration plate;

[0033] A determination module, configured to determine an optimal fourth relative posture relationship between the origin of the robot arm and the camera based on a plurality of the first relative posture relationships, a plurality of the second relative posture relationships, and a third relative posture relationship between the robot arm base and the calibration plate;

[0034] A calibration module is used to calibrate the installation position of the camera based on the optimal fourth relative posture relationship.

[0035] In a third aspect, an embodiment of the present application further provides a system for calibrating the camera installation position on a robot arm, the system comprising: a calibration device for calibrating the camera installation position on a robot arm and a calibration plate as described in the second aspect above,

[0036] The calibration plate includes a calibration object, which is placed within the visual range of the camera;

[0037] The calibration device is used to calibrate the installation position of the camera on the robotic arm.

[0038] In a fourth aspect, an embodiment of the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method as described in the first aspect above.

[0039] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0040] The above-mentioned calibration method, device and system for the camera installation position on the robotic arm solves the problem of low efficiency and accuracy in the calibration of the camera installation position on the robotic arm, and realizes efficient and accurate calibration of the camera installation position on the robotic arm, ensuring that the robotic arm can obtain accurate visual data when performing tasks, thereby improving the reliability and accuracy of the operation. Compared with the existing manual calibration method, the method of the present application can realize automatic calibration, reduce human intervention, thereby reducing human operation errors, and improving the automation level and efficiency of the calibration process; at the same time, the calibration method of the present application makes the camera installation and calibration process simpler and faster, suitable for different operators, and further saves time and cost.

[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 is a schematic diagram of an application environment of a method for calibrating a camera installation position on a robotic arm in an embodiment;

[0044] Figure 2 is a flow chart of a method for calibrating the installation position of a camera on a robotic arm in an embodiment;

[0045] Figure 3 is a schematic diagram of the rotation of a robot arm in one embodiment;

[0046] Figure 4 is a schematic diagram of the relative position relationship between the robot arm base and the calibration plate in an embodiment;

[0047] Figure 5 is a structural block diagram of a device for calibrating the installation position of a camera on a mechanical arm in one embodiment;

[0048] Figure 6 It is a schematic diagram of the structure of a computer device in an embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0051] The calibration method for the camera installation position on the robot arm provided in this application can be applied to Figure 1In the application environment shown. The robot includes a robot base 10 and a robot arm 20, the robot arm 20 rotates with the robot arm origin (the robot arm origin refers to the connection point between the robot arm and the robot base), the camera 30 is installed on the robot arm 20, and a calibration plate 40 is placed within the visible range of the camera 30, and the calibration object on the calibration plate 40 is a QR code. Among them, the robot base coordinate system is link_1, the robot arm origin coordinate system is link_2, the camera coordinate system is camera_link, and the calibration plate coordinate system is Tag_link.

[0052] The present application provides a method for calibrating the camera installation position on a robotic arm, and the method is applied to Figure 1 The application environment in is used as an example to illustrate. Figure 2 As shown, the method comprises the following steps:

[0053] S201, after the robot arm rotates a set angle each time, a first relative posture relationship between the robot arm base and the origin of the robot arm is obtained, and the calibration object on the calibration plate is recognized by the camera to obtain a second relative posture relationship between the camera and the calibration plate.

[0054] like Figure 3 As shown, the robot arm is controlled to automatically rotate around the origin of the robot arm for multiple times, and it is ensured that the QR code is completely contained in the picture taken by the camera within the rotation range of the robot arm. After each rotation of a certain angle, the first relative posture relationship between the robot arm base and the origin of the robot arm is obtained according to the rotation angle; at the same time, after each rotation of a certain angle, the camera is used to recognize the QR code on the calibration plate to obtain the second relative posture relationship between the camera and the calibration plate.

[0055] The calculation process of the first relative posture relationship A between the robot base and the origin of the robot upper arm is as follows:

[0056] Define the transformation matrix A:

[0057]

[0058] Among them, R is the rotation matrix, T is the translation vector, and the translation vector T represents the fixed position relationship between the robot base coordinate system link_1 and the robot arm origin coordinate system link_2:

[0059]

[0060] During the rotation of the robot arm, only the pitch angle changes between the robot arm base coordinate system link_1 and the robot arm origin coordinate system link_2. The pitch angle is a rotation around the y-axis. The rotation matrix around the y-axis (R y (θ)) is:

[0061]

[0062] Combining the rotation matrix R and the translation vector T, we get the transformation matrix A:

[0063]

[0064] The calculation process of the second relative posture relationship C between the camera and the calibration plate is as follows:

[0065] Define the transformation matrix C:

[0066]

[0067] Among them, R is the rotation matrix, T is the translation vector, and the translation vector T represents the positional relationship between the camera coordinate system camera_link and the calibration plate coordinate system Tag_link:

[0068]

[0069] Rotation matrix about the (x) axis for:

[0070]

[0071] The rotation matrix (R) around the (y) axis y §)for:

[0072]

[0073] The rotation matrix (R) around the (z) axis z (y)) is:

[0074]

[0075] The calculation formula of the rotation matrix R is as follows:

[0076]

[0077] exist Figure 1 and Figure 2 In the figure, the coordinate systems of the camera, calibration plate, robot arm, robot base, etc. are established with the y-axis perpendicular to the paper facing outward, and the x-axis and z-axis are as shown in the figure.

[0078] S202, determining the optimal fourth relative posture relationship between the robot arm origin and the camera based on multiple first relative posture relationships, multiple second relative posture relationships and a third relative posture relationship between the robot arm base and the calibration plate.

[0079] Specifically, multiple groups of first relative pose relationships and second relative pose relationships can be obtained through the above S201. In this step, the optimal fourth relative pose relationship between the origin of the robotic arm and the camera is determined according to multiple first relative pose relationships, multiple second relative pose relationships and the third relative pose relationship between the robotic arm base and the calibration plate. Since the camera is installed on the robotic arm, the position of the robotic arm base relative to the calibration plate is fixed. Therefore, the third relative pose relationship and the fourth relative pose relationship are fixed transformation matrices.

[0080] S203: Calibrate the installation position of the camera based on the optimal fourth relative posture relationship.

[0081] The above-mentioned calibration method for the camera installation position on the robot arm quickly and accurately determines the installation position of the camera on the robot arm's upper arm in an automated manner, solving the problem of low efficiency and accuracy in the calibration of the camera installation position on the robot arm, and realizing efficient and accurate calibration of the camera installation position on the robot arm, ensuring that the robot arm can obtain accurate visual data when performing tasks, thereby improving the reliability and accuracy of the operation. Compared with the existing manual calibration method, the method of the present application can realize automatic calibration, reduce human intervention, thereby reducing human operation errors, and improving the automation level and efficiency of the calibration process; at the same time, the calibration method of the present application makes the camera installation and calibration process simpler and faster, suitable for different operators, and further saves time and cost.

[0082] In one embodiment, obtaining the third relative position and posture relationship between the robotic arm base and the calibration plate includes the following:

[0083] like Figure 4 As shown, first, based on the first relative posture relationship A between the robot base and the origin of the robot arm upper arm and the fourth relative posture relationship B between the origin of the robot arm upper arm and the camera, the fifth relative posture relationship E between the robot base and the camera is obtained; then, based on the fifth relative posture relationship E between the robot base and the camera and the second relative posture relationship C between the camera and the calibration plate, the third relative posture relationship D between the robot base and the calibration plate is obtained.

[0084] Therefore, each group of the first relative posture relationship, the second relative posture relationship, the third relative posture relationship, and the fourth relative posture relationship has the following relationship: the third relative posture relationship = the first relative posture relationship * the second relative posture relationship * the fourth relative posture relationship, that is:

[0085] D = A × B × C;

[0086] Among them, A represents the first relative posture relationship between the robot base and the origin of the robot arm, C represents the second relative posture relationship between the camera and the calibration plate, B represents the fourth relative posture relationship between the origin of the robot arm and the camera, and D represents the third relative posture relationship between the robot base and the calibration plate.

[0087] In one embodiment, determining the optimal fourth relative posture relationship between the origin of the robot arm and the camera based on the plurality of the first relative posture relationships, the plurality of the second relative posture relationships, and the third relative posture relationship between the robot arm base and the calibration plate comprises the following steps:

[0088] S301, constructing an objective function based on the relationship between the plurality of the first relative posture relationships, the plurality of the second relative posture relationships, and the corresponding plurality of the third relative posture relationships, and the plurality of the fourth relative posture relationships.

[0089] S302: Determine the optimal third relative posture relationship and the optimal fourth relative posture relationship by minimizing the objective function.

[0090] Among them, S301 determines the corresponding multiple error terms according to multiple first relative posture relationships, multiple second relative posture relationships and the third relative posture relationship between the robot arm base and the calibration plate; and constructs the objective function based on the multiple error terms.

[0091] In one embodiment, based on the relationship between the first relative posture relationship, the second relative posture relationship, the third relative posture relationship, and the fourth relative posture relationship, a corresponding plurality of error terms are determined, and the error term is A i ×B×C i -D.

[0092] Based on the multiple error terms, the objective function is constructed, and the objective function is:

[0093]

[0094] Wherein, n represents the number of rotations of the robot arm, A irepresents the i-th relative position and posture relationship of the first, and B represents the fourth relative position and posture relationship; C i represents the i-th second relative posture relationship, D represents the i-th third relative posture relationship, |.| F represents the Frobenius norm.

[0095] The objective function is minimized by the following formula (1) to determine the optimal third relative posture relationship and the optimal fourth relative posture relationship:

[0096]

[0097] In an example embodiment, the QR code is placed within the range where the robot arm can be seen when it is rotated to the minimum angle and the maximum angle, and then the robot arm is rotated to the minimum angle, and the calibration program is started. The program automatically controls the rotation of the robot arm. After each rotation of a certain angle, the change matrix A and the change matrix C are calculated and recorded after the robot arm stops, and then the robot arm continues to rotate. When the number of records reaches a certain number, n groups of data are obtained, and each group of data contains the transformation matrix A. i and the transformation matrix C i , where i = 1, 2, ..., n. The objective function to be optimized is to find the transformation matrix B and the transformation matrix D so that for all i, A i *B*C i As close as possible to D, with the smallest difference between the two, specifically including the following steps:

[0098] Define the error function: For each set of data (A i ,C i ), define the error term diff i =A i *B*C i -D.

[0099] Construct an optimization problem: The goal is to find the optimal transformation matrix B and transformation matrix D so that the n error terms diff i The sum of squares is minimized, that is, we want to minimize the objective function:

[0100]

[0101] Parameterization and optimization: Use Ceres Solver for iterative optimization, take transformation matrix B and transformation matrix D as optimization variables, and use automatic differentiation to calculate the gradient of objective function f(B,D) with respect to transformation matrix B and transformation matrix D. CeresSolver updates transformation matrix B and transformation matrix D through iterative methods to minimize the objective function. CeresSolver is a C++ nonlinear optimization library open sourced by Google, which can solve nonlinear least squares problems with or without constraints. In Ceres Solver, each set of data (A i ,C i ) corresponds to a residual block, so it is necessary to i ,C i ) defines a residual block and adds it to the optimization problem. Ceres Solver will minimize the sum of squares of all residual blocks to achieve the optimization goal.

[0102] Assume that transformation matrix B and transformation matrix D are the solutions to the optimization problem, then:

[0103]

[0104] Ceres Solver finds such transformation matrices B and D through an iterative optimization process to achieve the following results for all given data sets (A i ,C i ), A i *B*C i The difference with D is minimal.

[0105] Furthermore, in one of the embodiments, the transformation matrix B and the transformation matrix D are expanded into vector form to facilitate iterative optimization, and the elements of the transformation matrix B and the transformation matrix D are optimization variables:

[0106] b=vec(B), d=vec(D)

[0107] For each set of data (A i ,C i ), define the residual vector r i for:

[0108] r i =vec(A i BC i -D)

[0109] Furthermore, the optimization objective function is:

[0110]

[0111] In one of the embodiments, calibrating the installation position of the camera based on the optimal fourth relative posture relationship includes: converting the optimal fourth relative posture relationship into a translation vector and a rotation Euler angle; and calibrating the installation position of the camera based on the translation vector and the rotation Euler angle.

[0112] After minimizing the objective function, this embodiment obtains the optimal fourth relative posture relationship B between the origin of the robot arm and the camera, and converts the optimal fourth relative posture relationship into a translation vector and a rotation Euler angle, which are used as the installation position of the camera for calibration, ensuring that the robot arm can obtain accurate visual data when performing tasks.

[0113] Based on the same inventive concept, the embodiment of the present application also provides a device for calibrating the installation position of a camera on a robotic arm, such as Figure 5 As shown, the device comprises:

[0114] The acquisition module 10 is used to obtain a first relative posture relationship between the robot base and the origin of the robot arm each time the robot arm rotates a set angle, and to obtain a second relative posture relationship between the camera and the calibration plate by using the camera to identify the calibration object on the calibration plate.

[0115] The determination module 20 is used to determine the optimal fourth relative posture relationship between the origin of the robot arm and the camera based on multiple first relative posture relationships, multiple second relative posture relationships and the third relative posture relationship between the robot arm base and the calibration plate.

[0116] The calibration module 30 is used to calibrate the installation position of the camera based on the optimal fourth relative posture relationship.

[0117] In one embodiment, each group of the first relative posture relationship, the second relative posture relationship, the third relative posture relationship, and the fourth relative posture relationship has the following relationship:

[0118] The third relative posture relationship=the first relative posture relationship*the second relative posture relationship*the fourth relative posture relationship.

[0119] In one embodiment, the determination module 20 is also used to: construct an objective function based on the relationship between multiple first relative posture relationships, multiple second relative posture relationships and the corresponding multiple third relative posture relationships and multiple fourth relative posture relationships; and determine the optimal third relative posture relationship and the optimal fourth relative posture relationship by minimizing the objective function.

[0120] In one embodiment, the determination module 20 is further configured to:

[0121] Based on the relationship between the first relative posture relationship, the second relative posture relationship, the third relative posture relationship, and the fourth relative posture relationship, a corresponding plurality of error terms are determined, and the error term is A i ×B×C i -D;

[0122] Based on the multiple error terms, the objective function is constructed, and the objective function is:

[0123]

[0124] Wherein, n represents the number of rotations of the robot arm, A i represents the i-th relative position and posture relationship of the first, and B represents the fourth relative position and posture relationship; C i represents the i-th second relative posture relationship, D represents the third relative posture relationship, |.| F represents the Frobenius norm.

[0125] The objective function is minimized by the following formula (1) to determine the optimal third relative posture relationship and the optimal fourth relative posture relationship:

[0126]

[0127] Expand the variable matrix corresponding to the third relative posture relationship and the fourth relative posture relationship in formula (1) into a vector form:

[0128] b=vec(B), d=vec(D)

[0129] The corresponding error term is vectorized into a residual vector r i :

[0130] r i =vec(A i BC i -D)

[0131] The objective function is optimized as:

[0132]

[0133] Among them, b represents the vector after the fourth relative posture relationship is expanded, and d represents the vector after the third relative posture relationship is expanded.

[0134] In one embodiment, the calibration module 30 is further used to: convert the optimal fourth relative posture relationship into a translation vector and a rotation Euler angle; and calibrate the installation position of the camera based on the translation vector and the rotation Euler angle.

[0135] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0136] Based on the same inventive concept, an embodiment of the present application also provides a calibration system for the installation position of a camera on a robotic arm, the system comprising a calibration device for the installation position of a camera on a robotic arm and a calibration plate as provided in the above embodiment, the calibration plate comprising a calibration object placed within the visible range of the camera; the calibration device is used to calibrate the installation position of the camera on the robotic arm.

[0137] Compared with the existing manual calibration method, the calibration system of this embodiment can realize automatic calibration, reduce human intervention, thereby reducing human operation errors, improving the automation level and efficiency of the calibration process, and realizing efficient and accurate calibration of the camera installation position on the robotic arm.

[0138] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for calibrating the installation position of a camera on a robotic arm is implemented.

[0139] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in any of the above-mentioned embodiments of the method for calibrating the installation position of a camera on a robotic arm are implemented.

[0141] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0142] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for calibrating the installation position of a camera on a robotic arm, wherein the robotic arm comprises a robotic arm base and a robotic arm upper arm, the robotic arm upper arm rotates about the robotic arm upper arm origin, the camera is installed on the robotic arm upper arm, and a calibration plate is placed within the visual range of the camera, characterized in that: The method comprises: After the robot arm rotates by a set angle each time, a first relative posture relationship between the robot arm base and the origin of the robot arm is obtained, and a second relative posture relationship between the camera and the calibration plate is obtained by using the camera to recognize the calibration object on the calibration plate; Determine an optimal fourth relative posture relationship between the origin of the upper arm of the robotic arm and the camera based on a plurality of the first relative posture relationships, a plurality of the second relative posture relationships, and a third relative posture relationship between the robotic arm base and the calibration plate; Based on the optimal fourth relative posture relationship, the installation position of the camera is calibrated.

2. The method according to claim 1, characterized in that Each group of the first relative posture relationship, the second relative posture relationship, the third relative posture relationship, and the fourth relative posture relationship has the following relationship: The third relative posture relationship=the first relative posture relationship*the second relative posture relationship*the fourth relative posture relationship.

3. The method according to claim 2, characterized in that Determining the optimal fourth relative posture relationship between the origin of the robot arm and the camera based on the plurality of the first relative posture relationships, the plurality of the second relative posture relationships, and the third relative posture relationship between the robot arm base and the calibration plate comprises: constructing an objective function based on the relationship between the plurality of the first relative posture relationships, the plurality of the second relative posture relationships, and the corresponding plurality of the third relative posture relationships and the plurality of the fourth relative posture relationships; By minimizing the objective function, the optimal third relative posture relationship and the optimal fourth relative posture relationship are determined.

4. The method according to claim 3, characterized in that The constructing of the objective function based on the plurality of the first relative posture relationships, the plurality of the second relative posture relationships, and the corresponding plurality of the third relative posture relationships, and the plurality of the fourth relative posture relationships comprises: Based on the relationship between the first relative posture relationship, the second relative posture relationship, the third relative posture relationship, and the fourth relative posture relationship, a corresponding plurality of error terms are determined, and the error term is A i ×B×C i -D; Based on the multiple error terms, the objective function is constructed, and the objective function is: Wherein, n represents the number of rotations of the robot arm, A i represents the i-th relative position and posture relationship of the first, and B represents the fourth relative position and posture relationship; C i represents the i-th second relative posture relationship, D represents the third relative posture relationship, |.| F represents the Frobenius norm.

5. The method according to claim 4, characterized in that The objective function is minimized by the following formula (1) to determine the optimal third relative posture relationship and the optimal fourth relative posture relationship:

6. The method according to claim 5, characterized in that Expand the variable matrix corresponding to the third relative posture relationship and the fourth relative posture relationship in formula (1) into a vector form: b=vec(B), d=vec(D) The corresponding error term is vectorized into a residual vector r i : r i =vec(A i BC i -D) The objective function is optimized as: Among them, b represents the vector after the fourth relative posture relationship is expanded, and d represents the vector after the third relative posture relationship is expanded.

7. The method according to claim 1, characterized in that The calibrating the installation position of the camera based on the optimal fourth relative posture relationship includes: Converting the optimal fourth relative posture relationship into a translation vector and a rotation Euler angle; The installation position of the camera is calibrated based on the translation vector and the rotation Euler angle.

8. A device for calibrating the installation position of a camera on a robotic arm, the robotic arm comprising a robotic arm base and a robotic arm upper arm, the robotic arm upper arm rotates about the robotic arm upper arm origin, the camera is installed on the robotic arm upper arm, a calibration plate is placed within the visual range of the camera, characterized in that: The device comprises: An acquisition module is used to obtain a first relative posture relationship between the base of the robotic arm and the origin of the robotic arm after the robotic arm rotates a set angle each time, and to obtain a second relative posture relationship between the camera and the calibration plate by using the camera to identify the calibration object on the calibration plate; A determination module, configured to determine an optimal fourth relative posture relationship between the origin of the robot arm and the camera based on a plurality of the first relative posture relationships, a plurality of the second relative posture relationships, and a third relative posture relationship between the robot arm base and the calibration plate; A calibration module is used to calibrate the installation position of the camera based on the optimal fourth relative posture relationship.

9. A calibration system for the camera installation position on a robotic arm, characterized in that: The system comprises: a calibration device for the camera installation position on a robot arm and a calibration plate as claimed in claim 7, The calibration plate includes a calibration object, which is placed within the visual range of the camera; The calibration device is used to calibrate the installation position of the camera on the robotic arm.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images