Method and system for camera-odometry calibration using jig

The jig-based calibration method addresses inaccuracies in camera-odometry by adjusting the marker board's position and angle relative to the robot, enabling precise calibration without robot movement, thus improving the robot's positional accuracy.

KR103046085B1Active Publication Date: 2026-11-11NAVER CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
KR1020230139397
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-10-18
Publication Date
2026-11-11
Estimated Expiration
2043-10-18

AI Technical Summary

Technical Problem

Existing camera-odometry calibration methods for robots are inaccurate due to uneven floors or narrow spaces, leading to unreliable odometry information, which affects the precision of robot movements based on camera images.

Method used

A calibration method using a jig to adjust the coupling position and angle between a marker board and a robot, allowing for precise collection of calibration data without actual robot movement, by analyzing images to determine the relative pose of the marker board's coordinate system with respect to the robot's coordinate system, and calculating a camera-odometry matrix to minimize projection errors.

Benefits of technology

Provides accurate camera-odometry calibration by using a jig, enhancing the precision of robot movements and reducing reliance on potentially inaccurate odometry information, thereby improving the robot's ability to understand rotation and translation relationships between its reference and camera coordinate systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112023114207857-PAT00077_ABST
    Figure 112023114207857-PAT00077_ABST
Patent Text Reader

Abstract

A method and system for camera-odometry calibration using a jig are disclosed. A calibration method according to one embodiment may include the steps of: receiving a plurality of images captured by a camera included in a robot while adjusting at least one of a coupling position and a coupling angle between a marker board on a jig linked to a robot and the jig; analyzing the plurality of images according to at least one of the coupling position and a coupling angle to collect the pose of the coordinate system of the marker board relative to the coordinate system of the robot as calibration data; and calibrating the pose of the coordinate system of the camera relative to the coordinate system of the robot using the collected calibration data by the at least one processor.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The following description relates to a camera-odometry calibration method and system using a jig. Background Technology

[0002] For a robot to perform precise tasks based on camera images, it is necessary to understand the rotation and translation relationships between the robot's reference coordinate system {Odom} and the camera coordinate system {Cam}. The calibration between a camera and a manipulator, such as a robot arm, is widely known as hand-eye calibration. The calibration method between the reference coordinate system {Odom} and the camera coordinate system {Cam} of a wheel-based mobile robot is also referred to as camera-odometry calibration. In this case, the mobile robot's reference coordinate system {Odom} typically has the center of both wheels as the origin and the straight direction as the x-axis. Meanwhile, it is assumed that the mobile robot moves on a plane.

[0003] The input value for camera-odometry calibration is the robot's relative pose at position j with respect to position i when the robot moves from position i to position j. and the pose of the marker board with respect to the camera at position i , and the pose of the marker board for the camera at position j It may include.

[0004] Here, and It is very accurate compared to odometry information. On the other hand, This can be obtained through odometry information; however, odometry information obtained through robot movement may be inaccurate if the floor is uneven or the space for the robot to move is narrow. Therefore, the odometry information obtained through It can also be inaccurate.

[0005] [Prior Literature Number]

[0006] Korean Published Patent No. 10-2023-0000585 The problem to be solved

[0007] A method and system for camera-odometry calibration using a jig are provided. means of solving the problem

[0008] A calibration method for a computer device comprising at least one processor, comprising: receiving a plurality of images captured by a camera included in a robot while adjusting at least one of a coupling position and a coupling angle between a marker board on a jig linked to a robot by the at least one processor; analyzing the plurality of images according to at least one of the coupling position and a coupling angle by the at least one processor to collect a pose relative to the coordinate system of the marker board with respect to the coordinate system of the robot as calibration data; and calibrating the pose of the coordinate system of the camera with respect to the coordinate system of the robot using the collected calibration data by the at least one processor.

[0009] According to one aspect, the jig includes a plurality of coupling positions located on a single straight line, and the step of receiving the plurality of images may be characterized by receiving a plurality of images of the marker boards of two or more of the plurality of coupling positions.

[0010] According to another aspect, the jig comprises a plurality of joining positions arranged in the form of a matrix having a plurality of columns and rows, and the step of receiving the plurality of images may be characterized by receiving a plurality of images of the marker boards of joining positions of different columns or different rows among the plurality of joining positions.

[0011] According to another aspect, the coupling angle is determined by the rotation of the marker board with respect to the y-axis of the coordinate system of the marker board, and the step of receiving the plurality of images may be characterized by receiving the plurality of images in which the marker board is photographed at the same coupling position with different coupling angles.

[0012] According to another aspect, the calibration step may be characterized by including: a step of extracting first coordinates of corner points of the markers included in the marker board by analyzing a marker image taken of the marker board at a first position of the robot; a step of projecting second coordinates of the corner points of the markers at a second position of the robot onto the marker image; and a step of calculating a camera-odometry matrix that minimizes a projection error based on the difference between the first coordinates and the second coordinates.

[0013] According to another aspect, the camera-odometry matrix may be characterized by including information regarding rotation and translation relationships between the coordinate system of the robot and the coordinate system of the camera.

[0014] According to another aspect, the second coordinates may be characterized by being calculated by the matrix product of a first matrix obtained for the camera, a second matrix representing the pose of the robot's coordinate system relative to the camera's coordinate system at the first position, a third matrix representing the pose of the robot's coordinate system at the second position relative to the robot's coordinate system at the first position, a fourth matrix representing the pose of the camera's coordinate system relative to the robot's coordinate system at the second position relative to the pose of the marker board's coordinate system relative to the camera's coordinate system at the second position relative to the pose of the marker board's coordinate system at the second position relative to the pose of the marker board's coordinate system at the second position relative to the first matrix.

[0015] According to another aspect, the third matrix is ​​calculated based on the 3-1 matrix representing the relative pose of the coordinate system of the marker board with respect to the coordinate system of the robot at the first position obtained without movement of the robot using the jig, and the 3-2 matrix representing the relative pose of the coordinate system of the marker board with respect to the coordinate system of the robot at the second position, and the first position and the second position may be the same position, but the coupling position or coupling angle of the marker board coupled with the jig may be different from each other.

[0016] According to another aspect, the second matrix includes the inverse of the camera-odometry matrix, and the fourth matrix includes the camera-odometry matrix, and the step of calculating the camera-odometry matrix may be characterized by calculating the fourth matrix that minimizes the projection error according to the difference between the first coordinates and the second coordinates while the first matrix, the third matrix, the fifth matrix, and the sixth matrix are fixed.

[0017] According to another aspect, the calibration step may be characterized by including: a step of analyzing a marker image captured of the marker board to extract first coordinates of corner points of the marker included in the marker board; a step of projecting second coordinates of the corner points of the marker onto the marker image; and a step of calculating a camera-odometry matrix that minimizes a projection error based on the difference between the first coordinates and the second coordinates.

[0018] According to another aspect, the second coordinates may be characterized by including the coordinates of the corner points of the marker obtained from an image of a marker board with a different combination position or combination angle than when the first coordinates were extracted.

[0019] According to another aspect, the second coordinates may be characterized by being calculated by the matrix product of a first matrix obtained for the camera, a second matrix representing the pose relative to the coordinate system of the robot with respect to the coordinate system of the camera, a third matrix representing the pose relative to the coordinate system of the marker board with respect to the coordinate system of the robot, and a fourth matrix obtained for the marker board.

[0020] According to another aspect, the second matrix may be characterized by being calculated based on matrices representing the relative poses of the coordinate system of the marker board with respect to the coordinate system of the robot obtained at different joint positions or different joint angles using the jig.

[0021] According to another aspect, the second matrix includes the camera-odometry matrix, and the step of calculating the camera-odometry matrix may be characterized by calculating the second matrix that minimizes the projection error according to the difference between the first coordinates and the second coordinates while the first matrix, the third matrix, and the fourth matrix are fixed.

[0022] A calibration method for a computer device comprising at least one processor is provided, the method comprising: a step of analyzing a marker image captured by the at least one processor at a first position of a robot to extract first coordinates of corner points of a marker included in the marker board; a step of projecting second coordinates of the corner points of the marker at a second position of the robot onto the marker image by the at least one processor; and a step of calculating a camera-odometry matrix that minimizes a projection error based on the difference between the first coordinates and the second coordinates by the at least one processor.

[0023] A computer program stored on a computer-readable recording medium is provided to be combined with a computer device to execute the above method on the computer device.

[0024] A computer-readable recording medium is provided on which a program for executing the above method is recorded on a computer device.

[0025] The present invention provides a computer device comprising at least one processor implemented to execute a command readable by a computer device, wherein the at least one processor receives a plurality of images captured by a camera included in the robot while adjusting at least one of a coupling position and a coupling angle between the marker board and the jig linked to the robot, analyzes the plurality of images according to at least one of the coupling position and a coupling angle to collect a pose relative to the coordinate system of the marker board with respect to the coordinate system of the robot as calibration data, and uses the collected calibration data to calibrate the pose of the coordinate system of the camera with respect to the coordinate system of the robot.

[0026] A computer device is provided that includes at least one processor implemented to execute a command readable by a computer device, wherein the at least one processor analyzes a marker image captured by a marker board at a first position of a robot to extract first coordinates of corner points of a marker included in the marker board, projects second coordinates of the corner points of the marker at a second position of the robot onto the marker image, and calculates a camera-odometry matrix that minimizes a projection error based on the difference between the first coordinates and the second coordinates. Effects of the invention

[0027] A camera-odometry calibration method and system using a jig can be provided. Brief explanation of the drawing

[0028] FIGS. 1 to 3 are drawings for explaining the process of collecting calibration data using a jig in an embodiment of the present invention. FIG. 4 is a drawing illustrating an actual implementation example of a jig in which the joining positions are arranged in the form of a matrix in one embodiment of the present invention. FIG. 5 is a diagram illustrating a calibration method using movement information of a robot in an embodiment of the present invention. FIG. 6 is a diagram illustrating a calibration method that does not use robot movement information in an embodiment of the present invention. FIG. 7 is a flowchart illustrating an example of a camera-odometry calibration method according to an embodiment of the present invention. FIG. 8 is a flowchart illustrating an example of a calibration method using robot movement information in an embodiment of the present invention. FIG. 9 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention. Specific details for implementing the invention

[0029] Hereinafter, embodiments will be described in detail with reference to the attached drawings.

[0030] Embodiments of the present invention are relative poses of the camera with respect to odometry A camera-odometry calibration method and system for calibrating are provided. can be expressed as shown in Equation 1 below. Here, r 11 to r 33 is the relationship of rotation between the robot's reference coordinate system and the camera coordinate system, x , y , z It can represent the translation relationship between the robot's reference coordinate system and the camera coordinate system.

[0031]

[0032] Also, the camera's relative pose to the odometry and the relative pose of odometry with respect to the camera It can have an inverse matrix relationship as shown in mathematical equation 2 below.

[0033]

[0034] First, a jig generally refers to an auxiliary device used in machining to easily and accurately determine the machining position; it is a special tool attached to a workpiece to determine the position of the machining part and simultaneously guide the machining process. In the embodiments of the present invention, the jig may refer to an auxiliary device precisely manufactured to accurately determine the relative position between a mobile robot and a marker board (more precisely, the relative pose of the marker board). For example, by connecting a marker board to a specific position on a jig linked to the robot while the robot is stationary, calibration data can be collected without moving the robot.

[0035] FIGS. 1 to 3 are drawings for explaining the process of collecting calibration data using a jig in an embodiment of the present invention.

[0036] First, FIG. 1 shows a robot (110), a jig (120), and a marker board (130). The robot (110) may include a camera (111) for capturing surrounding images and a wheel (112) for moving the robot (110). Additionally, the jig (120) may be linked to the robot (110) and may be implemented to include multiple connection positions for connecting the marker board (130) to a specific location on the jig (120). At this time, the connection angle of the marker board (130) may be changed as it rotates (for example, rotation about the y-axis of the marker coordinate system) while connected at a specific connection position on the jig (120).

[0037] FIG. 2 illustrates an example in which the connection position and connection angle of a marker board (130) connected to a jig (120) linked to a robot (110) are changed. At this time, the robot (110) is positioned in a fixed state, but the pose of the marker coordinate system {A} relative to the robot coordinate system {R}. It may change.

[0038] The change in relative position between the marker board (130) and the robot (110) due to the movement of the marker board (130) can produce the same effect as the movement of the robot (110) shown in FIG. 3. In other words, without moving the robot (110), by changing the pose (joining position and / or joining angle) of the marker board (130) of the jig (120), the pose of the marker coordinate system {A} relative to the robot coordinate system {R} is obtained as calibration data identical to the case where the robot (110) has moved. You will be able to obtain.

[0039] In the embodiments of FIGS. 1 to 3 described above, the jig (120) includes a plurality of joining positions located on a single straight line, but the joining positions may be arranged in the form of a matrix having a plurality of columns and rows.

[0040] FIG. 4 illustrates an actual implementation example of a jig in which coupling positions are arranged in the form of a matrix, in an embodiment of the present invention. The images in FIG. 4 show an implementation example of a jig having coupling positions in the form of a 2x3 matrix. In this case, by placing marker boards at various coupling positions of the jig at various coupling angles and capturing images, the pose of the marker coordinate system {A} relative to the robot coordinate system {R} can be obtained without moving the robot. It can be collected as calibration data.

[0041] The calibration method is described below. The calibration method can be performed by a computer device included in the robot or by a computer device implemented as separate hardware from the robot that receives data from the robot.

[0042] First, the calibration method may include a calibration method that uses robot movement information and / or a calibration method that does not use robot movement information.

[0043] FIG. 5 is a diagram illustrating a calibration method using robot movement information in an embodiment of the present invention. FIG. 5 shows a case where the robot actually moves, but as explained above through the embodiments of FIG. 1 to 3, the movement of the robot can be replaced by a method using a jig. In this case, a calibration method using robot movement information can be provided without actually moving the robot.

[0044] The computer device analyzes an image of the marker on the marker board taken at the robot's i-th position (hereinafter referred to as the 'i-th marker image') to determine the corner points of the marker It can be extracted. Here, the corner points It may include the coordinates in the marker coordinate system of representative points of the object contained by the marker. For example, if the marker includes a plurality of rectangles arranged as in the marker board shown in FIG. 5, the coordinates in the marker coordinate system of the vertices of the rectangles are corner points. It can be extracted as.

[0045] In addition, the computer device can project the corner points of the marker at the robot's j-th position onto the i-th marker image. At this time, the corner points projected onto the i-th marker image It may include the coordinates of projected points in the i-th marker image.

[0046] At this time, the computer device The coordinates of The projection error, which is the difference between the coordinates, is minimized as in Equation 3 (= X) can be calculated using the optimization method shown in mathematical formula 4 below.

[0047]

[0048]

[0049] Here, K can be a well-known camera matrix, and may be a value extracted from the produced marker board. "·" may mean matrix multiplication.

[0050] In this case, when a jig is used, as explained above through the embodiments of FIGS. 1 to 3, and You can obtain, Through matrix multiplication between can be obtained. At this time, is of mathematical formula 4 It can respond to.

[0051] The result obtained from the robot's actual movement Obtained using a jig It can provide a more accurate value.

[0052] However, in this embodiment, the position of the camera coordinate system {Cam} in the z-axis direction relative to the robot's reference coordinate system {Odom} is unknown.

[0053] FIG. 6 is a diagram illustrating a calibration method that does not use robot movement information in an embodiment of the present invention. FIG. 6 describes a calibration method in which the position of the camera coordinate system {Cam} relative to the robot's reference coordinate system {Odom} in the z-axis direction can be determined using a jig without actually moving the robot.

[0054] The computer device analyzes the image of the marker on the marker board and the corner points of the marker It can extract. In addition, the computer device can project the corner points of the marker onto the marker image. The marker image may exist in multiple forms depending on the connection position and / or connection angle of the marker board on the jig, rather than just one, and since the change in the connection position and / or connection angle of the marker board can correspond to the movement of the robot, the projected points and extracted corner points Projection errors, which are differences between them, may occur.

[0055] At this time, the computer device minimizes this projection error as shown in Equation 5 (= X) can be calculated using an optimization method such as Equation 5 below.

[0056]

[0057]

[0058] Here, K can be a well-known camera matrix as previously explained, and may be a value extracted for the created marker board. Also, (= Since ) can be obtained using a jig, the computer device can process camera-odometry calibration by calculating X that minimizes projection error.

[0059] While the embodiment of FIG. 5 is a calibration method applicable in both cases where the robot actually moves and where a jig is used, the embodiment of FIG. 6 can be applied only when a jig is used. Instead, as previously explained, through the embodiment of FIG. 6, the position in the z-axis direction of the camera coordinate system {Cam} relative to the robot's reference coordinate system {Odom} can be determined.

[0060] FIG. 7 is a flowchart illustrating an example of a camera-odometry calibration method according to an embodiment of the present invention. The calibration method according to the present embodiment may be performed by a computer device that is included in a robot or communicates with the robot. In this case, the processor of the computer device may be implemented to execute a control instruction according to the code of an operating system included in memory or the code of at least one computer program. Here, the processor may control the computer device so that the computer device performs steps (710 to 730) included in the method of FIG. 7 according to the control instruction provided by the code stored in the computer device.

[0061] In step (710), the computer device can receive multiple images captured by a camera included in the robot while adjusting at least one of the connection position and connection angle between the marker board and the jig on the jig connected to the robot. As previously described, the jig is connected to the robot and can help generate images that have the same effect as the robot moving by adjusting the connection position and / or connection angle of the marker board connected to the jig without the robot moving. The reception of these images can correspond to receiving from the camera if the computer device is included in the robot, and can correspond to receiving from the robot if the computer device is a separate hardware device from the robot.

[0062] In one embodiment, the jig may include a plurality of joining positions located on a single straight line. In this case, the computer device may receive a plurality of images of marker boards at two or more of the plurality of joining positions. In another embodiment, the jig may include a plurality of joining positions arranged in the form of a matrix having a plurality of columns and rows. In this case, the computer device may receive a plurality of images of marker boards at joining positions in different columns or different rows among the plurality of joining positions. In yet another embodiment, the joining angle may be determined by the rotation of the marker board with respect to the y-axis of the marker board's coordinate system. In this case, the computer device may receive a plurality of images of marker boards at different joining angles at the same joining position. According to an embodiment, the computer device may receive a plurality of images of marker boards at different joining positions and different joining angles.

[0063] In step (720), the computer device can analyze a plurality of images according to at least one of the coupling position and the coupling angle to collect the relative pose of the marker board coordinate system with respect to the robot coordinate system as calibration data. The pose of the marker coordinate system {A} with respect to the robot coordinate system {R} through the embodiments of FIGS. 1 to 3 above. It was explained that it was collected as calibration data. Practically, the pose It can take the form of a matrix. As explained earlier through Equation 2, Is Since it can be obtained as the inverse matrix, instead of the pose of the marker coordinate system {A} with respect to the robot coordinate system {R}, the pose of the robot coordinate system {R} with respect to the marker coordinate system {A} It may also be collected as calibration data.

[0064] In step (730), the computer device can calibrate the pose of the camera's coordinate system relative to the robot's coordinate system using the collected calibration data.

[0065] In one embodiment, the computer device can analyze a marker image of a marker board taken from a first position of the robot to extract first coordinates of the corner points of the markers included in the marker board. These first coordinates are as described above It can correspond to. In addition, the computer device can project the second coordinates of the corner points of the marker at the second position of the robot onto the marker image. Here, the second coordinates are as described above. It can correspond to. In this case, the computer device can calculate a camera-odometry matrix that minimizes the projection error resulting from the difference between the first coordinates and the second coordinates. Here, the camera-odometry matrix may include information regarding rotation relationships and translation relationships between the robot's coordinate system and the camera's coordinate system.

[0066] As a more specific example, the second coordinates can be calculated by the matrix product of a first matrix obtained for the camera, a second matrix representing the pose of the robot's coordinate system relative to the camera's coordinate system at the first position, a third matrix representing the pose of the robot's coordinate system at the second position relative to the robot's coordinate system at the first position, a fourth matrix representing the pose of the camera's coordinate system relative to the robot's coordinate system at the second position, a fifth matrix representing the pose of the marker board's coordinate system relative to the camera's coordinate system at the second position, and a sixth matrix obtained for the marker board. Here, the first matrix is ​​K in Equation 4, and the second matrix is ​​of Equation 4 E., the third matrix is ​​of Equation 4. E., the fourth matrix is ​​of mathematical formula 4. E., the fifth matrix is ​​of Equation 4. E., the 6th matrix is ​​of Equation 4. Each can correspond to.

[0067] Meanwhile, in an embodiment utilizing robot movement information, matrices based on the robot's position can actually be calculated and used; however, in this embodiment utilizing a jig, the third matrix can be calculated based on the 3-1 matrix, which represents the relative pose of the marker board's coordinate system to the robot's coordinate system at a first position obtained without robot movement using the jig, and the 3-2 matrix, which represents the relative pose of the marker board's coordinate system to the robot's coordinate system at a second position. Here, the 3-1 matrix is ​​the previously explained E, the 3-2 matrix is ​​the previously explained Each can correspond to one. As already explained, Through matrix multiplication between It can be saved, is of mathematical formula 4 It can correspond to. In this case, the first position and the second position described above may be the same position, provided that the joining position or joining angle of the marker board joined to the jig may be different from each other.

[0068] Additionally, the second matrix may include the inverse of the camera-odometry matrix, and the fourth matrix may include the camera-odometry matrix. In this case, the computer device can calculate a camera-odometry matrix containing information about the rotation relationship and translation relationship between the robot's coordinate system and the camera's coordinate system by calculating a fourth matrix that minimizes the projection error based on the difference between the first coordinates and the second coordinates while the first, third, fifth, and sixth matrices are fixed.

[0069] In another embodiment, the computer device can analyze a marker image captured of a marker board to extract first coordinates of the corner points of the markers included in the marker board. Here, the first coordinates are as described above It can correspond to. In addition, the computer device can project the second coordinates of the corner points of the marker onto the marker image. Here, the second coordinates are as described above. It can correspond to. In this case, the computer device can calculate a camera-odometry matrix that minimizes the projection error resulting from the difference between the first coordinates and the second coordinates. Here, the camera-odometry matrix may include information regarding rotation and translation relationships between the robot's coordinate system and the camera's coordinate system.

[0070] In this case, the second coordinates may include the coordinates of the corner points of the marker obtained from an image of the marker board at a different connection position or connection angle than when the first coordinates were extracted. As previously explained, a different connection position or connection angle of the marker board can produce the same effect as the robot moving from the perspective of the image in which the marker board was captured.

[0071] In addition, the second coordinates can be calculated by the matrix product of the first matrix obtained for the camera, the second matrix representing the pose of the robot's coordinate system relative to the camera's coordinate system, the third matrix representing the pose of the marker board's coordinate system relative to the robot's coordinate system, and the fourth matrix obtained for the marker board. Here, the first matrix is ​​K in Equation 6, and the second matrix is ​​of Equation 6 E., the third matrix is ​​of Equation 6. E., the fourth matrix is ​​of mathematical formula 6. Each can correspond to.

[0072] Meanwhile, the second matrix can be calculated based on matrices representing the relative poses of the marker board's coordinate system with respect to the robot's coordinate system, obtained at different joint positions or different joint angles using a jig. Previously (= It has been explained that ) can be obtained using a jig.

[0073] This second matrix may include a camera-odometry matrix, and the computer device can calculate a camera-odometry matrix containing information about rotation relationships and translation relationships between the robot's coordinate system and the camera's coordinate system by calculating a second matrix that minimizes projection errors based on the difference between the first coordinates and the second coordinates while the first matrix, the third matrix, and the fourth matrix are fixed.

[0074] Thus, according to embodiments of the present invention, a camera-odometry calibration method and system using a jig can be provided.

[0075] FIG. 8 is a flowchart illustrating an example of a calibration method using movement information of a robot in an embodiment of the present invention. The calibration method according to the present embodiment may be performed by a computer device that is included in the robot or communicates with the robot. In this case, the processor of the computer device may be implemented to execute a control instruction according to the code of an operating system included in memory or the code of at least one computer program. Here, the processor may control the computer device so that the computer device performs steps (810 to 830) included in the method of FIG. 8 according to the control instruction provided by the code stored in the computer device.

[0076] In step (810), the computer device can analyze a marker image of the marker board taken from a first position of the robot to extract first coordinates of the corner points of the markers included in the marker board. Here, the marker image may include an image of the marker board taken through a camera while the robot is actually moving.

[0077] In step (820), the computer device can project the second coordinates of the corner points of the marker at the second position of the robot onto the marker image. Here, the second coordinates can be calculated by the matrix product of a first matrix obtained for the camera, a second matrix representing the pose of the robot's coordinate system relative to the camera's coordinate system at the first position, a third matrix representing the pose of the robot's coordinate system at the second position relative to the robot's coordinate system at the first position, a fourth matrix representing the pose of the camera's coordinate system relative to the robot's coordinate system at the second position, a fifth matrix representing the pose of the marker board's coordinate system relative to the camera's coordinate system at the second position, and a sixth matrix obtained for the marker board. Here, the first matrix is ​​K of Equation 4, and the second matrix is ​​of Equation 4 E., the third matrix is ​​of Equation 4. E., the fourth matrix is ​​of mathematical formula 4. E., the fifth matrix is ​​of Equation 4. E., the 6th matrix is ​​of Equation 4. Each can correspond to.

[0078] In step (830), the computer device can calculate a camera-odometry matrix that minimizes the projection error resulting from the difference between the first coordinates and the second coordinates. Here, the camera-odometry matrix may include information regarding rotation and translation relationships between the robot's coordinate system and the camera's coordinate system.

[0079] The descriptions omitted in FIG. 8 can be easily understood through the embodiments of FIG. 1 to 7.

[0080] FIG. 9 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention. As shown in FIG. 9, the computer device (900) may include memory (910), a processor (920), a communication interface (930), and an input / output interface (940). The memory (910) is a computer-readable recording medium and may include a non-perishable mass storage device such as RAM (random access memory), ROM (read only memory), and a disk drive. Here, the non-perishable mass storage device such as ROM and the disk drive may be included in the computer device (900) as a separate permanent storage device distinct from the memory (910). Additionally, an operating system and at least one program code may be stored in the memory (910). These software components may be loaded into memory (910) from a computer-readable recording medium separate from memory (910). This separate computer-readable recording medium may include computer-readable recording media such as floppy drives, disks, tapes, DVD / CD-ROM drives, and memory cards. In another embodiment, software components may be loaded into memory (910) via a communication interface (930) rather than a computer-readable recording medium. For example, software components may be loaded into memory (910) of a computer device (900) based on a computer program installed by files received through a network (Network, 960).

[0081] The processor (920) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (920) via memory (910) or a communication interface (930). For example, the processor (920) may be configured to execute instructions received according to program code stored in a recording device such as memory (910).

[0082] The communication interface (930) may provide a function for the computer device (900) to communicate with other devices through a network (960). For example, requests, commands, data, files, etc. generated by the processor (920) of the computer device (900) according to program code stored in a recording device such as memory (910) may be transmitted to other devices through the network (960) under the control of the communication interface (930). Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device (900) through the communication interface (930) of the computer device (900) via the network (960). Signals, commands, data, etc. received through the communication interface (930) may be transmitted to the processor (920) or memory (910), and files, etc. may be stored in a storage medium (the permanent storage device described above) that the computer device (900) may further include.

[0083] The input / output interface (940) may be a means for interfacing with an input / output device (I / O device, 950). For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface (940) may be a means for interfacing with a device in which the functions for input and output are integrated into one, such as a touchscreen. The input / output device (950) may be composed of a computer device (900) and a single device.

[0084] Additionally, in other embodiments, the computer device (900) may include fewer or more components than the components of FIG. 9. However, it is not necessary to clearly illustrate most of the prior art components. For example, the computer device (900) may be implemented to include at least some of the input / output devices (950) described above, or may include other components such as a transceiver, a database, etc.

[0085] Thus, according to embodiments of the present invention, a camera-odometry calibration method and system using a jig can be provided.

[0086] The system or device described above may be implemented as a hardware component, or a combination of a hardware component and a software component. For example, the device and component described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0087] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0088] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Furthermore, the medium may be various recording or storage means in the form of a single or multiple hardware components, and is not limited to a medium directly connected to a computer system, but may also exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Additionally, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0089] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0090] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

Claim 1 A calibration method for a computer device comprising at least one processor, comprising: a step of receiving a plurality of images captured by a camera included in a robot while adjusting at least one of a coupling position and a coupling angle between a marker board on a jig linked to a robot by the at least one processor; a step of collecting a pose relative to the coordinate system of the marker board with respect to the coordinate system of the robot as calibration data by analyzing the plurality of images according to at least one of the coupling position and a coupling angle by the at least one processor; and a step of calibrating the pose of the coordinate system of the camera with respect to the coordinate system of the robot by using the collected calibration data by the at least one processor, wherein the jig includes a plurality of coupling positions arranged in the form of a matrix having a plurality of columns and rows, and the step of receiving the plurality of images is characterized by receiving a plurality of images captured by the marker board at a coupling position of a different column and row among the plurality of coupling positions. Claim 2 delete Claim 3 delete Claim 4 A calibration method according to claim 1, wherein the coupling angle is determined according to the rotation of the marker board with respect to the y-axis of the coordinate system of the marker board, and the step of receiving the plurality of images is characterized by receiving the plurality of images in which the marker board is photographed at the same coupling position at different coupling angles. Claim 5 A calibration method according to claim 1, wherein the calibration step comprises: a step of analyzing a marker image of the marker board taken at a first position of the robot to extract first coordinates of corner points of the marker included in the marker board; a step of projecting second coordinates of the corner points of the marker at a second position of the robot onto the marker image; and a step of calculating a camera-odometry matrix that minimizes a projection error based on the difference between the first coordinates and the second coordinates. Claim 6 A calibration method according to claim 5, characterized in that the camera-odometry matrix includes information regarding rotation and translation relationships between the coordinate system of the robot and the coordinate system of the camera. Claim 7 A calibration method according to claim 5, wherein the second coordinates are calculated by the matrix product of a first matrix obtained for the camera, a second matrix representing the pose of the robot's coordinate system relative to the camera's coordinate system at the first position, a third matrix representing the pose of the robot's coordinate system at the second position relative to the robot's coordinate system at the first position, a fourth matrix representing the pose of the camera's coordinate system relative to the robot's coordinate system at the second position relative to the pose of the marker board's coordinate system at the second position relative to the pose of the marker board's coordinate system at the second position. Claim 8 A calibration method according to claim 7, wherein the third matrix is ​​calculated based on a 3-1 matrix representing the relative pose of the coordinate system of the marker board with respect to the coordinate system of the robot at the first position obtained without movement of the robot using the jig, and a 3-2 matrix representing the relative pose of the coordinate system of the marker board with respect to the coordinate system of the robot at the second position, and wherein the first position and the second position are the same position, but the coupling position or coupling angle of the marker board coupled with the jig is different from each other. Claim 9 A calibration method according to claim 7, wherein the second matrix includes the inverse of the camera-odometry matrix, the fourth matrix includes the camera-odometry matrix, and the step of calculating the camera-odometry matrix is ​​characterized by calculating the fourth matrix which minimizes the projection error according to the difference between the first coordinates and the second coordinates while the first matrix, the third matrix, the fifth matrix, and the sixth matrix are fixed. Claim 10 A calibration method according to claim 1, wherein the calibration step comprises: a step of analyzing a marker image captured of the marker board to extract first coordinates of corner points of the marker included in the marker board; a step of projecting second coordinates of the corner points of the marker onto the marker image; and a step of calculating a camera-odometry matrix that minimizes a projection error based on the difference between the first coordinates and the second coordinates. Claim 11 A calibration method according to claim 10, characterized in that the second coordinates include the coordinates of the corner points of the marker obtained from an image of a marker board having a different combination position or combination angle than when the first coordinates were extracted. Claim 12 A calibration method according to claim 10, wherein the second coordinates are calculated by the matrix product of a first matrix obtained for the camera, a second matrix representing the pose of the robot's coordinate system relative to the camera's coordinate system, a third matrix representing the pose of the marker board's coordinate system relative to the robot's coordinate system, and a fourth matrix obtained for the marker board. Claim 13 A calibration method according to claim 12, wherein the second matrix is ​​calculated based on matrices representing the relative poses of the coordinate system of the marker board to the coordinate system of the robot obtained at different joint positions or different joint angles using the jig. Claim 14 A calibration method according to claim 12, wherein the second matrix includes the camera-odometry matrix, and the step of calculating the camera-odometry matrix is ​​characterized by calculating the second matrix that minimizes the projection error according to the difference between the first coordinates and the second coordinates while the first matrix, the third matrix, and the fourth matrix are fixed. Claim 15 A calibration method for a computer device comprising at least one processor, wherein the at least one processor analyzes a marker image captured by the at least one processor at a first position of a robot to extract first coordinates of corner points of a marker included in the marker board; and the at least one processor projects second coordinates of corner points of the marker at a second position of the robot onto the marker image. A calibration method comprising the step of calculating a camera-odometry matrix that minimizes a projection error based on the difference between the first coordinates and the second coordinates by the at least one processor, wherein the second coordinates are calculated by the matrix product of a first matrix obtained for the camera, a second matrix representing the pose of the robot's coordinate system relative to the camera's coordinate system at the first position, a third matrix representing the pose of the robot's coordinate system at the second position relative to the robot's coordinate system at the first position, a fourth matrix representing the pose of the camera's coordinate system relative to the robot's coordinate system at the second position, a fifth matrix representing the pose of the marker board's coordinate system relative to the camera's coordinate system at the second position, and a sixth matrix obtained for the marker board. Claim 16 A calibration method according to claim 15, characterized in that the camera-odometry matrix includes information regarding rotation and translation relationships between the coordinate system of the robot and the coordinate system of the camera. Claim 17 delete Claim 18 A computer program stored on a computer-readable recording medium to execute the method of any one of claims 1, 4 through 16 on the computer device. Claim 19 A computer device comprising at least one processor implemented to execute a readable command on a computer device, wherein the at least one processor receives a plurality of images captured by a camera included in a robot while adjusting at least one of a coupling position and a coupling angle between a marker board on a jig linked to a robot, and the plurality of images analyzed according to at least one of the coupling position and a coupling angle to collect a pose relative to the coordinate system of the marker board with respect to the coordinate system of the robot as calibration data, and calibrates the pose of the coordinate system of the camera with respect to the coordinate system of the robot using the collected calibration data, wherein the jig includes a plurality of coupling positions arranged in the form of a matrix having a plurality of columns and rows, and wherein, in order to receive the plurality of images, the at least one processor receives a plurality of images captured by the marker board at a coupling position of a different column and row among the plurality of coupling positions. Claim 20 A computer device according to claim 19, characterized in that the marker board captured in the plurality of images is captured in a state where at least one of the coupling position and coupling angle with the jig is different.

Citation Information

Patent Citations

  • Device and method for calibration of three-dimensional scanner, storage medium and processor

    CN108269287A