Position correction device, position correction method, and marker for position correction

Abstract

To provide a position correction device, a position correction method and a marker for position correction capable of expanding a range that can correct a position.SOLUTION: According to an embodiment, a position correction device 1 includes a camera 4, and a control device 3 for correcting a position on the basis of a marker 5. The marker 5 has a plurality of characteristic points 51 arranged in a first arrangement pattern, and the respective characteristic points 51 are composed of a set of fine images 511 arranged in a second arrangement pattern. The control device 3 executes the processing for: detecting the marker 5 from an image caught by the camera 4; correcting the position on the basis of the marker 5 when the marker 5 is detected, detecting any characteristic point 51 unless the marker 5 is detected; changing a visual field so as to make the detected characteristic point 51 to be the center of the visual field; and performing catching again with the camera 4 after changing the visual field.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a position correction device that uses a marker to correct a position, a position correction method, and a marker for position correction. [Background technology]

[0002] For example, a work robot must be in an appropriate positional relationship with respect to a workpiece or component because it performs repetitive movements. For this reason, for example, Patent Document 1 proposes using markers to correct the teaching points of the robot. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-77614 A Summary of the Invention [Problem to be solved by the invention]

[0004] Such a marker for position correction generally has a plurality of characteristic points arranged in a predetermined arrangement pattern, and the position can be corrected based on the positional relationship between the characteristic points.

[0005] However, the conventional method is based on the assumption that the entire marker is present within the field of view. Therefore, even if a feature point of a part of the marker is present within the field of view, the conventional method is unable to recognize the feature point as a feature point, and the range in which the position can be corrected is limited.

[0006] The present disclosure has been made in consideration of the above-mentioned circumstances, and its purpose is to provide a position correction device, a position correction method, and a marker for position correction that can expand the range in which position can be corrected. [Means for solving the problem]

[0007] The position correction device according to the present disclosure is a position correction device that corrects a position using a marker, and includes a camera and a control device that corrects the position based on the marker captured by the camera. The marker has a plurality of feature points arranged in a predetermined first arrangement pattern, and the position can be corrected based on the positional relationship of all the feature points, and is composed of a collection of fine images in which each feature point is arranged in a predetermined second arrangement pattern. The control device then executes a process of detecting the marker from the image captured by the camera, a process of correcting the position based on the marker if the marker is detected, a process of detecting any of the feature points if the marker is not detected, a process of changing the field of view so that the detected feature point is the center of the field of view, and a process of capturing an image again with the camera after changing the field of view. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating a configuration of a position correction device according to an embodiment. [Diagram 2] FIG. 1 is a diagram showing a typical example of a use of a position correction device; [Diagram 3] FIG. 2 is a schematic diagram showing an example of a marker structure. [Figure 4] FIG. 1 is a comparative example illustrating a detection mode of a marker by a conventional method. [Diagram 5] A diagram for explaining the flow of position correction processing [Figure 6] FIG. 1 is an example showing a detection mode of a marker. [Figure 7] FIG. 1 is a diagram showing a schematic example of a structure of a two-dimensional code as a marker; [Figure 8] FIG. 1 is a diagram illustrating a detection mode of a two-dimensional code. [Figure 9] FIG. 13 is a diagram showing another example of an image; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] As shown in Fig. 1, the position correction device 1 according to this embodiment includes a robot 2, a control device 3 that controls the robot 2, and a camera 4. In this embodiment, the robot 2 is a so-called six-axis robot of a vertical articulated type. As is well known, the robot 2 has six rotation axes from the first axis to the sixth axis (J6), and can change its posture by rotating the arms about each rotation axis based on a command from the control device 3 to respond to various tasks. The robot 2 may be a so-called seven-axis robot, or a so-called four-axis robot of a horizontal articulated type.

[0010] In this embodiment, the camera 4 is attached to the robot 2 with a central axis (JC) indicating the center of the field of view oriented in the same direction as a sixth axis (J6) indicating the direction of the hand of the robot 2. Therefore, the relative positional relationship between the camera 4 and the robot 2 is fixed, and therefore if the position of the camera 4 can be correctly corrected as described below, it becomes possible to appropriately control the posture of the robot 2. The camera 4 is connected so as to be able to output the captured image to the control device 3, and the control device 3 corrects the position based on the marker 5 included in the image captured by the camera 4.

[0011] The robot 2 is mounted on a mobile carriage 6 configured as an unmanned transport vehicle. The mobile carriage 6 includes a control unit 7, a driving unit 8, a sensor 9, etc., and is capable of moving to a predetermined position by itself while being guided by a guide member such as a magnetic tape (not shown). That is, in the case of this embodiment, the position correction device 1 is configured as a complex in which the mobile carriage 6, which is a moving body, is combined with the robot 2, which is a movable body. Note that the configuration of the position correction device 1 shown in FIG. 1 is an example, and is not limited thereto. For example, the mobile carriage 6 may be controlled by the control device 3.

[0012] Now, as shown in Fig. 2, for example, the position correction device 1 performs an operation of assembling parts 13 supplied from a supply tray 12 to a workpiece 11 placed on a workbench 10 at the destination work site after moving from a standby position or another work site. Note that the usage form shown in Fig. 2 is an example, and is not limited thereto.

[0013] A marker 5A is provided at a predetermined position on the workbench 10 for correcting the position relative to the workpiece 11. Also, a marker 5B is provided on the supply tray 12 for correcting the position relative to the parts 13 to be supplied. Also, a marker 5C is provided on the side of the workbench 10 facing the movable carriage 6 for correcting the stopping position of the movable carriage 6. Hereinafter, when a common explanation is given, they will be simply referred to as marker 5.

[0014] As shown in Fig. 3, the marker 5 has four feature points 51 in this embodiment. Each feature point 51 is an image with a roughly circular outer edge that is arranged so as to be located at each vertex of a roughly rectangular shape. The arrangement pattern of these four feature points 51 corresponds to a first arrangement pattern. Therefore, if roughly circular images are arranged at each vertex of a rectangle in the image captured by the camera 4, it can be detected as the marker 5.

[0015] In this case, the detection of the marker 5 may be appropriately performed by preparing a detection image for detecting the marker 5 in advance and detecting the marker 5 by so-called image matching with the image captured by the camera 4. The center position (PQ) of the marker 5 may be detected based on the positional relationship of each feature point 51 or by comparing with the detection image. The number and arrangement of the feature points 51 are not limited to this, and a different number and arrangement pattern may be used. In the following description, the upper left in the figure is referred to as feature point 51a, the lower left as feature point 51b, the upper right as feature point 51c, and the lower right as feature point 51d, and the center of these feature points 51 is referred to as the center position (PQ) of the marker 5.

[0016] Each feature point 51 of the marker 5 is composed of a collection of minute images 511 as shown in a partially enlarged view. In this embodiment, the minor crime image 511 means an image smaller than the feature point 51, that is, smaller than the apparent size of the feature point 51 shown by the broken line. In this embodiment, each minute image 511 is a circular image provided at the intersection of a plurality of virtual horizontal lines (LH1 to LHm) extending in the left-right direction and arranged at a predetermined interval (ΔD1) in the up-down direction in the figure, and a plurality of virtual vertical lines (LV1 to LVn) extending in the up-down direction and arranged at a predetermined interval (ΔW1) in the left-right direction in the figure, within the region of the roughly circular feature point 51. The arrangement pattern of the minute images 511 corresponds to the second arrangement pattern.

[0017] Next, the operation and effects of the above-mentioned configuration will be described. First, as shown as a detectable example in Fig. 4, in the conventional method of correcting the position using the conventional marker 105, when all of the conventional feature points 151 constituting the conventional marker 105 are present within the visual field (M1) of the camera 4, it is possible to detect it as the conventional marker 105. In other words, in the conventional method, even if some of the conventional feature points 151 are present within the visual field as shown as an undetectable example, they cannot be recognized as part of the conventional marker 105, and therefore cannot be recognized as the conventional marker 105.

[0018] As a result, as shown in the example of the correctable range, in the conventional method, the range in which the conventional marker 105 can be recognized without changing the field of view many times, that is, the range in which the position can be quickly corrected, is limited by the conventional marker 105 itself. Specifically, in the conventional method, the position correctable range (R1) was a range formed by the outer edges of the range in which the conventional marker 105 is entirely contained in the lower right as in the field of view (M1a), the range in which the conventional marker 105 is entirely contained in the upper right as in the field of view (M1b), the range in which the conventional marker 105 is entirely contained in the lower left as in the field of view (M1c), and the range in which the conventional marker 105 is entirely contained in the upper left as in the field of view (M1d).

[0019] Furthermore, in recent years, an automated guided vehicle or an autonomous mobile robot may be used to automatically transport an item to a predetermined location, or a robot 2 capable of general-purpose operations may be mounted on a cart and made to work in different locations as a mobile robot. In such cases, it is expected that the marker 5 may not be in the field of view or may be captured only very small during movement, or that the marker 5 may shift from the center of the field of view after stopping. For this reason, there is a demand for making it possible to recognize the marker 5 even when only a portion of the marker 5 is in the field of view.

[0020] Therefore, in this embodiment, the feature points 51 of the marker 5 for position correction are configured as a collection of fine images 511 as described above, and the position correction process shown in Fig. 5 is executed to expand the range in which the position can be corrected, that is, the range in which the marker 5 can be recognized. Note that although the process shown in Fig. 5 is executed by the control device 3, the following description will be mainly given with reference to the position correction device 1.

[0021] The position correction device 1 first captures an image with the camera 4 (S1), detects the marker 5 from the captured image (S2), and if the marker 5 is detected (S3: YES), corrects the position based on the marker 5 (S4). In this embodiment, the position correction device 1 changes the posture of the robot 2 and corrects the position of the camera 4 so that the center position (PQ) of the marker 5 is located at the center position (PC) of the field of view of the camera 4. Note that it is also possible to obtain the amount of deviation between the center position (PC) and the center position (PQ), treat the amount of deviation as a correction value in the control coordinate system of the robot 2, and correct the control command value when controlling the robot 2.

[0022] On the other hand, if the position correction device 1 cannot detect the marker 5 (S3: NO), it determines whether or not any of the feature points 51 has been detected (S5). In this case, if the position correction device 1 can detect a fine image 511 arranged in a predetermined arrangement pattern in the captured image, it detects it as a feature point 51. That is, in the case of the position correction device 1, as shown as a detectable example in FIG. 6, if some of the feature points 51 of the marker 5 are located in the field of view (M), it is possible to detect them as feature points 51, that is, to estimate the position of the marker 5 from the feature points 51.

[0023] Next, if the position correction device 1 detects the feature point 51 (S6: YES), it aligns the center position of the field of view to the detected feature point 51 (S7). That is, the position correction device 1 aligns, for example, the feature point 51a to become the center position (PC) of the field of view, as shown in Fig. 6 after alignment. Note that, if multiple feature points 51 are detected, it may be configured to align the feature points 51 so that the midpoint between the feature points 51 becomes the center position (PC) of the field of view.

[0024] Once the center position of the field of view is aligned with the feature point 51, the position correction device 1 proceeds to step S1 and captures an image with the camera 4 (S1). At this time, since the marker 5 is considered to be of a size that fits within the field of view of the camera 4 so that position correction is possible, and since the marker 5 for correcting the position of the robot 2 or the mobile cart 6 is considered not to be large enough to cover the entire field of view of the camera 4 except in special cases, it is considered that the entire marker 5 can be accommodated within the field of view by changing the field of view so that one of the feature points 51 becomes the center position (PC) of the field of view. In other words, it is considered that the marker 5 can be detected by proceeding to step S1 and capturing an image again.

[0025] 6 as an example of the correctable range, the position correctable range (R) is a range formed by the outer edges of a range in which the feature point 51a falls in the lower right as in the visual field (Ma), a range in which the feature point 51b falls in the upper right as in the visual field (Mb), a range in which the feature point 51c falls in the lower left as in the visual field (Mc), and a range in which the feature point 51d falls in the upper left as in the visual field (Md), and is a range that is significantly wider than that of the conventional marker 105. The position correctable range (R) is a range in which the marker 5 can be recognized without changing the visual field many times, and is a range in which the position can be corrected quickly.

[0026] Furthermore, if the position correction device 1 cannot detect the marker 5 (SS3: NO) and cannot detect any of the feature points 51 (S6: NO), it changes the field of view (S8) and then proceeds to step S1 to capture an image with the camera 4. In this case, even if the entire marker 5 is not within the field of view after the field of view is changed, if any of the feature points 51 is within the field of view, the marker 5 can be detected the next time an image is captured, which enables quicker position correction than with conventional methods.

[0027] According to the embodiment described above, the following effects can be obtained. The position correction device 1 corrects the position by using a marker 5, and includes a camera 4 and a control device 3 that corrects the position based on the marker 5 captured by the camera 4. In this case, the marker 5 has a plurality of feature points 51 arranged in a predetermined first arrangement pattern, and the position can be corrected based on the positional relationship of all the feature points 51, and each of the feature points 51 is composed of a collection of fine images 511 arranged in a predetermined second arrangement pattern.

[0028] The control device 3 then executes the following processes: detecting a marker 5 from an image captured by the camera 4; if the marker 5 is detected, correcting the position based on the marker 5; if the marker 5 is not detected, detecting any of the feature points 51; changing the field of view so that the detected feature point 51 becomes the center of the field of view; and capturing an image again with the camera 4 after changing the field of view.

[0029] As a result, if some of the feature points 51 that make up the marker 5 are within the field of view, it can be recognized that they are feature points 51 that make up the marker 5, that is, that they are part of the marker 5, and it becomes possible to grasp the presence of the marker 5. This makes it possible to expand the range in which the marker 5 can be detected without repeatedly changing the field of view, in other words, the position correction range in which quick position correction is possible.

[0030] Moreover, in the position correction device 1, the camera 4 is provided on the robot 2, and the control device 3 corrects the position by controlling the attitude of the robot 2. In this case, as in the embodiment, the position and orientation of the camera 4, i.e., the attitude of the robot 2, can be corrected so that the center position (PC) of the field of view of the camera 4 coincides with the center position (PQ) of the marker 5, or the control command value of the robot 2 can be corrected by obtaining the deviation amount of each center position (PC) and center position (PQ). This makes it possible to correct the starting position when making the robot 2 perform a task, and also to quickly correct the position, thereby shortening the cycle time.

[0031] In addition, the position correction method is a method of correcting the position using a marker 5, and includes the steps of detecting the marker 5 from an image captured by the camera 4, correcting the position based on the marker 5 if the marker 5 is detected, detecting any of the feature points 51 if the marker 5 is not detected, changing the field of view so that the detected feature point 51 becomes the center of the field of view, and capturing an image again with the camera 4 after changing the field of view.

[0032] With this type of position correction method, it is possible to grasp the presence of the marker 5 as long as some of the feature points 51 that make up the marker 5 are within the field of view, and therefore it is possible to obtain various effects similar to those of the position correction device 1, such as expanding the range in which the marker 5 can be detected without repeatedly changing the field of view, in other words, the position correction range in which rapid position correction is possible.

[0033] In addition, the marker 5 is composed of a plurality of feature points 51 arranged in a predetermined first arrangement pattern, and the position can be corrected based on the positional relationship of all the feature points 51, and is composed of a collection of microscopic images 511 in which each feature point 51 is arranged in a predetermined second arrangement pattern.

[0034] By using such a marker 5, it is possible to recognize the marker 5 as long as some of the feature points 51 that make up the marker 5 are within the field of view, and therefore it is possible to obtain various effects similar to those of the position correction device 1, such as expanding the range in which the marker 5 can be detected without repeatedly changing the field of view, in other words, the position correction range in which rapid position correction is possible.

[0035] So far, the configuration in which the camera 4 is provided on the robot 2 and the position of the robot 2, such as its posture, has been exemplified, but the camera 4 can be provided on a moving body such as a moving dolly 6, and the control device 3 can correct the position of the moving body by controlling the moving body. For example, in the example of FIG. 1, the marker 5C can be used to correct the stopping position of the moving body. In this case, the configuration can be such that the control unit 7 on the moving body side executes various processes for correcting the position of the moving body. Note that a configuration in which the robot 2 is placed on a moving body, the camera 4 is provided on the robot 2, and the camera 4 captures an image of the marker 5 while the moving body is moving, as in the embodiment, is also included in the configuration in which the camera 4 is provided on the moving body in a broad sense.

[0036] Even with this configuration, it is possible to grasp the presence of the marker 5 as long as some of the feature points 51 that make up the marker 5 are within the field of view, and therefore it is possible to obtain various effects similar to those of the position correction device 1, such as expanding the range in which the marker 5 can be detected without repeatedly changing the field of view, in other words, the position correction range in which rapid position correction is possible.

[0037] In the embodiment, a configuration in which a dedicated marker 5 for correcting the position is used has been exemplified, but the marker 5 may have other functions. For example, as shown in Fig. 7, a two-dimensional code 20 such as a QR code (registered trademark) in which light or dark areas according to information to be provided are assigned to cells arranged in a two-dimensional matrix can be used as the marker 5. The control device 3 can be configured to detect the two-dimensional code 20 as the marker 5 and detect feature points 51 based on the arrangement pattern of the images constituting the two-dimensional code 20.

[0038] In the case of a QR code, the cell images (CG) in the data area are arranged along virtual horizontal lines (LH11-LV1m) that extend in the left-right direction in the figure and are arranged at a predetermined interval (ΔD11), and virtual vertical lines (LV11-LV1n) that extend in the up-down direction in the figure and are arranged at a predetermined interval (ΔW11). In addition, in a QR code, a finder pattern (FP) is arranged in a predetermined image format, and the finder pattern (FP) is arranged at the three corners of the QR code.

[0039] That is, in a QR code, cell images (CG) and finder patterns (FP) are arranged in a predetermined arrangement pattern. In addition, in the case of a QR code, the positional relationship with the camera 4 can be detected when reading information, so it is easy to detect the center position (PQ) of the QR code. Note that the cell image may be an image that occupies the entire cell, or an image that is smaller than the cell.

[0040] In this case, as shown in, for example, detection example 1 in Figure 8, when a part of the data area of ​​the QR code is within the field of view (M), the position correction device 1 detects that the cell images (CG) are feature points 51 that constitute the QR code based on the arrangement pattern of the cell images (CG) and changes the field of view so that the feature point 51 becomes the center position (PC) of the field of view, thereby making it possible to quickly detect the QR code as a marker 5 and read its information, and also to correct the center position (PC) of the field of view to the center position (PQ) of the QR code.

[0041] Alternatively, as shown in detection example 2 in Figure 8, when a part of the data area of ​​the QR code or the finder pattern (FP) is within the field of view (M), the position correction device 1 detects that they are feature points 51 that constitute the QR code based on the arrangement pattern of the finder pattern (FP), the arrangement pattern of the cell image (CG), or the arrangement patterns of both, and changes the field of view so that the feature point 51 becomes the center position (PC) of the field of view, making it possible to quickly detect the QR code as a marker 5 and read its information, and also to correct the center position (PC) of the field of view to the center position (PQ) of the QR code.

[0042] With this configuration, it is possible to obtain various effects similar to those of the position correction device 1, such as expanding the range in which the marker 5 can be detected without repeatedly changing the field of view, that is, the range in which the position can be corrected quickly.

[0043] In the embodiment, as shown in Fig. 3, the feature points 51 are generally circular, the fine images 511 are also circular, and the collection of the fine images 511 is arranged so that the outer edge of the feature points 51 forms a circle, but the shapes of the feature points 51 and the fine images 511 are not limited to this. For example, as shown as another image example 1 in Fig. 9, other shapes such as a square, an ellipse, a cross, or a combination of these shapes may be used as long as they are identifiable as the feature points 51 and the fine images 511.

[0044] 9 as another image example 2, a configuration in which the black and white of the feature point 51 shown in FIG. 3 are inverted, that is, a circular image with a clear outer edge as the feature point 51 is arranged, and a white circular fine image 511, for example, is arranged inside it. In other words, the feature point 51 can be arranged in a configuration in which fine images 511 of the same size as the feature point 51 and fine images 511 smaller than the feature point 51 are mixed.

[0045] Also, as shown in another image example 3, the feature point 51 may be a circular image, and the image may be divided, for example, in the circumferential direction to form a plurality of minute images 511, which may be arranged with gaps between them. In this case, the images divided in the circumferential direction may be regarded as minute images 511, but the line segments filling the image with white may also be regarded as minute images 511.

[0046] Also, as shown in another image example 4, a configuration in which a plurality of fine images 511 of different sizes are mixed, for example, circular fine images 511 of different sizes are arranged concentrically, can be used. Alternatively, as shown in another image example 5, a configuration in which fine images 511 of different sizes and shapes are mixed, for example, a circular fine image 511 and a rectangular fine image 511 are arranged, can be used. Furthermore, as shown in another image example 5, a configuration in which fine images 511 of different sizes and shapes and also different arrangement patterns are mixed can be used. In other words, a configuration in which a plurality of second arrangement patterns in which fine images 511 are arranged can be provided for a marker 5 composed of feature points 51 arranged in one first arrangement pattern.

[0047] In the embodiment, the first arrangement pattern and the second arrangement pattern are different in the arrangement of images, but the arrangement of images may be similar. For example, in the first arrangement pattern in which the feature points 51 are arranged at the four vertices of a rectangle as shown in Fig. 3, the minute images 511 can be arranged in the second arrangement pattern in which they are arranged at the four vertices of a rectangle.

[0048] Although the present disclosure has been described based on the embodiment, it is understood that the present disclosure is not limited to the embodiment or structure. The present disclosure also includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, and other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and concept of the present disclosure. [Explanation of symbols]

[0049] In the drawing, 1 indicates a position correction device, 2 indicates a robot, 3 indicates a control device, 4 indicates a camera, 5 indicates a marker, 6 indicates a moving cart (moving body), 7 indicates a control unit, 20 indicates a two-dimensional code, 51 indicates a feature point, and 511 indicates a fine image.

Claims

1. A position correction device that corrects a position by using a marker, Camera and and a control device that corrects the position of the marker based on the image of the marker captured by the camera, the marker has a plurality of feature points arranged in a predetermined first arrangement pattern, the position of which can be corrected based on the positional relationship of all of the feature points, and is composed of a collection of fine images in which each of the feature points is arranged in a predetermined second arrangement pattern; The control device includes: A process of detecting the marker from an image captured by the camera; If the marker is detected, a process of correcting the position based on the marker; If the marker is not detected, a process of detecting any feature point; A process of changing the field of view so that the detected feature point is at the center of the field of view; A process of capturing an image again with the camera after changing the field of view; A position correction device that performs the above.

2. The camera is provided on a robot, 2. The position correction device according to claim 1, wherein the control device corrects the position by controlling the attitude of the robot.

3. The camera is provided on a moving object, 2. The position correction device according to claim 1, wherein the control device corrects the position by controlling the moving body.

4. The marker is a two-dimensional code in which light or dark areas are assigned to cells arranged in a two-dimensional matrix according to information to be provided, 2. The position correction device according to claim 1, wherein the control device detects the two-dimensional code as the marker, and detects feature points based on an arrangement pattern of images constituting the two-dimensional code.

5. A position correction method for correcting a position by using a marker, comprising the steps of: the marker has a plurality of feature points arranged in a predetermined first arrangement pattern, the position of which can be corrected based on the positional relationship of all of the feature points, and is composed of a collection of fine images in which each of the feature points is arranged in a predetermined second arrangement pattern; detecting the marker from an image captured by the camera; If the marker is detected, correcting the position based on the marker; If the marker is not detected, detecting any feature point; modifying the field of view so that the detected feature point is centered in the field of view; A process and step of capturing an image again with the camera after changing the field of view; A position correction method including:

6. A position correction marker that is composed of a plurality of feature points arranged in a predetermined first arrangement pattern, and whose position can be corrected based on the positional relationships of all of the feature points, and that is composed of a collection of fine images in which each of the feature points is arranged in a predetermined second arrangement pattern.

Images