Position detection system, screw fastening driver drive control system using the same, and position detection method

Abstract

To provide a position detection system for detecting the position of an object by imaging a marker arranged on the object with a camera, capable of detecting the position of the object at a high speed and with high accuracy.SOLUTION: A position detection system for detecting the position of an object from an image picked up by an imaging device includes an imaging device for imaging an object, and a position detection marker 2 disposed at the object. The position detection marker includes a color frame 8 of a specific color, and a two-dimensional pattern 9 disposed in an area surrounded with the color frame 8, detects an area in which the position detection marker 2 is located by detecting the color frame 8 included within an imaging range of the imaging device, and detects a center position in the area on the basis of the two-dimensional pattern 9.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a position detection system and a position detection method for detecting the position of an object, and in particular to a technique that is effective when applied to an electric motor-driven screw driver. [Background technology]

[0002] In manufacturing sites, screw tightening drivers driven by electric motors are widely used, and by electrifying the screw tightening work that was previously performed manually, the burden on workers is reduced and productivity and quality are improved.

[0003] In a conventional screw tightening process using an electric motor-driven screw driver, the workpiece is photographed with a camera and displayed on a monitor, and the operator tightens the screws by following instructions on the screen.

[0004] Background art in this technical field includes, for example, technology such as Patent Document 1. Patent Document 1 discloses a method for automatically processing not only the position of the center of a marker but also how the center of the marker forms the object area from the identification number by giving the marker an identification number based on a combination of colors when the marker is extracted from an image of the object.

[0005] Furthermore, Patent Document 2 discloses "an information identification marker to be attached to a managed object, the information identification marker comprising colored marks arranged at each of the four corners of a rectangle on a flat, monochromatic background, and an icon in a display area connecting the outermost sides of the marks at the four corners, in which data identifying the managed object is displayed as human-understandable characters, designs, or a combination of these." [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2005-309717 A [Patent Document 2] JP 2002-56371 A Summary of the Invention [Problem to be solved by the invention]

[0007] As described above, in a conventional screw tightening process using an electric motor-driven screw driver, an operator tightens the screws by following instructions on a screen.

[0008] However, since there are no limitations on the drive of the screwdriver and the screwdriver can be driven in a position other than its original position, there is a concern that defective products may be produced due to incorrect assembly.

[0009] The techniques of Patent Documents 1 and 2 above leave room for improvement in terms of detection accuracy of the target center position.

[0010] Therefore, an object of the present invention is to provide a position detection system that detects the position of an object by capturing an image of a marker placed on the object with a camera, capable of detecting the position of an object quickly and with high accuracy, and a drive control system and position detection method for a screwdriver using the same. [Means for solving the problem]

[0011] In order to solve the above problems, the present invention provides a position detection system that detects the position of an object from an image captured by an imaging device, comprising: an imaging device that captures an image of the object; and a position detection marker that is placed on the object, wherein the position detection marker has a color frame of a specific color and a two-dimensional pattern that is placed in an area surrounded by the color frame, and detects the area in which the position detection marker is located by detecting the color frame that is included in the imaging range of the imaging device, and detects the center position of the area based on the two-dimensional pattern.

[0012] The present invention is also characterized by a drive control system for a screwdriver using the above-mentioned position detection system.

[0013] The present invention also provides a position detection method for detecting a position of an object from an image captured by an imaging device, comprising the steps of: (a) capturing an image of an object on which a position detection marker is disposed by an imaging device; (b) detecting an area in which the position detection marker is located by detecting a color frame of the position detection marker included in an imaging range of the imaging device from the image captured in step (a); and (c) detecting a center position of the area detected in step (b) based on a two-dimensional pattern of the position detection marker. Effect of the Invention

[0014] According to the present invention, in a position detection system that detects the position of an object by capturing an image of a marker placed on the object with a camera, it is possible to realize a position detection system capable of detecting the position of an object quickly and with high accuracy, as well as a drive control system and a position detection method for a screwdriver using the same.

[0015] This improves the reliability of the screw tightening operation.

[0016] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram showing a schematic configuration of a position detection system according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram showing the appearance of a marker according to a first embodiment of the present invention. [Diagram 3] FIG. 13 is a diagram showing a modified example of the marker. [Figure 4] FIG. 13 is a diagram showing a modified example of a two-dimensional code constituting a marker. [Diagram 5] 2 is a diagram showing how a single object to be detected 5 is detected by the camera 3 in FIG. 1. FIG. [Figure 6]2 is a diagram showing how a plurality of detection target objects 5 are detected by the camera 3 in FIG. 1. FIG. [Figure 7] 1A and 1B are diagrams showing how a single object 5 to be detected is detected by stereo vision. [Figure 8] FIG. 1 is a diagram showing a three-dimensional position detection method using a single camera. [Figure 9] FIG. 13 is a diagram showing a comparison of detection accuracy between a color frame and a two-dimensional code. [Figure 10] FIG. 13 is a diagram showing a blurred state of a two-dimensional code. [Figure 11] FIG. 13 is a diagram showing a blurred state of a color frame. [Figure 12] 4 is a flowchart showing a position detection method according to the first embodiment of the present invention. [Figure 13] FIG. 13 is a diagram showing a detection range using a color frame and a two-dimensional code. [Figure 14] 10 is a flowchart showing a process performed by a drive control system for a screwdriver according to a second embodiment of the present invention. [Figure 15] FIG. 11 is a diagram showing a schematic configuration of a drive control system for a screwdriver according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and detailed description of overlapping parts will be omitted. EXAMPLES

[0019] First Embodiment A position detection system and a position detection method according to a first embodiment of the present invention will be described with reference to FIGS.

[0020] FIG. 1 is a diagram showing a schematic configuration of a position detection system 1 according to the present embodiment.

[0021] As shown in FIG. 1, the position detection system 1 of this embodiment mainly comprises a position detection marker 2 placed on a detectable object 5, a camera 3 for capturing an image of the detectable object 5 and detecting its position, and a processing device 4 for transmitting and receiving signals between the camera 3 and the detectable object 5 and processing these signals.

[0022] The object to be detected 5 may be, for example, a driver driven by an electric motor in a screw tightening device, a welding torch in a welding device, or any other device.

[0023] As described below, the marker 2 of the present invention is composed of a color frame for quickly detecting the marker detection range 7 within the camera field of view 6, and a two-dimensional code (also called a two-dimensional pattern) for detecting the center position within the color frame with high accuracy.

[0024] In FIG. 1, DFOV indicates the viewing angle (display field). local ;V>V local ;H local >H obj It is assumed that.

[0025] FIG. 2 is a diagram showing the appearance of the marker 2 of this embodiment.

[0026] As shown in FIG. 2, the marker 2 is composed of a color frame 8 and a two-dimensional code (two-dimensional pattern) 9. The color frame 8 has a specific color including a monochrome (black and white) pattern as shown in FIG. 2. The color of the two-dimensional code 9 may be the same as that of the color frame 8, or may be a different color. What is important here is that a gap 10 is provided between the color frame 8 and the two-dimensional code 9 so that the color frame 8 and the two-dimensional code 9 can be clearly distinguished from each other. In order to clearly distinguish the boundary between the color frame 8 and the two-dimensional code 9, the color of the gap 10 needs to be different from that of the color frame 8 and the two-dimensional code 9. Note that if the color of the color frame 8 and the color of the two-dimensional code 9 are different and the color frame 8 and the two-dimensional code 9 can be clearly distinguished from each other, it is not necessarily necessary to provide the gap 10.

[0027] FIG. 3 is a diagram showing a modified example of the marker 2. In FIG.

[0028] As shown in markers 2a to 2c in Fig. 3, the shape of color frame 8 is not limited, and two-dimensional code 9 can be applied by arranging it inside the outer frame. In addition to a rectangle (square) as shown in Fig. 2, color frame 8 may be a circle like marker 2a, a triangle (polygon) like marker 2b, or a special shape with projections and recesses like marker 2c.

[0029] FIG. 4 is a diagram showing a modified example of the two-dimensional code 9 constituting the marker 2. In FIG.

[0030] In addition to the pattern shown in FIG. 2, the two-dimensional code 9 can be replaced with other two-dimensional codes, such as a Data Matrix like two-dimensional code 9a, a QR Code (registered trademark) like two-dimensional code 9b, or AruCo like two-dimensional code 9c.

[0031] FIG. 5 is a diagram showing how a single object 5 to be detected is detected by the camera 3 in FIG.

[0032] The detection configuration of a single object to be detected 5 by one camera 3 of the position detection system 1 will be described with reference to FIG.

[0033] The position detection system 1 in Fig. 5 is composed of a marker 2 for position detection placed on one object 5 to be detected, and one camera 3 for detecting the position of the object 5 to be detected. The camera 3 is fixed to the top of the system.

[0034] The marker 2 is attached to a detectable object 5, which has a movable range within the field of view of the camera 3. The camera 3 detects the position of the detectable object 5 that exists within the camera field of view 6. First, the camera 3 detects a color frame 8 from the marker 2, thereby quickly identifying the marker detection range 7 of the detectable object 5. Next, the camera 3 detects a two-dimensional code 9 within the detected color frame 8, and identifies its center position with high accuracy.

[0035] With this mechanism, the position detection system 1 in FIG. 5 can detect the position of a single object to be detected 5 efficiently and with high accuracy using one camera 3.

[0036] FIG. 6 is a diagram showing how a plurality of detection target objects 5 are detected by the camera 3 in FIG.

[0037] A detection configuration for a plurality of detection target objects 5 using one camera 3 of the position detection system 1 will be described with reference to FIG.

[0038] 6 is composed of position detection markers 2d, 2e arranged on multiple (two in this example) objects 5 to be detected, and one camera 3 for detecting the positions of the multiple objects 5 to be detected. The camera 3 is fixed to the top of the system.

[0039] The markers 2d and 2e are attached to a plurality of detection target objects 5, respectively, and the plurality of detection target objects 5 have a movable range within the field of view of the camera 3. The markers 2d and 2e are used in combination with a different color frame 8 and a two-dimensional code 9 for each detection target object 5.

[0040] The camera 3 detects the position of each of the multiple detectable objects 5 within the range of the camera's field of view 6 by parallel processing. First, the camera 3 detects the color frames 8 of the markers 2d and 2e, and quickly identifies the detection ranges of the markers 2d and 2e using that information. Next, the camera 3 detects the two-dimensional code 9 within the color frames 8 and identifies its center position with high accuracy. This allows the position of each detectable object 5 to be detected.

[0041] With the above-described mechanism, the position detection system 1 in FIG. 6 can detect the positions of a plurality of target objects 5 simultaneously, efficiently, and with high accuracy, using one camera 3.

[0042] FIG. 7 is a diagram showing how a single object 5 to be detected is detected by stereo vision.

[0043] Stereo detection (stereo vision) using two cameras 3a and 3b shown in Fig. 7 uses the same principle as humans using their two eyes to estimate the distance to an object. Below, we will explain the basic configuration of stereo detection using two cameras 3a and 3b.

[0044] The two cameras, camera 3a and camera 3b, simultaneously capture the same scene from different angles. Like the human eye, the two cameras 3a, 3b see the scene from different perspectives, which allows the three-dimensional position of the detected object 5 to be calculated.

[0045] The images from the two cameras 3a, 3b are compared in a signal processor 4 (see Figure 1) through a process called stereo matching, which finds pixels that correspond to the same object in each image. This matching makes it possible to calculate disparity (the relative positional shift of the same object seen by the two cameras 3a, 3b) to estimate the distance of the object.

[0046] The calculated disparity is then used to estimate the distance to the object using the principles of triangulation, which is the basis for generating a depth map of the object and thus obtaining the three-dimensional (3D) position of the detected object 5.

[0047] As described above, each component works in conjunction to make it possible to estimate the three-dimensional position of an object.

[0048] FIG. 8 is a diagram showing a three-dimensional position detection method using a single camera 3. In FIG.

[0049] A three-dimensional position detection method using a single camera 3 will be described with reference to Figure 8. The position detection system 1 in Figure 8 detects the three-dimensional position of an object using a single camera 3. A marker 2 is attached to a detectable object 5, and as the detectable object 5 approaches the camera 3 (Z0), its area becomes larger, and as the detectable object 5 moves away from the camera 3 (Z1), its area becomes smaller.

[0050] Camera 3 detects the area of ​​two-dimensional code 9 of object 5 to be detected that is present within the range of camera field of view 6. By utilizing the characteristic that the area of ​​two-dimensional code 9 becomes larger when object 5 to be detected approaches camera 3 (Z0), and the area of ​​two-dimensional code 9 becomes smaller when object 5 to be detected moves away from camera 3 (Z1), it is possible to approximate the Z direction (depth) of object 5 to be detected.

[0051] As described above, this system can obtain position information not only in the XY directions but also in the Z direction (depth), making it possible to detect the three-dimensional position of the detection target object 5 using a single camera 3.

[0052] As described above, this system can use a single camera 3 to efficiently and accurately detect the position of an object in three-dimensional space by utilizing changes in the area of ​​the two-dimensional code 9 on the object to be detected 5.

[0053] FIG. 9 is a diagram showing a comparison of detection accuracy between a color frame and a two-dimensional code.

[0054] In position detection using the color frame 8, the accuracy of center detection may be reduced because noise may exist within the color frame extracted by the detection algorithm. However, the advantage of the method using the color frame 8 is that it is possible to detect the object even if the image is blurred.

[0055] On the other hand, when a two-dimensional code 9 is used, the accuracy of detecting the center is very high. However, the process of detecting the two-dimensional code 9 is heavier and takes more time than the process of detecting the color frame 8. Also, the two-dimensional code 9 is sensitive to blurring, and becomes difficult to detect when the image is blurred.

[0056] Fig. 10 is a diagram showing a blurred state of a two-dimensional code. The left diagram in Fig. 10 shows a state without blurring, and the right diagram shows a state with blurring.

[0057] As shown in the right diagram of FIG. 10, when the image of the two-dimensional code 9 is blurred, it becomes difficult to recognize and detect the features and edges of the two-dimensional code 9.

[0058] Fig. 11 is a diagram showing the blurred state of the color frame, where the left diagram in Fig. 11 shows a state without blurring, and the right diagram shows a state with blurring.

[0059] As shown in the right diagram of FIG. 11, even if the image of the color frame 8 is blurred, the color characteristics remain and can be detected without any problems.

[0060] It is important to select the detection method using the color frame 8 or the two-dimensional code 9 based on this information and depending on the application situation of the position detection system 1 and the state of the object 5 to be detected.

[0061] 12 is a flowchart showing the position detection method of this embodiment, which illustrates a detection process flow for a single object 5 to be detected by one camera 3 of the position detection system 1.

[0062] This process flow shows a process flow for analyzing the color and shape of an input image acquired from the camera 3 and detecting the center of the detection target object 5 from a specified region.

[0063] First, based on the image input in step S1, the lens distortion of the camera 3 is corrected in step S2.

[0064] Next, in step S3, the corrected image is converted into HSV color space to make the color information clearer.

[0065] Next, in step S4, a specific HSV region is detected as a color frame through user-set parameters.

[0066] Next, in step S5, it is determined whether or not a color frame is detected. If a color frame is detected (Yes), the process proceeds to step S6. On the other hand, if a color frame is not detected (No), the process returns to step S2 ((1) in FIG. 12) and repeats the processes from step S2 onward.

[0067] Once the color frame has been detected, in step S6, a noise filter is used to remove unnecessary information from edge noise of the color frame.

[0068] Then, in step S7, a shape analysis of the color frame is performed, and in step S8, a specific shape of the color frame is extracted based on a user specification. When shape detection is completed (Yes), in step S9, the color frame with the maximum area is extracted.

[0069] Next, in step S10, the center coordinates of the selected color frame (color blob) are detected, and based on this, in step S11, the coordinates of a local ROI (region of interest) are calculated from the center coordinates and the user-specified offset pixel number. In step S12, the local ROI is extracted using the calculated coordinates.

[0070] Next, in step S13, the image intensity of the extracted local ROI is normalized, and then in step S14, a marker (two-dimensional code) is detected from that region.

[0071] Next, in step S15, it is determined whether or not a two-dimensional code is present. If it is confirmed that a two-dimensional code is present (Yes), the center coordinates of the marker (two-dimensional code) are output in step S16. On the other hand, if a two-dimensional code is not detected (No), the center coordinates of the color frame are obtained and output as coordinates in step S17. This series of operations is performed as a loop.

[0072] FIG. 13 is a diagram showing the detection range using a color frame and a two-dimensional code.

[0073] The detection range when using a color frame and a two-dimensional code in combination will be described with reference to FIG.

[0074] Without a color frame, the 2D code 9 needs to be detected from the entire camera field of view. This requires processing a large amount of data, resulting in a very large time and processing load. This can be difficult to handle when real-time processing (video) is required, and can require expensive processing equipment.

[0075] On the other hand, when the color frame 8 and the two-dimensional code 9 are used together as in the present invention, the detection range is narrowed by the color frame 8, and the overall detection process is made lighter. This is a step in which the initial position is first specified by the color frame 8, and then the two-dimensional code 9 is detected only within that color frame.

[0076] This eliminates the need to scan the entire camera field of view, reducing computation time and load.

[0077] Therefore, the combined use of the color frame 8 and the two-dimensional code 9 enables efficient real-time processing and also leads to cost reduction. EXAMPLES

[0078] Second Embodiment A drive control system and a control method for a screwdriver according to a second embodiment of the present invention will be described with reference to FIGS.

[0079] In this embodiment, a use case will be described in which the position detection system and the position detection method of the first embodiment are applied to a screwdriver driven by an electric motor.

[0080] FIG. 14 is a flow chart showing the process performed by the drive control system of the screwdriver of this embodiment.

[0081] First, in step S1, a screw driver (detection target object 5) with a marker 2 attached thereto is detected from an image input from the camera 3. Through this detection, the position and center coordinates of the screw driver are identified.

[0082] Next, in step S2, it is confirmed whether the detected position (output coordinate) of the screw tightening driver is within a specific range. This is performed, for example, to confirm whether the driver is present within the screw tightening position or whether the driver is tightening the correct position.

[0083] If it is determined in step S2 that the output coordinates are within a specific range (Yes), the interlock (safety device) of the screw driver is released in step S3, which allows the screw driver to be driven.

[0084] FIG. 15 is a diagram showing a schematic configuration of a drive control system for the screw driver of this embodiment.

[0085] A use case when using the color frame 8 and the two-dimensional code 9 in combination will be described with reference to Figure 15. Figure 15 shows a drive control system for a screw driver linked to position detection.

[0086] In detecting the position of the screwdriver 11, a sticker (marker) combining a color frame 8 and a two-dimensional code 9 is attached to the screwdriver 11 and detected by the camera 3, enabling high-speed and high-precision detection.

[0087] In this way, by linking the position detection system 1 of the present invention with the drive control of the screw driver 11, it is possible to solve the problems of tightening the wrong type of screw into the work object 12 and of making mistakes in the work procedure.

[0088] The present invention is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0089] 1. Position detection system 2,2a,2b,2c,2d,2e...Marker 3, 3a, 3b…Camera 4...(Signal) processing device 5...Detection object 6,6a,6b…Camera field of view 7…Marker detection range 8…Color frame 9, 9a, 9b, 9c...2D code (2D pattern) 10…Gap (between the color frame and the 2D code) 11...Screwdriver 12...Work object.

Claims

1. A position detection system for detecting a position of an object from an image captured by an imaging device, An imaging device that captures an image of an object; A position detection marker disposed on the object, the position detection marker has a color frame of a specific color and a two-dimensional pattern arranged in an area surrounded by the color frame, A position detection system that detects the color frame included in an imaging range of the imaging device to detect an area in which the position detection marker is located, and detects a center position of the area based on the two-dimensional pattern.

2. 2. The position detection system according to claim 1, The position detection system, wherein the position detection marker has a gap between the color frame and the two-dimensional pattern.

3. 3. The position detection system according to claim 2, The gap is a different color than the color frame and the two-dimensional pattern.

4. 2. The position detection system according to claim 1, A position detection system in which the color frame is any one of a rectangle, a circle, a polygon including a triangle, and a shape having projections and recesses.

5. 2. The position detection system according to claim 1, The two-dimensional pattern is any one of a black and white pattern, a data matrix, a QR code (registered trademark), and an AruCo code.

6. 2. The position detection system according to claim 1, A plurality of the imaging devices are provided, A position detection system that detects a three-dimensional position of the object based on parallax between the multiple imaging devices.

7. 2. The position detection system according to claim 1, The imaging device includes: A position detection system that detects a three-dimensional position of an object based on a plurality of images captured by changing the distance between the one imaging device and the object.

8. 2. The position detection system according to claim 1, The position detection system includes a position detection marker having a movable range within an imaging range of the imaging device.

9. A drive control system for a screwdriver, which uses the position detection system according to any one of claims 1 to 8.

10. A position detection method for detecting a position of an object from an image captured by an imaging device, comprising: (a) capturing an image of an object on which a position detection marker is disposed by an imaging device; (b) detecting a color frame of the position detection marker included in an imaging range of the imaging device from the image captured in the (a) step, thereby detecting an area in which the position detection marker is located; (c) detecting a center position of the area detected in the step (b) based on the two-dimensional pattern of the position detection marker; A position detection method comprising:

Images