Detection method and computer

Abstract

This detection method comprises: a step for aligning a prescribed position A of an object (10) with a reference point O of an alignment member (7) on which a positioning marker (71) is disposed; a step for imaging the alignment marker (71) and a first marker (11); a step for acquiring, on the basis of the captured image, information on the relative three-dimensional coordinates of the prescribed position A of the object (10) with respect to the position of the first marker (11) attached to the object (10); and a step for imaging a second marker (14) attached to a member (13) different from the object (10) and the first marker (11) on the object (10), and thereby detecting the relative three-dimensional coordinates of the prescribed position A of the object (10) with respect to the other member (13), from the captured image that includes the first marker (11) and the second marker (14), on the basis of the acquired three-dimensional coordinates.

Description

Detection method and computer

[0001] The present invention relates to a sensing method and a computer.

[0002] Conventionally, a technique has been known in which a marker is attached to an object, the marker is captured by a camera, and the position and orientation of the object are measured from the image of the marker. Markers that can be attached to objects are disclosed, for example, in Japanese Patent Application Publication No. 2012-145559 (JP2012-145559A). A typical marker is a planar pattern that can be attached to an object. A typical marker includes a square black frame and a two-dimensional pattern code printed within it. Then, by capturing an image of the marker with a camera, the relative position and orientation of the marker and the camera can be recognized, and the position and orientation of the object to which the marker is attached can be recognized. Furthermore, the camera can also read information recorded in the marker.

[0003] There is a demand for technology that can detect the three-dimensional coordinates of a specific position on an object to which a marker is attached, which is difficult to image using a camera, and the three-dimensional direction vector of a specific direction related to this specific position, from an image of the marker, but such technology has not been known until now.

[0004] The present disclosure has been made in consideration of these points, and aims to provide a new detection method and computer for detecting, from an image of a marker, the three-dimensional coordinates of a predetermined position on an object to which a marker is attached, and a three-dimensional direction vector in a predetermined direction related to this predetermined position.

[0005] The detection method disclosed herein is a detection method for detecting a predetermined position of an object having a first marker attached thereto, and includes the steps of: aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is arranged; capturing an image of the alignment marker and the first marker arranged on the alignment member; acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object based on the captured image including the alignment marker and the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting the three-dimensional coordinates of the predetermined position of the object relative to the separate member from the captured image including the first marker and the second marker based on the acquired three-dimensional coordinates.

[0006] A detection method disclosed herein is a detection method for detecting a direction of an object to which a first marker is attached toward a predetermined position, the detection method comprising: a step of aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is arranged; a step of capturing an image of the alignment marker and the first marker arranged on the alignment member; a step of acquiring information on a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object based on the captured image including the alignment marker and the first marker; and a step of capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting a relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the separate member from the captured image including the first marker and the second marker based on the acquired three-dimensional direction vector.

[0007] A detection method disclosed herein is a detection method for detecting a predetermined position of an object to which a first marker is attached and a direction to the predetermined position, comprising: a step of aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is arranged; a step of capturing an image of the alignment marker and the first marker arranged on the alignment member; a step of acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object and information on a three-dimensional direction vector of the object relative to the first marker, based on the captured image including the alignment marker and the first marker; and a step of capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting, from the captured image including the first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position relative to the separate member, based on the acquired three-dimensional coordinates and three-dimensional direction vector.

[0008] In the detection method of the present disclosure, the alignment member may have a flat surface, and a predetermined reference point may be formed on the flat surface, and the alignment marker may be arranged on the flat surface.

[0009] The detection method disclosed herein is a detection method for detecting a predetermined position of an object to which a first marker is attached, and includes the steps of: placing the alignment marker at a predetermined reference point of an alignment member, and capturing an image of the alignment marker with a fixed-position imaging device; aligning the predetermined position of the object with the predetermined reference point of the alignment member, and capturing an image of the first marker with the imaging device; acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object, based on the captured image including the alignment marker and the captured image including the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting the three-dimensional coordinates of the predetermined position of the object relative to the separate member from the captured image including the first marker and the second marker, based on the acquired three-dimensional coordinates.

[0010] A detection method disclosed herein is a detection method for detecting a direction of an object to which a first marker is attached toward a predetermined position, comprising the steps of: placing an alignment marker at a predetermined reference point of an alignment member, and capturing an image of the alignment marker with a fixed-position imaging device; aligning the predetermined position of the object with the predetermined reference point of the alignment member, and capturing an image of the first marker with the imaging device; acquiring information about a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object, based on the captured image including the alignment marker and the captured image including the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting a relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the separate member from the captured image including the first marker and the second marker, based on the acquired three-dimensional direction vector.

[0011] A detection method according to the present disclosure is a detection method for detecting a predetermined position of an object to which a first marker is attached and a direction to the predetermined position, the detection method comprising the steps of: placing an alignment marker at a predetermined reference point of an alignment member, and capturing an image of the alignment marker with an imaging device that is fixed in position; aligning the predetermined position of the object with the predetermined reference point of the alignment member, and capturing an image of the first marker with the imaging device; and acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object and information on a three-dimensional direction vector of the predetermined position of the object relative to the first marker, based on the captured image including the alignment marker and the captured image including the first marker. and a step of capturing an image of the first marker of the object and a second marker attached to a member other than the object, and detecting, based on the acquired three-dimensional coordinates and three-dimensional direction vector, from an image including the captured first marker and second marker, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the other member.

[0012] In the detection method disclosed herein, a recess may be formed at the reference point of the alignment member, and the predetermined position of the object may be aligned with the reference point by inserting the predetermined position portion of the object into the recess.

[0013] The detection method disclosed herein includes the steps of: placing an alignment marker at a predetermined position of an object to which a first marker is attached, and capturing an image of the first marker and the alignment marker; acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object, based on the captured image including the alignment marker and the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting the three-dimensional coordinates of the predetermined position of the object relative to the separate member from the captured image including the first marker and the second marker, based on the acquired three-dimensional coordinates.

[0014] The detection method disclosed herein includes the steps of: placing an alignment marker at a predetermined position of an object to which a first marker is attached, and capturing an image of the first marker and the alignment marker; acquiring information about a three-dimensional direction vector in a predetermined direction extending from the predetermined position perpendicular to the first marker based on the captured image including the alignment marker and the first marker; and capturing an image of a second marker attached to a member other than the object and the first marker of the object, and detecting a relative three-dimensional direction vector of the predetermined direction with respect to the other member from the captured image including the first marker and the second marker based on the acquired three-dimensional direction vector.

[0015] The detection method disclosed herein includes the steps of: placing an alignment marker at a predetermined position of an object to which a first marker is attached, and capturing an image of the first marker and the alignment marker; acquiring, based on the captured image including the alignment marker and the first marker, information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object, and information on a three-dimensional direction vector in a predetermined direction extending from the predetermined position so as to be perpendicular to the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, detecting, from the captured image including the first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member and the relative three-dimensional direction vector in the predetermined direction with respect to the separate member, based on the acquired three-dimensional coordinates and three-dimensional direction vector.

[0016] The detection method of the present disclosure includes the steps of: placing a first alignment marker at a first predetermined position on a first object to which a first marker is attached, and capturing an image of the first marker and the first alignment marker; acquiring, based on the captured image including the first alignment marker and the first marker, information on three-dimensional coordinates of the first predetermined position on the first object relative to the position of the first marker attached to the first object, and information on a three-dimensional direction vector in a first predetermined direction extending from the first predetermined position so as to be perpendicular to the first alignment marker; placing a second alignment marker at a second predetermined position on a second object to which a second marker is attached, and capturing an image of the second marker and the second alignment marker; The method comprises: a step of acquiring, based on an image including the captured second alignment marker and the second marker, information on three-dimensional coordinates of the second predetermined position of the second object relative to the position of the second marker attached to the second object, and information on a three-dimensional direction vector of a second predetermined direction extending from the second predetermined position so as to be perpendicular to the second alignment marker; and a step of capturing an image of the first marker of the first object and the second marker of the second object, thereby detecting, from the captured image including the first marker and the second marker, the relative three-dimensional coordinates of the first predetermined position of the first object relative to the second predetermined position of the second object, and the relative three-dimensional direction vector of the first predetermined direction of the first object relative to the second predetermined direction of the second object.

[0017] The detection method disclosed herein is a detection method for detecting a predetermined position of an object to which a first marker is attached, and includes the steps of: acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object; and capturing an image of the first marker of the object and a second marker attached to a member separate from the object, and detecting the three-dimensional coordinates of the predetermined position of the object relative to the separate member from an image including the captured first marker and the second marker based on the acquired three-dimensional coordinates.

[0018] The detection method disclosed herein is a detection method for detecting a direction of an object to which a first marker is attached toward a predetermined position, and includes the steps of: acquiring information on a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object; and capturing an image of a second marker attached to a member other than the object and the first marker of the object, and detecting, based on the acquired three-dimensional direction vector, a relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the other member from an image including the captured first marker and the second marker.

[0019] A detection method disclosed herein is a detection method for detecting a predetermined position of an object to which a first marker is attached and a direction to the predetermined position, comprising: a step of acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object, and acquiring information on a three-dimensional direction vector of the object relative to the first marker; and a step of capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting, based on the acquired three-dimensional coordinates and three-dimensional direction vector, from an image including the captured first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position relative to the separate member.

[0020] In the computer of the present disclosure, the control unit executes a program to receive an image of an alignment marker and the first marker when a predetermined position of an object to which the alignment marker is attached is aligned with a predetermined reference point of an alignment member on which the alignment marker is placed; based on the received image including the alignment marker and the first marker, obtain information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object; and based on the obtained three-dimensional coordinates, detect the relative three-dimensional coordinates of the predetermined position of the object with respect to the separate member from an image including a second marker attached to the separate member from the object and the first marker of the object.

[0021] In the computer of the present disclosure, the control unit executes a program to receive an image of an alignment marker and a first marker when a predetermined position of an object to which the alignment marker is attached is aligned with a predetermined reference point of an alignment member on which the alignment marker is disposed; based on the received image including the alignment marker and the first marker, obtain information on a three-dimensional direction vector of the predetermined position of the object relative to the first marker attached to the object; and based on the obtained three-dimensional coordinates, detect the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member from an image including a second marker attached to the separate member and the first marker of the object.

[0022] In the computer of the present disclosure, the control unit executes a program to receive an image of the alignment marker and the first marker when a predetermined position of an object to which the alignment marker is attached is aligned with a predetermined reference point of an alignment member on which the alignment marker is disposed; based on the received image including the alignment marker and the first marker, obtain information on the relative three-dimensional coordinate of the predetermined position of the object with respect to the position of the first marker attached to the object, and obtain information on the three-dimensional direction vector of the object with respect to the first marker; and based on the obtained three-dimensional coordinates and three-dimensional direction vector, detect, from the image including the first marker and the second marker, the relative three-dimensional coordinate of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the other member.

[0023] The computer of the present disclosure, when executed by the control unit, receives an image of an alignment marker imaged by a fixed-position imaging device and placed at a predetermined reference point of an alignment member; receives an image of a first marker attached to an object imaged by the imaging device when a predetermined position of the object is aligned with the predetermined reference point of the alignment member; acquires information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object based on the received image including the alignment marker and the received image including the first marker; and detects the three-dimensional coordinates of the predetermined position of the object relative to the separate member from an image including a second marker attached to the separate member and the first marker of the object based on the acquired three-dimensional coordinates.

[0024] The computer of the present disclosure, when executed by the control unit, receives an image of an alignment marker imaged by a fixed-position imaging device and placed at a predetermined reference point of an alignment member; receives an image of a first marker attached to an object imaged by the imaging device when a predetermined position of the object is aligned with the predetermined reference point of the alignment member; acquires information on a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object based on the received image including the alignment marker and the received image including the first marker; and detects a relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to a separate member from the object based on the acquired three-dimensional direction vector from an image including a second marker attached to the separate member from the object and the first marker attached to the object.

[0025] The computer of the present disclosure, when executed by a control unit, receives an image of an alignment marker imaged by a fixed-position imaging device and placed at a predetermined reference point of an alignment member; receives an image of a first marker attached to the object imaged by the imaging device when a predetermined position of the object is aligned with the predetermined reference point of the alignment member; acquires, based on the received image including the alignment marker and the received image including the first marker, information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object and information on the relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker; and detects, based on the acquired three-dimensional coordinates and three-dimensional direction vector, from an image including a second marker attached to a member separate from the object and the first marker attached to the object, the relative three-dimensional coordinates of the predetermined position of the object with respect to the separate member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the separate member.

[0026] The computer of the present disclosure, when executed by a control unit, receives an image of a first marker and the alignment marker attached to an object when the alignment marker is placed at a predetermined position on the object, obtains, based on the received image including the alignment marker and the first marker, information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object, and detects, based on the obtained three-dimensional coordinates, from an image including a second marker attached to a member separate from the object and the first marker of the object, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member.

[0027] The computer of the present disclosure, when executed by a control unit, receives an image of a first marker and the alignment marker attached to an object when the alignment marker is placed at a predetermined position on the object, acquires information about a three-dimensional direction vector in a predetermined direction extending from the predetermined position perpendicular to the first marker based on the received image including the alignment marker and the first marker, and detects a relative three-dimensional direction vector in the predetermined direction with respect to a different member from the object from an image including a second marker attached to the different member and the first marker of the object based on the acquired three-dimensional direction vector.

[0028] The computer of the present disclosure, by executing a program by a control unit, receives an image of a first marker and the alignment marker attached to an object when the alignment marker is placed at a predetermined position on the object; acquires, based on the received image including the alignment marker and the first marker, information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object and information on a three-dimensional direction vector of a predetermined direction extending from the predetermined position so as to be perpendicular to the first marker; and detects, based on the acquired three-dimensional coordinates and three-dimensional direction vector, from an image including a second marker attached to a member separate from the object and the first marker of the object, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member and the relative three-dimensional direction vector of the predetermined direction with respect to the separate member.

[0029] The computer of the present disclosure, by executing a program by a control unit, receives an image of a first alignment marker attached to a first object when the first alignment marker is placed at a first predetermined position on the first object, and obtains, based on the received image including the first alignment marker, information on relative three-dimensional coordinates of the first predetermined position of the first object attached to the first object and information on a three-dimensional direction vector in a first predetermined direction extending from the first predetermined position so as to be perpendicular to the first alignment marker; receives an image of a second alignment marker attached to a second object when a second alignment marker is placed at a second predetermined position on the second object; and obtains, based on the received image including the second alignment marker, information on relative three-dimensional coordinates of the second predetermined position of the second object attached to the second object and information on a three-dimensional direction vector in a second predetermined direction extending from the second predetermined position so as to be perpendicular to the second alignment marker; When an image including the first alignment marker of the first object and the second alignment marker of the second object is received, the relative three-dimensional coordinates of the first predetermined position of the first object with respect to the second predetermined position of the second object and the relative three-dimensional direction vector of the first predetermined direction of the first object with respect to the second predetermined direction of the second object are detected from the received image including the first alignment marker and the second alignment marker.

[0030] In the computer of the present disclosure, the control unit executes a program to acquire information on the relative three-dimensional coordinates of a predetermined position of an object relative to the position of a first marker attached to the object, and based on the acquired three-dimensional coordinates, detects the relative three-dimensional coordinates of the predetermined position of the object relative to a second marker attached to a member separate from the object from an image including the first marker of the object and the separate member.

[0031] In the computer of the present disclosure, the control unit executes a program to acquire information on a three-dimensional direction vector of a predetermined position of an object relative to a first marker attached to the object, and based on the acquired three-dimensional coordinates, detects the relative three-dimensional coordinates of the predetermined position of the object relative to a second marker attached to a member other than the object from an image including the first marker of the object and the second marker attached to the member other than the object.

[0032] In the computer of the present disclosure, the control unit executes a program to acquire information on the relative three-dimensional coordinates of a predetermined position of the object relative to the position of a first marker attached to the object, as well as information on the three-dimensional direction vector of the object relative to the first marker, and detects, from an image including the first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object relative to the other component and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position relative to the other component based on the acquired three-dimensional coordinates and three-dimensional direction vector.

[0033] FIG. 1 is a schematic diagram showing a configuration of a position detection system according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram showing a method for calibrating the position detection system according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram showing one usage form of the position detection system according to an embodiment of the present disclosure, which is a schematic diagram showing detection of a part that cannot be seen from the outside, such as inside a body. FIG. 4 is a flowchart showing the flow of a calibration process in the position detection system according to an embodiment of the present disclosure. FIG. 5 is a flowchart showing the flow of a detection process in the position detection system according to an embodiment of the present disclosure. FIG. 6 is a schematic diagram showing a modified example of the position detection system according to an embodiment of the present disclosure, which is a form when an alignment marker is placed at a predetermined position on an object. FIG. 7 is a schematic diagram showing a modified example of the position detection system according to an embodiment of the present disclosure, which is a configuration of a position detection system when there are multiple objects. FIG. 8 is a schematic diagram showing an example of a usage form of the position detection system according to an embodiment of the present disclosure. FIG. 9 is a schematic diagram showing an example of a usage form of the position detection system according to an embodiment of the present disclosure, which is a form of drawing in three-dimensional space. FIG. 10 is a schematic diagram showing an example of a usage form of the position detection system according to an embodiment of the present disclosure, which is a form of using an object as a laser pointer.

[0034] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIGS. 1 to 5 are diagrams showing parts of a position detection system according to an embodiment of the present disclosure. Of these, FIG. 1 is a schematic diagram showing the configuration of the position detection system according to this embodiment, and FIG. 2 is a schematic diagram showing a method of calibrating the position detection system according to this embodiment. FIG. 3 is a schematic diagram showing one use form of the position detection system according to this embodiment, which is a schematic diagram showing detection of a part that cannot be seen from the outside, such as inside a body. FIG. 4 is a flowchart showing the flow of calibration processing in the position detection system according to this embodiment. FIG. 5 is a flowchart showing the flow of detection processing in the position detection system according to this embodiment.

[0035] —Position Detection System 1— FIG. 1 is a block diagram showing the configuration of a position detection system 1 according to this embodiment. The position detection system 1 according to this embodiment includes a computer 2 and an imaging device 3. As shown in FIGS. 2 and 3 , the imaging device 3 is used to position an object (target object 10) to be detected, and captures an alignment marker 71 attached to an alignment member 7, a first marker 11 attached to an object having a predetermined length, and a second marker 14 attached to another member 13 (e.g., a human body). Based on an image including the captured first marker 11 and alignment marker 71, the position detection system 1 acquires at least one of information on the three-dimensional coordinates and three-dimensional directional vectors of the first marker 11 relative to the alignment marker 71, and measures the three-dimensional coordinates and three-dimensional directional vectors of a predetermined position (point A) relative to the first marker 11. For example, the computer 2 is connected to the imaging device 3 so that they can communicate with each other. The image capturing device 3 and the computer 2 may be communicably connected to each other via a communication network such as the Internet. Each of the components of the position detection system 1 will be described below.

[0036] The alignment member 7 is a member used for calibration to define a predetermined position (A) of the object 10. The alignment member 7 is treated as a plate-like member or a virtual plane, and an alignment marker 71 is placed on the alignment member 7. Here, the predetermined position (A) refers to a specific location on the object 10, and in the case of a rod-shaped member, it may be the tip of the rod-shaped member, for example. In the present disclosure, the tip of a member having a longitudinal axis will be described as an example.

[0037] The alignment marker 71 is an AR marker that is placed on the alignment member 7 and is used for calibration together with the first marker 11. The alignment marker 71 is placed in a position on the virtual plane that does not overlap with a position defined as the origin (O).

[0038] The first marker 11 is an AR marker that is attached to an object (target object 10) having a predetermined length and is used to grasp the positional relationship of a predetermined position (A) of the target object 10. The size of the first marker 11 is such that it does not overlap with the alignment marker 71 when performing calibration.

[0039] The above-mentioned function is achieved by capturing images of the alignment marker 71 and the first marker 11 using the imaging device 3.

[0040] <Configuration of Computer 2> The computer 2 can be used via a communication network such as the Internet. The computer 2 has a control unit 4, a storage unit 5, and a communication unit 6. The control unit 4 is configured as a microcomputer including a CPU and semiconductor memory, and controls the operation of the computer 2. More specifically, the control unit 4 functions as a receiving unit 41, a calculating unit 42, and a detecting unit 43 by executing a program stored in the storage unit 5.

[0041] The accepting unit 41 accepts images captured by the imaging device 3. For example, the accepting unit 41 accepts image data of the alignment marker 71, the first marker 11, and the second marker 14 captured by the imaging device 3.

[0042] The calculation means 42 acquires information on the relative three-dimensional coordinates of a predetermined position (A) of the object 10 with respect to the attachment position of the first marker 11 attached to the object 10, based on image data including at least the alignment marker 71 and the first marker 11. First, the calculation means 42 calculates, from the image data captured by the imaging device 3, a homogeneous transformation matrix ｃ H ｍ１ Furthermore, based on the positions of the alignment marker 71 (m1) and the origin (O), the homogeneous transformation matrix Ｏ H ｍ１ －１ At this time, the predetermined position (A) of the object 10 in the m2 coordinate system coincides with the origin (O). ｃ Hｍ１ and Ｏ H ｍ１ －１ Based on ｃ H Ｏ ( ｃ H ｍ１Ｏ H ｍ１ －１ ) is calculated. The homogeneous transformation matrix is ​​used to efficiently calculate coordinate transformation and rigid body transformation by matrix calculation, and is expressed in three-dimensional space by the following equation.

[0043] (In Equation 1, r represents a rotation matrix, and t represents a translation vector.)

[0044] Furthermore, the calculation means 42 calculates a homogeneous transformation matrix based on the matrices at the positions of the image capture device 3 (point C) and the first marker 11 (point m2). ｃ H ｍ２ －１ Then, the calculation means 42 calculates the calculated ｃ H Ｏ and ｃ H ｍ２ －１ Based on ｍ２ H Ｏ ( ｃ H ｍ２ －１ ｃ H Ｏ In this state, the tilt of the first marker 11 (point m2) and the origin (O) is reflected, so for example, if the rotation matrix (R) is set to non-rotation and the resulting translation vectors are tx, ty, and tz, the following equation is obtained.

[0045]

[0046] In this way, calibration is performed in a state where the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7 .

[0047] The calculation means 42 calculates a homogeneous transformation matrix calculated based on the matrices at the positions of the image capture device 3 (point C) and the first marker 11 (m2). ｃ H ｍ２ ,and, ｍ２ H Ａ Based on ｃH Ａ ( ｃ H ｍ２ｍ２ H Ａ In this way, the calculation means 42 sets the predetermined position (A) of the object 10 in a coordinate system centered on m2, calculates the homogeneous transformation matrix including the rotation matrix, and extracts the translation (position) component, thereby determining the predetermined position (A) of the object 10 in the m2 coordinate system.

[0048] The detection means 43 captures an image of a second marker 14 attached to a member 13 separate from the object 10 and a first marker 11 on the object 10, and based on at least one of the acquired three-dimensional coordinates and three-dimensional direction vectors, detects at least one of the relative three-dimensional coordinates and three-dimensional direction vectors of a predetermined position (A) of the object 10 relative to the separate member 13 from an image including the captured first marker 11 and second marker 14.

[0049] The other member 13 is a member to which the second marker 14 is attached, and is an object for which the position of the predetermined position (A) of the object 10 is determined from an image including the first marker 11 and the second marker 14. There are no particular limitations on what can be used as the other member 13, and for example, when an endoscope is used as the object 10, the patient's body may be used as the other member 13.

[0050] The second marker 14 is an AR marker that is attached to another member 13 and is used in cooperation with the first marker 11 to grasp the positional relationship between a predetermined position (A) of the object 10 and the other member 13 .

[0051] The programs executed by the control unit 4 are not limited to those stored in the storage unit 5. The programs executed by the control unit 4 may be those transmitted from an external device to the control unit 4 via the communication unit 6, those stored in a storage medium such as a USB memory attached to the computer 2, or those stored in a server (for example, a cloud server) separate from the computer 2.

[0052] The storage unit 5 is composed of a hard disk drive (HDD), random access memory (RAM), read only memory (ROM), solid state drive (SSD), etc. Various programs including programs executed by the control unit 4 are stored in the storage unit 5. The storage unit 5 also stores images captured by the imaging device 3. Details of the information stored in the storage unit 5 will be described later.

[0053] The communication unit 6 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc., and includes an interface for performing wired or wireless communication. That is, the communication unit 6 is configured from a communication interface configured to communicate with other devices via a communication network.

[0054] The imaging device 3 is a camera device that is provided in one or more units in the position detection system 1. The imaging device 3 is grounded or suspended, and is placed in a position where it can capture images of the alignment marker 71, the first marker 11, the second marker 14, the reference point (origin (O)), and the target object 10, and captures these images to obtain image data.

[0055] <Calibration Process> First, the calibration process in this embodiment will be described with reference to the schematic diagram shown in Fig. 2 and the flowchart shown in Fig. 4. Note that the operation of the position detection system 1 described below is performed by the control unit 4 of the computer 2 executing a program stored in the storage unit 5.

[0056] First, the user places the alignment marker 71 on the alignment member 7 .

[0057] Next, the user aligns a predetermined position (A) of the object 10 to which the first marker 11 is attached with the reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is disposed (step S10). At this time, the imaging device 3 is installed in a position where it can capture images of the alignment marker 71 and the first marker 11.

[0058] Next, the image capturing device 3 captures images of the alignment marker 71 and the first marker 11 (step S11). As a result, the receiving means 41 receives and grasps information such as the position and layout relationship of the image information defining the alignment member (virtual plane) 7 registered in the alignment marker 71 and the image information defining the target object 10 registered in the first marker 11.

[0059] Next, the calculation means 42 acquires information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the attachment position of the first marker 11 based on the images of the alignment marker 71 and the first marker 11 received by the reception means 41 (step S12). As a result, calibration is performed in a state where the predetermined position (A) of the object 10 is aligned with a predetermined reference point (origin (O)) of the alignment member 7.

[0060] <Detection Processing> Next, the detection processing after calibration in this embodiment will be described with reference to the schematic diagram shown in FIG. 3 and the flowchart shown in FIG.

[0061] First, the imaging device 3 captures images of the second marker 14 attached to the other member 13 and the first marker 11 of the object 10 on the virtual plane defined by the alignment marker 71 (step S13). As a result, the receiving means 41 receives and understands information such as the position and arrangement relationship of the image information defining the other member 13 registered in the second marker 14 and the image information defining the object 10 registered in the first marker 11.

[0062] Next, based on the three-dimensional coordinates acquired in step S12 and the image information captured in step S13 and received by the receiving means 41, the detecting means 43 detects the three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the other member 13 (step S14). This makes it possible to detect the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the other member 13 from the captured image including the first marker 11 and the second marker 14, thereby enabling continuous position detection.

[0063] According to the detection method of the present embodiment configured as described above, when detecting the predetermined position (A) of the object 10 to which the first marker 11 is attached, the detection method of the present disclosure aligns the predetermined position (A) of the object 10 with a predetermined reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is arranged. Next, an image of the alignment marker 71 and the first marker 11 arranged on the alignment member 7 is captured. Next, based on the captured image including the alignment marker 71 and the first marker 11, information on the three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the position of the first marker 11 attached to the object 10 is obtained. Next, by capturing an image of a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative three-dimensional coordinates of a predetermined position (A) of the object 10 relative to the separate member 13 are detected from an image including the captured first marker 11 and second marker 14 based on the acquired three-dimensional coordinates.

[0064] According to this detection method, even when the predetermined position (A) of the object 10 cannot be visually recognized, the predetermined position (A) of the object 10 relative to another member 13 can be accurately determined. More specifically, a technology for detecting the three-dimensional coordinates of the predetermined position (A) of the object 10, to which a marker is attached, from an image of the marker has been desired. For example, when attempting to determine the position of the tip of an endoscope inside a body, it is difficult to visually recognize the tip from the outside, and it is also difficult to accurately determine where the tip is located inside the body. A technology for solving these problems has not been known. According to the detection method configured as described above, the three-dimensional coordinates of the predetermined position (A) of the object 10 to which a marker is attached can be detected from an image of the marker, thereby avoiding these problems.

[0065] Furthermore, in the detection method of this embodiment, as described above, the alignment member 7 has a flat surface, on which a predetermined reference point (origin (O)) is formed and on which the alignment marker 71 is also disposed. In this case, calibration can be performed efficiently.

[0066] Furthermore, in the detection method and computer 2 of this embodiment, as described above, the control unit 4 of the computer 2 of the present disclosure receives an image of the alignment marker 71 and the first marker 11 when the predetermined position (A) of the object 10 to which the first marker 11 is attached is aligned with a predetermined reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is disposed, by executing a program. Next, based on the received image including the alignment marker 71 and the first marker 11, information on the three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the position of the first marker 11 attached to the object 10 is acquired. Next, based on the acquired three-dimensional coordinates, the computer 2 detects the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the separate member 13 from the object 10 from an image including the second marker 14 attached to the separate member 13 and the first marker 11 of the object 10. In this case, the computer 2 can detect the three-dimensional coordinates of a predetermined position (A) on the object 10 to which the marker is attached from an image of the marker.

[0067] The detection method and computer 2 according to the present disclosure are not limited to the above-described aspects, and various modifications can be made.

[0068] In the present embodiment, the description has been given by way of example of a form in which information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 is acquired, but the present invention is not limited to this. For example, instead of the three-dimensional coordinates, information on a relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position with respect to another member 13 may be acquired.

[0069] Furthermore, information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 may be acquired, as well as information on the three-dimensional direction vector of the predetermined position (A) relative to the first marker 11. Alternatively, by capturing an image of the first marker 11 of the object 10 and a second marker 14 attached to a member 13 separate from the object 10, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the separate member 13 and the relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) relative to the separate member 13 may be detected from the captured image including the first marker 11 and the second marker 14 based on the acquired three-dimensional coordinates and three-dimensional direction vector. In this case, by acquiring the three-dimensional direction vector in addition to the three-dimensional coordinates, the predetermined position (A) on the object 10 can be identified with higher accuracy.

[0070] Furthermore, in the present embodiment, an example has been described in which the alignment marker 71 arranged on the alignment member 7 and the first marker 11 attached to the object 10 are simultaneously imaged, but this is not limiting. For example, in the detection method disclosed herein, the alignment marker 71 is arranged at a predetermined reference point (origin (O)) on the alignment member 7, and the alignment marker 71 is imaged by the fixed-position imaging device 3. Next, the alignment marker 71 is removed, a predetermined position (A) on the object 10 is aligned with the predetermined reference point (origin (O)) on the alignment member 7, and the first marker 11 is imaged by the imaging device 3. Next, based on the image including the imaged alignment marker 71 and the imaged first marker 11, information on the three-dimensional coordinates of the predetermined position (A) on the object 10 relative to the position of the first marker 11 attached to the object 10 is obtained. In this case, after capturing an image of the alignment marker 71 attached to a predetermined reference point (origin (O)), the first marker 11 is captured with a predetermined position (A) of the object 10 aligned with the predetermined reference point (origin (O)). This allows calibration even with a narrow angle of view based on three-dimensional coordinates. Next, by capturing an image of the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the separate member 13 are detected from the captured image including the first marker 11 and second marker 14 based on the acquired three-dimensional coordinates. This allows the detection process to be performed.

[0071] In another detection method disclosed herein, an alignment marker 71 is placed at a predetermined reference point (origin (O)) on the alignment member 7, and an image of the alignment marker 71 is captured by the fixed imaging device 3. Next, the alignment marker 71 is removed, a predetermined position (A) on the object 10 is aligned with the predetermined reference point (origin (O)) on the alignment member 7, and an image of the first marker 11 is captured by the imaging device 3. Next, based on the captured image including the alignment marker 71 and the captured image including the first marker 11, information on the three-dimensional direction vector of the predetermined position (A) on the object 10 relative to the first marker 11 attached to the object 10 is obtained. In this case, after capturing an image of the alignment marker 71 attached to a predetermined reference point (origin (O)), the first marker 11 is captured with a predetermined position (A) on the object 10 aligned with the predetermined reference point (origin (O)), and calibration is performed based on the three-dimensional direction vector. This allows calibration to be performed with a narrow angle of view and focusing on the three-dimensional direction. Next, by capturing an image of the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 on the object 10, a relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) relative to the separate member 13 is detected from the captured image including the first marker 11 and second marker 14 based on the acquired three-dimensional direction vector. This allows detection processing to be performed.

[0072] In yet another detection method of the present disclosure, an alignment marker 71 is placed at a predetermined reference point (origin (O)) on the alignment member 7, and an image of the alignment marker 71 is captured by the fixed imaging device 3. Next, the alignment marker 71 is removed, a predetermined position (A) on the object 10 is aligned with the predetermined reference point (origin (O)) on the alignment member 7, and an image of the first marker 11 is captured by the imaging device 3. Next, based on the captured image including the alignment marker 71 and the captured image including the first marker 11, information on the three-dimensional coordinates of the predetermined position (A) on the object 10 relative to the position of the first marker 11 attached to the object 10 and information on the three-dimensional direction vector of the predetermined position (A) on the object 10 relative to the first marker 11 are obtained. In this case, after capturing an image of the alignment marker 71 attached to a predetermined reference point (origin (O)), the first marker 11 is captured with a predetermined position (A) of the object 10 aligned with the predetermined reference point (origin (O)), and calibration is performed based on the three-dimensional coordinates and three-dimensional direction vector. This allows calibration to be performed with a narrow angle of view and by combining the three-dimensional coordinates and three-dimensional direction. Next, the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 on the object 10 are captured. Based on the acquired three-dimensional coordinates and three-dimensional direction vector, the captured image including the first marker 11 and the second marker 14 detects the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the separate member 13 and the relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) relative to the separate member 13. This allows detection processing to be performed.

[0073] Furthermore, in this embodiment, a recess is formed at the reference point (origin (O)) of the alignment member 7, and the predetermined position (A) of the object 10 may be aligned with the reference point (origin (O)) by inserting a portion of the object 10 at the predetermined position (A) into the recess. In this case, by inserting the portion of the object 10 relating to the predetermined position (A) into the recess, the predetermined position (A) of the object 10 can be aligned with the reference point (origin (O)).

[0074] Next, FIG. 6 will be described. In the detection method disclosed herein, a first marker 11 is placed at a predetermined position (B) on the object 101, and an alignment marker 12 is placed at point m2. Next, images of the first marker 11 and the alignment marker 12 are captured. Then, based on the captured image including the alignment marker 12 and the first marker 11, information on the relative three-dimensional coordinates of the predetermined position (point m2) of the alignment marker 12 with respect to the first marker 11 (point m1) placed on the object 101 is acquired. This allows calibration to be performed even if the alignment marker 12 is removed from the object 101. Next, an image of a second marker 14 attached to a member 13 separate from the object 101 and the first marker 11 on the object 101 is captured. Based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of point m2 with respect to the separate member 13 are detected from the captured image including the second marker 14 and the first marker 11. At this time, as shown in FIG. 6, the calculation means 42 calculates a homogeneous transformation matrix based on the matrices at the positions of the image pickup device 3 (point C) and the reference point (point m2) from the image data captured by the image pickup device 3. ｃ H ｍ２ －１ Furthermore, based on the matrices at the positions of the image capture device 3 (point C) and the first marker 11 (point m1), a homogeneous transformation matrix ｃ H ｍ１ And then, ｃ H ｍ１ and ｃ H ｍ２ －１ Based on ｍ１ H ｍ２ ( ｃ H ｍ２ －１ ｃ H ｍ１ ) is calculated. In this case, the m1 coordinate system and the m2 coordinate system are treated as the same coordinate system and calibration is performed based on three-dimensional coordinates, so calibration can be performed with a narrow angle of view and by calculating the position and orientation of the m2 coordinate system in the m1 coordinate system using three-dimensional coordinates. Therefore, for example, when assembling parts using a crane, it is also possible to estimate the positions at which parts are connected to each other. Note that in the detection process, ｃ Hｍ２ ( ｃ H ｍ１ｍ２ H ｍ１ －１ ) is required.

[0075] In addition, in the detection method disclosed herein, an alignment marker 12 is placed at a predetermined position (B) on the object 101 to which the first marker 11 is attached, and an image of the first marker 11 and the alignment marker 12 is captured. Next, based on the captured image including the alignment marker 12 and the first marker 11, information on a three-dimensional direction vector in a predetermined direction extending from the predetermined position (B) perpendicular to the first marker 11 is acquired, thereby performing calibration based on the three-dimensional direction vector. Next, an image of a second marker 14 attached to a member 13 separate from the object 101 and the first marker 11 of the object 101 is captured, and based on the acquired three-dimensional direction vector, a relative three-dimensional direction vector in the predetermined direction with respect to the separate member 13 is detected from the captured image including the first marker 11 and the second marker 14.

[0076] Furthermore, in the detection method disclosed herein, an alignment marker 12 is placed at a predetermined position (B) on the object 101 to which the first marker 11 is attached, and an image of the first marker 11 and the alignment marker 12 is captured. Next, based on the captured image including the alignment marker 12 and the first marker 11, information on the three-dimensional coordinate of the predetermined position (B) on the object 101 relative to the position of the first marker 11 attached to the object 101 and information on a three-dimensional direction vector in a predetermined direction extending from the predetermined position (B) so as to be perpendicular to the first marker 11 is acquired, thereby performing calibration based on the three-dimensional coordinates and the three-dimensional direction vector. Next, by capturing an image of a second marker 14 attached to a member 13 other than the object 101 and the first marker 11 of the object 101, the relative three-dimensional coordinates of a predetermined position (B) of the object 101 relative to the other member 13 and the relative three-dimensional direction vector in a predetermined direction relative to the other member 13 are detected from an image including the captured first marker 11 and second marker 14 based on the acquired three-dimensional coordinates and three-dimensional direction vector.

[0077] Next, FIG. 7 will be described. In the detection method disclosed herein, a first alignment marker 15 is placed at a first predetermined position (point A1) on a first object 111 to which a first marker 11 is attached, and an image of the first marker 11 and the first alignment marker 15 is captured. At this time, the first alignment marker 15 is positioned, for example, at point A1 in FIG. 7. Based on the captured image including the first alignment marker 15 and the first marker 11, information on the three-dimensional coordinates of the first predetermined position (point A1) on the first object 111 relative to the position of the first marker 11 attached to the first object 111 and information on a three-dimensional direction vector in a first predetermined direction extending perpendicularly from the first predetermined position (point A1) to the first alignment marker 15 are obtained. Next, a second alignment marker 16 is placed at a second predetermined position (point A2) on the second object 112 to which the second marker 14 is attached, and an image of the second marker 14 and the second alignment marker 16 is captured. At this time, the second alignment marker 16 is positioned at, for example, point A2 in FIG. 7 . Next, based on the captured image including the second alignment marker 16 and the second marker 14, information on the three-dimensional coordinates of the second predetermined position (point A2) on the second object 112 relative to the position of the second marker 14 attached to the second object 112 and information on a three-dimensional direction vector in a second predetermined direction extending from the second predetermined position (point A2) so as to be perpendicular to the second alignment marker 16 is acquired. Next, by capturing an image of the first marker 11 on the first object 111 and the second marker 14 on the second object 112, the relative three-dimensional coordinates of the first predetermined position (point A1) of the first object 111 with respect to the second predetermined position (point A2) of the second object 112 and the relative three-dimensional direction vector of the first predetermined direction of the first object 111 with respect to the second predetermined direction of the second object 112 are detected from the captured image including the first marker 11 and the second marker 14. In this case, even if the first object 111 and the second object 112 overlap and a blind spot exists when capturing an image by the imaging device 3, the position difference at the alignment location can be measured and the alignment of point A1 and point A2 can be performed.Furthermore, when calculating the difference in tilt between the first object 111 and the second object 112, the calculation means 42 can express it as, for example, the inner product of the image data captured by the image capture device 3 from the image capture device 3 (point C) to point A1 (C-A1) and the inner product of point C to point A' (C-A'). Then, the difference between point A1 and point A' is expressed as follows when point A1 is used as the reference. Ａ１ H Ａ '= ｃ H Ａ１ －１ ｃ H Ａ In this case, the difference in tilt at the alignment point can be measured, and the holes formed in the first object 111 and the second object 112 can be aligned. In this way, it becomes possible to smoothly insert a rod-shaped member into the communication hole formed by connecting the holes.

[0078] Furthermore, although the calibration process has been described in detail in this embodiment, the detection method of this embodiment may acquire information on the three-dimensional coordinates and three-dimensional direction vectors of the predetermined position (A) of the object 10 in advance. For example, the detection method of this embodiment acquires information on the three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the position of the first marker 11 attached to the object 10. Alternatively, by capturing an image of the first marker 11 of the object 10 and a second marker 14 attached to a member 13 separate from the object 10, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the separate member 13 may be detected from the captured image including the first marker 11 and the second marker 14 based on the acquired three-dimensional coordinates. In this case, information on the three-dimensional coordinates of the predetermined position (A) of the object 10 acquired in advance may be used.

[0079] Furthermore, for example, the detection method in this embodiment acquires information on a relative three-dimensional direction vector of a predetermined position (A) on the object 10 with respect to a first marker 11 attached to the object 10. Alternatively, by capturing an image of a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 on the object 10, a relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) with respect to the separate member 13 may be detected from an image including the captured first marker 11 and second marker 14 based on the acquired three-dimensional direction vector. In this case, information on the three-dimensional direction vector of the predetermined position (A) on the object 10 that has been acquired in advance can be used.

[0080] Furthermore, for example, the detection method in this embodiment acquires information on the three-dimensional coordinates of a predetermined position (A) on the object 10 relative to the position of a first marker 11 attached to the object 10, and also acquires information on a three-dimensional direction vector of the object 10 relative to the first marker 11. Alternatively, by capturing an image of a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 on the object 10, the relative three-dimensional coordinates of the predetermined position (A) on the object 10 relative to the separate member 13 and the relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) relative to the separate member 13 may be detected from the captured image including the first marker 11 and the second marker 14 based on the acquired three-dimensional coordinates and three-dimensional direction vector. In this case, information on the three-dimensional coordinates and three-dimensional direction vector of the predetermined position (A) on the object 10 acquired in advance may be used.

[0081] In the computer 2 of this embodiment, the control unit 4 executes a program to receive images of the alignment marker 71 and the first marker 11 when the predetermined position (A) of the object 10, to which the first marker 11 is attached, is aligned with a predetermined reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is disposed. Furthermore, based on the received image including the alignment marker 71 and the first marker 11, information on a three-dimensional direction vector of the predetermined position (A) of the object 10 relative to the first marker 11 attached to the object 10 is acquired. In this case, the computer 2 can perform calibration using the three-dimensional direction vector instead of the three-dimensional coordinates. Furthermore, based on the acquired three-dimensional coordinates, the computer 2 detects the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the separate member 13 from the object 10 from an image including the second marker 14 attached to the separate member 13 and the first marker 11 of the object 10.

[0082] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive images of the alignment marker 71 and the first marker 11 when the predetermined position (A) of the object 10 to which the first marker 11 is attached is aligned with a predetermined reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is disposed. Furthermore, based on the received image including the alignment marker 71 and the first marker 11, information on the three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the position of the first marker 11 attached to the object 10 is obtained, as well as information on the three-dimensional direction vector of the object 10 relative to the first marker 11. In this case, the computer 2 performs calculations including the three-dimensional direction vector in addition to the three-dimensional coordinates, thereby enabling the computer 2 to identify the predetermined position (A) on the object 10 with higher accuracy. In addition, based on the acquired three-dimensional coordinates and three-dimensional direction vector, the relative three-dimensional coordinates of a predetermined position (A) of the object 10 relative to another member 13 and the relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) relative to the other member 13 are detected from the image including the first marker 11 and the second marker 14.

[0083] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the alignment marker 71, which is placed at a predetermined reference point (origin (O)) of the alignment member 7 and is captured by the fixed-position imaging device 3. The computer 2 also receives an image of the first marker 11 attached to the object 10, which is captured by the imaging device 3 when the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7. Furthermore, based on the received image including the alignment marker 71 and the received image including the first marker 11, the computer 2 acquires information on the three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the position of the first marker 11 attached to the object 10. In this case, the computer 2 captures an image of the alignment marker 71 attached to the predetermined reference point (origin (O)), and then captures an image of the first marker 11 when the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)). This allows calibration even with a narrow angle of view based on the three-dimensional coordinates. In addition, based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of a predetermined position (A) of the object 10 relative to the other member 13 are detected from an image including a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10.

[0084] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the alignment marker 71, which is placed at a predetermined reference point (origin (O)) of the alignment member 7 and is captured by the fixed-position imaging device 3. The computer 2 also receives an image of the first marker 11, which is attached to the object 10 and is captured by the imaging device 3 when the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7. The computer 2 also obtains information about the three-dimensional direction vector of the predetermined position (A) of the object 10 relative to the first marker 11 attached to the object 10, based on the received image including the alignment marker 71 and the received image including the first marker 11. In this case, the computer 2 captures an image of the alignment marker 71 attached to a predetermined reference point (origin (O)), then captures an image of the first marker 11 with a predetermined position (A) on the object 10 aligned with the predetermined reference point (origin (O)), and performs calibration based on the three-dimensional direction vector, thereby enabling calibration with a narrow angle of view and focusing on the three-dimensional direction. Furthermore, based on the acquired three-dimensional direction vector, the computer 2 detects a relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) relative to the separate member 13 from the object 10 from an image including the second marker 14 attached to the separate member 13 and the first marker 11 attached to the object 10.

[0085] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the alignment marker 71, which is placed at a predetermined reference point (origin (O)) of the alignment member 7 and is captured by the fixed-position imaging device 3. The computer 2 also receives an image of the first marker 11 attached to the object 10 and captured by the imaging device 3 when the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7. The computer 2 also obtains, based on the received image including the alignment marker 71 and the received image including the first marker 11, information on the three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the position of the first marker 11 attached to the object 10 and information on the three-dimensional direction vector of the predetermined position (A) of the object 10 relative to the first marker 11. In this case, the computer 2 captures an image of the alignment marker 71 attached to a predetermined reference point (origin (O)), then captures an image of the first marker 11 with a predetermined position (A) of the object 10 aligned with the predetermined reference point (origin (O)), and performs calibration based on the three-dimensional coordinates and three-dimensional direction vector. This allows calibration to be performed with a narrow angle of view and by combining the three-dimensional coordinates and three-dimensional direction. Furthermore, based on the acquired three-dimensional coordinates and three-dimensional direction vector, the computer 2 detects, from an image including the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 attached to the object 10, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the separate member 13, and the relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) with respect to the separate member 13.

[0086] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the first marker 11 and the alignment marker 12 attached to the object 101 when the alignment marker 12 is placed at a predetermined position (B) on the object 101. Furthermore, based on the received image including the alignment marker 12 and the first marker 11, information on the three-dimensional coordinates of the predetermined position (B) on the object 101 relative to the position of the first marker 11 attached to the object 101 is acquired. In this case, the computer 2 treats the first marker 11 as being in the same coordinate system as the m1 coordinate system and the m2 coordinate system, thereby performing calibration based on the three-dimensional coordinates. This allows calibration with a narrow angle of view and by determining the position and orientation of the m2 coordinate system in the m1 coordinate system using three-dimensional coordinates. In addition, based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of a predetermined position (B) of the object 101 relative to the other member 13 are detected from an image including a second marker 14 attached to a member 13 other than the object 101 and the first marker 11 of the object 101.

[0087] In the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the first marker 11 and the alignment marker 12 attached to the object 101 when the alignment marker 12 is placed at a predetermined position (B) on the object 101. Furthermore, based on the received image including the alignment marker 12 and the first marker 11, information on a three-dimensional direction vector in a predetermined direction extending perpendicularly from the predetermined position (B) to the first marker 11 is acquired. In this case, the computer 2 can perform calibration based on the three-dimensional direction vector by treating the first marker 11 as being in the same coordinate system as the m1 coordinate system and the m2 coordinate system. Furthermore, based on the acquired three-dimensional direction vector, a relative three-dimensional direction vector in a predetermined direction with respect to the separate member 13 is detected from an image including the second marker 14 attached to the separate member 13 and the first marker 11 of the object 101.

[0088] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the first marker 11 and the alignment marker 12 attached to the object 101 when the alignment marker 12 is placed at a predetermined position (B) on the object 101. Based on the received image including the alignment marker 12 and the first marker 11, information on the three-dimensional coordinate of the predetermined position (B) on the object 101 relative to the position of the first marker 11 attached to the object 101 and information on a three-dimensional direction vector in a predetermined direction extending from the predetermined position (B) so as to be perpendicular to the first marker 11 is acquired. In this case, the computer 2 can perform calibration based on the three-dimensional coordinates and the three-dimensional direction vector by treating the first marker 11 as being in the same coordinate system as the m1 coordinate system and the m2 coordinate system. In addition, based on the acquired three-dimensional coordinates and three-dimensional direction vector, the relative three-dimensional coordinates of a predetermined position (B) of the object 101 relative to the other member 13 and the relative three-dimensional direction vector in a predetermined direction relative to the other member 13 are detected from an image including a second marker 14 attached to a member 13 other than the object 101 and the first marker 11 of the object 101.

[0089] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the first alignment marker 15 attached to the first object 111 when the first alignment marker 15 is placed at a first predetermined position on the first object 111. Furthermore, based on the received image including the first alignment marker 15, information on the relative three-dimensional coordinates of the first predetermined position of the first object 111 attached to the first object 111 and information on a three-dimensional direction vector in a first predetermined direction extending from the first predetermined position so as to be perpendicular to the first alignment marker 15 is acquired. Furthermore, an image of the second alignment marker 16 attached to the second object 112 when the second alignment marker 16 is placed at a second predetermined position on the second object 112 is received. Furthermore, based on the received image including the second alignment marker 16, the image processing unit 100 acquires information on the relative three-dimensional coordinates of a second predetermined position of the second object 112 attached to the second object 112 and information on a three-dimensional direction vector of a second predetermined direction extending from the second predetermined position perpendicular to the second alignment marker 16. Furthermore, when an image including the first alignment marker 15 of the first object 111 and the second alignment marker 16 of the second object 112 is received, the image processing unit 100 detects, from the received image including the first alignment marker 15 and the second alignment marker 16, the relative three-dimensional coordinates of the first predetermined position of the first object 111 with respect to the second predetermined position of the second object 112 and the relative three-dimensional direction vector of the first predetermined direction of the first object 111 with respect to the second predetermined direction of the second object 112. In this case, even if the first object 111 and the second object 112 overlap and there is a blind spot for the imaging device 3, the computer 2 can perform alignment and measure the position difference and tilt difference at the alignment point.

[0090] The computer 2 in this embodiment may also acquire information on the three-dimensional coordinates and three-dimensional direction vector of the predetermined position (A) of the object 10 in advance. For example, the computer 2 in this embodiment acquires information on the three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the position of the first marker 11 attached to the object 10 by causing the control unit 4 to execute a program. Furthermore, based on the acquired three-dimensional coordinates, the computer 2 may detect the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the separate member 13 from the object 10 from an image including the first marker 11 of the object 10 and the second marker 14 attached to the separate member 13. In this case, the previously acquired information on the three-dimensional coordinates of the predetermined position (A) of the object 10 can be used.

[0091] Furthermore, for example, the computer 2 in this embodiment may acquire information on a three-dimensional direction vector of a predetermined position (A) of the object 10 relative to a first marker 11 attached to the object 10 by causing the control unit 4 to execute a program, and may also detect, based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to another member 13 from an image including a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10. In this case, information on the three-dimensional direction vector of the predetermined position (A) on the object 10 that has been acquired in advance may be used.

[0092] Furthermore, for example, in the computer 2 according to the present embodiment, the control unit 4 executes a program to acquire information on the relative three-dimensional coordinates of a predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10, as well as information on a three-dimensional direction vector of the object 10 with respect to the first marker 11. Furthermore, based on the acquired three-dimensional coordinates and three-dimensional direction vector, the computer 2 may detect, from an image including the first marker 11 and the second marker 14, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to another member 13, and the relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) with respect to the other member 13. In this case, information on the three-dimensional coordinates and three-dimensional direction vector of the predetermined position (A) on the object 10 that has been acquired in advance can be used.

[0093] 8, when the target object 10 is treated as, for example, a part of a welding robot, the detection method and computer 2 according to this embodiment can measure whether the welding robot is operating in the correct position by capturing images of the alignment marker 71 and the first marker 11 with the imaging device 3. In this case, by attaching an AR marker to a member that can measure not only the position but also the welding angle, it is possible to determine whether the part is placed in the specified position and the welding point for that part.

[0094] Furthermore, by using the detection method and computer 2 according to this embodiment, it is possible to measure, for example, a displacement of the ground surface due to a landslide or the like at an erosion control site, as shown in Fig. 9. At this time, a fixed imaging device 3 captures images of an object 10 installed on the ground surface and having a first marker 11 attached thereto over time, capturing images of the first marker 11 before and after the ground surface moves. In this case, it is possible to grasp the ground surface, the amount of movement, and the direction of movement.

[0095] Furthermore, by using the detection method and computer 2 according to this embodiment, it is also possible to perform drawing in a three-dimensional space, as shown in Fig. 10. In this case, for example, a rectangular parallelepiped marker is attached to one end of the object 10 that is not the handle portion when drawing. First, in the alignment process, from the calibration process according to this embodiment, ｃ H Ａ Then, ｍ１ H Ａ = ｃ H ｍ１ －１ ｃ H Ａ from ｍ１ H Ａ = ｃ H ｍ１ －１ ｃ H ｍ２ｍ２ H Ａ By guiding ｍ１ H Ａ t ｚ The quantity is the vertical distance from the imaginary plane to point A, and t ｚ is equal to or less than the threshold value, the drawing can be performed in the coordinate system of point A. Therefore, by measuring the length from the virtual plane generated using the alignment marker 71 to point A, the contact between the two can be defined.

[0096] 11, the target 10 can also be used as a laser pointer. ｍ１ H ｍ２ = ｃ H ｍ１ －１ ｃ H ｍ２ By calculating the above, the position and inclination of the first marker 11 are first determined when the m1 coordinate system is used as a reference. Here, if the first marker 11 can be positioned on the central axis of the object 10, alignment is not required. Then, the Z-axis direction of the first marker 11 can be treated as the direction of the laser.

Claims

1. A detection method for detecting a predetermined position of an object to which a first marker is attached, comprising the steps of: aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is arranged; capturing an image of the alignment marker and the first marker arranged on the alignment member; acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object based on the captured image including the alignment marker and the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, detecting the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member from the captured image including the first marker and the second marker based on the acquired three-dimensional coordinates.

2. A detection method for detecting a direction to a predetermined position of an object to which a first marker is attached, comprising the steps of: aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is arranged; capturing an image of the alignment marker and the first marker arranged on the alignment member; acquiring information of a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object based on the captured image including the alignment marker and the first marker; and detecting a relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the separate member from the captured image including the first marker and the second marker, based on the acquired three-dimensional direction vector by capturing an image of a second marker attached to a separate member from the object and the first marker of the object.

3. A detection method for detecting a predetermined position of an object to which a first marker is attached and a direction to the predetermined position, comprising: a step of aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is arranged; a step of imaging the alignment marker and the first marker arranged on the alignment member; a step of acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object and information on a three-dimensional direction vector of the object relative to the first marker, based on the captured image including the alignment marker and the first marker; and a step of imaging a second marker attached to a member separate from the object and the first marker of the object, to detect, from the captured image including the first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the separate member, based on the acquired three-dimensional coordinates and three-dimensional direction vector.

4. A detection method according to any one of claims 1 to 3, wherein the alignment member has a flat surface, a predetermined reference point is formed on the flat surface, and the alignment marker is arranged on the flat surface.

5. A detection method for detecting a predetermined position of an object to which a first marker is attached, comprising the steps of: placing an alignment marker at a predetermined reference point of an alignment member, and capturing an image of the alignment marker with an imaging device that is position fixed; aligning the predetermined position of the object with the predetermined reference point of the alignment member, and capturing an image of the first marker with the imaging device; acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object, based on the captured image including the alignment marker and the captured image including the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member from the captured image including the first marker and the second marker, based on the acquired three-dimensional coordinates.

6. A detection method for detecting a direction to a predetermined position of an object to which a first marker is attached, comprising the steps of: placing an alignment marker at a predetermined reference point of an alignment member, and capturing an image of the alignment marker with an imaging device that is fixed in position; aligning the predetermined position of the object with the predetermined reference point of the alignment member, and capturing an image of the first marker with the imaging device; acquiring information of a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object, based on the captured image including the alignment marker and the captured image including the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, detecting a relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the separate member from the captured image including the first marker and the second marker, based on the acquired three-dimensional direction vector.

7. A detection method for detecting a predetermined position of an object to which a first marker is attached and a direction to the predetermined position, comprising the steps of: placing an alignment marker at a predetermined reference point of an alignment member, and capturing an image of the alignment marker with an imaging device that is fixed in position; aligning the predetermined position of the object with the predetermined reference point of the alignment member, and capturing an image of the first marker with the imaging device; and acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object and information on a three-dimensional direction vector of the predetermined position of the object relative to the first marker, based on the captured image including the alignment marker and the captured image including the first marker. and detecting, from an image including the captured first marker and the captured first marker, based on the acquired three-dimensional coordinates and three-dimensional direction vector, three-dimensional coordinates of the predetermined position of the object relative to the other member and a relative three-dimensional direction vector of a direction from the first marker to the predetermined position on the other member.

8. A detection method according to any one of claims 1 to 3 and 5 to 7, in which a recess is formed at the reference point of the alignment member, and the predetermined position of the object is aligned with the reference point by inserting a portion of the object at the predetermined position into the recess.

9. A detection method comprising the steps of: placing an alignment marker at a predetermined position of an object to which a first marker is attached, and capturing an image of the first marker and the alignment marker; acquiring information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object based on the captured image including the alignment marker and the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member from the captured image including the first marker and the second marker based on the acquired three-dimensional coordinates.

10. A detection method comprising the steps of: placing an alignment marker at a predetermined position of an object to which a first marker is attached, and capturing an image of the first marker and the alignment marker; acquiring information about a three-dimensional direction vector in a predetermined direction extending from the predetermined position perpendicular to the first marker based on the captured image including the alignment marker and the first marker; and capturing an image of a second marker attached to a member other than the object and the first marker of the object, and detecting a relative three-dimensional direction vector of the predetermined direction with respect to the other member from the captured image including the first marker and the second marker based on the acquired three-dimensional direction vector.

11. A detection method comprising the steps of: placing an alignment marker at a predetermined position of an object to which a first marker is attached, and capturing an image of the first marker and the alignment marker; acquiring, based on the captured image including the alignment marker and the first marker, information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object, and information on a three-dimensional direction vector in a predetermined direction extending from the predetermined position perpendicular to the first marker; and capturing an image of a second marker attached to a member separate from the object and the first marker of the object, detecting, from the captured image including the first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object with respect to the separate member and the relative three-dimensional direction vector in the predetermined direction with respect to the separate member, based on the acquired three-dimensional coordinates and three-dimensional direction vector.

12. A process of placing a first alignment marker at a first predetermined position of a first object to which a first marker is attached, and capturing an image of the first marker and the first alignment marker; acquiring information on three-dimensional coordinates of the first predetermined position of the first object relative to the position of the first marker attached to the first object based on an image including the first alignment marker and the captured image, and information on a three-dimensional direction vector in a first predetermined direction extending from the first predetermined position perpendicular to the first alignment marker; placing a second alignment marker at a second predetermined position of a second object to which a second marker is attached, and capturing an image of the second marker and the second alignment marker; a step of acquiring information on three-dimensional coordinates of the second predetermined position of the second object relative to a position of the second marker attached to the second object and information on a three-dimensional direction vector of a second predetermined direction extending from the second predetermined position perpendicular to the second alignment marker, based on an image including the captured second alignment marker and the second marker; and a step of detecting, from the image including the captured first marker and the second marker, the relative three-dimensional coordinates of the first predetermined position of the first object relative to the second predetermined position of the second object and the relative three-dimensional direction vector of the first predetermined direction of the first object relative to the second predetermined direction of the second object by capturing an image of the first marker of the first object and the second marker of the second object.

13. A detection method for detecting a predetermined position of an object to which a first marker is attached, comprising: a step of acquiring information on three-dimensional coordinates of the predetermined position of the object relative to a position of the first marker attached to the object; and a step of capturing an image of a second marker attached to a member other than the object and the first marker of the object, and detecting, based on the acquired three-dimensional coordinates, from an image including the first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member.

14. A detection method for detecting a direction to a predetermined position of an object to which a first marker is attached, comprising: a step of acquiring information of a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object; and a step of capturing an image of a second marker attached to a member other than the object and the first marker of the object, and detecting, based on the acquired three-dimensional direction vector, a relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the separate member from an image including the captured first marker and the second marker.

15. A detection method for detecting a predetermined position of an object to which a first marker is attached and a direction to the predetermined position, comprising: a step of acquiring information on three-dimensional coordinates of the predetermined position of the object relative to a position of the first marker attached to the object, and acquiring information on a three-dimensional direction vector of the object relative to the first marker; and a step of capturing an image of a second marker attached to a member separate from the object and the first marker of the object, and detecting, based on the acquired three-dimensional coordinates and three-dimensional direction vector, from an image including the captured first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position relative to the separate member.

16. A computer in which a control unit executes a program to: receive an image of an alignment marker and the first marker when a predetermined position of an object to which the alignment marker is attached is aligned with a predetermined reference point of an alignment member on which the alignment marker is disposed; obtain, based on the received image including the alignment marker and the first marker, information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object; and detect, based on the obtained three-dimensional coordinates, from an image including a second marker attached to a member separate from the object and the first marker of the object, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member.

17. A computer in which a control unit executes a program to: receive an image of an alignment marker and a first marker when a predetermined position of an object to which a first marker is attached is aligned with a predetermined reference point of an alignment member on which an alignment marker is disposed; obtain, based on the received image including the alignment marker and the first marker, information on a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object; and detect, based on the obtained three-dimensional coordinates, from an image including a second marker attached to a member separate from the object and the first marker of the object, the relative three-dimensional coordinates of the predetermined position of the object with respect to the separate member.

18. A computer in which a control unit executes a program to: receive an image of an alignment marker and the first marker when a predetermined position of an object to which a first marker is attached is aligned with a predetermined reference point of an alignment member on which an alignment marker is disposed; obtain, based on the received image including the alignment marker and the first marker, information on the relative three-dimensional coordinate of the predetermined position of the object with respect to the position of the first marker attached to the object, and information on the three-dimensional direction vector of the object with respect to the first marker; and detect, from the image including the first marker and the second marker, the relative three-dimensional coordinate of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the other member, based on the obtained three-dimensional coordinates and three-dimensional direction vector.

19. A computer in which a control unit executes a program to: accept an image of an alignment marker captured by a fixed-position imaging device and placed at a predetermined reference point of an alignment member; accept an image of a first marker attached to an object captured by the imaging device when a predetermined position of the object is aligned with the predetermined reference point of the alignment member; obtain information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object based on the received image including the alignment marker and the received image including the first marker; and detect the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member from an image including a second marker attached to a member separate from the object and the first marker of the object based on the obtained three-dimensional coordinates.

20. A computer in which a control unit executes a program to: accept an image of an alignment marker captured by a fixed-position imaging device and placed at a predetermined reference point of an alignment member; accept an image of a first marker attached to an object captured by the imaging device when a predetermined position of the object is aligned with the predetermined reference point of the alignment member; obtain information on a relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object based on the received image including the alignment marker and the received image including the first marker; and detect a relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the separate member from an image including a second marker attached to a member other than the object and the first marker attached to the object based on the obtained three-dimensional direction vector.

21. A computer in which a control unit executes a program to: accept an image of an alignment marker captured by a fixed-position imaging device and placed at a predetermined reference point of an alignment member; accept an image of a first marker attached to an object captured by the imaging device when a predetermined position of the object is aligned with the predetermined reference point of the alignment member; acquire information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object and information on a relative three-dimensional direction vector of the predetermined position of the object relative to the first marker, based on the accepted image including the alignment marker and the accepted image including the first marker; and detect, based on the acquired three-dimensional coordinates and three-dimensional direction vector, from an image including a second marker attached to a member separate from the object and the first marker attached to the object, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position relative to the separate member.

22. A computer in which, by executing a program by a control unit, the computer receives an image of a first marker and the alignment marker attached to an object when the alignment marker is placed at a predetermined position on the object, obtains information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object based on the received image including the alignment marker and the first marker, and detects the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member from an image including a second marker attached to a separate member from the object and the first marker of the object based on the obtained three-dimensional coordinates.

23. A computer in which, by executing a program by a control unit, the computer receives an image of a first marker and the alignment marker attached to an object when the alignment marker is placed at a predetermined position on the object, obtains information on a three-dimensional direction vector in a predetermined direction extending from the predetermined position perpendicular to the first marker based on the received image including the alignment marker and the first marker, and detects a relative three-dimensional direction vector in the predetermined direction with respect to a different member from the object from an image including a second marker attached to the different member and the first marker of the object based on the obtained three-dimensional direction vector.

24. A computer in which a control unit executes a program to: receive an image of a first marker and the alignment marker attached to an object when the alignment marker is placed at a predetermined position on the object; obtain, based on the received image including the alignment marker and the first marker, information on three-dimensional coordinates of the predetermined position of the object relative to the position of the first marker attached to the object and information on a three-dimensional direction vector of a predetermined direction extending from the predetermined position perpendicular to the first marker; and detect, based on the obtained three-dimensional coordinates and three-dimensional direction vector, from an image including a second marker attached to a member separate from the object and the first marker of the object, the relative three-dimensional coordinates of the predetermined position of the object relative to the separate member and the relative three-dimensional direction vector of the predetermined direction relative to the separate member.

25. A control unit executes a program to: receive an image of a first alignment marker attached to a first object when the first alignment marker is placed at a first predetermined position on the first object; and obtain, based on the received image including the first alignment marker, information on relative three-dimensional coordinates of the first predetermined position of the first object attached to the first object and information on a three-dimensional direction vector in a first predetermined direction extending from the first predetermined position perpendicular to the first alignment marker; receive an image of a second alignment marker attached to a second object when a second alignment marker is placed at a second predetermined position on the second object; and obtain, based on the received image including the second alignment marker, information on relative three-dimensional coordinates of the second predetermined position of the second object attached to the second object and information on a three-dimensional direction vector in a second predetermined direction extending from the second predetermined position perpendicular to the second alignment marker; a computer that, when receiving an image including the first alignment marker of the first object and the second alignment marker of the second object, detects, from the received image including the first alignment marker and the second alignment marker, a relative three-dimensional coordinate of the first predetermined position of the first object with respect to the second predetermined position of the second object and a relative three-dimensional direction vector of the first predetermined direction of the first object with respect to the second predetermined direction of the second object.

26. A computer in which a control unit executes a program to acquire information on three-dimensional coordinates of a predetermined position of an object relative to the position of a first marker attached to the object, and based on the acquired three-dimensional coordinates, detects the relative three-dimensional coordinates of the predetermined position of the object relative to a different member from the object from an image including the first marker of the object and a second marker attached to the different member.

27. A computer in which a control unit executes a program to acquire information on a relative three-dimensional direction vector of a predetermined position of an object relative to a first marker attached to the object, and based on the acquired three-dimensional coordinates, detects the relative three-dimensional coordinates of the predetermined position of the object relative to a second marker attached to a member other than the object from an image including the first marker of the object and the second marker attached to the member.

28. A computer in which a control unit executes a program to obtain information on three-dimensional coordinates of a predetermined position of an object relative to the position of a first marker attached to the object, and information on a three-dimensional direction vector of the object relative to the first marker, and based on the obtained three-dimensional coordinates and three-dimensional direction vector, detects, from an image including the first marker and the second marker, the relative three-dimensional coordinates of the predetermined position of the object relative to another member, and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position relative to the other member.

Images