Detection method and computer

Abstract

To provide a new detection method and a computer, for position detection from an image in the three-dimensional coordinate or the like of a predetermined position in an object.SOLUTION: The detection method includes a step for aligning the predetermined position A of an object 10 with the reference point O of an alignment member 7 in which an alignment marker 71 is arranged, a step for imaging the alignment marker 71 and a first marker 11, a step for acquiring the information of the relative three-dimensional coordinate of the predetermined position A of the object 10 with respect to the position of the first marker 11 attached to the object 10 on the basis of the captured image, and a step for imaging a second marker 14 attached to a member 13 different from the object 10 and the first marker 11 of the object 10, to detect the relative three-dimensional coordinate of the predetermined position A of the object 10 with respect to the different member 13 from the image including the captured first marker 11 and second marker 14 on the basis of the acquired three-dimensional coordinate.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to a detection method and a computer. [Background technology]

[0002] Conventionally, there is a known technique for measuring the position and orientation of an object by attaching a marker to the object and imaging the marker with a camera. Examples of markers that can be attached to an object include those disclosed in Patent Document 1, for example. A typical marker is a planar pattern that can be attached to an object. A typical marker includes a square black frame and a two-dimensional pattern code printed inside it. By imaging the marker with a camera and acquiring an image of the marker, the relative position and orientation of the marker and the camera can be recognized, and the position and orientation of the object to which the marker is attached can be recognized. Furthermore, the camera can also read the information recorded on the marker. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2012-145559 [Overview of the project] [Problems that the invention aims to solve]

[0004] There is a need for a technology that can detect the 3D coordinates of a predetermined position on an object to which a marker is attached, and the 3D directional vector in a predetermined direction related to this predetermined position, from the image of the marker, but such a technology has not been known until now.

[0005] This disclosure has been made with these points in mind and aims to provide a novel detection method and computer for detecting the three-dimensional coordinates of a predetermined position on an object to which a marker is attached, and a three-dimensional direction vector in a predetermined direction related to this predetermined position, from an image of the marker. [Means for solving the problem]

[0006] The detection method disclosed herein is: A detection method for detecting a predetermined position on an object to which a first marker is attached, A step of aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is placed, A step of imaging the alignment marker and the first marker that are arranged on the alignment member, A step of acquiring information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, based on an image including the captured alignment marker and the first marker, The method includes the step of detecting the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member from an image including the captured first and second markers, based on the acquired three-dimensional coordinates, by imaging a second marker attached to a member other than the object and the first marker of the object.

[0007] The detection method disclosed herein is: A detection method for detecting the direction toward a predetermined position of an object to which a first marker is attached, A step of aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is placed, A step of imaging the alignment marker and the first marker that are arranged on the alignment member, A step of acquiring information on the relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object, based on an image including the captured alignment marker and the first marker, The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional direction vector, detecting a relative three-dimensional direction vector from the image including the captured first marker and second marker to the other member in the direction from the first marker to the predetermined position, It is equipped with.

[0008] The detection method disclosed herein is: A detection method for detecting a predetermined position and direction to a predetermined position of an object to which a first marker is attached, A step of aligning the predetermined position of the object with a predetermined reference point of an alignment member on which an alignment marker is placed, A step of imaging the alignment marker and the first marker that are arranged on the alignment member, A step of acquiring information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, and information on the three-dimensional direction vector of the object with respect to the first marker, based on an image including the captured alignment marker and the first marker. The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional coordinates and three-dimensional direction vectors, detecting the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the other member from the image including the first marker and the second marker, It is equipped with.

[0009] In the detection method disclosed herein, The alignment member has a flat surface, and a predetermined reference point may be formed on the flat surface, and the alignment marker may be placed on it.

[0010] The detection method disclosed herein is: A detection method for detecting a predetermined position on an object to which a first marker is attached, The process involves placing an alignment marker at a predetermined reference point on the alignment member and imaging the alignment marker with a fixed-position imaging device. The steps include aligning the predetermined position of the object with the predetermined reference point of the alignment member, and imaging the first marker with the imaging device, A step of acquiring information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, based on the image including the captured alignment marker and the image including the captured first marker. The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional coordinates, detecting the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member from the image including the captured first and second markers, It is equipped with.

[0011] The detection method disclosed herein is: A detection method for detecting the direction toward a predetermined position of an object to which a first marker is attached, The process involves placing an alignment marker at a predetermined reference point on the alignment member and imaging the alignment marker with a fixed-position imaging device. The steps include aligning the predetermined position of the object with the predetermined reference point of the alignment member, and imaging the first marker with the imaging device, A step of acquiring information on the relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object, based on the image including the captured alignment marker and the image including the captured first marker, The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional direction vector, detecting a relative three-dimensional direction vector from the image including the captured first marker and second marker to the other member in the direction from the first marker to the predetermined position, It is equipped with.

[0012] The detection method disclosed herein is: A detection method for detecting a predetermined position and direction to a predetermined position of an object to which a first marker is attached, The process involves placing an alignment marker at a predetermined reference point on the alignment member and imaging the alignment marker with a fixed-position imaging device. The steps include aligning the predetermined position of the object with the predetermined reference point of the alignment member, and imaging the first marker with the imaging device, A step of acquiring information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, and information on the relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker, based on an image including the captured alignment marker and an image including the captured first marker. The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional coordinates and three-dimensional direction vectors, detecting the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the other member from the image including the first marker and the second marker, It is equipped with.

[0013] In the detection method disclosed herein, A recess is formed at the reference point of the alignment member, and the predetermined position of the object may be aligned with the reference point by placing the portion of the object at the predetermined position into the recess.

[0014] The detection method disclosed herein is: The process involves placing an alignment marker at a predetermined position on an object to which a first marker is attached, and imaging the first marker and the alignment marker. A step of acquiring information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, based on an image including the captured alignment marker and the first marker, The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional coordinates, detecting the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member from the image including the captured first and second markers, It is equipped with.

[0015] The detection method disclosed herein is: The process involves placing an alignment marker at a predetermined position on an object to which a first marker is attached, and imaging the first marker and the alignment marker. A step of acquiring information on a three-dimensional direction vector in a predetermined direction extending orthogonally from the predetermined position to the first marker, based on an image including the captured alignment marker and the first marker, The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional direction vector, detecting the relative three-dimensional direction vector in a predetermined direction with respect to the other member from the image including the captured first and second markers, It is equipped with.

[0016] The detection method disclosed herein is: The process involves placing an alignment marker at a predetermined position on an object to which a first marker is attached, and imaging the first marker and the alignment marker. A step of acquiring, based on an image including the captured alignment marker and the first marker, information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, and information on a three-dimensional direction vector in a predetermined direction extending orthogonally from the predetermined position to the first marker, The process involves imaging a second marker attached to a member separate from the object and the first marker of the object, and based on the acquired three-dimensional coordinates and three-dimensional direction vectors, detecting the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vector in the predetermined direction with respect to the other member from the image including the first and second markers. It is equipped with.

[0017] The detection method disclosed herein is: The process involves placing a first alignment marker at a first predetermined position on a first object to which a first marker is attached, and imaging the first marker and the first alignment marker. A step of acquiring information on the relative three-dimensional coordinates of a first predetermined position of a first object with respect to the position of the first marker attached to the first object, and information on a three-dimensional direction vector in a first predetermined direction extending orthogonally from the first predetermined position to the first alignment marker, based on the image of the captured first alignment marker and the image including the first marker. The process involves placing a second alignment marker at a second predetermined position on a second object to which a second marker is attached, and imaging the second marker and the second alignment marker. A step of acquiring information on the relative three-dimensional coordinates of the second predetermined position of the second object with respect to the position of the second marker attached to the second object, and information on a three-dimensional direction vector in a second predetermined direction extending orthogonally from the second predetermined position to the second alignment marker, based on the captured second alignment marker and the image including the second marker. The process involves imaging the first marker of the first object and the second marker of the second object, thereby detecting from the image including the first and second markers the relative three-dimensional coordinates of the first predetermined position of the first object with respect to the second predetermined position of the second object, and the relative three-dimensional direction vector of the first predetermined direction of the first object with respect to the second predetermined direction of the second object. It is equipped with.

[0018] The detection method disclosed herein is: A detection method for detecting a predetermined position on an object to which a first marker is attached, A step of acquiring information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional coordinates, detecting the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member from the image including the captured first and second markers, It is equipped with.

[0019] The detection method disclosed herein is: A detection method for detecting the direction toward a predetermined position of an object to which a first marker is attached, A step of acquiring information on the relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object, The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional direction vector, detecting a relative three-dimensional direction vector from the image including the captured first marker and second marker to the other member in the direction from the first marker to the predetermined position, It is equipped with.

[0020] The detection method disclosed herein is: A detection method for detecting a predetermined position and direction to a predetermined position of an object to which a first marker is attached, A step of acquiring information on the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, and acquiring information on the three-dimensional direction vector of the object with respect to the first marker, The process involves imaging a second marker attached to a member other than the object and the first marker of the object, and based on the acquired three-dimensional coordinates and three-dimensional direction vectors, detecting the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position with respect to the other member from the image including the first marker and the second marker, It is equipped with.

[0021] The computer disclosed herein is The control unit executes the program, Images of the alignment marker and the first marker are received when the predetermined position of the object to which the first marker is attached is aligned with a predetermined reference point of the alignment member on which the alignment marker is located. Based on the received image including the alignment marker and the first marker, information is obtained regarding the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object. Based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member are detected from an image including a second marker attached to a member other than the object and the first marker of the object.

[0022] The computer disclosed herein is The control unit executes the program, Images of the alignment marker and the first marker are received when the predetermined position of the object to which the first marker is attached is aligned with a predetermined reference point of the alignment member on which the alignment marker is located. Based on the received image including the alignment marker and the first marker, information on the relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object is obtained. Based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member are detected from an image including a second marker attached to a member other than the object and the first marker of the object.

[0023] The computer disclosed herein is The control unit executes the program, Images of the alignment marker and the first marker are received when the predetermined position of the object to which the first marker is attached is aligned with a predetermined reference point of the alignment member on which the alignment marker is located. Based on the received image including the alignment marker and the first marker, information is obtained regarding the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, and information is obtained regarding the three-dimensional direction vector of the object with respect to the first marker. Based on the acquired three-dimensional coordinates and three-dimensional direction vectors, the relative three-dimensional coordinates of the predetermined position of the object relative to the other member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position relative to the other member are detected from the image including the first marker and the second marker.

[0024] The computer disclosed herein is The control unit executes the program, The system receives images of alignment markers captured by a fixed-position imaging device positioned at a predetermined reference point on the alignment member. When the predetermined position of the object is aligned with the predetermined reference point of the alignment member, an image of the first marker attached to the object and captured by the imaging device is received. Based on the received image including the alignment marker and the received image including the first marker, information is obtained regarding the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object. Based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member are detected from an image including a second marker attached to a member other than the object and the first marker of the object.

[0025] The computer disclosed herein is The control unit executes the program, The system receives images of alignment markers captured by a fixed-position imaging device positioned at a predetermined reference point on the alignment member. When the predetermined position of the object is aligned with the predetermined reference point of the alignment member, an image of the first marker attached to the object and captured by the imaging device is received. Based on the received image including the alignment marker and the received image including the first marker, information on the relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker attached to the object is obtained. Based on the acquired three-dimensional direction vector, a relative three-dimensional direction vector from the first marker to the predetermined position relative to the other member is detected from an image including a second marker attached to a member other than the object and the first marker attached to the object.

[0026] The computer disclosed herein is The control unit executes the program, The system receives images of alignment markers captured by a fixed-position imaging device positioned at a predetermined reference point on the alignment member. When the predetermined position of the object is aligned with the predetermined reference point of the alignment member, an image of the first marker attached to the object and captured by the imaging device is received. Based on the received image including the alignment marker and the received image including the first marker, information is obtained regarding the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, and information regarding the relative three-dimensional direction vector of the predetermined position of the object with respect to the first marker. Based on the acquired three-dimensional coordinates and three-dimensional direction vectors, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vectors of the direction from the first marker to the predetermined position with respect to the other member are detected from an image including a second marker attached to a member other than the object and the first marker attached to the object.

[0027] The computer disclosed herein is The control unit executes the program, When an alignment marker is placed at a predetermined position on the object, images of the first marker attached to the object and the alignment marker are received. Based on the received image including the alignment marker and the first marker, information is obtained regarding the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object. Based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member are detected from an image including a second marker attached to a member other than the object and the first marker of the object.

[0028] The computer disclosed herein is The control unit executes the program, When an alignment marker is placed at a predetermined position on the object, images of the first marker attached to the object and the alignment marker are received. Based on the received image including the alignment marker and the first marker, information on a three-dimensional direction vector in a predetermined direction extending orthogonally from the predetermined position to the first marker is obtained. Based on the acquired three-dimensional direction vector, the relative three-dimensional direction vector in the predetermined direction to the other member is detected from an image including a second marker attached to a member other than the object and the first marker on the object.

[0029] The computer disclosed herein is The control unit executes the program, When an alignment marker is placed at a predetermined position on the object, images of the first marker attached to the object and the alignment marker are received. Based on the received image including the alignment marker and the first marker, information is obtained regarding the relative three-dimensional coordinates of the predetermined position of the object with respect to the position of the first marker attached to the object, and information regarding the three-dimensional direction vector in a predetermined direction extending orthogonally from the predetermined position to the first marker. Based on the acquired three-dimensional coordinates and three-dimensional direction vectors, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member and the relative three-dimensional direction vector in the predetermined direction with respect to the other member are detected from an image including a second marker attached to a member other than the object and the first marker of the object.

[0030] The computer disclosed herein is The control unit executes the program, When the first alignment marker is placed at a first predetermined position on the first object, an image of the first alignment marker attached to the first object is received. Based on the received image including the first alignment marker, information is obtained regarding the relative three-dimensional coordinates of the first predetermined position of the first object attached to the first object, and information regarding the three-dimensional direction vector in the first predetermined direction extending orthogonally from the first predetermined position to the first alignment marker. When the second alignment marker is placed at a second predetermined position on the second object, an image of the second alignment marker attached to the second object is received. Based on the received image including the second alignment marker, information is obtained regarding the relative three-dimensional coordinates of the second predetermined position of the second object attached to the second object, and information regarding the three-dimensional direction vector of the second predetermined direction extending orthogonally from the second predetermined position to the second alignment marker. When an image is received that includes the first alignment marker of the first object and the second alignment marker of the second object, the system detects from the received image containing the first and second alignment markers the relative three-dimensional coordinates of the first predetermined position of the first object with respect to the second predetermined position of the second object, and the relative three-dimensional direction vector of the first predetermined direction of the first object with respect to the second predetermined direction of the second object.

[0031] The computer disclosed herein is The control unit executes the program, The system acquires information on the relative three-dimensional coordinates of a predetermined position on an object with respect to the position of a first marker attached to the object. Based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member are detected from an image including a second marker attached to a member other than the object and the first marker of the object.

[0032] The computer disclosed herein is The control unit executes the program, Information on the relative three-dimensional direction vector of a predetermined position of an object with respect to a first marker attached to the object is obtained. Based on the acquired three-dimensional coordinates, the relative three-dimensional coordinates of the predetermined position of the object with respect to the other member are detected from an image including a second marker attached to a member other than the object and the first marker of the object.

[0033] The computer disclosed herein is The control unit executes the program, The system acquires information on the relative three-dimensional coordinates of a predetermined position on an object with respect to the position of a first marker attached to the object, and also acquires information on the three-dimensional direction vector of the object with respect to the first marker. Based on the acquired three-dimensional coordinates and three-dimensional direction vectors, the relative three-dimensional coordinates of the predetermined position of the object relative to the other member and the relative three-dimensional direction vector of the direction from the first marker to the predetermined position relative to the other member are detected from the image including the first marker and the second marker. [Effects of the Invention]

[0034] The detection method and computer of this disclosure provide a novel detection method and computer that can detect the three-dimensional coordinates of a predetermined position on an object to which a marker is attached, and the three-dimensional direction vector in a predetermined direction related to this predetermined position, from an image of the marker. [Brief explanation of the drawing]

[0035] [Figure 1] This is a schematic diagram showing the configuration of a location detection system according to an embodiment of the present disclosure. [Figure 2] This is a schematic diagram illustrating a calibration method for a position detection system according to an embodiment of the present disclosure. [Figure 3] This is a schematic diagram illustrating one use case of a location detection system according to an embodiment of the present disclosure, which is a schematic diagram for detecting parts that are not visible from the outside, such as inside the body. [Figure 4] This flowchart shows the flow of the calibration process in the position detection system according to the embodiment of the present disclosure. [Figure 5] This is a flowchart showing the flow of the detection process in the location detection system according to the embodiment of the present disclosure. [Figure 6] This is a schematic diagram showing a modified version of the position detection system according to the embodiment of the present disclosure, in which an alignment marker is placed at a predetermined position on an object. [Figure 7] This is a schematic diagram showing a modified example of the location detection system according to the embodiment of the present disclosure, illustrating the configuration of the location detection system when there are multiple objects. [Figure 8] This is a schematic diagram showing an example of how the location detection system according to the embodiments of this disclosure is used. [Figure 9] This is a schematic diagram showing an example of how the location detection system according to the embodiments of this disclosure is used. [Figure 10] This is a schematic diagram illustrating an example of how the position detection system according to the embodiments of this disclosure is used, specifically a configuration in which it is drawn in three-dimensional space. [Figure 11] This is a schematic diagram illustrating an example of how the position detection system according to the embodiments of this disclosure can be used, specifically in which an object is used as a laser pointer. [Modes for carrying out the invention]

[0036] Embodiments of this disclosure will be described below with reference to the drawings. Figures 1 to 5 are diagrams showing a part of the position detection system according to the embodiments of this disclosure. Of these, Figure 1 is a schematic diagram showing the configuration of the position detection system according to this embodiment, and Figure 2 is a schematic diagram showing the calibration method of the position detection system according to this embodiment. Figure 3 is a schematic diagram showing one usage form of the position detection system according to this embodiment, which is a schematic diagram for detecting parts that cannot be seen from the outside, such as inside the body. Figure 4 is a flowchart showing the flow of the calibration process in the position detection system according to this embodiment. Figure 5 is a flowchart showing the flow of the detection process in the position detection system according to this embodiment.

[0037] —Location detection system 1— Figure 1 is a block diagram showing the configuration of the position detection system 1 according to this embodiment. The position detection system 1 according to this embodiment comprises a computer 2 and an imaging device 3. As shown in Figures 2 and 3, the imaging device 3 is used to position the object to be detected (object 10) and photographs an alignment marker 71 attached to an alignment member 7, a first marker 11 attached to an object of a predetermined length, and a second marker 14 attached to another member 13 (for example, the human body). The position detection system 1 also acquires at least one of the information on the relative three-dimensional coordinates of the first marker 11 with respect to the alignment marker 71 and the information on the three-dimensional direction vector based on the image including the captured first marker 11 and the alignment marker 71, and measures the three-dimensional coordinates and three-dimensional direction vector of a predetermined position (point A) with respect to the first marker 11. For example, the computer 2 is connected to the imaging device 3 in a communicative manner. The imaging device 3 and the computer 2 may also be connected to each other in a communicative manner via a communication network such as an internet line. Each component of this position detection system 1 is described below.

[0038] The alignment member 7 is a member used for calibration to define a predetermined position (A) of the object 10. The alignment member 7 is treated as a plate-shaped member or a virtual plane, and an alignment marker 71 is placed on it. Here, the predetermined position (A) means a specific location on the object 10, and for example, if it is a rod-shaped member, it may be the tip of the rod-shaped member. In this disclosure, the tip of a member having a longitudinal axis will be used as an example.

[0039] The alignment marker 71 is positioned on the alignment member 7 and is an AR marker used for calibration together with the first marker 11. The alignment marker 71 is positioned in the virtual plane at a location that does not overlap with the location defined as the origin (O).

[0040] The first marker 11 is an AR marker attached to an object (object 10) having a predetermined length, and is used to determine the positional relationship of a predetermined position (A) on the object 10. Furthermore, the size of the first marker 11 is such that it does not overlap with the alignment marker 71 when calibration is performed.

[0041] The above function is achieved by photographing the alignment marker 71 and the first marker 11 with the imaging device 3.

[0042] <Configuration of Computer 2> Computer 2 can be accessed, for example, through a communication network such as the Internet. Computer 2 has a control unit 4, a storage unit 5, and a communication unit 6. The control unit 4 is composed of a microcomputer including a CPU and semiconductor memory, and controls the operation of computer 2. More specifically, the control unit 4 functions as a receiving means 41, a calculation means 42, and a detection means 43 by executing a program stored in the storage unit 5.

[0043] The reception means 41 receives the images captured by the imaging device 3. For example, the reception means 41 receives the image data of the alignment marker 71, the first marker 11, and the second marker 14 captured by the imaging device 3.

[0044] The calculation means 42 acquires information on the three-dimensional coordinates of the relative position of the predetermined position (A) of the object 10 with respect to the attachment position of the first marker 11 attached to the object 10 based on the image data including at least the alignment marker 71 and the first marker 11. First, the calculation means 42, based on the matrices at the respective positions of the imaging device 3 (point C) and the alignment marker 71 (m1) in the image data captured by the imaging device 3, obtains the homogeneous transformation matrix H m1 and obtains the homogeneous transformation matrix O H m1 -1 based on the respective positions of the alignment marker 71 (m1) and the origin (O). At this time, the predetermined position (A) of the object 10 and the origin (O) in the m2 coordinate system coincide. Then, based on the obtained c H m1 and O H m1 -1 , c H O ( c H m1O H m1 -1 ) is obtained. The homogeneous transformation matrix is for efficiently calculating coordinate transformation and rigid body transformation by matrix calculation, and in three-dimensional space, it is represented by the following formula.

[0045]

Equation

[0046] Also, the calculation means 42, based on the matrices at the respective positions of the imaging device 3 (point C) and the first marker 11 (point m2), obtains the homogeneous transformation matrix c H m2 -1The calculation means 42 then calculates the c H O and c H m2 -1 Based on, m2 H O ( c H m2 -1 c H O We find the following equation. In this state, the slopes of the first marker 11 (point m2) and the origin (O) are reflected, so for example, if we set the rotation matrix (R) to no rotation and let the resulting translation vectors be tx, ty, and tz, we obtain the following equation.

[0047]

number

[0048] In this way, the object 10 is calibrated when its predetermined position (A) coincides with the predetermined reference point (origin (O)) of the alignment member 7.

[0049] Furthermore, the calculation means 42 obtains a homogeneous transformation matrix based on the matrices at the respective positions of the imaging device 3 (point C) and the first marker 11 (m2). c H m2 ,and, m2 H A Based on, c H A ( c H m2m2 H A The calculation means 42 sets a predetermined position (A) of the object 10 in a coordinate system centered on m2, performs operations on the homogeneous transformation matrix including the rotation matrix, and extracts the translation (position) component to determine the predetermined position (A) of the object 10 in the m2 coordinate system.

[0050] The detection means 43, by imaging a second marker 14 attached to a member 13 separate from the object 10 and a first marker 11 on the object 10, detects at least one of the acquired three-dimensional coordinates and three-dimensional direction vectors from the image including the captured first marker 11 and second marker 14, the relative three-dimensional coordinates and three-dimensional direction vectors of a predetermined position (A) of the object 10 with respect to the other member 13.

[0051] Another component 13 is a component to which the second marker 14 is attached, and is an object to which the placement of the object 10 at a predetermined position (A) is determined from an image including the first marker 11 and the second marker 14. There are no particular limitations on what can be used as the other component 13; for example, if an endoscope is used as the object 10, the patient's body may be used as the other component 13.

[0052] The second marker 14 is attached to another component 13 and is an AR marker used in cooperation with the first marker 11 to determine the positional relationship between a predetermined position (A) of the object 10 and the other component 13.

[0053] Furthermore, the programs executed by the control unit 4 are not limited to those stored in the storage unit 5. Programs executed by the control unit 4 may include those transmitted to the control unit 4 from an external device via the communication unit 6, those stored on a storage medium such as a USB memory attached to the computer 2, or those stored on a server other than the computer 2 (for example, a cloud server).

[0054] The storage unit 5 consists of an HDD (Hard Disk Drive), RAM (Random Access Memory), ROM (Read Only Memory), or SSD (Solid State Drive), etc. The storage unit 5 stores various programs, including programs executed by the control unit 4. Furthermore, the storage unit 5 stores images captured by the imaging device 3. Details of the information stored in the storage unit 5 will be described later.

[0055] The communication unit 6 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc., and includes an interface for wired or wireless communication. In other words, the communication unit 6 consists of a communication interface configured to communicate with other devices via a communication network.

[0056] The imaging device 3 is a camera device provided in one or more locations within the position detection system 1. The imaging device 3 is grounded or suspended and is positioned to capture the alignment marker 71, the first marker 11, the second marker 14, the reference point (origin (O)), and the target object 10. It photographs these objects and acquires image data.

[0057] <Calibration process> First, the calibration process in this embodiment will be explained using the schematic diagram shown in Figure 2 and the flowchart shown in Figure 4. The operation of the position detection system 1 described below is performed by the control unit 4 in the computer 2 executing a program stored in the storage unit 5.

[0058] First, the user places the alignment marker 71 on the alignment member 7.

[0059] Next, the user aligns the predetermined position (A) of the object 10 to which the first marker 11 is attached with the reference point (origin (O)) of the alignment member 7 where the alignment marker 71 is located (step S10). At this time, the imaging device 3 is positioned so that the alignment marker 71 and the first marker 11 can be imaged.

[0060] Next, the imaging device 3 images the alignment marker 71 and the first marker 11 (step S11). As a result, the receiving means 41 receives and understands information such as the position and arrangement relationship of the image information defining the alignment member (virtual plane) 7 registered in the alignment marker 71 and the image information defining the object 10 registered in the first marker 11.

[0061] Next, the calculation means 42 acquires information on the relative three-dimensional coordinates of a predetermined position (A) of the object 10 with respect to the mounting position of the first marker 11, based on the images of the alignment marker 71 and the first marker 11 received by the receiving means 41 (step S12). This allows the predetermined position (A) of the object 10 to be calibrated to coincide with a predetermined reference point (origin (O)) of the alignment member 7.

[0062] <Detection process> Next, the detection process after calibration in this embodiment will be explained using the schematic diagram shown in Figure 3 and the flowchart shown in Figure 5.

[0063] First, the imaging device 3 images a second marker 14 attached to another member 13 and a first marker 11 on the object 10 on a virtual plane defined by the alignment marker 71 (step S13). As a result, information such as the position and arrangement relationship of the image information defining the other member 13 registered in the second marker 14 and the image information defining the object 10 registered in the first marker 11 is received and grasped by the receiving means 41.

[0064] Next, based on the three-dimensional coordinates acquired in step S12 and the image information captured in step S13 and received by the receiving means 41, the detection means 43 detects the three-dimensional coordinates of a predetermined position (A) of the object 10 relative to another member 13 (step S14). This allows for the detection of the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to another member 13 from the captured image including the first marker 11 and the second marker 14, enabling continuous position detection.

[0065] According to the detection method of this embodiment, which has the configuration described above, the detection method of this disclosure detects a predetermined position (A) of an object 10 to which the first marker 11 is attached, by aligning the predetermined position (A) of the object 10 with a predetermined reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is located. Next, the alignment marker 71 and the first marker 11 located on the alignment member 7 are imaged. Next, based on the image including the imaged alignment marker 71 and the first marker 11, information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10 is obtained. Next, by imaging the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 on the object 10, the relative three-dimensional coordinates of a predetermined position (A) of the object 10 with respect to the other member 13 are detected from the image including the imaged first marker 11 and second marker 14, based on the acquired three-dimensional coordinates.

[0066] This detection method allows for the accurate determination of the predetermined position (A) of the object 10 relative to another component 13, even when the predetermined position (A) of the object 10 cannot be visually identified. More specifically, there has been a need for a technology to detect the three-dimensional coordinates of a predetermined position (A) on the object 10 to which a marker is attached, which is difficult to image using a camera, from the image of the marker. For example, even when trying to determine the position of the tip of an endoscope inside the body, it is difficult to visually identify the tip from the outside, and it is also difficult to accurately determine where the tip is located inside the body. And until now, no technology has been known to solve these problems. With the detection method with the above configuration, since the three-dimensional coordinates of the predetermined position (A) on the object 10 to which the marker is attached can be detected from the image of the marker, these problems can be avoided.

[0067] Furthermore, in the detection method of this embodiment, as described above, the alignment member 7 has a plane, and a predetermined reference point (origin (O)) may be formed on the plane, and an alignment marker 71 may be placed thereon. In this case, calibration can be performed efficiently.

[0068] Furthermore, in the detection method and computer 2 of this embodiment, as described above, the computer 2 of this disclosure receives an image of the alignment marker 71 and the first marker 11 when the predetermined position (A) of the object 10 to which the first marker 11 is attached is aligned with a predetermined reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is located, by the control unit 4 executing a program. Next, based on the received image including the alignment marker 71 and the first marker 11, information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10 is obtained. Next, based on the obtained three-dimensional coordinates, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to another member 13 are detected from an image including the second marker 14 attached to another member 13 and the first marker 11 of the object 10. In this case, the computer 2 can detect the three-dimensional coordinates of a predetermined position (A) on the object 10 to which the marker is attached from the image of the marker.

[0069] Furthermore, the detection method and computer 2 relating to this disclosure are not limited to the embodiments described above, and can be modified in various ways.

[0070] In this embodiment, an example of acquiring information on the relative three-dimensional coordinates of a predetermined position (A) of the object 10 has been described, but the embodiment is not limited to this. For example, instead of three-dimensional coordinates, information on a relative three-dimensional direction vector from the first marker 11 to the predetermined position on another member 13 may be acquired.

[0071] Furthermore, information on the relative three-dimensional coordinates of a predetermined position (A) of the object 10 may be acquired, as well as information on the three-dimensional direction vector of the predetermined position (A) relative to the first marker 11. Alternatively, by imaging a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 relative to the other member 13 and the relative three-dimensional direction vector from the first marker 11 to the predetermined position (A) relative to the other member 13 may be detected from the image including the first marker 11 and the second marker 14, based on the acquired three-dimensional coordinates and three-dimensional direction vector. In this case, by acquiring the three-dimensional direction vector in addition to the three-dimensional coordinates, the predetermined position (A) on the object 10 can be identified with higher accuracy.

[0072] Furthermore, although this embodiment describes an example in which an alignment marker 71 placed on the alignment member 7 and a first marker 11 attached to the object 10 are simultaneously imaged, the invention is not limited to this. For example, in the detection method of this disclosure, an alignment marker 71 is placed at a predetermined reference point (origin (O)) of the alignment member 7, and the alignment marker 71 is imaged by a fixed-position imaging device 3. Next, the alignment marker 71 is removed, a predetermined position (A) of the object 10 is aligned with a predetermined reference point (origin (O)) of the alignment member 7, and the first marker 11 is imaged by the imaging device 3. Next, based on the imaged image including the alignment marker 71 and the imaged image including the first marker 11, information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10 is obtained. In this case, after imaging the alignment marker 71 attached to a predetermined reference point (origin (O)), the first marker 11 is imaged with the predetermined position (A) of the object 10 aligned to the predetermined reference point (origin (O)). This allows calibration even with a narrow field of view based on three-dimensional coordinates. Next, by imaging the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the other member 13 are detected from the image including the imaged first marker 11 and second marker 14, based on the acquired three-dimensional coordinates. This enables detection processing.

[0073] Furthermore, in other detection methods of this disclosure, an alignment marker 71 is placed at a predetermined reference point (origin (O)) of the alignment member 7, and the alignment marker 71 is imaged by a fixed imaging device 3. Next, the alignment marker 71 is removed, the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7, and the first marker 11 is imaged by the imaging device 3. Next, based on the imaged image including the alignment marker 71 and the imaged image including the first marker 11, information on the relative three-dimensional direction vector of the predetermined position (A) of the object 10 with respect to the first marker 11 attached to the object 10 is obtained. In this case, after imaging the alignment marker 71 attached to a predetermined reference point (origin (O)), the first marker 11 is imaged with the predetermined position (A) of the object 10 aligned to the predetermined reference point (origin (O)), and calibration is performed based on the three-dimensional direction vector. This allows for calibration with a narrow field of view and focused on the three-dimensional direction. Next, by imaging the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative three-dimensional direction vector from the first marker 11 to the predetermined position (A) relative to the other member 13 is detected from the image including the imaged first marker 11 and second marker 14, based on the acquired three-dimensional direction vector. This enables detection processing.

[0074] Furthermore, in yet another detection method of this disclosure, an alignment marker 71 is placed at a predetermined reference point (origin (O)) of the alignment member 7, and the alignment marker 71 is imaged by a fixed imaging device 3. Next, the alignment marker 71 is removed, the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7, and the first marker 11 is imaged by the imaging device 3. Next, based on the imaged image including the alignment marker 71 and the imaged image including the first marker 11, information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10, and information on the relative three-dimensional direction vector of the predetermined position (A) of the object 10 with respect to the first marker 11 are obtained. In this case, after imaging the alignment marker 71 attached to a predetermined reference point (origin (O)), the first marker 11 is imaged with the predetermined position (A) of the object 10 aligned to the predetermined reference point (origin (O)), and calibration is performed based on 3D coordinates and 3D direction vectors. This allows calibration to be performed with a narrow field of view and by combining 3D coordinates and 3D direction. Next, by imaging the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative 3D coordinates of the predetermined position (A) of the object 10 with respect to the other member 13 and the relative 3D direction vector from the first marker 11 to the predetermined position (A) with respect to the other member 13 are detected from the image including the imaged first marker 11 and second marker 14, based on the acquired 3D coordinates and 3D direction vectors. This enables detection processing.

[0075] Furthermore, in this embodiment, a recess is formed at the reference point (origin (O)) of the alignment member 7, and the predetermined position (A) of the object 10 may be aligned with the reference point (origin (O)) by placing the portion of the object 10 at the predetermined position (A) into the recess. In this case, the predetermined position (A) of the object 10 can be aligned with the reference point (origin (O)) by placing the portion of the object 10 at the predetermined position (A) into the recess.

[0076] Next, Figure 6 will be described. In the detection method of this disclosure, a first marker 11 is placed at a predetermined position (B) on the object 101, and an alignment marker 12 is placed at the position of point m2. Next, the first marker 11 and the alignment marker 12 are imaged. Then, based on the imaged image including the alignment marker 12 and the first marker 11, information on the relative three-dimensional coordinates of the predetermined position (point m2) of the alignment marker 12 with respect to the first marker 11 (point m1) placed on the object 101 is obtained, thereby enabling calibration even if the alignment marker 12 is removed from the object 101. Next, by imaged a second marker 14 attached to a member 13 separate from the object 101 and the first marker 11 on the object 101, the relative three-dimensional coordinates of point m2 with respect to the other member 13 are detected from the imaged image including the second marker 14 and the first marker 11, based on the acquired three-dimensional coordinates. At this time, as shown in Figure 6, the calculation means 42 calculates a homogeneous transformation matrix from the image data captured by the imaging device 3, based on the matrices at the respective positions of the imaging device 3 (point C) and the reference point (point m2). c H m2 -1 We will determine the homogeneous transformation matrix based on the matrices at the respective positions of the imaging device 3 (point C) and the first marker 11 (point m1). c H m1 We seek... And we find... c H m1 and c H m2 -1 Based on, m1 H m2 ( c H m2 -1 c H m1 ) is determined. In this case, the m1 coordinate system and the m2 coordinate system are treated as the same coordinate system, and calibration is performed based on 3D coordinates. Therefore, it is possible to determine the position and orientation of the m2 coordinate system in the m1 coordinate system using a narrow field of view and 3D coordinates and perform calibration. For example, when assembling parts using a crane, it is possible to estimate the position where the parts are connected. Note that in the detection process, c Hm2 ( c H m1m2 H m1 -1 )

[0077] Furthermore, the detection method described herein includes: The first marker 11 is placed at a predetermined position (B) on the object 101, and the alignment marker 12 is placed at point m2. Next, The first marker 11 and the alignment marker 12 are imaged. Next, based on the image including the imaged alignment marker 12 and the first marker 11, information on a three-dimensional direction vector in a predetermined direction extending orthogonally from a predetermined position (B) to the first marker 11 is obtained, and calibration is performed based on the three-dimensional direction vector. Next, the second marker 14 attached to a member 13 separate from the object 101 and the first marker 11 of the object 101 are imaged, and based on the obtained three-dimensional direction vector, the relative three-dimensional direction vector in a predetermined direction to the other member 13 is detected from the image including the imaged first marker 11 and the second marker 14.

[0078] Furthermore, the detection method described herein includes: The first marker 11 is placed at a predetermined position (B) on the object 101, and the alignment marker 12 is placed at point m2. Next, The first marker 11 and the alignment marker 12 are imaged. Next, based on the image including the imaged alignment marker 12 and the first marker 11, information on the relative three-dimensional coordinates of a predetermined position (B) of the object 101 with respect to the position of the first marker 11 attached to the object 101, and information on a three-dimensional direction vector in a predetermined direction extending orthogonally from the predetermined position (B) to the first marker 11 are obtained, thereby performing calibration based on the three-dimensional coordinates and the three-dimensional direction vector. Next, by imaged the second marker 14 attached to a member 13 separate from the object 101 and the first marker 11 of the object 101, the relative three-dimensional coordinates of the predetermined position (B) of the object 101 with respect to the other member 13 and the relative three-dimensional direction vector in a predetermined direction with respect to the other member 13 are detected from the image including the imaged first marker 11 and the second marker 14, based on the obtained three-dimensional coordinates and three-dimensional direction vector.

[0079] Next, Figure 7 will be described. In the detection method of this disclosure, a first alignment marker 15 is placed at a first predetermined position (point A1) of the first object 111 to which the first marker 11 is attached, and images of the first marker 11 and the first alignment marker 15 are captured. At this time, the first alignment marker 15 is positioned to match, for example, point A1 in Figure 7. Based on the captured image including the first alignment marker 15 and the first marker 11, information on the relative three-dimensional coordinates of the first predetermined position (point A1) of the first object 111 with respect to the position of the first marker 11 attached to the first object 111, and information on the three-dimensional direction vector in a first predetermined direction extending orthogonally from the first predetermined position (point A1) to the first alignment marker 15 are obtained. Next, a second alignment marker 16 is placed at a second predetermined position (point A2) on the second object 112 to which the second marker 14 is attached, and images of the second marker 14 and the second alignment marker 16 are captured. At this time, the second alignment marker 16 is positioned to match point A2 in Figure 7, for example. Next, based on the captured images including the second alignment marker 16 and the second marker 14, information on the relative three-dimensional coordinates of the second predetermined position (point A2) of the second object 112 with respect to the position of the second marker 14 attached to the second object 112, and information on the three-dimensional direction vector of the second predetermined direction extending orthogonally from the second predetermined position (point A2) to the second alignment marker 16 are obtained. Next, by imaging the first marker 11 of the first object 111 and the second marker 14 of the second object 112, the relative three-dimensional coordinates of the first predetermined position (point A1) of the first object 111 with respect to the second predetermined position (point A2) of the second object 112, and the relative three-dimensional direction vector of the first predetermined direction of the first object 111 with respect to the second predetermined direction of the second object 112 are detected from the image including the captured first marker 11 and second marker 14. In this case, even if the first object 111 and the second object 112 overlap and there is a blind spot when the imaging device 3 takes pictures, the positional difference at the alignment point can be measured and the alignment of points A1 and A2 can be performed.Furthermore, when calculating the difference in inclination between the first object 111 and the second object 112, the calculation means 42 can express the result from the image data captured by the imaging device 3, for example, as the dot product of the points C to A1 (C-A1) and C to A' (C-A'). The difference between points A1 and A' is then calculated with respect to point A1. A1 H A '= c H A1 -1 c H A It can be expressed as '. In this case, the difference in inclination at the alignment point can be measured, and the holes formed in the first object 111 and the second object 112 can be aligned. In this way, it becomes possible to smoothly insert a rod-shaped member into the connecting hole formed by connecting the respective holes.

[0080] Furthermore, although the calibration process has been described in detail in this embodiment, in the detection method of this embodiment, information on the 3D coordinates and 3D direction vectors of a predetermined position (A) of the object 10 may be acquired in advance. For example, the detection method of this embodiment acquires information on the relative 3D coordinates of the predetermined position (A) of the object 10 with respect to the position of a first marker 11 attached to the object 10. Alternatively, by imaging a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative 3D coordinates of the predetermined position (A) of the object 10 with respect to the other member 13 may be detected based on the acquired 3D coordinates from an image including the captured first marker 11 and second marker 14. In this case, information on the 3D coordinates of the predetermined position (A) of the object 10 that has been acquired in advance can be used.

[0081] Furthermore, for example, the detection method in this embodiment acquires information on the relative three-dimensional direction vector of a predetermined position (A) of the object 10 with respect to a first marker 11 attached to the object 10. Alternatively, by imaging a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative three-dimensional direction vector from the first marker 11 to the predetermined position (A) with respect to the other member 13 may be detected based on the acquired three-dimensional direction vector from the image including the captured first marker 11 and second marker 14. In this case, information on the three-dimensional direction vector of the predetermined position (A) on the object 10, which has been acquired in advance, can be used.

[0082] Furthermore, for example, the detection method in this embodiment acquires information on the relative three-dimensional coordinates of a predetermined position (A) of the object 10 with respect to the position of a first marker 11 attached to the object 10, and also acquires information on the three-dimensional direction vector of the object 10 with respect to the first marker 11. Alternatively, by imaging a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10, the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the other member 13 and the relative three-dimensional direction vector from the first marker 11 to the predetermined position (A) with respect to the other member 13 may be detected based on the acquired three-dimensional coordinates and three-dimensional direction vector. In this case, information on the three-dimensional coordinates and three-dimensional direction vector of the predetermined position (A) on the object 10 that has been acquired in advance can be used.

[0083] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the alignment marker 71 and the first marker 11 when the predetermined position (A) of the object 10 to which the first marker 11 is attached is aligned with a predetermined reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is located. Based on the received image including the alignment marker 71 and the first marker 11, the computer 2 acquires information on the relative three-dimensional direction vector of the predetermined position (A) of the object 10 with respect to the first marker 11 attached to the object 10. In this case, the computer 2 can perform calibration using a three-dimensional direction vector instead of three-dimensional coordinates. Furthermore, based on the acquired three-dimensional coordinates, the computer 2 detects the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to another member 13 from an image including the second marker 14 attached to another member 13 and the first marker 11 of the object 10.

[0084] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the alignment marker 71 and the first marker 11 when the predetermined position (A) of the object 10 to which the first marker 11 is attached is aligned with a predetermined reference point (origin (O)) of the alignment member 7 on which the alignment marker 71 is located. Based on the received image including the alignment marker 71 and the first marker 11, the computer 2 obtains information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10, as well as information on the three-dimensional direction vector of the object 10 with respect to the first marker 11. In this case, the computer 2 can determine the predetermined position (A) of the object 10 with higher accuracy by calculating including the three-dimensional direction vector in addition to the three-dimensional coordinates. Furthermore, based on the acquired 3D coordinates and 3D direction vectors, the relative 3D coordinates of a predetermined position (A) of the object 10 with respect to another member 13, and the relative 3D direction vector from the first marker 11 to the predetermined position (A) with respect to the other member 13 are detected from the image including the first marker 11 and the second marker 14.

[0085] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the alignment marker 71 captured by the fixed-position imaging device 3, which is positioned at a predetermined reference point (origin (O)) of the alignment member 7. It also receives an image of the first marker 11, which is attached to the object 10 and captured by the imaging device 3, when the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7. Based on the received image including the alignment marker 71 and the received image including the first marker 11, the computer 2 acquires information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10. In this case, the computer 2 captures the alignment marker 71 attached to the predetermined reference point (origin (O)) and then captures the first marker 11 when the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)), so that calibration can be performed even with a narrow field of view based on three-dimensional coordinates. Furthermore, based on the acquired 3D coordinates, the relative 3D coordinates of a predetermined position (A) of the object 10 with respect to another member 13 are detected from an image that includes a second marker 14 attached to a member 13 separate from the object 10 and a first marker 11 on the object 10.

[0086] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the alignment marker 71 captured by the fixed-position imaging device 3, which is positioned at a predetermined reference point (origin (O)) of the alignment member 7. It also receives an image of the first marker 11, which is attached to the object 10 and captured by the imaging device 3, when the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7. Based on the received image including the alignment marker 71 and the received image including the first marker 11, information on the relative three-dimensional direction vector of the predetermined position (A) of the object 10 with respect to the first marker 11 attached to the object 10 is obtained. In this case, the computer 2, after imaging the alignment marker 71 attached to a predetermined reference point (origin (O)), images the first marker 11 with the predetermined position (A) of the object 10 aligned to the predetermined reference point (origin (O)), and performs calibration based on the three-dimensional direction vector. This allows for calibration with a narrow field of view and focused on the three-dimensional direction. Furthermore, based on the acquired three-dimensional direction vector, the computer 2 detects the relative three-dimensional direction vector from the first marker 11 to the predetermined position (A) relative to the other member 13, from an image including the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 attached to the object 10.

[0087] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the alignment marker 71 captured by the fixed-position imaging device 3, which is positioned at a predetermined reference point (origin (O)) of the alignment member 7. It also receives an image of the first marker 11, which is attached to the object 10 and captured by the imaging device 3, when the predetermined position (A) of the object 10 is aligned with the predetermined reference point (origin (O)) of the alignment member 7. Based on the received image including the alignment marker 71 and the received image including the first marker 11, the computer 2 obtains information on the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10, and information on the relative three-dimensional direction vector of the predetermined position (A) of the object 10 with respect to the first marker 11. In this case, the computer 2, after imaging the alignment marker 71 attached to a predetermined reference point (origin (O)), images the first marker 11 with the predetermined position (A) of the object 10 aligned to the predetermined reference point (origin (O)), and performs calibration based on 3D coordinates and 3D direction vectors. This allows calibration with a narrow field of view and by combining 3D coordinates and 3D directions. Furthermore, based on the acquired 3D coordinates and 3D direction vectors, the computer 2 detects the relative 3D coordinates of the predetermined position (A) of the object 10 with respect to the other member 13, and the relative 3D direction vector from the first marker 11 to the predetermined position (A) with respect to the other member 13, from an image including the second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 attached to the object 10.

[0088] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the first marker 11 and the alignment marker 12 attached to the object 101 when the alignment marker 12 is placed at a predetermined position (B) on the object 101. Based on the received image including the alignment marker 12 and the first marker 11, the computer 2 obtains information on the relative three-dimensional coordinates of the predetermined position (B) of the object 101 with respect to the position of the first marker 11 attached to the object 101. In this case, the computer 2 treats the first marker 11 as being in the same coordinate system as the m1 coordinate system and the m2 coordinate system, and performs calibration based on three-dimensional coordinates, so that it can perform calibration with a narrow field of view and by determining the position and orientation of the m2 coordinate system in the m1 coordinate system using three-dimensional coordinates. Furthermore, based on the acquired 3D coordinates, the relative 3D coordinates of a predetermined position (B) of the object 101 with respect to another member 13 are detected from an image that includes a second marker 14 attached to a member 13 separate from the object 101 and a first marker 11 on the object 101.

[0089] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the first marker 11 and the alignment marker 12 attached to the object 101 when the alignment marker 12 is placed at a predetermined position (B) on the object 101. Based on the received image including the alignment marker 12 and the first marker 11, the computer 2 acquires information on a three-dimensional direction vector in a predetermined direction extending orthogonally from the predetermined position (B) to the first marker 11. In this case, the computer 2 can perform calibration based on the three-dimensional direction vector by treating the first marker 11 as being in the same coordinate system as the m1 coordinate system and the m2 coordinate system. Furthermore, based on the acquired three-dimensional direction vector, the computer 2 detects a relative three-dimensional direction vector in a predetermined direction with respect to another member 13 from an image including a second marker 14 attached to another member 13 and the first marker 11 of the object 101.

[0090] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the first marker 11 and the alignment marker 12 attached to the object 101 when the alignment marker 12 is placed at a predetermined position (B) on the object 101. Based on the received image including the alignment marker 12 and the first marker 11, the computer 2 obtains information on the relative three-dimensional coordinates of the predetermined position (B) of the object 101 with respect to the position of the first marker 11 attached to the object 101, and information on a three-dimensional direction vector in a predetermined direction extending orthogonally from the predetermined position (B) to the first marker 11. In this case, the computer 2 can perform calibration based on the three-dimensional coordinates and three-dimensional direction vector by treating the first marker 11 as being in the same coordinate system as the m1 coordinate system and the m2 coordinate system. Furthermore, based on the acquired 3D coordinates and 3D direction vectors, the relative 3D coordinates of a predetermined position (B) of the object 101 with respect to the other member 13, and the relative 3D direction vector in a predetermined direction with respect to the other member 13 are detected from an image including a second marker 14 attached to a member 13 separate from the object 101 and a first marker 11 on the object 101.

[0091] Furthermore, in the computer 2 of this embodiment, the control unit 4 executes a program to receive an image of the first alignment marker 15 attached to the first object 111 when the first alignment marker 15 is placed at a first predetermined position on the first object 111. Based on the received image including the first alignment marker 15, the control unit 4 obtains information on the relative three-dimensional coordinates of the first predetermined position on the first object 111 and information on the three-dimensional direction vector of the first predetermined direction extending orthogonally from the first predetermined position to the first alignment marker 15. In addition, the control unit 4 receives an image of the second alignment marker 16 attached to the second object 112 when the second alignment marker 16 is placed at a second predetermined position on the second object 112. Furthermore, based on the received image including the second alignment marker 16, information on the relative three-dimensional coordinates of the second predetermined position of the second object 112 attached to the second object 112, and information on the three-dimensional direction vector of the second predetermined direction extending orthogonally from the second predetermined position to the second alignment marker 16 are obtained. Additionally, when an image including the first alignment marker 15 of the first object 111 and the second alignment marker 16 of the second object 112 is received, the relative three-dimensional coordinates of the first predetermined position of the first object 111 with respect to the second predetermined position of the second object 112, and the relative three-dimensional direction vector of the first predetermined direction of the first object 111 with respect to the second predetermined direction of the second object 112 are detected from the received image including the first alignment marker 15 and the second alignment marker 16. In this case, even if the first object 111 and the second object 112 overlap and there is a blind spot for the imaging device 3, the computer 2 can perform alignment and measure the positional difference and tilt difference at the alignment point.

[0092] In the computer 2 of this embodiment, information on the 3D coordinates and 3D direction vector of a predetermined position (A) of the object 10 may be acquired in advance. For example, in the computer 2 of this embodiment, the control unit 4 executes a program to acquire information on the relative 3D coordinates of the predetermined position (A) of the object 10 with respect to the position of the first marker 11 attached to the object 10. Alternatively, based on the acquired 3D coordinates, the relative 3D coordinates of the predetermined position (A) of the object 10 with respect to another member 13 may be detected from an image including a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10. In this case, information on the 3D coordinates of the predetermined position (A) of the object 10 that has been acquired in advance can be used.

[0093] Furthermore, for example, in this embodiment, the computer 2 may, by having the control unit 4 execute a program, acquire information on the relative three-dimensional direction vector of a predetermined position (A) of the object 10 with respect to a first marker 11 attached to the object 10. Alternatively, based on the acquired three-dimensional coordinates, the computer 2 may detect the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to another member 13 from an image including a second marker 14 attached to a member 13 separate from the object 10 and the first marker 11 of the object 10. In this case, information on the three-dimensional direction vector of the predetermined position (A) on the object 10, which has been acquired in advance, can be used.

[0094] Furthermore, for example, in this embodiment, the computer 2, through the execution of a program by the control unit 4, acquires information on the relative three-dimensional coordinates of a predetermined position (A) of the object 10 with respect to the position of a first marker 11 attached to the object 10, and also acquires information on the three-dimensional direction vector of the object 10 with respect to the first marker 11. In addition, based on the acquired three-dimensional coordinates and three-dimensional direction vector, the computer 2 may detect the relative three-dimensional coordinates of the predetermined position (A) of the object 10 with respect to another member 13, and the relative three-dimensional direction vector of the direction from the first marker 11 to the predetermined position (A) with respect to the other member 13, from an image including the first marker 11 and the second marker 14. In this case, information on the three-dimensional coordinates and three-dimensional direction vector of the predetermined position (A) on the object 10 that has been acquired in advance can be used.

[0095] Furthermore, as shown in Figure 8, the detection method and computer 2 according to this embodiment can measure whether the welding robot is operating in the correct position by imaging the alignment marker 71 and the first marker 11 with the imaging device 3 when the object 10 is treated as, for example, part of a welding robot. In this case, by attaching the AR marker to a component that can measure not only the position but also the welding angle, it is possible to determine whether the component is placed in a predetermined position and to identify the welding point on that component.

[0096] Furthermore, by using the detection method and computer 2 according to this embodiment, as shown in Figure 9, it is possible to measure, for example, the displacement of the ground surface due to landslides at a erosion control site. In this case, the fixed imaging device 3 images the object 10, which is installed on the ground surface and to which the first marker 11 is attached, over time, and images the first marker 11 before and after the movement of the ground surface. In this case, the ground surface, the amount of movement, and the direction of movement can also be determined.

[0097] Moreover, by using the detection method and the computer 2 according to this embodiment, as shown in FIG. 10, drawing can also be performed in a three-dimensional space. At this time, for example, a rectangular parallelepiped marker is attached to one end portion of the object 10 that is not the handle portion during drawing. First, in the alignment process, from the calibration process according to this embodiment, c H A is calculated in advance. Then, m1 H A = c H m1 -1 c H A From m1 H A = c H m1 -1 c H m2m2 H A By deriving m1 H A The amount of t of z becomes the vertical distance from the virtual plane to point A, and drawing can be performed in the coordinate system of point A when t z is below the threshold value. Therefore, by measuring the length from the virtual plane generated using the alignment marker 71 to point A, the contact between the two can be defined.

[0098] Also, as shown in FIG. 11, the object 10 can also be used as a laser pointer. In this case, m1 H m2 = c H m1 -1 c H m2 By calculating

Explanation of Signs

[0099] 1 Position Detection System 2 Computers 3. Imaging device 4. Control Unit 5 Storage section 6 Communications Department 7 Alignment member 10 Objects 11 First Marker 12 Alignment markers 13 Another component 14. Second Marker 15. First alignment marker 16. Second alignment marker 41. Reception Methods 42 Calculation means 43 Detection means 71 Alignment marker 101 Object 111 First Object 112 Second Object

Claims

1. The process involves placing a first alignment marker at a first predetermined position on a first object to which a first marker is attached, and imaging the first marker and the first alignment marker. A step of acquiring information on the relative three-dimensional coordinates of a first predetermined position of a first object with respect to the position of the first marker attached to the first object, and information on the three-dimensional orientation of the first object, based on the image of the captured first alignment marker and the image including the first marker. The process involves placing a second alignment marker at a second predetermined position on a second object to which a second marker is attached, and imaging the second marker and the second alignment marker. A step of acquiring information on the relative three-dimensional coordinates of the second predetermined position of the second object with respect to the position of the second marker attached to the second object, and information on the three-dimensional orientation of the second object, based on the image of the captured second alignment marker and the image including the second marker. The process involves imaging the first marker of the first object and the second marker of the second object, thereby detecting the relative three-dimensional coordinates of the first predetermined position of the first object with respect to the second predetermined position of the second object, and the three-dimensional orientation of the first object with respect to the second object, from the image including the first and second markers captured. A detection method equipped with [the necessary components].

2. The control unit executes the program, When the first alignment marker is placed at a first predetermined position on the first object, an image of the first alignment marker attached to the first object is received. Based on the received image including the first alignment marker, information on the relative three-dimensional coordinates of the first predetermined position of the first object attached to the first object and information on the three-dimensional orientation of the first object are obtained. When the second alignment marker is placed at a second predetermined position on the second object, an image of the second alignment marker attached to the second object is received. Based on the received image including the second alignment marker, information on the relative three-dimensional coordinates of the second predetermined position of the second object attached to the second object and information on the three-dimensional orientation of the second object are obtained. A computer that, upon receiving an image including a first alignment marker for the first object and a second alignment marker for the second object, detects the relative three-dimensional coordinates of the first predetermined position of the first object with respect to a second predetermined position of the second object, and the three-dimensional orientation of the first object with respect to the second object, from the received image including the first and second alignment markers.

Images