Marker extraction system and program

Abstract

To simply display a marker corresponding to a desired position.SOLUTION: A marker extraction system comprises: a reference point identification unit for identifying a reference point corresponding to a position of a terminal apparatus among reference points whose position in real space can be grasped; a marker extraction unit for extracting the marker corresponding to the reference point identified by the reference point identification unit based on correspondence information in which the information about the reference point is associated with a marker; and a marker output unit for outputting the marker extracted by the marker extraction unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a marker extraction system and a program.

Background Art

[0002] When using AR (Augmented Reality) technology, it is necessary to grasp the position and orientation of the device that displays AR (hereinafter may be simply referred to as an AR display device). Conventionally, a marker installed at a specific position is imaged by an AR display device, and based on the size, contour shape, and orientation of the design of the marker represented in the captured image, the distance and angle of the AR display device with respect to the marker are obtained to grasp the position and orientation of the AR display device. Patent Document 1 discloses a technique for grasping the position of a device using markers installed on a wall, ceiling, etc. Such markers are fixedly installed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, it may be difficult to fixedly install a marker. For example, since a public road is a public property owned by a country or a local public entity, a marker cannot be left installed. Therefore, in order to use AR technology on a public road or the like, it is necessary to temporarily install a marker corresponding to the target position each time AR technology is used and collect it after the use of AR technology. In this case, in order to use AR technology at multiple positions, marker papers printed with markers corresponding to each position need to be prepared respectively. And since it is necessary to select and install the marker paper corresponding to the target position, there is a possibility of misselecting the marker paper.

[0005] Therefore, the present invention has been made in view of the above points, and an object thereof is to simply display a marker corresponding to a target position.

Means for Solving the Problems

[0006] One aspect of the present invention is a marker extraction system including: a reference point specifying unit that specifies a reference point corresponding to the position of a terminal device among reference points capable of grasping a position in the real space; a marker extraction unit that extracts the marker corresponding to the reference point specified by the reference point specifying unit based on correspondence information in which information regarding the reference point and the marker are associated with each other; and a marker output unit that outputs the marker extracted by the marker extraction unit.

[0007] Another aspect of the present invention is a program for causing a computer to execute: a reference point specifying step of specifying a reference point corresponding to the position of a terminal device among reference points capable of grasping a position in the real space; a marker extraction step of extracting the marker corresponding to the reference point specified by the reference point specifying step based on correspondence information in which information regarding the reference point and the marker are associated with each other; and a marker output step of outputting the marker extracted by the marker extraction step.

Advantages of the Invention

[0008] According to the present invention, a marker corresponding to a target position can be simply displayed.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0010] [Embodiment] Regarding the marker extraction system and program according to the present embodiment, preferred embodiments will be described in detail below with reference to the accompanying drawings. Note that the present embodiment is not limited to these embodiments, but also includes those with various modifications or improvements. That is, the components described below include those that can be easily assumed by those skilled in the art and substantially identical ones, and the components described below can be combined as appropriate. Also, the present embodiment can perform various omissions, substitutions, or changes of components without departing from the gist of the present invention.

[0011] First, with reference to FIGS. 1 and 2, the aspect of the marker extraction system 1 according to the present embodiment will be described.

[0012] FIG. 1 is a schematic diagram for explaining an example of a marker display method. With reference to this figure, an example of displaying a marker on the marker display device 10 in the marker extraction system 1 will be described. The marker display device 10 (which may be described as a terminal device) displays a marker according to a reference point R. A marker is something that contains information for identifying the reference point R. The marker may be composed of, for example, a pattern of a specific pattern, an illustration, a photograph, a figure, a symbol such as a character or a number. Also, the marker may, for example, look different according to the viewing angle, that is, not be point-symmetric.

[0013] FIG. 2 is a schematic diagram for showing an example of displaying an AR image on an AR display device. The AR display device 30 is a device that displays an AR image. The AR display device 30 images a marker installed at a reference point R which is a target position. The reference point R is a point whose position in the real space can be grasped. The AR display device 30 overlays and displays AR in the real space based on the relative position between the marker and the AR display device 30 obtained from the appearance of the imaged marker.

[0014] FIG. 3 is a block diagram for explaining the functional configuration of each device according to the marker extraction system of the present embodiment. With reference to this figure, the marker extraction system 1 according to the present embodiment will be specifically described. The marker extraction system 1 includes a marker display device 10, a marker extraction device 20, and an AR display device 30. The marker display device 10 and the marker extraction device 20 are operated by a user.

[0015] The marker display device 10 and the marker extraction device 20, and the AR display device 30 and the marker extraction device 20 are connected to each other via a network NW. The network NW is a predetermined communication network configured using a line. An example of the network NW is the Internet. Each functional unit of the marker display device 10, the marker extraction device 20, and the AR display device 30 may be realized using an electronic circuit as necessary. Further, each functional unit may not be included in a single device, and may be configured in such a manner that the marker display device 10, the marker extraction device 20, and the AR display device 30 are each composed of a plurality of devices.

[0016] The marker display device 10 includes a position information measurement unit 11, a direction information acquisition unit 12, an inclination information acquisition unit 13, a terminal information transmission unit 14, a marker acquisition unit 15, and a marker display unit 16.

[0017] The position information measurement unit 11 acquires position information LI indicating the position of the marker display device 10. The position information measurement unit 11 may acquire the position information LI using, for example, a Global Navigation Satellite System (hereinafter sometimes simply referred to as GNSS). The position information LI includes information indicating positions such as latitude, longitude, altitude, etc. The position information measurement unit 11 transmits the acquired position information LI to the terminal information transmission unit 14.

[0018] The direction information acquisition unit 12 acquires direction information indicating the direction in which the marker display device 10 is facing from a direction sensor. The direction sensor may be, for example, a magnetic sensor provided in the marker display device 10. The direction in which the marker display device 10 is facing is, for example, in a plane horizontal to the display screen of the marker display device 10, passing through the center line in the vertical direction of the display screen, and is the direction from the lower side to the upper side of the display screen (hereinafter sometimes referred to as the upper side direction of the display screen). The direction information acquisition unit 12 transmits the acquired direction information to the terminal information transmission unit 14.

[0019] The inclination information acquisition unit 13 acquires inclination information indicating the inclination of the marker display device 10 from an inclination sensor provided in the marker display device 10. The inclination of the marker display device 10 is, for example, the angle formed between a horizontal plane orthogonal to the direction of gravity acting on the marker display device 10 and a plane horizontal to the display screen of the marker display device 10. The inclination information acquisition unit 13 transmits the acquired inclination information to the terminal information transmission unit 14.

[0020] The terminal information transmission unit 14 acquires the position information LI from the position information measurement unit 11. Also, the terminal information transmission unit 14 acquires direction information from the direction information acquisition unit 12. Further, the terminal information transmission unit 14 acquires inclination information from the inclination information acquisition unit 13. Hereinafter, the direction information and inclination information indicating the current posture of the marker display device 10 at the reference point R, and the position information LI indicating the position of the marker display device 10 may be collectively referred to as terminal information. The terminal information transmission unit 14 transmits the acquired terminal information to the terminal information acquisition unit 21 provided in the marker extraction device 20 via the network NW.

[0021] The marker acquisition unit 15 acquires, from a marker output unit 24 described later, information regarding a reference point R corresponding to the position of the marker display device 10, a marker (marker information MI) associated with the reference point R, and installation information regarding the posture when installing the marker display device 10. Hereinafter, the marker associated with the reference point R may be simply referred to as the marker of the reference point R. The marker acquisition unit 15 transmits the acquired information regarding the reference point R, the marker of the reference point R, and the installation information to the marker display unit 16.

[0022] The marker display unit 16 acquires information regarding the reference point R, the marker of the reference point R, and the installation information from the marker acquisition unit 15. The marker display unit 16 displays the acquired information regarding the reference point R, the marker of the reference point R, and the installation information. That is, the marker display unit 16 displays information regarding the reference point R corresponding to the position of the marker display device 10, the marker of the reference point R, and the content regarding the posture when installing the marker display device 10.

[0023] The marker extraction device 20 includes a terminal information acquisition unit 21, a reference point specification unit 22, a marker extraction unit 23, a marker output unit 24, a storage unit 25, an imaging image acquisition unit 26, a self-position estimation unit 27, an AR information generation unit 28, and an AR information transmission unit 29.

[0024] The terminal information acquisition unit 21 acquires terminal information from the terminal information transmission unit 14. The terminal information acquisition unit 21 transmits position information LI among the acquired terminal information to the reference point specification unit 22. Further, among the acquired terminal information, the direction information and the inclination information are transmitted to the self-position estimation unit 27.

[0025] The storage unit 25 stores in advance corresponding information CI and information regarding the content to be displayed by the AR display device 30. The user may input and update information in the storage unit 25 as necessary. The corresponding information CI is information in which information regarding the reference point R, markers, and installation information are associated. Information regarding the reference point R may include, for example, information indicating the coordinates of the reference point R such as latitude, longitude, altitude, etc., a reference point ID which is an identifier for identifying each of the reference points R, information that serves as a mark (hereinafter sometimes referred to as a reference point mark) for the user to identify the reference point R at the site, and the like. The installation information may include, for example, information that predefines the posture when installing the marker display device 10, such as installing the marker display device 10 facing north and the plane parallel to the display screen of the marker display device 10 being perpendicular to gravity. Information regarding the content displayed by the AR display device 30 includes, for example, information regarding the appearance of the AR object which is an object displayed in the AR, such as the shape, color, pattern, texture, etc. of the AR object, the position of the AR object in the virtual space, the position and the direction the AR display device 30 is facing in the virtual space, and the like.

[0026] The reference point specifying unit 22 acquires the position information LI from the terminal information acquisition unit 21. Also, the reference point specifying unit 22 refers to the storage unit 25 to acquire the corresponding information CI. The reference point specifying unit 22 specifies a reference point R corresponding to the position of the marker display device 10 based on the acquired position information LI and the coordinates of the reference point R included in the corresponding information CI. The reference point specifying unit 22 may, for example, specify the reference point R closest to the marker display device 10. Also, the reference point specifying unit 22 does not necessarily have to specify the closest reference point R, and may specify another reference point R within a certain vicinity range from the closest reference point R. The reference point specifying unit 22 transmits at least any one of the information regarding the specified reference point R (hereinafter sometimes referred to as specified information) to the marker extraction unit 23. Also, the reference point specifying unit 22 transmits the information regarding the coordinates of the reference point R among the information regarding the specified reference point R to the self-position estimation unit 27. In the above description, the case where there are a plurality of reference points R has been described. However, this embodiment is not limited to this example, and the reference point R may be one. When the reference point R is one, the reference point specifying unit 22 specifies that reference point R.

[0027] In addition, when the interval between the positions where the reference points R are set is longer than twice the error that may occur when the position information measurement unit 11 measures the position information LI, the reference point identification unit 22 can identify the reference point R closest to the marker display device 10 based on the position information LI and the coordinates of the reference point R even if a measurement error occurs in the position information measurement unit 11. Specifically, the case where the measurement error by the position information measurement unit 11 is less than 10 [m], the interval between two reference points R is 20 [m], and the marker display device 10 is installed at one of the reference points R will be taken as an example to explain the above content. When the distance to the other reference point R with the position of one reference point R as the reference (0 [m]) is 20 [m], if the position of the marker display device 10 indicated by the position information LI is less than 10 [m], the reference point identification unit 22 can identify one reference point R as the reference point R closest to the marker display device 10. Since the measurement error by the position information measurement unit 11 is less than 10 [m] and the marker display device 10 is installed at the position (0 [m]) of one reference point R, the reference point identification unit 22 can identify one reference point R, that is, the reference point R where the marker display device 10 is installed, regardless of the magnitude of the measurement error by the position information measurement unit 11.

[0028] The marker extraction unit 23 acquires specific information from the reference point identification unit 22. Further, the marker extraction unit 23 refers to the storage unit 25 to acquire the correspondence information CI. The marker extraction unit 23 extracts the marker of the identified reference point R, the information related to the reference point R, and the installation information based on the specific information and the correspondence information CI. The marker extraction unit 23 transmits the extracted marker information MI related to the marker, the information related to the reference point R, and the installation information to the marker output unit 24.

[0029] FIG. 4 is a diagram for explaining an example of correspondence information. In the correspondence information CI shown in FIG. 4, a marker, the coordinates (latitude, longitude, altitude) of a reference point R, a reference point ID, a reference point mark, and the direction in which the marker display device 10 is installed when installed are associated with each other. The correspondence information CI-1 with the reference point ID being "0001" to the correspondence information CI-4 with the reference point ID being "0004" exemplify that there are a plurality of pieces of correspondence information CI in which the above-described information is associated (for example, for each reference point R). The reference point ID may be, for example, a symbol such as a character or a number. The reference point mark may be any name that can distinguish a specific reference point R from other reference points. For example, it may be a name of a building, an intersection, a store, a bridge, etc. that is near the reference point R and is not confused with other reference point marks. A plurality of reference point marks may be defined for one reference point R. By associating the marker with the coordinates of the reference point R, the marker extraction unit 23 can extract the marker of the reference point R corresponding to the position of the marker display device 10. By further associating the reference point ID and the reference point mark, even when the user accidentally heads to another reference point R, the user can notice the mistake by looking at the reference point ID and the reference point mark.

[0030] Returning to FIG. 3, the marker extraction system 1 will be continuously described. The marker output unit 24 acquires the marker of the reference point R, information regarding the reference point R, and installation information from the marker extraction unit 23. The marker output unit 24 transmits the acquired marker of the reference point R, information regarding the reference point R, and installation information to the marker acquisition unit 15 provided in the marker display device 10 via the network NW. When there are a plurality of marker display devices 10, the marker extraction device 20 transmits marker information MI to the marker display device 10 that has transmitted the position information LI.

[0031] The captured image acquisition unit 26 acquires the captured image obtained by the AR display device 30 capturing the marker from the captured image transmission unit 32. The captured image acquisition unit 26 outputs the acquired captured image to the self-position estimation unit 27.

[0032] The self-position estimation unit 27 acquires the direction information and the inclination information from the terminal information acquisition unit 21. Also, the self-position estimation unit 27 acquires the coordinates of the reference point R specified by the reference point specifying unit 22. Further, the self-position estimation unit 27 acquires the captured image from the captured image acquisition unit 26. The self-position estimation unit 27 estimates, based on the captured image, the position (relative position) and the direction in which the marker extraction device 20 is facing with respect to the marker from the appearance of the captured marker. Also, the self-position estimation unit 27 estimates the position (absolute position) and the direction in which the AR display device 30 is facing in the real space based on the coordinates of the reference point R (coordinates of the marker), the direction information, the inclination information, the estimated relative position, and the direction in which it is facing. The self-position estimation unit 27 can more accurately estimate the position of the AR display device 30 in the real space by using the direction information and the inclination information in addition to the coordinates of the marker and the estimated relative position and the direction in which it is facing. The self-position estimation unit 27 transmits self-position information indicating the absolute position and the direction in which it is facing to the AR information generation unit 28. Note that the appearance of the marker represented in the captured image may be based on, for example, the size, the contour shape, the design, etc. of the marker.

[0033] The AR information generation unit 28 acquires the self-position information from the self-position estimation unit 27. Also, the AR information generation unit 28 refers to the storage unit 25 and acquires information regarding the content to be displayed by the AR display device 30. The AR information generation unit 28 generates AR information, which is information for displaying a display as if an AR object exists in the real space, based on the self-position information and the information regarding the content to be displayed by the AR display device 30. The AR information generation unit 28 transmits the AR information to the AR information transmission unit 29.

[0034] The AR information transmission unit 29 acquires the AR information from the AR information generation unit 28. The AR information transmission unit 29 transmits the acquired AR information to the AR information acquisition unit 33 provided in the AR display device 30 via the network NW.

[0035] The AR display device 30 includes an imaging unit 31, an imaging image transmission unit 32, an AR information acquisition unit 33, and an AR display unit 34.

[0036] The imaging unit 31 included in the AR display device 30 is, for example, a camera that images the front angular field region. The imaging unit 31 images the marker displayed on the marker display unit 16. The imaging unit 31 transmits the captured image in which the marker is captured to the captured image transmission unit 32.

[0037] The captured image transmission unit 32 acquires the captured image from the imaging unit 31. The captured image transmission unit 32 transmits the acquired captured image to the captured image acquisition unit 26 included in the marker extraction device 20 via the network NW.

[0038] The AR information acquisition unit 33 acquires the AR information from the AR information transmission unit 29. The AR information acquisition unit 33 transmits the acquired AR information to the AR display unit 34.

[0039] The AR display unit 34 acquires the AR information from the AR information acquisition unit 33. The AR display unit 34 displays an AR image as if an AR object exists in the real space based on the acquired AR information. The self-position estimation unit 27 may, for example, use a technique called SLAM (Simultaneous Localization and Mapping) to grasp the amount of movement and the direction in which the AR display device 30 is facing. Based on the self-position information estimated by the self-position estimation unit 27 and the information about the amount of movement and the direction in which the AR display device 30 is facing, even if the AR display device 30 is moved, the position and the direction in which the AR display device 30 is facing can be grasped. Therefore, the AR display device 30 does not necessarily have to continue imaging the marker after imaging the marker once.

[0040] FIG. 5 is a flowchart for showing an example of the processing flow of the marker extraction device. With reference to the figure, an example of the processing of outputting a marker corresponding to the position of the marker display device 10 based on the position information LI of the marker display device 10 will be described.

[0041] First, the user brings the marker display device 10 near the reference point R for the purpose of observing the AR image. The position information measurement unit 11 acquires the position information LI, for example, according to the user's operation, before the marker display device 10 is installed at the reference point R or after it is installed at the reference point R. (Step S101) The terminal information acquisition unit 21 acquires, via the network NW, the position information LI indicating the position of the marker display device 10 measured by the position information measurement unit 11.

[0042] (Step S102) The reference point identification unit 22 identifies the reference point closest to the marker display device 10 based on the position information LI and the correspondence information CI, that is, the position of the marker display device 10 and the position (coordinates) of the reference point R.

[0043] (Step S103) The marker extraction unit 23 extracts the marker of the reference point R corresponding to the position of the marker display device 10 from the correspondence information CI.

[0044] (Step S104) The marker output unit 24 outputs marker information MI regarding the marker to the marker display device 10.

[0045] FIG. 6 is a diagram for explaining a specific example of an image displayed on the display screen of the marker display device. With reference to this figure, the image displayed on the display screen (marker display unit 16) of the marker display device 10 will be described. FIG. 6(A) shows a first display image IM1 which is a first example of the display image. FIG. 6(B) shows a second display image IM2 which is a second example of the display image. The first display image IM1 and the second display image IM2 each include a current position display unit 1002, a first display button 1003, a marker image display unit 1004, a reference point selection unit 1005, and a second display button 1006.

[0046] In the marker extraction system 1, the guidance display unit 1001 displays to the user the content for guiding the marker display device 10 to be installed in the same posture as the predetermined posture (associated with the correspondence information CI or shared among users) when installing. When the display screen of the marker display device 10 is orthogonal to the gravitational direction and installed facing north, the case where the installation posture is predetermined will be described. In FIGS. 6(A) and 6(B), examples are shown in which a compass and an azimuth angle based on north are displayed for guidance. In FIG. 6(A), since the azimuth angle is displayed as 0 [degrees], the user does not need to adjust the direction in which the marker display device 10 is facing. In FIG. 6(B), since the azimuth angle is displayed as 29 [degrees] (northeast), the user rotates the direction in which the marker display device 10 is facing counterclockwise by 29 [degrees] to face north (0 [degrees]). Note that the guidance display unit 1001 does not necessarily need to display the compass and the azimuth angle. For example, the content for guidance may be output by a visual method such as characters like "Please turn clockwise" or an arrow, or an auditory method such as voice reading the content for guidance. Further, the guidance display unit 1001 may guide the marker display device 10 to face the correct direction by vibrating when the marker display device 10 is facing the wrong direction. Further, the guidance display unit 1001 is not limited to an example of displaying the content for guiding the direction in which the marker display device 10 is facing, and may display the content for guiding the marker display device 10 to be installed at a predetermined inclination. By displaying the content for guiding the installation in the predetermined posture, the user can install the marker display device 10 in the predetermined posture by looking at the content for guidance even when the user has forgotten the installation posture or has recognized it incorrectly. Note that since the guidance display unit 1001 is for displaying the content for guiding the user to install the marker display device 10 in the predetermined posture, it does not necessarily need to include the first display image IM1 and the second display image IM2.

[0047] The current position display unit 1002 displays information indicating the current position of the marker display device 10. FIGS. 6(A) and 6(B) show an example in which the values of latitude and longitude among the position information LI of the marker display device 10 are displayed. Note that the guidance display unit 1001 may display the value of altitude among the position information. By the current position display unit 1002 displaying information indicating the current position of the marker display device 10, when the user arrives at a reference point R different from the reference point R at the position where the AR technology is planned to be used by mistake, the user can notice the mistake by looking at the information indicating the current position. Further, on the current position display unit 1002, for the reference point R corresponding to the position information LI of the marker display device 10, a reference point mark (for example, the name of a building near the reference point R) may be displayed, or a reference point ID (for example, a number for identifying each reference point R) may be displayed. By the current position display unit 1002 displaying the reference point mark or the reference point ID, the user can intuitively understand the current position as compared with the case where latitude and longitude are displayed. Note that since the current position display unit 1002 displays content for assisting the user in understanding the current position, it is not necessarily required to include the first display image IM1 and the second display image IM2.

[0048] The first display button 1003 is a button for causing the marker image display unit 1004 to display a marker corresponding to the position information LI. When the user selects the first display button 1003, the marker image display unit 1004 displays a marker of the reference point R corresponding to the position information LI of the marker display device 10, that is, the current position.

[0049] The marker image display unit 1004 displays a marker of the reference point R according to the position of the marker display device 10. FIGS. 6(A) and 6(B) show examples in which a marker having a reference point ID in the marker is displayed. By displaying the reference point ID in the marker, even for a marker using a point-symmetric pattern or symbol, the self-position estimation unit 27 can estimate the angle at which the marker is visually recognized (the direction in which the AR display device 30 is facing) from the appearance of the captured marker. Note that in the marker image display unit 1004, a reference point mark may be displayed in the marker, or the marker, the reference point ID, and the reference point mark may be displayed separately and independently.

[0050] The reference point selection unit 1005 is a button for the user to select the reference point R. FIGS. 6(A) and 6(B) show an example of specifying the reference point R by selecting the reference point ID. However, this embodiment is not limited to this example, and the reference point R may be specified by selecting a reference point mark, or the reference point R may be specified by inputting the coordinates of the reference point R. The marker extraction system 1 can display a marker regardless of whether GNSS can be received when the user selects the reference point R.

[0051] The second display button 1006 is a button for displaying the marker of the reference point R selected by the user on the marker image display unit 1004. Note that since the first display button 1003 is labeled "Display AR marker from current position" and the second display button 1006 is labeled "Display selected AR marker", the user can distinguish between the respective buttons.

[0052] In addition, in this embodiment, an example is shown in which the position information measurement unit 11 acquires the position information LI in accordance with a user operation (for example, selecting the first display button 1003). However, this embodiment is not limited to this example. The position information measurement unit 11 may acquire the position information LI at regular intervals. When the position information measurement unit 11 acquires the position information LI of the marker display device 10 at regular intervals, the first display image IM1 and the second display image IM2 do not necessarily have to include the first display button 1003. In that case, the first display image IM1 and the second display image IM2 may update the position displayed on the current position display unit 1002 and the marker or reference point ID displayed on the marker image display unit 1004 each time the position indicated by the position information LI changes. Since the display screen is updated each time the position changes, the user can move while referring to the display screen, aiming at the reference point R corresponding to the position where the AR technology is used.

[0053] In addition, in this embodiment, an example is shown in which the marker display device 10 acquires the direction information and the inclination information and uses them for estimating its own position by the self-position estimation unit 27. However, this embodiment is not limited to this example. For example, when the direction along the road, the angle between the horizontal plane orthogonal to the gravity direction and the ground, etc. are associated with the correspondence information CI in advance, that is, when the installation posture for each reference point R is determined in advance, it is not necessarily required for the marker display device 10 to acquire the direction information and the inclination information, or for the self-position estimation unit 27 to use them. By determining in advance the posture when installing based on the situation around the reference point R such as the direction along the road or the slope of the road, and associating and storing it in the correspondence information CI, the processing in the marker extraction system 1 can be simplified. Also, when the display screen of the marker display device 10 is orthogonal to the gravitational direction and installed facing north, etc., and the installation postures among users are unified (shared), it is not always necessary for the marker display device 10 to acquire the direction information and the inclination information, or for the self-position estimation unit 27 to use them. When the installation posture for each reference point R is determined and shared among users, since there is no need to change the installation posture for each reference point R, the processing in the marker extraction system 1 can be simplified. Also, if the installation posture for each reference point R is changed, when the user installs the marker display device 10, the user may be confused or install it in an incorrect posture. Since the installation posture when installing the marker display device 10 is unified among users, the user does not need to change the installation posture for each reference point R, and it is possible to prevent the marker display device 10 from being installed in an incorrect posture.

[0054] In this embodiment, an example in which the marker display device 10 displays a marker by the marker output unit 24 transmitting marker information MI to the marker display device 10 is shown. However, this embodiment is not limited to this example, and the marker may be displayed on a medium other than the marker display device 10. For example, the marker output unit 24 may output marker information MI corresponding to the reference point R to a portable printer and cause it to be printed. The user can use AR technology by installing the printed marker paper at the reference point R and imaging the marker printed on the marker paper with the AR display device 30. By discharging the marker paper each time AR technology is used, it is possible to prevent misidentifying the marker paper corresponding to the target position and the marker paper at other positions.

[0055] In this embodiment, the marker display device 10 acquires direction information and inclination information. The self-position estimation unit 27 estimates the position (relative position) and the direction in which the marker extraction device 20 faces with respect to the marker. However, when the direction and inclination of the reference marker (marker display device 10) are different from those determined in advance, the position (absolute position) and the direction in which the AR display device 30 faces in the real space estimated based on the relative position may be different from the actual ones. In this embodiment, the self-position estimation unit 27 can accurately estimate the position of the AR display device 30 in the real space by using the direction information and the inclination information in addition to the coordinates of the marker, the estimated relative position, and the direction in which it faces.

[0056] In this embodiment, an example of displaying a marker on the display screen of the marker display device 10 is shown by the marker extraction device 20 specifying the reference point R based on the position information LI acquired by the marker display device 10 and outputting the marker information MI regarding the marker of the specified reference point R to the marker display device 10. However, this embodiment is not limited to this example, and the marker display device 10 may perform the process of acquiring the position information LI, specifying the reference point R, and extracting the marker. That is, the marker display device 10 may include a reference point specifying unit 22, a marker extraction unit 23, and a storage unit 25. The marker display device 10 can display the marker of the reference point R corresponding to the position information LI of the marker display device 10 even when it cannot be connected to the network NW by performing the processes from step S101 to step S104 without passing through the marker extraction device 20. In that case, instead of the marker output unit 24, the marker display unit 16 provided in the marker display device 10 outputs the marker to the user.

[0057] In the above description, the case of using the AR technology is taken as an example. However, the present embodiment is not limited to this example, and it may be used when using technologies such as VR (Virtual Reality) and MR (Mixed Reality). Further, the marker extraction system 1 according to the present embodiment can be used for confirmation when viewed from the construction site or the ground by displaying the position in the ground, such as a shield machine which is an excavator used in the shield method, a propulsion pipe and an excavator used in the propulsion method, etc., by AR or the like.

[0058] [Summary of the Present Embodiment] According to the above-described embodiment, among the reference points R for grasping the position in the real space, a reference point specifying unit 22 for specifying a reference point R corresponding to the position of the marker display device 10 (terminal device), information regarding the reference point R, and a marker are provided. Based on the correspondence information CI in which they are associated, a marker extraction unit 23 that extracts a marker corresponding to the reference point R specified by the reference point specifying unit 22, and a marker output unit 24 that outputs marker information MI regarding the marker extracted by the marker extraction unit 23. According to the marker extraction system 1 according to the present embodiment, since the marker of the reference point R corresponding to the position where the AR technology is used is displayed on the marker display device 10, it is not necessary to prepare a plurality of marker papers. Further, according to the marker extraction system 1 according to the present embodiment, since the marker of the reference point R corresponding to the position where the AR technology is used is displayed on the marker display device 10, it is possible to prevent the marker of another reference point R from being erroneously installed. Therefore, the marker extraction system 1 according to the present embodiment can easily display a marker corresponding to the target position.

[0059] Note that the entire functions or some of the functions of each part included in the marker display device 10, the marker extraction device 20, and the AR display device 30 in the above-described embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, reading the program recorded on this recording medium into a computer system, and executing it. Here, the "computer system" is assumed to include hardware such as an OS and peripheral devices.

Explanation of Reference Numerals

[0060] 1…Marker extraction system, 10…Marker display device, 11…Position information measurement unit, 12…Direction information acquisition unit, 13…Tilt information acquisition unit, 14…Terminal information transmission unit, 15…Marker acquisition unit, 16…Marker display unit, 20…Marker extraction device, 21…Terminal information acquisition unit, 22…Reference point identification unit, 23…Marker extraction unit, 24…Marker output unit, 25…Storage unit, 26…Captured image acquisition unit, 27…Self-position estimation unit, 28…AR information generation unit, 29…AR information transmission unit, 30…AR display device, 31…Imaging unit, 32…Captured image transmission unit, 33…AR information acquisition unit, 34…AR display unit, R…Reference point, LI…Position information, MI…Marker information, CI…Corresponding information

Claims

1. Among the reference points that can grasp the position in the real space, a reference point specifying unit that specifies a reference point corresponding to the position of the terminal device, Based on the correspondence information in which the information about the reference point and the marker are associated, a marker extraction unit that extracts the marker corresponding to the reference point specified by the reference point specifying unit, A marker output unit that outputs the marker extracted by the marker extraction unit, A marker extraction system comprising:

2. Further comprising a position information measurement unit that measures the position information of the terminal device, The information about the reference point includes the coordinates of the reference point, The reference point specifying unit specifies the reference point corresponding to the position of the terminal device based on the position information measured by the position information measurement unit and the coordinates of the reference point included in the correspondence information The marker extraction system according to claim 1.

3. A marker display unit that displays the marker output from the marker output unit, An imaging unit that images the marker displayed by the marker display unit, Based on the size, contour shape, and design of the marker represented in the image captured by the imaging unit and the coordinates of the reference point specified by the reference point specifying unit, an own position estimation unit that estimates the position and the direction facing in the real space of the AR display device for displaying AR, An AR display unit that displays AR information corresponding to the position and the direction estimated by the own position estimation unit on the AR display device, The marker extraction system according to claim 2, further comprising:

4. Further comprising a direction information acquisition unit that acquires direction information about the direction in which the terminal device is facing at the reference point, The own position estimation unit estimates the position and the direction based on the appearance of the marker represented in the image captured by the imaging unit and the direction information The marker extraction system according to claim 3.

5. Further comprising an inclination information acquisition unit that acquires inclination information about the inclination of the terminal device at the reference point, The own position estimation unit estimates the position and the direction based on the appearance of the marker represented in the image captured by the imaging unit and the inclination information The marker extraction system according to claim 3.

6. The reference point specifying unit specifies the reference point corresponding to the information about the reference point input by the user to the terminal device The marker extraction system according to claim 1.

7. The terminal device outputs information including content that guides the user to direct the terminal device in a predetermined direction. The marker extraction system according to any one of claims 1 to 6.

8. The information regarding the reference points includes identifiers for identifying each of the plurality of reference points. The marker output unit further outputs the identifier of the reference point specified by the reference point specifying unit. The marker extraction system according to any one of claims 1 to 6.

9. The interval between the positions where the plurality of reference points are set is longer than twice the error that may occur when the position information measuring unit measures the position information. The marker extraction system according to any one of claims 2 to 5.

10. The marker output unit further outputs the position information measured by the position information measuring unit. The marker extraction system according to any one of claims 2 to 5.

11. A program that causes a computer to perform a reference point specifying step of specifying a reference point corresponding to the position of the terminal device among the reference points that can grasp the position in the real space, a marker extraction step of extracting the marker corresponding to the reference point specified in the reference point specifying step based on the correspondence information in which the information regarding the reference point and the marker are associated, and a marker output step of outputting the marker extracted in the marker extraction step. ​

Images