Image display device, image display system, and image display method

Abstract

To provide an image display method and an image display device, which enable users to view an image of a separate space with a more realistic feeling or to eliminate a feeling of boredom when moving in a real space.SOLUTION: An image display device that displays an image of a separate space differing in position from a real space in which a user is actually located, comprises: a position detection unit configured to detect the position of the user; a communication interface for acquiring, from the outside, course information that includes image data of the separate space and an imaged position; a memory for preserving the image data and the course information acquired by the communication interface; a display unit configured to display the image of the separate space preserved in the memory; and a control unit configured to control the display of the image of the separate space on the display unit. The control unit maps a course composed of a plurality of separate space points to a course 27a composed of a plurality of real space points, and when the user moves along the real space course 27a, displays images 24a to 24f of mapped separate space points in accordance with the user's position.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an image display device, an image display system, and an image display method for displaying an image in a different space from a real space in which a user is actually located. [Background technology]

[0002] Well-known technologies include Virtual Reality (VR), which creates and displays image content of a virtual world using a computer or other device, allowing users to experience the sensation of being in that virtual world, and Augmented Reality (AR), which superimposes image content created using a computer or other device onto real scenery. These technologies have been put to practical use using devices such as head mounted displays (HMDs) and smartphones.

[0003] As an example of a game that applies the above technology, Patent Document 1 discloses an entertainment system in which a player uses a mobile terminal to actually move around the real world and obtain real information from real stores, facilities, etc. This game is structured as follows: "A map of the user's surroundings is displayed on the user's mobile terminal, and the user is prompted to select an event spot. When an event spot is selected, a quiz or mini-game is presented to the user. The user travels to the actual event spot using the map as a guide, gets hints to play the quiz or mini-game, and earns points when they clear the quiz or mini-game." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2002-49681 A Summary of the Invention [Problem to be solved by the invention]

[0005] With VR and AR technologies, a user can experience being in another space by viewing images of a space located differently from the real space (hereinafter referred to as "another space") while being in the space in which the user actually resides (hereinafter referred to as "real space").

[0006] For example, when a user views a landscape of a memorable place that the user has visited in the past or a landscape of an interesting place that the user has not yet visited as an image of another space, the user cannot experience the place with a sense of realism by simply playing back the image of the place. In order to enhance the sense of realism, the real space and the other space are merged, for example, the user can experience the other place as if he or she were actually moving through the other place by viewing an image of the other place while actually moving through the current location. Alternatively, when a user moves (walks or runs) along a fixed course in the real space, it may be boring to only see the same fixed scenery in the real space. In such a case, if the user can view an image of another space while moving, the walk or run can be made fresh and the user's sense of boredom can be eliminated.

[0007] In Patent Document 1, a user plays a game in a virtual space using a mobile device, but the user moves through real space using a map displayed on the mobile device to obtain quiz hints, and the game can be considered to be a combination of virtual and real spaces. However, the configuration described in Patent Document 1 does not combine real space with another space, i.e., it does not combine the space where the user is actually located with a space in a different location. Therefore, the user cannot experience images of another space with a sense of realism or eliminate boredom when moving around in real space. This does not mean that

[0008] An object of the present invention is to provide an image display device, an image display system, and an image display method that enable a user to view an image in another space with a more realistic feel, or to eliminate the sense of boredom felt when moving through a real space. [Means for solving the problem]

[0009] The present invention is an image display device worn by a user and displaying an image of another space different from the real space where the user is actually located, the image display device comprising: a position detection unit that detects the position of the user; an orientation / attitude detection unit that detects the orientation and attitude of the user in the front direction (the azimuth and elevation angle of the user's line of sight); a communication interface that acquires image data of the other space and course information including a shooting position from the outside; a memory that stores the image data and course information acquired by the communication interface; a display unit that displays the image of the other space stored in the memory; an audio output unit that outputs audio; and a control unit that controls the display of the image of the other space on the display unit. The control unit associates a course consisting of a plurality of points in the other space with a course consisting of a plurality of points in the real space based on the course information of the other space, and displays an image of the associated point of the other space according to the user's position when the user moves along the course in the real space. Effect of the Invention

[0010] According to the present invention, when a user moves along a course in real space, the user can view the scenery of a corresponding position in another space as a still image or a moving image, which has the effect of providing a realistic experience as if the user were actually moving through another space, or eliminating the boredom felt when moving through real space. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing a configuration of an image display system 1 according to a first embodiment. [Figure 2A] FIG. 2 is a diagram showing the appearance of a user wearing the image display device 2 (head mounted display 19). [Figure 2B] FIG. 2 is a diagram showing an example of a display screen of the image display device 2. [Diagram 3] FIG. 13 is a diagram showing an example of acquiring images of each point on a course in a different space. [Figure 4] FIG. 13 is a diagram showing an example of displaying a course setting in a real space and an image captured in another space. [Figure 5A] FIG. 13 is a diagram showing a process for registering course information for another space. [Figure 5B]FIG. 5B is a diagram showing the contents of course information registered by the processing of FIG. 5A. [Figure 6] FIG. 13 illustrates the process of displaying registered images by traversing a course in real space. [Figure 7] FIG. 11 is a diagram showing an example of a course in a different space having a difference in altitude according to the second embodiment. [Figure 8] FIG. 13 is a diagram showing an example of a flat course with no altitude difference in real space. [Figure 9] FIG. 11 is a diagram showing a course in another space according to the third embodiment. [Figure 10] FIG. 1 is a diagram showing a course around a track as a course in real space. [Figure 11] FIG. 2 is a diagram showing a user wearing an image display device 2 (smartphone 40). [Figure 12A] FIG. 4 is a diagram showing an example of a display screen of a smartphone 40. [Figure 12B] FIG. 4 is a diagram showing an example of a display screen of a smartphone 40. [Figure 12C] FIG. 4 is a diagram showing an example of a display screen of a smartphone 40. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same parts are generally designated by the same reference numerals, and the repeated description will be omitted. Example 1

[0013] In the first embodiment, an image display system 1 that displays an image captured in another space when a user moves to a corresponding position in the real space will be described.

[0014] [Image display system] 1 is a diagram showing a configuration of an image display system 1 according to a first embodiment. In the image display system 1, an image display device 2, an imaging device 3, and a Web server 4 are connected via a communication network 5. The communication network 5 includes a wireless communication network and the Internet. Here, a case where a head mounted display (HMD) is used as the image display device 2 will be described. The internal configuration of each device will be described below.

[0015] The image display device 2 has an imaging element 6a, a position detection unit 7a, a memory 8a, a communication interface 9a, a control unit 10a, an orientation / attitude detection unit 11a, a display unit 12a, and an audio output unit 13. The imaging element 6a includes an optical lens (not shown) and is a unit that captures still images or moving images, and the captured image data is stored in the memory 8a. The position detection unit 7a measures latitude, longitude, and altitude as position information using, for example, a global positioning system (GPS). The orientation / attitude detection unit 11a includes a geomagnetic sensor and a gyro sensor (not shown) and measures an azimuth angle (angle measured clockwise from the north direction) and an elevation angle (angle measured upward from the horizontal plane). The display unit 12a captures or generates and displays images captured in another space, images synthesized with the images, images of courses, etc. The communication interface 9a is connected to a communication network 5 and transmits and receives data between the Web server 4 and the imaging device 3. The audio output unit 13 includes a speaker (not shown) and generates an audio signal by synthesis or the like to output the audio. The control unit 10a is made up of a CPU and controls all the functional blocks of the image display device 2. The memory 8a stores a display control application, a Web search application, a map application, and the like (not shown).

[0016] Here, the image display device 2 includes an imaging element 6a and is configured to also function as the image capture device 3 described next, but when used only for the image display function, the imaging element 6a may be omitted.

[0017] The imaging device 3 has an imaging element 6b, a position detection unit 7b, a memory 8b, a communication interface 9b, a control unit 10b, an orientation / attitude detection unit 11b, and a display unit 12b. Descriptions of functional blocks with the same names as those of the image display device 2 described above will be omitted because they have the same configurations and functions.

[0018] The imaging device 3 is for photographing another space, and photographs still images and videos using the imaging element 6, and stores the image data in the memory 8b. The position detection unit 7b and the orientation / posture detection unit 11b obtain information (position, orientation, orientation, etc.) at the time of photographing, and store the information in the memory 8b as additional information (metadata) of the image data. At that time, the image data and the additional information are paired and stored as one image file, making it easy to associate them. The stored image file is transmitted to the Web server 4 or the image display device 2 via the communication interface 9b and the communication network 5. Alternatively, it can be passed to the Web server 4 or the image display device 2 via a removable recording medium such as an SD card (not shown). The display unit 12b reproduces and displays the image at the time of photographing and the data stored in the memory 8b.

[0019] The Web server 4 has a communication interface 9c, a control unit 10c, and a memory 8c, and image files and Web page documents are stored in the memory 8c. Image data captured by the imaging device 3 and an image file including its associated information are received via the communication interface 9c and taken into the memory 8c. The control unit 10c generates course information based on the acquired data and publishes it on the Internet as a Web page document. Methods for disclosing the course information include a method of directly indicating latitude, longitude, and altitude as point information on the course, a method of specifying the shooting location as a point on the course from the associated information of the image file, and a combination of these methods. The above course is called a course in another space in this embodiment. In this way, by publishing the course in another space as a Web page document, the image data in the other space can be used not only by the photographer but also by other users who are interested.

[0020] The operation of the image display system 1 of this embodiment is as follows. First, the imaging device 3 captures images at each point on the course in the other space, and transmits the image data and the attached information of the capturing points to the Web server 4 for registration. This registration process may be performed by the user who owns the image display device 2, or by another user. When the user himself / herself performs the registration, the imaging element 6a in the image display device 2 may be used, in which case the imaging device 3 is not necessary. Next, when the user wears the image display device 2 and moves along a course in the real space, the image of the other space registered in the Web server 4 is displayed on the image display device 2 according to the moving position. At that time, when the user specifies the desired positions (starting point and ending point) to move along in the real space on the map, the course in the real space is determined in which the distance and direction are approximated to the course in the other space by a predetermined geometric relationship, and the registered image of the other space is displayed at the moving point in the real space corresponding to the capturing point in the other space.

[0021] [Image display device] 2A and 2B are diagrams showing an example of the appearance and display screen of the image display device 2. The image display device 2 is a see-through type HMD 19. FIG. 2A shows a state in which a user 20 wears the see-through type HMD 19 on his / her head. The display unit 12a of the see-through type HMD 19 is configured such that a semi-transmissive prism 22 is disposed in front of one eye (the right eye) 21a, and a reproduced image of a display element 23 on the side is irradiated onto the prism 22. In addition, an imaging element 6a is disposed in front of the see-through type HMD 19, and a control unit (drive unit) 10a is disposed on the upper part.

[0022] 2B shows an example of the display screen (display unit 12a) when the user 20 looks at the prism 22. A reconstructed image 24 (shown by a solid line) of another space by the display element 23 can be seen superimposed on a landscape image 25 (shown by a dashed line) in front of the user 20 in the real space through the prism 22. When the reconstructed image 24 is displayed, its display position (display direction) is, for example, the center of the display screen 12a. Furthermore, by comparing the orientation information of the image display device 2 and the image capturing direction by the orientation / posture detection units 11a and 11b of the image capturing direction of the image capturing device 3, when the user's line of sight direction in the real space coincides with the orientation of the image capturing direction of the image capturing device 3 in the other space, the corresponding reconstructed image 24 can be displayed.

[0023] In this example, the see-through HMD 19 is a monocular type in which the prism 22 and the display element 23 are placed on one eye (right eye) 21a, but it can also be a binocular type in which they are placed on the other eye (left eye) 21b. The transmittance of the prism 22 is set appropriately depending on whether it is a monocular or binocular type so that both the landscape image 25 and the reproduced image 24 can be viewed in a balanced manner.

[0024] [Course in a different space] 3 is a diagram showing an example of acquiring images of each point on a course in another space. While moving along a course 26a (indicated by a dashed arrow) in another space, the photographer (the user of the image display device 2 himself / herself, or another user) photographs the surrounding scenery to be recommended at each point on the course (scenery that will become a memory for the user) with the imaging device 3. Note that a route that can be traveled without overlapping in order to photograph each recommended object is called a recommended course.

[0025] Specifically, at the start point P111 of the recommended course 26a, the landmark gate 28 is photographed, and at point P112, a building 29 is photographed. At the branching point P113, a landscape including the branching road is photographed. At point P114, a tower 30 is photographed, and at the end point P115 of the course, a torii gate 31 is photographed. If there are other recommended objects in a position outside the recommended course 26a (points P111 to P115), for example, a stone monument 32 is photographed at point P116. The image data photographed at these points and their associated information (such as location information) are stored in memory 8b as image files and transmitted to the Web server 4. The data at each point is stored with a distinction made as to whether or not it is on the recommended course 26a.

[0026] [Real space course] FIG. 4 is a diagram showing an example of setting a course in the real space and displaying an image taken in another space. A course 27a (indicated by a solid arrow) is set in the real space, and points P211 to P215 on the course are shown. Points outside the course are shown by dashed lines 39a and 39b. The procedure for setting the course will be described. A user who wishes to move in the real space uses a map application or the like to display a map of the real space on the display unit 12a of the image display device 2, and specifies the start point P211 and end point P215 of the course. As a result, the start point P211 is associated with the start point P111 of the other space shown in FIG. 3, and the end point P215 is associated with the point P115 of the other space. Next, a course that can be moved in the real space is determined so that it has a predetermined geometric relationship (direction deviation and scale) approximately with the course in the other space. Naturally, since the conditions of roads and buildings are different between the real space and the other space, the shape of the course that the user can move in the real space does not strictly match the shape of the course in the other space. Therefore, routes and intermediate points that have similar geometric relationships are set. Details of the course setting will be explained in the PAD section below. As a result, intermediate points P212 to P214 in the real space are determined so as to correspond to intermediate points P112 to P114 of the course in another space shown in Figure 3, and a course 27a in the real space connecting these points is set.

[0027] When the user moves along the course 27a in the real space and approaches each of the set points P211 to P215, the photographed images 24a to 24e of the corresponding recommended objects (structures and scenery at branching points) photographed in another space are displayed on the display unit 12a of the image display device 2. The direction of the user's line of sight is detected by the orientation / posture detection unit 11a, and the registered recommended objects are displayed when the direction of the registered recommended objects is in the direction of the user's line of sight, thereby enhancing the sense of realism. Among these, the point P213 corresponds to the point P113, which is a branching point in another space. When the user approaches this branching point P213, a branching guide map is displayed in which not only the corresponding photographed image 24c but also an arrow 24c' indicating the branching direction registered in advance is image-combined with the corresponding photographed image 24c.

[0028] Although the intermediate points P112 and P114 on the different space course 26a are not branching points, the corresponding points P212 and P214 on the real space course 27a require a change of course (right turn) at the immediately preceding intersection. Therefore, the position detection unit 7a of the image display device 2 detects that the user has reached the intersection, and the audio output unit 13 instructs the user by voice to turn right. This allows the user to move without straying from the course 27a set in the real space. Note that if the user moves along the course of the dashed lines 39a and 39b that deviates from the recommended course, the image 24f of the stone monument is displayed at the position of the point P216.

[0029] As a result, when the user walks along the course 27a in the real space, the user can see a landscape image of a different space from the real space according to the moving position, and can experience the different space with a sense of realism. Furthermore, at the branching point, the user can see a branching guide map combined with the landscape image, which has the effect of allowing the user to preview the route at the branching point.

[0030] [Processing PAD and Course Information] 5A is a diagram showing the process of registering course information for another space, and shows the process structure as a hierarchical problem analysis diagram (PAD). The control unit 10b of the imaging device 3 reads out the captured image data and its associated information from the memory 8b, and transmits them to the Web server 4 via the communication interface 9b. Since multiple courses are registered in the Web server 4, a course number is assigned to each one.

[0031] In step S11, image data of another space captured by the imaging device 3 and its associated information (such as location information) are registered as an image file. At this time, an image file that is not on the recommended course may be included (for example, data of the stone monument 32 in FIG. 3).

[0032] In step S12, the following process is repeated for each point (P111 to P115) on the recommended course. In step S12a, an image file (images 28 to 31 at P111 to P115 in FIG. 3) to be registered for the specified point is selected. In step S12b, point information is set from the attached information of the selected image file. At this time, if there is information on the shooting location, this is used as the point information, and if there is no information on the shooting location, the point information is specified from a map application or the like and registered. In addition, related figures and symbols to be displayed superimposed on the image, such as a branch guidance arrow, are arbitrarily registered as one piece of attached information. In the example of FIG. 3 and FIG. 4, an arrow symbol 24c' that serves as route guidance at the branch point P213 (P113) is registered as attached information.

[0033] FIG. 5B is a diagram showing the contents of the course information registered by the process of FIG. 5A. The course information 44 includes a course number, course point information, and image file information. The course number is a number for identifying the course. The course point information is position information (latitude, longitude, altitude) of points from the start point to the end point of the course. The altitude can be omitted if it is not covered by the function of the position detection unit. The image file information is a list (or link) of image files 45 of images taken at points on or around the course. In addition, the image file 45 referenced from the image file information includes image data 16 and its associated information 17 (generally also called metadata). The associated information 17 includes the shooting position and related figures and symbols.

[0034] 6 is a diagram showing the process of moving along a course in real space and displaying registered images, and the processing structure is shown as a hierarchical PAD. The control unit 10a of the image display device 2 receives image data and its associated information from the Web server 4 via the communication interface 9a, sets a course in real space, and displays the corresponding image according to the moving position.

[0035] Step S21 is to specify a course in another space in which an image to be displayed is stored. When the user inputs a course number (e.g., No. 1) in step S21a, the corresponding course information and image file are retrieved from the Web server 4 in step S21b.

[0036] In step S22, the user specifies a start point and a finish point in the real space. The method for specifying the positions includes a method of specifying the position from a map application, and a method of specifying the user's current position from the position detection unit 7a of the image display device 2.

[0037] In step S23, the following process is repeated for each section defined between adjacent points on the course. That is, the end point (target point) of the section is set at the start point of the section, and when the user reaches the end point of the section, the registered image is displayed and the end point of the next section is set.

[0038] In step S23a, in order to set a course in the real space that is similar to the course in another space, the section end point is determined based on the geometric relationship between the two courses. For this purpose, the linear direction and distance when the section start point Ps (the starting point in the first time) is expected to the course end point Pg (the goal point) in the other space and the real space are compared, and the direction difference and scale between the two are obtained. For example, the linear direction from the section start point Ps to the goal point Pg in the other space is eastward, and the distance is 5 km. On the other hand, the linear direction from the section start point Ps' to the goal point Pg' in the real space is northward, and the distance is 2 km. In this case, the direction difference of the real space with respect to the other space is a 90-degree counterclockwise rotation, and the scale is 0.4 times. Using these geometric relationships (direction difference and scale), the section end point Pe' in the real space corresponding to the section end point Pe in the other space is obtained. However, the position of the section end point Pe' determined in this manner is not necessarily a place that can be reached in real space, and may overlap with an area where buildings exist, for example. Therefore, a map application in the real space is referenced to determine a position that can be reached close to position Pe' (for example, on a road), and this is set as the target section end point Pe". Then, the map application is referenced to determine the route from the section start point Ps' to the section end point Pe", and the course in the real space is set.

[0039] In step S23b, it is determined whether the user has approached the section end point Pe". The moved position of the user in real space is detected by the position detection unit 7a. Then, when the moved position reaches the vicinity of the section end point Pe", for example, within a range of 10% of the section distance from the section end point Pe", it is determined that the user has approached the section end point Pe".

[0040] If the result of the determination in step S23b is that the section end point is being approached, in step S23c, the registered image at the corresponding point in the other space is displayed on the display unit 12a. After this, in step S23d, it is repeatedly determined whether the section end point Pe" on the course in the real space has been reached. If it is determined that the section has ended, that position is set as the start point of the next section in the real space and the process returns to step S23.

[0041] As a condition for determining the end of a section in step S23d, if there is a branch point on the map application and the branch point is not a dead end, the branch point is regarded as the end point of the section. Point P213 in Fig. 4 corresponds to this case. Other conditions for determining the end of a section include when the section distance in real space reaches a predetermined distance even if it is not a branch point, or when it is a point specified by the user. Points P212 and P214 in Fig. 4 correspond to this case.

[0042] In addition, the real-space course 27a can be automatically created by performing a simulation run on a map application. Furthermore, in order to create a real-space course 27a that matches the user's preferences from the course 26a in another space, the real-space course automatically created by the process of Fig. 6 may be manually corrected before or during the movement. For example, when the user arrives at the point P213, a round-trip course 39a to the point P216 outside the automatically created course 27a may be added.

[0043] Furthermore, when the azimuth difference between the traveling direction on the course 26a in another space and the traveling direction on the course 27a in the real space reaches or exceeds a predetermined value, the audio output unit 13 may notify the user by voice or a message may be displayed on the display unit 12a. This allows the user to move along the course while being aware that the difference between the course in another space and the course in the real space has become large due to the geographical constraints of the real space.

[0044] Furthermore, when it is detected that the user's position has deviated from the course 27a in the real space by a predetermined value or more, an alarm may be issued by the audio output unit 13 or a warning message may be displayed on the display unit 12a, thereby enabling the user to move along the course 27a in the real space without making any mistakes.

[0045] In the above description, the image captured by the imaging device 3 is registered in the Web server 4, and the image display device 2 retrieves and displays the registered image from the Web server 4. On the other hand, when the user himself captures and displays the image, the image can be captured and stored on the image display device 2. Therefore, the imaging device 3 and the Web server 4 can be omitted.

[0046] According to this embodiment, when a user moves along a course in real space, the scenery of the corresponding position in another space can be viewed as a still image or a moving image, so that the user can experience a sense of realism as if he or she were actually moving through the other space. In addition, the course in the real space along which the user moves is not only similar to the course in the other space, but is also flexibly set according to the geographical conditions such as roads in the real space, making it highly practical. Example 2

[0047] In the second embodiment, setting of a course in real space will be described in a case where the course in another space has an altitude difference and the course in real space is flat.

[0048] FIG. 7 is a diagram showing an example of a course in another space with altitude differences. The other space is undulating, and the altitude differences are represented by contour lines 34 (shown by dotted lines), with altitude [m] written on each contour line. The course 26b in the other space (shown by solid arrows) is a course with altitude differences from the start point P121 to the end point P126. Point P122 has the highest altitude, and point P123 has the lowest altitude. Points P124 and P125 are highlands. Therefore, the walking speed decreases in the uphill section (points P121 to P122, points P123 to P124). Conversely, the walking speed increases in the downhill section (points P122 to P123, points P125 to P126). There is a hut 33 at point P124, and the photographed image of the hut is registered. The same applies to the photographed objects at other points, but the description will be omitted here.

[0049] Image data captured at each point is registered in the Web server 4 in the same manner as in the first embodiment, but as shown in the additional information 17 in FIG. 5B, altitude is added to the information on the capture position obtained by the position detection unit 7b at the time of capture. The altitude information can also be measured by an altitude sensor. Alternatively, the altitude value of the position can be read from map information.

[0050] FIG. 8 shows an example of a flat course 27b without altitude difference in the real space, which is associated with the course 26b in the other space in FIG. 7. Since there is no altitude difference in the real space, the contour lines are omitted. The course 27b in the real space is from the start point P221 to the end point P226, and the direction of each section between adjacent points corresponds to the direction of each section of the course 26b in the other space with a predetermined direction difference. However, the distance of each section in the real space is set in consideration of the fact that the walking speed changes due to the altitude difference in the other space. In other words, the position information and altitude information of each point of the course 26b in the other space are obtained from the Web server 4, and the walking time of each section is assumed. Then, intermediate points are set on the course 27b so that the walking time for each section of the course 27b in the real space corresponds to the walking time for each section of the course 26b in the other space at a predetermined scale.

[0051] Specifically, the distance is increased for sections corresponding to uphill sections (points P221-P222, points P223-P224), and decreased for sections corresponding to downhill sections (points P222-P223, points P225-P226). When the user approaches a point of each section (for example, point P224), the position detection unit 7a detects it, and similar to the first embodiment, a registered image of another space (for example, 24g) is displayed on the display unit 12a.

[0052] In this way, in Example 2, by taking into account the altitude difference in the other space and correcting and setting the distances between points in the real space so as to cancel out the difference in travel time of the user due to the altitude difference, it is possible to move through the real space with a physical sensation that is closer to the course in the other space in terms of the time required and fatigue level.

[0053] Here, we have assumed that the course in another space has an altitude difference and the course in real space is flat. However, conversely, if the course in another space is flat and the course in real space has an altitude difference, or if the altitude difference between the two courses is different, the course in real space can also be set taking walking time into consideration. Example 3

[0054] In the third embodiment, an image display device 2 that allows a user to run a circular course in a real space while viewing an image of the course in another space will be described. Here, a case where a smartphone 40 is used as the image display device 2 will be described.

[0055] 9 is a diagram showing a course 26c in another space. Here, a course used in a road race is assumed, starting at point P131 and finishing at point P135. A building 35 is located at intermediate point P132, and a photographed image of the building is registered in the Web server 4.

[0056] FIG. 10 is a diagram showing the course 27c in the real space. Here, the course is a lap course around a track for athletics. The shape of the course 27c in the real space is not related to the shape of the course 26c in the other space, and points are set at positions corresponding to the running distance. For example, the course starts at point P231 on the start line 38, and the goal is the same point P235 after two laps around the track. The scale is calculated from the ratio of the total length of the course 26c in the other space to the course 27c in the real space, and the passing points on the way of the course 27c in the real space are automatically determined as positions at a distance obtained by multiplying the distance from the start of the course 26c in the other space to the passing points by the scale. In this method of calculating the distance, the image display device 2 itself must measure the running distance of the user on the course 27c in the real space. As an alternative method, a beacon is installed at the passing point position on the course 27c in the real space, and the reaching of the passing point is detected by a received signal from the beacon.

[0057] Here, the latter method using beacons will be described. Assuming a race involving multiple people, beacons 36a to 36d are placed at the start / finish and each passing point on the course 27c in real space so that position information can be measured accurately. The receivable range where signals from the beacons can be received at a certain threshold or higher is indicated by a dashed line 37. The user starts at point P231 (beacon 36a) and proceeds counterclockwise, passes point P232 (beacon 36c), and then completes one lap at the same point as the start (beacon 36a). The user then passes point P233 (beacon 36b) and point P234 (beacon 36d), completes two laps at point P235 (beacon 36a) where the finish line 38 is drawn, and finishes.

[0058] 11 is a diagram showing a state of a user wearing an image display device. Here, a smartphone 40, which is an image display device 2, is attached to the body of a user 20 using a fixture 41 so that the display screen can be viewed hands-free. The smartphone 40 detects signals emitted by the beacons 36a to 36d by using a Bluetooth (registered trademark) function as a position detection unit 7a, and can know the current position. When going around the track multiple times as in this example, it is possible to determine which of points P231 to P235 has been reached by combining the number of times the beacon signal has been received (i.e., the number of laps).

[0059] An application that associates points P231 to P235 on course 27c in the real space with points P131 to P135 on course 26c in another space is installed in smartphone 40. Image data on course 26c in another space is either downloaded in advance from Web server 4 via communication network 5, or is kept in a state where it can be downloaded from Web server 4 at any time in response to a request from the application.

[0060] A specific example will be described in which a large number of users race on a course 27c in real space. After launching an application on a smartphone 40, a first group of athletes waits in front of the starting point P231, within the reception range 37 of a signal from a beacon 36a, and starts the race at 00 seconds per minute. After this start, a second group of athletes waits in front of point P231 in the same way. This method enables a wave start in which the athletes are divided into groups of several people and start. The starter can either send a start signal according to the posted time at 00 seconds per minute, or just watch over the athletes to make sure they do not start early.

[0061] When each runner passes through points P233 and P234 in the real space and approaches the goal point P235, the smartphone 40 measures the time when the signal received from the beacon 36a reaches a maximum as the goal time. The goal time of each runner is transmitted to a counting device (not shown) of the race organizer via, for example, the wireless communication network 5 and is used to count the race rankings.

[0062] 12A to 12C are diagrams showing examples of the display screen of the smartphone 40. Here, the display when the user approaches point P232 in the real space corresponding to point P132 in the other space is shown. In the course 27c in the real space, the user goes straight near point P232, but in the course 26c in the other space, the user turns right at point P132. As the user runs, the image 24h in FIG. 12A showing a right turn in the other space is changed to the image 24i in FIG. 12B and displayed on the display unit 12a. These images include buildings 35a, 35b and roads 42a, 42b photographed in the other space. In the other space, the user turns right at this point, but in the real space, the image is displayed as one in which the traveling direction is always seen as a front. In other words, since the correspondence relationship between the traveling direction between the other space and the real space is not fixed, the course shape of the real space relative to the other space can be freely set. Note that the correspondence relationship between the traveling direction is fixed, for example, in a state in which the traveling direction is always shifted by 30 degrees clockwise.

[0063] FIG. 12C shows a display example that reflects the user's front direction azimuth and posture (azimuth and elevation angles of the user's line of sight). FIG. 12B shows a display image 24j when the user twists his / her upper body to the left and looks down while passing by an image 35b of a building in another space. In normal image display of a smartphone, even if the image has a wide viewing angle such as a 360° viewing angle, only images within the range of the viewing angle of the smartphone orientation that matches the traveling direction are displayed. Therefore, the orientation / posture detection unit 11a of the smartphone 40 can detect the movement of the user's upper body and display images within the range of the viewing angle of the viewing direction. As a result, in the display image 24j, an image 35c of a building is displayed in front and an image 42c of a road is displayed below it.

[0064] According to the third embodiment, the present invention can be applied to a race in which runners run around the same course while looking at different scenery. In particular, in a long-distance race with many laps, runners tend to get bored, but being able to run while looking at different scenery from time to time has the effect of eliminating boredom.

[0065] FIG. 11 shows an example in which smartphone 40 is attached to the body using fixing device 41 so that it can be viewed hands-free. However, typically, smartphone 40 is placed in a pocket of a backpack or the like, and smartphone 40 can be set to emit an alarm when approaching each point. When the user hears the alarm, it means that the user is approaching the next point, so the user can pick up smartphone 40 and look at the screen of smartphone 40.

[0066] In addition, in the third embodiment, not only still images but also videos can be displayed on the display unit. In the above description, a beacon is used, but at the very least, it is sufficient to be able to play videos only within the range where the beacon is received. Alternatively, if it is desired to display outside the beacon receiving range, the running speed of the user between adjacent beacons can be calculated, and the playback speed of the captured video can be changed accordingly.

[0067] In addition, if the real space is a circular course and the passing points can be identified only by the position detection unit without using beacons, the video can be aligned based on the position coordinates obtained from the position detection unit and played back over the entire course. In that case, position information can be added to the captured video at regular time intervals or when passing through a point.

[0068] In the above embodiments, the user moves on foot or by running, but the user may also move by bicycle, a car that the user does not drive, or a public means of transportation. The user may also change the intermediate points or the goal of the course during the movement. When displaying images on the image display device while moving within each section, the images may be images of scenery from a car window that have been photographed in advance in a different space and registered in a state accessible from the image display device.

[0069] Although several embodiments have been described above, the present invention is not limited to the above-described embodiments, and various modified examples are included. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. It is also possible to add the configuration of one embodiment to the configuration of another embodiment. It is also possible to add, delete, or replace other configurations with respect to part of each embodiment. [Explanation of symbols]

[0070] 1: image display system, 2: image display device, 3: imaging device, 4: web server, 5: communication network, 6a, 6b: imaging elements, 7a, 7b: position detection unit, 8a, 8b, 8c: memory, 9a, 9b, 9c: communication interface, 10a, 10b, 10c: control unit, 11a, 11b: orientation / attitude detection unit, 12a, 12b: display unit, 13: audio output unit, 19: head mounted display (HMD), 20: user, 40: smartphone.

Claims

1. An image display method for displaying image information of a second space different from the first space by an image display device of a first space, comprising: a course information acquisition step of acquiring course information including at least one set of section start point information and section end point information in the second space and image information related to the section end point information; a map information display step of displaying map information of the first space; a start point position input step of accepting a designation of a start point position on the map information; a recommended course and image display point setting step of setting a recommended course and the image display points along which the image display device can move in the first space based on the map information, the positional relationship in the second space between the section start point information and the section end point information and the positional relationship in the first space between the start point position and the image display point being on an approximate scale; a position information acquiring step of acquiring position information of the image display device in the first space; an image information display step of displaying image information related to the section end point information when the position information is in the vicinity of the image display point when the image display device moves on the recommended course from the start point position; An image display method comprising:

2. The image display device displays image information of a second space different from the first space in an image display device of a first space, a position information acquisition unit that acquires position information of the image display device in the first space; a communication interface that acquires course information from an external device, the course information including at least one set of section start point information and section end point information in the second space, and image information related to the section end point information; an input interface that accepts designation of a start point position in the first space; a display unit for displaying map information and the image information; A control unit; Equipped with The control unit is Displaying map information of the first space on the display unit; a positional relationship in the second space between the section start point information and the section end point information and a positional relationship in the first space between the start point position and an image display point are approximated in scale, and a recommended course along which the image display device can move in the first space and the image display points are set based on the map information; When the image display device moves on the recommended course from the starting point position, if the position information is in the vicinity of the image display point, image information related to the section end point information is displayed on the display unit.

1. An image display device comprising:

Images