Apparatus for displaying virtual object and display method thereof

Abstract

The invention provides an apparatus for displaying a virtual object and a display method thereof, which can improve the presence of a user even under the condition that the virtual object cannot be seen. To this end, the method for displaying a virtual object comprises: a map information processing step in which a first map element corresponding to position information and a second map element corresponding to a predetermined pattern are extracted from map data; a virtual object processing step of arranging a virtual object on a real object of a real space corresponding to the first map element; an auxiliary information processing step of generating a map element object corresponding to the second map element; a display processing step in which, among the virtual object and the map element object, a drawing method is made different for a portion located in front of the real object and a portion located behind the real object; and a display step: displaying the virtual object and the map element object processed by the display processing step in a real space in an overlapping manner.

Description

Technical Field

[0001] The present invention relates to a device for displaying a virtual object such as a head mounted display and a display method thereof. Background Art

[0002] Mixed reality (MR) technology, which superimposes virtual objects (virtual objects) generated by CG (Computer Graphics) on real space, has been widely used in games, sports, telemedicine, maintenance work, etc.

[0003] As a device that displays a virtual object, there are, for example, information processing devices such as HMD (Head Mounted Display), HUD (Head Up Display) mounted on a vehicle or an airplane, a car navigation system, and a smartphone.

[0004] In MR technology, for example, in the case of an HMD, a virtual object drawn in response to the movement of the HMD is displayed as an image of the virtual space on the display unit, superimposed on the image of the real space seen through the display unit. Alternatively, in the HMD, an image of the real space captured by a camera and a virtual object are superimposed and displayed on a non-perspective reflective display unit.

[0005] As a background technology in this technical field, there is Patent Document 1. Patent Document 1 discloses a technology in which a user of an HMD is a runner and a virtual object as a virtual runner is displayed on the HMD to enhance the sense of running and make it easy for the user to grasp information about the running.

[0006] Prior art literature Patent Literature Patent Document 1: Japanese Patent Application Publication No. 2011-67277 Summary of the invention

[0007] Technical problem to be solved by the invention In MR technology, the distance between the user's real object and the virtual object in real space is compared, and an occlusion process is performed to cover part of the virtual object that is farther away than the real object to achieve a three-dimensional viewing effect. Therefore, if the virtual object is completely covered, the user cannot see the virtual object.

[0008] Patent Document 1 has a problem that if the user cannot see the virtual object continuously, the user cannot recognize the virtual runner for a long time, making it difficult to achieve the purpose of improving the sense of presence. As such, Patent Document 1 does not consider how to deal with the situation where the virtual object cannot be seen due to occlusion processing between the virtual object and the real object.

[0009] The present invention is made in view of the above situation, and its purpose is to provide a device for displaying virtual objects and a display method thereof, which can improve the user's sense of presence even when the virtual objects cannot be seen.

[0010] Technical solutions to the problem The present invention cites an example of a method for displaying a virtual object, which is composed of: a map information processing step, extracting a first map element corresponding to position information and a second map element corresponding to a specified pattern from map data; a virtual object processing step, configuring a virtual object on a real object in a real space corresponding to the first map element; an auxiliary information processing step, generating a map element object corresponding to the second map element; a display processing step, making the drawing method different for the part located in front of the real object and the part located behind the real object in the virtual object and the map element object; and a display step, superimposing and displaying the virtual object and the map element object processed by the display processing step on the real space.

[0011] Effects of the Invention According to the present invention, a device for displaying a virtual object and a display method thereof can be provided, which can cope with a situation where the virtual object cannot be seen and can improve the user's sense of presence. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A 2 is a diagram showing the appearance of an HMD in the embodiment.

[0013] Figure 1B FIG. 4 is a diagram showing the appearance of another HMD in the embodiment.

[0014] Figure 2 is a functional block diagram of the HMD in the embodiment.

[0015] Figure 3 4 is a block diagram of the hardware structure of the HMD in the embodiment.

[0016] Figure 4 is a flowchart of the MR process in the embodiment.

[0017] Figure 5 4 is a flowchart of a virtual object generation process in the embodiment.

[0018] Figure 64 is a flowchart of a configuration process of a virtual object in an embodiment.

[0019] Figure 7 1 is a flowchart of a process for generating a route object in the embodiment.

[0020] Figure 8 This is an example of display of a visible virtual object in the embodiment.

[0021] Fig. 9 This is an example of displaying a virtual object when it cannot be seen in the embodiment.

[0022] Fig.10 This is an example of configuring a virtual object at a specific position in the embodiment.

[0023] Fig.11 This is a display example when the virtual object in the embodiment is located behind the user.

[0024] Fig.12 This is a display example when the virtual object in the embodiment is outside the viewing range. DETAILED DESCRIPTION

[0025] The embodiments of the present invention are described below with reference to the accompanying drawings. In this embodiment, an HMD is used as an example of a device for displaying a virtual object.

[0026] Example Figure 1A : is the appearance structure diagram of the HMD in this embodiment. Figure 1A 1 is an HMD, 10 is a camera, 11 is a distance measuring sensor, 12a and 12b are a pair of left and right projection units (projectors), 13 is a semi-transmissive screen, 14 is a speaker, 15 is a microphone, 16 is a housing, 17 is a support unit, and 18 is a control unit.

[0027] The user of the HMD 1 wears the HMD 1 on his / her face using the housing 16 and the support 17. The camera 10 captures the real space in front of the HMD 1, and the distance measuring sensor 11 measures the distance between the HMD 1 and a real object in the real space captured by the camera.

[0028] The projection units 12a, 12b and the screen 13 constitute a display unit of the HMD 1. The projection units 12a, 12b project an image of a virtual object for the left eye and an image of a virtual object for the right eye, respectively, onto the screen 13, and three-dimensionally display the projected image, i.e., the virtual object, as if it were located at a predetermined distance in the real space.

[0029] In this embodiment, an optical see-through type HMD is described in which the HMD user views an image of the real space in front of him through the screen 13 , but a video see-through type HMD may be used in which an image of the real space captured by a camera is displayed on the screen 13 for viewing.

[0030] Here, in the display of the HMD, an occlusion process is performed according to the front-to-back relationship of the distance between the real object and the virtual object. The occlusion process refers to processing the drawing data of the virtual object so that a part of the virtual object appears to be hidden by a part of the real object when a part of the real object is located in front of a part of the virtual object (that is, closer to the user), thereby making it possible to display an image of the MR space with a sense of perspective.

[0031] The control unit 18 imports the image of the real space captured by the camera 10 and supplies it to the internal memory and CPU. In addition, the HMD 1 has a built-in sensor group such as GPS, gyroscope, orientation, and acceleration sensors. The control unit 18 detects the position and movement of the HMD based on the information from the sensor group. Furthermore, the control unit 18 generates images to be projected by the projection units 12a and 12b and sounds to be output to the speaker 14. The control unit 18, the camera 10, the distance sensor 11, the speaker 14, and the microphone 15 are arranged in the housing 16. In addition, their arrangement locations are not limited to Figure 1A .

[0032] Figure 1B FIG. 4 is a diagram showing the appearance of another HMD in this embodiment. Figure 1B For Figure 1A The same functions are denoted by the same symbols, and their description is omitted. Figure 1B Zhongyu Figure 1A The difference is that the control unit 18 is divided into 18a and 18b.

[0033] Figure 1B In the embodiment, the control unit 18a is connected to the control unit 18b via a wired or wireless interface. The control unit 18b is, for example, a general information terminal such as a smart phone or a smart watch. Figure 1B The structure of HMD2 shown in FIG. Figure 1A Since a part of the control unit 18 shown in the figure can be provided as the control unit 18 a , there is an advantage that the HMD 2 can be made smaller and lighter.

[0034] Figure 2 : is a functional block diagram of the HMD in this embodiment. Figure 2 express Figure 1A In the case of HMD 1, the functional block diagram of the control unit 18 is particularly shown in detail. Figure 1AThe same modules are marked with the same symbols and their descriptions are omitted. Figure 1A The projection units 12A and 12B are collectively referred to as the projection unit 12, and the microphone, speaker, etc. are omitted.

[0035] Figure 2 In the figure, the control unit 18 includes an image recognition processing unit 20 , a communication unit 21 , a map information processing unit 22 , a virtual object processing unit 23 , a display processing unit 24 , a position detection processing unit 25 , an auxiliary information processing unit 26 , and an overall control unit 27 .

[0036] The position detection processing unit 25 includes GPS, orientation, gyro sensors, etc., and detects the position and orientation of the HMD. The overall control unit 27 obtains the distance between the HMD (= user) and the virtual object (= competitor) calculated by the virtual object processing unit 23 based on the information detected by the position detection processing unit 25. Then, the overall control unit 27 specifies the position and range to the map data server (not shown) through the communication unit 21 based on this information, and requests download. The map data downloaded by the communication unit 21 is input to the map information processing unit 22.

[0037] The map information processing unit 22 extracts map elements such as roads that are pre-set as running routes from the map data, and outputs the extracted information to the virtual object processing unit 23. In addition, the map information processing unit 22 retains or obtains the user's scheduled running route data via the communication unit 21 for extracting map elements from the map data.

[0038] The image recognition processing unit 20 inputs the captured image of the camera 10 and the distance data of the distance measuring sensor 11 , recognizes real objects such as roads and buildings in the real space captured by the captured image, and adds distance data to feature points of the real objects.

[0039] The virtual object processing unit 23 calculates the position of the competitor based on the competitor's running pace information, and then generates image data of the virtual object. The image data of the virtual object can also be obtained from an external server via the communication unit 21. Furthermore, the virtual object processing unit 23 obtains the current position of the user from the position detection processing unit 25, extracts information according to the running route, etc., and configures the virtual object accordingly. Furthermore, by sending the position of the virtual object to the position detection processing unit 25, the range of the map data to be downloaded is determined. The download range of the map data is constantly changing, but the difference between the downloaded range can be updated to suppress the downloaded data capacity to a small level.

[0040] The auxiliary information processing unit 26 extracts information based on the running route, etc. and performs processing, such as generating a route object of auxiliary information that conforms to the route data, etc. Furthermore, the display processing unit 24 inputs the route object from the auxiliary information processing unit 26, the virtual object from the virtual object processing unit 23, and the real object from the image recognition processing unit 20, performs occlusion processing between the virtual object and the route object and the real object, and sends the virtual object image and the route object image to the projection unit 12 for display on the screen 13.

[0041] Figure 3 is a block diagram of the hardware structure of the HMD in this embodiment, showing Figure 1A Case of HMD1. Figure 3 In Figure 1A The same modules are marked with the same reference numerals and their descriptions are omitted. Figure 1B The control units 18a and 18b are shown.

[0042] Figure 3 The control unit 18 a includes a sensor group 28 including a GPS, an orientation sensor, a gyro sensor, etc., and an interface unit 29 , and the control unit 18 b includes a communication unit 30 , a CPU 31 , a RAM 32 , a Flash ROM (FROM) 33 , and an interface unit 36 ​​.

[0043] The communication unit 30 of the control unit 18b selects appropriate processing from several communication processing such as 4G, 5G mobile communication, wireless LAN, etc., connects the HMD to the network, and downloads map data from an external server. Furthermore, the FROM 33 includes a basic program 34 and an MR processing program 35 as processing programs. These processing programs are loaded into the RAM 32 and software processing is performed by the CPU 31, thereby achieving Figure 2 The various functions shown in the figure. Furthermore, the data required to execute the processing program is stored in FROM33. FROM33 can be a single storage medium as shown in the figure, or it can be composed of multiple storage media. Furthermore, it can also be a non-volatile storage medium other than FlashROM.

[0044] in addition, Figure 3 In the adoption of Figure 1A In the case of the structure of the HMD 1 shown in the figure, which includes the control unit 18 in which the control units 18 a and 18 b are integrally configured, the interface units 29 and 36 may not be provided.

[0045] In addition, Figure 1B In the case where the control unit 18 shown in the figure is divided into the control units 18a and 18b, Figure 3In the embodiment, the control unit 18b is separated from the HMD, and the control units 18a and 18b are connected via the interface units 29 and 36. In this case, the interface units 29 and 36 may be wired interfaces such as USB (registered trademark), or wireless interfaces such as wireless LAN and Bluetooth (registered trademark). As described above, if the control unit 18 is divided into the control units 18a and 18b, the HMD only needs to be provided with the control unit 18a, which is a part of the control unit 18, so the HMD can be miniaturized and lightweight.

[0046] Figure 4 This is a flowchart of the MR process in this embodiment. Figure 4 In step S10, the process starts, and step S11 is Figure 2 In step S11, the current position of the HMD is detected based on the data from the sensor group 28, and a request for downloading map data is output together with the position of a virtual object described later. In addition, in step S12, the current position of the HMD and the position of the virtual object are saved as a running record at regular intervals.

[0047] Steps S13 to S15 are Figure 2 The map information processing unit 22 described in the above is used for map information processing. In S13, map data is downloaded, in S15, route data is read, and in S14, the read route data is referred to and roads set as running routes are extracted as map elements.

[0048] Step S16 is a step in which the camera processing unit performs camera shooting and distance measurement, and imports camera images and distance data.

[0049] Steps S17 and S18 are Figure 2 In S17, real objects such as roads and buildings are recognized from the captured image which is an image of the real space, and in S18, distance data is associated with feature points of the recognized real objects.

[0050] Steps S20 to S23 are Figure 2 The virtual object processing of the virtual object processing unit 23 described in the above is performed. In S21, the saved running pace data is read, in S20, the running distance of the virtual object is calculated, and in S22, an image of the virtual object (= competitor) is generated. The size of the image of the virtual object changes according to the direct viewing distance from the HMD, and the orientation changes according to the direction relative to the HMD. Furthermore, in S23, the virtual object is arranged on the road of the extracted running route. Furthermore, the arrangement position of the virtual object is sent to the position detection processing S11, and the range of the map data to be downloaded is determined.

[0051] Step S24 is Figure 2 The auxiliary information processing of the auxiliary information processing unit 26 described in the above description generates a route object (= auxiliary information object) corresponding to the extracted running route. The route object is an object representing a road along the running route, and may be a three-dimensional object reflecting distance data. Furthermore, as an auxiliary information object, a distance information object to which numerical data representing distance information is attached may be added like a milestone. Furthermore, in the case where the competitor is outside the user's field of vision, a simulated route object may be generated instead of the route object. The simulated route object may only reflect the sense of distance from the competitor. In addition, these route objects, distance information objects, simulated route objects, etc. are objects associated with a map, and are therefore also referred to as map element objects.

[0052] Steps S25 to S27 are Figure 2 In step S25, an occlusion process of the real object and the virtual object (including the route object) is performed. In the occlusion process, the real object and the virtual object in the real space are compared based on the distance to the user, and the part of the virtual object that is closer to the real object and the part of the virtual object that is farther from the real object are distinguished.

[0053] When viewed from the user, the portion of the virtual object that is located farther than the real object is the portion that is hidden by the real object and cannot be seen. However, if the state of being unable to see the virtual object continues to occur, the user cannot recognize the presence of the competitor, which no longer helps to increase the user's enthusiasm by allowing the user to recognize the competitor while running. Therefore, in S26, the portion of the virtual object that is hidden by the real object and cannot be seen is depicted differently from the portion of the virtual object. The different depiction is implemented, for example, by using different color schemes.

[0054] In step S27, the virtual object and the auxiliary information object are output and projected on the display unit of the HMD. Then, in step S28, it is confirmed whether the program is finished. If it is not finished (No), it returns to S10. If it is finished (Yes), it ends in S29.

[0055] Figure 5 yes Figure 4 Flowchart of the virtual object generation process (S22) in FIG. Figure 5 In S50, it is determined whether the competitor is ahead of the user. If the competitor is ahead ("yes"), a virtual object is generated in the real space in S51. If the competitor is not ahead ("no"), a rearview mirror object is generated in S52, and the virtual object is configured in the rearview mirror object.

[0056] Figure 6 yes Figure 4Flowchart of the configuration process (S23) of the virtual object in FIG. Figure 6 In S60, the width of the road is determined for the road element obtained as the extracted information from the map data. If the road width is wide ("yes"), it is further determined in S61 whether there is a sidewalk. If there is a sidewalk ("yes"), a virtual object is arranged on the sidewalk in S62. If there is no sidewalk ("no"), a virtual object is arranged on the side of the road in S63. If the road width is narrow ("no") in the determination of S60, a virtual object is arranged in the road in S64. The arrangement of the virtual objects in S62 to S64 is arranged at the position of the running distance measured along the running route.

[0057] Figure 7 yes Figure 4 This is a flowchart of the process of generating a route object ( S25 ) in FIG. Figure 7 In S70, it is determined whether the competitor is significantly ahead, and in S71, it is determined whether the route is set to return in the direction opposite to the running direction of the user, and both are states where the competitor is outside the user's field of vision. If the competitor is within the user's field of vision (both S70 and S71 are "No"), a route object along the road is generated in S72. If the competitor is outside the user's field of vision (either S70 or S71 is "Yes"), a simulated route object is generated in S73. The simulated route object is a road object that only reflects the sense of distance between the competitor and the competitor and has nothing to do with the road in the real space.

[0058] Next, use Figure 8 to Figure 12 A specific example of display in this embodiment will be described. Figure 8 This is an example of a display in which the competitor is ahead of the user and is not obscured by a real object and can be seen. Figure 8 In FIG. 5 , the left side is a display image 50 of the HMD viewed by the user, and the right side is a corresponding map 60. Furthermore, Figure 8 In FIG. 5 , 51 and 61 are users, 52 and 62 are competitors (virtual objects, especially 52 is an avatar), and 63 is a running route. Figure 8 On the left side, the user 51 is not included in the display image 50, but the position of the user is recorded for reference. In addition, an image of a competitor 52 as a virtual object is displayed.

[0059] Fig. 9 This is a display example in which the competitor is ahead of the user and is hidden by a building that is a real object and cannot be seen. Fig. 9 In Figure 8 The same structures are denoted by the same symbols, and their description is omitted. Fig. 9 and Figure 8The difference is that a route object 54 and a distance information object 55 are added as auxiliary information objects.

[0060] exist Fig. 9 On the left, the virtual contestant 52 is hidden by the real building and cannot be seen. Figure 8 The running route is shown in a different manner, for example, by changing the display color and displaying it differently compared to the case where the running route is being run, so that the user does not lose track of the runner. In addition, the route object 54 is also partially covered by the building as a real object, but the covered portion is displayed in a different manner than the uncovered portion, so as to guide the user to the running route. In addition, the distance information object 55 is displayed so that the user can get a correct sense of distance.

[0061] In this way, by changing the way the masked part of the competitor, i.e., the virtual object, is depicted, and by displaying the virtual route using the running route as the route object and changing the way the masked part of the route object is depicted, the user's sense of presence can be enhanced, which can help improve the runner's performance.

[0062] Fig.10 This is an example of placing a virtual object at a specific location. Fig.10 In Figure 8 The same structures are denoted by the same symbols, and their description is omitted. Fig.10 In the case where the road width of the running course is wide and a sidewalk 56 is provided, the competitor 52 as a virtual object is arranged on the sidewalk 56 in such a way that obstacles such as vehicles on the lane do not overlap with the competitor 52 as a virtual object.

[0063] Fig.11 This is an example of a display when a competitor is lagging behind the user. Fig.11 In Figure 8 The same structures are denoted by the same symbols, and their description is omitted. Fig.11 In the embodiment, a rearview mirror object 57 is displayed at a specific position, for example, at the upper part of the display image 50 of the HMD. A virtual object of a competitor and a distance information object are displayed in the rearview mirror object 57, so that it is easy to recognize that the competitor is lagging behind the user.

[0064] Fig.12 is a situation where the competitor is significantly ahead of the user, or Fig.12 The map on the right shows a case where the running route is winding and the competitor is outside the user's field of view (the display range of the display image of the HMD), in other words, it is an example of a display when the virtual object is outside the viewing range. Fig.12 In Fig. 9 The same structures are marked with the same symbols and their descriptions are omitted. Fig.12As shown on the left side, a simulated route object 58 including a route object 54 is displayed, and a competitor 52 as a virtual object, a distance information object 55, etc. are displayed on the simulated route object 58. The simulated route object 58 is a three-dimensional object that gradually moves away from the user 51, for example, using perspective. Then, the competitor 52 as a virtual object is arranged on the simulated route object 58 at a position after equivalent transformation of the distance corresponding to the distance ahead of the competitor, so that the distance when the user observes the competitor can reflect the actual distance. In this way, the actual distance sense of the virtual route can be improved, which helps to improve the performance of the runner. In addition, only one of the simulated route object 58 and the competitor 52 as a virtual object can be displayed.

[0065] As described above, the display method of virtual objects in this embodiment includes display processing, position detection processing, map information processing, virtual object processing, auxiliary information processing and image recognition processing. Position detection processing is used to determine the user's position on the map, and map information processing is used to extract roads that are pre-set as running routes, etc. Virtual object processing is used to generate virtual objects such as competitors. For virtual objects, a relative position with respect to the user and a configuration position of the virtual object on the running route are assigned. Image recognition processing is used to identify real objects from images in real space, and display processing is used to determine whether the virtual object is within the user's viewing range, and to display virtual objects outside the viewing range in a different manner from virtual objects within the viewing range. Furthermore, auxiliary information processing is used to display information such as running routes extracted by map information processing as auxiliary information objects.

[0066] In addition, the device for displaying virtual objects in this embodiment includes a display processing unit, a position detection processing unit, a map information processing unit, a virtual object processing unit, an auxiliary information processing unit, a camera unit, and an image recognition processing unit. The position detection processing unit is used to determine the user's position on the map, and the map information processing unit is used to extract roads that are pre-set as running routes. The virtual object processing unit is used to generate virtual objects such as competitors. For virtual objects, a relative position with the user and a configuration position of the virtual object on the running route are given. The image recognition processing unit is used to identify real objects from the image of the real space obtained by the camera, and the display processing unit is used to determine whether the virtual object is within the user's viewing range, and display virtual objects outside the viewing range in a different way from virtual objects within the viewing range. Furthermore, the auxiliary information processing unit is used to display information such as running routes extracted by map information processing as auxiliary information objects.

[0067] As described above, according to the present embodiment, a device for displaying virtual objects and a display method thereof can be provided. In an MR space realized by the fusion of real space and virtual space, even when the virtual object cannot be seen, the user will not lose sight of the virtual object by changing the depiction method, thereby improving the user's sense of presence. In addition, for virtual objects outside the viewing range, the user's sense of presence can also be improved by depicting them as simulated objects.

[0068] The above embodiments are described, but the present invention is not limited to the above embodiments, and includes various modifications. For example, the present invention is not limited to having all the structures described in the embodiments. In addition, as a device for displaying a virtual object, in addition to an HMD, it can also be a smartphone or a car navigation system.

[0069] In addition, in this embodiment, the functions of the above-mentioned embodiment are described as software processing, but part or all of them can also be implemented by hardware, for example, by integrated circuit design. In addition, the scope of software implementation is not limited, and hardware and software can also be used at the same time. In addition, part or all of each function can also be implemented by a server. In addition, as long as the server can perform functions in collaboration with other components via communication, such as a local server, cloud server, edge server, network service, etc., the method is arbitrary. In addition, information such as programs, tables, files, etc. that implement each function can be stored in a recording device such as a memory, hard disk, SSD (Solid State Drive), or a recording medium such as an IC card, SD card, DVD, etc., or can be stored in a device on the communication network.

[0070] In addition, the programs described in each processing example may be independent programs, or a plurality of programs may constitute one application program. In addition, the order of each processing may be switched and executed.

[0071] Description of Reference Numerals 1, 2: HMD, 10: camera, 11: distance measuring sensor, 12, 12a, 12b: projection unit, 13: screen, 18, 18a, 18b: control unit, 20: image recognition processing unit, 21, 30: communication unit, 22: map information processing unit, 23: virtual object processing unit, 24: display processing unit, 25: position detection processing unit, 26: auxiliary information processing unit, 27: overall control unit, 31: CPU, 33: FROM, 35: MR processing program, 50: display image, 60: map, 51, 61: user, 52: competitor (virtual object), 62: competitor, 63: running route, 54: route object, 55: distance information object, 56: sidewalk, 57: rearview mirror object, 58: simulated route object.

Claims

1. A method for displaying a virtual object, characterized in that: include: A map information processing step of extracting a first map element corresponding to the position information and a second map element corresponding to a prescribed pattern from the map data; a virtual object processing step of configuring a virtual object on a real object in a real space corresponding to the first map element; an auxiliary information processing step, generating a map element object corresponding to the second map element; A display processing step of making the drawing method different for the portion located in front of the real object and the portion located behind the virtual object and the map element object; and A display step is to display the virtual object and the map element object processed by the display processing step in a superimposed manner on the real space.

2. The method for displaying a virtual object according to claim 1, wherein: The virtual object is a competitor in the virtual space, The map feature object is a route object, In the display step, the competitor and the course object are displayed superimposed on the real space.

3. The method for displaying a virtual object according to claim 2, wherein: The map element object includes a distance information object. In the display step, the competitor, the route object, and the distance information object are displayed in a superimposed manner on the real space.

4. The method for displaying a virtual object according to claim 1, wherein: The virtual object is a competitor in the virtual space, The map element object is a distance information object. In the display step, the competitor and the distance information object are displayed superimposed at a specific position in the real space.

5. The method for displaying a virtual object according to claim 2, wherein: In the auxiliary information processing step, when the competitor is outside the display range of the image displayed in the display step, a simulated route object is generated as the map element object, In the display step, the competitor is arranged on the simulated course object and displayed in such a manner that a sense of distance of the competitor seen by a user corresponds to a distance between the user and the competitor.

6. A device for displaying a virtual object, characterized in that: include: a map information processing unit that extracts a first map element corresponding to the position information and a second map element corresponding to a prescribed pattern from the map data; a virtual object processing unit configured to arrange a virtual object on a real object in a real space corresponding to the first map element; an auxiliary information processing unit, which generates a map element object corresponding to the second map element; and A display processing unit that makes the drawing method different for the part of the virtual object and the map element object located in front of the real object and the part located behind the real object, and superimposes and displays the virtual object and the map element object obtained by making the drawing method different on the real space.

7. The device for displaying a virtual object according to claim 6, characterized in that: The virtual object is a competitor in the virtual space, The map feature object is a route object, The display processing unit displays the competitor and the course object in a superimposed manner on the real space.

8. The device for displaying a virtual object according to claim 7, characterized in that: The map element object includes a distance information object. The display processing unit displays the competitor, the course object, and the distance information object in a superimposed manner on the real space.

9. The device for displaying a virtual object according to claim 6, wherein: The virtual object is a competitor in the virtual space, The map element object is a distance information object. The display processing unit displays the competitor and the distance information object in a superimposed manner at a specific position in the real space.

10. The device for displaying a virtual object according to claim 7, wherein: The auxiliary information processing unit generates a simulated route object as the map element object when the competitor is outside a display range of the image displayed on the display processing unit. The display processing unit arranges the competitor on the simulated course object and displays the competitor in such a manner that a sense of distance of the competitor seen by a user corresponds to the distance between the user and the competitor.

Images