Information display device and display method

Abstract

To provide an information display device and a display method for the information display device that make menu display and the like easy to be viewed so as to be usable even when a display section is inclined.SOLUTION: An information display device having a display section comprises: an object data acquisition section that acquires information related to an AR object and generates the AR object; an additional object generation section that generates an additional object related to the AR object; an attitude detection section that detects an inclination of the information display device relative to a gravity direction in a real space; and a display control section that displays the AR object so that the AR object is inclined relative to the display section in accordance with the inclination of the information display device, displays the additional object so that the additional object is inclined relative to the display section in accordance with the inclination of the information display device when the inclination of the information display device is less than a predetermined value, and displaying the additional object using the same vertical reference as for the display section so that the additional object is not inclined relative to the display section regardless of the inclination of the information display device when the inclination is equal to or greater than the predetermined value.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to an information display device and a display method thereof.

Background Art

[0002] As one of information display devices, there is a head-mounted display device (hereinafter referred to as HMD: Head Mounted Display) that is worn on the head to view information. The HMD can superimpose and display an image of a virtual object (AR object: Argument Reality Object) such as an image or characters on the real space for visual recognition, and in recent years, its use in fields such as game applications and work support has been spreading.

[0003] Here, in order to create a state of being immersed in the virtual world, the virtual space that the user visually recognizes by wearing the HMD needs to be changed according to the movement of the user's head.

[0004] As the background art in this technical field, there is Patent Document 1. Patent Document 1 discloses an image processing device that presents a stereoscopic image by an HMD, including a reference image acquisition unit that acquires a reference image composed of a pair of left-eye image and right-eye image representing an object viewed from two viewpoints separated in the horizontal direction, an inclination angle acquisition unit that acquires an inclination angle of the vertical axis of the head of a user wearing the HMD from the vertical direction in a plane parallel to the screen, an image conversion unit that performs a conversion process of rotating the left-eye image and the right-eye image around a common axis in a virtual space including the object based on the inclination angle, and an output unit that outputs the data of the left-eye image and the right-eye image subjected to the conversion process to the HMD.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

[0006] Patent Document 1 describes a display image control system that rotates the image on the display unit in accordance with the user's head tilt angle. Therefore, when the user tilts their head, all images on the display unit as seen by the user are displayed at an angle. Consequently, if there is a menu display, for example, the menu display will also be displayed at an angle, and the issue of the menu display becoming difficult to read has not been taken into consideration.

[0007] In view of the above problems, the present invention aims to provide an information display device and a display method that are easy to use and that make menu displays and the like easy to see even when the display unit is tilted. [Means for solving the problem]

[0008] One example of the present invention is an information display device having a display unit, comprising: an object data acquisition unit that acquires information about AR objects and generates AR objects; an additional object generation unit that generates additional objects related to AR objects; an attitude detection unit that detects the tilt of the information display device with respect to the direction of gravity in real space; and a display control unit that displays AR objects in a tilted state on the display unit according to the tilt of the information display device, and displays additional objects in a tilted state on the display unit according to the tilt of the information display device if the tilt of the information display device is less than a predetermined value, and displays them without tilting on the display unit if the tilt is greater than or equal to the predetermined value, using the same vertical reference as the display unit regardless of the tilt of the information display device. [Effects of the Invention]

[0009] According to the present invention, it is possible to make menu displays and other information easily visible even when the display unit is tilted, thereby providing an easy-to-use information display device and its display method. [Brief explanation of the drawing]

[0010] [Figure 1] Block diagram showing the hardware configuration of the HMD in Example 1. [Figure 2] This shows the appearance and mounting diagram of the HMD in Example 1. [Figure 3] This is a functional configuration diagram of the HMD in Example 1. [Figure 4A] This is a conceptual diagram illustrating the viewing conditions of the virtual space using the HMD in Example 1. [Figure 4B] This figure illustrates the image displayed on the HMD's display unit in the state shown in Figure 4A. [Figure 5A] This is a conceptual diagram illustrating the view of the virtual space by the HMD when the user tilts their head 30 degrees in Example 1. [Figure 5B] This figure illustrates the image displayed on the HMD's display unit, based on the AR object, in the state shown in Figure 5A. [Figure 5C] This figure illustrates the image displayed on the HMD's display unit in the state shown in Figure 5A, with the HMD's display unit as the reference point. [Figure 6A] This is a conceptual diagram illustrating the view of the virtual space by the HMD when the user tilts their head 45 degrees in Example 1. [Figure 6B] This figure illustrates the image displayed on the HMD's display unit, based on the AR object, in the state shown in Figure 6A. [Figure 6C] This figure illustrates the image displayed on the HMD's display unit in the state shown in Figure 6A, with the HMD's display unit as the reference unit. [Figure 7] This is an explanatory diagram showing the relationship between the user's head tilt angle and the image displayed on the display unit in Example 1. [Figure 8] This is a flowchart of the object display control process in Example 1. [Figure 9A] This is an explanatory diagram of the additional object display control in Example 1. [Figure 9B] This table summarizes the display control of AR objects and additional objects according to the user's head tilt α in the state shown in Figure 9A. [Figure 10]It is an explanatory diagram of display control of an AR object and an additional object according to the inclination α of the user's head in Example 2. [Figure 11A] It is a diagram for explaining an image displayed on a display unit of an HMD worn by a user in Example 3 with the head tilted to the right. [Figure 11B] It is a diagram for explaining an image displayed on a display unit when the user further tilts the head to the right from the state of FIG. 11A and the inclination of the user's head becomes a predetermined value or more. [Figure 11C] It is an explanatory diagram of additional object display control in Example 3.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, an HMD will be described as an example of an information display device.

Example

[0012] FIG. 1 is a block diagram showing the hardware configuration of an HMD 100 in this embodiment. In FIG. 1, 101 is a main control unit (CPU / MCU, etc.), 102 is a bus which is a transmission / reception path of commands and data, 103 is a RAM which is a work area when executing a basic operation program and other operation programs, and 110 is a storage unit which is a non-volatile storage medium such as FlashROM / EEPROM / SSD / HDD.

[0013] 120 is an operation input unit which is a user operation interface, and has operation keys 121 such as SWs, a power key, and a volume key, and a touch sensor 122 such as a touch pad.

[0014] 130 is an image processing unit such as an image (video) processor, and has a display unit (display) 131, an image signal processing unit (image (video) signal processor) 132, a first image input unit 133 which is an external camera for forward shooting, and a second image input unit 134 which is an internal camera for gaze detection.

[0015] 140 is an audio processing unit (audio processor) and includes an audio output unit (speaker) 141, an audio signal processing unit (audio signal processor) 142, and an audio input unit (microphone) 143.

[0016] 150 is a location information acquisition unit (for receiving location information signals such as GPS signals). 160 is a sensor unit, which includes a gyro sensor 161, a geomagnetic sensor 162, and an acceleration sensor 163.

[0017] 170 is a communication unit (communication interface) consisting of a LAN (Wi-Fi®) communication unit, a mobile communication unit, a Bluetooth® communication unit, etc., and 180 is an expansion interface unit, such as a USB interface, which is used for data transmission and reception, charging, etc.

[0018] Figure 2 shows the external appearance and wearing diagram of the HMD in this embodiment. In Figure 2, the top figure is a top view of the HMD 100 worn on the head of user U1, the middle figure is a front view, and the bottom figure is a side view. As shown in Figure 2, the HMD 100 in this embodiment is a goggle-type, non-transparent HMD.

[0019] In Figure 2, the display unit 131 is non-transparent and is positioned inside the HMD 100 housing where it can be seen by the user U1. The first image input unit 133, which is an external camera, displays the user's field of view image on the display unit 131. The display unit 131 may also be transparent, in which case the outside world can be directly seen through the display unit 131.

[0020] 133L is the first image input unit (L), which is the external camera on the left, and 133R is the first image input unit (R), which is the external camera on the right. Note that there may be only one first image input unit; for example, a configuration with one first image input unit 133 in the center is also possible.

[0021] Furthermore, 122L is a touch sensor (L) and 122R is a touch sensor (R) (not shown), and there may be only one touch sensor on either the left or right side. Also, 141L is an audio output unit (L) (stereo speaker L) and 141R is an audio output unit (R) (stereo speaker R) (not shown). Although not shown, a monaural microphone may be included as an audio input unit 143.

[0022] Figure 3 is a functional configuration diagram of the HMD in this embodiment. In Figure 3, the storage unit 110 consists of a basic operation program 1001, which is a basic program such as the OS; an application 1002, which includes applications (hereinafter abbreviated as "apps") that execute the functions of this embodiment and other apps; and various data storage areas 1009, which are areas for storing various operation settings and various information (videos / still images / audio, etc.).

[0023] The programs / applications stored in the storage unit 110 are loaded into the RAM 103, and the main control unit 101 executes the loaded programs / applications through software processing, thereby executing the functional units and other functional units of this embodiment.

[0024] In Figure 3, RAM103 contains a list of executable programs to be deployed, categorized by function. In other words, RAM103 includes a basic operation function unit 1101 that executes the basic operation program of HMD100, an object data acquisition unit 1111 that acquires information about AR objects (hereinafter, objects may be referred to as OBJ) and generates AR objects to be displayed on the display unit 131 of HMD100 based on the acquired information, an attitude detection unit 1112 that detects the tilt of HMD100 with respect to the direction of gravity in real space (the tilt of the user U1's head), an additional object generation unit 1113 that generates additional information (additional objects) related to the AR objects generated by the object data acquisition unit 1111, and a display control unit 1114 that controls the display mode on the display unit 131 of the AR objects generated by the object data acquisition unit 1111 and the additional objects generated by the additional object generation unit 1113 based on the direction of gravity set in the virtual space where the AR objects are placed, the tilt of HMD100 with respect to the direction of gravity in real space detected by the attitude detection unit 1112, and the operation mode of HMD100.

[0025] Furthermore, RAM103 has a temporary storage area 1199, which is a temporary storage area for various types of information created / acquired by the application.

[0026] Figures 4A and 4B are conceptual diagrams illustrating the viewing conditions of the virtual space using the HMD in this embodiment.

[0027] Figure 4A shows user U1 wearing the HMD100 while standing upright with their head not tilted relative to the vertical direction, which is the direction of gravity G in real space. S1 is the virtual space that user U1 sees when wearing the HMD100. In Figure 4A, the virtual space S1 is the space where OBJ1-3 and other objects are placed, and is not limited to the ellipse.

[0028] OBJ1 is the AR object of a person, OBJ2 is the AR object of a cup, and OBJ3 is the AR object of a table, each positioned according to the world coordinate system. 131V is the field of view in the virtual space S1 that user U1 sees when wearing the HMD100, and it moves in conjunction with the movement of the HMD100 worn by user U1. g is the direction of gravity set in the virtual space S1 where the AR objects OBJ1-3 are placed.

[0029] By displaying AR objects OBJ1 to 3 on the display unit 131 of the HMD100, user U1 can view a predetermined range of the virtual space S1, which includes the AR objects OBJ1 to 3, through the viewing range 131V.

[0030] Figure 4B is a diagram illustrating the image displayed on the HMD's display unit in the state shown in Figure 4A. In Figure 4B, the display unit 131 shows the image 131V1 displayed in the state shown in Figure 4A. Note that OBJM are additional objects consisting mainly of text, such as menus and pop-ups for object descriptions.

[0031] Figures 5A, 5B, and 5C are conceptual diagrams illustrating the view of the virtual space by the HMD when the user tilts their head 30 degrees in this embodiment.

[0032] In Figures 5A, 5B, and 5C, the same components as in Figures 4A and 4B are denoted by the same reference numerals, and their explanations are omitted. Figure 5A shows the case where, from the state in Figure 4A, user U1 tilts their head 30 degrees to the right, that is, the HMD 100 rotates 30 degrees clockwise in the XY plane of real space. In other words, it shows the case where the angle of the vertical axis of the head with respect to the vertical direction in a plane parallel to the display unit 131 is 30 degrees.

[0033] When user U1 tilts their head 30 degrees to the right, the HMD100 also tilts, and the viewing range 131V also tilts 30 degrees to the right. In other words, the viewing range 131V rotates 30 degrees clockwise within the xy plane of virtual space S1. Each AR object OBJ1-3 and additional object OBJM, which are positioned based on the world coordinate system, maintain their positions within virtual space S1 even when user U1 or the viewing range 131V moves, in order to create an immersive virtual world experience. That is, each AR object OBJ1-3 and additional object OBJM maintains a state where they are displayed with the direction of gravity g in virtual space as the vertical reference.

[0034] Figure 5B shows the state in which the video 131V3 displayed in the display unit 131 is rendered based on each AR object OBJ1 to 3, as shown in Figure 5A.

[0035] Figure 5C shows the state in Figure 5A, where the video 131V3 displayed in the display unit 131 is drawn relative to the display unit 131. In other words, it shows the video displayed on the display unit 131 as seen by user U1 with their head tilted.

[0036] When user U1 tilts their head to the right, the display unit 131 and viewing range 131V of the HMD 100 worn on user U1's head also tilt to the right. However, each AR object OBJ1-3 placed in the virtual space S1 maintains its orientation based on the gravity direction g of the virtual space S1 in order to create the feeling of immersion in the virtual world. Therefore, as shown in Figure 5C, each AR object OBJ1-3 and additional object OBJM displayed on the display unit 131 are displayed tilted to the left, opposite to the rightward direction in which user U1 tilts their head. As a result, even when user U1 tilts their head, the objects themselves appear fixed, giving the user the feeling that they actually exist in the virtual world.

[0037] Figures 6A, 6B, and 6C are conceptual diagrams illustrating the view of the virtual space by the HMD when the user tilts their head at a 45-degree angle in this embodiment.

[0038] In Figures 6A, 6B, and 6C, the same components as in Figures 5A, 5B, and 5C are denoted by the same reference numerals, and their descriptions are omitted. Figure 6A shows the case where user U1 tilts their head further to the right, by 45 degrees, from the state shown in Figure 5A. That is, it shows the HMD100 rotating 45 degrees clockwise in the XY plane of real space.

[0039] Figures 6B and 6C show the same display within the display unit 131 as in Figures 5B and 5C. The difference in Figures 6B and 6C compared to Figures 5B and 5C is that, unlike AR objects OBJ1-3, the additional object OBJM has the same tilt as the user's head, and is displayed facing the user directly.

[0040] In this embodiment, a predetermined value for the user's head tilt angle is set to 40 degrees. If the angle is less than 40 degrees, both the AR objects OBJ1-3 and the additional object OBJM are displayed on the display unit 131 tilted according to the head tilt, as shown in Figure 5C. On the other hand, if the angle is 40 degrees or more, unlike the AR objects OBJ1-3, the additional object OBJM is tilted to match the user's head tilt and is displayed facing directly towards the user U1 who has tilted their head, as shown in Figure 6C.

[0041] This is because if there is a large difference between the tilt of the user's head and the tilt of the attached object, the attached object becomes difficult to read. In this embodiment, the predetermined value of the tilt angle is set to 40 degrees, and when the tilt angle of the head is 40 degrees or more, the tilt of the attached object is made the same as the tilt of the user's head. This has the effect of making attached objects easier to read if they are text or the like. The predetermined value of the tilt angle is preferably around 30 to 40 degrees. Alternatively, the predetermined value of the tilt angle may be 0 degrees. That is, the attached object OBJM may always have the same tilt as the tilt of the user's head and be displayed so as to be directly facing the user U1 when the head is tilted. Furthermore, the predetermined value of the tilt angle may be changed by user settings.

[0042] Figure 7 is an explanatory diagram showing the relationship between the user's head tilt angle and the image displayed on the display unit in this embodiment. Figure 7 shows the display patterns of AR objects and additional objects displayed on the display unit when the user's head tilt angle is changed in 15-degree increments, and shows the case where the predetermined value of the tilt angle is 40 degrees.

[0043] As shown in Figure 7, at tilt angles of 15 and 30 degrees, which are less than the predetermined value of 40 degrees, AR objects OBJ1-3 and the attached object OBJM are displayed tilted to the left, opposite to the right direction in which the user tilts their head. On the other hand, at tilt angles of 45 to 90 degrees, which are greater than the predetermined value of 40 degrees, AR objects OBJ1-3 are displayed tilted to the left, opposite to the right direction in which the user tilts their head, but the attached object OBJM is displayed tilted in the same direction as the user's head tilt, that is, facing the user who has tilted their head.

[0044] Figure 8 is a flowchart of the object display control process in this embodiment. In Figure 8, first, in step S101, the display control unit 1114 checks whether the state of the HMD 100 is in AR object display On mode. If it is in AR object display On mode, the process proceeds to step S102. If it is not in AR object display On mode, the process ends.

[0045] In step S102, the attitude detection unit 1112 detects the position and orientation of the HMD 100 in real space, as well as the tilt of the HMD 100. Here, the tilt of the HMD 100 is the angle between the vertical direction of the display unit 131 and the vertical direction on a plane parallel to the display unit 131, or in other words, the angle of the vertical axis of the head with respect to the vertical direction on a plane parallel to the display unit 131, which is 0° when the head is upright.

[0046] Then, in step S103, the object data acquisition unit 1111 acquires information about AR objects to be placed in the virtual space S1 from a server device on the network. Then, in step S104, the object data acquisition unit 1111 generates AR objects to be displayed on the display unit 131.

[0047] Then, in step S105, the display control unit 1114 displays the AR object generated in the S104 process on the display unit 131. Note that the AR object displayed here is displayed using the direction of gravity g in the virtual space as the vertical reference, regardless of the tilt of the HMD 100 detected in the S102 process. In other words, it is displayed in a way that maintains an orientation based on the direction of gravity g in the virtual space S1. That is, a rotation process is performed so that the AR object is tilted in the opposite direction by the same amount as the tilt of the HMD 100 detected in the S102 process, and then it is displayed.

[0048] Next, in step S106, the additional object generation unit 1113 checks whether the HMD 100 is in the additional object display On mode. If it is in the additional object display On mode, the process proceeds to step S107. If it is not in the additional object display On mode, the process returns to step S101.

[0049] In step S107, the additional object generation unit 1113 generates additional objects related to the AR object generated in the processing of S104.

[0050] Then, in step S108, it is checked whether the tilt of the HMD100 with respect to the vertical, detected in step S102, is less than a predetermined value. If the tilt with respect to the vertical is less than the predetermined value, the process proceeds to step S109. If the tilt with respect to the vertical is not less than the predetermined value, the process proceeds to step S110.

[0051] In step S109, the display control unit 1114 displays the additional object generated in S107, using the gravity direction g of the virtual space as the vertical reference, regardless of the tilt of the HMD 100 detected in the S102 process. That is, similar to the AR object displayed in the S105 process, the additional object is displayed tilted in the opposite direction by the same amount as the tilt of the HMD 100 detected in the S102 process.

[0052] In step S110, the display control unit 1114 displays the additional object generated in S107, using the same vertical reference as the display unit 113. That is, the additional object is displayed so that it is in a position without tilt within the display unit 131.

[0053] Figures 9A and 9B are explanatory diagrams of the additional object display control in this embodiment. Figure 9A shows the state in which user U1 has tilted their head to the right (clockwise) by α degrees. That is, α is the angle of inclination of user U1's head with respect to the vertical direction, which is the direction of gravity G in real space, and it shows the case when the HMD100 rotates clockwise by α degrees in the XY plane of real space.

[0054] Figure 9B is a table summarizing the display control of AR objects and additional objects in this embodiment according to the user's head tilt α, in the state shown in Figure 9A.

[0055] In Figure 9B, if the user's head tilt α, i.e., the HMD tilt α, is 0 ≤ α < predetermined value, the AR object is displayed based on the gravity direction g of the virtual space S1, regardless of the tilt of the HMD 100. Similarly, the added object is also displayed based on the gravity direction g of the virtual space S1, regardless of the tilt of the HMD 100.

[0056] On the other hand, if the HMD's tilt α is less than or equal to a predetermined value, the AR object will similarly be displayed based on the gravity direction g of the virtual space S1, regardless of the tilt of the HMD100. However, the added object will be displayed in accordance with the tilt of the HMD100, so that the relative tilt difference with the HMD100 becomes zero.

[0057] As described above, according to this embodiment, in order to create a state of immersion in the virtual world, the AR object, which is the main image, is tilted according to the tilt angle of the user's head, i.e., the tilt angle of the display unit, but additional objects such as menu displays are not tilted. This makes it possible to make menu displays and other elements easy to see even when the display unit is tilted, and provides an easy-to-use information display device and display method. [Examples]

[0058] This embodiment describes a modified example of controlling the display of an additional object when the user tilts their head.

[0059] Figure 10 is an explanatory diagram illustrating the display control of AR objects and additional objects in this embodiment according to the user's head tilt α.

[0060] In Figure 10, assuming a state similar to Figure 9A where user U1 tilts their head to the right (clockwise) by α degrees, and setting multiple predetermined values ​​such that predetermined value 1 < predetermined value 2 < predetermined value 3 < ... < predetermined value n, if the HMD tilt α is 0 ≤ α < predetermined value 1, the AR object will be displayed based on the gravity direction g of the virtual space S1, regardless of the tilt of the HMD 100. Additionally, the added object will be displayed with a tilt of (predetermined value 1 ÷ 2) relative to the vertical. For example, if predetermined value 1 is 30 degrees, and α is 0 ≤ α < 30, it will be displayed with a tilt of (30 ÷ 2) = 15 degrees.

[0061] Furthermore, if the HMD's tilt α is a predetermined value 1 ≤ α < predetermined value 2, the AR object will be displayed based on the gravity direction g of the virtual space S1, regardless of the tilt of the HMD 100. However, the added object will be displayed with a tilt of [predetermined value 1 + {(predetermined value 2 - predetermined value 1) ÷ 2}] relative to the vertical direction. For example, if predetermined value 1 is 30 degrees and predetermined value 2 is 60 degrees, then if α is 30 ≤ α < 60, it will be displayed with a tilt of [30 + {(60 - 30) ÷ 2}] = 45 degrees.

[0062] Similarly, if the HMD's tilt α is a predetermined value 2 ≤ α < predetermined value 3, the AR object will be displayed based on the gravity direction g of the virtual space S1, regardless of the tilt of the HMD 100, but the added object will be displayed with a tilt of [[predetermined value 2 + {(predetermined value 3 - predetermined value 2) ÷ 2}] relative to the vertical direction. For example, if predetermined value 2 is 60 degrees and predetermined value 3 is 90 degrees, then if α is 60 ≤ α < 90, it will be displayed with a tilt of [60 + {(90 - 60) ÷ 2}] = 75 degrees.

[0063] Ultimately, when the HMD's tilt α is a predetermined value n ≤ α ≤ 180, the AR object will be displayed based on the gravity direction g of the virtual space S1, regardless of the tilt of the HMD100, but the added object will be displayed with a tilt of [predetermined value n + {(180 - predetermined value n) ÷ 2}] relative to the vertical direction.

[0064] Thus, in this embodiment, the added objects are displayed with a stepped angle of inclination according to the user's head tilt α, that is, the tilt α of the HMD's display unit.

[0065] As a result, according to this embodiment, the difference in tilt between the user's head tilt and the added object can be kept to a minimum without significantly increasing the difference in tilt between the AR object and the added object. Furthermore, it is possible to prevent the display of the added object from fluctuating in small ways in response to small changes in the user's head tilt, creating an immersive experience in the virtual world, and also making the added object easier to read if it is text or the like. [Examples]

[0066] This embodiment describes further variations of the display control of additional objects when the user tilts their head.

[0067] Figures 11A, 11B, and 11C illustrate the images displayed on the HMD's display when a user wearing the HMD in this embodiment tilts their head to the right (clockwise).

[0068] In Figure 11A, the HMD's display unit 131 shows video 131V5. Video 131V5 displays AR objects OBJ1-3 and additional objects OBJP1, which are explanatory pop-ups related to the AR objects (for example, diagrams, tables, and comments explaining AR object OBJ2), similar to Figure 4A. In Figure 11A, the user's head tilt is less than a predetermined value, and the additional objects OBJP1 maintain their position based on the gravity direction of the virtual space S1.

[0069] Figure 11B shows the case where the user tilts their head further to the right from the state shown in Figure 11A, and the user's head tilt exceeds a predetermined value. In Figure 11B, because the user's head tilt exceeds the predetermined value, the display control described in Example 1 causes the additional object OBJP1 to be displayed with the same vertical reference as the HMD. However, since the AR objects OBJ1-3 maintain their position based on the gravity direction of the virtual space S1, the display of the additional object OBJP1 and the display of the AR objects may overlap, as shown in Figure 11B.

[0070] Therefore, in this embodiment, as shown in Figure 11C, in order to eliminate the overlap between the display of the additional object OBJP1 and the display of each AR object in the state shown in Figure 11B, the additional object OBJP1 is moved to a position where it does not overlap with the display of each AR object and displayed. In other words, for the additional object, not only the rotation process as in Embodiment 1 but also the movement process can be performed simultaneously. This makes it possible to make menu displays and other elements easier to see even when the user tilts their head.

[0071] Furthermore, the destination of the additional object OBJP1 during the above process can be a predetermined distance away in a predetermined direction. Alternatively, since the display control unit 1114 can determine the area occupied by each AR object within the display unit 131 based on the shape of each AR object, the additional object OBJP1 can be moved to a position that avoids the area occupied by each AR object within the display unit 131.

[0072] Furthermore, when the above-mentioned additional object OBJP1 is moved, in order to prevent the position of the additional object from fluctuating slightly in response to small changes in the user's head tilt, the relative positional relationship between the position of the additional object and the AR object may be fixed when the change in the user's head tilt is minute. This makes it possible to prevent the visibility of the additional object from being impaired even if the user's head moves slightly.

[0073] As described above, according to this embodiment, when a user tilts their head, that is, when the display unit is tilted, the AR object is tilted according to the tilt angle of the user's head, that is, the tilt angle of the display unit, while additional objects such as menu displays are not tilted. Furthermore, by moving them to a position where they do not overlap with the AR object, it is possible to make menu displays and the like easier to see, thereby providing a more user-friendly information display device and display method.

[0074] Although embodiments have been described above, the present invention is not limited to the embodiments described above, and various modifications are included. For example, although an HMD was used as an example of an information display device in the embodiments, the present invention is not limited to an HMD, and can also be applied to information display devices such as smartphones in which the angle of the display screen with respect to the vertical direction can be changed.

[0075] Furthermore, in addition to the process of tilting objects according to the tilt angle of the user's head or the tilt angle of the display unit, the user may also be able to select, using the input field, to fix the tilt of the objects. For example, when an AR object and an attached object are displayed tilted according to the tilt of the display unit, if the user selects to fix the display, the display will continue at the selected tilt regardless of subsequent tilts of the display unit. Some users may not like the object rotation process to follow the tilt of the display unit. Therefore, allowing the user to select to fix the display can improve user experience.

[0076] Furthermore, although the functions of the present invention described in the examples were explained as being implemented in software by a CPU or the like interpreting and executing an operating program that realizes each function, some or all of them may be implemented in hardware, for example, by designing an integrated circuit, a general-purpose processor, or a special-purpose processor. A processor includes transistors and other circuits and is considered circuitry or processing circuitry. Hardware and software may also be used in combination. In addition, some or all of each function may be implemented by a server. The server only needs to be able to execute functions in cooperation with other components via communication, and its form is not limited to, for example, a local server, cloud server, edge server, network service, etc. Information such as programs, tables, files that realize each function may be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD, or it may be stored in a device on a communication network.

[0077] Furthermore, the programs described in each processing example may be independent programs, or multiple programs may constitute a single application program. The order in which each processing step is executed may also be changed.

[0078] Furthermore, the embodiments described above are detailed explanations provided to facilitate understanding of the present invention, and are not necessarily limited to those comprising all the described configurations. It is also possible to replace parts of the configuration of one embodiment with those of another embodiment, and to add configurations from other embodiments to a given embodiment. Additionally, it is possible to add, delete, or replace parts of the configurations in each embodiment with those of other embodiments. [Explanation of symbols]

[0079] 100: HMD (Head-Mounted Display Device), 101: Main Control Unit, 103: RAM, 110: Storage Unit, 120: Operation Input Unit, 130: Image Processing Unit, 131: Display Unit, 131V: Visibility Range, 140: Audio Processing Unit, 150: Location Information Acquisition Unit, 160: Sensor Unit, 170: Communication Unit, 180: Expansion Interface Unit, 1001: Basic Operation Program, 1002: Application, 1009: Various Data Storage Areas, 1101: Basic Operation Function Unit, 1111: Object Data Acquisition Unit, 1112: Attitude Detection Unit, 1113: Additional Object Generation Unit, 1114: Display Control Unit, 1199: Temporary Storage Area, U1: User, S1: Virtual Space, OBJ1~3: AR Objects, OBJM, OBJP1: Additional Objects

Claims

1. In an information display device, Display unit and A detection unit for detecting the tilt of the information display device with respect to the direction of gravity in real space, It comprises a control unit and, The control unit, An AR object and an additional object which is additional information related to the AR object are generated. The system controls the display unit to display the AR object in a tilted state according to the tilt of the information display device detected by the detection unit. An information display device characterized by controlling the display of the additional object on the display unit in an untilted state, regardless of the tilt of the information display device.

2. In the information display device according to claim 1, It also has a communications unit, The information display device is characterized in that the control unit acquires information about the AR object from a server device using the communication unit, and generates the AR object and the additional object based on the acquired information.

3. In the information display device according to claim 1, The information display device is characterized in that the control unit controls the additional object to move to a position where it does not overlap with the AR object and to display it on the display unit.

4. In the information display device according to claim 1, The information display device is characterized in that the aforementioned additional object is composed of characters.

5. In the information display device according to claim 1, The aforementioned AR object is an object representing a person or an object, The information display device is characterized in that the aforementioned additional object is an object representing a menu containing text.

6. In the information display device according to claim 1, It is further equipped with an operation input section, The control unit is characterized in that, when the user of the information display device selects to fix the tilt of the AR object using the operation input unit, the control unit controls the display of the AR object to continue at the selected tilt, regardless of the tilt of the information display device.

7. In the information display device according to any one of claims 1 to 6, The information display device is characterized in that the information display device is a head-mounted display device.

8. In the information display device according to claim 7, The tilt of the information display device is the tilt angle of the head of the user wearing the head-mounted display device. The control for displaying the AR object on the display unit in a tilted state is a control for displaying the object on the display unit in a state tilted in the opposite direction by the same amount as the tilt angle of the head, The control for displaying the additional object on the display unit without tilting it is characterized in that the tilt is the same as the tilt of the head and the object is displayed so as to face the user.

Images