Head-mounted display device

Abstract

To provide a head-mounted display device that can avoid danger to a user when wearing the head-mounted display device.SOLUTION: A head-mounted display device is a video-transparent type that acquires an image of the outside world using a first image input unit and displays it on a display unit, and includes a second image input unit that acquires an image of the state of the wearer's eyes, and a control unit, and the control unit is configured to determine that the wearer is in an awake state when the eyes do not remain closed for a predetermined period of time or more from the image of the wearer's eyes acquired by the second image input unit, and to determine that the wearer is in an asleep state when the eyes remain closed for a predetermined period of time or more, and to display the image of the outside world acquired by the first image input unit on the display unit when the wearer changes from the asleep state to the awake state.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a head mounted display device (hereinafter referred to as HMD). [Background technology]

[0002] HMDs, which are equipped with a display and worn on the head to view information, have been increasingly used in gaming and work support fields in recent years.

[0003] Background art in this technical field is Patent Document 1. Patent Document 1 discloses a head-mounted display device that includes display means that displays an image based on a video signal on a display screen and that is configured to transmit external light, and that is capable of controlling the ratio between the image display portion and the external light transmitting portion and that is mountable on a user's head, detection means that detects the direction in which the user intends to view or the posture or position of the user's head, and display control means that controls the ratio between the image display portion and the external light transmitting portion on the display screen of the display means in accordance with detection by the detection means. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 8-328512 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, a user can view the HMD display image and an external world image while wearing the HMD, depending on the direction the user wishes to look or the posture or position of the user's head.

[0006] However, there is a possibility that a user may fall asleep while wearing an HMD, and then wake up or become semi-awake and forget that they are wearing the HMD. In this case, the user's field of vision is blocked by the HMD, preventing them from checking their surroundings. This could lead to a dangerous situation if they act in a panic, as they are unable to grasp their own situation. Patent Document 1 does not take such situations into consideration.

[0007] In view of the above-mentioned problems, an object of the present invention is to provide an HMD that can avoid danger to the user when wearing the HMD. [Means for solving the problem]

[0008] To give one example, the present invention is a video-transparent head-mounted display device that acquires an image of the outside world using a first image input unit and displays it on a display unit, and has a second image input unit that acquires an image of the state of the wearer's eyes, and a control unit, and the control unit is configured to determine, from the image of the wearer's eyes acquired by the second image input unit, that the wearer is in an awake state if both eyes are not closed for a predetermined time or more, and to determine that the wearer is in a asleep state if both eyes are closed for a predetermined time or more, and to display the image of the outside world acquired by the first image input unit on the display unit when the wearer changes from a asleep state to an awake state. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an HMD that can avoid danger to the user when wearing the HMD. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is a block diagram showing the hardware configuration of the HMD according to the first embodiment. [Figure 2] 1A and 1B are diagrams illustrating the appearance and wearing of an HMD according to a first embodiment. [Figure 3] FIG. 2 is a functional configuration diagram of an HMD according to the first embodiment. [Figure 4] 1 is a processing flowchart of an HMD according to the first embodiment. [Figure 5] 10 is a processing flowchart of an HMD according to the second embodiment. [Figure 6] 10 is a display example of a warning message in the second embodiment. [Figure 7] 11 is a processing flowchart of an HMD according to the third embodiment. [Figure 8] 10 is a processing flowchart of an HMD according to the fourth embodiment. [Figure 9] 13 is a display example of a warning message of the HMD in the fourth embodiment. [Figure 10] 13 is an example of an external display of an HMD in Example 4. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [Example]

[0012] Fig. 1 is a block diagram showing the hardware configuration of an HMD 100 in this embodiment. In Fig. 1, 101 is a main control unit (CPU / MCU, etc.), 102 is a bus which is a path for sending and receiving commands and data, 103 is a RAM which serves as a work area when executing basic operation programs and other operation programs, and 110 is a storage unit which is a non-volatile storage medium such as Flash ROM / EEPROM / SSD / HDD.

[0013] Reference numeral 120 denotes an operation input unit that is a user operation interface, and includes operation keys 121 such as switches, a power key, and a volume key, and a touch sensor 122 such as a touch pad.

[0014] Also, 130 is an image processing unit such as an image (video) processor, and has a display unit (display) 131, an image signal processing unit (image (video) signal processor) 132, a first image input unit 133 which is an external camera for acquiring outside world images, and a second image input unit 134 which is an internal camera for binocular confirmation.

[0015] Reference numeral 140 denotes an audio processor, which includes an audio output unit (speaker) 141, an audio signal processor (audio signal processor) 142, and an audio input unit (microphone) 143.

[0016] Also, reference numeral 151 denotes a location information acquisition unit (GPS receiving unit), and 160 denotes a sensor unit, which includes a gyro sensor 161, a geomagnetic sensor 162, and an acceleration sensor 163.

[0017] Also, 170 is a communication unit including a LAN (Wi-Fi (registered trademark)) communication unit, a mobile communication unit, a Bluetooth (registered trademark) communication unit, etc., and 180 is an expansion interface unit having functions such as a charging terminal, for example.

[0018] Figure 2 shows the appearance and wearing of the HMD in this embodiment. As shown in Figure 2, the HMD in this embodiment is a goggle-type HMD, and is not an optical transmission type with a window on the front, but a closed video transmission type HMD in which an external image is captured by a camera and displayed on a display unit within the HMD. (A) is a top view of the HMD 100 worn on the head of user U1, (B) is a front view, and (C) is a side view.

[0019] 2, 133L is a first image input unit (L) which is an external camera on the left side, and 133R is a first image input unit (R) which is an external camera on the right side. Note that there may be only one first image input unit (for example, one first image input unit 133 in the center).

[0020] Also, 122L is a touch sensor (L), and 122R is a touch sensor (R) (not shown), and there may be only one touch sensor on either the left or right side. Furthermore, 141L denotes an audio output unit (L) (stereo speaker L), and 141R denotes an audio output unit (R) (stereo speaker R) (not shown).

[0021] Although not shown in the figures, the HMD has a monaural microphone as an audio input unit 143, and a display unit 131 made up of an LCD or the like arranged inside the housing of the HMD so as to be visible to the user. The display unit 131 may be made up of a display unit 131L for the left eye and a display unit 131R for the right eye. The HMD also has a second image input unit 134, which is an internal camera arranged so as to be able to check the state of both of the user's eyes, inside the housing. The second image input unit 134 may also be made up of a second image input unit 134L that can check the state of the left eye and a second image input unit 134R that can check the state of the right eye.

[0022] Fig. 3 is a functional configuration diagram of the HMD in this embodiment. In Fig. 3, the storage unit 110 is composed of a basic operation program 1001, which is a basic program such as an OS, an application 1002 including applications (hereinafter abbreviated as apps) that execute the functions of this embodiment and other apps, and a various data storage area 1009, which is an area for storing various operation setting values ​​and various information (video / still images / audio, etc.).

[0023] The programs / applications stored in the storage unit 110 are deployed (loaded) into the RAM 103, and the main control unit 101 executes the deployed programs / applications, thereby executing the functional units of this embodiment and other functional units.

[0024] 3, the RAM 103 lists executable programs to be loaded by function. Specifically, the RAM 103 includes a basic operation function unit 1101 that executes a basic operation program of the HMD 100, a binocular state confirmation unit 1111 that analyzes an image acquired by the second image input unit (internal camera) 134 to confirm the state of both eyes and the gaze position of the user U1 wearing the HMD 100, a communication state confirmation unit 1112 that analyzes information received via the communication unit 170 to confirm whether the received information is an emergency alert or the like, an application control unit 1113 that controls processes such as launching / executing various applications in response to operation instructions from the user U1 wearing the HMD 100 and generates an application screen (application image), and a display control unit 1114 that selects one of the image of the application screen (application image) generated by the application control unit 1113 and an external image acquired by the first image input unit (external camera) in response to the state of both eyes and the gaze position of the user confirmed by the binocular state confirmation unit 1111 or whether the received information confirmed by the communication state confirmation unit 1112 is an emergency alert or the like, and controls the display on the display unit 131. The RAM 103 also has a temporary storage area 1199 that is a temporary storage area for various types of information created / acquired by the application.

[0025] 4 is a processing flowchart of the HMD in this embodiment. In FIG. 4, first, in step S101, the display control unit 1114 displays on the display unit 131 an application screen (application image) that the application control unit 1113 has generated by activating / executing it in accordance with a user's operation instruction, and the process proceeds to S102. In S102, the second image input unit (internal camera) 134 acquires an image of the user's both eyes, and the process proceeds to S103. In S103, the binocular state confirmation unit 1111 analyzes the image of the user's both eyes acquired in the process of S102, and if it is determined that both of the user's eyes are closed, the process proceeds to S104. If it is determined that both of the user's eyes are not closed, the process returns to S102. In S104, the binocular state confirmation unit 1111 initializes the count value of a timer, and the process proceeds to S105. In S105, the second image input unit (internal camera) 134 acquires an image of the user's both eyes, and the process proceeds to S106. In S106, the binocular state confirmation unit 1111 analyzes the binocular image of the user acquired in the process of S105, and if it is determined that one of the user's eyes is open, the process returns to S102. If it is determined that neither of the user's eyes is open, the process proceeds to S107. In S107, the binocular state confirmation unit 1111 counts up the count value of the timer, and proceeds to S108. In S108, the binocular state confirmation unit 1111 compares the count value of the timer with a predetermined value, and if the count value of the timer is equal to or greater than the predetermined value, the process proceeds to S109. If the count value of the timer is not equal to or greater than the predetermined value, the process returns to S105.

[0026] In S109, it is assumed that the user has entered a sleeping state, and the display control unit 1114 generates a sleep screen (a dark screen with no video) and displays it on the display unit 131, and the process proceeds to S110. Note that the process of S109 may not be performed.

[0027] In S110, the second image input unit (internal camera) 134 acquires an image of both eyes of the user, and the process proceeds to S111. In S111, the binocular state confirmation unit 1111 analyzes the image of both eyes of the user acquired in S110, and if it is determined that one of the user's eyes is open, the process proceeds to S112. If it is determined that neither of the user's eyes is open, the process returns to S110. Note that a timer may be added after S111, and the process may proceed to S112 if it is detected that the eyes have been open for a predetermined period of time or more.

[0028] In S112, it is assumed that the user is in an awake state (or semi-awake state: the same applies below), and the first image input unit (external camera) 133 acquires an external world image, and the process proceeds to S113. In S113, the display control unit 1114 displays the external world image acquired in S112 on the display unit 131, and the process ends.

[0029] As described above, in this embodiment, the internal camera checks the state of the wearer's (user's) eyes, and if the wearer's (user's) eyes are not closed for a predetermined period of time or more, the wearer is determined to be awake. If the wearer's (user's) eyes are closed for a predetermined period of time or more, the wearer is determined to be asleep ... change from the awake state to the asleep state, the application image is switched to a sleep screen (no image), and if the wearer's (user's) eyes change to the awake state, the sleep screen is switched to a video-through image (external camera image). This allows the wearer (user) to check the situation around them (real space) even if they fall asleep while wearing the HMD and then wake up.

[0030] When the wearer goes from awake to asleep, the app image may continue to be displayed without switching to a sleep screen (no image), but switching to a sleep screen with no image and a dark screen can save battery power when the wearer is asleep. Also, when the wearer goes from awake to asleep, the app image may be switched to a video-through image (image from an external camera).

[0031] Furthermore, if the application screen (application video) displayed on the display unit 131 in S101 is accompanied by audio output (audio output from the audio output unit 141), the audio output may also be stopped when switching to a sleep screen (no video) in S109. Alternatively, to indicate that the HMD is (was) in operation, the audio output may be continued when switching to a sleep screen (no video) in S109. Alternatively, the audio output may be continued with the volume lowered.

[0032] As described above, according to this embodiment, even if the user falls asleep while wearing the HMD and wakes up later, the user can check the situation around them, and an HMD can be provided that allows the user to avoid danger. [Example]

[0033] In the first embodiment, a configuration was described in which, when the user wakes up from a sleep state, the user can check the surrounding situation by switching to an external camera image, thereby enabling the user to avoid danger. In this embodiment, another example of danger avoidance will be described.

[0034] Fig. 5 is a processing flowchart of the HMD in this embodiment. In Fig. 5, the same processes as in Fig. 4 are assigned the same reference numerals, and their explanations will be omitted. Fig. 5 differs from Fig. 4 in that steps S122 and S123 display a message to the wearer (user) that the HMD is being worn when the wearer wakes up from a sleeping state.

[0035] That is, in Fig. 5, steps S101 to S111 are the same as the operations in Fig. 4. Although step S109 is not shown in Fig. 5, a sleep screen may be displayed when the awake state changes to the sleeping state, as in Fig. 4.

[0036] If the result of S111 is Yes, the state has changed from a sleeping state to an awake state, and the process proceeds to S122. In S122, the display control unit 1114 generates a message to warn the user that the HMD is being worn, and the process proceeds to S123. In S123, the display control unit 1114 displays the warning message generated in S122 on the display unit 131, superimposed on the application screen (application image) that the application control unit 1113 has generated by starting / executing the application in accordance with the user's operation instruction, and the process ends.

[0037] 6 is a display example of a warning message in this embodiment. In FIG. 6, the warning message M11 generated in S122 is displayed on the display unit 131 superimposed on the application screen (application image) AP11. When a sleep screen is displayed when the user transitions from an awake state to a sleeping state, the warning message M11 is displayed superimposed on the sleep screen, which is a dark screen without any image. In addition to the above, examples of the warning message display may include a message such as "You have been asleep for X hours" or simply a display of the current time. In other words, it is sufficient if the message makes the user aware that they are wearing an HMD.

[0038] 6, the warning message may not only be displayed on the display unit 131 but also be output as audio from the audio output unit 141. In this case, repeating the same warning message multiple times can reduce the possibility that the user will miss the warning message. Also, the audio output of the warning message may be repeated until the user issues an instruction to stop the audio output of the warning message.

[0039] As described above, according to this embodiment, even if the user falls asleep while wearing the HMD and wakes up later, the user can check the warning message, and an HMD that can avoid danger can be provided.

[0040] When the awake state changes to the sleep state, a warning message may be superimposed on the application screen that was displayed in the awake state on the display unit 131. Furthermore, if the user wishes to switch to displaying an external image after checking the warning message, the user can do so by issuing an instruction or gesture from the operation input unit 120. Furthermore, the warning message may be erased and the display of the app screen may be resumed in response to an instruction from the user. [Example]

[0041] In this embodiment, another example of danger avoidance when wearing an HMD will be described.

[0042] Fig. 7 is a processing flowchart of the HMD in this embodiment. In Fig. 7, first, in step S201, the display control unit 1114 displays on the display unit 131 an application screen (application image) that the application control unit 1113 has started / executed and generated in accordance with a user's operation instruction, and the process proceeds to S202.

[0043] In S202, the communication unit 170 receives a message sent from a device on an external network or a device in cooperation with the communication unit 170, for example, a mobile information terminal such as a smartphone, and proceeds to S203. In S203, the communication status confirmation unit 1112 analyzes the message received in S202, and if it determines that the received message is an emergency alert or the like, it proceeds to S204. If it determines that the received message is not an emergency alert or the like, it returns to S202.

[0044] In S204, the first image input unit (external camera) 133 acquires an outside world image, and the process proceeds to S205. In S205, the display control unit 1114 displays the outside world image acquired in S204 on the display unit 131, and the process ends.

[0045] In this way, in this embodiment, when the HMD receives an emergency alert (email, etc.) such as an earthquake early warning via the communication unit, it switches the application image to a video-through image (external world image) regardless of whether the user is awake or asleep. This allows the user to check their surroundings by switching to an external world image in an emergency, even when wearing a closed-type video-transparent HMD, and provides an HMD that can avoid danger.

[0046] A warning message informing the user of the content of the emergency alert may be superimposed on the display unit.

[0047] In addition, the audio input unit 143 may acquire audio information (instead of the process of S202), analyze the acquired audio information, and if it determines that the audio information is an emergency bell or the like (instead of the process of S203), switch to an external world image. [Example]

[0048] In the second embodiment, an HMD capable of avoiding danger by superimposing a warning message on the display screen when the user wakes up from a sleep state has been described.

[0049] In this embodiment, a description will be given of switching to an external image by gazing at a predetermined icon after a warning message is superimposed.

[0050] Fig. 8 is a processing flowchart of the HMD in this embodiment. In Fig. 8, the steps enclosed by dotted lines are the processes from S101 to S111 in Fig. 5, which are abbreviated and their description will also be omitted.

[0051] If the result of S111 is Yes, the state has changed from a sleeping state to an awake state, and the process proceeds to S124. In S124, the display control unit 1114 generates a message to warn the user that the HMD is being worn, and the process proceeds to S125. In S125, the display control unit 1114 displays on the display unit 131 the warning message generated in S124 superimposed on the application screen (application image) that the application control unit 1113 has generated by starting / executing the application in accordance with the user's operation instruction.

[0052] 9 is a display example of a warning message in this embodiment. In FIG. 9, the warning message M12 generated in S124 is displayed on the display unit 131 superimposed on the application screen (application image) AP11. Also displayed is a predetermined icon O11, which is a mark indicating that the emergency door is closed. The warning message M12 displays a message indicating that the HMD is being worn, as well as a message saying, "If you want to check the outside, please look at the door mark." As will be described later, if you focus your gaze on this predetermined icon O11 for a predetermined period of time or longer, the image displayed on the display unit will switch from the application image to an image of the outside world.

[0053] In S125, only the predetermined icon O11 may be displayed. That is, the display of the predetermined icon O11 may include the meaning of the warning message M12.

[0054] Next, in S302, the second image input unit (internal camera) 134 acquires an image of the user's both eyes, and the process proceeds to S303. In S303, the binocular state confirmation unit 1111 analyzes the image of the user's both eyes acquired in S302 to detect the user's gaze position, and the process proceeds to S304. In S304, the binocular state confirmation unit 1111 confirms the user's gaze position detected in S303, and if it is determined that the user's gaze position is on a predetermined icon, the process proceeds to S305. If it is determined that the user's gaze position is not on the predetermined icon, the process returns to S302. Note that if the user's gaze position is within a predetermined range that includes the predetermined icon and is larger than the predetermined icon, it may be determined that the user's gaze position is on the predetermined icon.

[0055] In S305, it is determined that the user is looking at a predetermined icon, and the binocular state confirmation unit 1111 initializes the count value of the timer, and the process proceeds to S306. In S306, the second image input unit (internal camera) 134 acquires an image of the user's both eyes, and the process proceeds to S307. In S307, the binocular state confirmation unit 1111 analyzes the image of the user's both eyes acquired in S306 to detect the user's gaze position, and the process proceeds to S308. In S308, the binocular state confirmation unit 1111 confirms the user's gaze position detected in the process of S307, and if it is determined that the user's gaze position has deviated from the predetermined icon (within a predetermined range), the process returns to S302. If it is determined that the user's gaze position has not deviated from the predetermined icon (within a predetermined range), the process proceeds to S309. In S309, the binocular state confirmation unit 1111 increments the count value of the timer, and the process proceeds to S310. In S310, the binocular state confirmation unit 1111 compares the count value of the timer with a predetermined value, and if the count value of the timer is equal to or greater than the predetermined value, it determines that the gaze is fixed on a predetermined icon for a predetermined period of time or more, i.e., the user is gazing at the icon, and proceeds to S311. If the count value of the timer is not equal to or greater than the predetermined value, it returns to S306.

[0056] In S311, the first image input unit (external camera) 133 acquires an outside world image, and the process proceeds to S312. In S312, the display control unit 1114 displays the outside world image acquired in S311 on the display unit 131, and the process ends.

[0057] Fig. 10 is an example of an outside world display in this embodiment. In Fig. 10, an outside world image OW11 is displayed on the display unit 131 by S312. That is, when the gaze is focused on the predetermined icon O11 in Fig. 9 for a predetermined period of time or more, the image displayed on the display unit 131 switches from the warning message superimposed on the application image to the outside world image, and also changes from the predetermined icon O11 with a mark of the emergency door in a closed state to a predetermined icon O12 with a mark of the emergency door in an open state.

[0058] If the user focuses his / her gaze on the predetermined icon O12 for a predetermined period of time or longer, the image displayed on the display unit 131 switches from the external image to the application image. Furthermore, the design of the predetermined icon also changes from the predetermined icon O12 with a mark of an open emergency door to the predetermined icon O11 with a mark of a closed emergency door.

[0059] The predetermined icon does not have to be an emergency door mark, but may be any icon that indicates a display of the outside world.

[0060] The waiting loop for switching the external world based on gaze may be interrupted, that is, switched to the application screen display based on an instruction from the user via the operation input unit 120.

[0061] As described above, according to this embodiment, even if the user falls asleep while wearing the HMD and wakes up later, the user can check the warning message and can automatically switch to an external image by gazing at a specified icon.

[0062] The predetermined icon O11 may be displayed superimposed on the application image AP11 at all times while the application is running, regardless of the user's transition between a sleeping state and a waking state. In other words, since the user may feel uneasy if their field of vision is blocked from the real space by the application image AP11 for a long period of time, danger can be avoided by allowing the user to check the outside world by focusing their gaze on the predetermined icon O11 at any time.

[0063] Furthermore, in the above-described embodiments, the case where the HMD is a closed-type video-transmitting HMD has been described. However, even if the HMD is an optically transmittable HMD, if an object generated by executing an application covers most of the display section, by performing processing similar to that described in the above-described embodiments, if the wearer (user) falls asleep while wearing the HMD and wakes up later, the wearer (user) can check the situation around them (real space) without their view in real space being obstructed by the application screen (application image). Note that, if the HMD is an optically transmittable HMD, the processing for switching to a video-through image (external camera image) in the above-described embodiments can be changed to processing for stopping the display of the application image.

[0064] Although the embodiments have been described above, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations. [Explanation of symbols]

[0065] 100: HMD (head mounted display device), 101: main control unit, 103: RAM, 110: storage unit, 120: operation input unit, 121: operation keys, 122: touch sensor, 130: image processing unit, 131: display unit, 132: image signal processing unit, 133: first image input unit, 134: second image input unit, 140: audio processing unit, 141: audio output unit, 142: audio signal processing unit, 143: audio input unit, 151: position information acquisition unit, 160: sensor unit, 170 : Communication unit, 180: Extended interface unit, 1001: Basic operation program, 1002: Application, 1009: Various data storage area, 1101: Basic operation function unit, 1111: Binocular state confirmation unit, 1112: Communication state confirmation unit, 1113: Application control unit, 1114: Display control unit, 1199: Temporary storage area, U1: User, M11, M12: Warning message, AP11: Application screen (application image), O11, O12: Prescribed icon, OW11: Outside world image.

Claims

1. A video see-through type head mounted display device that acquires an external image by a first image input unit and displays it on a display unit, a second image input unit for acquiring an image of the state of both eyes of the wearer; A control unit is provided. The control unit determines, from the image of the state of the wearer's eyes acquired by the second image input unit, that the wearer is in an awake state if both eyes are not closed for a predetermined period of time or more, and determines that the wearer is in a asleep state if both eyes are closed for a predetermined period of time or more, and when the wearer changes from the asleep state to the awake state, displays the external world image acquired by the first image input unit on the display unit.

2. The head-mounted display device according to claim 1, The head-mounted display device is characterized in that, when the awake state is changed to the sleep state, the control unit switches the display of the display unit from the application image displayed in the awake state to a blank screen, and then, when the sleep state is changed to the awake state, the control unit switches the display of the display unit from the blank screen to the external world image.

3. The head-mounted display device according to claim 1, The head-mounted display device is characterized in that, when the awake state is changed to the sleep state, the control unit continues to display the application image that was displayed in the awake state as the display on the display unit, and then, when the sleep state is changed to the awake state, the control unit switches the display on the display unit from the application image to the external world image.

4. The head-mounted display device according to claim 1, The head-mounted display device is characterized in that, when the awake state is changed to the sleep state, the control unit switches the display of the display unit from the application image displayed in the awake state to the external world image.

5. A video see-through type head mounted display device that acquires an external image by a first image input unit and displays it on a display unit, a second image input unit for acquiring an image of the state of both eyes of the wearer; A control unit is provided. The control unit determines, from the image of the state of the wearer's eyes acquired by the second image input unit, that the wearer is in an awake state if both eyes are not closed for a predetermined period of time or more, and determines that the wearer is in a asleep state if both eyes are closed for a predetermined period of time or more, and when the wearer changes from the asleep state to the awake state, displays a warning message on the display unit to warn that the head-mounted display device is being worn.

6. The head-mounted display device according to claim 5, The head-mounted display device is characterized in that, when the awake state changes to the sleep state, the control unit switches the display on the display unit from the application image displayed in the awake state to a blank screen, and then, when the sleep state changes to the awake state, the control unit superimposes the warning message on the blank screen on the display unit.

7. The head-mounted display device according to claim 5, The head-mounted display device is characterized in that, when the awake state changes to the sleep state, the control unit continues to display the application image that was displayed in the awake state on the display unit, and then, when the awake state changes from the sleep state, the control unit superimposes the warning message on the application image on the display unit.

8. The head-mounted display device according to claim 5, The head-mounted display device is characterized in that, when the awake state is changed to the sleep state, the control unit displays the warning message on the display unit, superimposed on the application image that was displayed in the awake state.

9. A video see-through type head mounted display device that acquires an external image by a first image input unit and displays it on a display unit, The Communications Department and A control unit is provided. A head-mounted display device characterized in that, when an emergency alert is received via the communication unit, the control unit switches the display of the display unit from the application image that was being displayed to an external image acquired by the first image input unit.

10. The head-mounted display device according to claim 9, The head-mounted display device is characterized in that the control unit displays a warning message on the display unit, superimposed thereon, the warning message indicating the content of the emergency alert.

11. The head-mounted display device according to claim 5, The control unit displays, as the warning message, an icon suggesting an external world display and a message indicating that an image of the external world can be confirmed by looking at the icon, and when it determines from an image of the wearer's eyes acquired by the second image input unit that the wearer is looking at the icon for a predetermined period of time or more, the control unit switches the display on the display unit to the external world image acquired by the first image input unit.

12. The head-mounted display device according to claim 11, The head-mounted display device is characterized in that the icon is a mark indicating that an emergency door is closed.

Images