Head-mounted display device

Abstract

To provide a head mount display apparatus capable of avoiding the danger of a user who wears the head mount display apparatus.SOLUTION: A video see-through head mount display apparatus, which obtains an external world image at a first image input unit and displays it on a display unit, comprises a second image input unit that obtains an image of a state of both eyes of a wearer, and a control unit. The control unit determines that the wearer is in an awake state when the second image input unit does not continue to obtain the image of both eyes of the wearer in a closed state for a predetermined period of time or longer, and determines that the wearer is in a sleep state when the second image input unit continues to obtain the image of both eyes of the wearer in a closed state for the predetermined period of time or longer. The control unit displays the external world image obtained at the first image input unit on the display unit when the wearer is awakened from the sleep state.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a head-mounted display device (hereinafter referred to as an HMD: Head Mount Display).

Background Art

[0002] An HMD equipped with a display and worn on the head to view information has been increasingly used in recent years in the fields of gaming and work support.

[0003] As the background art in this technical field, there is Patent Document 1. Patent Document 1 discloses a display means configured to display an image by a video signal on a display screen and to transmit external light, capable of controlling the ratio between the video display portion and the external light transmission portion, and being wearable on the user's head, a detection means for detecting the direction in which the user intends to look, the posture of the user's head, or the position of the head, and a display control means for controlling the ratio between the video display portion and the external light transmission portion on the display screen of the display means in response to the detection by the detection means. A head-mounted display device is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In Patent Document 1, while the user wears the HMD, it is possible to view the display image of the HMD and the external video according to the direction in which the user intends to look, the posture of the user's head, or the position of the head.

[0006] However, it is conceivable that the user may fall asleep while wearing the HMD and then forget that they are wearing the HMD when waking up or semi-waking up after sleep. In this case, since the user's vision is blocked by the HMD and they cannot check the surrounding situation, they cannot grasp their own situation, and may cause a dangerous situation if they act in a hurry. Such a situation is not considered in the above Patent Document 1.

[0007] In view of the above problems, an object of the present invention is to provide an HMD capable of avoiding risks to the user when wearing the HMD.

Means for Solving the Problems

[0008] Taking an example, the present invention is a video see-through type head-mounted display device that acquires an external image with a first image input unit and displays it on a display unit, and has a second image input unit that acquires an image of the states of both eyes of the wearer, and a control unit. The control unit determines that the wearer is in an awake state when the state of closing both eyes does not continue for a predetermined time or more based on the image of the states of both eyes of the wearer acquired by the second image input unit, and determines that the wearer is in a sleeping state when the state of closing both eyes continues for a predetermined time or more. When changing from the sleeping state to the awake state, it is configured to display the external image acquired by the first image input unit on the display unit.

Effects of the Invention

[0009] According to the present invention, it is possible to provide an HMD capable of avoiding risks to the user when wearing the HMD.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

MODE FOR CARRYING OUT THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

EXAMPLE

[0012] FIG. 1 is a block diagram showing the hardware configuration of the HMD 100 in the present embodiment. In FIG. 1, 101 is a main control unit (CPU / MCU, etc.), 102 is a bus which is a transmission / reception path for commands and data, 103 is a RAM which serves as a work area when executing a basic operation program and other operation programs, and 110 is a storage unit which is a non-volatile storage medium such as FlashROM / EEPROM / SSD / HDD.

[0013] Also, 120 is an operation input unit which is a user operation interface, and has operation keys 121 such as SWs, a power key, and a volume key, and a touch sensor 122 such as a touch pad.

[0014] Also, 130 is an image processing unit such as an image (video) processor, and has a display unit (display) 131, an image signal processing unit (image (video) signal processor) 132, a first image input unit 133 which is an external camera for acquiring an external image, and a second image input unit 134 which is an internal camera for binocular confirmation.

[0015] Further, 140 is an audio processing unit (audio processor), which includes an audio output unit (speaker) 141, an audio signal processing unit (audio signal processor) 142, and an audio input unit (microphone) 143.

[0016] Further, 151 is a position information acquisition unit (GPS receiver). Further, 160 is a sensor unit, which includes a gyro sensor 161, a geomagnetic sensor 162, and an acceleration sensor 163.

[0017] Further, 170 is a communication unit including a LAN (Wi-Fi (registered trademark)) communication unit, a mobile communication unit, a Bluetooth (registered trademark) communication unit, etc., and 180 is an expansion interface unit, which has functions such as a charging terminal.

[0018] Figure 2 is an external view and a wearing diagram of the HMD in this embodiment. As shown in Figure 2, the HMD in this embodiment is of a goggle type, not an optical transmission type with a window on the front surface, but a video transmission type sealed HMD that captures an external image with a camera and displays it on a display unit inside the HMD. (A) is a top view of the state where the HMD 100 is worn on the head of the user U1, (B) is a front view, and (C) is a side view.

[0019] In Figure 2, 133L is a first image input unit (L), which is an external camera on the left side, and 133R is a first image input unit (R), which is an external camera on the right side. Note that there may be only one first image input unit (for example, one first image input unit 133 in the central part).

[0020] Further, 122L is a touch sensor (L), and 122R is a touch sensor (R) (not shown). There may be only one touch sensor on either the left or right side. Further, 141L is an audio output unit (L) (stereo speaker L), and 141R is an audio output unit (R) (stereo speaker R) (not shown).

[0021] Although illustration is omitted, the voice input unit 143 has a monaural microphone, and inside the housing of the HMD, there is a display unit 131 composed of an LCD or the like that is visible to the user. Further, the display unit 131 may be composed of a left-eye display unit 131L and a right-eye display unit 131R. Further, inside the housing of the HMD, there is also a second image input unit 134, which is an internal camera arranged so that the state of both eyes of the user can be confirmed. Further, the second image input unit 134 may also be composed of a second image input unit 134L capable of confirming the state of the left eye and a second image input unit 134R capable of confirming the state of the right eye.

[0022] Figure 3 is a functional configuration diagram of the HMD in this embodiment. In Figure 3, the storage unit 110 is composed of a basic operation program 1001, which is a basic program such as an OS, an application (hereinafter abbreviated as an app) that executes the functions of this embodiment, and other apps, and an application 1002, and various data storage areas 1009, which are areas for storing various operation setting values and various information (videos / still images / audio, etc.).

[0023] The programs / apps stored in the storage unit 110 are expanded (loaded) into the RAM 103, and the main control unit 101 executes the expanded programs / apps, whereby the functional units and other functional units of this embodiment are executed.

[0024] In FIG. 3, the RAM 103 describes the expanded execution program by function. That is, a basic operation function unit 1101 that executes the basic operation program of the HMD 100, a binocular state confirmation unit 1111 that analyzes the acquired image of the second image input unit (internal camera) 134 to confirm the state of both eyes and the line-of-sight position of the user U1 wearing the HMD 100, a communication state confirmation unit 1112 that analyzes the information received via the communication unit 170 to confirm whether the received information is an emergency warning or the like, an application control unit 1113 that controls processes such as activation / execution of various applications according to the operation instructions of the user U1 wearing the HMD 100 and generates an application screen (application video), and the application screen video (application video) generated by the application control unit 1113 according to whether the state of both eyes and the line-of-sight position of the user confirmed by the binocular state confirmation unit 1111 or the received information confirmed by the communication state confirmation unit 1112 is an emergency warning or the like, and one of the external world videos acquired by the first image input unit (external camera) are selected to control the display to the display unit 131. The RAM 103 also has a temporary storage area 1199 which is a temporary storage area for various information created / acquired by applications.

[0025] Figure 4 is a processing flowchart of the HMD in this embodiment. In Figure 4, first in step S101, the display control unit 1114 displays the app screen (app video) generated by the app control unit 1113 starting / executing according to the user's operation instruction on the display unit 131, and proceeds to S102. In S102, the second image input unit (internal camera) 134 acquires the binocular images of the user, and proceeds to S103. In S103, the binocular state confirmation unit 1111 analyzes the binocular images of the user acquired in the process of S102. If it is determined that both eyes of the user are closed, it proceeds to S104. If it is determined that both eyes of the user are not closed, it returns to S102. In S104, the binocular state confirmation unit 1111 initializes the count value of the timer and proceeds to S105. In S105, the second image input unit (internal camera) 134 acquires the binocular images of the user, and proceeds to S106. In S106, the binocular state confirmation unit 1111 analyzes the binocular images of the user acquired in the process of S105. If it is determined that either one of the user's eyes is open, it returns to S102. If it is determined that neither of the user's eyes is open, it proceeds to S107. In S107, the binocular state confirmation unit 1111 increments the count value of the timer and proceeds to S108. In S108, the binocular state confirmation unit 1111 compares the count value of the timer with a predetermined value. If the count value of the timer ≧ the predetermined value, it proceeds to S109. If the count value of the timer < the predetermined value, it returns to S105.

[0026] In S109, assuming that the user has fallen asleep, the display control unit 1114 generates a sleep screen (a dark screen without video) and displays it on the display unit 131, and proceeds to S110. Note that the process of S109 may not be provided.

[0027] In S110, the second image input unit (internal camera) 134 acquires the binocular images of the user and proceeds to S111. In S111, the binocular state confirmation unit 1111 analyzes the binocular images of the user acquired in S110. If it determines that either of the user's eyes is open, it proceeds to S112. If it determines that neither of the user's eyes is open, it returns to S110. Additionally, a timer may be added after S111 to proceed to S112 when a state where the eyes are open for a predetermined time or longer is detected.

[0028] In S112, assuming that the user has become awake (or semi-awake; the same applies hereinafter), the first image input unit (external camera) 133 acquires an external video image and proceeds to S113. In S113, the display control unit 1114 displays the external video image acquired in S112 on the display unit 131 and ends the process.

[0029] As described above, in this embodiment, the internal camera checks the state of both eyes of the wearer (user). If the state of closing both eyes does not continue for a predetermined time or longer, it is determined that the wearer (user) is awake. If the state of closing both eyes continues for a predetermined time or longer, it is determined that the wearer (user) is in a sleeping state. Then, when changing from the awake state to the sleeping state, the app video is switched to a sleep screen (no video). After that, when becoming awake, the sleep screen is switched to a video-through video (external camera video). Thereby, even when the user falls asleep while wearing the HMD and then wakes up, the wearer (user) can check the situation of the surrounding (real space).

[0030] Note that when changing from the awake state to the sleeping state, the display of the app video may be continued without switching to the sleep screen (no video). However, by switching to the sleep screen, which is a dark screen with no video, it is possible to save the battery consumption of the HMD when the wearer is in the sleeping state. Also, when changing from the awake state to the sleeping state, the app video may be switched to a video-through video (external camera video).

[0031] Also, when the app screen (app video) displayed on the display unit 131 in S101 is accompanied by audio output (audio output is being performed from the audio output unit 141), the audio output may also be stopped when switching to the sleep screen (no video) in S109. Alternatively, in order to indicate that the HMD is in operation (was in operation), the audio output may continue as it is when switching to the sleep screen (no video) in S109. Also, the volume may be lowered to continue the audio output.

[0032] As described above, according to this embodiment, even when the user falls asleep while wearing the HMD and then wakes up later, the user can check the surrounding situation and an HMD that can avoid danger can be provided.

Embodiment

[0033] In the first embodiment, when waking up from the sleep state, by switching to the external camera video, a configuration has been described in which the user can check the surrounding situation and avoid danger. In this embodiment, other examples of danger avoidance will be described.

[0034] FIG. 5 is a processing flowchart of the HMD in this embodiment. In FIG. 5, the same processes as in FIG. 4 are denoted by the same reference numerals and their descriptions are omitted. In FIG. 5, the differences from FIG. 4 are steps S122 and S123, which is to display a message indicating that the wearer (user) is wearing the HMD when waking up from the sleep state.

[0035] That is, in FIG. 5, S101 to S111 are the same as the operations in FIG. 4. Note that in FIG. 5, S109 is not described, but when changing from the awake state to the sleep state as in FIG. 4, a sleep screen may be displayed.

[0036] If the answer is Yes in S111, it means the user has woken up from the sleep state and proceeds to S122. In S122, the display control unit 1114 generates a message for warning the user that they are wearing the HMD and then proceeds to S123. In S123, the display control unit 1114 superimposes the warning message generated in S122 on the app screen (app video) generated and produced by the app control unit 1113 according to the user's operation instructions, and then displays it on the display unit 131, and the process ends.

[0037] Figure 6 is an example of the display of the warning message in this embodiment. In Figure 6, on the display unit 131, the warning message M11 generated in S122 is superimposed and displayed on the app screen (app video) AP11. When the sleep screen is to be displayed when changing from the awake state to the sleep state, the warning message M11 is superimposed and displayed on the sleep screen which is a dark screen without video. In addition to the above-mentioned example of the display of the warning message, it may also be a display such as "You have slept for X hours" or simply the current time display. That is, anything that can make the user aware that they are wearing the HMD is acceptable.

[0038] Also, as shown in Figure 6, the warning message may not only be displayed on the display unit 131, but also be output from the audio output unit 141 as an audio output. In this case, if the same warning message is repeated multiple times, the possibility that the user will miss the warning message can be reduced. Also, the audio output of the warning message may be repeated until there is a user instruction to stop the audio output of the warning message.

[0039] As described above, according to this embodiment, even if the user falls asleep while wearing the HMD and then wakes up, the user can check the warning message, and an HMD that can avoid danger can be provided.

[0040] When changing from the awake state to the sleep state, the warning message may be superimposed and displayed on the app screen that was being displayed in the awake state on the display unit 131. Also, after the warning message is confirmed, if the user wishes to switch to the display of the external video, it may be switched by an instruction from the operation input unit 120 by the user, a gesture operation, or the like. Further, according to the instruction of the user, the warning message may be erased and the display of the application screen may be resumed.

Embodiment

[0041] In this embodiment, other danger avoidance examples when wearing the HMD will be described.

[0042] FIG. 7 is a processing flowchart of the HMD in this embodiment. In FIG. 7, first, in step S201, the display control unit 1114 displays the application screen (application video) generated by starting / executing according to the user's operation instruction by the application control unit 1113 on the display unit 131, and proceeds to S202.

[0043] In S202, the communication unit 170 receives a message transmitted from a device on the external network or a device during cooperative operation, for example, a portable information terminal such as a smartphone, and proceeds to S203. In S203, the communication state confirmation unit 1112 analyzes the message received in S202, and if it is determined that the received message is an emergency warning or the like, proceeds to S204. If it is determined that the received message is not an emergency warning or the like, returns to S202.

[0044] In S204, the first image input unit (external camera) 133 acquires the external video, and proceeds to S205. In S205, the display control unit 1114 displays the external video acquired in S204 on the display unit 131, and ends the process.

[0045] As described above, in this embodiment, when the HMD receives an emergency warning (such as an earthquake early warning by email) via the communication unit, it switches the application video to the video-through video (external video) regardless of the waking state or the sleeping state. Thereby, even when wearing a video-transmissive sealed HMD, in an emergency situation, by switching to the external video, the user can check the surrounding situation, and an HMD capable of avoiding danger can be provided.

[0046] In addition, a warning message for notifying the content of an emergency warning may be superimposed and displayed on the display unit.

[0047] Also, when the voice input unit 143 acquires voice information (as an alternative to the process of S202) and analyzes the acquired voice information and determines that the voice information is a fire alarm or the like (as an alternative to the process of S203), it may be switched to an external video.

Embodiment

[0048] In the second embodiment, an HMD capable of avoiding danger by superimposing and displaying a warning message on the display screen when waking up from a sleeping state was described.

[0049] In this embodiment, the point of switching to an external video by gazing at a predetermined icon after the warning message is superimposed and displayed will be described.

[0050] FIG. 8 is a processing flowchart of the HMD in this embodiment. In FIG. 8, first, the steps enclosed by the dotted line are the processes from S101 to S111 in FIG. 5, which are omitted from the description, and the description thereof is also omitted.

[0051] If Yes at S111, it is in a state of waking up from a sleeping state, and the process proceeds to S124. At S124, the display control unit 1114 generates a message for warning the user that the HMD is being worn, and the process proceeds to S125. At S125, the display control unit 1114 superimposes the warning message generated at S124 on the app screen (app video) generated and executed by the app control unit 1113 according to the user's operation instruction, and displays it on the display unit 131.

[0052] FIG. 9 shows an example of displaying a warning message in this embodiment. In FIG. 9, on the display unit 131, the warning message M12 generated in S124 is displayed superimposed on the app screen (app video) AP11. Also, a predetermined icon O11 which is a mark of the closed state of the emergency door is displayed. In the warning message M12, a message "Please look at the door mark if you want to check the outside" is displayed together with a display indicating that the HMD is being worn. As will be described later, when the line of sight is directed at this predetermined icon O11 for a predetermined time or more, the video displayed on the display unit switches from the app video to the outside world video.

[0053] Note that in S125, only the predetermined icon O11 may be displayed. That is, the meaning of the warning message M12 may be included in that the predetermined icon O11 is displayed.

[0054] Next, in S302, the second image input unit (internal camera) 134 acquires the binocular images of the user and proceeds to S303. In S303, the binocular state confirmation unit 1111 analyzes the binocular images of the user acquired in S302 to detect the line-of-sight position of the user and proceeds to S304. In S304, the binocular state confirmation unit 1111 confirms the line-of-sight position of the user detected in S303, and if it is determined that the line-of-sight position of the user is on the predetermined icon, proceeds to S305. If it is determined that the line-of-sight position of the user is not on the predetermined icon, returns to S302. Note that when the line-of-sight position of the user is within a predetermined range including the predetermined icon and larger than the predetermined icon, it may be determined that the line-of-sight position of the user is on the predetermined icon.

[0055] In S305, when it is determined that the user is looking at a predetermined icon, the binocular state confirmation unit 1111 initializes the count value of the timer and proceeds to S306. In S306, the second image input unit (internal camera) 134 acquires a binocular image of the user and proceeds to S307. In S307, the binocular state confirmation unit 1111 analyzes the binocular image of the user acquired in S306 to detect the user's gaze position and proceeds to S308. In S308, the binocular state confirmation unit 1111 confirms the user's gaze position detected in the process of S307. If it is determined that the user's gaze position has deviated from above the predetermined icon (within the predetermined range), the process returns to S302. If it is determined that the user's gaze position has not deviated from above the predetermined icon (within the predetermined range), the process proceeds to S309. In S309, the binocular state confirmation unit 1111 counts up the count value of the timer and proceeds to S310. In S310, the binocular state confirmation unit 1111 compares the count value of the timer with a predetermined value determined in advance. If the count value of the timer ≧ the predetermined value, it is determined that the user has gazed at the predetermined icon for a predetermined time or longer, that is, is staring, and the process proceeds to S311. If the count value of the timer ≧ the predetermined value is not satisfied, the process returns to S306.

[0056] In S311, the first image input unit (external camera) 133 acquires an external video image and proceeds to S312. In S312, the display control unit 1114 displays the external video image acquired in S311 on the display unit 131, and the process ends.

[0057] FIG. 10 is an example of external display in this embodiment. In FIG. 10, an external video image OW11 is displayed on the display unit 131 by S312. That is, when the user gazes at the predetermined icon O11 in FIG. 9 for a predetermined time or longer, the video image displayed on the display unit 131 switches from a warning message in which an application video image is superimposed to an external video image, and at the same time, the predetermined icon O11 with a mark indicating the closed state of the emergency door changes to the predetermined icon O12 with a mark indicating the open state of the emergency door.

[0058] In addition, when the line of sight is directed at a predetermined icon O12 for a predetermined time or longer, the video displayed on the display unit 131 switches from an external video to an app video. Further, the design of the predetermined icon also changes from the predetermined icon O12 with a mark indicating the open state of the emergency door to the predetermined icon O11 with a mark indicating the closed state of the emergency door.

[0059] Note that the predetermined icon does not have to be an emergency door mark, and any icon that suggests an external display may be used.

[0060] Also, the waiting loop for switching to the external view by line-of-sight fixation may be an interrupt process, that is, it may be switched to the app screen display according to an instruction from the user via the operation input unit 120.

[0061] As described above, according to this embodiment, even if the user falls asleep while wearing the HMD and then wakes up later, the user can check the warning message and automatically switch to the external video by gazing at the predetermined icon.

[0062] Note that the predetermined icon O11 may be always displayed superimposed on the app video AP11 during app execution regardless of the transition between the user's sleeping state and waking state. That is, since the user may feel uneasy when the field of view is blocked from the real space by the app video AP11 for a long time, if the user can check the outside world by directing the line of sight at the predetermined icon O11 at any time, danger can be avoided.

[0063] In addition, in each of the above-described embodiments, the case where the HMD is a video-transmissive sealed HMD has been described. However, even when the HMD is an optically transmissive HMD, if the object generated by the execution of the application covers most of the display unit, by performing the same processing as the processing described in each of the above-described embodiments, when the user falls asleep while wearing the HMD and then wakes up, the wearer (user) can check the situation of the surroundings (real space) without the field of view in the real space being blocked by the application screen (application video). When the HMD is an optically transmissive HMD, the process of switching to the video-through video (external camera video) in each of the above-described embodiments may be changed to a process of stopping the display of the application video.

[0064] Although the above embodiments have been described, the above-described embodiments have been described in detail for the purpose of explaining the present invention clearly, and are not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can also be added to the configuration of one embodiment. Also, for a part of the configuration of each embodiment, addition, deletion, or replacement with other configurations is possible.

Description of Reference Numerals

[0065] 100: HMD (Head-Mounted Display Device), 101: Main Control Unit, 103: RAM, 110: Storage Unit, 120: Operation Input Unit, 121: Operation Key, 122: Touch Sensor, 130: Image Processing Unit, 131: Display Unit, 132: Image Signal Processing Unit, 133: First Image Input Unit, 134: Second Image Input Unit, 140: Audio Processing Unit, 141: Audio Output Unit, 142: Audio Signal Processing Unit, 143: Audio Input Unit, 151: Position Information Acquisition Unit, 160: Sensor Unit, 170: Communication Unit, 180: Expansion Interface Unit, 1001: Basic Operation Program, 1002: Application, 1009: Various Data Storage Areas, 1101: Basic Operation Function Unit, 1111: Binocular State Confirmation Unit, 1112: Communication State Confirmation Unit, 1113: Application Control Unit, 1114: Display Control Unit, 1199: Temporary Storage Area, U1: User, M11, M12: Warning Messages, AP11: Application Screen (Application Video), O11, O12: Predetermined Icons, OW11: External World Video.

Claims

1. A video see-through type head-mounted display device that acquires an external image by a first image input unit and displays it on a display unit, a second image input unit that acquires an image of the states of both eyes of the wearer, and has a control unit, wherein the control unit judges that the wearer is in an awake state when a state of closing both eyes does not continue for a predetermined time or more from the image of the states of both eyes of the wearer acquired by the second image input unit, and judges that the wearer is in a sleeping state when a state of closing both eyes continues for a predetermined time or more, when changing from the sleeping state to the awake state, displays a warning message warning that the head-mounted display device is being worn on the display unit, and when changing from the awake state to the sleeping state, switches the display on the display unit from the application image being displayed in the awake state to a no-video screen, and then, when changing from the sleeping state to the awake state, superimposes and displays the warning message on the no-video screen on the display unit. A head-mounted display device characterized by this.

2. A video see-through type head-mounted display device that acquires an external image by a first image input unit and displays it on a display unit, a second image input unit that acquires an image of the states of both eyes of the wearer, and has a control unit, wherein the control unit judges that the wearer is in an awake state when a state of closing both eyes does not continue for a predetermined time or more from the image of the states of both eyes of the wearer acquired by the second image input unit, and judges that the wearer is in a sleeping state when a state of closing both eyes continues for a predetermined time or more, when changing from the sleeping state to the awake state, displays a warning message warning that the head-mounted display device is being worn on the display unit, and when changing from the awake state to the sleeping state, continues to display the application image being displayed in the awake state as the display on the display unit, and then, when changing from the sleeping state to the awake state, superimposes and displays the warning message on the application image on the display unit. A head-mounted display device characterized by this.

3. A video see-through type head-mounted display device that acquires an external image by a first image input unit and displays it on a display unit, a second image input unit that acquires an image of the states of both eyes of the wearer, and has a control unit, wherein the control unit When it is determined that the state of the wearer's both eyes being closed for a predetermined time or more does not continue from the images of the states of the wearer's both eyes acquired by the second image input unit, it is determined that the wearer is in an awake state. When the state of the wearer's both eyes being closed for a predetermined time or more continues, it is determined that the wearer is in a sleeping state. When changing from the awake state to the sleeping state, the head-mounted display device is characterized in that a warning message warning that the head-mounted display device is being worn is superimposed and displayed on the application image being displayed in the awake state on the display unit.

4. The head-mounted display device according to any one of Claims 1 to 3, wherein the control unit displays, as the warning message, an icon suggesting an external display and a message indicating that an external image can be confirmed by looking at the icon. When it is determined from the images of the states of the wearer's both eyes acquired by the second image input unit that the wearer has been looking at the icon for a predetermined time or more, the display on the display unit is switched to the external image acquired by the first image input unit. The head-mounted display device is characterized by this.

5. The head-mounted display device according to Claim 4, wherein the icon is a mark of the closed state of an emergency door. The head-mounted display device is characterized by this.

Images