Control device

Abstract

To provide a technique that enables a head-mounted display device such as an HMD to be conveniently turned off (stop display) and turned on (restart display).SOLUTION: A control device includes first detection means for detecting the movement of a head-mounted display device, second detection means for detecting the wearing of the display device by a user, and control means for transitioning the state of the display device to a first display stop state when a duration time of a state where movement of the display device is not detected reaches a first time in a state where the wearing of the display device by the user is not detected in a case where the state of the display device is a display state, transitioning the state of the display state to the display state when the movement of the display device is detected in a case where the state of the display device is a first display stop state, and transitioning the state of the display device to a second display stop state when the duration time of the state where the wearing of the display device by the user is not detected reaches a second time in a case where the state of the display device is the display state.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a control device that controls a head-mounted display device such as a head-mounted display. [Background technology]

[0002] Mixed reality (MR) technology and augmented reality (AR) technology are known as technologies that seamlessly fuse real space and virtual space in real time. Virtual reality (VR) technology is also known as a technology that represents only virtual space. These technologies are sometimes used in head-mounted display devices such as head-mounted displays (HMDs).

[0003] Displaying an HMD when the user is not using the HMD is unnecessary display, which is undesirable because it leads to unnecessary power consumption and degradation of the display (eg, burn-in).

[0004] Patent Document 1 discloses a technique for performing display on an HMD only when the HMD is worn by a user and the HMD is in motion. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-76680 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology disclosed in Patent Document 1 is inconvenient because the HMD is unnecessarily turned off (display is stopped) or turned on (resuming display) late. For example, when a user is staring at a screen displayed on the HMD, the HMD is unnecessarily turned off even though the user is using the HMD. Also, when a user passes the HMD to another user, the other user is likely to use the HMD immediately. However, the HMD is turned off while the user is passing the HMD to the other user, and the process for turning on the HMD is started after the other user wears the HMD, so the other user cannot immediately start viewing with the HMD.

[0007] An object of the present invention is to provide a technique that can suitably turn off (stop display) and turn on (resume display) a head-mounted display device such as an HMD. [Means for solving the problem]

[0008] A first aspect of the present invention is a control device comprising: a first detection means for detecting movement of a head-mounted display device; a second detection means for detecting wearing of the display device by a user; and a control means for transitioning the state of the display device to a first display stop state in which the display means does not display when a duration of a state in which wearing of the display device by the user is not detected and movement of the display device is not detected reaches a first time when the state of the display device is in a display state in which the display means of the display device performs display, transitioning the state of the display device to the display state when movement of the display device is detected when the state of the display device is in the first display stop state, and transitioning the state of the display device to a second display stop state in which the display means does not display when a duration of a state in which wearing of the display device by the user is not detected reaches a second time when the state of the display device is in the display state. It is a device.

[0009] A second aspect of the present invention is a control method comprising the steps of: detecting movement of a head-mounted display device; detecting wearing of the display device by a user; and, when the state of the display device is a display state in which a display means of the display device displays, if the wearing of the display device by the user is not detected and the duration of the state in which movement of the display device is not detected reaches a first time, transitioning the state of the display device to a first display stop state in which the display means does not display; when the state of the display device is in the first display stop state, if movement of the display device is detected, transitioning the state of the display device to the display state; and, when the state of the display device is in the display state, if the duration of the state in which wearing of the display device by the user is not detected reaches a second time, transitioning the state of the display device to a second display stop state in which the display means does not display.

[0010] A third aspect of the present invention is a head-mounted display device having a display means and the above-mentioned control device. A fourth aspect of the present invention is a program for causing a computer to function as each of the means of the above-mentioned control device. Effect of the Invention

[0011] According to the present invention, it is possible to suitably turn off (stop display) and turn on (resume display) a head-mounted display device such as an HMD. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram of a display system according to a first embodiment. [Diagram 2] 4 is a flowchart of a stop determination process according to the first embodiment. [Diagram 3] 4 is a flowchart of a restart determination process according to the first embodiment. [Figure 4] 4 is a flowchart of the process of S302 in FIG. 3. [Diagram 5] 10 is a flowchart of a stop determination process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] <Embodiment 1> A first embodiment of the present invention will be described. FIG. 1 is a block diagram of a display system according to the first embodiment. The display system of FIG. 1 includes a head mounted display (HMD) 100 and a control device 200. The HMD 100 is a head mounted display device, and is a video see-through display device. The video see-through HMD 100 displays an image of a mixed reality space (or an augmented reality space) in which an image of a real space and an image of a virtual space are combined. The control device 200 is a device that controls the HMD 100, and is, for example, a personal computer, a controller, or a server. The HMD 100 may be an optical see-through display device that displays an image of a virtual space so that a real space that is not an image can be visually recognized. In addition, an example in which a mixed reality (MR) technology or an augmented reality (AR) technology is used in the HMD 100 will be described, but is not limited thereto. For example, a virtual reality (VR) technology may be used in the HMD 100.

[0014] The HMD 100 will now be described. The real space image acquisition unit 11 has a lens and an image sensor, and captures an image of real space. The real space image acquisition unit 11 captures an image in the direction in which the HMD 100 is pointed (the direction in which the face of a user wearing the HMD 100 on his head is pointed). The real space image acquisition unit 11 outputs an image of the real space (real space image) to a transmission unit 16. The HMD 100 may have one real space image acquisition unit 11, or may have multiple real space image acquisition units. For example, the HMD 100 has a real space image acquisition unit 12 for acquiring a real space image for the right eye. and a real space image acquisition unit that acquires a real space image for the left eye.

[0015] The peripheral information acquisition unit 12 has a lens, an imaging element, and an image processing circuit, acquires information (peripheral information) about the situation around the HMD 100, and outputs the peripheral information to the transmission unit 16. The imaging element captures an image of real space. The image processing circuit detects, for example, at least one of markers and feature points from an image captured of real space (real space image). The peripheral information acquisition unit 12 then outputs information including the real space image and the detection results of at least one of the markers and feature points as peripheral information.

[0016] The wearing detection unit 13 detects that the user wears the HMD 100, and outputs wearing information indicating the detection result to the transmission unit 16. For example, the wearing detection unit 13 detects that the user wears the HMD 100 using a gaze sensor or a proximity sensor. The gaze sensor has, for example, a lens, an imaging element, and an image processing circuit. The lens and the imaging element are provided so as to be able to capture an image of the user's eyes when the display unit 18 is in front of the user's eyes. The image processing circuit detects the user's gaze from the image obtained by the imaging element. Then, when the user's gaze is not detected, the image processing circuit determines that the user is not wearing the HMD 100, and when the user's gaze is detected, the image processing circuit determines that the user is wearing the HMD 100. For example, the proximity sensor is provided so as to be able to detect the approach of an object (part of the user's face) when the display unit 18 is in front of the user's eyes. When the proximity sensor does not detect the approach of an object, it determines that the user is not wearing the HMD 100, and when it detects the approach of an object, it determines that the user is wearing the HMD 100. The detection method of the attachment detector 13 is not particularly limited.

[0017] The movement / orientation detection unit 14 detects the movement and orientation of the HMD 100, and outputs movement / orientation information indicating the detection results to the transmission unit 16. For example, the movement / orientation detection unit 14 detects the movement and orientation of the HMD 100 using an acceleration sensor or a gyro sensor.

[0018] The operation unit 15 has one or more operation members such as a button and a switch, and receives operations by the user on the HMD 100. The operation unit 15 outputs operation information to the transmission unit 16 in response to the operation by the user.

[0019] The transmission unit 16 acquires a real space image from the real space image acquisition unit 11, acquires peripheral information from the peripheral information acquisition unit 12, acquires wearing information from the wearing detection unit 13, acquires movement / posture information from the movement / posture detection unit 14, and acquires operation information from the operation unit 15. Then, the transmission unit 16 transmits the acquired information to the control device 200. The reception unit 17 receives information transmitted from the control device 200 (an image of the mixed reality space (MR image) and a display control command for controlling the display unit 18). The communication between the HMD 100 and the control device 200 may be wired communication, wireless communication, or a combination of these.

[0020] The display unit 18 displays various images using a display element such as a liquid crystal element or an EL element. The display unit 18 is provided so as to be located in front of the eyes of a user wearing the HMD 100 on the head. The display unit 18 displays the MR image received by the receiving unit 17 according to a display control command received by the receiving unit 17. The display unit 18 has an optical system, and a virtual image obtained by enlarging the MR image is presented to the user by the optical system. The HMD 100 may have one display unit 18, or may have a display unit for the right eye and a display unit for the left eye. The display unit 18 may be a retinal projection type display unit that projects an image directly onto the retina.

[0021] In the first embodiment, a user wearing the HMD 100 on his / her head can change the state of the HMD 100 from a facing state in which the display unit 18 is placed in front of the user's eyes to a retracted state in which the display unit 18 is retracted from in front of the user's eyes by pushing up the HMD 100. By pressing down on the HMD 100, the wearing state of the HMD 100 can be changed from the retracted state to the facing state. Hereinafter, a state change (change) from the facing state to the retracted state will be referred to as "pull up," and a state change (change) from the retracted state to the facing state will be referred to as "pull down." The HMD 100 may have a mechanism for pulling up and pulling down.

[0022] The control device 200 will be described. The receiving unit 21 receives information (real space image, peripheral information, wearing information, movement / posture information, and operation information) transmitted by the transmitting unit 16 of the HMD 100. The receiving unit 21 transfers the real space image to the image processing unit 24 and the peripheral information to the position / posture calculation unit 22, and transfers the wearing information, movement / posture information, and operation information to the display control unit 26.

[0023] The first processing unit 201 has a position / orientation calculation unit 22 and a CG generation unit 23. The position / orientation calculation unit 22 calculates the position and orientation of the HMD 100 (real space image acquisition unit 11) using peripheral information of the HMD 100 acquired from the reception unit 21. The position and orientation of the HMD 100 can be calculated using various well-known techniques. For example, the position / orientation calculation unit 22 acquires the position (spatial position) of the detected marker in real space using the detection result of the marker (type of detected marker) included in the peripheral information. The position of the marker in real space is determined in advance. In addition, the position / orientation calculation unit 22 acquires the position (image position) of the detected marker in the real space image from the peripheral information. Then, the position / orientation calculation unit 22 calculates the position and orientation of the HMD 100 based on the spatial position and image position of the detected marker. The position / orientation calculation unit 22 may calculate the position and orientation of the HMD 100 using movement orientation information (for example, output value of an acceleration sensor or a gyro sensor). The CG generating section 23 generates an image that appears according to the position and orientation of the HMD 100 calculated by the position / orientation calculating section 22 as an image of the virtual space (virtual space image, graphics).

[0024] The second processing unit 202 has an image processing unit 24 and a synthesis unit 25. The image processing unit 24 performs various image processing on the real space image acquired from the receiving unit 21. The image processing unit 24 performs image processing for obtaining an image suitable for display, for example. The image processing unit 24 may perform development processing, may perform processing to extract a part from the real space image, may perform perspective projection conversion, or may perform equirectangular conversion. The synthesis unit 25 synthesizes (superimposes) the virtual space image generated by the CG generation unit 23 on the real space image after image processing by the image processing unit 24. As a result, an MR image is generated, which is a synthetic image in which the virtual space image is synthesized with the real space image.

[0025] The display control unit 26 uses the wearing information, movement / posture information, and operation information acquired from the receiving unit 21 to determine whether to stop display on the display unit 18 of the HMD 100 and whether to resume display on the display unit 18. For example, the display control unit 26 detects that the user has worn the HMD 100 according to the wearing information acquired from the receiving unit 21, and detects the movement and state change of the HMD 100 according to the movement / posture information acquired from the receiving unit 21. The display control unit 26 also performs various settings related to the HMD 100 according to the operation information acquired from the receiving unit 21. Then, the display control unit 26 determines whether to stop display on the display unit 18 and whether to resume display on the display unit 18 based on the processing results. The display control unit 26 generates a display control command according to these determination results. The display control unit 26 also controls execution / non-execution of processing in the first processing unit 201 and execution / non-execution of processing in the second processing unit 202 using the wearing information, movement / posture information, and operation information acquired from the receiving unit 21.

[0026] The transmission unit 27 transmits the MR image generated by the synthesis unit 25 and the display control command generated by the display control unit 26 to the HMD 100.

[0027] These operations may be realized by one processor or by multiple processors. For example, if the processing of the first processing unit 201 and the processing of the second processing unit 202 are executed in parallel by two different processors, the processing time of the control device 200 can be shortened. Furthermore, the time from when the real space image is acquired by the real space image acquisition unit 11 to when the MR image is displayed by the display unit 18 can be shortened, and real-time performance can be improved.

[0028] 2 is a flowchart showing the operation of the control device 200, and is a flowchart of a stop determination process for determining whether or not to stop display by the display unit 18 of the HMD 100. The stop determination process in FIG. 2 is performed when the state of the HMD 100 is a display state in which the display unit 18 performs display.

[0029] In S201, the display control unit 26 determines whether or not the HMD 100 is worn by the user (detection of wearing of the HMD 100) by using the wearing information acquired from the wearing detection unit 13. If it is determined that the HMD 100 is worn by the user, the process proceeds to S206, and if not, the process proceeds to S202.

[0030] In S202, the display control unit 26 determines whether the power saving priority mode (power saving priority setting) is ON (enabled). If it is determined that the power saving priority mode is ON, the process proceeds to S208, and if not (if it is determined that the power saving priority mode is OFF (disabled)), the process proceeds to S203. The power saving priority mode is an operation mode that enhances the power saving effect by stopping the display of the display unit 18 under looser conditions. The user can use the operation unit 15 to switch the power saving priority mode between ON and OFF.

[0031] In S203, the display control unit 26 judges whether the state of the HMD 100 is a pull-up state (a state in which the HMD 100 has been pulled up). If it is judged that the state of the HMD 100 is a pull-up state, the process proceeds to S208, and if not, the process proceeds to S204. The pull-up state is a retracted state in which the display unit 18 is retracted from in front of the user's eyes while the HMD 100 is worn by the user. The display control unit 26 can detect the pull-up and pull-down of the HMD 100 by using, for example, the motion / posture information acquired from the motion / posture detection unit 14 (using an acceleration sensor or a gyro sensor). For example, if the HMD 100 shows a specific motion such as the rotation of the display unit 18 around a specific axis during the pull-up, the display control unit 26 can detect the pull-up by judging whether the motion / posture information shows a specific motion (a feature amount of the specific motion). A pull-down can also be detected in a similar manner.

[0032] In the first embodiment, in the pull-up state, even if the HMD 100 is worn by a user, the wearing of the HMD 100 is not detected. Therefore, the process of S203 is performed only when the wearing of the HMD 100 is not detected. Note that the wearing of the HMD 100 may be detected even in the pull-up state, and the process of S203 may be performed even when the wearing of the HMD 100 is detected.

[0033] In S204, the display control unit 26 uses the movement / orientation information acquired from the movement / orientation detection unit 14 to determine whether or not there has been movement in the HMD 100 (detection of movement of the HMD 100). For example, when the movement / orientation information indicates the output value of an acceleration sensor or a gyro sensor, the display control unit 26 can detect the movement of the HMD 100 by determining whether or not the output value has changed. Then, the display control unit 26 determines whether or not the duration of the state in which movement of the HMD 100 is not detected (stillness time) has reached a threshold time T1. If it is determined that the stillness time has reached the threshold time T1, the process proceeds to S208, and if not, the process proceeds to S209. Go to 205.

[0034] In S205, the display control unit 26 determines whether or not the duration (non-wearing time) of the state in which the user is not wearing the HMD 100 has reached the threshold time T2. If it is determined that the non-wearing time has reached the threshold time T2, the process proceeds to S207, and if not, the process proceeds to S206.

[0035] In S206, the display control unit 26 generates a display control command to maintain the display on the display unit 18, and transmits the display control command to the HMD 100. This maintains the state of the HMD 100 in the display state. Note that the state of the HMD 100 may be maintained in the display state without transmitting a display control command to the HMD 100. When it is determined in S201 that the HMD 100 is worn by the user, and when it is determined in S205 that the non-wearing time of the HMD 100 has not reached the threshold time T2, the display control unit 26 performs the process of S206.

[0036] In each of S207 and S208, the display control unit 26 generates a display control command to stop the display on the display unit 18, and transmits the display control command to the HMD 100. This causes the state of the HMD 100 to transition to a display stop state in which the display unit 18 does not perform display.

[0037] When it is determined in S205 that the non-wearing time of the HMD 100 reaches the threshold time T2, the display control unit 26 performs the process of S207. In this case, the user has not used the HMD 100 for a long time, and power saving should be prioritized over quickly resuming the display of the display unit 18 when the user resumes use of the HMD 100. Therefore, in S207, the display control unit 26 reduces power consumption by stopping the process of the first processing unit 201 (generation of a virtual space image) and the process of the second processing unit 202 (generation of an MR image).

[0038] If it is determined in S202 that the power saving priority mode is ON, if it is determined in S203 that the state of the HMD 100 is in the pull-up state, or if it is determined in S204 that the rest time of the HMD 100 has reached the threshold time T1, the display control unit 26 performs the process of S208. In these cases, the user may immediately resume use of the HMD 100 by pulling down the HMD 100 or wearing the HMD 100. Therefore, in S208, the display control unit 26 does not stop the process of the first processing unit 201 (generation of a virtual space image) and the process of the second processing unit 202 (generation of an MR image) (the process of the first processing unit 201 and the process of the second processing unit 202 are continued). In this way, the MR image can be continued to be generated, and the display of the display unit 18 can be resumed quickly (in a short time).

[0039] The user may be able to switch the display resume priority mode (display resume priority setting) between ON (enabled) and OFF (disabled) using the operation unit 15. The display resume priority mode is an operation mode that prioritizes quickly resuming the display on the display unit 18 over power saving. When stopping the display on the display unit 18, if the display resume priority mode is ON (enabled), the process of S208 may be performed, and if the display resume priority mode is OFF (disabled), the process of S207 may be performed.

[0040] 3 is a flowchart showing the operation of the control device 200, and is a flowchart of a restart determination process for determining whether or not to restart display by the display unit 18 of the HMD 100. The restart determination process of FIG. 3 is performed when the state of the HMD 100 is a display stop state in which the display unit 18 does not perform display.

[0041] In S301, the display control unit 26 judges whether or not the trigger for stopping the display on the display unit 18 is the process of S203 (determination that the state of the HMD 100 is in the pull-up state). If it is determined that the trigger is the process of S203, the process proceeds to S302; If not, the process proceeds to S303.

[0042] In S302, the display control unit 26 uses the motion / posture information acquired from the motion / posture detection unit 14 (using the acceleration sensor or gyro sensor) to determine whether or not the HMD 100 has been pulled down (detection of pull-down of the HMD 100). If it is determined that pull-down has been performed, the process proceeds to S306, and if not, the process proceeds to S307.

[0043] In S303, the display control unit 26 determines whether the trigger for stopping the display on the display unit 18 is the process of S205 (determining that the non-wearing time of the HMD 100 has reached the threshold time T2). If it is determined that the trigger is the process of S205, the process proceeds to S305. If not (if the trigger is the process of S202 (determining that the power saving priority mode is ON) or the process of S204 (determining that the stationary time of the HMD 100 has reached the threshold time T1)), the process proceeds to S304.

[0044] In S304, the display control unit 26 determines whether or not there has been a movement in the HMD 100 (detection of the movement of the HMD 100) using the movement / posture information acquired from the movement / posture detection unit 14. If it is determined that there has been a movement in the HMD 100, the process proceeds to S306, and if not, the process proceeds to S307.

[0045] In S305, the display control unit 26 determines whether or not the HMD 100 is worn by the user (detection of wearing of the HMD 100) by using the wearing information acquired from the wearing detection unit 13. If it is determined that the HMD 100 is worn by the user, the process proceeds to S306, and if not, the process proceeds to S307.

[0046] In S306, the display control unit 26 generates a display control command to resume display on the display unit 18, and transmits the display control command to the HMD 100. At this time, if the processing of the first processing unit 201 (generation of a virtual space image) and the processing of the second processing unit 202 (generation of an MR image) are stopped, the display control unit 26 resumes the processing of the first processing unit 201 and the processing of the second processing unit 202. As a result, the state of the HMD 100 transitions to a display state in which the display unit 18 performs display. If it is determined in S302 that a pull-down has been performed, if it is determined in S304 that there has been a movement in the HMD 100, or if it is determined in S305 that the HMD 100 has been worn by the user, the display control unit 26 performs the process of S306.

[0047] In S307, the display control unit 26 generates a display control command to maintain the display of the display unit 18 stopped, and transmits the display control command to the HMD 100. This maintains the state of the HMD 100 in the display stopped state. Note that the state of the HMD 100 may be maintained in the display stopped state without transmitting the display control command to the HMD 100. If it is determined in S302 that the pull-down is not being performed (the pull-up state is maintained), and if it is determined in S304 that there is no movement in the HMD 100, the display control unit 26 performs the process of S307. If it is determined in S305 that the HMD 100 is not worn by the user, the display control unit 26 also performs the process of S307.

[0048] In S302, the method of detecting pull-down may be switched depending on whether the display restart priority mode is ON or not. FIG. 4 is a flowchart of the process of S302. In S401, the display control unit 26 judges whether the display restart priority mode (predetermined mode) is ON or not. If it is judged that the display restart priority mode is ON, the process proceeds to S402, and if not (if it is judged that the display restart priority mode is OFF), the process proceeds to S403. In S402, the display control unit 26 judges whether or not a pull-down of the HMD 100 has been performed using the movement / posture information acquired from the movement / posture detection unit 14 (using the acceleration sensor or gyro sensor). For example, in S402, if there is a pull-down movement (posture change), Then, if it is determined that a pull-down movement has occurred, it is determined that a pull-down has been performed, and if not, it is determined that a pull-down has not been performed. In S403, the display control unit 26 uses the wearing information acquired from the wearing detection unit 13 (using a proximity sensor or a line-of-sight sensor) to determine whether or not a pull-down of the HMD 100 has been performed. For example, in S403, it determines whether or not the HMD 100 has been worn by the user. Then, if it is determined that the HMD 100 has been worn by the user, it is determined that a pull-down has been performed, and if not, it is determined that a pull-down has not been performed. If it is determined that a pull-down has been performed, the process proceeds to S306, and if not, the process proceeds to S307.

[0049] A specific example of the operation of the display system (FIG. 1) according to the first embodiment will be described.

[0050] When a user wears the HMD 100 and the display unit 18 is in front of the user's eyes, it can be considered that the user is using the HMD 100. In the first embodiment, it is possible to determine in S201 that the user is wearing the HMD 100 and maintain the display on the display unit 18. As described above, when the user's line of sight is not detected, it may be determined that the user is not wearing the HMD 100, and when the user's line of sight is detected, it may be determined that the user is wearing the HMD 100. By doing so, when the user falls asleep while using the HMD 100, it is possible to determine in S201 that the user is not wearing the HMD 100 and stop the display on the display unit 18.

[0051] When the state of the HMD 100 is the pull-up state, it can be considered that the user is not looking at the display unit 18. In the first embodiment, the pull-up state can be detected in S203 and the display of the display unit 18 can be stopped. The display of the display unit 18 is not resumed until the pull-down is detected in S302. Here, it is assumed that the process of S203 is omitted, and the process proceeds to S204 when it is determined in S202 that the power saving priority mode is OFF. When the user is wearing the HMD 100, the movement of the HMD 100 is detected in S204. Therefore, even if the state of the HMD 100 is the pull-up state, the display of the display unit 18 is maintained until it is determined in S205 that the non-wearing time of the HMD 100 has reached the threshold time T2. As a result, it takes a long time from when the HMD 100 is pulled up until the display of the display unit 18 is stopped, and unnecessary display is performed for a long time. By performing the process of S203, the display of the display unit 18 can be stopped in a short time, and the time of unnecessary display can be shortened.

[0052] When the user removes the HMD 100 from the head and places it on a desk or the like, it can be considered that the user is not using the HMD 100. When the HMD 100 is placed on a desk or the like, it can be considered that the state in which the HMD 100 is not moving continues for a long time. In the first embodiment, it is possible to stop the display on the display unit 18 by determining in S204 that the stationary time of the HMD 100 has reached the threshold time T1. In addition, when the user resumes the use of the HMD 100, the display resume process is started in response to the detection of the movement of the HMD 100 in S304 as a trigger. The display resume process is a process for resuming the display on the display unit 18, and includes, for example, the generation of a display control command, the transmission and reception of the display control command, and the driving of the display unit 18. Therefore, the user can resume viewing on the HMD 100 in a shorter time than when the display start process is performed after the display unit 18 is placed in front of the user's eyes. Similarly, when the user pulls down from a pull-up state to resume use of the HMD 100, if the pull-down is detected using the movement / posture information, the user can immediately resume viewing on the HMD 100.

[0053] When a user passes the HMD 100 to another user, it is considered that the other user will immediately use the HMD, and it is considered that the display on the display unit 18 should be maintained. While the user passes the HMD 100 to the other user, the HMD 100 moves. In the first embodiment, the movement of the HMD 100 is detected in S204, and the display on the display unit 18 can be maintained. If the user does not intend to hand over the HMD 100 to another user, the power saving priority mode can be turned on to stop the display on the display unit 18 and suppress unnecessary display.

[0054] When the user forgets to turn off the power of the HMD 100 and carries the HMD 100 in a bag or the like, it can be considered that the user is not using the HMD 100 and the display on the display unit 18 should be stopped. When the HMD 100 is being carried, there is movement in the HMD 100, but the HMD 100 is not worn by the user. In the first embodiment, it is possible to stop the display on the display unit 18 by determining in S205 that the non-wearing time of the HMD 100 reaches the threshold time T2. Note that if the process of S205 is omitted and the process proceeds to S206 when the movement of the HMD 100 is detected in S204, the display on the display unit 18 will continue.

[0055] When a user uses the HMD 100 at a high place or while walking, if a state occurs in which the user cannot see the real space, the safety of the user is threatened. In the first embodiment, when quick resumption of display on the display unit 18 is to be prioritized over power saving, the processing of the first processing unit 201 and the processing of the second processing unit 202 are continued, so that the MR image can be continuously generated and the display on the display unit 18 can be quickly (in a short time) resumed.

[0056] As described above, according to the first embodiment, it is possible to suitably turn off (stop display) and turn on (resume display) a head-mounted display device such as an HMD.

[0057] <Embodiment 2> A description will now be given of a second embodiment of the present invention. Note that in the following, descriptions of the same points as in the first embodiment (for example, the same configurations and processes as in the first embodiment) will be omitted as appropriate.

[0058] FIG. 5 is a flowchart of the stop determination process according to the second embodiment. S501 to S507 are the same as S201 to S207 in FIG. 2. In S508, similar to S208, the display control unit 26 generates a display control command to stop the display of the display unit 18, and transmits the display control command to the HMD 100. However, unlike S208, the display control unit 26 does not stop the process of the second processing unit 202 (generation of an MR image), but stops the process of the first processing unit 201 (generation of a virtual space image). When the process of the first processing unit 201 is stopped and the process of the second processing unit 202 is continued, the second processing unit 202 generates a real space image (through image) as an MR image. By doing so, not only can the MR image be continued to be generated and the display of the display unit 18 be quickly (in a short time) resumed, but the process of the first processing unit 201 can be stopped to reduce power consumption.

[0059] The above embodiment (including the modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of the above embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above embodiments. For example, a part of the processing of the HMD 100 may be performed by the control device 200, or a part of the processing of the control device 200 may be performed by the HMD 100. The entire control device 200 may be built into the HMD 100.

[0060] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0061] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) a first detection means for detecting a movement of the head-mounted display device; A second detection means for detecting attachment of the display device by a user; when the state of the display device is a display state in which a display means of the display device performs display, if a duration of a state in which wearing of the display device by the user is not detected and a state in which movement of the display device is not detected reaches a first time period, transitioning the state of the display device to a first display stop state in which the display means does not perform display; When a state of the display device is the first display stop state, if a movement of the display device is detected, the state of the display device is transitioned to the display state; a control means for transitioning the state of the display device to a second display stop state in which the display means does not perform display when a duration of a state in which wearing of the display device by the user is not detected reaches a second time period while the state of the display device is the display state; A control device comprising: (Configuration 2) The display device can be in a first state in which the display means is disposed in front of the user while being worn by the user, and in a second state in which the display means is retracted from in front of the user while being worn by the user, The control device further includes a third detection means for detecting a state change between the first state and the second state, The control means when a state change from the first state to the second state is detected while the state of the display device is the display state, transitioning the state of the display device to a third display stop state in which the display means does not perform display; The control device according to configuration 1, characterized in that when a state change from the second state to the first state is detected while the state of the display device is the third display stop state, the control device transitions the state of the display device to the display state. (Configuration 3) The third detection means detects a state change between the first state and the second state by using an acceleration sensor or a gyro sensor. 3. The control device according to configuration 2. (Configuration 4) The third detection means is Detecting a state change from the first state to the second state using an acceleration sensor or a gyro sensor; when the state of the display device is the third display stop state and a predetermined mode is set, detecting a state change from the second state to the first state using the acceleration sensor or the gyro sensor; When the state of the display device is the third display stop state and the predetermined mode is not set, the third detection means detects a state change from the second state to the first state using a proximity sensor or a line of sight sensor. 3. The control device according to configuration 2. (Configuration 5) When displaying an image on the display means, a graphic is generated, a composite image is generated by combining the graphic with an image in real space, and the composite image is displayed on the display means; In the first display stop state, the generation of the graphic and the generation of the composite image are continued, In the second display stop state, the generation of the graphics and the generation of the composite image are stopped. Will be 5. The control device according to any one of configurations 1 to 4. (Configuration 6) When displaying an image on the display means, a graphic is generated, a composite image is generated by combining the graphic with an image in real space, and the composite image is displayed on the display means; In the first display stop state, generation of the graphic is stopped, and generation of the composite image is continued such that the image of the real space is generated as the composite image; In the second display stop state, generation of the graphic and generation of the composite image are stopped. 5. The control device according to any one of configurations 1 to 4. (Configuration 7) The first detection means detects the movement of the display device using an acceleration sensor or a gyro sensor. 7. The control device according to any one of configurations 1 to 6. (Configuration 8) The second detection means detects that the user is wearing the display device by using a proximity sensor or a line-of-sight sensor. 8. The control device according to any one of configurations 1 to 7. (Configuration 9) A display means; A control device according to any one of configurations 1 to 8. A head-mounted display device comprising: (method) Detecting movement of the head mounted display device; detecting attachment of the display device by a user; when the state of the display device is a display state in which a display means of the display device performs display, if a duration of a state in which wearing of the display device by the user is not detected and a state in which movement of the display device is not detected reaches a first time period, transitioning the state of the display device to a first display stop state in which the display means does not perform display; When a state of the display device is the first display stop state, if a movement of the display device is detected, the state of the display device is transitioned to the display state; and when a duration of a state in which wearing of the display device by the user is not detected reaches a second time period while the state of the display device is the display state, transitioning the state of the display device to a second display stop state in which the display means does not perform display. A control method comprising the steps of: (program) A program for causing a computer to function as each of the means of the control device according to any one of configurations 1 to 8. [Explanation of symbols]

[0062] 200: Control device 26: Display control unit

Claims

1. a first detection means for detecting a movement of the head mounted display device; A second detection means for detecting attachment of the display device by a user; when the state of the display device is a display state in which a display means of the display device performs display, if a duration of a state in which wearing of the display device by the user is not detected and a state in which movement of the display device is not detected reaches a first time period, transitioning the state of the display device to a first display stop state in which the display means does not perform display, When a state of the display device is the first display stop state, if a motion of the display device is detected, the state of the display device is transitioned to the display state; a control means for transitioning the state of the display device to a second display stop state in which the display means does not perform display when a duration of a state in which wearing of the display device by the user is not detected reaches a second time period while the state of the display device is the display state; A control device comprising:

2. The display device can be in a first state in which the display means is disposed in front of the user while being worn by the user, and in a second state in which the display means is retracted from in front of the user while being worn by the user, The control device further includes a third detection means for detecting a state change between the first state and the second state, The control means when a state change from the first state to the second state is detected while the state of the display device is the display state, transitioning the state of the display device to a third display stop state in which the display means does not perform display; The control device according to claim 1, characterized in that when a state change from the second state to the first state is detected while the state of the display device is the third display stop state, the control device transitions the state of the display device to the display state.

3. The third detection means detects a state change between the first state and the second state by using an acceleration sensor or a gyro sensor.

3. The control device according to claim 2.

4. The third detection means is Detecting a state change from the first state to the second state using an acceleration sensor or a gyro sensor; when the state of the display device is the third display stop state and a predetermined mode is set, detecting a state change from the second state to the first state using the acceleration sensor or the gyro sensor; When the state of the display device is the third display stop state and the predetermined mode is not set, the third detection means detects a state change from the second state to the first state by using a proximity sensor or a line of sight sensor.

3. The control device according to claim 2.

5. When displaying an image on the display means, a graphic is generated, a composite image is generated by combining the graphic with an image in real space, and the composite image is displayed on the display means; In the first display stop state, the generation of the graphic and the generation of the composite image are continued, In the second display stop state, generation of the graphics and generation of the composite image are stopped. The control device according to claim 1 .

6. When displaying an image on the display means, a graphic is generated, a composite image is generated by combining the graphic with an image in real space, and the composite image is displayed on the display means; In the first display stop state, generation of the graphic is stopped, and generation of the composite image is continued such that the image of the real space is generated as the composite image; In the second display stop state, generation of the graphics and generation of the composite image are stopped. The control device according to claim 1 .

7. The first detection means detects the movement of the display device using an acceleration sensor or a gyro sensor. The control device according to claim 1 .

8. The second detection means detects that the user is wearing the display device by using a proximity sensor or a line-of-sight sensor. The control device according to claim 1 .

9. A display means; A control device according to any one of claims 1 to 8. A head-mounted display device comprising:

10. Detecting movement of the head mounted display device; detecting attachment of the display device by a user; when the state of the display device is a display state in which a display means of the display device performs display, if a duration of a state in which wearing of the display device by the user is not detected and a state in which movement of the display device is not detected reaches a first time period, transitioning the state of the display device to a first display stop state in which the display means does not perform display, When a state of the display device is the first display stop state, if a motion of the display device is detected, the state of the display device is transitioned to the display state; when a duration of a state in which wearing of the display device by the user is not detected reaches a second time period while the state of the display device is the display state, transitioning the state of the display device to a second display stop state in which the display means does not perform display; A control method comprising the steps of:

11. A program for causing a computer to function as each of the means of the control device according to any one of claims 1 to 8.

Images