Head-mounted display device, authentication method, and authentication program

Abstract

To perform high-security authentication by use of a head-mounted display.SOLUTION: An HMD including a face direction detection unit which detects a face direction of a user, and a line-of-sight detection unit which detects a line of sight of the user, changes an object to be displayed in an image display area of a display, in accordance with a result detected by the face direction detection unit. The HMD executes user authentication when the line-of-sight detection unit detects that the user looks at an authentication object which has been displayed in the image display area in accordance with user action.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a head mounted display device, an authentication method, and an authentication program. [Background technology]

[0002] User authentication may be performed when starting up an information processing device or when accessing a computer network, etc. Only if this user authentication is successful can the information processing device be used or the computer network be accessed.

[0003] For example, user authentication requires input of a password via a keyboard. However, to prevent password leakage due to the use of a keyboard, more secure user authentication methods have been developed.

[0004] As a highly secure user authentication method, Patent Document 1 discloses a display system that detects the line of sight of a user gazing at an image displayed on a screen, detects the image the user is gazing at based on the line of sight, and performs user authentication using the detected image as authentication information. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-149326 Summary of the Invention [Problem to be solved by the invention]

[0006] In recent years, head mounted displays (HMDs), which have a display attached to the user's head, have become popular. Although it is expected that head mounted display devices will use user authentication that detects the user's line of sight as described in Patent Document 1, authentication with higher security is required.

[0007] The present invention has been made in view of the above circumstances, and has an object to provide a head mounted display device, an authentication method, and an authentication program that can perform authentication with higher security. [Means for solving the problem]

[0008] In order to solve the above problems, the head mounted display device, authentication method, and authentication program of the present invention employ the following means.

[0009] In order to solve the above problem, a "head-mounted display device" which is one aspect of the present invention is a head-mounted display device in which a display is mounted on the head of a user so that it can be seen by the user, and comprises: a face direction detection means which detects the direction of the user's face; a gaze detection means which detects the line of sight of the user; an image display control means which changes an object to be displayed in an image display area of ​​the display according to the detection result by the face direction detection means; and an authentication object used for authentication is displayed in the image display area according to the user's actions, and authentication means which performs authentication when the gaze detection means detects that the user has focused their line of sight on the authentication object.

[0010] In order to solve the above problem, one aspect of the present invention is an "authentication method" for a head-mounted display device in which a display is attached to the user's head so that the user can see it, and which is equipped with a face direction detection means for detecting the direction of the user's face and a gaze detection means for detecting the user's gaze, and which includes a first step of changing an object to be displayed in an image display area of ​​the display according to the detection result by the face direction detection means, and a second step of displaying an authentication object to be used for authentication in the image display area according to the user's movement, and performing authentication when the gaze detection means detects that the user has focused their gaze on the authentication object.

[0011] In order to solve the above problem, an "authentication program" which is one aspect of the present invention causes a computer possessed by a head-mounted display device in which a display is attached to a user's head so that the user can see it, and which is equipped with a face direction detection means for detecting the direction of the user's face and a gaze detection means for detecting the user's gaze, to function as: an image display control means for changing an object to be displayed in an image display area of ​​the display in accordance with the detection result by the face direction detection means; and an authentication means for displaying an authentication object used for authentication in the image display area in accordance with the user's actions, and for performing authentication when the gaze detection means detects that the user has focused their gaze on the authentication object.

[0012] Various technical limitations may be imposed on the above-mentioned "head mounted display device," as exemplified below. Furthermore, technical limitations of the same nature may be imposed on the processing steps executed by the "authentication method" or the functions of the "authentication program."

[0013] The authentication object is not displayed in the image display area when authentication starts, but is displayed in the image display area depending on the user's action.

[0014] An authentication display area is set in the image display area, and the authentication means performs authentication when the authentication object is displayed in the authentication display area in response to the user's actions and the gaze detection means detects that the user has focused their gaze on the authentication object.

[0015] The user's action is the user changing the direction of his / her face or the user moving the authentication object.

[0016] The authentication means performs authentication based on the detection results of the line of sight detection means and the detection results of the face direction detection means.

[0017] When a real, pre-registered identifying object is displayed on the display, the authentication object is displayed on the display.

[0018] The authentication object is a real identifying entity that is displayed on the display.

[0019] The authentication means determines that the user has started the authentication operation when it detects that the user has directed his / her gaze at the authentication object for a predetermined time or longer, and that the user has ended the authentication operation when it detects that the user has averted his / her gaze from the authentication object for a predetermined time or longer.

[0020] The authentication means starts or ends the authentication operation when it detects that the user has blinked a predetermined number of times or more within a predetermined time period.

[0021] The authentication means starts or ends the authentication operation when it detects that the user has made a predetermined movement of the head. [Effects of the Invention]

[0022] According to the present invention, it is possible to perform authentication with higher security using a head-mounted display. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a configuration diagram of a display system according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing an image display area of ​​the HMD according to the first embodiment of the present invention. [Figure 3] 1 is a functional block diagram showing the electrical configuration of an HMD according to a first embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram showing user authentication using an HMD according to the first embodiment of the present invention. [Figure 5] FIG. 2 is a schematic diagram showing an authentication display area according to the first embodiment of the present invention. [Figure 6] 4 is a flowchart showing the flow of user authentication processing according to the first embodiment of the present invention. [Figure 7] FIG. 10 is a schematic diagram showing user authentication using an HMD according to a second embodiment of the present invention. [Figure 8] FIG. 10 is a schematic diagram showing user authentication using an HMD according to a third embodiment of the present invention. [Figure 9] FIG. 10 is a functional block diagram showing the electrical configuration of an HMD according to a third embodiment of the present invention. [Figure 10] 10 is a flowchart showing the flow of user authentication processing according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0024] Hereinafter, an embodiment of a head mounted display device, an authentication method, and an authentication program according to the present invention will be described with reference to the drawings.

[0025] [1. First embodiment] A first embodiment of the present invention will be described below.

[0026] [1-1. Display system configuration] FIG. 1 is a diagram showing the configuration of a display system 10 according to this embodiment.

[0027] The display system 10 includes a head mounted display device (hereinafter referred to as an “HMD”) 12 and a server 14 .

[0028] The HMD 12 is provided with a display 16 such as an LCD (Liquid Crystal Display) on the user's head so that the display 16 can be viewed by the user. The display 16 is fixed to the user's head by, for example, an elastic band 18.

[0029] The external shape of the HMD 12 shown in FIG. 1 is an example, and the display 16 is positioned in front of both eyes. However, the present invention is not limited to this, and the HMD 12 may be a type that projects an image onto the lenses of glasses, as in the case of glasses. Furthermore, the HMD 12 may be a type that does not allow the user to view an image with both eyes, but allows the user to view an image with one eye. The HMD 12 may be either a transparent type that allows the user to view the outside world even when worn by the user, or a non-transparent type that does not allow the user to view the outside world. The image that the HMD 12 displays on the display 16 may be either virtual reality (VR) or augmented reality (AR). In the HMD 12 according to this embodiment, VR is displayed on the display 16.

[0030] The HMD 12 and the server 14 are capable of transmitting and receiving information via a communication line 20. The HMD 12 may have a communication function for directly accessing the communication line 20, or may access the communication line 20 via another information processing device such as a mobile phone terminal.

[0031] The server 14 stores various information such as programs executed by the HMD 12, and transmits and receives information to and from the HMD 12 as necessary.

[0032] 2, the HMD 12 has an image display area 22 provided on the display 16 so as to cover the user's field of view. Various images (also referred to as objects) are displayed in the image display area 22. Note that the image display area 22 according to this embodiment substantially coincides with the field of view of the user wearing the HMD 12.

[0033] The HMD 12 according to this embodiment performs user authentication when the user activates the HMD 12, and allows the user to operate the HMD 12 if the user authentication is successful. An authentication object 24, which is an object used for user authentication, is displayed in the image display area 22. Then, when a line-of-sight detection unit 32 (see FIG. 3) provided in the HMD 12 detects that the user has focused their line of sight on the authentication object 24, user authentication is performed.

[0034] As an example, the authentication object 24 according to this embodiment is formed of a plurality of elements (hereinafter referred to as "authentication elements") 26, of which authentication is successful when at least one of them is visually recognized. Furthermore, in user authentication according to this embodiment, authentication is successful when the plurality of authentication elements 26 are visually recognized in a predetermined order, but this is not limiting, and the order in which the plurality of authentication elements 26 are visually recognized may not be predetermined, or the condition for successful authentication may be the length of time the authentication elements 26 are continuously visually recognized. Note that the authentication object 24 may be a single image instead of being formed of a plurality of authentication elements 26.

[0035] [1-2. HMD functional block diagram] FIG. 3 is a functional block diagram showing the electrical configuration of the HMD 12 according to this embodiment.

[0036] In addition to the display 16, the HMD 12 includes a face direction detection unit 30, a line of sight detection unit 32, a main control unit , a storage unit , and a communication unit .

[0037] The face direction detection unit 30 includes, for example, an acceleration sensor, a gyro sensor, or the like, and detects the direction of the face of the user wearing the HMD 12.

[0038] The line-of-sight detection unit 32 includes, for example, a camera that captures an image of the user's eyes, and detects the line of sight of the user based on the image of the user's eyes.

[0039] The main control unit 34 is, for example, a CPU (Central Processing Unit), a microprocessor, a DSP (Digital Signal Processor), or the like, and controls the overall operation of the HMD 12.

[0040] The memory unit 36 ​​is composed of, for example, RAM (Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, and stores a work area when the main control unit 34 executes processing based on various programs, various data such as images, and programs used for processing by the main control unit 34.

[0041] The communication unit 38 has a function of connecting to the communication line 20, and is, for example, a NIC (Network Interface Controller), and connects to a communication network such as a mobile phone network. Note that the communication unit 38 may have, instead of or together with the NIC, a function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), and a function of enabling short-range wireless communication such as Bluetooth (registered trademark), infrared communication, etc.

[0042] The display 16, face direction detection unit 30, line of sight detection unit 32, main control unit 34, storage unit 36, and communication unit 38 are electrically connected to one another via a system bus. Therefore, the main control unit 34 can display images on the display 16, perform various processes using the detection results of the face direction detection unit 30 and line of sight detection unit 32, access the storage unit 36, and access various communication networks and other terminals via the communication unit 38. In addition to the above configuration, the HMD 12 is also equipped with a speaker and earphone jack for outputting sound, a USB (Universal Serial Bus) connector for transmitting and receiving data to and from other information processing devices, etc.

[0043] The HMD 12 can be connected to a controller by wire or wirelessly, and can be operated by the user using the controller.

[0044] The main control unit includes an image display control unit 40 and an authentication processing unit 42. Each unit included in the main control unit is realized by a program stored in the storage unit .

[0045] The image display control unit 40 outputs data indicating an image (object) to be displayed in the image display area 22 to the display 16. In addition, the image display control unit 40 changes the object to be displayed in the image display area 22 according to the detection result by the face direction detection unit 30.

[0046] The authentication processing unit 42 executes processing related to user authentication.

[0047] [1-3. User authentication according to the first embodiment] User authentication according to this embodiment will be described.

[0048] In this embodiment, user authentication is performed when an authentication object 24 is displayed in the image display area 22 in response to the user's actions, and the gaze detection unit 32 detects that the user has focused their gaze on the authentication object 24.

[0049] Fig. 4 is a schematic diagram showing user authentication according to this embodiment, and shows an image space 44 displayed by the HMD 12. Since the HMD 12 according to this embodiment displays virtual reality (VR), the image space 44 shown in Fig. 4 is a VR space. The area surrounded by a dashed line in Fig. 4 is the entire image space 44 that can be viewed by a user wearing the HMD 12 by moving their head. On the other hand, the area surrounded by a solid line is the image space 44 displayed in the image display area 22, and is the range that is actually viewed by the user wearing the HMD 12.

[0050] As shown in Fig. 4(A), the authentication object 24 exists in the image space 44, but is not displayed in the image display area 22 at the start of user authentication. Therefore, the user turns his or her face to cause the authentication object 24 existing in the image space 44 to be displayed in the image display area 22, as shown in Fig. 4(B).

[0051] Furthermore, in user authentication according to this embodiment, an authentication display area 22A is set in the image display area 22. As shown in Fig. 5, the image display area 22 is divided into a pseudo-authentication display area 22A and a non-authentication display area 22B. Then, user authentication is performed when the authentication object 24 is displayed in the authentication display area 22A.

[0052] When the authentication object 24 is displayed in the image display area 22, a display may be made to prompt the user to display the authentication object 24 in the authentication display area 22A. For example, the HMD 12 flashes the authentication display area 22A, displays letters or arrows, or makes an audio announcement to prompt the user to place the authentication object 24 into the authentication display area 22A.

[0053] 5 is an example, and other divisions may be used. Furthermore, the image display area 22 may not be provided with the non-authentication display area 22B, and the authentication display area 22A may be the entire image display area 22.

[0054] The HMD 12 then determines whether authentication is successful or not based on whether the result of gaze detection by the gaze detection unit 32 matches the registered authentication gaze information. The authentication gaze information is generated when the user looks at the authentication object 24 in order to register it, and is stored in advance in the storage unit 36.

[0055] [1-4. Determining the start or end of authentication operations] Next, the start or end of the authentication operation by the user will be described.

[0056] The authentication processing unit 42 determines that the authentication operation starts when the user performs a predetermined operation on the controller connected to the HMD 12, and that the authentication operation ends when the user performs another predetermined operation on the controller. Note that the predetermined operation on the controller is, for example, clicking a predetermined button.

[0057] Then, the gaze detection unit 32 starts detecting the user's gaze after the authentication processing unit 42 receives the start of the authentication operation by the user. Also, the gaze detection unit 32 ends the detection of the user's gaze after the authentication processing unit 42 receives the end of the authentication operation by the user. Thereafter, the authentication processing unit 42 determines whether the user authentication has been successful or not based on the gaze detection result by the gaze detection unit 32.

[0058] The start or end of the authentication operation by the user may be determined based on the detection result by the line of sight detection unit 32 or the face direction detection unit 30, without being limited to a predetermined operation on the controller.

[0059] For example, the authentication processing unit 42 determines that the user's authentication operation has started when the gaze detection unit 32 detects that the user has directed their gaze at the authentication object 24 for a predetermined period of time or longer. The authentication processing unit 42 then determines that the user's authentication operation has ended when the gaze detection unit 32 detects that the user has averted their gaze from the authentication object 24 for a predetermined period of time or longer. Note that the predetermined period of time is not the period during which the user unconsciously directs or averts their gaze, but a period of time during which it can be determined that the user did so intentionally, such as a few seconds.

[0060] Furthermore, the authentication processing unit 42 starts or ends the authentication operation when the gaze detection unit 32 detects that the user has blinked a predetermined number of times or more within a predetermined time. A predetermined number of times or more within a predetermined time is a number that can be determined to be blinks that the user intentionally performed, rather than blinks that the user unconsciously performed, such as three or more blinks within one second.

[0061] Furthermore, the authentication processing unit 42 starts or ends the authentication operation when the face direction detection unit 30 detects that the user has made a predetermined head movement. The predetermined movement is, for example, a predetermined number of times in a predetermined direction within a predetermined time, which is not an unconscious movement made by the user but a number of times that can be determined to be an intentional movement made by the user, such as tilting the head to the right twice in one second.

[0062] In addition, the authentication processing unit 42 may determine that the user has started the authentication operation when the gaze detection unit 32 detects that the user has directed their gaze at a specified object displayed on the display 16 for a specified period of time or more, and may determine that the user has ended the authentication operation when the gaze detection unit 32 detects that the user has again directed their gaze at the specified object for a specified period of time or more.

[0063] The start or end of any authentication operation is determined when the authentication object 24 is displayed in the authentication display area 22A. When the HMD 12 receives a user's action indicating the start or end of the authentication operation, it issues a notification to that effect (hereinafter referred to as an "authentication start notification" or an "authentication end notification"). The authentication start notification or authentication end notification is, for example, the display of a predetermined image on the display 16 or the output of a predetermined sound.

[0064] The notification of the start of authentication allows the user to recognize that the authentication operation can be performed, and the notification of the end of authentication allows the user to recognize that the HMD 12 has accepted the authentication operation performed by the user.

[0065] In this way, the HMD 12 according to this embodiment accepts the start or end of the authentication operation when the user performs a predetermined action, which allows the user to perform user authentication at their own timing and prevents erroneous detection of the line of sight during user authentication.

[0066] [1-5. User authentication process according to this embodiment] 6 is a flowchart showing the flow of user authentication processing executed by the main control unit 34. A program for executing the user authentication processing is stored in advance in a predetermined area of ​​the storage unit 36. Note that the user authentication processing according to this embodiment is executed, for example, when the user starts up the HMD 12.

[0067] First, in step 100, it is determined whether or not the authentication object 24 is displayed in the authentication display area 22A, and if the determination is affirmative, the process proceeds to step 102. Note that when the HMD 12 is started up, the authentication object 24 is not displayed in the authentication display area 22A. Therefore, the user moves their head or the like to cause the authentication object 24 to be displayed in the authentication display area 22A.

[0068] In step 102, the authentication processing unit 42 determines whether or not the user has performed a predetermined action indicating the start of the authentication operation, and if the determination is affirmative, the process proceeds to step 104. Note that if the determination is affirmative, the authentication processing unit 42 issues a notification that authentication has started.

[0069] In step 104, the line of sight detection unit 32 performs line of sight detection for user authentication.

[0070] In the next step 106, the authentication processing unit 42 determines whether or not the user has performed a predetermined action indicating that the authentication operation is to be terminated, and if the determination is affirmative, the process proceeds to step 108, whereas if the determination is negative, the process returns to step 104. Note that if the determination is affirmative, the authentication processing unit 42 notifies the user that authentication has been terminated.

[0071] In step 108, the authentication processing unit 42 performs authentication based on the result of the gaze detection, and then ends the user authentication process. Specifically, the success or failure of authentication is determined depending on whether the result of gaze detection on the authentication object 24 matches the registered authentication gaze information. If authentication is successful, the user is allowed to operate the HMD 12. On the other hand, if user authentication fails, the process returns to step 100, and the user authentication process is performed again.

[0072] As described above, the HMD 12 according to this embodiment includes a face direction detection unit 30 that detects the direction of the user's face and a gaze detection unit 32 that detects the user's gaze, and changes the object displayed in the image display area 22 of the display 16 according to the detection result by the face direction detection unit 30. The HMD 12 then displays the authentication object 24 in the image display area according to the user's actions, and performs user authentication when the gaze detection unit 32 detects that the user has focused their gaze on the authentication object 24.

[0073] In this way, the HMD 12 according to this embodiment finds the authentication object 24 when the user moves his or her head, and displays the authentication object 24 on the display 16. The HMD 12 then performs user authentication by detecting the user's line of sight aligned with the displayed authentication object 24, thereby enabling authentication with higher security.

[0074] The authentication object 24 may also be movable by the user in a simulated manner. In this case, for example, when the authentication object 24 is displayed in the non-authentication display area 22B, the user simulates grasping the authentication object 24 with their hand and moving the authentication object 24 to the authentication display area 22A. The operation of grasping the authentication object 24 may be performed via a controller connected to the HMD 12.

[0075] Alternatively, multiple authentication objects 24 may exist in the image space 44. In this case, only one of the multiple authentication objects 24 is registered in advance as the object to be used for user authentication. The user moves his or her head to find one of the registered authentication objects 24 and have it displayed on the display 16, thereby performing user authentication.

[0076] The authentication object 24 may also move. In this case, for example, a plurality of authentication objects 24 are displayed on the display 16 while moving. The user moves his / her head to find one of the registered authentication objects 24 and have it displayed on the display 16, thereby performing user authentication.

[0077] [2. Second Embodiment] A second embodiment of the present invention will now be described.

[0078] [2-1. Configuration of HMD according to the second embodiment]

[0079] The configurations of the display system 10 and HMD 12 according to the second embodiment are similar to the configurations of the display system 10 and HMD 12 according to the first embodiment shown in FIGS. 1 to 3, and therefore a description thereof will be omitted.

[0080] [2-2. User authentication according to the second embodiment] The authentication processing unit 42 according to this embodiment performs user authentication based on the result of gaze detection by the gaze detection unit 32 and the result of face direction detection by the face direction detection unit 30. That is, the condition for successful user authentication is that the user looks at the authentication object 24 and that the direction of the face is a pre-registered direction.

[0081] The pre-registered directions are, for example, up, down, right, and left. In the example of Fig. 7, authentication is successful when the face direction detected by the face direction detection unit 30 and the gaze direction detected by the gaze detection unit 32 are opposite to each other.

[0082] This allows the HMD 12 according to this embodiment to perform user authentication with higher security.

[0083] 3. Third Embodiment A third embodiment of the present invention will now be described.

[0084] [3-1. Configuration of HMD according to the third embodiment]

[0085] The configurations of the display system 10 and the HMD 12 according to this embodiment are shown in FIGS. 1 and 2. The configuration of the display system 10 and the HMD 12 according to the first embodiment is the same as that of the display system 10 and the HMD 12 according to the first embodiment, and therefore the description thereof will be omitted.

[0086] The image displayed on the display 16 by the HMD 12 according to this embodiment is AR. For this reason, a camera 56, which will be described in detail later, is provided on the front of the HMD 12, and captures an image (hereinafter referred to as a "real space image") of the surroundings (hereinafter referred to as a "real space") of the user wearing the HMD 12. AR is realized by displaying an object together with the real space image on the display 16.

[0087] [3-2. User authentication according to the third embodiment] As shown in FIG. 8 , the HMD 12 according to this embodiment displays an authentication object 24 on the display 16 when a pre-registered identifying object (hereinafter referred to as a "registered identifying object") 50 is displayed on the display 16. The registered identifying object 50 is, for example, an existing object, and characteristic parts such as shape and color for identifying the registered identifying object 50 are stored in the memory unit 36. The characteristic parts of the registered identifying object 50 are extracted by performing image analysis on a real-space image acquired when registering the registered identifying object 50.

[0088] In the example of FIG. 8 , the registered identification object 50 is a monitor 54 placed on a desk 52, and when this monitor 54 is displayed on the display 16, the authentication object 24 is displayed on the display 16. In other words, the initial position of the authentication object 24 is predetermined in real space. That is, the authentication object 24 does not exist in the image space until the registered identification object 50 is displayed in the authentication display area 22A. Therefore, when the registered identification object 50 is not displayed on the display 16, the user moves his or her head to cause the registered identification object 50 to be displayed on the display 16.

[0089] The registered identification object 50 shown in Fig. 8 is just an example, and may be another object such as a clock or a window, but an object whose position does not change in real space is preferable. Also, in the example of Fig. 8, the authentication object 24 is displayed superimposed on the registered identification object 50, but superimposition is not necessarily required.

[0090] When the authentication object 24 is displayed on the display 16, the user moves his / her head so that the authentication object 24 is displayed in the authentication display area 22A. After the authentication object 24 is displayed in the authentication display area 22A, the user performs user authentication by aligning his / her line of sight with the authentication object 24.

[0091] In addition, if the user moves their head after the authentication object 24 is displayed on the display 16 and the registered identification object 50 is no longer displayed on the display 16, the HMD 12 may hide the authentication object 24 that was displayed, or may continue to display the authentication object 24, thereby enabling user authentication.

[0092] Fig. 9 is a functional block diagram showing the electrical configuration of the HMD 12 according to this embodiment. Note that the same components in Fig. 9 as those in Fig. 3 are given the same reference numerals as in Fig. 3, and their description will be omitted.

[0093] The camera 56 is provided on the front surface of the HMD 12 and captures a real space image of the user's surroundings.

[0094] The image display control unit 40 causes the display 16 to display in real time the real-space image acquired by the camera 56, and also causes the display 16 to display a non-existent image (object) superimposed on the real-space image.

[0095] The camera image analysis unit 58 performs image analysis on the real space image to identify the object captured by the camera 56. That is, the real space image is subjected to image analysis while being displayed on the display 16.

[0096] The authentication processing unit 42 determines whether the object identified by the camera image analysis unit 58 matches the registered identification object 50 by image matching or the like. That is, when the real-space image includes an object that matches the registered identification object 50, the authentication processing unit 42 determines that the registered identification object 50 is being displayed on the display 16, and causes the authentication object 24 to be displayed on the display 16.

[0097] Fig. 10 is a flowchart showing the flow of user authentication processing according to this embodiment. Note that the same steps in Fig. 10 as those in Fig. 6 are given the same reference numerals as those in Fig. 6, and their description will be omitted in part or in whole.

[0098] First, in step 200, the authentication processing unit 42 determines whether or not the registered identification object 50 is displayed on the display 16, in other words, whether or not the registered identification object has been imaged by the camera 56, and if the determination is affirmative, the process proceeds to step 202. Note that if the HMD 12 is a see-through type and the determination is affirmative in step 200, the image of the actually existing registered identification object 50 has been captured in a predetermined region (e.g., the central region) of the imaging range of the camera 56, and the user is directly viewing the registered identification object 50 that has passed through the display 16. In other words, if the HMD 12 is a see-through type, the image captured by the camera 56 does not need to be displayed on the display 16.

[0099] In step 202, the authentication processing unit 42 displays the authentication object 24 on the display 16, and the process proceeds to step 100.

[0100] In steps 100 to 108, the same processing as the user authentication processing according to the first embodiment is performed. If the user authentication fails, the processing returns to step 200, and the user authentication processing is performed again.

[0101] As described above, the HMD 12 of this embodiment realizes AR by displaying objects together with real-space images on the display 16, and when a registered identification object 50 is displayed on the display 16, an authentication object 24 is displayed on the display 16.

[0102] This allows the HMD 12 according to this embodiment to perform user authentication with higher security.

[0103] It should be noted that a plurality of registered identification devices 50 may be registered in advance, and the registered identification device 50 used for user authentication may be changed for each user authentication or every predetermined number of days, etc. When the registered identification device 50 used for user authentication changes, information regarding the changed registered identification device 50 is notified to the user's pre-registered email address, etc.

[0104] Furthermore, the registered identification object 50 is not limited to an object such as the monitor 54, but may be, for example, an identification code such as a barcode or QR code (registered trademark), a predetermined image, etc. These identification codes and images may be printed on paper or the like, or may be displayed on a computer monitor, a mobile information terminal, etc.

[0105] Furthermore, the registered identification object 50 may not be an actual object, but may be a virtual object displayed on the display 16. In this case, the user moves to the registered identification object 50 in the VR space and displays it on the display 16, or holds the registered identification object 50 in the VR space and displays it on the display 16.

[0106] Furthermore, the real or virtual registered identification object 50 itself may be the authentication object 24. In this case, when the registered identification object 50 is displayed in the authentication display area 22A, user authentication is performed. Furthermore, when the HMD 12 is a see-through type, user authentication may be performed when the real registered identification object 50 is captured in a predetermined area (for example, the central area) of the imaging range of the camera 56. good.

[0107] 4. Other Embodiments Although the present invention has been described above using the above-mentioned embodiments, the technical scope of the present invention is not limited to the scope described in the above-mentioned embodiments. Various modifications and improvements can be made to the above-mentioned embodiments without departing from the gist of the invention, and modifications and improvements are also included in the technical scope of the present invention. Furthermore, the above-mentioned embodiments may be combined as appropriate.

[0108] For example, in each of the above embodiments, the HMD 12 is described as having the authentication processing unit 42, but the present invention is not limited to this, and the server 14 may be configured to have the authentication processing unit 42. In this configuration, the detection result detected by the HMD 12 for user authentication is transmitted to the server 14, and the server 14 performs user authentication based on the received detection result and transmits the authentication result to the HMD 12.

[0109] Furthermore, in the above-described embodiments, the authentication according to the present invention has been described as being applied to user authentication performed when the HMD 12 is started up, but the present invention is not limited to this and may be applied to, for example, user authentication performed when software executed on the HMD 12 is started up. The present invention may also be applied to authentication within a game executed on the HMD 12. In a form applied to game authentication, the HMD 12 may perform authentication when it detects that the user's line of sight has been aligned with the authentication object 24, and a predetermined event may occur if the authentication is successful.

[0110] Furthermore, in the first and second embodiments, the HMD 12 displays VR, but the present invention is not limited to this, and the HMD 12 may display AR.

[0111] Furthermore, the flow of the user authentication process described in each of the above embodiments is also an example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged within the scope of the present invention. [Explanation of symbols]

[0112] 12 HMD (Head Mounted Display Device) 16 Display 22 Image display area 22A Authentication display area 24 Authentication Objects 30 Face direction detection unit (face direction detection means) 32 Line of sight detection unit (line of sight detection means) 40 Image display control unit (image display control means) 42 Authentication processing unit (authentication means)

Claims

1. A head-mounted display device in which a display is mounted on a user's head so as to be visible to the user, a face direction detection means for detecting a face direction of the user; a gaze detection means for detecting the gaze of the user; an image display control means for changing an object to be displayed in an image display area of ​​the display in accordance with a detection result by the face direction detection means; an authentication means for displaying an authentication object used for authentication in the image display area in response to a movement of the user, and for performing authentication when the line-of-sight detection means detects that the user has focused their line of sight on the authentication object; A head-mounted display device comprising:

2. 2. The head mounted display device according to claim 1, wherein the authentication object is not displayed in the image display area when authentication starts, but is displayed in the image display area in response to a movement of the user.

3. an authentication display area is set in the image display area; The head-mounted display device according to claim 1 or 2, wherein the authentication means performs authentication when the authentication object is displayed in the authentication display area in response to the user's actions and the gaze detection means detects that the user has focused their gaze on the authentication object.

4. 4. The head mounted display device according to claim 1, wherein the user's action is the user changing the direction of his / her face or the user moving the authentication object.

5. 5. The head mounted display device according to claim 1, wherein the authentication means performs authentication based on the detection results of the line of sight detection means and the face direction detection means.

6. 6. The head mounted display device according to claim 1, wherein when a pre-registered identifier is displayed on the display, the authentication object is displayed on the display.

7. It has a camera on the front, 7. The head mounted display device according to claim 1, wherein when the camera captures an image of a pre-registered real-world identifying object, the authentication object is displayed on the display.

8. 6. The head mounted display device according to claim 1, wherein the authentication object is a pre-registered identifier.

9. A head-mounted display device as described in any one of claims 1 to 8, wherein the authentication means determines that the user has started the authentication operation when it detects that the user has directed their gaze at the authentication object for more than a predetermined time, and that the user has ended the authentication operation when it detects that the user has looked away from the authentication object for more than a predetermined time.

10. 9. The head mounted display device according to claim 1, wherein the authentication means starts or ends an authentication operation when a predetermined number of blinks of the user or more are detected within a predetermined time period.

11. 9. The head mounted display device according to claim 1, wherein the authentication means starts or ends an authentication operation when it is detected that the user has made a predetermined head movement.

12. An authentication method for a head-mounted display device, in which a display is mounted on a user's head so as to be visible to the user, and which is equipped with a face direction detection means for detecting a face direction of the user and a line of sight detection means for detecting a line of sight of the user, a first step of changing an object to be displayed in an image display area of ​​the display in accordance with a detection result by the face direction detection means; a second step of displaying an authentication object used for authentication in the image display area in response to a movement of the user, and performing authentication when the line-of-sight detection means detects that the user has focused their line of sight on the authentication object; An authentication method having the following:

13. A head-mounted display device is provided with a computer, the computer being mounted on the head of a user so that the display can be viewed by the user, the computer being equipped with a face direction detection means for detecting the direction of the user's face and a line of sight detection means for detecting the line of sight of the user, an image display control means for changing an object to be displayed in an image display area of ​​the display in accordance with a detection result by the face direction detection means; an authentication means for displaying an authentication object used for authentication in the image display area in response to a movement of the user, and for performing authentication when the line-of-sight detection means detects that the user has focused their line of sight on the authentication object; Certification program to ensure it functions as a

Images