Information processing apparatus, method for controlling information processing apparatus, and program

Abstract

To provide an information processing apparatus that more easily determines that a user wearing a HMD has changed.SOLUTION: An information processing apparatus determines whether a user wearing a head-mounted display having display means has changed, and has: image acquisition means that acquires a first picked-up image of the eyes of a first user at a first time point, and a second picked-up image of the eyes of a second user at a second time point later than the first time point; detection means that detects the positions of the organs in the eyes of the first user on the basis of the first image, and detects the positions of the organs in the eyes of the second user on the basis of the second image; and determination means that determines whether the first user and the second user are different persons on the basis of the positions detected based on the first image and the positions detected based on the second image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a control method for an information processing device, and a program. [Background technology]

[0002] There is an IPD adjustment technology that adjusts the position of the display mounted on an HMD (head-mounted display) to match the IPD (inter-pupilar distance) of an authenticated individual.

[0003] Authentication technologies include iris authentication and face authentication. Patent Document 1 discloses the basic algorithm technology for iris authentication. If an HMD is worn by a user different from the user who wore the HMD immediately before, the authentication technology makes it possible to control the position of the display to the position where the image on the display looks the most clear (the so-called sweet spot) according to the IPD of the authenticated individual. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 5,291,560 Summary of the Invention [Problem to be solved by the invention]

[0005] However, iris recognition has various high requirements, such as "high image resolution required for imaging" and "the position where the image of the eye needs to be as directly facing the eye as possible." To meet these requirements, a separate imaging device dedicated to iris recognition is required. Therefore, implementing such a function would be expensive and would increase the size of the HMD. Furthermore, facial recognition requires capturing an image of the entire face to authenticate an individual, but it is difficult to capture an image of the entire face with systems such as HMDs, making it unrealistic to implement.

[0006] Therefore, an object of the present invention is to provide a technique that makes it possible to more easily determine whether the user wearing the HMD has been replaced. [Means for solving the problem]

[0007] One aspect of the present invention is a method for producing a medicament for the treatment of a pulmonary arthritis. An information processing device that determines whether a user wearing a head-mounted display having a display means has changed, an image acquisition means for acquiring a first image of a first user's eye at a first time point and a second image of a second user's eye at a second time point after the first time point; a detection means for detecting a position of an organ in the eye of the first user based on the first image and detecting a position of an organ in the eye of the second user based on the second image; a determination means for determining whether the first user and the second user are different persons based on the position detected based on the first image and the position detected based on the second image; The information processing device is characterized by having:

[0008] One aspect of the present invention is a method for producing a medicament for the treatment of a pulmonary arthritis. A control method for an information processing device that determines whether a user wearing a head-mounted display having a display means has changed, comprising: a first image acquisition step of acquiring a first image of an eye of a first user at a first time point; a first detection step of detecting a position of an organ in the eye of the first user based on the first image; a second image acquisition step of acquiring a second image of an eye of a second user at a second time point that is later than the first time point; a second detection step of detecting a position of an organ in the eye of the second user based on the second image; a determination step of determining whether the first user and the second user are different people based on the position detected based on the first image and the position detected based on the second image; The control method is characterized by having the following. [Effects of the Invention]

[0009] According to the present invention, it is possible to more easily determine whether the user wearing the HMD has been replaced. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram of an HMD system according to an embodiment. [Figure 2] 1 is a configuration diagram of an HMD system according to a first embodiment. [Figure 3] FIG. 2 is a diagram illustrating detection of an eye organ according to the first embodiment. [Figure 4] FIG. 2 is a diagram illustrating a difference in the position of an eye organ according to the first embodiment. [Figure 5] FIG. 1 is a diagram illustrating personal identification according to the first embodiment. [Figure 6] 1A and 1B are diagrams illustrating an image of an eye captured from the front or the like according to the first embodiment. [Figure 7] 4 is a flowchart of processing of the HMD system according to the first embodiment. [Figure 8] FIG. 10 is a configuration diagram of an HMD system according to a second embodiment. [Figure 9] 10 is a flowchart of processing of the HMD system according to the second embodiment. [Figure 10] FIG. 10 is a configuration diagram of an HMD system according to a third embodiment. [Figure 11] 11 is a flowchart of processing of an HMD system according to a third embodiment. [Figure 12] FIG. 10 is a configuration diagram of an HMD system according to a fourth embodiment. [Figure 13] 10 is a flowchart of processing of an HMD system according to a fourth embodiment. [Figure 14]FIG. 10 is a configuration diagram of an HMD system according to a fifth embodiment. [Figure 15] 13 is a flowchart of processing of an HMD system according to a fifth embodiment. [Figure 16] FIG. 13 is a configuration diagram of an HMD system according to a sixth embodiment. [Figure 17] 13 is a flowchart of processing of an HMD system according to a sixth embodiment. [Figure 18] FIG. 13 is a configuration diagram of an HMD system according to a seventh embodiment. [Figure 19] 13 is a flowchart of the processing of the HMD system according to the seventh embodiment. [Figure 20] 13 is a diagram illustrating a frequency distribution of the state of the HMD according to the seventh embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, each embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0012] First, matters common to each embodiment will be described. Fig. 1 is a block diagram showing the configuration of an HMD system according to each embodiment. The HMD system (head-mounted system) mainly displays an image in which an image of real space is superimposed on virtual computer graphics. Each embodiment can be applied to various HMD systems including HMDs.

[0013] The HMD system of FIG. 1 includes an HMD 10 and an information processing device 20. The HMD 10 is a display device that can be worn on the head of a user. The information processing device 20 may include part of the configuration of the HMD 10, or the HMD 10 may include all or part of the configuration of the information processing device 20. That's fine.

[0014] (About the HMD configuration) The HMD 10 includes a lens unit 101, an imaging unit 102, an imaging signal processing unit 103, an image signal processing unit 104, an image synthesis unit 105, a display unit 106, an eyepiece optical system 107, an imaging control unit 108, and an HMD control unit 109. The HMD 10 also includes an eye camera control unit 110, an IPD adjustment control unit 111, an eye camera unit 112, an eye detection unit 113, a power supply unit 114, an HMD operation unit 115, a shake detection unit 116, a self-position estimation unit 117, and an HMD communication control unit 118. The HMD 10 also includes a microphone unit 119, a microphone control unit 120, an eye organ detector 121, a position comparison unit 122, and a personal identification unit 123.

[0015] The imaging unit 102 captures an image of the real space (captured image) via the lens unit 101.

[0016] The imaging signal processing unit 103 performs image processing on the captured image output from the imaging unit 102 .

[0017] The image signal processing unit 104 performs further image processing on the captured image that has been subjected to image processing by the image capture signal processing unit 103 .

[0018] The image synthesis unit 105 synthesizes the computer graphics (CG) generated by the CG generation unit 125 with the captured image that has been subjected to image processing, thereby generating a synthesized image.

[0019] The composite image is displayed on the display unit 106. The composite image displayed on the display unit 106 is focused on the retina of the eye through the eyepiece optical system 107.

[0020] The imaging control unit 108 controls exposure in the imaging unit 102 .

[0021] The HMD control unit 109 includes a CPU. The HMD control unit 109 controls the entire HMD system. The HMD control unit 109 communicates with a CG communication control unit 131 via an HMD communication control unit 118.

[0022] The eye camera control unit 110 controls the eye camera unit 112 .

[0023] The IPD adjustment control unit 111 controls the positions of the components of the HMD so that an image transmitted from the display unit 106 through the eyepiece optical system 107 is correctly formed on the retina of the eye. In this case, the IPD adjustment control unit 111 moves the positions of the eye camera unit 112, the display unit 106, and the eyepiece optical system 107. Hereinafter, such a movement of positions will be simply referred to as "IPD control."

[0024] Furthermore, the IPD adjustment control unit 111 determines whether the gaze direction detected by the gaze detection unit 113 is perpendicular to the display surface of the display unit 106, based on the image of the eye captured by the gaze camera unit 112. If the IPD adjustment control unit 111 determines that the gaze direction is not perpendicular to the display surface, it moves the display unit 106 and the gaze camera unit 112 so that the gaze direction and the display surface are perpendicular to each other. This enables IPD adjustment.

[0025] The eye camera unit 112 captures images of the eye area.

[0026] The gaze detection unit 113 estimates the gaze direction of the user based on the image of the eye area captured by the gaze camera unit 112.

[0027] The power supply unit 114 supplies power (electricity) to the entire system depending on the application.

[0028] The HMD operation unit 115 is an operation unit that the user uses to operate the HMD system. The HMD operation unit 115 outputs an operation signal corresponding to the operation to the HMD control unit 109.

[0029] The shake detection unit 116 detects the amount of shake applied to the HMD 10. The shake detection unit 116 outputs a detection signal indicating the amount of shake to the self-position estimation unit 117 together with the captured image processed by the image signal processing unit 104.

[0030] The self-position estimation unit 117 estimates the self-position (position) of the HMD 10 based on the detected amount of shaking and the captured image.

[0031] The HMD communication control unit 118 communicates with a CG communication control unit 131 of the information processing device 20 .

[0032] The microphone unit 119 converts information about sounds around the HMD 10 into electrical signals.

[0033] The microphone control unit 120 converts the electrical signal into audio stream data. The audio stream data is sent to the CG generation unit 125 together with the captured image processed by the image signal processing unit 104, and is stored in the storage unit 124 as image data.

[0034] The eye organ detector 121 detects the positions of the eye organs (corner, inner corner, upper and lower ends of the eye, inner and outer edges of the eyebrows, etc.) based on the image of the eye area captured by the eye camera unit 112.

[0035] The position comparison unit 122 compares the positions of the eye organs detected at two points in time.

[0036] The individual identification unit 123 simply identifies (recognizes) the user (individual) wearing the HMD 10 based on the comparison result by the position comparison unit 122. Hereinafter, if it is determined that a user different from the user wearing the HMD 10 immediately before is wearing the HMD 10, the IPD adjustment control unit 111 performs IPD adjustment again.

[0037] (Configuration of information processing device) The information processing device 20 has a storage unit 124 , a CG generation unit 125 , a menu configuration switching unit 126 , a guardian generation unit 127 , an operability parameter switching unit 128 , a volume parameter switching unit 129 , a brightness switching unit 130 , and a CG communication control unit 131 .

[0038] The storage unit 124 stores various data such as image data.

[0039] The CG generation unit 125 generates computer graphics (CG). The CG generation unit 125 can also generate an image (hereinafter referred to as a "gazing image") for the user to look at a specific position. The gaze image is displayed on the display unit 106.

[0040] Based on the determination result of identifying the user, the menu configuration switching unit 126 switches the current menu to a menu (= the menu displayed on the display unit 106) configured to correspond to the identified user. Specifically, the menu configuration switching unit 126 performs control such as changing the display order of the components of the menu to suit the user and making it easier to access frequently used components.

[0041] Based on the result of the determination of the user, the guardian generation unit 127 switches the current setting to the setting of the guardian area corresponding to the identified user. The guardian area indicates an area in real space that the user can safely move in. For example, if the user enters an area outside the guardian area, the HMD 10 issues a warning to the user.

[0042] Based on the determination result of identifying the user, the operability parameter switching unit 128 switches the current parameters to operability parameters (parameters related to operation) corresponding to the identified user. The operability parameters indicate, for example, the correspondence relationship between the amount of operation by the user and the amount of movement of the control target.

[0043] Based on the result of the determination of the user, the volume parameter switching unit 129 switches the current output volume to the volume corresponding to the identified user.

[0044] Based on the determination result of identifying the user, the brightness switching unit 130 switches the current brightness of the display unit 106 (display) to a brightness corresponding to the identified user.

[0045] (Outline of HMD system operation) Next, the general operation of the HMD system will be described.

[0046] The HMD operation unit 115 accepts an operation to display a menu. Then, the user selects the IPD adjustment menu. When the IPD adjustment menu is selected, the CG generation unit 125 generates an image (hereinafter referred to as a "gaze image") for the user to look at a specific position. The display unit 106 displays the gaze image. In the following, the specific position is assumed to be the center position of the display surface of the display unit 106.

[0047] While the user is looking at a specific position, the gaze detection unit 113 detects the user's gaze direction based on the image of the user's eyes captured by the gaze camera unit 112. The IPD adjustment control unit 111 moves the gaze camera unit 112 and the display unit 106 so that the detected gaze direction is perpendicular to the display surface of the display unit 106. After this movement, when the user looks at the display unit 106, the image formation position correctly overlaps with the user's retina. This allows the user to see a clear image.

[0048] <Embodiment 1> In the first embodiment, the HMD system detects the positions of the eye organs (corner, inner corner, upper and lower corners of the eye, inner and outer corners of the eyebrows, etc.) using an eye organ detector. Then, the HMD system performs simple personal identification based on the difference in the positions of the eye organs, and if it determines that the user wearing the HMD is a different user from the user at the immediately preceding (previous) point in time, it performs IPD adjustment.

[0049] 2 shows a configuration diagram of the HMD system according to embodiment 1. The HMD system includes an optical system 201, an image sensor 202, an image processing unit 203, a synthesis unit 204, a display unit 206, a camera control unit 207, a position estimation unit 208, a CG generation unit 209, an eye organ detector 210, a gaze image sensor 211, and a gaze image generation unit 212. The HMD system also includes a position comparison unit 213, a personal identification unit 214, an IPD control unit 215, and an IPD adjustment unit 216.

[0050] The image sensor 202 is an image acquisition unit that acquires a captured image of the real space via the optical system 201. The image sensor 202 corresponds to the imaging unit .

[0051] The image processing unit 203 performs image processing on the captured image. The image processing unit 203 corresponds to the image capture signal processing unit 103 and the image signal processing unit 104.

[0052] The synthesis unit 204 synthesizes the captured image that has been subjected to image processing and the CG generated by the CG generation unit 209. The synthesis unit 204 synthesizes the image data with the computer graphics. In this way, the synthesis unit 204 generates a synthesized image. The synthesis unit 204 corresponds to the image synthesis unit 105.

[0053] The composite image is displayed on the display unit 206. The display unit 206 corresponds to the display unit .

[0054] The camera control unit 207 controls the imaging sensor 202. The camera control unit 207 corresponds to the imaging control unit .

[0055] The position estimation unit 208 estimates the position of the HMD. The position estimation unit 208 corresponds to the self-position estimation unit 117.

[0056] The CG generation unit 209 generates computer graphics based on the estimated position of the HMD. The CG generation unit 209 corresponds to the CG generation unit 125.

[0057] The eye organ detector 210 detects the positions of the eye organs (the outer corner of the eye, the inner corner of the eye, the upper and lower edges of the eye, the inner and outer corners of the eyebrows, etc.) based on the image of the eye captured by the line-of-sight imaging sensor 211. The eye organ detector 210 corresponds to the eye organ detector 121.

[0058] The gaze imaging sensor 211 captures an image of the eye area by capturing an image of the eye, and outputs the image of the eye area to the eye organ detector 210 and the IPD adjustment unit 216. The gaze imaging sensor 211 corresponds to the gaze camera unit 112.

[0059] The gaze image generating unit 212 generates a gaze image for the user to look at the center position of the display surface of the display unit 206. The gaze image is displayed on the display unit 206.

[0060] The position comparison unit 213 compares the “position of the eye organ stored immediately before” with the “position of the newly detected eye organ.” The position comparison unit 213 corresponds to the position comparison unit 122.

[0061] The individual identification unit 214 is a determination unit that determines that different users are wearing the HMDs at the two points in time when the positions of the eye organs at the two points in time have a specific relationship. The individual identification unit 214 corresponds to the individual identification unit 123.

[0062] The IPD control unit 215 drives and moves the line-of-sight image sensor 211 and the display unit 206 so that the user's line of sight is perpendicular to the display surface of the display unit 206. As a result, the image displayed by the display unit 206 is correctly focused on the retina, allowing the user to see a clear image. The IPD control unit 215 corresponds to the IPD adjustment control unit 111.

[0063] The IPD adjustment unit 216 detects the direction of the user's gaze based on an image of the user's eyes captured by the gaze image sensor 211. The IPD adjustment unit 216 corresponds to the gaze detection unit 113.

[0064] 3 shows images of the eye regions of user A and user B captured by the gaze imaging sensor 211 at a position diagonally downward from the eye position (cross-eyed position). When an eye organ detector is applied to the image shown in FIG. 3, the position 302 of the inner corner of the eye and the position 303 of the outer corner of the eye can be detected. Information about the line 301 connecting the detected position 302 of the inner corner of the eye and the position 303 of the outer corner of the eye is also obtained. Here, information about the position 302 of the inner corner of the eye, the position 303 of the outer corner of the eye, and the line 301 is stored at the timing of IPD adjustment.

[0065] Next, an eye organ detector detects the position 305 of the inner corner of the eye and the position 306 of the outer corner of the eye from the image of the eyes of user B. After that, information on a line 304 connecting the detected position 305 of the inner corner of the eye and the position 306 of the outer corner of the eye is obtained.

[0066] This makes it possible to compare the "distance between the inner corner position 302 and the outer corner position 303 of the eye" at the timing of the most recent IPD adjustment with the newly detected "distance between the inner corner position 305 and the outer corner position 306 of the eye." If the difference between these two distances is equal to or greater than a predetermined value, it is determined that the image of user A's eye and the image of user B's eye are images of different users (people). Furthermore, by comparing line 301 and line 304, if the difference in length is equal to or greater than a first threshold and the difference in slope is equal to or greater than a second threshold, it may be determined that the image of user A's eye and the image of user B's eye are images of different users.

[0067] 4A and 4B are diagrams illustrating a method for calculating the difference between the positions of the left and right eyes. Fig. 4A shows the relationship between the display and the eyes before IPD adjustment. Fig. 4B shows the relationship between the display and the eyes after IPD adjustment.

[0068] To calculate the difference in position between the left and right eyes, it is necessary to know the difference in position between eye camera 403 that images the left eye and eye camera 405 that images the right eye. Eye camera 403 and eye camera 405 are included in eye imaging sensor 211. Display 401 seen by the left eye and display 402 seen by the right eye are included in display unit 206.

[0069] Eye camera 403 is fixed to display 401 and moves with display 401. Eye camera 405 is fixed to display 402 and moves with display 402.

[0070] At this time, the distance 404 between the eye camera 403 and the eye camera 405 is determined by the positions of the left structure (display 401 and eye camera 403) and the right structure (display 402 and eye camera 405).

[0071] In addition, a motor can be driven to move the left structure (display 401 and gaze camera 403). An encoder and decoder are provided to identify the position of the left structure. Therefore, the position comparator 213 can grasp the position of the left structure.

[0072] Similarly, a motor can be driven to move the right structure (display 402 and gaze camera 405). An encoder and decoder are provided to identify the position of the right structure. Therefore, the position comparator 213 can grasp the position of the right structure.

[0073] In this way, since the positions of the left and right structures can be grasped, the position comparison unit 213 can calculate the distance 404 between the eye camera 403 and the eye camera 405 .

[0074] Therefore, the eye detector 210 detects the positions of the left and right eye organs based on the image of the eyes captured with the positions of the left and right eye cameras known, and the position comparator 213 can calculate the distance between the left and right eyes based on the positions of the left and right eye organs.

[0075] A method for determining that user A and user B are different users (people) based on the results of measuring the distance between the left and right eyes will be described with reference to FIGS. 5A and 5B.

[0076] 5A, eye organ detector 210 detects left inner corner 503 and outer corner 502 of the eye based on an image of user A's eyes captured by gaze camera 403. Also, eye organ detector 210 detects right inner corner 504 and outer corner 505 of the eye based on an image of user A's eyes captured by gaze camera 405. As a result, distance 501 between the left and right outer corners of the eye and distance 506 between the left and right inner corners of the eye are obtained.

[0077] 5B, left inner corner 509 and outer corner 508 of the eye are detected based on an image of user B's eyes captured by gaze camera 403. Right inner corner 510 and outer corner 511 of the eye are detected based on an image of user B's eyes captured by gaze camera 405. Then, distance 507 between the left and right outer corners of the eyes and distance 512 between the left and right inner corners of the eyes are obtained.

[0078] Therefore, for example, the individual identification unit 214 compares the distance 501 with the distance 507, and if the difference between the two distances is less than a predetermined value, determines that the user A and the user B are the same user (person). Also, the individual identification unit 214 compares the distance 501 with the distance 507, and if the difference between the two distances is equal to or greater than a predetermined value, determines that the user A and the user B are different users.

[0079] Furthermore, the individual identification unit 214 may compare the distance 506 with the distance 512, and if the difference between the two distances is less than a predetermined value, determine that the user A and the user B are the same user (person). Furthermore, the individual identification unit 214 may compare the distance 506 with the distance 512, and if the difference between the two distances is equal to or greater than a predetermined value, determine that the user A and the user B are different users.

[0080] Figures 6A and 6B are diagrams explaining that it is possible to detect eye organs (inner corner of the eye, outer corner of the eye, inner corner of the eyebrow, outer corner of the eyebrow, upper edge of the eye, and lower edge of the eye) even when using an image of the eye captured from the front or below the eye.

[0081] The eye organ detector 210 detects the inner corner 603 and outer corner 602 of the left eye based on an image of the eye captured from a position in front of or below the left eye of user C. As shown in FIG. 6A, the eye organ detector 210 can also calculate a line 601 connecting the detected inner corner 603 and the detected outer corner 602 of the eye, allowing for processing equivalent to that when using an eye-gaze camera in a cross-eyed position. Furthermore, as shown in FIG. 6B, the eye organ detector 210 can also calculate a line 604 connecting the inner corner 606 and the outer corner 605 of the eyebrows, and line 604 can also be used as an index for identifying an individual. As shown in FIG. 6B, the eye organ detector 210 can also calculate a line 609 connecting the upper end 607 of the eye and the lower end 608 of the eye, and line 609 can also be used as an index for identifying an individual.

[0082] The processing of the HMD system according to the first embodiment will be described with reference to the flowchart of FIG.

[0083] In step S702, the IPD control unit 215 starts IPD adjustment. A gaze image is displayed on the display unit 206, and control of the positions of the display unit 206 and the gaze image sensor 211 (IPD control) is performed based on the gaze direction of the user viewing the gaze image.

[0084] In step S703, the gaze image sensor 211 captures images of the eyes using the gaze cameras 403 and 405.

[0085] In step S704, the eye organ detector 210 detects the positions of the eye organs (each organ in the eye) based on the image of the eye.

[0086] In step S705, the position comparison unit 213 calculates the distance DA between the outer corner and inner corner of the left eye (or right eye) based on the result of detecting the position of the eye organ in step S704.

[0087] In step S706, the position comparison unit 213 calculates the distance DB between the corners of the left and right eyes based on the result of detecting the positions of the eye organs in step S704.

[0088] In step S707, the position comparison unit 213 calculates the distance DC between the inner corners of the left and right eyes based on the result of detecting the positions of the eye organs in step S704.

[0089] In step S708, the position comparison unit 213 stores information on the distances DA, DB, and DC in a memory or the like.

[0090] In step S709, the position comparison unit 213 uses a self-position estimation technique such as SLAM to determine whether the HMD has moved more than a predetermined amount since the processing of step S703 was performed. If it is determined that the HMD has moved more than the predetermined amount, the process proceeds to step S710. If it is determined that the HMD has not moved more than the predetermined amount, the process of step S709 is repeated.

[0091] In step S710, the eye gaze imaging sensor 211 captures images of the eyes using the eye gaze cameras 403 and 405.

[0092] In step S711, the eye organ detector 210 detects the position of the eye organ based on the image of the eye.

[0093] In step S712, the position comparison unit 213 calculates the distance DA' between the outer corner and inner corner of the left eye (or right eye) based on the result of detecting the position of the eye organ in step S711.

[0094] In step S713, the position comparison unit 213 calculates the distance DB' between the corners of the left and right eyes based on the result of detecting the positions of the eye organs in step S711.

[0095] In step S714, the position comparison unit 213 calculates the distance DC' between the inner corners of the left and right eyes based on the result of detecting the positions of the eye organs in step S711.

[0096] In step S715, the position comparison unit 213 compares "distance DA, distance DB, distance DC" with "distance DA', distance DB', distance DC'." Hereinafter, the difference between distance DA and distance DA' will be referred to as "difference IA." The difference between distance DB and distance DB' will be referred to as "difference IB." The difference between distance DC and distance DC' will be referred to as "difference IC."

[0097] In step S716, the personal identification unit 214 determines whether different users were wearing the HMD at the time of imaging in step S703 (the previous time point) and the time of imaging in step S710 (the current time point) based on the comparison result by the position comparison unit 213. For example, if the difference IA is equal to or greater than the threshold THA, the difference IB is equal to or greater than the threshold THB, and the difference IC is equal to or greater than the threshold THC, the personal identification unit 214 determines that different users were wearing the HMD at the two time points. Otherwise, the personal identification unit 214 determines that the same user was wearing the HMD at the two time points. Furthermore, the personal identification unit 214 may determine that different users were wearing the HMD at the two time points if at least one of the following conditions is satisfied: the difference IA is equal to or greater than the threshold THA, the difference IB is equal to or greater than the threshold THB, or the difference IC is equal to or greater than the threshold THC.

[0098] Note that instead of distance DA and distance DA', which are the distances between the outer corner and inner corner of the eye, the gradient of a line connecting the positions of the outer corner and inner corner of the eye may be used. Instead of distance DB and distance DB', which are the distances between the outer corners of the left and right eyes, the gradient of a line connecting the positions of the outer corners of the left and right eyes may be used. Instead of distance DC and distance DC', which are the distances between the inner corners of the left and right eyes, the gradient of a line connecting the positions of the inner corners of the left and right eyes may be used. In other words, if the difference between the positional relationship of the multiple eye organs at the time of step S703 and the positional relationship of the multiple eye organs at the time of step S710 is greater than a certain value, the individual identification unit 214 determines that different users at the two times used the HMD. It may be determined that the person was wearing

[0099] If it is determined in step S716 that different users were wearing the HMD at the two points in time, the process proceeds to step S702, where the IPD adjustment is performed again. On the other hand, if it is determined in step S716 that the same user was wearing the HMD at the two points in time, the process proceeds to step S717.

[0100] In step S717, the IPD adjustment unit 216 determines whether the power has been turned off. If it is determined that the power has been turned off, the process of this flowchart ends. If it is determined that the power has not been turned off, the process proceeds to step S709.

[0101] According to the first embodiment, even when a low-resolution eye camera is used, the HMD system can determine whether the user wearing the HMD has changed by comparing the positions of the eye organs detected based on the image acquired by the eye camera. Therefore, the HMD system can more simply (easily) determine whether the user wearing the HMD has changed, using an HMD and an information processing device with a simple configuration. Then, the HMD system can start IPD adjustment at a more appropriate timing.

[0102] <Embodiment 2> In the second embodiment, when the HMD system determines that a user different from the user immediately before is wearing the HMD, it automatically switches the current setting to the guardian setting corresponding to the user wearing the HMD.

[0103] The configuration of the HMD system according to the second embodiment will be described with reference to Fig. 8. The HMD system according to the second embodiment has a guardian setting switching unit 812 in addition to the configuration of the HMD system according to the first embodiment.

[0104] If a guardian setting has already been registered for the user wearing the HMD, the guardian setting switching unit 812 switches the guardian setting currently being used in the HMD to the registered guardian setting. Switching of the guardian setting can be achieved by reading out the guardian setting associated with an individual stored in the guardian setting switching unit 812 and switching to the setting associated with the identified individual. If no guardian setting has been registered for the user wearing the HMD, the guardian setting switching unit 812 registers a new guardian setting for the user and switches to the registered guardian setting.

[0105] The processing according to the second embodiment will be described with reference to the flowchart of Fig. 9. Note that steps with the same names in the flowchart of Fig. 9 and the flowchart of Fig. 7 perform similar processing in the two flowcharts, and therefore descriptions thereof will be omitted.

[0106] In step S901, the guardian setting switching unit 812 switches the guardian setting currently being used in the HMD to the guardian setting corresponding to the user wearing the HMD.

[0107] In step S916, the individual identification unit 214 determines whether different users are wearing the HMD at the time of imaging in step S703 (the previous time point) and the time of imaging in step S710 (the current time point) based on the comparison result by the position comparison unit 213. If it is determined that different users are wearing the HMD at the two time points, the process proceeds to step S901, and the guardian setting is switched again. On the other hand, if it is determined that the same user is wearing the HMD at the two time points, the guardian setting is not switched. Then, the process proceeds to step S717.

[0108] According to the second embodiment, when the user wearing the HMD is changed, the guardian setting can be appropriately switched to a setting that corresponds to the user wearing the HMD.

[0109] <Embodiment 3> In the third embodiment, when the HMD system determines that a user different from the previous user is wearing the HMD, it automatically switches the current settings to menu settings corresponding to the user wearing the HMD. Therefore, as shown in Fig. 10, the HMD system according to the third embodiment has a menu configuration switching unit 1012 instead of the guardian setting switching unit 812 of the HMD system according to the second embodiment.

[0110] If menu configuration settings have already been registered for the user wearing the HMD, the menu configuration switching unit 1012 switches the menu configuration settings currently used in the HMD to the registered settings. If menu configuration settings have not been registered for the user wearing the HMD, the menu configuration switching unit 1012 newly registers menu configuration settings for the user and switches to the registered menu configuration settings.

[0111] 9, the process of step S1101 in the flowchart of Fig. 11 may be performed. In step S1101, the menu configuration switching unit 1012 switches the menu configuration settings currently used in the HMD to the menu configuration settings corresponding to the user wearing the HMD. Specifically, the menu configuration switching unit 1012 customizes the configuration and settings of the menu displayed on the display unit 206.

[0112] According to the third embodiment, when the user wearing the HMD is changed, it is possible to appropriately switch to the settings of the menu configuration corresponding to the user wearing the HMD.

[0113] <Embodiment 4> In the fourth embodiment, when the HMD system determines that a user different from the previous user is wearing the HMD, it automatically switches the current operability parameters to operability parameters (parameters related to operation) corresponding to the user wearing the HMD. Therefore, as shown in Fig. 12, the HMD system according to the fourth embodiment has an operability parameter switching unit 1212 instead of the guardian setting switching unit 812 of the HMD system according to the second embodiment.

[0114] If operability parameters have been registered for the user wearing the HMD, the operability parameter switching unit 1212 switches the operability parameters currently used in the HMD to the registered operability parameters. If operability parameters have not been registered for the user wearing the HMD, the operability parameter switching unit 1212 newly registers operability parameters for the user and switches to the registered operability parameters.

[0115] Furthermore, instead of step S901 in the flowchart of Fig. 9, the processing of step S1301 in the flowchart of Fig. 13 may be performed. In step S1301, the operability parameter switching unit 1212 switches the operability parameters currently being used in the HMD to operability parameters corresponding to the user wearing the HMD.

[0116] According to the fourth embodiment, when the user wearing the HMD is changed, the operability parameters can be appropriately switched to those corresponding to the user wearing the HMD.

[0117] <Embodiment 5> In the fifth embodiment, when the HMD system determines that a user different from the previous user is wearing the HMD, it automatically switches the current volume parameters to volume parameters (parameters related to output volume) corresponding to the user wearing the HMD. Therefore, as shown in Fig. 14, the HMD system according to the fifth embodiment has a volume parameter switching unit 1412 instead of the guardian setting switching unit 812 of the HMD system according to the second embodiment.

[0118] If volume parameters have been registered for the user wearing the HMD, the volume parameter switching unit 1412 switches the volume parameters currently used in the HMD to the registered volume parameters. If volume parameters have not been registered for the user wearing the HMD, the volume parameter switching unit 1412 newly registers volume parameters for the user and switches to the registered volume parameters.

[0119] 9, the process of step S1301 in the flowchart of Fig. 15 may be performed. In step S1501, the volume parameter switching unit 1412 switches the volume parameter (output volume) currently used in the HMD to the volume parameter (output volume) corresponding to the user wearing the HMD.

[0120] According to the fifth embodiment, when the user wearing the HMD is changed, the volume parameters (output volume) can be appropriately switched to those corresponding to the user wearing the HMD.

[0121] <Embodiment 6> In the sixth embodiment, when the HMD system determines that a user different from the previous user is wearing the HMD, it automatically switches the current brightness parameter to the brightness parameter (brightness of the display unit 206) corresponding to the user wearing the HMD. Therefore, as shown in Fig. 16, the HMD system according to the fourth embodiment has a brightness switching unit 1612 instead of the guardian setting switching unit 812 of the HMD system according to the second embodiment.

[0122] If a brightness parameter has been registered for a user wearing the HMD, the brightness switching unit 1612 switches the brightness parameter currently used in the HMD to the registered operability parameter. If a brightness parameter has not been registered for a user wearing the HMD, the brightness switching unit 1612 newly registers a brightness parameter for the user and switches to the registered brightness parameter.

[0123] 9, the process of step S1701 in the flowchart of Fig. 17 may be performed. In step S1701, the luminance switching unit 1612 switches the luminance parameter currently used in the HMD (the luminance of the display unit 206) to the luminance parameter corresponding to the user wearing the HMD.

[0124] According to the sixth embodiment, when the user wearing the HMD is replaced, the brightness parameters can be appropriately switched to those corresponding to the user wearing the HMD.

[0125] <Embodiment 7> In the seventh embodiment, the HMD system performs frequency decomposition on the outputs from the acceleration sensor and angular velocity sensor mounted on the HMD. Depending on the frequency distribution, the HMD system determines whether the HMD is "placed on a desk," "held and moving in the hand," or "mounted on the head and stationary." This allows the HMD system to detect the timing when the HMD is newly worn, and at that timing, determine the eye organs' The HMD system performs simple personal identification by comparing the difference in the positions of the eye organs, and if it determines that the user wearing the HMD is different from the previous user, it performs IPD adjustment.

[0126] The configuration of an HMD system according to the seventh embodiment will be described with reference to Fig. 18. The HMD system according to the seventh embodiment includes an acceleration sensor 1812, an angular velocity sensor 1813, a frequency decomposition unit 1814, and an HMD state determination unit 1815 in addition to the configuration of the HMD system according to the first embodiment.

[0127] The acceleration sensor 1812 acquires the acceleration of the HMD as a sensor output.

[0128] The angular velocity sensor 1813 acquires the angular velocity of the HMD as a sensor output.

[0129] The frequency decomposition unit 1814 performs frequency decomposition on the sensor output from the acceleration sensor 1812 or the angular velocity sensor 1813 .

[0130] The HMD state determination unit 1815 determines the state of the HMD based on the frequency distribution of the frequency-decomposed sensor output. If the sensor output is close to 0, the HMD state determination unit 1815 can determine that the HMD is placed somewhere. If the sensor output is distributed over a wide range of frequencies, the HMD state determination unit 1815 can determine that the HMD is being held or moving. If the sensor output has a weak distribution around 1 Hz, the HMD state determination unit 1815 can determine that the HMD is worn on the head and the user is stationary.

[0131] Here, when the state of the HMD changes, there is a high possibility that a new user will be wearing the HMD. Therefore, when the state of the HMD changes, the image sensor 1811 captures an image of the eye area again.

[0132] The processing of the seventh embodiment will be described with reference to the flowchart of Fig. 19. Note that steps with the same names in the flowchart of Fig. 19 and the flowchart of Fig. 7 perform similar processing in the two flowcharts, so their description will be omitted.

[0133] In step S1909, the frequency decomposition unit 1814 performs frequency decomposition on the sensor output from the acceleration sensor 1812 or the angular velocity sensor 1813.

[0134] In step S1910, the HMD state determination unit 1815 determines the state of the HMD based on the frequency distribution of the frequency-resolved sensor output.

[0135] In step S1911, the HMD state determination unit 1815 compares the previous determination result of the HMD state with the current determination result to determine whether the state of the HMD has changed. If it is determined that the state of the HMD has not changed, the process returns to step S1909. If it is determined that the state of the HMD has changed, the process proceeds to step S710.

[0136] FIG. 20 explains the state of the HMD and the frequency distribution after frequency decomposition of the sensor output (output of the angular velocity sensor and acceleration sensor). In the frequency distribution 2001 in the state 2002 in which the HMD is placed, it can be seen that the output (signal) is 0 in any frequency band. In the frequency distribution 2003 in the state 2004 in which the HMD is being moved by hand, it can be seen that a strong output is generated in the band centered between 1 Hz and 1 KHz. In the frequency distribution 2005 in the state 2006 in which the HMD is worn on the head and is stationary, it can be seen that a weak output is generated centered around 1 Hz. The HMD state determination unit 1815 performs frequency decomposition. By detecting changes in the fabric, it is possible to detect changes in the state of the HMD.

[0137] In step S1917, the IPD adjustment unit 216 determines whether the power has been turned off. If it is determined that the power has been turned off, the process of this flowchart ends. If it is determined that the power has not been turned off, the process proceeds to step S1909.

[0138] According to the seventh embodiment, the IPD adjustment can be started at the timing when the user wearing the HMD is changed.

[0139] Furthermore, in the above, "If A is greater than or equal to B, proceed to step S1; if A is less than (lower than) B, proceed to step S2" may be read as "If A is greater than (higher than) B, proceed to step S1; if A is less than or equal to B, proceed to step S2." Conversely, "If A is greater than (higher than) B, proceed to step S1; if A is less than (lower than) B, proceed to step S2" may be read as "If A is greater than (higher than) B, proceed to step S1; if A is less than (lower than) B, proceed to step S2." Therefore, unless a contradiction arises, "greater than or equal to A" may be read as "greater than (higher; longer; more) than A," and "less than or equal to A" may be read as "less than (lower; shorter; fewer) than A." Furthermore, "greater than (higher; longer; more) than A" may be read as "greater than or equal to A," and "less than (lower; shorter; fewer) than A" may be read as "less than or equal to A."

[0140] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). The entire device may be controlled by multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) sharing the processing.

[0141] The above processor is a processor in the broad sense, and includes general-purpose processors and dedicated processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Dedicated processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0142] Although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0143] <Other embodiments> The present invention can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program, or by a circuit that realizes one or more functions.

[0144] The disclosure of the above embodiments includes the following configurations, methods, and programs. (Configuration 1) Whether or not the user wearing the head-mounted display having the display means has been changed An information processing device for determining an image acquisition means for acquiring a first image of a first user's eye at a first time point and a second image of a second user's eye at a second time point after the first time point; a detection means for detecting a position of an organ in the eye of the first user based on the first image and detecting a position of an organ in the eye of the second user based on the second image; a determination means for determining whether the first user and the second user are different persons based on the position detected based on the first image and the position detected based on the second image; An information processing device comprising: (Configuration 2) further comprising an adjustment means for adjusting a position of the display means so as to correspond to the second user when the determination means determines that the first user and the second user are different persons; 2. The information processing device according to configuration 1, (Configuration 3) The device further includes a first switching means for switching a guardian setting, which is a range in which a person can safely act, to correspond to the second user when the determination means determines that the first user and the second user are different people. 3. The information processing device according to configuration 1 or 2. (Configuration 4) further comprising second switching means for switching a menu configuration displayed on the display means to correspond to the second user when the determination means determines that the first user and the second user are different persons; 4. The information processing device according to any one of configurations 1 to 3. (Configuration 5) further comprising a third switching means for switching an operation parameter to correspond to the second user when the determining means determines that the first user and the second user are different persons; 5. The information processing device according to any one of configurations 1 to 4. (Configuration 6) further comprising a fourth switching means for switching a parameter relating to a volume to be output so as to correspond to the second user when the determining means determines that the first user and the second user are different persons; 6. The information processing device according to any one of configurations 1 to 5. (Configuration 7) and a fifth switching means for switching the brightness of the display means to correspond to the second user when the determining means determines that the first user and the second user are different people. 7. The information processing device according to any one of configurations 1 to 6. (Configuration 8) The second time point is a time point at which it is determined that the display means has moved by more than a predetermined amount since the first time point. 8. The information processing device according to any one of configurations 1 to 7. (Configuration 9) The second time point is a time point at which it is determined that the state of the display means has changed. 8. The information processing device according to any one of configurations 1 to 7. (Configuration 10) The display device further includes a state determination means for determining whether or not the state of the display means has changed based on the frequency distribution of the sensor output output from the angular velocity sensor or the acceleration sensor. 10. The information processing device according to configuration 9. (Configuration 11) the determination means determines that the first user and the second user are different people when a difference between the first distance and the second distance is greater than a first threshold value; the first distance is a distance between a position of a first organ and a position of a second organ detected based on the first image; the second distance is a distance between the position of the first organ and the position of the second organ detected based on the second image; 11. The information processing device according to any one of configurations 1 to 10. (Configuration 12) the determination means determines that the first user and the second user are different people when a difference between the first gradient and the second gradient is greater than a second threshold value; the first gradient is a gradient of a line connecting the position of a first organ and the position of a second organ detected based on the first image; The second gradient is a gradient of a line connecting the position of the first organ and the position of the second organ detected based on the second image. 12. The information processing device according to any one of configurations 1 to 11. (Configuration 13) the first organ is the outer corner of the first eye of either the left or right eye, The second organ is the inner corner of the first eye. 13. The information processing device according to configuration 11 or 12. (Configuration 14) the first organ is the outer corner of the first eye of either the left or right eye, The second organ is the outer corner of a second eye different from the first eye. 13. The information processing device according to configuration 11 or 12. (Configuration 15) The first organ is the inner corner of the first eye of either the left or right eye, The second organ is the inner corner of a second eye different from the first eye. 13. The information processing device according to configuration 11 or 12. (method) A control method for an information processing device that determines whether a user wearing a head-mounted display having a display means has changed, comprising: a first image acquisition step of acquiring a first image of an eye of a first user at a first time point; a first detection step of detecting a position of an organ in the eye of the first user based on the first image; a second image acquisition step of acquiring a second image of an eye of a second user at a second time point that is later than the first time point; a second detection step of detecting a position of an organ in the eye of the second user based on the second image; a determination step of determining whether the first user and the second user are different people based on the position detected based on the first image and the position detected based on the second image; A control method comprising: (program) 16. A program for causing a computer to function as each means of the information processing device according to any one of configurations 1 to 15. [Explanation of symbols]

[0145] 10: HMD, 20: information processing device, 202: Imaging sensor (image acquisition unit), 210: Eye organ detector, 214: Personal identification unit (determination unit)

Claims

1. An information processing device that determines whether a user wearing a head-mounted display having a display means has changed, an image acquisition means for acquiring a first image of a first user's eye at a first time point and a second image of a second user's eye at a second time point after the first time point; a detection means for detecting a position of an organ in the eye of the first user based on the first image and detecting a position of an organ in the eye of the second user based on the second image; a determination means for determining whether the first user and the second user are different persons based on the position detected based on the first image and the position detected based on the second image; An information processing device comprising:

2. and an adjustment unit that adjusts the position of the display unit so as to correspond to the second user when the determination unit determines that the first user and the second user are different people.

2. The information processing apparatus according to claim 1, wherein:

3. The device further includes a first switching means for switching a guardian setting, which is a range in which a person can safely move, to correspond to the second user when the determination means determines that the first user and the second user are different people.

2. The information processing apparatus according to claim 1, wherein:

4. The system further comprises a second switching means for switching a configuration of a menu displayed on the display means to correspond to the second user when the determination means determines that the first user and the second user are different persons.

2. The information processing apparatus according to claim 1, wherein:

5. and a third switching means for switching an operation parameter to correspond to the second user when the determining means determines that the first user and the second user are different persons.

2. The information processing apparatus according to claim 1, wherein:

6. further comprising a fourth switching means for switching a parameter relating to a volume to be output so as to correspond to the second user when the determining means determines that the first user and the second user are different persons; 2. The information processing apparatus according to claim 1, wherein:

7. and a fifth switching means for switching the brightness of the display means to correspond to the second user when the determining means determines that the first user and the second user are different people.

2. The information processing apparatus according to claim 1, wherein:

8. The second time point is a time point at which it is determined that the display means has moved by more than a predetermined amount since the first time point.

2. The information processing apparatus according to claim 1, wherein:

9. The second time point is a time point at which it is determined that the state of the display means has changed.

2. The information processing apparatus according to claim 1, wherein:

10. The display device further includes a state determination means for determining whether or not the state of the display means has changed based on the frequency distribution of the sensor output output from the angular velocity sensor or the acceleration sensor.

10. The information processing apparatus according to claim 9,

11. the determination means determines that the first user and the second user are different people when a difference between the first distance and the second distance is greater than a first threshold value; the first distance is a distance between a position of a first organ and a position of a second organ detected based on the first image; the second distance is a distance between a position of the first organ and a position of the second organ detected based on the second image; 2. The information processing apparatus according to claim 1, wherein:

12. the determination means determines that the first user and the second user are different people when a difference between the first tilt and the second tilt is greater than a second threshold value; the first gradient is a gradient of a line connecting the position of a first organ and the position of a second organ detected based on the first image; the second gradient is a gradient of a line connecting the position of the first organ and the position of the second organ detected based on the second image; 2. The information processing apparatus according to claim 1, wherein:

13. the first organ is the outer corner of a first eye of either the left or right eye, The second organ is the inner corner of the first eye.

12. The information processing apparatus according to claim 11,

14. the first organ is the outer corner of a first eye of either the left or right eye, The second organ is a corner of the eye of a second eye different from the first eye.

12. The information processing apparatus according to claim 11,

15. the first organ is the inner corner of a first eye of either the left or right eye, The second organ is the inner corner of a second eye different from the first eye.

12. The information processing apparatus according to claim 11,

16. A control method for an information processing device that determines whether a user wearing a head-mounted display having a display means has changed, comprising: a first image acquisition step of acquiring a first image of an eye of a first user at a first time point; a first detection step of detecting a position of an organ in the eye of the first user based on the first image; a second image acquisition step of acquiring a second image of an eye of a second user at a second time point that is later than the first time point; a second detection step of detecting a position of an organ in the eye of the second user based on the second image; a determining step of determining whether the first user and the second user are different persons based on the position detected based on the first image and the position detected based on the second image; A control method comprising:

17. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 15.

Images