Control device, control method, and program

Abstract

To determine whether detected positional deviation of a display optical system is deviation intended by a user when detecting the positional deviation, and switch control.SOLUTION: A control device controls a head-mounted information processing apparatus, and comprises a control unit that controls a driving unit that can adjust the position of a display optical system included in the information processing apparatus. The control unit controls the driving unit on the basis of a first relative position indicating a relative position of the display optical system to the eyes of a user wearing the information processing apparatus, and a second relative position indicating a relative position of the display optical system to a body unit of the information processing apparatus.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a control device, a control method, and a program. [Background technology]

[0002] Conventionally, a technology has been developed that determines the eye position from a pupil image or the like, and drives the position of the display optical system of the head mounted display accordingly to adjust the interpupillary distance (IPD: InterPupillary Distance). There is a configuration that adjusts the IPD (InterPupillary Distance) to enable such suitable observation. For example, Patent Document 1 discloses a technology that matches a pupil image with user profile data and adjusts the IPD appropriately for the user wearing the device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6144681 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in the above-mentioned Patent Document 1 has a problem when the IPD adjustment result and the display optical system position suitable for the user are different. In other words, there is a problem when the user's preference and the IPD adjustment position performed by the device differ, and the user wants to make further adjustments, or when it is difficult to detect the eye position suitable for the user's observation due to strabismus. In other words, the display optical system position is automatically adjusted to a position that is inappropriate for the user by IPD adjustment. Furthermore, for example, even if the user touches and moves the display optical system position with his or her hand to resolve this, it is assumed that the head mounted display will determine that the detected eye position has shifted and perform IPD adjustment again. In this case, the user will continue to observe with the head mounted display with the IPD adjusted to an inappropriate position.

[0005] Therefore, the present invention has been made in consideration of the above-mentioned problems, and its object is to provide a control device that, when a positional misalignment of the display optical system is detected, determines whether the positional misalignment is intended by the user and switches control. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, according to one aspect of the present invention, there is provided a control device for controlling a head-mounted information processing device, comprising a control unit that controls a drive unit capable of adjusting the position of a display optical system provided in the information processing device, and the control unit controls the drive unit based on a first relative position indicating the relative position of the display optical system with respect to the eyeball of a user wearing the information processing device, and a second relative position indicating the relative position of the display optical system with respect to a main body of the information processing device. Effect of the Invention

[0007] According to the present invention, when a positional deviation of the display optical system is detected, it is possible to determine whether or not the positional deviation is one intended by the user, and to switch control. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an external perspective view of a head mounted display 100 according to an embodiment of the present invention. [Diagram 2] FIG. 1 is an explanatory diagram for explaining an example of a functional configuration of a control box 320 according to an embodiment of the present invention. [Diagram 3] FIG. 1 is a diagram showing a state in which a display body 101 according to an embodiment of the present invention is attached to a user's head by a mounting part 301. [Figure 4] 1 is a schematic diagram of a display optical system according to an embodiment of the present invention; [Diagram 5] A table summarizing the results of the posture detection means including the gaze detection camera 104, the encoder 109, and the posture detection camera 105, the corresponding expected movements of the display body 101, and how to deal with them. [Figure 6] Flowchart of the process for judging each case DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0010] Fig. 1 is an external perspective view of a head mounted display 100 according to an embodiment of the present invention. Fig. 1(a) is an external perspective view of the head mounted display 100 as viewed from the subject side. The head mounted display 100 is an example of a head-mounted information processing device. As shown in Fig. 1(a), the head mounted display 100 according to this embodiment has a display main body 101 and a mounting unit 301.

[0011] The display body 101 is a main body of the head mounted display 100 that includes a display panel 102 (described later) and an eyepiece lens 103 that is a display optical system, and can provide an XR (Extended Reality) experience to a user. Note that XR here can include VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality).

[0012] An orientation detection camera 105 and a background image camera 106 are arranged on the left and right sides in front of the display main body 101. The orientation detection camera 105 detects the orientation of the display main body 101 and aligns the drawing position of the CG image viewed by the user through the eyepiece lens 103 with the surrounding image that becomes the background. The orientation detection camera 105 is also used as an orientation detection means for detecting the user's orientation, as will be described later. The background image camera 106 plays the role of the eyes in the video see-through image, and captures an image that becomes the background image of the parts that are not CG images.

[0013] The wearing unit 301 is an attachment member for fixing the display main body 101 in front of the user's eyes by wearing it on the head like a headband. Since it is sufficient to fix the display main body 101 in front of the eyes, this form is not limited to a headband type, and it may be a helmet type that covers the entire upper head, or a glasses type that hangs on the ears.

[0014] The connecting mechanism unit 207 is a connecting means for connecting the display main body 101 and the mounting portion 301 .

[0015] The head mounted display 100 according to this embodiment is connected to a control box 302 which is a device that controls the position and the like of the display main body 101. Hereinafter, an example of the functional configuration of the control box 302 will be described with reference to FIG.

[0016] 2 is an explanatory diagram for explaining an example of a functional configuration of the control box 302. The control box 302 is a control device that performs various controls of the head mounted display 100. The control box 302 is also a housing that controls the image captured by the background image camera 106 and the CG image to be displayed on the display panel 102. The control box 302 includes a storage unit 310 and a control unit 330.

[0017] The storage unit 310 has a function of storing various information for the operation of the control box 302. The storage unit 310 can be realized by, for example, a read only memory (ROM) or a random access memory (RAM).

[0018] The control unit 330 functions as an arithmetic processing unit or a control device, and controls the overall operation or a part of the operation of each component of the control box 302 based on various programs recorded in the storage unit 310 etc. The control unit 330 can be realized by a processor such as a CPU (Central Processing Unit) or an MCU (Micro Controller Unit).

[0019] The control unit 330 controls, for example, an IPD adjustment motor 201 capable of adjusting the position of the display panel 103 included in the head mounted display 100. For example, the control unit 330 controls the IPD adjustment motor 201 based on the relative position of the display panel 103 with respect to the eyeballs of the user wearing the head mounted display 100 and the relative position of the display panel 103 with respect to the display main body 101. Details will be described later. Note that, hereinafter, the relative position of the display panel 103 with respect to the eyeballs of the user may be expressed as a first relative position. Also, the relative position of the display panel 103 with respect to the display main body 101 may be expressed as a second relative position.

[0020] The control box 302 also includes a communication unit 320 that performs various communications with the head mounted display 100. The control box 302 is communicatively connected to the display body 101 via a connection cable 303. The communication unit 320 can be connected to the Internet, for example, by a wired LAN (Local Area Network), a wireless LAN, or Wi-Fi (registered trademark). The control box 302 may be connected to the display body 101 by a wireless device, and in this form, the user can operate the display body 101 without being bothered by the connection cable 303. In addition, each function of the control unit 330 mounted on the control box 302 may be implemented as an application on a general PC. Furthermore, the control box 302 may be mounted on the head mounted display 100, and specifically, various functions of the control box 302, such as a determination means and a control means, may be implemented on a CPU mounted on a board in the display body 101. In this case, the user can experience XR using only the display main body 101, reducing the hassle of handling the control box 302 and connection cable 303 during the experience.

[0021] FIG. 1(b) is an external perspective view of the display main body 101 as seen from the user side. The eyepiece 103 is a display optical system through which the user observes the image on the display panel 102. The user can have an appropriate XR experience by observing the image captured by the background image camera 106 and the CG image displayed on the display panel 102 through the eyepiece 103. As will be described later, a gaze detection camera 104, which is a gaze detection means, is disposed behind the eyepiece 103. The gaze detection camera 104 captures an image of the user's eyes through the eyepiece 103 and obtains gaze information on where the user's gaze is directed. This makes it possible to detect the position of the user's eyes relative to the eyepiece 103, as will be described later.

[0022] The wearing detection sensor 110 is a wearing detection means that determines whether the display body 101 is in a position where the user can observe it, that is, whether the display body 101 is worn by the user. For example, this is configured with a proximity sensor device equipped with an element that emits infrared rays and an element that receives infrared rays, and determines whether an obstruction is present near the sensor. In this embodiment, it is possible to determine that the display body 101 is being worn if the user's nose is in the vicinity.

[0023] FIG. 3 is a diagram showing a state in which the display body 101 according to the embodiment of the present invention is attached to the user's head by the attachment unit 301. FIG. 3(a) shows a state in which the display body 101 is in front of the user's eyes (i.e., in front of the field of vision), and at this time, the user can experience XR. FIG. 3(b) shows a state in which the display body 101 has moved relative to the attachment unit 301 through the connection with the coupling mechanism unit 207. In other words, FIG. 3(b) shows a state in which the display body 101 is outside the field of vision of the user, and at this time, the user can check the surrounding situation and things in front of the user without the display body 101 blocking the field of vision. In the present invention, the state of FIG. 3(b) is called a flip-up state, and the change from the state of FIG. 3(a) to the state of FIG. 3(b) is called flip-up, flip-up drive, etc. Also, the state in FIG. 3(a) is the wearing state that enables the user to properly experience XR, but in the present invention, this is referred to as the flip-down state corresponding to the flip-up state, and driving from the state in FIG. 3(b) to the state in FIG. 3(a) is sometimes referred to as flip-down drive.

[0024] FIG. 4 is a schematic diagram of the display optical system and its periphery according to an embodiment of the present invention. As described above, the display panel 102 is a display panel that displays the image captured by the background image camera 106 and the CG image, and the user observes the display image of the display panel 102 through the eyepiece 103. The lens housing 107 is a housing part that holds the eyepiece 103 and the display panel 102. The gaze detection camera 104 is a gaze detection means that detects the direction of the user's gaze and the position of the eye relative to the eyepiece 103. The LED 108 is a light projection means for projecting appropriate infrared light onto the periphery of the user's eye when observing the periphery of the eye with the gaze detection camera 104. Here, in this embodiment, the observation of the eyeball by the gaze detection camera 104 is configured with an optical path that passes through the lens of the eyepiece 103, thereby realizing a reduction in the lens diameter direction of the eyepiece 103. Furthermore, by fixing the positions of eyepiece 103 and gaze detection camera 104 relative to one another and moving them in conjunction, it is possible to appropriately adjust the position of eyepiece 103 in relation to the results of detection of the position of the user's eye by gaze detection camera 104. With this configuration, if there is space radially outside eyepiece 103, it is also possible to configure observation without using eyepiece 103, for example by locating gaze detection camera 104 outside eyepiece 103. This makes it possible to observe the eyes while suppressing optical distortion and image degradation caused by passing through eyepiece 103.

[0025] The IPD adjustment motor 201 is an example of a driving unit, and is a stepping motor for positioning the eyepiece 103 at a position that matches the interpupillary distance of the user. The adjustment for adjusting the position of the eyepiece 103 (display optical system) to a position that matches the interpupillary distance of the user is called IPD adjustment. Generally, a mechanism is known that automatically calculates an IPD adjustment position suitable for the user from the image capture result of the gaze detection camera 104 and controls the IPD adjustment motor 201 to perform automatic IPD adjustment. In addition, even in a configuration without the IPD adjustment motor 201, the IPD adjustment can be configured such that the user directly touches the lens housing 107 with their fingers to adjust it. Even in such a configuration, the gaze detection camera 104 can detect the position of the eye relative to the eyepiece 103, and it is possible to determine whether the user's eye is in a position suitable for the design relative to the eyepiece 103.

[0026] In FIG. 4, the shaft 206 connected to the lens housing 107 is a guide member when driving the display optical system in the horizontal direction. FIG. 4 illustrates a configuration in which two IPD adjustment motors 201 are arranged to drive the left and right display optical systems, respectively. This enables IPD adjustment to move the display optical system to a position more suitable for each of the left and right eyes. On the other hand, the lens housing 107 may be connected to worm gears with teeth cut in opposite directions on the left and right, and driven in a direction to widen or narrow the interpupillary distance depending on the driving direction of the IPD adjustment motor 201. This is based on the assumption that the position of the display optical system during IPD adjustment is basically symmetrical on the left and right. With this configuration, it is possible to reduce the number of components for driving, including the IPD adjustment motor 201, and costs can be reduced. The encoder 109 is an encoder that detects the position of the eyepiece 103 relative to the display body 101. This makes it possible to determine whether the driving by the IPD adjustment motor 201 is performed as expected, and to determine whether the eyepiece 103 is misaligned relative to the display body 101 due to impact or the like. In the following description, the direction in which the interpupillary distance widens or narrows may be collectively referred to as the interpupillary distance direction.

[0027] In addition, in this configuration, when a load is applied with a finger or the like to drive the eyepiece 103 or lens housing 107 in the interpupillary direction, the IPD adjustment motor 201 rotates freely. This makes it possible to prevent damage to the display body 101 even when the user adjusts the IPD to a suitable position with their own hand or when the eyepiece 103 or lens housing 107 is unintentionally subjected to an impact.

[0028] 5 is a table summarizing the results of the posture detection means including the gaze detection camera 104, the encoder 109, and the posture detection camera 105, the corresponding assumed movements of the display main body 101, and how to deal with them. In the present invention, first, a case is considered in which the detection results of the gaze detection camera 104, which is the gaze detection means, and the encoder 109, which is the display system position detection means, deviate from the detection results at a certain point in time, for example, immediately after IPD adjustment. In this respect, one of the features is that the control of the IPD adjustment by the control unit 330 is switched depending on the detection results of the posture detection means. Each case will be explained below.

[0029] Case 1 indicates a state in which neither the detection result of the line-of-sight detection camera 104 nor the detection result of the encoder 109 is misaligned. This corresponds to a state in which the relative position of the eyepiece 103 with respect to the position of the user's eye nor the relative position of the eyepiece 103 with respect to the display body 101 is misaligned from each reference after the IPD adjustment is completed. In this case, the control unit 330 does not perform drive control for the IPD adjustment and holds the eyepiece 103 at the corresponding position. A preset value prepared in advance may be used to determine the misalignment. Specifically, the control unit 330 may not need to readjust the position of the eyepiece 103 when the following two conditions are satisfied. The first condition may be that the amount of misalignment of the relative position of the eyepiece 103 with respect to the position of the user's eye from the reference is less than a first preset value. The second condition may be that the amount of misalignment of the relative position of the eyepiece 103 with respect to the display body 101 from the reference is less than a second preset value. Here, the relative position of the eyepiece 103 with respect to the position of the user's eye is an example of a first relative position, and the relative position of the eyepiece 103 with respect to the display main body 101 is an example of a second relative position.

[0030] Case 2 shows a state where there is no deviation in the detection result of the gaze detection camera 104, but there is deviation in the detection result of the encoder 109. This corresponds to a state where the relative position of the eyepiece 103 with respect to the position of the user's eye has not deviated from the state where the IPD adjustment is completed, but the relative position of the eyepiece 103 with respect to the display body 101 has deviated. This is nothing but a situation where the eyepiece 103 moves with respect to the display body 101 and the encoder 109 detects the deviation, but the gaze detection camera 104 does not detect the deviation. Such a situation can be considered when the eyepiece 103 moves with respect to the display body 101 and at the same time the display body 101 moves with respect to the head, canceling the position deviation detected by the gaze detection camera 104. Or when it moves within the detection resolution of the gaze detection camera 104. However, in any case, it is difficult to assume such a situation from the viewpoint of the detection accuracy of the encoder 109 and the gaze detection camera 104, and this embodiment does not mention any special measures for such a situation. The specifications of the device may be determined from the design situation of these devices. For example, if the design detection resolution of the gaze detection camera 104 is coarse, the deviation detected by the encoder 109 may be determined as a deviation that needs to be corrected, and the IPD may be adjusted again. On the other hand, if the design detection error of the encoder 109 is large, the situation may be determined to be due to an error in the encoder 109, and the eyepiece 103 may be held in that position without adjusting the IPD.

[0031] Case 3 shows a state in which the detection result of the gaze detection camera 104 is misaligned, but the detection result of the encoder 109 is not misaligned. This is considered to be a situation in which the position of the display body 101 relative to the user's head is misaligned. In this case, it is considered preferable to readjust the position of the eyepiece 103 relative to the user's position. In this situation, the control unit 330 may control the IPD adjustment motor 201 to adjust the IPD again. For example, when the amount of deviation from the reference of the first relative position described above is equal to or greater than a first specified value and the amount of deviation from the reference of the second relative position is less than a second specified value, the control unit 330 controls the IPD adjustment motor 201 to readjust the position of the eyepiece 103.

[0032] Below, we will explain the difference in control between Case 4 and Case 5. In this embodiment, when the gaze detection camera 104 detects a deviation in this way and the encoder 109 also detects a deviation, the control by the control unit 330 is switched according to the detection result of the orientation detection means including the orientation detection camera 105.

[0033] For example, as in case 4, there are cases where the movement of the display body 101 is detected as a result of detection by the attitude detection means. If it is determined that there is a reasonable reason for this, it can be assumed that the eyepiece 103 is positioned in an unsuitable position for the user's observation. In such a case, it is desirable to redo the IPD adjustment. This may be the case when, for example, the mounting unit 301 is misaligned with respect to the user's head, causing a shift in the detection positions of the gaze detection camera 104 and the encoder 109.

[0034] On the other hand, as in case 5, even though the gaze detection camera 104 and the encoder 109 detect a misalignment, if the movement of the display body 101 is so small that it is unreasonable from the detection result of the attitude detection means, the IPD adjustment is not redone. This is because if the position of the eyepiece 103 moves even though the movement of the display body 101 is so small, it is considered that the user has moved only the display optical system by touching it with his / her finger. In other words, this situation is a situation in which the result of the IPD adjustment is not suitable for the user's observation, and it is nothing other than the user himself / herself moving the display optical system to a position where the user can observe it suitably. Therefore, in this case, it is determined that holding the eyepiece 103 at that position is suitable for the user, and the control unit 330 does not perform drive control for the IPD adjustment and holds the eyepiece 103 at that position.

[0035] A preset value may be used to determine whether or not the attitude detection means detects a movement (attitude change) of the display main body 101. For example, the control unit 330 may use the amount of attitude change when the amount of deviation of the first relative position from the reference is equal to or greater than a first preset value and the amount of deviation of the second relative position from the reference is less than a second preset value. For example, in the above case, the control unit 330 may further control the IPD adjustment motor 201 to readjust the position of the eyepiece 103 when the amount of attitude change is equal to or greater than a third preset value (Case 4).

[0036] On the other hand, in the above case, when the amount of change in attitude is further less than a third specified value, the control unit 330 may not need to readjust the position of the eyepiece 103 (Case 5). The third specified value may be set based on whether the above-mentioned misalignment of the eyepiece 103 with respect to the position of the user's eye and the misalignment of the eyepiece 103 with respect to the display body 101 are within an expected range possible from attitude detection. The first, second, and third specified values ​​may be set manually by the user, or may be set automatically from data obtained in the past, etc.

[0037] Furthermore, control unit 330 may determine whether the position of eyepiece 103 has been intentionally or unintentionally shifted by the user, and may not return eyepiece 103 to its original position if the user intentionally shifted it, but may return eyepiece 103 to its original position if the user unintentionally shifted it. In this way, the criteria for determining whether or not to adjust the position of eyepiece 103 may be appropriately modified in design from the above-mentioned concept.

[0038] 6 is a flowchart of the process related to the judgment of each case. Step S501 is a process of performing IPD adjustment. In this embodiment, the IPD adjustment is performed automatically, so that process is shown. After the IPD adjustment is performed in this step, the process proceeds to the next step S502.

[0039] Step S502 is a step of detecting the output of encoder 109 and judging whether the position of eyepiece 103 relative to display body 101 has shifted from the state immediately after step S501. Here, encoder 109 may be a system that performs deviation detection processing by intermittent operation such as several times per second, rather than constant monitoring. This enables power-saving operation to the extent that no inconvenience is felt in actual use. In step S502, if deviation is detected, the process proceeds to the next step S503, and if deviation is not detected, the process transitions to step S502 and continues monitoring again.

[0040] Step S503 is a step of detecting the output of the gaze detection camera 104 and judging whether the position of the eyepiece 103 with respect to the user's eye is shifted from the state immediately after step S501. This step is executed after the shift is detected in step S502, so that power saving operation is possible. The order of steps S502 and S503 may be reversed. In that case, the gaze detection camera 104 may not monitor continuously, but may perform the shift detection process by intermittent operation such as several times per second. This enables power saving operation within a range that does not cause inconvenience in actual use. In step S503, if there is a shift, the process proceeds to the next step S504, and if there is no shift, the process proceeds to step S502. This is because when it is judged that there is a shift in S502 and some movement of the display body 101 or the eyepiece 103 is expected, it is basically considered that there is a shift in S503 from the viewpoint of device accuracy as described above. In other words, if there is no deviation in S503, it is preferable to consider that there is doubt about the detection result of the encoder 109 in step S502.

[0041] Step S504 is a step for monitoring the output of the attitude detection means including the attitude detection camera 105, and detecting the movement of the display main body 101 relative to the state immediately after step S501. After detecting the movement, the process proceeds to the next step S505, where it is determined whether the movement is reasonable with respect to the detection results of S502 and S503.

[0042] In step S505, it is determined whether the movement of the display body 101 detected in S504 is within a range expected from the detection result of the encoder 109 detected in S502 and the detection result of the gaze detection camera 104 detected in step S503. For example, when the display body 101 is pulled to the left by the connection cable 303, the detection result of the attitude detection means indicates that the display body 101 has suddenly moved from right to left. At this time, it is expected from the detection results of the gaze detection camera 104 and the encoder 109 that the eyepiece lenses 103 for both eyes will move in the opposite direction to the movement of the display body 101 because they remain stationary relative to the display body 101 due to inertia.

[0043] In other words, if the movement of the gaze detection camera 104 and the encoder 109 is similarly opposite to the movement of the display body 101, it can be said that this is within the expected range that can be calculated from the detection result of the attitude detection means. In addition, a case can be considered where a result is detected in which the eyepiece 103 has moved only with respect to the display body 101 for the right eye in response to a result that the display body 101 has moved. Here, the movement of the display body 101 can be a situation where the display body 101 has been flipped up or down. In addition, a result in which the eyepiece 103 has moved only with respect to the display body 101 for the right eye can be considered where the gaze detection camera 104 and the encoder 109 detect this. This is considered to be a case where the user flips up or down the display body 101 with the right hand and touches the eyepiece 103 or the lens housing 107 for the right eye at that time. In such a case, it can be said that the detection result of the encoder 109 and the detection result of the gaze detection camera 104 are within the expected range that can be calculated from the movement of the display body 101.

[0044] On the other hand, even if the movement of the display body 101 is minute and it is considered that the display body 101 hardly moves at all, the movement detected by the encoder 109 and the gaze detection camera 104 may be a large value, such as several millimeters or more. This is a behavior that is difficult to predict as a calculable value from the detection result of the attitude detection means. In other words, the movement of the eyepiece 103 cannot be calculated from the movement of the display body 101. If it is determined in this way and it is within the expected range, it is determined that a situation has occurred in the eyepiece 103 that requires IPD adjustment again, and the process proceeds to step S501, where the IPD adjustment is performed again. On the other hand, if it is not within the expected range, it is determined that the user has intentionally touched and moved the eyepiece 103 or the lens housing 107, and the process proceeds to step S506.

[0045] Here, as a method of determination, for example, the direction of deviation detected by the gaze detection camera 104 and the encoder 109 may be consistent for the left eye display optical system and the right eye display optical system. This is based on the assumption that when the user moves the display optical system with his / her own hand as in case 5, the deviation width detected by the gaze detection camera 104 and the deviation width detected by the encoder 109 do not differ significantly. Also, for example, the direction of deviation detected by the gaze detection camera 104 and the encoder 109 may be approximately symmetrical, such as both the left and right display optical systems moving inward or outward, and this may be used as a criterion for determining case 4 or case 5. In other words, the control unit 330 may control the IPD adjustment motor 201 based on the direction of movement of the eyepiece 103 corresponding to the user's left eye and the direction of movement of the eyepiece 103 corresponding to the user's right eye. This is based on the assumption that if the user holds the display optical system with his / her fingers and moves it to adjust the IPD, the movement will basically be symmetrical. By making such a judgment, it is possible to judge whether or not it is case 5 with higher accuracy than by using only the attitude detection means.

[0046] Such a situation may occur, for example, in the following cases. When the pupil position detected by the gaze detection camera 104 and used as the reference for adjustment differs from the position of the eyepiece 103 where the user can observe favorably, for example, when the user has strabismus. Also, when the optical distance to the image forming surface of the eyepiece 103 when the gaze detection camera 104 detects the eye position and uses it as the reference for adjustment differs from the optical distance of the image that the user focuses on when actually experiencing XR, and the parallax differs between when the IPD is adjusted and when it is in use. Also, when the central image height normally used for IPD adjustment differs from the image height that the user focuses on during the XR experience, and the IPD adjustment position is adjusted so that the performance other than the center is improved as the resolution performance of the eyepiece 103. Also, for example, when a user touches a position within the range of precision or a minute deviation that does not require readjustment as a normal IPD adjustment specification, in search of a more suitable position. Also, for example, when the user becomes fatigued after long-term use and it becomes difficult to fuse the images of the left and right display panels 102, and it is desired to reduce the parallax at the focus point. In such a case, as in case 5, it is conceivable that the user will adjust the eyepiece 103 or the lens housing 107 by touching it with their fingers from the IPD adjustment position. In this case, it is determined that the current position is appropriate for the user, and the IPD adjustment will not be redone.

[0047] As a control method, instead of the control unit 330 making the judgment in step S505, a configuration in which a message such as "It is misaligned" or "Do you want to readjust?" is displayed on the display panel 102 and an operation by the user is waited for (an instruction is received). At this time, the user expresses his / her intention by pressing or not pressing some operation button or menu button depending on whether or not he / she wants to adjust the IPD again, and the presence or absence of IPD adjustment is switched accordingly. While the specification for automatically making the judgment in step S505 does not require user operation and is not troublesome, if a configuration is adopted in which a user operation is received, it is possible to more accurately judge whether or not the case is Case 5.

[0048] In step S506, the difference between the position adjusted in the IPD adjustment in step S501 and the current position is calculated and stored (saved) as a correction value. This allows the eyepiece 103 to be adjusted to a position suitable for the user, using the correction value stored in the storage unit 310 as a reference, even when the IPD is adjusted again due to a new instruction from the user.

[0049] Now, it is more useful to apply this correction value as a user-specific value when adjusting the IPD thereafter. Step S507 is a step for determining whether to continue to hold the correction value or to reset and erase it. For this determination, for example, a specification can be considered in which the correction value is held until a predetermined time has passed in a state where the head mounted display is removed, based on the detection result of the mounting detection sensor 110. In other words, if a user removes and reattaches the head mounted display within a very short period of time, it is assumed that the same user is removing and mounting the head mounted display, so a specification that adjusts the IPD to a suitable position using the same correction value is useful.

[0050] There is also a technology that performs personal authentication using an image of the eye, including the iris, acquired by the gaze detection camera 104 or the like. This allows for a specification in which a correction value is provided for each user. In this case, the IPD adjustment has the advantage that the adjustment is performed to a position suitable for each user. In step S507, it is constantly monitored whether the conditions are met, and if the conditions are met, the process proceeds to step S508.

[0051] In step S508, the correction value obtained in S506 is reset. As a result, in the subsequent IPD adjustment, the IPD is adjusted to a position appropriate for the design according to the detection results of devices including the line of sight detection camera 104.

[0052] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0053] (Other embodiments) The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-mentioned embodiments are supplied to a system or device via a network or various storage media, and the computer (or control unit, MPU, etc.) of the system or device reads and executes the program code. In this case, the program and the storage medium storing the program constitute the present invention.

[0054] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0055] Each functional unit in each of the above embodiments (variations) may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of a plurality of functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Furthermore, each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, the device may have a processor and a memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the device may be realized by the processor reading and executing the control program from the memory.

[0056] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0057] [Configuration 1] A control device for controlling a head-mounted information processing device, A control unit that controls a drive unit that can adjust a position of a display optical system included in the information processing device; Equipped with The control unit is controlling the drive unit based on a first relative position indicating a relative position of the display optical system with respect to an eyeball of a user wearing the information processing device, and a second relative position indicating a relative position of the display optical system with respect to a main body of the information processing device; Control device.

[0058] [Configuration 2] The control unit is controlling the driving unit based on a first deviation amount indicating a deviation amount of the first relative position from a reference and a deviation amount of the second relative position from a reference; 2. The control device according to configuration 1.

[0059] [Configuration 3] The control unit is controlling the driving unit to readjust a position of the display optical system when a deviation amount of the first relative position from a reference is equal to or greater than a first specified value and a deviation amount of the second relative position from a reference is less than a second specified value; 3. The control device according to configuration 2.

[0060] [Configuration 4] The control unit is when the deviation amount of the first relative position from the reference is less than a first specified value and the deviation amount of the second relative position from the reference is less than a second specified value, the position of the display optical system is not readjusted. 4. The control device according to configuration 2 or 3.

[0061] [Configuration 5] the first relative position is based on a detection result obtained by a gaze detection means provided in the information processing device, A control device according to any one of configurations 1 to 4.

[0062] [Configuration 6] The control unit is and controlling the driving unit based on a detection result obtained by an attitude detection means of the information processing device. A control device according to any one of configurations 1 to 5.

[0063] [Configuration 7] The control unit is controlling the drive unit to readjust a position of the display optical system when a deviation amount of the first relative position from a reference is equal to or greater than a first specified value, a deviation amount of the second relative position from a reference is equal to or greater than a second specified value, and further, the amount of change in attitude is equal to or greater than a third specified value. The control device according to configuration 6.

[0064] [Configuration 8] The control unit is when an amount of deviation of the first relative position from a reference is equal to or greater than a first specified value, an amount of deviation of the second relative position from a reference is equal to or greater than a second specified value, and the amount of change in attitude is equal to or greater than a third specified value, and an instruction for readjustment from a user is received, controlling the drive unit to readjust a position of the display optical system. 8. The control device according to configuration 7.

[0065] [Configuration 9] The control unit is when the deviation amount of the first relative position from a reference is equal to or greater than a first specified value, the deviation amount of the second relative position from a reference is equal to or greater than a second specified value, and further, the amount of change in attitude is less than a third specified value, the position of the display optical system is not readjusted. The control device according to configuration 6.

[0066] [Configuration 10] The control unit is controlling the driving unit based on a moving direction of a display optical system corresponding to the left eye of the user and a moving direction of a display optical system corresponding to the right eye of the user; 2. The control device according to configuration 1.

[0067] [Configuration 11] The control unit is when an amount of deviation of the first relative position from a reference is equal to or greater than a first specified value, an amount of deviation of the second relative position from a reference is equal to or greater than a second specified value, and further, the amount of change in posture is less than a third specified value, the current first relative position and the second relative position are stored in a storage unit as references. The control device according to configuration 5.

[0068] [Configuration 12] The control device is mounted on the information processing device. 12. The control device according to any one of configurations 1 to 11.

[0069] [method] A control method for controlling a head-mounted information processing device, comprising: Controlling a drive unit capable of adjusting a position of a display optical system included in the information processing device; Including, Controlling the drive unit includes: controlling the drive unit based on a first relative position indicating a relative position of the display optical system with respect to an eyeball of a user wearing the information processing device, and a second relative position indicating a relative position of the display optical system with respect to a main body of the information processing device; A control method implemented by a computer, comprising:

[0070] [program] Computer, A control device for controlling a head-mounted information processing device, A control unit that controls a drive unit that can adjust a position of a display optical system included in the information processing device; Equipped with The control unit is controlling the drive unit based on a first relative position indicating a relative position of the display optical system with respect to an eyeball of a user wearing the information processing device, and a second relative position indicating a relative position of the display optical system with respect to a main body of the information processing device; A program that functions as a control device.

Claims

1. A control device for controlling a head-mounted information processing device, A control unit that controls a drive unit that can adjust a position of a display optical system included in the information processing device; Equipped with The control unit is controlling the drive unit based on a first relative position indicating a relative position of the display optical system with respect to an eyeball of a user wearing the information processing device, and a second relative position indicating a relative position of the display optical system with respect to a main body of the information processing device; Control device.

2. The control unit is controlling the driving unit based on a first deviation amount indicating a deviation amount of the first relative position from a reference and a deviation amount of the second relative position from a reference; The control device according to claim 1 .

3. The control unit is controlling the driving unit to readjust a position of the display optical system when a deviation amount of the first relative position from a reference is equal to or greater than a first specified value and a deviation amount of the second relative position from a reference is less than a second specified value; The control device according to claim 2.

4. The control unit is when the deviation amount of the first relative position from the reference is less than a first specified value and the deviation amount of the second relative position from the reference is less than a second specified value, the position of the display optical system is not readjusted. The control device according to claim 2.

5. the first relative position is based on a detection result obtained by a gaze detection means provided in the information processing device, The control device according to claim 1 .

6. The control unit is and controlling the driving unit based on a detection result obtained by an attitude detection means of the information processing device. The control device according to claim 1 .

7. The control unit is controlling the drive unit to readjust a position of the display optical system when a deviation amount of the first relative position from a reference is equal to or greater than a first specified value, a deviation amount of the second relative position from a reference is equal to or greater than a second specified value, and further, the amount of change in attitude is equal to or greater than a third specified value. The control device according to claim 6.

8. The control unit is when an amount of deviation of the first relative position from a reference is equal to or greater than a first specified value, an amount of deviation of the second relative position from a reference is equal to or greater than a second specified value, and the amount of change in attitude is equal to or greater than a third specified value, and an instruction for readjustment from a user is received, controlling the drive unit to readjust a position of the display optical system. The control device according to claim 7.

9. The control unit is when the deviation amount of the first relative position from a reference is equal to or greater than a first specified value, the deviation amount of the second relative position from a reference is equal to or greater than a second specified value, and further, the amount of change in attitude is less than a third specified value, the position of the display optical system is not readjusted. The control device according to claim 6.

10. The control unit is controlling the driving unit based on a moving direction of a display optical system corresponding to the left eye of the user and a moving direction of a display optical system corresponding to the right eye of the user; The control device according to claim 1 .

11. The control unit is when an amount of deviation of the first relative position from a reference is equal to or greater than a first specified value, an amount of deviation of the second relative position from a reference is equal to or greater than a second specified value, and further, the amount of change in posture is less than a third specified value, the current first relative position and the second relative position are stored in a storage unit as references. The control device according to claim 5.

12. The control device is mounted on the information processing device. A control device according to any one of claims 1 to 11.

13. A control method for controlling a head-mounted information processing device, comprising: Controlling a drive unit capable of adjusting a position of a display optical system included in the information processing device; Including, Controlling the drive unit includes: controlling the drive unit based on a first relative position indicating a relative position of the display optical system with respect to an eyeball of a user wearing the information processing device, and a second relative position indicating a relative position of the display optical system with respect to a main body of the information processing device; A control method implemented by a computer, comprising:

14. Computer, A control device for controlling a head-mounted information processing device, A control unit that controls a drive unit that can adjust a position of a display optical system included in the information processing device; Equipped with The control unit is controlling the drive unit based on a first relative position indicating a relative position of the display optical system with respect to an eyeball of a user wearing the information processing device, and a second relative position indicating a relative position of the display optical system with respect to a main body of the information processing device; A program that functions as a control device.

Images