Video display device and its control method, program

Abstract

To provide a video display device that can change diopter in correspondence with a usage state.SOLUTION: Display units 102a, 102b and lenses 103a, 103b included in a video display device 1 form a plurality of display optical systems that display videos on the right and left eyes of a user. The video display device 1 comprises: a diopter change instruction unit 106 and diopter change driving units 104a, 104b that can change a diopter related to the plurality of display optical systems; and an object position calculation unit 108 that calculates an object position of a displayed object in the videos. The diopter change instruction unit 106 issues a drive instruction to the diopter change driving units 104a, 104b based on information acquired from a user information recording unit 109 and an operation mode recording unit 110 that record information on the state of usage of the video display device 1.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for adjusting visibility in a video display device.

Background Art

[0002] There are video display devices that are worn on the user's head for use, and video display devices that are worn by the user like glasses for use. A display unit is disposed near the user's eyes, and parallax video display processing is performed for each of the user's left and right eyes. By the user visually recognizing the displayed parallax video, a stereoscopic effect for the object displayed in the parallax video can be obtained.

[0003] In Patent Document 1, a technique for reducing the discomfort of a displayed image and reducing the dizziness of an observer during long-term continuous use is disclosed. Based on the state of the user's eyes, image processing is performed so that other portions of the displayed parallax video except for the portions focused on the retina are blurred.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, when a human looks at an object, the convergence angle, which is the angle at which the lines of sight of the two eyes intersect, and the focal length of the lens in the eyeball change according to the distance from the object to the eyes. In the real space, the relationship between the convergence angle and the focal length of the lens always approximately matches.

[0006] In the prior art disclosed in Patent Document 1, changes in the position of an object are represented by the displayed parallax image, but the display position (diopter) of the parallax image in the optical axis direction of the image display device is always constant. Therefore, when a user views the image of an object displayed as a parallax image, the user's convergence angle changes according to the position of that object. At this time, the focal length of the lens changes according to the magnitude of the convergence angle based on experience in real space. In this case, there is a possibility that the focal length of the lens and the diopter of the image display device will not match. As a result, the user may not be able to focus on the displayed parallax image, making it difficult to see clearly, and good visibility may not be ensured.

[0007] Therefore, when adjusting the diopter of an image display device, it is desirable to change the diopter according to the position of the object in the parallax image that determines the user's convergence angle, by driving the lens within the device. However, the relationship between the user's convergence angle and the focal length of the lens varies from person to person due to factors such as age and visual acuity. If the diopter of the image display device were to be uniformly changed according to the position of the displayed object in the parallax image, some users may not be able to focus on the displayed parallax image, making it difficult to see clearly and thus failing to ensure good visibility. In addition, the appropriate lens drive speed may differ depending on whether the image display device is in normal mode without power saving or in power-saving mode for long-term use. Due to such differences in users (individual differences) and usage conditions, including the operating mode of the image display device, the appropriate diopter adjustment operation may differ. The object of the present invention is to provide an image display device that enables diopter adjustment according to the usage conditions. [Means for solving the problem]

[0008] An embodiment of the present invention is an image display device comprising first and second display optical systems that display images to the left and right eyes of a user, respectively, and includes a calculation means for calculating the object position of an object in the image produced by the first and second display optical systems, and the distance between the object position and the user. ChangeDiopter changing means for changing the diopter of the first and second display optical systems based on information on the amount of change in diopter relative to, The diopter changing means uses the information on the amount of diopter change to determine the diopter changing speed when changing the diopter of the first and second display optical systems. It is characterized by the following: [Effects of the Invention]

[0009] The present invention provides a display device that enables diopter adjustment in accordance with the usage conditions. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram showing the general configuration of the video display device of this embodiment. [Figure 2] This is a schematic diagram illustrating the state of a video display device when the lens moves. [Figure 3] This is a schematic diagram illustrating an example of video data. [Figure 4] This is a schematic diagram showing how the image on the display unit appears to an observer's eye through the lens. [Figure 5] This is a schematic diagram showing the change in the user's diopter with respect to the distance to an object. [Figure 6] This is a flowchart explaining the operation of a video display device. [Figure 7] This is a schematic diagram illustrating the diopter adjustment operation of a video display device. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. While the embodiments show examples of diopter change control in image display devices such as head-mounted or eyeglass-type devices, the present invention can be applied to various forms of image display devices capable of displaying parallax images. A parallax image is an image having parallax, consisting of multiple images from different viewpoints.

[0012] Figure 1 shows a schematic configuration of the video display device 1 according to this embodiment. Details of each part will be described later, but the video data acquisition unit 100, display processing unit 101, diopter change instruction unit 106, object position calculation unit 108, user information recording unit 109, and operation mode recording unit 110 are realized by one or more processors, such as a CPU, reading and executing a program. The video display device 1 can be used in two forms: one in which the user wears it on their head, and another in which it is worn like eyeglasses. In either form, the video display device 1 can be fixed in the vicinity of the user's left eye 2a and right eye 2b. The symbol 'a' is appended to the code of components related to the user's left eye 2a, and the symbol 'b' is appended to the code of components related to the user's right eye 2b to distinguish them.

[0013] The video display device 1 comprises a video data acquisition unit 100 and a display processing unit 101. The video data acquisition unit 100 acquires video data for display via an external device or network. The display processing unit 101 performs adjustments such as display magnification on the acquired video data. The processed video data is sent to two display units 102a and 102b for display. For example, there is a configuration in which the video data is divided and displayed as data for each of the two display units 102a and 102b. The system is not limited to this configuration, and it may also be a configuration in which the screen of one display unit is divided into two and the video data is displayed on the divided screen.

[0014] The image display device 1 includes first and second display optical systems corresponding to the left eye 2a and the right eye 2b, respectively. The first display optical system has a lens 103a, and the second display optical system has a lens 103b. The images displayed on the display units 102a and 102b are presented to the left eye 2a and the right eye 2b through the corresponding lenses 103a and 103b, respectively.

[0015] The diopter change driving units 104a and 104b have a driving source such as a motor, and drive the lenses 103a and 103b respectively. That is, the lenses 103a and 103b move in the directions along their optical axes (see arrows 104a1 and 104b1) by the diopter change driving units 104a and 104b. In the present embodiment, although the optical axes 103a1 and 103b1 of the lenses 103a and 103b are described as passing through the centers of the display units 102a and 102b and the eyes 2a and 2b respectively, it is not limited thereto.

[0016] FIG. 2 is an explanatory diagram of the state when the lenses 103a and 103b are moved by the diopter change driving units 104a and 104b. FIG. 2(A) is a diagram showing the state when the lenses 103a and 103b are respectively moved close to the display units 102a and 102b. FIG. 2(B) is a diagram showing the state when the lenses 103a and 103b are respectively moved close to the eyes 2a and 2b. The user, who is an observer, visually recognizes virtual images 105a and 105b when viewing the display units 102a and 102b with their eyes 2a and 2b through the lenses 103a and 103b. The position of the virtual images 105a and 105b in the direction of the optical axes 103a1 and 103b1 based on the position of the observer's eyes 2a and 2b is defined as the virtual image formation position i.

[0017] As shown in FIGS. 2(A) and 2(B), the virtual image formation position i can be changed by changing the positions of the lenses 103a and 103b. For example, as the lenses 103a and 103b approach the display units 102a and 102b, the virtual image formation position i approaches the eyes 2a and 2b. Conversely, when the lenses 103a and 103b approach the eyes 2a and 2b, the virtual image formation position i moves away from the eyes 2a and 2b. Therefore, by moving the lenses 103a and 103b by the diopter change driving units 104a and 104b, the diopter can be changed.

[0018] In this embodiment, when the lenses 103a and 103b approach the display units 102a and 102b, the virtual image formation position i approaches the eyes 2a and 2b. However, the present invention is not limited to this configuration. When the lenses 103a and 103b approach the display units 102a and 102b, the virtual image formation position i may move away from the eyes 2a and 2b. Also, in order to change the diopter, a configuration is shown in which the lenses 103a and 103b are moved by the diopter change drive units 104a and 104b, but it is not limited to the movement of the optical member. For example, a configuration in which the display units 102a and 102b are moved by the diopter change drive units 104a and 104b may be used. Further, an embodiment in which the diopter change drive units 104a and 104b are configured by adopting a method that does not involve the movement of the lenses 103a and 103b, such as a method using a liquid lens, may also be used. Also, in a configuration in which the diopter is changed using a motor, the driving method of the motor is not limited. However, since the video display device 1 is located near the observer's ears, a highly quiet motor is preferable. For example, a voice coil motor, a vibration wave motor, or the like is preferable. Also, in FIGS. 1 and 2, the lenses 103a and 103b are represented by a single lens, but each may be composed of a plurality of lenses, and a configuration in which the positions of the plurality of lenses are moved during diopter change may also be used.

[0019] The diopter change instruction unit 106 in FIG. 1 determines the amount of diopter change with respect to the position of the displayed object in the displayed video and gives an instruction corresponding to the change amount. Specifically, the diopter change instruction unit 106 acquires information on the current virtual image formation position i from the diopter change drive units 104a and 104b, determines the next virtual image formation position i, and instructs the diopter change drive units 104a and 104b. Here, since there is a correspondence relationship between the virtual image formation position i and the positions of the lenses 103a and 103b, information on the current positions of the lenses 103a and 103b may be used as information on the current virtual image formation position i. In the case of a configuration in which the display units 102a and 102b are moved, information on the current positions of the display units 102a and 102b may be used as information on the current virtual image formation position i. As described above, information on the relative positions of the display units 102a and 102b and the lenses 103a and 103b may be used as information on the current virtual image formation position i.

[0020] Please refer to Figure 3 for an explanation of the video data. Figure 3 is a schematic diagram illustrating an example of video data 100D. The video data 100D consists of, for example, data for the left eye video 100a displayed on the left eye display unit 102a and data for the right eye video 100b displayed on the right eye display unit 102b.

[0021] In this embodiment, an example is shown where the video data 100D is composed of data from two videos 100a and 100b, but it is not limited to this. For example, the display processing unit 101 may process the data based on 3D data to generate parallax images that are displayed on the two display units 102a and 102b. Alternatively, video data captured by a camera mounted on the video display device 1 may be treated as at least a part of the video data 100D. Such a configuration makes it possible to display video in an augmented reality representation format on the video display device 1. For example, video data 100D is created by superimposing video captured by a camera mounted on the video display device 1 with video based on pre-created 3D data, and the composite image is displayed.

[0022] As shown in Figure 3, the data for the left eye image 100a and the right eye image 100b each contain data for the displayed objects 107a and 107b. In this embodiment, the case where there is one displayed object each for 107a and 107b is shown, but it is not limited to this. For example, it is also possible to have a configuration in which multiple objects are displayed in the image captured by the camera. In that case, for example, a gaze direction detection sensor capable of detecting the gaze direction of the user's eyes 2a and 2b is mounted on the device. Processing is performed to treat the object that the user's gaze is directed towards as the displayed object 107a and 107b.

[0023] Figure 4 is a schematic diagram showing the state when an observer visually views the displayed image through the lens. The left-eye image 100a, corresponding to the image data 100D shown in Figure 3, is displayed on the left-eye display unit 102a, and the right-eye image 100b is displayed on the right-eye display unit 102b. The observer observes the displayed images on the display units 102a and 102b through the lenses 103a and 103b with their eyes 2a and 2b. At this time, virtual images 105a and 105b are projected onto the observer's eyes 2a and 2b, as shown in Figure 4. The displayed objects 107a and 107b are displayed within the virtual images 105a and 105b, in a direction perpendicular to the optical axes 103a1 and 103b1, and at positions distanced from the optical axes 103a1 and 103b1 by distances Xa and Xb, respectively.

[0024] When an observer fixates on display objects 107a and 107b with eyes 2a and 2b, respectively, eyes 2a and 2b rotate as indicated by the arrows to face the direction of display objects 107a and 107b. At this time, the optical axes 2a1 and 2b1 of the lenses of eyes 2a and 2b are tilted to pass through display objects 107a and 107b, respectively. The angle between optical axes 2a1 and 2b1 is called the convergence angle θ. In this state, the display objects 107a and 107b appear to the observer as fused together and as a single object. In other words, the display object 107c appears to exist at a virtual object position Z in the direction of optical axes 103a1 and 103b1, relative to the positions of eyes 2a and 2b. There is a correlation between the position Z of the object perceived by the observer and the convergence angle θ, and humans have this correlation as a sense derived from everyday experience. Therefore, the observer (user) recognizes that the displayed object 107c is located at the virtual object position Z, based on the convergence angle θ when viewing the displayed objects 107a and 107b.

[0025] The object position calculation unit 108 shown in Figure 1 can calculate the virtual object position Z from the geometric calculation shown in Figure 4, based on the distances Xa and Xb of the display objects 107a and 107b related to the video data 100D from the optical axes 103a1 and 103b1.

[0026] The user information recording unit 109 and the operation mode recording unit 110 shown in Figure 1 are examples of usage status recording units that record information about the usage status of the video display device 1 in memory connected to the processor. In this embodiment, examples of user information and operation mode information are described, but the invention is not limited thereto, and various modifications and changes are possible within the scope of the gist of the present invention.

[0027] The user information recording unit 109 can record user information. A specific example is shown with reference to Figure 5. Figure 5 is a schematic graph showing the change in the user's diopter (horizontal axis) with respect to object distance (vertical axis). As shown by line (A) in Figure 5, a person's diopter generally changes according to the object distance. When a person is looking at an object that is close, their diopter adjusts to the close position, and conversely, when they are looking at an object that is far away, their diopter adjusts to the far position. As a result, a person can see objects clearly whether they are close or far away.

[0028] In Figure 5, the relationship between object distance and diopter, shown by line (A), shows that the diopter changes significantly as the object distance changes. On the other hand, in the relationship between object distance and diopter, shown by line (B), the diopter does not change much even when the object distance changes. This can be due to, for example, a decrease in the focusing ability of the eye's lens due to aging. Therefore, younger people tend to show a relationship like line (A), while middle-aged and older people tend to show a relationship like line (B). In people who show a relationship like line (B), the range of object distances at which they can see clearly is narrow, and it is often difficult to see objects that are far or close clearly. In such cases, using glasses with different diopters depending on the viewing position, such as reading glasses, can generally allow them to see objects that are far or close clearly.

[0029] In Figure 5, the relationship between object distance and diopter shown by line (C) is the same as in the case of line (A), even though the relationship between the change in object distance and the change in diopter is the same. However, the diopter at the same object distance is different. This is because the object distance at which objects appear clear differs due to the effects of nearsightedness, farsightedness, etc. In such cases, corrective glasses are often used to make the relationship between object distance and diopter as shown by line (A).

[0030] As described above, there are individual differences in a user's diopter in relation to object distance, and the amount of change in diopter in relation to the change in object distance differs from person to person. The user information recording unit 109 records information based on the relationship between object distance and diopter for each user. In the recording method, for example, the display units 102a and 102b display different diopter measurements for the virtual object position Z. The diopter change drive units 104a and 104b change the diopter, and a process is executed to record the positions of the lenses 103a and 103b that match the diopter for the virtual object position Z. Furthermore, it is generally possible to estimate whether the user's diopter change is closer to the characteristics shown by line (A) or line (B) based on their age. For example, one method is to input the user's age at the time of use, and record the data estimated from the input age in the user information recording unit 109. Note that the method of recording the information recorded in the user information recording unit 109 is not limited to a specific method.

[0031] The operation mode recording unit 110 shown in Figure 1 can record the operation mode of the video display device 1. Specifically, the video display device 1 has a normal mode that does not reduce power consumption and a power-saving mode. The operation mode recording unit 110 records data indicating which of the two modes is set. Note that the above two modes are examples, and data for various operation modes can be recorded in the operation mode recording unit 110 as needed.

[0032] The operation of the video display device 1 will be explained with reference to the flowchart in Figure 6. The flowchart in Figure 6 is executed from S1 when the power to the video display device 1 is turned on or otherwise initiated to start displaying images on the display units 102a and 102b. In S1, the object position calculation unit 108 calculates the virtual object positions Z of the displayed objects 107a and 107b related to the video data 100D as seen by the user's eyes 2a and 2b.

[0033] In S2, the diopter change instruction unit 106 calculates appropriate operation instruction information regarding the diopter change operation performed by the diopter change drive units 104 and 104b. This is done using the virtual object position Z information calculated in S1, and the user information and operation mode information recorded in the user information recording unit 109 and the operation mode recording unit 110, respectively.

[0034] The user information used in S2 is, for example, information based on the relationship between the object distance and diopter relative to the user, as shown in Figure 5. The diopter change instruction unit 106 calculates instruction information for the diopter change operation to be performed by the diopter change drive units 104a and 104b based on the relationship between the object distance and diopter relative to the user.

[0035] Figure 7 is a graph showing the relationship between the virtual object position Z (vertical axis) and the diopter of the video display device 1 (horizontal axis), calculated by the diopter change instruction unit 106. For example, consider a first user with the characteristics shown by line (A) in Figure 5. In this case, operation instructions are given to the diopter change drive units 104a and 104b so that the characteristics are as shown by line (D) in Figure 7. Similarly, for second and third users with the characteristics shown by lines (B) and (C) in Figure 5, operation instructions are given to the diopter change drive units 104a and 104b so that the characteristics are as shown by lines (E) and (F) in Figure 7, respectively. Therefore, the diopter change range of the video display device 1 is narrower for the second user compared to the first user. Also, compared to the first user, the diopter change range of the video display device 1 is the same for the third user, but the diopter is different when the virtual object position Z is the same. By performing diopter adjustments according to the user in this way, even if the amount of diopter change in relation to object distance differs for each user, a clear display at the appropriate diopter is possible regardless of the user.

[0036] Furthermore, the operating mode information used in S2 in Figure 6 is the operating mode information recorded in the operating mode recording unit 110. In S2, the driving speed of the diopter change drive units 104a and 104b is changed according to the operating mode so that the diopter is calculated based on the virtual object position Z and user information. If the operating mode recorded in the operating mode recording unit 110 is the normal mode, the diopter change drive units 104a and 104b change the diopter at the maximum possible driving speed. On the other hand, if the operating mode recorded in the operating mode recording unit 110 is the power saving mode, the diopter change drive units 104a and 104b change the diopter at a slower speed than the driving speed in the normal mode. If the driving speed for diopter change is small, timing discrepancies in diopter may occur, potentially causing discomfort in the display. To suppress discomfort in the display, it is desirable to maximize the driving speed for diopter change. However, increasing the driving speed for diopter change requires driving the lenses 103a and 103b at high speed, which requires more power. Therefore, in normal mode, diopter adjustment is performed at a higher drive speed to minimize any display distortion. In power-saving mode, however, some display distortion is tolerated, and diopter adjustment is performed at a lower drive speed than in normal mode, resulting in lower power consumption. In other words, it is possible to control the drive time for diopter adjustment according to the operating mode. With this configuration, appropriate driving can be achieved according to the usage state of the video display device 1.

[0037] This embodiment demonstrates a configuration that uses both user information and operating mode to control the operation of appropriate diopter adjustment. However, the configuration is not limited to this, and may also be one that controls the operation of appropriate diopter adjustment based on either user information or operating mode. For example, the drive speed of diopter adjustment may be changed based on user information, and the drive speed of diopter adjustment may be slower when the diopter adjustment range is narrower than when the diopter adjustment range is wider. Furthermore, a configuration may be used to control the operation of diopter adjustment with respect to the virtual object position Z based on information other than user information and operating mode.

[0038] In S3 of Figure 6, the diopter change instruction unit 106 instructs the diopter change drive units 104a and 104b, respectively, to perform the diopter change operation determined in S2, and the diopter change drive units 104a and 104b operate. The diopter of the image display device 1 is changed by driving the lenses 103a and 103b. Next, in S4, the display units 102a and 102b display the image corresponding to the image data 100D according to the instructions of the display processing unit 101.

[0039] In this embodiment, the display processing of the display units 102a and 102b is performed in S4 after the diopter change operation performed in S3. However, the embodiment is not limited to this configuration, and a configuration in which the processing of S3 and S4 is performed simultaneously, or a configuration in which the processing of S4 is performed before S3, is also possible.

[0040] With the configuration described above, the diopter adjustment operation based on the virtual object position Z can be appropriately performed according to the usage status of the video display device 1, such as differences in the user and operating mode. Appropriate diopter adjustment operation can be achieved regardless of the usage status of the video display device 1, such as individual differences in the relationship between object distance and diopter, which differ from user to user, and differences in the operating mode of the video display device 1. According to this embodiment, a clear display is possible regardless of the virtual object position Z even when the user is different, and the video display device 1 can be used for a long time depending on the usage status. Information such as user information, which is information about the usage status of the video display device 1, may be input by the user or an external device each time it is used.

[0041] Although preferred embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of its gist.

[0042] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of Symbols]

[0043] 1. Video display device 101 Display Processing Unit 102a,102b Display section 103a, 103b lenses 104a, 104b Diopter adjustment drive unit 106 Diopter adjustment indicator 108 Object position calculation section 109 User Information Recording Section 110 Operation Mode Recording Unit

Claims

1. An image display device comprising first and second display optical systems that display images to the user's left eye and right eye, respectively, A calculation means for calculating the object position of a displayed object in the image produced by the first and second display optical systems, The system includes diopter changing means for changing the diopter of the first and second display optical systems based on information on the amount of change in diopter in response to changes in the object position and the user's distance from the object, The diopter changing means uses the information on the amount of diopter change to determine the diopter changing speed when changing the diopter of the first and second display optical systems. A video display device characterized by the following features.

2. The system includes recording means for recording information on the amount of change in diopter. The video display device according to feature 1.

3. The diopter changing means determines the amount of diopter change with respect to the object position using the information on the amount of diopter change. The video display device according to claim 1 or 2.

4. The diopter changing means uses the information on the amount of diopter change to determine the diopter changing range for the first and second display optical systems. The video display device according to feature 3.

5. The diopter changing means uses the information on the amount of diopter change to determine the diopter changing speed for the first and second display optical systems. The video display device according to feature 3.

6. The first and second display optical systems display an image with parallax. The video display device according to any one of claims 1 to 5.

7. A control method performed on an image display device equipped with first and second display optical systems that display images to the user's left eye and right eye, respectively, A step of calculating the object position of the display object in the image by the first and second display optical systems, The process includes a step of changing the diopter of the first and second display optical systems based on the information of the object's position and the information of the amount of change in diopter in response to the user's change in object distance, In the process of changing the diopter, the diopter change speed when changing the diopter of the first and second display optical systems is determined using the information on the amount of change in the diopter. A method for controlling an image display device, characterized by the features described above.

8. The computer of the video display device is made to perform each of the steps described in claim 7. A program characterized by the following features.

Images