Linearly actuated display and implementation method thereof

Dynamically adjust the view projection of the projector array through the user's eye detection system, solving the problem of insufficient angle resolution of the automatic stereo display at a wide viewing angle, achieving smooth motion parallax and reducing the number of projectors, improving the performance and efficiency of the display .

CN110392248BActive Publication Date: 2024-11-22TEXAS INSTRUMENTS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910293328.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-02-28
Filing Date
2019-04-12
Publication Date
2024-11-22
Estimated Expiration
2039-04-12

AI Technical Summary

Technical Problem

Existing automatic stereo displays are difficult to provide sufficient angular resolution when providing a wide viewing angle, resulting in jitter when moving, and a large number of projectors are required to achieve smooth motion parallax.

Method used

Through the user eye detection system, dynamically adjust the projector projection view in the projector array, use image sensors and controllers to determine the user's eye position, select appropriate view projection, increase the angle resolution and reduce the number of projectors, and achieve smooth horizontal and vertical Motion parallax.

Benefits of technology

Increase angular resolution without increasing projectors, reduce the cost and volume of the display, provide a smooth motion parallax effect, and reduce the usage of the graphics processing unit.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110392248B_ABST
    Figure CN110392248B_ABST
Patent Text Reader

Abstract

The present application relates to a linearly actuated display and implementation methods thereof. An apparatus includes an image sensor (716), a linear actuator (706), a projector (710) coupled to the linear actuator (706), and a controller (702) coupled to the image sensor (716) and the linear actuator (706). The controller (702) is configured to determine the position of a user's eye based on image data from the image sensor (716). The controller (702) is further configured to activate the linear actuator (706) to drive the projector (710) to an orientation associated with projecting to the eye located at the position. The controller (702) is further configured to initiate projection of an image depicting a view of a three-dimensional scene via the projector (710), the image being selected based on the position.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 637,895 filed on March 2, 2018, U.S. Provisional Application No. 62 / 661,316 filed on April 23, 2018, U.S. Provisional Application No. 62 / 786,818 filed on December 31, 2018, and U.S. Provisional Application No. 62 / 786,830 filed on December 31, 2018, which are hereby incorporated by reference. Technical Field

[0003] The present invention relates to a display, and more particularly to a linearly actuated display and an implementation method thereof. Background Art

[0004] A stereoscopic display presents a different image of a scene to each of the user's eyes. This presentation of different images to each of the eyes is called stereoscopic vision. The different images correspond to views from slightly different angles to simulate binocular disparity, which would occur if the user were observing a scene in the natural world due to the horizontal separation of the eyes (parallax). Thus, stereoscopic vision creates an impression of depth. Stereoscopic displays typically use some type of glasses (active or passive) to switch the left and right eye images from the display to the appropriate eye.

[0005] In addition, autostereoscopic displays provide left-eye images and right-eye images without special glasses. This is usually done by forming a so-called multi-view display. Such displays use any of several technologies to produce multiple views, which may involve lenticular lenses, lens arrays, parallax barriers, or multiple projectors. In the case of multiple projectors, the number of views formed by the display is usually equal to the number of projectors. Therefore, if 10 projectors are used, the system can be regarded as a 10-view 3D display. However, when the user changes orientation relative to the display, 10 views may not be enough to provide smooth motion parallax.

[0006] If the autostereoscopic display is to have a wide viewing angle, enough views must be generated to provide a high angular resolution of the image. For example, in a display with a 30 degree viewing angle and 10 views, the angular resolution of each view is 3 degrees. This means that when the user moves relative to the screen, the image on the display may appear to have "judder", because the image from one view to the next view will have a large change in perspective angle. More views (and corresponding projectors) can be introduced to increase the angular resolution to eliminate "judder". Super multi-view displays provide the smoothest motion parallax effect and simulate true light field displays. These displays provide multiple views to each of the user's eyes to achieve an angular resolution of approximately 0.1 °. However, such displays may require more than 100 projectors to provide this excellent angular resolution and smooth parallax. Summary of the invention

[0007] Disclosed are systems and methods for increasing angular resolution in an autostereoscopic display based on user eye detection. In particular, the disclosed systems and methods can increase angular resolution without adding additional projectors. Furthermore, the disclosed systems and methods can project fewer views at a time compared to a system that projects views simultaneously from each of a larger array of projectors. Thus, the disclosed systems and methods can have reduced rendering complexity. Thus, the cost and size / weight of the display can be reduced because fewer projectors can be used and also because fewer or cheaper graphics processing units can be used.

[0008] In some examples, the display includes an array of projectors, a camera, and a controller. The array of projectors is arranged in a linear manner or in a grid manner. For example, the array of projectors may be arranged horizontally. The projector is configured to project into a viewing window in space, and the viewing window is sized and spaced so that the user's eyes are located in different viewing windows. The camera captures an image of the user's eyes and based on the image, the controller determines the position of the user's eyes in the viewing window associated with the projector in the array. The controller realizes stereoscopic vision by causing different images of the scene to be projected into different windows in which the user's eyes are located. In addition, the controller selects an image based on the position of the user's eyes in the viewing window. When the eyes move horizontally in the viewing window associated with the projector, the controller changes the image projected by the projector accordingly. Since the controller selects the image projected into the viewing window based on the position of the eyes in the viewing window, each viewing window of each projector is capable of displaying more than one view. Therefore, the display has an increased horizontal angular resolution without adding an additional projector. Therefore, when the user moves horizontally relative to the display, the display can provide smooth horizontal motion parallax. In addition, the controller can form a vertical motion parallax effect by selecting the projected images further based on the vertical position of the user's eyes within the viewing window. Thus, both a vertical parallax effect and a horizontal parallax effect can be formed by the controller using a row of projectors. In an alternative example, a vertical parallax effect can be generated at least in part by using a grid of projectors.

[0009] An apparatus includes a projector array, the projector array including a plurality of projectors. Each projector is configured to project to a corresponding viewing window. The apparatus further includes an image sensor and a controller. The controller is configured to determine a first position of a user's first eye within a first viewing window based on image data from the image sensor. The first viewing window corresponds to a first projector among a plurality of projectors. The controller is further configured to determine a second position of a user's second eye within a second viewing window based on the image data. The second viewing window corresponds to a second projector among the plurality of projectors. The controller is further configured to initiate projection of a first image depicting a first view of a three-dimensional scene via the first projector. The first image is selected based on the first position. The controller is further configured to initiate projection of a second image depicting a second view of a three-dimensional scene via a second projector. The second image is selected based on the second position.

[0010] A method includes determining a first position of a first eye of a user within a first viewing window based on image data from an image sensor. The first viewing window corresponds to a first projector of a plurality of projectors. The method further includes determining a second position of a second eye of the user within a second viewing window based on the image data. The second viewing window corresponds to a second projector of the plurality of projectors. The method further includes initiating projection of a first image depicting a first view of a three-dimensional scene via the first projector. The first image is selected based on the first position. The method further includes initiating projection of a second image depicting a second view of the three-dimensional scene via a second projector. The second image is selected based on the second position.

[0011] A computer-readable storage device stores instructions that are executable by one or more processors to determine a first position of a user's first eye within a first viewing window based on image data from an image sensor. The first viewing window corresponds to a first projector among a plurality of projectors. The instructions are further executable by the one or more processors to determine a second position of a user's second eye within a second viewing window based on the image data. The second viewing window corresponds to a second projector among the plurality of projectors. The instructions are further executable by the one or more processors to initiate projection of a first image depicting a first view of a three-dimensional scene via the first projector. The first image is selected based on the first position. The instructions are further executable by the one or more processors to initiate projection of a second image depicting a second view of a three-dimensional scene via a second projector. The second image is selected based on the second position.

[0012] In some examples, the display includes a linear actuator, a projector coupled to the linear actuator, a camera, and a controller. The linear actuator is configured to move the projector in a certain direction (e.g., horizontal direction). The camera captures an image of the user's eye and based on the image, the controller determines the position of the user's eye. Based on the position of the user's eye, the controller activates the linear actuator to move the projector to an orientation associated with a portion of the pupil of the eye projected at the position. The controller further selects an image based on the position and starts projecting the image via the projector. When the eye moves horizontally, the controller changes the projected image and drives the linear actuator accordingly. Therefore, the display has an increased horizontal angular resolution without adding an additional projector. Therefore, when the user moves horizontally relative to the display, the display can provide smooth horizontal motion parallax. In addition, the controller can form a vertical motion parallax effect by further selecting a projected image based on the vertical position of the user's eye. The viewing window can be extended by a diffuser screen in the vertical direction. Therefore, the user's eyes can move vertically within the viewing window without repositioning the corresponding projector.

[0013] An apparatus includes an image sensor, a linear actuator, a projector coupled to the linear actuator, and a controller coupled to the image sensor and the linear actuator. The controller is configured to determine the position of a user's eye based on image data from the image sensor. The controller is further configured to activate the linear actuator to drive the projector to an orientation associated with projecting to the eye at the position. The controller is further configured to initiate projection of an image depicting a view of a three-dimensional scene via the projector, the image being selected based on the position.

[0014] A method includes receiving image data from an image sensor. The method further includes determining a position of an eye of a user based on the image data. The method further includes activating a linear actuator to drive a projector to an orientation associated with projecting to the eye located at the position. The method further includes initiating projection of an image depicting a view of a three-dimensional scene via the projector, the image selected based on the position.

[0015] A computer readable storage device stores instructions executable by one or more processors to receive image data from an image sensor. The instructions are further executable by the one or more processors to determine a position of an eye of a user based on the image data. The instructions are further executable by the one or more processors to activate a linear actuator to drive a projector to an orientation associated with projecting onto the eye at the position. The instructions are further executable by the one or more processors to initiate projection of an image depicting a view of a three-dimensional scene via the projector, the image being selected based on the position. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To describe various examples in detail, reference will now be made to the accompanying drawings, in which:

[0017] Figure 1 A diagram showing a system for increasing angular resolution in an autostereoscopic multi-view display based on user eye detection.

[0018] Figure 2A Illustrated is a system for increasing angular resolution in an autostereoscopic multi-view display that projects a first view of a three-dimensional scene to an eye located at a first position.

[0019] Figure 2B Illustrated is a system for increasing angular resolution in an autostereoscopic multi-view display that projects a second view of a three-dimensional scene to an eye at a second position after the eye moves horizontally within a viewing window.

[0020] Figure 3A Illustrated is a system for increasing angular resolution in an autostereoscopic multi-view display that projects a first view of a three-dimensional scene to an eye located at a first position.

[0021] Figure 3B Illustrated is a system for increasing angular resolution in an autostereoscopic multi-view display that projects a second view of a three-dimensional scene to an eye at a second location after the eye moves vertically within a viewing window.

[0022] Figure 4 Illustrated is a system for increasing angular resolution in an autostereoscopic multi-view display, causing two projectors to project the same image to an eye in response to the eye being located in viewing windows associated with both projectors.

[0023] Figure 5A is a first diagram showing the control of the summed intensity of light projected into the various viewing windows when the eyes are located in the two viewing windows.

[0024] Figure 5B is a second diagram showing control of the sum intensity of light projected to the various viewing windows when the eye is located in one of the viewing windows. The first and second diagrams illustrate that the eye experiences a consistent light intensity when the eye moves from an area where the viewing windows overlap to an area where the viewing windows do not overlap.

[0025] Figure 6 is a flow chart of a method for increasing angular resolution in an autostereoscopic multi-view display.

[0026] Figure 7 An additional system for increasing angular resolution in an autostereoscopic multi-view display is illustrated, the additional system including an actuator to move the projector.

[0027] Fig. 8A The diagram illustrates an example of an additional system further comprising three projectors per eye.

[0028] Figure 8B The diagram illustrates that the eye simultaneously receives three different images depicting different views of a three-dimensional scene.

[0029] Fig. 9 Illustrated is an example of an additional system in which a single projector projects three views to each eye simultaneously.

[0030] Fig.10 The diagram illustrates an example of an additional system in which additional actuators are included to drive the projector in two dimensions.

[0031] Fig.11 is a flow chart of another method for increasing angular resolution in an autostereoscopic multi-view display.

[0032] Fig.12 is a block diagram of a computer system that can perform the functions described herein. DETAILED DESCRIPTION

[0033] refer to Figure 1 , a diagram showing a system 100 for increasing angular resolution in an autostereoscopic multi-view display based on user eye detection is shown. The system 100 includes a controller 102, a projector array 104, an image sensor 106, and a screen assembly 108. The system 100 may include more components than illustrated. For example, the system 100 may include one or more speakers, one or more communication interfaces, an optical media drive, etc.

[0034] Controller 102 includes one or more processing units. Examples of processing units include central processing units (CPUs), graphics processing units (GPUs), etc. In some implementations, controller 102 includes a GPU for each projector in projector array 104. Although not illustrated, system 100 may further include one or more computer-readable storage devices (e.g., computer memories) storing instructions that can be executed by controller 102 to perform one or more of the functions described herein. As used herein, a computer-readable storage device is a manufactured object and is not a signal. In some implementations, controller 102 includes dedicated hardware configured to perform one or more of the functions described herein, such as a field programmable gate array. Image sensor 106 may correspond to a camera or other type of imaging device.

[0035] In the illustrated example, the projector array 104 includes a first projector 104a, a second projector 104b, a third projector 104c, a fourth projector 104d, a fifth projector 104e, and a sixth projector 104f. Alternative examples include more or fewer projectors. The projectors 104a-104f may correspond to a digital light processing projectors (DLP is a registered trademark of Texas Instruments Inc. of Dallas, Texas), Liquid Crystal on Silicon (LCoS), 104a to 104f are arranged horizontally, but other orientations are possible. The screen assembly 108 may include a diffuser screen, one or more Fresnel lenses, a holographic screen, or a combination thereof. For example, the screen assembly 108 may include a diffuser screen placed between two Fresnel lenses. In another example, the screen assembly 108 does not include a diffuser screen. The screen assembly is configured to direct light from each of the array of projectors 104a to 104f to a corresponding one of a plurality of viewing windows 112 within the viewing zone 110. The plurality of viewing windows 112 include a first viewing window 112a, a second viewing window 112b, a third viewing window 112c, a fourth viewing window 112d, a fifth viewing window 112e, and a sixth viewing window 112f. The screen assembly 108 is configured to direct light from the first projector 104a to the sixth viewing window 112f, direct light from the second projector 104b to the fifth viewing window 112e, direct light from the third projector 104c to the fourth viewing window 112d, direct light from the fourth projector 104d to the third viewing window 112c, direct light from the fifth projector 104e to the second viewing window 112b, and direct light from the sixth projector 104f to the first viewing window 112a. In the illustrated example, the screen assembly 108 is shown as directing light 107 from the second projector 104b to the fifth viewing window 112e. It should be noted that although the viewing windows 112a - 112f are depicted with gaps therebetween for ease of illustration, in many implementations, some of the viewing windows 112a - 112f overlap one another.

[0036] In the illustrated example, projectors 104a to 104f are at a distance s1 from the diffuser screen, and viewing zone 110 is at a distance s2 from screen assembly 108 and has a width w. Each of viewing windows 112a to 112f has a width D'. D', w, and the position of viewing windows 112a to 112f are based on s1 and s2. The manufacturer of system 100 can set s1. In some implementations, image sensor 106 corresponds to a stereo camera and controller 102 is configured to calculate the distance from system 100 to the user or system 100 further includes a separate depth sensor, which is configured to generate depth data indicating the distance from system 100 to the user. Controller 102 can then calculate s2 based on the depth data and then calculate w, D', and the position of viewing windows 112a to 112f within viewing zone 110 based on s2. In other implementations, s2, w, D', the positions of viewing windows 112a-f, or a combination thereof are hard coded.

[0037] In operation, the image sensor 106 generates image data depicting a scene in front of the diffuser screen 108. When a user is using the system 100, the image data depicts the user. The controller 102 receives the image data from the image sensor 106 and identifies the position of a first eye of the user located within one of the viewing windows 112a to 112f. The controller 102 identifies a first position of a first eye (e.g., a left eye) of the user located within one of the viewing windows 112a to 112f and identifies a second position of a second eye (e.g., a right eye) of the user located within one of the plurality of viewing windows 112a to 112f. The projector array 104 is arranged so that when the user's eyes are aligned generally horizontally (or otherwise aligned with the projector array 104), the width D' of each of the plurality of viewing windows 112a to 112f prevents the first eye and the second eye from being located in the same viewing window. The controller 106 selects a first image depicting a first view of the three-dimensional scene based on the first position and initiates projection of the first image via a projector in the projector array 104 that corresponds to a viewing window in the plurality of viewing windows 112 that includes the first position. Similarly, the controller 106 selects a second image depicting a second view of the three-dimensional scene based on the second position and initiates projection of the second image via a projector in the projector array 104 that corresponds to a viewing window in the plurality of viewing windows 112 that includes the second position. The first view and the second view of the three-dimensional scene are from slightly different viewing angles. Therefore, the user's eyes will each receive a different view of the three-dimensional scene, thereby forming a stereoscopic effect. In addition, since the controller 102 selects images based on the position of the eyes within the plurality of viewing windows 112, different views may be projected to the user's eyes when the eyes move through a particular one of the plurality of viewing windows 112, such as Figure 2A to 2B as well as Figure 3A and 3B As illustrated in . Thus, the angular resolution can be increased without adding projectors.

[0038] Figure 2A to 2B The diagram illustrates an example of a system 100 that changes the image projected by a projector as the eye moves horizontally within a viewing window. Specifically, Figure 2A to 2B It is shown that when the user's eye 206 moves horizontally across the fifth viewing window 112e, the controller 102 causes the projector to project a different image. Figure 1 As shown in FIG. 1 , the second projector 104b projects onto the fifth viewing window 112e. Figure 2A to 2B, the controller 102 determines based on image data from the camera 106 that the eye 206 is located at a first position 208 within the fifth viewing window 112e at a first time. Based on the first position 208 within the fifth viewing window 112e, the controller 102 initiates projection of the first image 204 via the second projector 104b. The controller 102 then determines based on the image data from the camera 106 that the eye 206 has moved to a second position 210 within the fifth viewing window 112e at a second time. Based on the second position 210 within the fifth viewing window 112e, the controller 102 initiates projection of the second image 206 via the second projector 104b. The first image 204 and the second image 206 correspond to different views of the same three-dimensional scene. Thus, each projector of the system 100 can project more than one view of the three-dimensional scene. Thus, the system 100 supports an increased number of horizontal views (e.g., horizontal angular resolution) without increasing the projector count. Thus, compared to another system with an equivalent number of projectors that does not track the user's eyes, the system 100 creates a relatively smooth horizontal parallax effect when the user moves horizontally across the plurality of viewing windows 112. It should be noted that in instances where the projector array 104 is arranged along an axis that is not horizontal, the system 100 increases the angular resolution along the axis in the same manner. Furthermore, the system 100 provides additional benefits, as described below.

[0039] Figures 3A to 3B The diagram illustrates an example of a system 100 that changes the image projected by a projector as the eye moves vertically within a viewing window. Specifically, Figures 3A to 3B It is shown that when the user's eye 206 moves vertically across the fifth viewing window 112e, the controller 102 causes the projector to project a different image. Figures 3A to 3BIn the example shown in , the controller 102 determines that the eye 206 is located at a first position 402 within the fifth viewing window 112e at a first time based on image data from the camera 106. Based on the first position 402, the controller 102 starts to project a first image 406 via the second projector 104b. The controller 102 then determines that the eye 206 has moved to a second position 404 within the fifth viewing window 112e at a second time based on the image data from the camera 106. Based on the second position 404, the controller 102 starts to project a second image 408 via the second projector 104b. The first image 406 and the second image 408 correspond to different views of the same three-dimensional scene. Therefore, each projector of the system 100 can project more than one view of the three-dimensional scene in the vertical direction. Therefore, the system 100 supports more than one view in the vertical direction (e.g., vertical angular resolution) without increasing the projector count. Therefore, compared to a system that employs one projector per view, the system 100 can use fewer projectors to form a vertical parallax effect when the user moves vertically through multiple viewing windows 112. Furthermore, in the case where the user's eyes move both horizontally and vertically, the controller 102 can simultaneously create a horizontal parallax effect and a vertical parallax effect. It should be noted that in the example where the projector array 104 is arranged along an axis that is not horizontal, the system 100 increases the angular resolution along a second axis perpendicular to the axis in the same manner.

[0040] refer to Figure 4 , showing a use case where the eye is in two viewing windows at the same time. As explained above, some of the plurality of viewing windows 112 may overlap. A system in which viewing windows overlap can present a continuous view of a scene by avoiding gaps in the viewing windows. However, if the eye receives conflicting images from the two projectors, image distortion (e.g., blurring or ghosting) may occur. Figure 4 In the illustrated example of FIG. 1 , the fifth viewing window 112e overlaps the sixth viewing window 112f and the user's eye 206 is located at a position 302 that is within both the fifth viewing window and the sixth viewing window. The controller 102 initiates projection of the image 306 via both the first projector 104a and the second projector 104b so that the eye 206 receives the same image 306 from both projectors 104a, 104b rather than two different images. Thus, the system 100 can present a clearer view than a system in which each projector projects a different image.

[0041] In addition to or in lieu of reducing blurring effects, system 100 may also manage the intensity of light projected by projector array 104 so that a user's eye receives the same (or approximately the same) light intensity when the eye moves from an area within a single viewing window of the plurality of viewing windows 112 to an area within more than one viewing window of the plurality of viewing windows 112. Figure 5A and 5B , shows a diagram illustrating the total light intensity projected to the viewing window. In some examples, controller 102 is configured to adjust the intensity of light projected by projector array 104 based on the position of eye 206 as eye 206 moves through viewing zone 110. Figure 5A and 5B In the example illustrated in , controller 102 adjusts the projector so that eye 206 sees a constant intensity of x lumens. Figure 5A The eye 206 is depicted at a position within the first viewing window 112a and the second viewing window 112b. Therefore, the controller 102 adjusts the fifth projector 104e and the sixth projector 104f to project x / 2 lumens to the second viewing window 112b and the first viewing window 112a, respectively, so that the sum of the intensities received by the eye 206 is x lumens. It should be noted that the intensity distribution of the projector array 104 can be different from Figure 5A . For example, the distribution may be Gaussian rather than flat or "square". Controller 102 may take into account the intensity distribution of projector array 104 when calculating the summed intensity of the projectors at a point in space. Figure 5A It is further depicted that the controller 102 may disable the projector or otherwise cause the projector to stop projecting in response to determining that no eyes are located in the corresponding viewing window. This may reduce the power consumed by the system 100 and reduce the demand on rendering components. Thus, the system 100 may utilize simpler (and potentially cheaper) hardware and consume less power than other systems.

[0042] Figure 5B The eye 206 is depicted only at a position within the second viewing window 112b. Therefore, the controller adjusts the fifth projector 104e to project x lumens into the second viewing window 112b so that the eye 206 also receives x lumens at this position.

[0043] therefore, Figure 5A and 5BThe depiction system 100 can monitor the orientation of the eye and adjust one or more projectors so that the eye experiences a constant light intensity as the user moves horizontally across the viewing window. Thus, the system 100 can produce images that reduce eye fatigue and present a more consistent image than other systems. It should be noted that the various operations described herein performed by the system 100 with respect to one eye can be performed with respect to both eyes of the user. Thus, while illustrated and described with respect to one eye for ease of explanation, a combination of operations can be performed by the system 100 for both eyes of the user. Figure 2A , 2B , 3A, 3B, 4, 5A and 5B described techniques.

[0044] refer to Figure 6 , a flow chart illustrating a method 600 for position-based projection of a user's eye is shown. The method 600 may be performed by the controller 102 of the system 100. The method 600 includes, at 602, determining a first position of a first eye of a user within a first viewing window based on image data from an image sensor, the first viewing window corresponding to a first projector of a plurality of projectors. For example, the controller 102 may determine that the user's left eye is located at a first particular position within the first viewing window 112a. As explained above, the first viewing window 112a corresponds to the sixth projector 104f.

[0045] The method 600 further includes determining, based on the image data, a second position of a second eye of the user within a second viewing window corresponding to a second projector of the plurality of projectors at 604. For example, the controller 102 may determine, based on the image data, that the right eye of the user is located at a second particular position within a fourth viewing window 112d. As explained above, the fourth viewing window 112d corresponds to the third projector 104c.

[0046] The method 600 further includes initiating projection of a first image depicting a first view of a three-dimensional scene via a first projector, the first image being selected based on a first position, at 606. For example, the controller 102 may select the first image of the three-dimensional scene based on a first particular position of a user's left eye in the first viewing window 112a and initiate projection of the first image from the sixth projector 104f.

[0047] The method 600 further includes initiating projection of a second image depicting a second view of the three-dimensional scene via a second projector, the second image being selected based on the second position, at 608. For example, the controller 102 may select the second image of the three-dimensional scene based on the second particular position of the user's right eye in the fourth viewing window 112d and initiate projection of the second image from the third projector 104c.

[0048] The method 600 further includes, at 610, in response to detecting that the first eye has moved to a third position within the first viewing window, initiating projection of a third image selected based on the third position via the first projector. For example, the controller 102 may detect that the user's left eye is located at a new particular position within the first viewing window 112a based on image data from the image sensor 106. Thus, the controller 102 may select a new image for projection by the sixth projector 104f based on the new position within the first viewing window 112a. This may induce a horizontal parallax effect, a vertical parallax effect, or both without requiring the eye to move to a viewing window associated with a different projector.

[0049] The method 600 further includes, at 612, in response to detecting that the second eye moves to a fourth position within the second viewing window, initiating projection of a fourth image selected based on the fourth position via the second projector. For example, the controller 102 may detect that the user's right eye is located at a new particular position within the fourth viewing window 112d based on image data from the image sensor 106. Thus, the controller 102 may select a new image for projection by the third projector 104c based on the new position within the fourth viewing window 112d. This may induce a horizontal parallax effect, a vertical parallax effect, or both without requiring the eye to move to a viewing window associated with a different projector.

[0050] Because the image is selected based on the position of the eye, the method 600 may provide increased angular resolution and / or reduced projector count compared to systems in which each projector projects a specified view.

[0051] refer to Figure 7 , a diagram showing another system 700 for increasing angular resolution in an autostereoscopic multi-view display based on user eye detection is shown. System 700 includes a controller 702, a linear actuator 706, a first projector 710, a second projector 712, a screen assembly 714, and an image sensor 716. Some examples of system 700 include more components, such as one or more speakers, a depth sensor, a housing, one or more communication interfaces, additional projectors, etc. Screen assembly 714 can include a diffuser screen, one or more Fresnel lenses, a holographic optical element screen, or a combination thereof.

[0052] Controller 702 includes one or more processing units. In some implementations, controller 702 includes a GPU for each projector in projectors 710, 712. Although not illustrated, system 700 further includes one or more computer-readable storage devices (e.g., computer memory) storing instructions that can be executed by controller 702 to perform one or more of the functions described herein. As used herein, a computer-readable storage device is a manufactured object and not a signal. In some implementations, controller 702 includes dedicated hardware configured to perform one or more of the functions described herein, such as a field programmable gate array. Image sensor 716 may correspond to a camera or other type of imaging device.

[0053] exist Figure 7 In the example of , linear actuator 706 includes motor 708 and screw 709, but linear actuator 706 can be replaced with other types of linear actuators. Projectors 710, 712 are arranged on an axis and attached to linear actuator 706. In some examples, the axis is horizontal. Linear actuator 706 is configured to move projectors 710, 712 along the axis in response to a command from controller 702. Projectors 710, 712 project images to screen assembly 714. Screen assembly 714 is configured to focus light received from the projectors to a specific area. Specifically, screen assembly 714 guides light from each projector to an area approximately as wide as a human pupil and diffuses light from projectors 710, 712 in a vertical direction, so that the viewing window receiving images from projectors 710, 712 expands in a vertical direction. Due to the width and position of the image as output by the screen assembly 714, each of the user's first eye 718 and second eye 720 can receive images from a different one of the projectors 710, 712. Thus, the system 700 can present a different view of the three-dimensional scene to each of the eyes 718, 720.

[0054] In operation, the controller 702 determines the position of the first eye 718 based on image data from the image sensor 716 and activates the linear actuator 706 to drive one of the projectors 710, 712 to an orientation associated with projecting to the first eye 718. When the user moves parallel to the axis of the linear actuator, the controller 702 identifies the updated position of the first eye 718 and activates the linear actuator 706 to drive one of the projectors 710, 712 to an orientation associated with projecting to the updated position. Aligning the first eye 718 with one of the projectors 710, 712 may result in aligning the second eye 720 with the other of the projectors 710, 712. In some implementations, the system 700 further includes an additional actuator configured to adjust the spacing between the projectors 710, 712 along the linear actuator 706 based on the detected interpupillary distance of the user. For example, the controller 702 may detect the interpupillary distance of the user based on image data from the image sensor 716 during the calibration phase. Although the first eye 718 is illustrated as the left eye, the controller 702 may drive the actuator based on the position of the right eye of the user. Alternatively, the controller 702 may activate the actuator to drive the projectors 710, 712 based on the positions of both the left eye and the right eye. Thus, the projectors 710, 712 are moved by the controller 702 so that one of the projectors 710, 712 projects to one of the eyes 718, 720, while the other projector 710, 712 projects to the other eye 718, 720. Thus, the eyes 718, 720 may receive images corresponding to different views of the three-dimensional scene and a stereoscopic effect may be achieved.

[0055] In addition, the controller 702 selects an image of the three-dimensional scene to project based on the position of the eyes 718, 720. For example, when the first eye 718 moves horizontally, the controller 702 may cause the second projector 712 to project an image corresponding to a view of the three-dimensional scene from different perspectives that change horizontally. In addition, when the first eye 718 moves vertically, the controller 702 may cause the second projector 712 to project an image corresponding to a view of the three-dimensional scene from different perspectives that change vertically. Therefore, both horizontal and vertical parallax effects can be achieved by the system 700. In the illustrated example, the first camera 710 projects a first image 722 to the second eye 720, and the second camera 712 projects a second image 724 to the first eye 718. The first image 722 corresponds to a first view of the three-dimensional scene and the second image 724 corresponds to a second view of the three-dimensional scene from a slightly different perspective. As explained above, the controller 702 selects the images 722 , 724 based on the position of the eyes 718 , 720 , as determined based on the image data from the image sensor 702 .

[0056] Thus, system 700 corresponds to an autostereoscopic display that supports both vertical and horizontal parallax effects and uses fewer projectors (e.g., 1) per eye compared to systems that rely on an array of fixed projectors to produce a stereoscopic effect display with motion parallax.

[0057] The human eye focuses on an object through a process called accommodation and vergence. The human eye includes a lens and accommodation refers to the eye adjusting the optical power of the eye to focus on a point by changing the shape of the lens. Accommodation to a point causes light rays from that point to converge on the back of the eye. Vergence refers to the movement of the two eyes to a point, a common point. Vergence and accommodation are usually linked together by accommodation reflex. Therefore, human eyes usually accommodate and converge to a common point. However, accommodation and vergence can be separate. For example, in a stereoscopic display system, different slightly changing images are presented to the user's eyes to produce a three-dimensional effect. However, when the user focuses on an object displayed by the stereoscopic display system, the user's eyes converge to the point associated with the object on the display screen while trying to adjust to the point in space where the object appears to exist, and vice versa. Therefore, the light rays associated with the point to which the user's eyes converge may not converge on the back of the user's eyes. This separation of accommodation and steering is referred to as steering accommodation conflict and can cause discomfort to some users of stereoscopic displays. However, providing each eye with more than one different view (e.g., 3 or more views) of a three-dimensional scene that arrives at the eye at a particular angle can reduce or resolve steering accommodation conflict by causing the user's eyes to converge and accommodate to a common point. Therefore, an autostereoscopic display system that controls the angle at which the user's eyes receive different images of a scene can reduce or resolve steering accommodation conflict. In particular, providing multiple images to the user's eyes at a particular angle can cause the user's eyes to accommodate to infinity. Therefore, no matter which point in space the user's eyes are turned to, that point is in focus.

[0058] refer to Fig. 8A, showing an example 900 of the system 700 configured to use more than one projector per eye to compensate for steering adjustment conflicts. In addition to the first projector 710 and the second projector 712, the example 900 also includes a third projector 920, a fourth projector 922, a fifth projector 924, and a sixth projector 926. The controller 702 is configured to activate the linear actuator 706 to drive the second projector 712 to a first position associated with projecting to a first portion of the pupil of the first eye 718 located at the location of the first eye 718, drive the sixth projector 926 to a second position associated with projecting to a second portion of the pupil of the first eye 718 located at the location of the first eye 718, and drive the fifth projector 924 to a third position associated with projecting to a third portion of the pupil of the first eye 718 located at the location of the first eye 718. Similarly, the linear actuator 706 drives the first projector 710 to a fourth orientation associated with projecting onto a first portion of the second eye 720 located at the location of the second eye 720, drives the third projector 920 to a fifth orientation associated with projecting onto a second portion of the second eye 720 located at the location of the second eye 720, and drives the fourth projector 922 to a sixth orientation associated with projecting onto a third portion of the second eye 720. As illustrated, the second projector 712 projects the second image 724 onto a first portion of the pupil of the first eye 718, the sixth projector 926 projects the third image 938 onto a second portion of the pupil of the first eye 718, and the fifth projector 924 projects the fourth image 936 onto a third portion of the pupil of the first eye 718. Similarly, the first projector 710 projects the first image 722 onto a first portion of the pupil of the second eye 720, the third projector 920 projects the fifth image 932 onto a second portion of the pupil of the second eye 720, and the fourth projector 922 projects the sixth image 934 onto a third portion of the pupil of the second eye 720. The controller 702 selects the images 722, 724, 932, 934, 936, 938 based on the positions of the eyes 718, 720. The difference between the images 724, 936, 938 received by the first eye 718 and the images 722, 932, 934 received by the second eye 720 can reduce steering adjustment conflicts.

[0059] Figure 8B The diagram illustrates a close-up view of an eye receiving three different images. Specifically, Figure 8BA first portion of the pupil 982 of the first eye 718 is shown receiving the second image 724, a second portion of the pupil 982 receiving the third image 938, and a third portion of the pupil 982 receiving the fourth image 936. Each of the images as received at the pupil 982 may have a width between .67 millimeters and 2.67 millimeters to accommodate human eye pupils ranging in width from 2 mm to 8 mm. In some examples, the width is 1.3 millimeters (e.g., 1 / 3 of the average human eye pupil width of 4 mm).

[0060] refer to Fig. 9 , shows an example 1000 of system 700 configured to use one projector per eye to compensate for steering adjustment conflicts. Fig. 9 , the first projector 710 may project each of the first image 722, the third image 932, and the fourth image 934 and the second projector 712 may project each of the second image 724, the fifth image 936, and the sixth image 938. Thus, the system 700 may use one projector per eye or multiple projectors per eye to resolve steering accommodation conflicts.

[0061] refer to Fig.10 , shows an example 1100 of the system 700, which is configured to drive the projector along two axes. In example 1100, the linear actuator 706 is mounted to the second linear actuator 1102. In the illustrated example, the second linear actuator 1102 includes a second motor 1104 and a second screw 1106, but other forms of linear actuators can be used. The linear actuator 706 is configured to drive the projectors 710, 712 along a first axis (e.g., a horizontal axis), and the second linear actuator 1102 is configured to drive the linear actuator 706 along a second axis (e.g., a vertical axis). Therefore, the linear actuators 706, 1102 are configured to drive the projectors 710, 712 in two dimensions. The controller 702 is configured to activate the linear actuators 706, 1102 to drive the projectors 710, 712 to an orientation associated with the position of the projected eye to the user, as described above.

[0062] refer to Fig.11 , another method 1200 for increasing angular resolution in an autostereoscopic multi-view display based on user eye detection is shown. The method 1200 may be performed by the system 700. The method 1200 includes, at 1202, receiving image data from an image sensor. For example, the controller 702 may receive the image data from the image sensor 716.

[0063] The method 1200 further includes determining the position of the user's eyes based on the image data, at 1204. For example, based on the image data, the controller 702 may determine the position of the user's first eye 718.

[0064] The method 1200 further includes activating the linear actuator to drive the projector to an orientation associated with projecting to the eye at the location, at 1206. For example, the controller 702 may activate the linear actuator 706 to drive the second projector 712 to an orientation where light from the projector is directed by the diffuser screen 714 to the location of the first eye 718.

[0065] The method 1200 further includes initiating projection of an image depicting a view of the three-dimensional scene via a projector, the image being selected based on the position, at 1208. For example, the controller 702 may initiate projection of the second image 724 via the second projector 712 to the position of the first eye 718. As explained above, the second image 724 is selected by the controller 702 based on the position of the first eye 718.

[0066] The method 1200 further includes, at 1210, initiating projection of another image depicting a different view of the three-dimensional scene via another projector offset from the projector, the other image based on the position of a second eye of the user. For example, the controller 702 may initiate projection of the first image 722 to the position of the second eye 720 via the first projector 710. As explained above, the first image 722 is selected by the controller 702 based on the position of the second eye 720. By initiating projection of different images depicting different views of the scene to the eyes 718, 720, the controller 702 may create a stereoscopic effect.

[0067] Compared to other systems, method 1200 can provide increased angular resolution because method 1200 determines which view of a three-dimensional scene a projector projects based on the position of the user's eyes rather than causing the projector to project a fixed view. In addition, because method 1200 causes the projector to move to an orientation associated with the position of the projection to the eyes, method 1200 can use fewer projectors than methods that rely on a fixed array of projectors. In addition, moving the projector allows the method to direct the image to the user's eyes without the use of a diffuser screen. Therefore, the depth of focus of a display implementing method 1200 can be longer and the image will remain in focus at a variety of turning points compared to other displays.

[0068] refer to Fig.12, a block diagram of a computer system 1300 that can provide increased angular resolution in an autostereoscopic multi-view display is shown. The computer system 1300 includes a computing device 1302. The computing device 1302 may correspond to the controller 102 or the controller 702. The computing device includes one or more processors 1304 and one or more computer-readable storage devices 1306. The one or more processors 1304 may include one or more CPUs, one or more GPUs, one or more other processors, or a combination thereof. The one or more computer-readable storage devices 1306 may include one or more read-only memory (ROM) devices, one or more random access memory (RAM) devices, one or more disk drive devices, one or more other types of memory devices, or a combination thereof. The one or more computer-readable storage devices 1306 store projector system instructions 1308, which can be executed by the one or more processors 1304 to perform one or more of the functions described herein.

[0069] Computer system 1300 further includes image sensor 1310. Image sensor 1310 corresponds to a camera or another type of imaging device. For example, image sensor 1310 may correspond to image sensor 106 or image sensor 716. Computer system 1300 further includes projector system 1312. Projector system 1312 includes one or more projectors, one or more actuators, or a combination thereof. For example, projector system 1312 may correspond to projector array 104, projectors 710, 712, actuator 706, projectors 920, 922, 924, 926, second actuator 1102, or a combination thereof.

[0070] The one or more processors 1304 executing instructions 1308 are configured to perform the operations described herein. For example, the one or more processors 1304 may receive image data from the image sensor 1310 and determine the location of the user's eyes. Based on the location, the one or more processors 1304 select one or more images to go to the eyes and control the projector system 1312 accordingly. In addition, the one or more processors 1304 may control one or more actuators of the projector system 1312 to drive the one or more projectors into an orientation associated with the location of the projected eyes of the user.

[0071] It should be noted that the described systems and methods and examples thereof may be combined in whole or in part.To illustrate, the example 900 of the system 700 may include a second linear actuator 1102 .

[0072] In this description, the term "couple" or "couples" means an indirect or direct wired or wireless connection. Thus, if a first device is coupled to a second device, the connection may be through a direct connection or through an indirect connection via other devices and connections. Furthermore, in some implementations, devices illustrated as being directly coupled may be indirectly coupled. The statement "based on" means "based at least in part on." Thus, if X is based on Y, X may be dependent on Y and any number of other factors.

[0073] Modifications are possible in the described embodiments, and other embodiments are possible within the scope of the claims.

Claims

1. A device for a linearly actuated display, the device include: an image sensor configured to provide image data; First Projector; Second projector; Third projector; an actuator coupled to the first projector, the second projector, and the third projector; a controller coupled to the image sensor, the actuator, the first projector, the second projector, and the third projector, the controller being configured to: receiving the image data from the image sensor; determining a first eye position based on the image data; instructing the actuator based on the first eye position to drive the first projector to a first orientation associated with projecting onto a first portion of a pupil of a first eye located at the first eye position, to drive the second projector to a second orientation associated with projecting onto a second portion of the pupil of the first eye located at the first eye position, and to drive the third projector to a third orientation associated with projecting onto a third portion of the pupil of the first eye located at the first eye position; Based on the first eye position: instructing the first projector to project a first view of a three-dimensional scene; instructing the second projector to project a second view of the three-dimensional scene; and instructing the third projector to project a third view of the three-dimensional scene; as well as The first, second, and third projectors are adjusted based on the first eye position so that the first eye sees a constant light intensity as a user moves horizontally across a viewing window. 2 . The apparatus of claim 1 , wherein the controller is configured to select the first view, the second view, and the third view based on the first eye position.

3. The apparatus of claim 1, wherein the controller is configured to: determining an updated position of the first eye based on the image data; and Based on the updated position of the first eye, the actuator is controlled. 4 . The apparatus of claim 3 , wherein the controller is configured to select the first view, the second view, and the third view based on the determined first eye position to induce a motion parallax effect.

5. The apparatus of claim 2, further comprising a fourth projector coupled to the actuator and the controller, and the controller is further configured to: determining a second eye position based on the image data; instructing the actuator based on the second eye position to drive the fourth projector to a fourth orientation associated with projecting onto a portion of a pupil of a second eye located at the second eye position; as well as The fourth projector is instructed to project a fourth view of the three-dimensional scene based on the second eye position.

6. The apparatus of claim 1, further comprising a screen assembly and a viewing zone, wherein the screen assembly is configured to direct the first view from the first projector to a particular viewing zone in the viewing zone.

7. The apparatus of claim 6, wherein the controller is configured to: determining whether the first eye position is within the particular viewing zone of the viewing zone; and The screen assembly is instructed based on the determination to direct a view to be projected to the first eye to the particular one of the viewing zones.

8. The apparatus of claim 6, wherein the screen assembly comprises a Fresnel lens, a diffuser screen, a holographic optical element, or a combination thereof.

9. The apparatus of claim 1, wherein the actuator comprises a first linear actuator and a second linear actuator, the first linear actuator being configured to adjust the first orientation along a first axis, and the second linear actuator being configured to adjust the first orientation along a second axis, the second axis being perpendicular to the first axis.

10. The apparatus of claim 1, wherein the actuator comprises a motor and a screw.

11. The apparatus of claim 1, wherein the first projector is configured to project the first view to a viewing window, the viewing window having a width between .67 mm and 2.67 mm.

12. A method for a linearly actuated display, the method include: receiving, by the controller, image data from the image sensor; determining, by the controller, a first eye position based on the image data; instructing, by the controller, an actuator based on the first eye position to drive a first projector to a first orientation associated with projecting onto a first portion of a pupil of a first eye located at the first eye position, to drive a second projector to a second orientation associated with projecting onto a second portion of the pupil of the first eye located at the first eye position, and to drive a third projector to a third orientation associated with projecting onto a third portion of the pupil of the first eye located at the first eye position; instructing, by the controller based on the first eye position, the first projector to project a first image depicting a first view of a three-dimensional scene, the first image being selected based on the first eye position; instructing, by the controller based on the first eye position, the second projector to project a second image depicting a second view of the three-dimensional scene; instructing, by the controller based on the first eye position, the third projector to project a third image depicting a third view of the three-dimensional scene; as well as The first, second, and third projectors are adjusted based on the first eye position so that the first eye sees a constant light intensity as a user moves horizontally across a viewing window.

13. The method according to claim 12, further comprising: include: determining, by the controller, a second eye position based on the image data; instructing, by the controller based on the second eye position, the actuator to drive a fourth projector to a fourth orientation associated with projecting onto a portion of a pupil of a second eye located at the second eye position; as well as The fourth projector is instructing, by the controller, to project a fourth view of the three-dimensional scene based on the second eye position.

14. A non-transitory computer-readable storage medium storing instructions executable by one or more processors to: receiving image data from an image sensor; determining a first eye position based on the image data; instructing an actuator based on the first eye position to drive a first projector to a first orientation associated with projecting onto a first portion of a pupil of a first eye located at the first eye position, to drive a second projector to a second orientation associated with projecting onto a second portion of the pupil of the first eye located at the first eye position, and to drive a third projector to a third orientation associated with projecting onto a third portion of the pupil of the first eye located at the first eye position; instructing the first projector to project a first image depicting a first view of a three-dimensional scene based on the first eye position, the first image being selected based on the first eye position; instructing the second projector to project a second image depicting a second view of the three-dimensional scene based on the first eye position; instructing the third projector to project a third image depicting a third view of the three-dimensional scene based on the first eye position; as well as The first, second, and third projectors are adjusted based on the first eye position so that the first eye sees a constant light intensity as a user moves horizontally across a viewing window.

15. The non-transitory computer readable storage medium of claim 14, wherein the instructions are further executable by the one or more processors to: determining a second eye position based on the image data; instructing the actuator based on the second eye position to drive a fourth projector to a fourth orientation associated with projecting onto a portion of a pupil of a second eye located at the second eye position; as well as The fourth projector is instructed to project a fourth view of the three-dimensional scene based on the second eye position.