Simulating mirrors with lenticular displays

Abstract

The invention relates to simulating a mirror with a lenticular display. In one implementation, a method of operating a lenticular display is performed by a device including a processor, a non-transitory memory, and a lenticular display. The method includes capturing a first image of a user from a first camera location in a physical environment. The method includes capturing a second image of the user from a second camera location in the physical environment. The method includes transforming the first image and the second image by horizontally flipping the first image and the second image. The method includes displaying a transformed first image on the lenticular display at a display angle corresponding to a first eye position of the user and a transformed second image at a display angle corresponding to a second eye position of the user.

Description

Technical Field

[0001] The present disclosure relates generally to lenticular displays, and in particular to systems, methods, and apparatus for simulating mirrors using lenticular displays. Background Art

[0002] The lenticular display can display different content at different angles. For example, in some embodiments, when the lenticular display is viewed from a first horizontal angle, the object is seen from a first viewing angle, and when the lenticular display is viewed from a second horizontal angle, the object is seen from a second viewing angle different from the first viewing angle. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] So that the present disclosure can be understood by those of ordinary skill in the art, a more detailed description may be obtained with reference to aspects of some exemplary implementations, some of which are illustrated in the accompanying drawings.

[0004] Figures 1A to 1C An operating environment including a lenticular display according to some implementations is shown.

[0005] FIG. 2A to FIG. 2C shows that according to some specific implementations, after the user has moved Figures 1A to 1C operating environment.

[0006] Figure 3 is a flowchart representation of a method of operating a lenticular display according to some specific implementations.

[0007] Figure 4 It is implemented according to some specific Figure 1A A block diagram of an example of a device.

[0008] As a matter of practice, the various features shown in the drawings may not be drawn to scale. Therefore, for clarity, the sizes of the various features may be arbitrarily enlarged or reduced. In addition, some drawings may not depict all components of a given system, method, or device. Finally, throughout the specification and drawings, the same reference numerals may be used to represent the same features. Summary of the invention

[0009] Various embodiments disclosed herein include devices, systems, and methods for operating a lenticular display. In various embodiments, a method is performed at a device including a processor, a non-volatile memory, and a lenticular display. The method includes capturing a first image of a user from a first camera position in a physical environment. The method includes capturing a second image of the user from a second camera position in the physical environment. The method includes transforming the first image and the second image by horizontally flipping the first image and the second image. The method includes displaying the transformed first image on the lenticular display at a display angle corresponding to a first eye position of the user and displaying the transformed second image at a display angle corresponding to a second eye position of the user.

[0010] According to some specific implementations, a device includes one or more processors, non-volatile memory, and one or more programs; one or more programs are stored in the non-volatile memory and are configured to be executed by one or more processors. One or more programs include instructions for executing or causing the execution of any method described herein. According to some specific implementations, a non-volatile computer-readable storage medium stores instructions that, when executed by one or more processors of the device, cause the device to execute or cause the execution of any of the methods described herein. According to some specific implementations, a device includes: one or more processors, non-volatile memory, and a device for executing or causing the execution of any of the methods described herein. DETAILED DESCRIPTION

[0011] In order to provide a thorough understanding of the example implementations shown in the drawings, many details are described. However, the drawings only illustrate some example aspects of the present disclosure and should not be considered limiting. It will be understood by those of ordinary skill in the art that other effective aspects and / or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits are not described in detail to avoid obscuring more relevant aspects of the example implementations described herein.

[0012] The lenticular display can display different content at different angles. For example, in various implementations, when the lenticular display is viewed from a first horizontal angle, the object is seen from a first viewing angle, and when the lenticular display is viewed from a second horizontal angle, the object is seen from a second viewing angle different from the first viewing angle.

[0013] In various implementations, a lenticular display includes a matrix of pixels on which a lenticular pattern is superimposed. In various implementations, a first set of pixel matrices is visible from a first horizontal angle, a second set of pixel matrices is visible from a second horizontal angle, a third set of pixel matrices is visible from a third horizontal angle, and so on. In various implementations, each set of pixel matrices includes a subset of the columns of the matrix. For example, in various implementations, a first set includes columns 1, 1+N, 1+2N, 1+3N, etc., a second set includes columns 2, 2+N, 2+2N, 2+3N, etc., a third set includes columns 3, 3+N, 3+2N, 3+3N, etc., where N is the number of sets and the number of horizontal angles at which the lenticular display can display different content.

[0014] In various implementations described below, this feature of the lenticular display is used to present different views at different horizontal angles relative to the lenticular display. For example, in various implementations, different content is presented to different eyes of a user at different horizontal angles. Specifically, an object is displayed from the perspective of the left eye at the horizontal angle of the user's left eye, and the object is displayed from the perspective of the right eye at the horizontal angle of the user's right eye, thereby providing a stereoscopic (or three-dimensional) view of the object. In various implementations, the object is a user, thereby simulating a reflector.

[0015] Figure 1A A top view of an operating environment 100 including a lenticular display 132 is shown according to some implementations. The operating environment 100 includes an electronic device 130 and a user 120. The electronic device 130 includes a left image sensor 131L and a right image sensor 131R for capturing images of the user 120. The electronic device 130 includes the lenticular display 132 to display images with optional enhancements to the user 120.

[0016] Operating environment 100 has a three-dimensional coordinate system. Similarly, electronic device 130 has a three-dimensional coordinate system (represented by axis 401) that is related to the three-dimensional coordinate system of operating environment 100 through a coordinate system transformation that changes as electronic device 130 moves in operating environment 100. In various implementations, the three-dimensional coordinate system of electronic device 130 includes a horizontal dimension represented by an x-coordinate ( Figure 1A left and right on the page in ), the vertical dimension represented by the y-coordinate ( Figure 1A ) and the depth dimension represented by the z-coordinate ( Figure 1A ). Figure 1A , the user 120 is approximately at a distance d in front of the electronic device 130 in the depth dimension. The user 120 has a left eye 121L and a right eye 121R, and the left eye is at a first left angle θ1 relative to the lenticular display 132 (eg, relative to the yz plane)L The right eye forms a first right angle θ1 relative to the lenticular lens display 132 R .

[0017] Figure 1B A first left captured image 150L of the user 120 captured by the left image sensor 131L of the electronic device 130 and a first right captured image of the user 120 captured by the right image sensor 131R of the electronic device 130 are shown. The first left captured image 150L includes an image representation of the user 120. Therefore, in the first left captured image 150L, ​​the image representation of the user 120 appears approximately at a distance d from the left image sensor 131L in the depth dimension. In addition, the image representation of the user 120 is not flipped horizontally. Therefore, although the left eye 121L of the user 120 is painted with a gray shade, the image representation of the user 120 also appears as the left eye 121L having a gray shade. Similarly, the first right captured image 150R includes an image representation of the user 120. Therefore, in the first right captured image 150R, the image representation of the user 120 appears approximately at a distance d from the right image sensor 131R in the depth dimension. In addition, the image representation of the user 120 is not flipped horizontally. Thus, even though the left eye 121L of the user 120 is shaded gray, the image representation of the user 120 also appears as the left eye 121L having a shade of gray.

[0018] Figure 1C A first left view 160L of the operating environment 100 from the perspective of the left eye 121L of the user 120 and a first right view 160R of the operating environment 100 from the perspective of the right eye 121R of the user 120 are shown. The first left view 160L and the first right view 160R each include a view of the electronic device 130.

[0019] The lenticular lens display 132 displays a first left image at a first left angle and displays a first right image different from the first left image at a first right angle. Therefore, a first left view 160L from the perspective of the left eye 121L of the user 120 includes the first left image displayed by the lenticular lens display 132 at the first left angle, and a first right view 160R from the perspective of the right eye 121R of the user 120 includes the first right image displayed by the lenticular lens display 132 at the first right angle. It is noteworthy that in the first left image, the right eye 121R of the image representation of the user 120 is located at the center of the first left image, and in the first right image, the left eye 121L of the image representation of the user 120 is located at the center of the first right image.

[0020] The first left captured image 150L is transformed by reprojecting it to a virtual left camera position 133L and flipping it horizontally, thereby generating a first left image. The virtual left camera position 133L corresponds to the position of the left eye 121L of the user 120, which is reflected across the plane spanned by the lenticular display 132 (e.g., by multiplying the z-coordinate of the position of the left eye 121L of the user 120 by -1). Similarly, the first right captured image 150R is transformed by reprojecting it to a virtual right camera position 133R and flipping it horizontally, thereby generating a first right image. The virtual right camera position 133R corresponds to the position of the right eye 121R of the user 120, which is reflected across the plane spanned by the lenticular display 132 (e.g., by multiplying the z-coordinate of the position of the right eye 121R of the user 120 by -1).

[0021] By viewing the first left image and the first right image stereoscopically, the user 120 perceives a three-dimensional mirror view of the user 120 when the user 120 is not wearing a head mounted device and the electronic device 130 does not include a reflective surface.

[0022] Figure 2A 1 shows a top view of operating environment 100 after user 120 has moved to the right (e.g., in the horizontal dimension). Figure 2A , the user 120 is approximately at a distance d in front of the lenticular digital mirror 130 in the depth dimension. The left eye 121L of the user 120 is at a second left angle θ2 relative to the lenticular display 132 (eg, relative to the yz plane) L , and the right eye 121R of the user 120 is at a second right angle θ2 relative to the lenticular lens display 132 R .

[0023] Figure 2B A second left captured image 250L of the user 120 captured by the left image sensor 131L of the electronic device 130 and a second right captured image 250R of the user 120 captured by the right image sensor 131R of the electronic device 130 are shown. The second left captured image 250L includes an image representation of the user 120. Therefore, in the second left captured image 250L, the image representation of the user 120 appears approximately at a distance d from the left image sensor 131L in the depth dimension. In addition, the image representation of the user 120 is not flipped horizontally. Therefore, although the left eye 121L of the user 120 is painted with a gray shade, the image representation of the user 120 also appears as the left eye 121L having a gray shade. Compared to the first left captured image 150L, ​​because the user 120 has moved to the right, in the second left captured image 250L, the image representation of the user 120 has moved to the left.

[0024] Similarly, the second right captured image 250R includes an image representation of the user 120. Thus, in the second right captured image 250R, the image representation of the user 120 appears approximately at a distance d from the right image sensor 131R in the depth dimension. In addition, the image representation of the user 120 is not flipped horizontally. Thus, although the left eye 121L of the user 120 is painted with a gray shade, the image representation of the user 120 also appears as the left eye 121L having a gray shade. Compared to the first right captured image 150R, because the user 120 has moved to the right, in the second right captured image 250R, the image representation of the user 120 has moved to the left.

[0025] Figure 2C A second left view 260L of the operating environment 100 from the perspective of the left eye 121L of the user 120 and a second right view 260R of the operating environment 100 from the perspective of the right eye 121R of the user are shown. The second left view 260L and the second right view 260R each include a view of the electronic device 130.

[0026] The lenticular lens display 132 displays the second left image at a second left angle and displays a second right image different from the second left image at a second right angle. Therefore, the second left view 260L includes the second left image displayed by the lenticular lens display 132 at a second left angle, and the second right view 260R includes the second right image displayed by the lenticular lens display 132 at a second right angle.

[0027] The second left captured image 250L is transformed by reprojecting the second left captured image 250L to the updated virtual left camera position 233L and horizontally flipping the second left captured image 250L, thereby generating a second left image. The updated virtual left camera position 233L corresponds to the updated position of the left eye 121L of the user 120, which is reflected across the plane spanned by the lenticular display 132 (e.g., by multiplying the z-coordinate of the position of the left eye 121L of the user 120 by -1). Similarly, the second right captured image 250R is transformed by reprojecting the second right captured image 250R to the updated virtual right camera position 233R and horizontally flipping the first right captured image 250R, thereby generating a second right image. The updated virtual right camera position 233R corresponds to the updated position of the right eye 121R of the user 120, which is reflected across the plane spanned by the lenticular display 132 (e.g., by multiplying the z-coordinate of the position of the right eye 121R of the user 120 by -1).

[0028] A second left image and a second right image are also generated by presenting virtual content (e.g., virtual arrow 565) from the updated virtual left camera position 233L and the updated virtual right camera position 233R and compositing the corresponding virtual content with the corresponding transformed images. In various implementations, the virtual content provides feedback to the user during a calibration process of the electronic device 130. In various implementations, the virtual content provides an XR experience, such as allowing the user 120 to see themselves while wearing virtual sunglasses without wearing a head mounted device and the electronic device includes a reflective surface.

[0029] Figure 3 3 is a flowchart representation of a method 300 of operating a lenticular display according to some implementations. In various implementations, the method 300 is performed by a device (e.g., Figure 1A In some implementations, the method 300 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some implementations, the method 300 is performed by a processor that executes instructions (e.g., code) stored in a non-transitory computer-readable medium (e.g., a memory).

[0030] Method 300 begins at block 310, where a device captures a first image of a user from a first camera position in a physical environment. For example, Figure 1B A first left captured image 150L including the user 120 is shown captured by the left image sensor 131L of the electronic device 130. In various implementations, the first image of the user includes the first eye of the user. In various implementations, the first image of the user may further include additional parts of the user, such as the second eye of the user, the face of the user, the head of the user, the torso of the user, or the whole body of the user.

[0031] Method 300 continues at block 320, where the device captures a second image of the user from a second camera position in the physical environment. Figure 1B A first right captured image 150R including the user 120 is shown captured by the right image sensor 131R of the electronic device 130. In various implementations, the second image of the user includes the second eye of the user. In various implementations, the second image may further include additional parts of the user, such as the first eye of the user, the face of the user, the head of the user, the torso of the user, or the whole body of the user.

[0032] In various implementations, the first camera position and the second camera position are the same position. Thus, in various implementations, the first image and the second image are captured from the same position. In various implementations, the first image and the second image are captured by the same image sensor. In various implementations, the first image and the second image are the same image.

[0033] In contrast, in various implementations, the first camera position and the second camera position are different positions. Thus, in various implementations, the first image and the second image are captured by different image sensors. In various implementations, the first image and the second image are different images. For example, in Figure 1B , the first left captured image 150L and the first right captured image 150R are captured by the left image sensor 131L and the right image sensor 131R and are different images (for example, in the first left captured image 150L, ​​the representation of the user 120 is in the left half of the image, and in the first right captured image 150R, the representation of the user 120 is in the right half of the image).

[0034] The method 300 continues in block 330, where the device transforms the first image and the second image by horizontally flipping the first image and the second image. For example, in various implementations, the image is an N×M pixel matrix, each pixel having a corresponding pixel value u ij , where i is between 1 and N, and j is between 1 and M. The flipped image is also an N×M matrix of pixels with pixel values ​​v ij , where i is between 1 and N, and j is between 1 and M, where v ij =u (N+1-i)j .

[0035] Method 300 continues in block 340 where the device displays the transformed first image on the lenticular display at a display angle corresponding to a first eye position of the user and displays the transformed second image at a display angle corresponding to a second eye position of the user.

[0036] In various implementations, the first eye position and the second eye position are different positions, and the first eye is different from the second eye. For example, in various implementations, the first eye is the user's left eye, and the second eye is the user's right eye (or vice versa). Furthermore, it should be understood that the first eye position is different from the first camera position and the second camera position, and the second eye position is different from the first camera position and the second camera position.

[0037] In various implementations, the first camera position, the first eye position, the second camera position, and the second eye position are each associated with a corresponding set of three-dimensional coordinates of a three-dimensional coordinate system of the physical environment. In various implementations, the first camera position, the first eye position, the second camera position, and the second eye position are each associated with a corresponding set of three-dimensional coordinates of a three-dimensional coordinate system of the device. The three-dimensional coordinate system of the device and the three-dimensional coordinate system of the physical environment are related by a coordinate system transformation that changes as the device moves in the physical environment.

[0038] In various embodiments, the three-dimensional coordinate system of the device includes a horizontal dimension (defined by the x-coordinate) that is parallel to the line between the left image sensor and the right image sensor of the device. In various embodiments, the three-dimensional coordinate system of the device includes a horizontal dimension (defined by the x-coordinate) that is parallel to the pixel rows of the lenticular display. In various embodiments, the three-dimensional coordinate system of the device includes a vertical dimension (defined by the y-coordinate) that is defined by the inertial measurement unit (IMU) of the device. In various embodiments, the three-dimensional coordinate system of the device includes a vertical dimension (defined by the y-coordinate) that is parallel to the pixel columns of the lenticular display. In various embodiments, the three-dimensional coordinate system includes a depth dimension (defined by the z-coordinate) that is parallel to the optical axis of at least one image sensor of the device. In various embodiments, the three-dimensional coordinate system includes a dimension that is perpendicular to the other two dimensions (regardless of how they are defined).

[0039] In various implementations, method 600 includes determining at least one of a first camera position, a first eye position, a second camera position, or a second eye position. In various implementations, method 600 includes determining at least one of the first camera position, the first eye position, the second camera position, or the second eye position in a three-dimensional coordinate system of a physical environment (e.g., determining a set of three-dimensional coordinates in the three-dimensional coordinate system of the physical environment). In various implementations, method 600 includes determining at least one of the first camera position, the first eye position, the second camera position, or the second eye position in a three-dimensional coordinate system of a device (e.g., determining a set of three-dimensional coordinates in the three-dimensional coordinate system of the device).

[0040] In various implementations, the first camera position is at a first reflection position corresponding to a first eye position that is reflected across a plane spanned by the lenticular display (e.g., the xy plane in various implementations.) Thus, the first image (after being flipped horizontally) is approximately what the user's first eye would see if the lenticular display was instead a mirror.

[0041] However, in various implementations, the first camera position is not at the first reflection position. Thus, in various implementations, transforming the first image (in block 330) further includes transforming the first image based on (and taking into account) a difference between the first camera position and the first reflection position. For example, in various implementations, transforming the first image further includes at least one of shifting or rescaling the first image based on a difference between the first camera position and a first reflection position corresponding to a first eye position that is reflected across a plane spanned by the lenticular display. For example, shifting the first image may be based on (and taking into account) a difference between the first camera position and the first reflection position in a horizontal and / or vertical dimension of a three-dimensional coordinate system of the device, and rescaling the first image may be based on (and taking into account) a difference between the first camera position and the first reflection position in a depth dimension.

[0042] In various implementations, shifting and / or rescaling the first image produces an imperfect approximation of what the user's first eye would see if the lenticular display was instead a reflector. Thus, in various implementations, transforming the first image further includes: performing a projective transformation of the first image based on a difference between a first camera position and a first reflection position corresponding to a first eye position, the first eye position reflected across a plane spanned by the lenticular display, and depth information of the physical environment. In various implementations, the depth information includes a depth map that indicates, for each pixel of the first image, a distance between an image sensor that captured the first and a portion of the physical environment represented by the pixel.

[0043] In various implementations, the projective transformation is a forward mapping, wherein for each pixel of the first image at a pixel position in the untransformed space, a new pixel position is determined in the transformed space of the transformed first image. In various implementations, the projective transformation is a reverse mapping, wherein for each pixel of the transformed first image at a pixel position in the transformed space, a source pixel position is determined in the untransformed space of the first image.

[0044] In various implementations, the source pixel position or the destination pixel position is determined based on a set of four-dimensional homogeneous coordinates, at least one of which is proportional (or inversely proportional) to the depth at the pixel position. In addition, in various implementations, the source pixel position or the destination pixel position is determined based on a 4×4 view projection matrix of a first perspective and / or a second perspective of an image sensor that captures the first image. In various implementations, the second perspective is from a first reflection position. In various implementations, the second perspective is from a position closer to the first reflection position than the first camera position. In various implementations, the second perspective shares one, two, or three coordinates with the first reflection position in the three-dimensional coordinate system of the device. In various implementations, the source pixel position or the destination pixel position is determined based on a 4×4 re-projection matrix, which is a multiplication of a view projection matrix of a first perspective of the image sensor that captures the first image and a view projection matrix of a second perspective (or, for at least one view matrix, its inverse).

[0045] Moreover, in various implementations, the 4×4 re-projection matrix is ​​further a multiplication of a 4×4 horizontal reflection matrix (eg, [-10 0 0; 0 1 0 0; 0 0 1 0; 0 0 0 1]). Thus, in various implementations, the projection transformation and horizontal flipping can be performed as a single transformation.

[0046] In various implementations, performing the projective transformation results in holes in the transformed first image, e.g., pixel positions of the transformed first image for which there are no corresponding pixel positions of the first image. Such holes may be filled via interpolation or using additional images, such as another image from a different perspective (e.g., one or more second images of the user captured at different times). Thus, in various implementations, performing the projective transformation is based on the second image.

[0047] Likewise, in various implementations, the second camera position is not at a second reflection position corresponding to a second eye position that is reflected across a plane spanned by the lenticular display (e.g., in various implementations, the xy plane). Thus, in various implementations, transforming the second image further includes transforming the second image based on a difference between the second camera position and the second reflection position, including any of the transformations discussed above with respect to the first image.

[0048] In various embodiments, transforming the first image further includes compositing the first virtual content with the first image. Thus, in various embodiments, method 600 further includes presenting the first virtual content. In various embodiments, the device presents the first virtual content from the perspective of the first eye position. In various embodiments, transforming the second image further includes compositing the second virtual content with the second image. Thus, in various embodiments, method 600 further includes presenting the second virtual content. In various embodiments, the device presents the second virtual content from the perspective of the second eye position. In various embodiments, the first virtual content and the second virtual content are the same virtual object presented from different perspectives. For example, in Figure 2C , the virtual arrow 265 is viewed from a first perspective in the second left view 260L and from a second perspective in the second right view 260R.

[0049] In various implementations, the first virtual content and / or the second virtual content provides feedback to the user during a calibration process of the device. For example, in various implementations, the first virtual content and / or the second virtual content provides feedback during a facial scan for, e.g., facial recognition, determining pupil distance, etc. In various implementations, the first virtual content and / or the second virtual content provides an XR experience, such as allowing a user to view themselves while wearing virtual sunglasses without wearing a head mounted device.

[0050] In various embodiments, displaying the transformed first image at a display angle corresponding to a first eye position includes displaying the transformed first image with a first set of pixel columns of a lenticular display, and displaying the transformed second image at a display angle corresponding to a second eye position includes displaying the transformed second image with a second set of pixel columns of the lenticular display.

[0051] In various implementations, displaying the transformed first image and the transformed second image includes interleaving columns of the transformed first image with columns of the transformed second image to form a display image, and displaying the display image on the lenticular display. In various implementations, interleaving columns of the transformed first image with columns of the transformed second image further includes interleaving blank columns into the display image. In various implementations, displaying the display image includes activating columns of pixels corresponding to the transformed first image and the transformed second image and deactivating columns of pixels corresponding to blank columns. Thus, portions of the lenticular display may be deactivated for blank columns to save power.

[0052] In various implementations, as the user moves relative to the device, the device updates the first image, the second image, and the corresponding angles at which the transformed updated image is displayed by the lenticular display. Thus, in various implementations, method 600 further includes: capturing an updated first image of the user from an updated first camera position in the physical environment; capturing an updated second image of the user from an updated second camera position in the physical environment; transforming the updated first image and the updated second image by horizontally flipping the updated first image and the updated second image; and displaying the transformed updated first image on the lenticular display at a display angle corresponding to the updated first eye position of the user and displaying the transformed updated second image at a display angle corresponding to the updated second eye position of the user.

[0053] Figure 4 It is implemented according to some specific Figure 1A 16x, GSM, CDMA, TDMA, GPS, IR, BLUETOOTH, ZIGBEE, and / or similar types of interfaces), one or more programming (e.g., I / O) interfaces 410, a lenticular display 132, image sensors 131L and 131R, a memory 420, and one or more communication buses 404 for interconnecting these components and various other components.

[0054] In some implementations, the one or more communication buses 404 include circuits that interconnect and control communications between system components. In some implementations, the one or more I / O devices and sensors 406 include at least one of the following: an inertial measurement unit (IMU), an accelerometer, a gyroscope, a thermometer, one or more microphones, one or more speakers, one or more biometric sensors (e.g., blood pressure monitor, heart rate monitor, respiration monitor, galvanic skin monitor, blood oxygen sensor, blood glucose sensor, etc.), a haptic engine, one or more depth sensors (e.g., structured light, time of flight, etc.), etc.

[0055] In some implementations, the lenticular display 432 is configured to display different content at different angles. In some implementations, the lenticular display 432 includes holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light emitting field effect transistor (OLET), organic light emitting diode (OLED), surface conduction electron emitter display (SED), field emission display (FED), quantum dot light emitting diode (QD-LED), microelectromechanical system (MEMS) and / or similar display types. In some implementations, the lenticular display 432 corresponds to a diffraction, reflection, polarization, holographic, etc. waveguide display. In various implementations, the lenticular display 432 can present mixed reality and / or virtual reality content.

[0056] In various implementations, image sensors 431L and 431R include one or more RGB cameras (e.g., with complementary metal oxide semiconductor (CMOS) image sensors or charge coupled device (CCD) image sensors), one or more infrared (IR) cameras, one or more event-based cameras, and the like.

[0057] The memory 420 includes a high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid-state memory devices. In some specific implementations, the memory 420 includes a non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices or other non-volatile solid-state storage devices. The memory 420 optionally includes one or more storage devices remotely located from the one or more processing units 402. The memory 420 includes a non-transitory computer-readable storage medium. In some specific implementations, the memory 420 or the non-transitory computer-readable storage medium of the memory 420 stores the following programs, modules and data structures or a subset thereof, which includes an optional operating system 430 and a content presentation module 440.

[0058] The operating system 430 includes processes for handling various basic system services and for performing hardware-related tasks. In some implementations, the content presentation module 440 is configured to present different content to different users at different angles via the lenticular display 132. To this end, in various implementations, the content presentation module 440 includes an image transformation unit 442 and a content presentation unit 444.

[0059] In some implementations, the image transformation unit 442 is configured to transform (by horizontally flipping) the images captured by the image sensors 131L and 131R. To this end, in various implementations, the image transformation unit 442 includes instructions and / or logic components for instructions and heuristics and metadata for the heuristics.

[0060] In some implementations, the content presentation unit 444 is configured to display the transformed image at different angles of the lenticular display 132. To this end, in various implementations, the content presentation unit 444 includes instructions and / or logic for instructions and heuristics and metadata for the heuristics.

[0061] Although image transformation unit 442 and content presentation unit 444 are shown as residing on a single device (e.g., Figure 1A However, it should be understood that in other specific implementations, the image transformation unit 442 and the content presentation unit 444 may be located in separate computing devices.

[0062] also, Figure 4 It is more of a functional description of various features that may be present in a particular implementation, as opposed to a schematic diagram of the structure of the implementation described herein. As one of ordinary skill in the art will recognize, items shown separately may be combined, and some items may be separated. For example, Figure 4 Some functional modules shown separately in the figure may be implemented in a single module, and the various functions of a single functional block may be implemented by one or more functional blocks in various specific implementations. The actual number of modules and the division of specific functions and how the features are distributed among them will vary depending on the specific implementation, and in some specific implementations, it depends in part on the specific combination of hardware, software and / or firmware selected for a specific specific implementation.

[0063] Although various aspects of specific implementations within the scope of the appended claims are described above, it should be apparent that the various features of the above-mentioned specific implementations can be embodied in a variety of forms, and any specific structures and / or functions described above are merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that the aspects described herein can be implemented independently of any other aspects, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement a device and / or a method can be practiced. In addition, in addition to or different from one or more aspects set forth herein, other structures and / or functions can be used to implement such a device and / or such a method can be practiced.

[0064] It will also be understood that, although the terms "first", "second", etc. may be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description, as long as all occurrences of the "first node" are consistently renamed and all occurrences of the "second node" are consistently renamed. Both the first node and the second node are nodes, but they are not the same node.

[0065] The terms used herein are only for the purpose of describing a specific implementation and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms of "a", "an" and "the" are intended to also cover the plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and covers any and all possible combinations of one or more items in the associated listed items. It will also be understood that when used in this specification, the terms "comprises" and / or "comprising" specify the presence of specified features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groups.

[0066] As used herein, the term “if” may be interpreted to mean “when the antecedent is true” or “when the antecedent is true” or “in response to determining” or “upon determining” or “in response to detecting” that the antecedent is true, depending on the context. Similarly, the phrase “if it is determined that [the antecedent is true]” or “if [the antecedent is true]” or “when [the antecedent is true]” is interpreted to mean “upon determining that the antecedent is true” or “in response to determining” or “upon determining” that the antecedent is true or “when detecting that the antecedent is true” or “in response to detecting” that the antecedent is true, depending on the context.

Claims

1. A method comprising: At a device including one or more processors, non-transitory memory, an image sensor, and a lenticular display: capturing a first image of a user from a first camera position in a physical environment; capturing a second image of the user from a second camera location in the physical environment; transforming the first image and the second image by flipping the first image and the second image horizontally; as well as The transformed first image is displayed on the lenticular display at a display angle corresponding to a first eye position of the user and the transformed second image is displayed at a display angle corresponding to a second eye position of the user.

2. The method of claim 1, wherein transforming the first image further comprises: The first image is at least one of shifted or rescaled based on a difference between the first camera position and a first reflection position corresponding to the first eye position reflected across a plane spanned by the lenticular display.

3. The method of claim 1 , wherein transforming the first image further comprises: A projective transformation of a first image is performed based on a difference between the first camera position and a first reflection position corresponding to the first eye position and depth information of the physical environment, the first reflection position being reflected across a plane spanned by the lenticular display.

4. The method of claim 3, wherein performing the projective transformation is based on a re-projection matrix based on a multiplication or inverse multiplication of a view projection matrix of a first perspective and a view projection matrix of a second perspective of an image sensor that captured the first image. The method of claim 4 , wherein the second viewing angle is at a position closer to the first reflection position than the first camera position. The method of claim 4 , wherein the second viewing angle is at a location that shares at least one coordinate with the first reflection location. The method of claim 4 , wherein the second viewing angle is at the first reflecting position. The method of claim 4 , wherein the re-projection matrix is ​​further based on a horizontal reflection matrix.

9. The method of claim 3, wherein performing the projective transformation is based on the second image.

10. The method of claim 1, wherein transforming the first image further comprises compositing first virtual content with the first image.

11. The method of claim 1 , wherein displaying the transformed first image at the display angle corresponding to the first eye position comprises displaying the transformed first image at a first set of pixel columns of the lenticular display, and displaying the transformed second image at the display angle corresponding to the second eye position comprises displaying the transformed second image at a second set of pixel columns of the lenticular display.

12. The method of claim 1 , wherein displaying the transformed first image and the transformed second image comprises: interleaving columns of the first image with columns of the second image to form a display image; as well as The display image is displayed on the lenticular display.

13. The method of claim 12, wherein interleaving columns of the first image with columns of the second image further comprises interleaving blank columns into the display image. 14 . The method of claim 13 , wherein displaying the display image comprises activating pixel columns corresponding to the first image and the second image and deactivating pixel columns corresponding to the blank columns.

15. The method according to claim 1, further comprising: capturing an updated first image of the user from an updated first camera position in the physical environment; capturing an updated second image of the user from an updated second camera position in the physical environment; transforming the updated first image and the updated second image by horizontally flipping the updated first image and the updated second image; as well as The transformed updated first image is displayed on the lenticular display at a display angle corresponding to the updated first eye position of the user and the transformed updated second image is displayed at a display angle corresponding to the updated second eye position of the user.

16. A device comprising: Lenticular display; as well as non-transitory memory; as well as One or more processors configured to: capturing a first image of a user from a first camera position in a physical environment; capturing a second image of the user from a second camera location in the physical environment; transforming the first image and the second image by flipping the first image and the second image horizontally; as well as The transformed first image is displayed on the lenticular display at a display angle corresponding to a first eye position of the user and the transformed second image is displayed at a display angle corresponding to a second eye position of the user.

17. A device according to claim 16, wherein the one or more processors are used to transform the first image in the following manner: further performing a projective transformation of the first image based on a difference between the first camera position and a first reflection position corresponding to the first eye position reflected across a plane spanned by the lenticular display and depth information of the physical environment.

18. The apparatus of claim 16, wherein the one or more processors are configured to display the transformed first image and the transformed second image by: interleaving columns of the first image with columns of the second image to form a display image; and The display image is displayed on the lenticular display.

19. The apparatus of claim 18, wherein the one or more processors are configured to interleave columns of the first image with columns of the second image by further interleaving blank columns into the display image.

20. A non-transitory memory storing one or more programs that, when executed by one or more processors of a device having a lenticular display, cause the device to: capturing a first image of a user from a first camera position in a physical environment; capturing a second image of the user from a second camera location in the physical environment; transforming the first image and the second image by flipping the first image and the second image horizontally; as well as The transformed first image is displayed on the lenticular display at a display angle corresponding to a first eye position of the user and the transformed second image is displayed at a display angle corresponding to a second eye position of the user.

Images