Opposite rotation of the display panel and / or virtual camera in the HMD

Abstract

A head-mounted display (HMD) system may include an HMD comprising a housing and a pair of counter-rotated display panels mounted within the housing. The compositor of the HMD system may also be configured to provide camera pose data with counter-rotated camera orientations to a running application (e.g., a video game application) and resample frames received from the application with or without clockwise and counter-clockwise rotation adjustments depending on whether the HMD's display panels are oriented upright or counter-rotated. A combined approach may use counter-rotated camera orientations in combination with counter-rotated display panels to provide an HMD with optimized display performance.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a PCT application claiming priority to U.S. Patent Application No. 16 / 366,981, entitled "COUNTERROTATION OF DISPLAY PANELS AND / OR VIRTUAL CAMERAS IN A HMD," filed March 27, 2019, which claims priority to co-pending U.S. Provisional Patent Application No. 62 / 792,290, filed January 14, 2019. Applications Nos. 16 / 366,981 and 62 / 792,290 are incorporated herein by reference in their entireties. [Background technology]

[0002] A head mounted display (HMD) may be worn by a user for the purpose of immersing the user in a virtual reality (VR) environment (e.g., a VR game) or an augmented reality (AR) environment. HMDs operate by displaying images on one or more display panels based on frames output by an application (e.g., a video game). These images are viewed by the user through optics (e.g., lenses) included in the HMD, causing the user to perceive the images as if the user were immersed in the VR or AR environment.

[0003] Some HMDs employ a single display panel and others employ a pair of display panels. One reason for using a single display panel is that a single display is less expensive (in terms of materials and / or manufacturing costs) than a pair of display panels. However, using two display panels offers several advantages, such as increased display bandwidth, the ability to utilize more of the available pixels, and allowing the user to adjust the distance between the two display panels, among other advantages.

[0004] All HMDs currently on the market have in common that they have one or more display panels in an upright panel orientation. For example, an HMD with two display panels orients the display panels directly in front of the user's face in a coplanar, side-by-side arrangement with their respective side edges parallel to one another. This type of upright panel orientation suffers from several drawbacks. For example, panel artifacts such as stripes or bands running the length of the panel often appear on upright oriented panels with the displayed image. These panel artifacts visually distract the user's view. In addition, the field of view (FOV) available to the user is limited to or restricted by the width of the display panel, which is typically rectangular. In addition, the basic filters (e.g., box filters) used to resample upright oriented images often result in sampling artifacts (e.g., jagged lines) along the vertical and horizontal directions of the displayed image. These sampling artifacts also visually distract the user's view considerably, especially when combined with the panel artifacts described above.

[0005] Provided herein are technical solutions for improving and enhancing these and other systems. [Brief description of the drawings]

[0006] The detailed description will now be described with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number refers to the figure in which the reference number first appears, and the use of the same reference number in different figures indicates similar or identical parts or features. [Figure 1] FIG. 1 illustrates an exemplary head mounted display (HMD) including a pair of display panels with oppositely rotated orientations. [Diagram 2]FIG. 2 shows front and side views of a display panel (one of the display panels in the HMD) in an upright panel orientation. FIG. 2 also shows reference planes for describing the orientation of the display panel relative to the upright panel orientation after rotating or tilting the display panel about one or more axes. [Diagram 3] A figure showing a front view of a pair of display panels in oppositely rotated orientations, a top view of the pair of display panels in oppositely rotated orientations, and an additional top view of the pair of display panels after tilting the display panels. [Figure 4] FIG. 13 is a flow diagram of an exemplary process for sending camera pose data to a running application with a counter-rotated camera orientation, resampling frames output by the application without rotational adjustments based on the camera pose data or the display panel orientation, and outputting pixel data for the resampled frames to a frame buffer for display on a counter-rotated display panel. [Diagram 5] FIG. 2 illustrates a displayed image from the perspective of a user viewing through the lenses of an HMD according to an embodiment disclosed herein. [Figure 6] FIG. 13 is a flow diagram of an exemplary process for sending camera pose data to a running application in an upright camera orientation, rotationally adjusting and resampling frames output by the application in a clockwise / counterclockwise direction, and outputting pixel data of the resampled frames to a frame buffer for display on a counter-rotated display panel. [Figure 7] 1A-1C show a front view of a pair of display panels in an upright panel orientation, a top view of the pair of display panels in an upright panel orientation, and an additional top view of the pair of display panels after tilting the display panels. [Figure 8]FIG. 13 is a flow diagram of an exemplary process for sending camera pose data to a running application with a counter-rotation camera orientation, rotationally adjusting and resampling frames output by the application in a clockwise / counterclockwise direction, and outputting pixel data of the resampled frames to a frame buffer for display on an upright oriented display panel. [Figure 9] 1 illustrates example components of an HMD, such as a VR headset, in which the technology disclosed herein may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Described herein, among other things, is a head mounted display (HMD) system including an HMD having a housing and a pair of display panels mounted within the housing, the display panels being in oppositely rotated orientations. The pair of display panels may include a first display panel (e.g., a left display panel configured to be viewed by the left eye of a user wearing the HMD) and a second display (e.g., a right display panel configured to be viewed by the right eye of a user wearing the HMD). The first display panel may be oriented in a first orientation by rotating the first display panel relative to an upright panel orientation in a first direction (e.g., clockwise) about a first axis perpendicular to the frontal plane of the first display panel, and the second display panel may be oriented in a second orientation by rotating the second display panel relative to an upright panel orientation in a second direction opposite the first direction (e.g., counterclockwise) about a second axis perpendicular to the frontal plane of the second display panel. In a configuration in which the display panels are coplanar and adjacent to one another, the side edges of the first display panel and the side edges of the second display panel are not parallel (e.g., the side edges of each of the pair of display panels exclusively form an angle between 0 and 180 degrees).

[0008] Also described herein are techniques for providing a running application (e.g., a video game application) with camera pose data for a counter-rotation camera orientation depending on whether the display panel of the HMD is oriented upright or rotated in the opposite direction, and resampling frames received from the application with or without rotation adjustments. For example, computer-readable instructions (e.g., a compositor) of an HMD system executed to resample frames output by an application (e.g., a video game application) may send to the application a first rotation camera orientation of a first virtual camera (e.g., a left virtual camera) rotated in a first direction (e.g., counterclockwise), and a second rotation camera orientation of a second virtual camera (e.g., a right virtual camera) rotated in a second direction opposite to the first direction (e.g., clockwise). The compositor may then receive the frames output by the application according to the counter-rotation camera orientation. This contra-rotation camera orientation approach can be used with an upright oriented display panel of an HMD, in which case the compositor resamples the frame by rotating first and second subsets of the pixel data of the frame in a direction opposite to the rotated camera direction to display a pair of upright oriented images on the upright oriented display panel. This means that the disclosed techniques for providing a contra-rotation camera orientation to a running application can be performed independent of an opposite rotation of the display panel of the HMD.

[0009] In some embodiments, the combined approach is implemented in an HMD system that includes an HMD with a pair of display panels with oppositely rotated orientations, whereby the compositor provides a running application (e.g., a video game application) with camera pose data with oppositely rotated camera orientations, and the compositor resamples frames received from the application without making rotational adjustments based on the camera pose data or based on the display panel orientation. In this way, an upright oriented image is displayed on the oppositely rotated display panel without making rotational adjustments (based on the camera pose data or the display panel orientation) in the resampling step. This combined approach provides several advantages, many related to improving the display performance of the HMD. For example, oppositely rotating the display panels of the HMD reduces panel artifacts, such as stripes or bands that run the length of the panel in the vertical and horizontal directions, by disguising the panel artifacts. That is, panel artifacts that appear on the individual display panels are still present, but are no longer visible to both eyes of the user at the same location on the display because the display panels have been rotated. As a result, panel artifacts are rendered less noticeable to the viewing user so as not to be visually distracting. Counter-rotated display panels also provide a wider field of view (FOV) to the viewing user compared to upright oriented display panels, and they also allow the HMD manufacturer (and / or user) to reduce the depth dimension of the HMD by allowing the display panel to be moved closer to the user's face, which may provide a smaller, lighter, more comfortable to wear, and more ergonomic HMD. Additionally, in embodiments in which the HMD's compositor is programmed to send camera pose data of the counter-rotated camera orientation to the running application, sampling artifacts (e.g., jagged lines) are also mitigated.This is because the filter (e.g., a box filter) used to resample an image that is counter-rotated no longer causes sampling artifacts that occur when the same filter is used to resample an upright oriented image, thereby providing smoother appearing lines in the displayed image.

[0010] It should be understood that the present disclosure describes a system (e.g., an HMD system) configured to implement the techniques and processes disclosed herein. In some embodiments, the HMD includes a pair of display panels with counter-rotated orientations. In other embodiments, the HMD may include an upright-oriented display panel, in which case the counter-rotated camera orientation is provided to the running application by the HMD's compositor in the process of rendering the frame. Also disclosed herein is a non-transitory computer-readable medium that stores computer-executable instructions for implementing the techniques and processes disclosed herein. Although many of the following examples are described in the context of video game applications, particularly VR game applications, it should be understood that the techniques and systems described herein may provide benefits to other applications, including, but not limited to, non-VR applications (e.g., AR applications), and / or non-game applications such as industrial machinery applications, defense applications, robotics applications, etc.

[0011] FIG. 1 illustrates an exemplary head mounted display (HMD) 100 worn by a user 102. The example HMD 100 of FIG. 1 is shown to include a pair of display panels 104(1) and 104(2) in oppositely rotated orientations. The display panel 104, the lens barrel 106, and the lens 108 respectively constitute a lens and display assembly. Thus, the HMD 100 may include a pair of lens and display assemblies including a first lens and display assembly having a first display 104(1), a first lens barrel 106(1), and a first lens 108(1), and a second lens and display assembly having a second display 104(2), a second lens barrel 106(2), and a second lens 108(2). The first lens and display assembly may correspond to the left eye of the user 102, while the second lens and display assembly may correspond to the right eye of the user 102. Each lens and display assembly is aligned on its own primary optical axis, referred to as the Z-axis in Figure 1. Each Z-axis (associated with each lens and display assembly) is perpendicular to the frontal plane of the display panel 104. Reference planes such as the frontal plane mentioned above are described in more detail below with reference to Figure 2.

[0012] The pair of lenses and display assemblies shown in FIG. 1 may be mounted within a housing 110 of the HMD 100. Although an outer shell of the housing 110 is shown in FIG. 1, it should be understood that the housing 110 may include various components and / or structures used to mount various electronic components and printed circuit boards within the housing 110. The pair of display panels 104(1) and 104(2) (collectively 104) and the pair of lenses 108(1) and 108(2) (collectively 108) may be attached to opposing ends of the lens barrels 106(1) and 106(2) (collectively 106), respectively, using any suitable mounting mechanism (e.g., screws, adhesives, latches, pins, etc.). Additionally, the lenses and display assemblies may be attached to an intermediate frame within the housing 110 adjacent to a rear surface of the display panels 104. This intermediate frame may house other components of the HMD, such as a motherboard with various sensors, processors, etc. The housing 110 may further include a shrouded (or hooded) intermediate frame that is attached to the lens and display assembly with an opening for the lens 108. The shrouded intermediate frame may be configured to block ambient light from entering the space between the user's eyes and the lenses 108(1) and 108(2) (collectively 108) when the user 102 is wearing the HMD 100.

[0013] In general, the HMD 100 may be worn by a user 102 for the purpose of immersing the user in a virtual reality (VR) environment (e.g., a VR game) and / or an augmented reality (AR) environment. Thus, the HMD 100 may represent a VR headset for use in a VR system, such as for use in a VR game system, in some examples. However, the HMD 100 may additionally or alternatively be implemented as an AR headset for use in an AR application. In AR, the user 102 sees virtual objects superimposed on a real-world environment, whereas in VR, the user 102 does not see a real-world environment but is fully immersed in a virtual environment as perceived through the display panel 104 and optics (e.g., lens 108) of the HMD 100. Although the examples described herein relate primarily to a VR-based HMD 100, it should be understood that the HMD 100 is not limited to implementation in VR applications.

[0014] The pair of display panels 104 are configured to display images based on a series of image frames (hereafter referred to as "frames") output by an application (e.g., a video game). These images are viewed by the user 102 through a lens 108, which may include various types of optics to provide a display close to the eyes. In this way, the user 102 perceives the images as if the user 102 were immersed in the VR or AR environment. The compositor of the HMD 100 may resample the frames output by the application utilizing various techniques, such as performing chromatic aberration, reprojection, and the like. In some embodiments, the compositor may resample the frames by counter-rotating a subset of pixel data of each frame (in a clockwise and counterclockwise direction, respectively) to ensure that the images are displayed upright oriented on the display panel 104, as described in more detail below. Regardless of whether the resampling step includes a rotation adjustment based on the camera pose data and / or the orientation of the display panel, the corrected pixel data of the resampled frames may be output to a frame buffer to display the images on the display panel 104. For example, the frame buffer used by the HMD 100 may be a stereo frame buffer that renders a pair of images on a pair of display panels. For example, pixel data for 2160×1200 pixels may be output to the frame buffer for display on a pair of display panels 104 of the HMD 100 (e.g., 1080×1200 pixels per display panel 104).

[0015] The display panel 104 may utilize any suitable type of display technology, such as a light-emitting display that utilizes light-emitting elements to emit light during presentation of frames on the display panel(s) 104 of the HMD 100. By way of example, the display panel 104 of the HMD 100 may include a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an inorganic light-emitting diode (ILED) display, or any other suitable type of display that utilizes a suitable display technology used for HMD 100 applications. In some embodiments, the display system of the HMD 100 may be a low-persistence display system, meaning that the light-emitting elements emit light for a small portion of the frame time (e.g., approximately 1 millisecond (ms) of a frame time of 11.11 ms) for each frame used to render an image on the display panel.

[0016] 2 is a diagram illustrating front and side views of display panel 104 in an upright panel orientation 200. Panel 104 in FIG. 2, which may represent either display panel 104(1) or 104(2), has an upright panel orientation 200. FIG. 2 also illustrates three reference planes, the frontal plane, the midsagittal plane, and the transverse plane, that are referenced herein to describe how each display panel 104 may be rotated and / or tilted into a particular orientation within housing 110 of HMD 100.

[0017] As shown in FIG. 2, the frontal plane of the display panel 104 is parallel to the front and rear surfaces of the display panel 104. The front of the display panel 104 is the surface of the display panel 104 that the user 102 is looking at during image presentation. The frontal plane can bisect the display panel 104 into a front half and a rear half, with the front half having the front (viewing) surface of the display panel 104. Also in FIG. 2, the axis perpendicular to the frontal plane of the display panel 104 is referred to as the Z-axis. This Z-axis may correspond to the primary optical axis along which light runs from the display panel 104 along the eye of the user 102 in some configurations. While the positive and negative directions can be defined in any manner, examples herein are described with reference to a positive Z-direction pointed from the display panel 104 towards the user 102 wearing the HMD 100. Thus, the negative Z-direction is considered to be in the direction pointed from the user 102 wearing the HMD 100 towards the display panel 104.

[0018] The midsagittal plane bisects the display panel 104 vertically to create a left half and a right half. The axis perpendicular to the midsagittal plane of the display panel 104 is referred to as the X-axis in FIG. 2. Again, the positive and negative directions may be defined in any manner, but the examples herein are described with reference to a positive X-direction pointing from the left half of the display panel 104 toward the right half of the display panel 104 (from the perspective of the user 102 looking at the front of the display panel 104). Thus, the negative X-direction is considered to point in the direction from the right half of the display panel 104 toward the left half of the display panel 104 (again, from the perspective of the user 102 looking at the front of the display panel 104).

[0019] The cross-section horizontally bisects the display panel 104 to create a top half and a bottom half. The axis perpendicular to the cross-section of the display panel 104 is referred to as the Y-axis in FIG. 2. Again, the positive and negative directions may be defined in any manner, but the examples herein are described with reference to a positive Y-direction that points from the bottom half of the display panel 104 to the top half of the display panel 104. The negative Y-direction is therefore considered to be in a direction that points from the top half of the display panel 104 to the bottom half of the display panel 104.

[0020] FIG. 3 illustrates a front view 300 of the pair of display panels 104(1) and 104(2) in oppositely rotated orientations, as well as a top view 302 of the pair of display panels 104(1) and 104(2) in oppositely rotated orientations, and an additional top view 304 of the pair of display panels 104(1) and 104(2) after display panels 104(1) and 104(2) have been tilted in opposite directions.

[0021] Front view 300 represents a view of the front of display panels 104(1) and 104(2), the front being the surface on which images are presented via the array of pixels on each display panel 104. Thus, first display panel 104(1) may represent a left display panel corresponding to the left eye of user 102, and second display panel 104(2) may represent a right display panel corresponding to the right eye of user 102. As shown in FIG. 3, first display panel 104(1) may be oriented in a first direction (e.g., a clockwise direction) about a first axis (e.g., a Z-axis) orthogonal to the frontal plane of first display panel 104(1), relative to upright panel orientation 200. Meanwhile, the second display panel 104(2) may be oriented in a second direction relative to the upright panel orientation 200 by rotating the second display panel 104(2) about a second axis (e.g., a Z axis) perpendicular to the frontal plane of the second display panel 104(2) in a second direction opposite to the first direction (e.g., a counterclockwise direction). Thus, the pair of display panels 104 are each rotated about their respective Z axes in opposite directions relative to one another and are therefore in oppositely rotated orientations.

[0022] In some embodiments, the second display panel 104(2) is adjacent to the first display panel 104(1), or vice versa, and the two display panels 104(1) and 104(2) are in the same plane. The spacing between the display panels 104 is configurable and may be based on a heuristic associated with the user's interpupillary distance (IPD). In some embodiments, the innermost points (e.g., adjacent top corners) of the two display panels 104 may touch (i.e., touch), or they may be separated by a small gap, such as a distance of about 1-3 millimeters (mm). In some embodiments, the display panels 104(1) and 104(2) may be positioned differently from one another. For example, instead of positioning display panels 104(1) and 104(2) directly in front of the eyes of user 102, display panel 104 may be located elsewhere in the HMD (e.g., near the temples of user 102, directly in front of the forehead of user 102, etc.) and optical components such as waveguides, mirrors, etc. may be used to transmit images output on display panel 104 via lens 108 to the eyes of user 102. Even in those embodiments in which display panels 104(1) and 104(2) are not adjacent to one another in HMD 100, display panels 104(1) and 104(2) may still be oppositely rotated by rotating the first display panel 104(1) about an axis perpendicular to its frontal plane in a first direction, and by rotating the second display panel 104(2) about an axis perpendicular to its frontal plane in a second direction opposite the first direction.

[0023] The position and orientation of the display panels 104 within the housing 110 of the HMD may be fixed (i.e., non-adjustable), which may allow for calibration of the HMD 100 at a manufacturer so that a calibrated HMD 100 is provided "out of the box." However, the housing 110 of the HMD 100 may include one or more mechanisms that allow a user to adjust the position and / or orientation of each display panel 104 individually or relative to one another within the housing 110. For example, a threaded rod may connect together the pair of lens and display assemblies shown in FIG. 1, and the HMD 100 may include a knob or similar adjustment mechanism that the user 102 may manipulate to rotate the threaded rod, increasing or decreasing the distance between the pair of display panels 104 depending on the direction the user 102 rotates the knob. A similar mechanism may be used to adjust the vertical position of the display panel 104 (e.g., moving the display panel 104 up or down along the Y axis), or adjust the amount of rotation (e.g., about the Z axis), and / or adjust the amount the display panel 104 is tilted (e.g., about the X and / or Y axes). This may provide additional flexibility to the user 102 to adjust the position and / or orientation of the display panel 104, but may create some additional difficulties in keeping the HMD 100 calibrated.

[0024] 3 also illustrates a configuration in which the display panels 104 are adjacent to one another and in the same plane, such that the side edges 306(1) of the first display panel 104(1) and the side edges 306(2) of the second display panel 104(2) are not parallel. This can be contrasted with a pair of display panels 104 that are oriented in an upright orientation 200, because when the pair of display panels 104 are each oriented in an upright orientation 200, the side edges 306(1) and 306(2) are parallel. In counter-rotated display panels 104, an angle within the range of 0 degrees and 180 degrees (exclusive) is formed between the side edges 306(1) and 306(2) of the pair of display panels 104(1) and 104(2).

[0025] The front view 300 of FIG. 3 also shows that the first display panel 104(1) and the second display panel 104(2) are each rotated by a substantially equal amount of rotation θ. Rotating each panel by a different amount of rotation may cause additional problems that degrade the user's viewing experience. Therefore, each panel 104 should be rotated by a substantially equal amount of rotation θ. As used herein, "substantially equal amounts of rotation" means within 2 degrees. Thus, if the first display panel 104(1) is rotated 45 degrees and the second display panel 104(2) is rotated 44 degrees, they are considered to be rotated by a substantially equal amount of rotation herein.

[0026] The amount of rotation θ by which each panel 104 is rotated may be in the range of 10 degrees to 80 degrees, relative to the upright panel orientation 200. A rotation amount outside this range may reduce the extent to which the benefit of the counter-rotated panel orientation is perceived by the user 102. In other words, visually distracting defects that appear in an axis-aligned manner (e.g., along the vertical and horizontal directions of the display panel) may remain noticeable to the user 102 if the display panels are rotated by a small amount (e.g., less than 10 degrees) or too much (e.g., 80 to 90 degrees). In some embodiments, for a pair of counter-rotated orientation display panels 104, the amount of rotation θ by which each display panel 104 is rotated may be in the range of 20 to 70 degrees, in the range of 30 to 60 degrees, or in the range of 40 to 50 degrees, relative to the upright panel orientation 200. In some embodiments, the display panels 104 may be counter-rotated by a rotation amount θ equal to 45 degrees, relative to the upright panel orientation 200. A 45 degree rotation may provide the greatest mitigation of panel artifacts depending on the widest FOV 308 as well as the pixel layout / pattern of the display panel 104, as shown in the front view 300 of FIG. 3. A typical pixel pattern is one in which pixels are arranged in vertical and horizontal columns on the display panel. For this type of pixel pattern, a 45 degree rotation may provide the greatest mitigation of panel artifacts. However, other pixel patterns that do not have pixels arranged in vertical columns and horizontal rows are contemplated. For these other types of pixel patterns, a different amount of rotation θ may optimally mitigate panel artifacts. Additionally, a rotation amount between 10 degrees and less than 45 degrees may be optimal to follow the contours of the nose 312 of the user 102, which may allow for optimized ergonomic benefits (e.g., moving the display panel 104 as close as possible to the face of the user 102).

[0027] Various advantages realized by orienting the display panel 104 in a counter-rotated panel orientation will now be described. One advantage is an increased (e.g., wider) FOV 308. This increased FOV 308 may be provided to a viewing user "at will" because pixels not utilized in an existing HMD 100 having a display panel oriented in an upright panel orientation 200, as shown in FIG. 3, can utilize the display panel 104 when their orientation is rotated in the opposite direction. For example, pixels located at the corners of the display panel may not be utilized when the display panel is oriented in an upright panel orientation 200 due to the inverse pincushion distortion caused by the lens, and because the HMD "cuts" the corners and does not utilize the pixels at the corners to provide a more aesthetically pleasing profile of the image to the user. The front view 300 of FIG. 3 shows how the FOV 308 may be increased to a maximum in the horizontal dimension of rectangular shaped display panels 104 when these display panels 104 are rotated by a rotation amount θ equal to 45 degrees relative to the upright panel orientation 200. The increased FOV 308 in the horizontal dimension (i.e., in the positive and negative X directions relative to the upright panel orientation 200) may, in fact, provide a more realistic sensation to the viewing user 102, who in the real world can see over a much wider range of FOV using peripheral vision.

[0028] Another benefit realized by orienting the display panel 104 in a counter-rotated panel orientation is the mitigation of panel artifacts (e.g., stripes or bands that run the length of the panel) by disguising them so that they are no longer visually distracting. To explain, display panels often contain defects called panel artifacts, which often occur with axis alignment specifications (i.e., correlated with horizontal or vertical on the display panel). At least one type of panel artifact, such as light and dark stripes or bands that run vertically the length of the display, is caused by a display driving scheme called "inversion," which uses alternating positive and negative voltage waveforms to drive the display every frame. Although the positive and negative voltages are meant to be applied with equal amounts of voltage, in practice they are usually applied with different voltages. The voltage difference between the positive and negative cycles of this alternating voltage waveform causes a flick output for every pixel column every frame as the image is displayed on the display panel. This is visually perceived as vertical stripes (e.g., light and dark stripes, crossed stripes) that are unrelated to the content being displayed in the image. These stripes are visually distracting to the user. In an HMD, this visual distraction is even more pronounced due to the unique way in which the user's head and eyes are counter-rotated while using the HMD. Furthermore, with an upright oriented display panel, both eyes will see light and dark stripes in the same location on the display across the frame, causing visual distraction for the user.

[0029] Another example of a panel artifact is caused by the vertical alignment of red, green, and blue pixels, which is manifested on an upright oriented display panel as red, green, and blue colored bands running the length of the upright oriented display panel that are seen by both eyes in the same location. This can be noticeable if the user 102 is moving their head at a precise rotational rate.

[0030] By orienting the display panel 104 in a counter-rotated panel orientation, the panel artifacts described above are mitigated. FIG. 3 shows a first group of pixels 310(1) on a first display panel 104(1) and a second group of pixels 310(2) on a second display panel 104(2) as oriented when the panel 104 is in a counter-rotated panel orientation. This counter-rotated panel orientation causes the panel defects described above to appear different to each eye, which actually disguises the panel defects and makes them unnoticed by the viewing user 102. This is because the pixel groups 310(1) and 310(2) are no longer aligned in each eye due to their counter-rotated orientation, which means that each eye sees a different defect for its respective location in a scene imaged on the counter-rotated display panel 104, and the user 102 consequently no longer perceives the panel defect in the scene. While a purely random distribution of red, green, and blue pixels may be optimal for optimized mitigation of panel defects, it is recognized that a purely random distribution may be infeasible and / or cost prohibitive. The counter-rotated panel orientation is sufficient to cause the pixels to be misaligned, thereby preventing stereo fusion in which both eyes see the same pixel defect at the same location on both panels 104 at the same time. Even if some defects align themselves to both counter-rotated display panels 104 over consecutive frames, these defects will appear on a per-pixel level and will not be noticed by the user 102 wearing the HMD 100 due to the counter-rotated panel orientation. Thus, the counter-rotated panel orientation shown in FIG. 3 shifts the aforementioned panel defects vertically and horizontally in the FOV of the user 102, meaning that the user 102 does not see the same panel defects in both eyes as a result, and the user's 102 field of view will not "snag" over the panel defects, so that they remain unnoticeable.

[0031] Another advantage realized by orienting the display panels 104 in a counter-rotated panel orientation is improved ergonomics. This can be seen in the top view 302 of FIG. 3 where the nose 312 of the user 102 can pass through the frontal plane of each of the coplanar display panels 104. Upright oriented display panels often do not provide enough space between the display panels, meaning that the display panels cannot be brought closer to the user's face than the point where they compete with the user's nose 312. By orienting the display panels 104 in a counter-rotated orientation, sufficient space for the nose 312 of the user 102 is provided between the display panels 104, which can then be moved closer to the user's 102 face. That is, the depth of the HMD 100 can be further reduced compared to existing HMDs having upright oriented display panels directly in front of the user's 102 eyes. A smaller, lighter, and closer fitting HMD 100 improves the ergonomics of the HMD 100 by making the HMD 100 more comfortable to wear for longer periods of time and / or by allowing it to be used by younger users (e.g., children) who may be more sensitive to heavier / bulkier HMDs 100.

[0032] An additional top view 304 of Figure 3 illustrates yet another orientation of the display panels 104(1) and 104(2) that can be provided by tilting each display panel 104 about an axis (e.g., the Y-axis) that is orthogonal to the transverse plane of the upright panel orientation 200. For example, the first display panel 104(1) can be tilted in a first direction (e.g., a counterclockwise direction) about the orthogonal axis relative to the transverse plane, and the second display panel 104(2) can be tilted in a second direction (e.g., a clockwise direction) opposite the first direction about the orthogonal axis relative to the transverse plane. Additionally or alternatively, the display panels 104 can be tilted about a second axis (e.g., the X-axis) that is orthogonal to the midsagittal plane of the upright panel orientation 200.

[0033] The processes described herein are illustrated as a collection of blocks in a logic flow graph, which represent a sequence of operations that can be implemented in hardware, software, firmware, or combinations thereof (i.e., logic). In the software context, the blocks represent computer-executable instructions that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order and / or in parallel to implement a process.

[0034] 4 is a flow diagram of an exemplary process 400 for sending camera pose data with a counter-rotated camera orientation to a running application, resampling frames output by the application without requiring rotational adjustments based on the camera pose data or the orientation of the display panel, and outputting pixel data for the resampled frames to a frame buffer for display on a counter-rotated display panel. For discussion purposes, process 400 will be described with reference to the previous figure.

[0035] 4 illustrates an application 407, which may represent a video game application or any other type of graphics application. The application 407 may be executed on the HMD 100 itself, or on a computing device (e.g., a personal computer (PC), a game console, etc.) associated with and coupled to the HMD 100 as part of an HMD system. The application 407 may thus be configured to output a series of frames that are resampled by a compositor 403 of the HMD 100 (or HMD system) and ultimately displayed as an image on the display panel(s) 104 of the HMD 100. Prior to rendering a frame, the application 407 may receive various types of data from the compositor 403. At least one type of data that the compositor 403 sends to the application 407 for use by the application 407 in rendering frames is camera pose data 405, which includes a set of poses having positions and orientations of a pair of virtual cameras. To provide rotated and / or tilted camera orientations to application 407, application 407 may, in some embodiments, signal its capabilities to compositor 403 (e.g., indicating that it is capable of receiving more complex camera orientations than an untilted upright camera orientation). The graphics logic of HMD 100 may be asynchronous or synchronous. In an asynchronous system, compositor 403 runs separately (on a separate asynchronous thread) from application 407 on the graphics processing unit(s) (GPU(s)) of HMD 100 (or HMD system).

[0036] At 402, a compositor 403 of the HMD 100 (or HMD system) may send camera pose data 405 to an executing application 407 configured to output a frame of pixel data. A first subset of this pixel data is associated with the first display panel 104(1), a first image, a first virtual camera, and / or a first eye of a user wearing the HMD 100, while a second subset of this pixel data is associated with the second display panel 104(2), a second image, a second virtual camera, and / or a second eye of a user wearing the HMD 100. The camera pose data 405 informs the application 407 how to render the next frame in a sequence of frames according to the camera pose data 405. The operation(s) at block 402 may be executed as part of an application programming interface (API) call.

[0037] As indicated by sub-block 404, camera pose data 405 may include a first rotated camera orientation associated with a first image for a frame rendered on first display panel 104(1), which specifies (or indicates) that a first virtual camera of a pair of virtual cameras is rotated in a first direction (e.g., a counterclockwise direction) relative to an upright camera orientation. The amount of rotation and direction specified in the first rotated camera orientation may be based on the amount of rotation and direction of first display panel 104(1) with respect to the rotated panel's orientation. That is, if first display panel 104(1) is rotated 45 degrees clockwise relative to the upright panel orientation 200, then the first rotated camera orientation may indicate that the first virtual camera is rotated 45 degrees counterclockwise relative to the upright camera orientation. The first rotated camera orientation may further specify (or indicate) that when the first display panel 104(1) is tilted in that orientation, as described herein, the first virtual camera is tilted.

[0038] As indicated by sub-block 406, camera pose data 405 may further include a second rotated camera orientation associated with a second image for a frame rendered on second display panel 104(2), which specifies (or indicates) that a second virtual camera of the pair of virtual cameras is rotated in a second direction (e.g., a clockwise direction) opposite the first direction, relative to the upright camera orientation. The amount of rotation and direction specified in the second rotated camera orientation may be based on the amount of rotation and direction of the second display panel 104(2) with respect to the rotated panel's orientation. That is, if the second display panel 104(2) is rotated 45 degrees counterclockwise relative to the upright panel orientation 200, then the second rotated camera orientation may indicate that the second virtual camera is rotated 45 degrees clockwise relative to the upright camera orientation. The second rotated camera orientation may further specify (or indicate) that the second virtual camera is tilted if the second display panel 104(2) happens to be tilted in that orientation, as described herein.

[0039] At 408, the compositor 403 may receive frames including a first image 409(1) (e.g., a left image) and a second image 409(2) (e.g., a right image) from the application 407. It should be understood that these images 409(1) and 409(2) (collectively 409) may be received from the application 407 and by the compositor 403 as pixel data. The pixel data for each image 409 may, in some embodiments, include a two-dimensional array of per-pixel values ​​(e.g., color values). In some embodiments, the pixel data further includes additional data or metadata, such as a depth value. In some embodiments, the pixel data may include data for each pixel represented by a single color and a set of alpha values ​​(e.g., one color value for a red channel, one color value for a green channel, one color value for a blue channel, and one or more values ​​for one or more alpha channels). As shown in FIG. 4, the pair of images 409 have been rendered by the application 407 according to camera pose data 405 that includes counter-rotating camera orientations. Thus, the scene in the pair of images 409 is counter-rotated by a corresponding amount of rotation based on the counter-rotation camera orientation. In other words, the application 407 naturally renders the counter-rotation camera orientation of the virtual camera.

[0040] At 410, compositor 403 may resample the frame without any rotational adjustments based on the camera pose data or the orientation of the display panel. Resampling may include applying one or more digital transformations to pixel data of the frame output by application 407 to modify / tweak / correct the frame rendered by the application before it is output to a frame buffer for presentation on display panel 104. For example, compositor 403 may resample the frame to obtain a resampled frame by modifying the pixel data of frame 409(1) to obtain changed pixel data of the resampled frame. Notably, this modification of the pixel data of the frame at block 410 does not involve any rotational adjustment of the pixel data based on camera pose data 405 or the orientation of display panel 104. For example, the resampling operation(s) in block 410 may include, but are not limited to, correcting for lens distortion (e.g., chromatic distortion), applying a reprojection transformation to modify the pixel data according to a predicted head pose of the user 102, and / or other mathematical operations that modify the pixel data to create a resampled frame that is improved or otherwise corrected from a visual standpoint. The reprojection may include rotational transformations that effectively rotate the frame about a reference axis, but these rotational transformations are based on the predicted head pose of the user 102 at the reprojection, and not based on the camera pose data 405 with respect to the virtual camera or the rotation of the display panel 104 relative to upright. By not making rotational adjustments in block 410 based on the camera pose data 405 or the orientation of the display panel 104, the rendering pipeline is made more efficient as compared to other embodiments described herein that involve rotational adjustments (clockwise / counterclockwise) in the resampling step.

[0041] At 412, compositor 403 may output the modified pixel data associated with the resampled frame to a frame buffer so that the resampled frame may be output as a pair of images on counter-rotated display panels 104(1) and 104(2) of HMD 100.

[0042] At 414, a pair of images based on the resampled frames may be displayed on counter-rotated display panels 104(1) and 104(2) of HMD 100. As shown in FIG. 4, the image presented on display panel 104 is in an upright orientation despite the counter-rotated orientation of display panel 104. This is due to the counter-rotated camera orientation provided to application 407 in camera pose data 405. It should be understood that while process 400 involves rendering a single frame, process 400 may be repeated to render multiple successive frames during execution of application 407.

[0043] An example of an additional benefit provided from using a counter-rotating camera orientation, as in process 400, is the mitigation of sampling artifacts (e.g., jagged lines) in the displayed image. To illustrate, application 407 may itself perform various sampling steps, such as supersampling, anti-aliasing (e.g., resolving a multi-sample anti-aliasing (MSAA) buffer from 16 samples per pixel (4×4) to 1 sample per pixel), in the process of rendering the frame including images 409(1) and 409(2). These sampling steps in application 407 introduce sampling artifacts that tend to align themselves vertically and horizontally in the frame. In addition, compositor 403 typically has anywhere between 8-11 milliseconds to resample a frame received from application 407, and therefore uses fast filters (e.g., box filters, linear filters, etc.) to speed up the resampling operation. These types of filters often have axially aligned properties, causing sampling artifacts to appear in the axially aligned resampled frame (vertically and horizontally in the frame). Because vertically and horizontally aligned objects are typically ubiquitous in virtual reality environments, the resulting image, when an upright camera orientation is used for the virtual camera, often contains many sampling artifacts along the vertical and horizontal directions of the image, such as jagged horizontal lines and jagged vertical lines that appear as "stairs" in the displayed image. This can be quite visually distracting, especially when these jagged lines are coming from the field of view. In contrast, lines running diagonally across the image often appear smooth and have no discernible jagged appearance.For example, as described in process 400, by specifying a counter-rotation camera orientation in the camera pose data 405 provided to application 407 for rendering a frame, these sampling artifacts that end up at non-vertical and non-horizontal angles in the final image are mitigated, making objects in the image more aesthetically pleasing.

[0044] 5 is a diagram illustrating a view 502 of a displayed image from the perspective of a viewing user 102 through the lenses 108 of an HMD 100, according to an embodiment disclosed herein. The user 102 is unable to determine that the display panel 104 is not in an upright panel orientation 200, yet still obtains many of the benefits described herein (e.g., mitigated panel artifacts, mitigated sampling artifacts, increased FOV, etc.).

[0045] 6 is a flow diagram of an example process 600 for sending camera pose data to a running application in an upright camera orientation, rotationally adjusting and resampling frames output by the application in a clockwise / counterclockwise direction, and outputting pixel data of the resampled frames to a frame buffer for output to a counter-rotated display panel. For discussion purposes, process 600 will be described with reference to the previous figure.

[0046] At 602, a compositor 403 of the HMD 100 (or HMD system) may send camera pose data 605 to an executing application 407 configured to output a frame of pixel data. A first subset of this pixel data is associated with a first display panel 104(1), a first image, a first virtual camera, and / or a first eye of a user wearing the HMD 100, while a second subset of this pixel data is associated with a second display panel 104(2), a second image, a second virtual camera, and / or a second eye of a user wearing the HMD 100. The camera pose data 605 informs the application 407 on how to render the next frame in a sequence of frames according to the camera pose data 605. The operation(s) at block 602 may be executed as part of an API call. In some embodiments, the compositor 403 may further request the application 407 to render additional pixels to those that the compositor would normally request the application 407 to render if the display panel was upright oriented. This is because the image of the frame will be counter-rotated in the resampling operation(s) in block 610 below, and the additional pixels can be used to fill in gaps resulting from rotating the image in orientation during resampling.

[0047] As indicated by sub-block 604, camera pose data 605 may include a first upright camera orientation associated with a first image for a frame rendered on first display panel 104(1), which specifies (or indicates) that a first virtual camera of the pair of virtual cameras is in an upright camera orientation. The first upright camera orientation may further specify (or indicate) that the first virtual camera is tilted if first display panel 104(1) is tilted in that orientation, as described herein.

[0048] As indicated by sub-block 606, the camera pose data 605 may further include a second upright camera orientation associated with a second image for a frame rendered on the second display panel 104(2), which second upright camera orientation specifies (or indicates) that the second virtual camera of the pair of virtual cameras is in an upright camera orientation. The second upright camera orientation may further specify (or indicate) that the second virtual camera is tilted if the second display panel 104(2) happens to be tilted in that orientation, as described herein.

[0049] At 608, compositor 403 may receive frames including a first image 609(1) (e.g., a left image) and a second image 609(2) (e.g., a right image) from application 407. It should be understood that these images 609(1) and 609(2) (collectively 609) may be received as pixel data from application 407 and by compositor 403. Further, as shown in FIG. 6, pair of images 609 has been rendered by application 407 according to camera pose data 605 that includes an upright camera orientation. Thus, the scene in pair of images 609 is upright and not rotated, as in process 400.

[0050] At 610, the compositor 403 may resample the frame by rotating the image 609 of the frame in the opposite direction. For example, the compositor 403 may resample the frame by rotating a first subset of pixel data of the frame associated with the first virtual camera in a first direction (e.g., a counterclockwise direction) to obtain a first subset of modified pixel data of the resampled frame corresponding to the first rotated image 611(1) and rotating a second subset of pixel data of the frame associated with the second virtual camera in a second direction opposite the first direction (e.g., a clockwise direction) to obtain a second subset of modified pixel data of the resampled frame corresponding to the second rotated image 611(2). These rotation adjustments may utilize an XY mapping of pixels and may perform mathematical calculations to move each pixel from a starting coordinate (X1, Y1) to a destination coordinate (X2, Y2). The resampling operation(s) in block 610 may include additional digital transformations such as, but not limited to, correcting lens distortion (e.g., chromatic distortion), applying a reprojection transformation to modify the pixel data according to the predicted head pose of the user 102, etc.

[0051] At 612, compositor 403 may output the modified pixel data associated with the resampled frame to a frame buffer so that the resampled frame may be output as a pair of images on counter-rotated display panels 104(1) and 104(2) of HMD 100.

[0052] At 614, a pair of rotated images 611 of the resampled frames may be displayed on the counter-rotated display panels 104(1) and 104(2) of the HMD 100. As shown in FIG. 6, the image presented on the display panel 104 is in an upright orientation despite the counter-rotated orientation of the display panel 104. This is due to the rotational adjustment in the resampling operation(s) in block 610 that compensates for the counter-rotated camera orientation provided to the application 407 in the camera pose data 605. It should be understood that the process 600 involves rendering a single frame, and that the process 600 may be repeated to render multiple successive frames during the execution of the application 407. While the sampling artifacts described herein may manifest themselves in the images displayed in the embodiment of FIG. 6, the counter-rotated orientation of the display panel 104 provides the remaining advantages described herein, such as mitigation of panel artifacts, increased FOV, and improved ergonomics.

[0053] FIG. 7 illustrates a front view 700 of the pair of display panels 104(1) and 104(2) in an upright panel orientation 200, as well as a top view 702 of the pair of display panels 104(1) and 104(2) in the upright panel orientation 200, and an additional top view 704 of the pair of display panels 104(1) and 104(2) after tilting the display panels. The orientation of the display panels 104(1) and 104(2) in FIG. 7 is presented in the context of the embodiment described in FIG. 8 below. That is, in this embodiment, the display panels 104 are not rotated in opposition, but instead are oriented in the upright panel orientation 200. In some embodiments, the display panels 104 may be tilted, such as by tilting each display panel 104 about an axis (e.g., the Y-axis) that is perpendicular to the transverse plane of the upright panel orientation 200. For example, the first display panel 104(1) may be tilted in a first direction (e.g., a counterclockwise direction) about an axis orthogonal to the transverse plane, and the second display panel 104(2) may be tilted in a second direction (e.g., a clockwise direction) opposite the first direction about the axis orthogonal to the transverse plane. Additionally or alternatively, the display panels 104 may be tilted about a second axis (e.g., an X-axis) that is orthogonal to the midsagittal plane of the upright panel orientation 200.

[0054] Figure 8 is a flow diagram of an exemplary process 800 for sending camera pose data for a counter-rotated camera orientation to a running application, resampling frames output by the application with rotational adjustments in the clockwise / counterclockwise direction, and outputting pixel data for the resampled frames to a frame buffer for display on an upright oriented display panel, such as the display panel shown in Figure 7. For discussion purposes, process 800 will be described with reference to the previous figure.

[0055] At 802, a compositor 403 of the HMD 100 (or HMD system) may send camera pose data 805 to an executing application 407 configured to output a frame of pixel data. A first subset of this pixel data is associated with a first display panel 104(1), a first image, a first virtual camera, and / or a first eye of a user wearing the HMD 100, while a second subset of this pixel data is associated with a second display panel 104(2), a second image, a second virtual camera, and / or a second eye of a user wearing the HMD 100. The camera pose data 805 informs the application 407 how to render the next frame in a sequence of frames according to the camera pose data 805. The operation(s) at block 802 may be executed as part of an API call. In some embodiments, compositor 403 may further request application 407 to render additional pixels for pixels it would normally request application 407 to render if compositor 403 sends an upright camera orientation in camera pose data 805. This is because the image for the frame will be rotated inversely in the resampling operation(s) in block 810 below to counter the counter-rotated camera orientation in camera pose data 805, and the additional pixels can be used to fill in gaps resulting from rotating the image orientation during resampling.

[0056] As indicated by sub-block 804, the camera pose data 805 may include a first rotated camera orientation associated with a first image for a frame rendered on the first display panel 104(1), the first rotated camera orientation specifying (or indicating) that a first virtual camera of a pair of virtual cameras is rotated in a first direction (e.g., a counterclockwise direction) relative to an upright camera orientation. The amount of rotation specified in the first rotated camera orientation may be within a range of 10 to 80 degrees, within a range of 20 to 70 degrees, within a range of 30 to 60 degrees, or within a range of 40 to 50 degrees, relative to an upright camera orientation. A 45 degree rotation may provide the greatest reduction in sampling artifacts. The first rotated camera orientation may further specify (or indicate) that the first virtual camera is tilted when the first display panel 104(1) is tilted in that orientation, as described herein.

[0057] As indicated by sub-block 806, the camera pose data 805 may further include a second rotated camera orientation associated with a second image for a frame rendered on the second display panel 104(2), which second rotated camera orientation specifies (or indicates) that a second virtual camera of the pair of virtual cameras is rotated in a second direction (e.g., a clockwise direction) opposite the first direction, relative to the upright camera orientation. The amount of rotation specified in the second rotated camera orientation may be substantially equal to the amount of rotation specified in the first rotated camera orientation, or may be within a rotation range specified with reference to sub-block 804. The second rotated camera orientation may further specify (or indicate) that the second virtual camera is tilted if the second display panel 104(2) happens to be tilted in that orientation, as described herein.

[0058] At 808, the compositor 403 may receive frames including a first image 809(1) (e.g., a left image) and a second image 809(2) (e.g., a right image) from the application 407. It should be understood that these images 809(1) and 809(2) (collectively 809) may be received as pixel data from the application 407 and by the compositor 403. Furthermore, as shown in FIG. 8, the pair of images 809 has been rendered by the application 407 according to the camera pose data 805 including the counter-rotating camera orientation. Thus, the scene in the pair of images 809 is counter-rotated by a corresponding amount of rotation based on the counter-rotating camera orientation. In other words, the application 407 naturally renders the counter-rotating camera orientation of the virtual camera.

[0059] At 810, compositor 403 may resample the frame by rotating the image 809 of the frame in the opposite direction. For example, compositor 403 may rotate a first subset of pixel data of a frame associated with a first virtual camera in a first direction (e.g., a clockwise direction) to obtain a first subset of modified pixel data of a resampled frame corresponding to a first rotated image 811(1), and rotate a second subset of pixel data of a frame associated with a second virtual camera in a second direction opposite the first direction (e.g., a counterclockwise direction) to obtain a second subset of modified pixel data of a resampled frame corresponding to a second rotated image 811(2), and resample the frame. The amount of rotation by which the second subset of pixel data is rotated may be substantially equal to the amount of rotation by which the first subset of pixel data is rotated and may be within a rotation range specified for subblock 804. The resampling operation(s) in block 810 may include additional digital transformations such as, but not limited to, correcting lens distortion (e.g., chromatic distortion), applying a reprojection transformation to modify the pixel data according to the predicted head pose of the user 102, etc.

[0060] At 812, compositor 403 may output the modified pixel data associated with the resampled frame to a frame buffer so that the resampled frame may be output as a pair of images on upright-oriented display panels 104(1) and 104(2) of HMD 100 as shown in FIG. 7.

[0061] At 814, a pair of rotated images 811 may be displayed on the upright oriented display panels 104(1) and 104(2) of the HMD 100. As shown in FIG. 8, the image presented on the display panel 104 is in an upright orientation. This is due to the rotational adjustment in the resampling operation(s) at block 810 that compensates for the counter-rotated camera orientation provided to the application 407 in the camera pose data 805. It should be understood that the process 800 involves rendering a single frame, and that the process 800 may be repeated to render multiple successive frames during execution of the application 407. Although the process 800 is used in an HMD 100 having an upright oriented display panel 104, it mitigates sampling artifacts (e.g., jagged lines) as described herein to improve the display performance of the HMD 100.

[0062] FIG. 9 illustrates exemplary components of an HMD 100 (or an HMD system including the HMD 100), such as a VR headset, according to embodiments disclosed herein that may be embodied. The HMD 100 may be implemented as a standalone device to be worn by the user 102 (e.g., on the head of the user 102). In some embodiments, the HMD 100 may be head-mountable by allowing the user 102 to fasten the HMD 100 to the user's head using a fastening mechanism (e.g., an adjustable band) sized to fit around the head of the user 102. In some embodiments, the HMD 100 includes a virtual reality (VR) or augmented reality (AR) headset that includes near-eye or near-to-eye display(s). Thus, the terms "wearable device," "wearable electronic device," "VR headset," "AR headset," and "head mounted display (HMD)" may be used interchangeably herein to refer to the device 100 of FIG. 1. However, it should be understood that these types of devices are merely examples of the HMD 100, and that the HMD 100 may be implemented in a variety of other form factors. It should also be understood that some or all of the components shown in FIG. 9 may be implemented on the HMD 100. Thus, in some embodiments, a subset of the components shown in FIG. 9 may be implemented on a computing device that is part of the HMD system but separate from the HMD 100 itself, such as a PC, a game console, or any other suitable computing device.

[0063] In the illustrated implementation, the HMD 100 includes one or more processors 902 and memory 904 (e.g., computer-readable media 904). In some implementations, the processor(s) 902 may include a central processing unit (CPU)(s), a graphics processing unit (GPU)(s), both a CPU(s) and a GPU(s), a microprocessor(s), a digital signal processor(s), or other processing units or components known in the art. Alternatively, or in addition, the functionality described functionally herein may be implemented, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like. Additionally, each of the processor(s) 902 may have its own local memory that may also store program modules, program data, and / or one or more operating systems.

[0064] The memory 904 may include volatile and non-volatile memory, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Such memory includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical memory, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, RAID storage systems, or any other medium that can be used to store the desired information and that can be accessed by a computing device. The memory 904 may be implemented as a computer-readable storage medium ("CRSM"), which may be any available physical medium that can be accessed by the processor(s) 902 to execute instructions stored in the memory 904. In one basic implementation, the CRSM may include random access memory ("RAM") and flash memory. In other implementations, the CRSM may include, but is not limited to, read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), or any other tangible medium that can be used to store desired information and that can be accessed by the processor(s) 902.

[0065] In general, the HMD 100 may include logic (e.g., software, hardware, and / or firmware, etc.) configured to implement the techniques, functions, and / or operations described herein. The computer-readable medium 904 is shown as including various modules, such as instructions, data stores, etc., that may be configured to execute on the processor(s) 902 to perform the techniques, functions, and / or operations described herein. Although several exemplary functional modules are shown as stored on the computer-readable medium 904 and executable on the processor(s) 902, the same functions may alternatively be implemented in hardware, firmware, or as a system-on-a-chip (SOC) and / or other logic.

[0066] The operating system module 906 may be configured to manage hardware within the HMD 100 and coupled to the HMD 100 for the convenience of other modules. In addition, in some instances, the HMD 100 may include one or more applications 407 stored in the memory 904 or otherwise accessible to the HMD 100. In this implementation, the application(s) 407 include video game application(s) 407. However, the HMD 100 may include any number or type of applications and is not limited to the specific example shown here. The video game application(s) 407 may be configured to initiate gameplay of a video-based interactive game (e.g., a VR game) playable by the user 102 and output frames to be rendered on the display panel 104 of the HMD 100. The compositor 403, in combination with other logic of the HMD 100, may be configured to implement the techniques described herein to provide the camera pose data 405 / 605 / 805 to the application 407, resample the frames, and output pixel data for the resampled frames to a frame buffer.

[0067] In general, the HMD 100 has input devices 908 and output devices 910. The input devices 908 may include control buttons. In some implementations, one or more microphones may function as the input devices 908 for receiving audio input, such as user voice input. In some implementations, one or more cameras or other types of sensors (e.g., inertial measurement units (IMUs)) may function as the input devices 908 for receiving gestural input, such as hand and / or head movements of the user 102. In some embodiments, additional input devices 908 may be provided in the form of a keyboard, keypad, mouse, touch screen, joystick, etc. In other embodiments, the HMD 100 may omit a keyboard, keypad, or other similar form of mechanical input. Instead, the HMD 100 may be implemented with a relatively simple form of input device 908, a network interface (wireless or wired based), a power source, and processing / memory capabilities. For example, a limited set of one or more input components (e.g., dedicated buttons for initiating configuration, powering on / off, etc.) may be used and the HMD 100 may thereafter be used. In one implementation, the input device(s) 908 may include controls such as basic volume control button(s) for increasing / decreasing the volume, as well as power and reset buttons.

[0068] The output device 910 may include a pair of display panels 104(1) and 104(2). The output device 910 may further include, without limitation, lighting elements (e.g., LEDs), a vibrator for producing tactile sensations, speaker(s) (e.g., headphones), etc. There may also be simple light elements (e.g., LEDs) to indicate a state, such as when the power is on.

[0069] The HMD 100 may further include a wireless unit 912 coupled to an antenna 914 to facilitate wireless connection to a network. The wireless unit 912 may implement one or more of a variety of wireless technologies, such as Wi-Fi, Bluetooth, radio frequency (RF), etc. It should be appreciated that the HMD 100 may further include a physical port to communicate with a network, connected peripheral devices (including PCs, game consoles, etc.), or other wireless networks, and to facilitate a wired connection to a plug-in network device that may be part of the HMD system.

[0070] The HMD 100 may further include an optical subsystem 916 that directs light from the display panel 104 to the user's eyes using one or more optical elements. The optical subsystem 916 may include various types and combinations of different optical elements, including, but not limited to, apertures, lenses (e.g., Fresnel lenses, convex lenses, concave lenses, etc.), filters, etc. Each of the lenses 108(1) and 108(2) may include some or all of these optical elements. In some embodiments, one or more optical elements in the optical subsystem 916 may have one or more coatings, such as an anti-reflective coating. The magnification of the image light by the optical subsystem 916 allows the display panel 104 to be physically smaller, lighter, and less power-hungry than larger displays. In addition, the magnification of the image light may increase the field of view (FOV) of the displayed content (e.g., an image). For example, the FOV of the displayed content is such that the displayed content is presented using nearly all (e.g., 120-150 degrees diagonal), and possibly all, of the user's FOV. When combined with the oppositely rotated orientation of the display panels 104(1) and 104(2), this FOV can be further increased horizontally. AR applications may have a narrower FOV (e.g., about 40 degrees FOV). The optical subsystem 916 can be designed to correct one or more optical errors, such as, but not limited to, barrel distortion, pincushion distortion, longitudinal chromatic aberration, transverse aberration, spherical aberration, field curvature, astigmatism, etc. In some embodiments, the content provided to the display panel 104 for display is pre-distorted, and the optical subsystem 916 corrects the distortion when receiving image light from the display panel 104 generated based on the content. In some embodiments, the compositor 403 is configured to resample frames in the rendering pipeline to correct the distortion, as described above.

[0071] The HMD 100 may further include one or more sensors 918, such as sensors used to generate motion, position, and orientation data. These sensors 918 may be or include gyroscopes, accelerometers, magnetometers, video cameras, color sensors, or other motion, position, and orientation sensors. The sensors 918 may also include secondary portions of sensors, such as a set of active or passive markers that may be viewed externally by a camera or color sensor to generate motion, position, and orientation data. For example, a VR headset may include multiple markers on its exterior, such as reflectors or lights (e.g., infrared or visible light) that, when viewed by an external camera or illuminated with light (e.g., infrared or visible light), may provide one or more reference points for interpretation by software to generate motion, position, and orientation data. The HMD 100 may include light sensors that are sensitive to light (e.g., infrared or visible light) projected or scattered by a base station in the environment of the HMD 100 to generate motion, position, and orientation data.

[0072] In one example, the sensor(s) 918 may include an inertial measurement unit (IMU) 920. The IMU 920 may be an electronic device that generates calibration data based on measurement signals received from accelerometers, gyroscopes, magnetometers, and / or other sensors suitable for detecting motion, correcting errors associated with the IMU 920, or some combination thereof. Based on the measurement signals, such motion-based sensors, such as the IMU 920, may generate calibration data indicative of an estimated position of the HMD 100 relative to an initial position of the HMD 100. For example, multiple accelerometers may measure translational motion (forward / backward, up / down, left / right) and multiple gyroscopes may measure rotational motion (e.g., pitch, yaw, roll). The IMU 920 may, for example, rapidly sample the measurement signals and calculate an estimated position of the HMD 100 from the sampled data. For example, the IMU 920 may integrate measurement signals received from an accelerometer over time to estimate a velocity vector, and integrate the velocity vector over time to determine an estimated position of a reference point on the HMD 100. A reference point is a point that may be used to describe the position of the HMD 100. Although a reference point may generally be defined as a point in space, in various embodiments, the reference point is defined as a point within the HMD 100 (e.g., the center of the IMU 920). Alternatively, the IMU 920 provides sampled measurement signals to an external console (or other computing device), which determines the calibration data.

[0073] The sensor 918 may operate at a relatively high frequency to provide sensor data at a high rate. For example, the sensor data may be generated at a rate of 1000 Hz (or one sensor reading every millisecond). In this manner, 1000 readings are generated per second. If the sensor generates this much data at this rate (or even faster), the data set used to predict movement becomes very large, even over a relatively short period of time, such as tens of milliseconds.

[0074] As mentioned, in some embodiments, the sensor 918 may include a light sensor that is sensitive to light emitted by a base station in the environment of the HMD 100 for purposes of tracking the position and / or orientation, posture, etc. of the HMD 100 in 3D space. Calculations of position and / or orientation may be based on timing characteristics of the light pulses and the presence or absence of light detected by the sensor 918.

[0075] The HMD 100 may further include an eye tracking module 922. A camera or other optical sensor inside the HMD 100 may capture image information of the user's eyes, and the eye tracking module 922 may use the captured information to determine the interpupillary distance, the three-dimensional (3D) position of each eye relative to the HMD 100, including the magnitude of twist and rotation (i.e., roll, pitch, and yaw) and the gaze direction of each eye (e.g., for distortion adjustment purposes). In one example, infrared light is emitted within the HMD 100 and reflected from each eye. The reflected light is received or detected by a camera of the HMD 100 and analyzed to extract eye rotation from changes in the infrared light reflected by each eye. Many methods for tracking the eyes of the user 102 may be used by the eye tracking module 922. Thus, the eye tracking module 922 may track up to six degrees of freedom (i.e., 3D position, roll, pitch, and yaw) of each eye, and at least a subset of the tracked quantities may be combined from the two eyes of the user 102 to estimate a point of gaze (i.e., a 3D location or position within the virtual scene where the user is looking). For example, the eye tracking module 922 may integrate information from past measurements, measurements identifying the position of the user 102's head, and 3D information describing the scene presented by the display panel 104. Thus, the position and orientation information of the user 102's eyes is used to determine a point of gaze within the virtual scene presented by the HMD 100 where the user 102 is looking.

[0076] The HMD 100 may further include a head tracking module 924. The head tracking module 924 may utilize one or more of the sensors 918 to track head movements, including head rotation, of the user 102, as described above. For example, the head tracking module 924 may track up to six degrees of freedom (i.e., 3D position, roll, pitch, and yaw) of the HMD 100. These calculations may be performed for each frame (or frame pair) of a series of frames (or frame pairs), so that the application 407 may determine how to render a scene in the next frame depending on the head position and orientation. In some embodiments, the head tracking module 924 and / or the compositor 403 using the head tracking module 924 are configured to predict the future position and / or orientation of the HMD 100 based on current and / or past data. This is because the application 407 is required to render a frame (or frame pair) before the user 102 actually sees the light (and therefore the image) on the display panel 104. Thus, the next frame (or frame pair) may be rendered based on this future prediction of head position and / or orientation, made at an earlier time, such as approximately 25-30 milliseconds (ms) prior to rendering the frame (or frame pair). The rotational data provided by the head tracking module 924 may be used to determine both the rotational direction of the HMD 100 and the amount of rotation of the HMD 100, in any suitable units of measurement. For example, the rotational direction may be output simply in terms of positive or negative horizontal directions and positive or negative vertical directions corresponding to left, right, up, and down. The amount of rotation may be in units of degrees, radians, etc. An angular velocity may be calculated to determine the rotational speed of the HMD 100.

[0077] Unless otherwise indicated, all numbers expressing quantities used in the specification and claims should be understood to be modified in all instances by the term "about". Accordingly, unless indicated to the contrary, the numerical parameters set forth in this specification and the appended claims are approximations that may vary depending on the desired properties sought to be obtained by the present disclosure. At the very least, and not as an attempt to limit the application of the doctrine of equivalents to the scope of the claims, each numerical parameter should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Where further clarity is needed, the term "about" has the meaning reasonably given by one of ordinary skill in the art when used in conjunction with a stated numerical value or range, i.e., within ±20% of the stated value, ±19% of the stated value, ±18% of the stated value, ±17% of the stated value, ±16% of the stated value, ±15% of the stated value, ±14% of the stated value, ±13% of the stated value, ±12% of the stated value, ±11% of the stated value, ±10% of the stated value, ±9% of the stated value, ±8% of the stated value, ±7% of the stated value, ±6% of the stated value, ±5% of the stated value, ±4% of the stated value, ±3% of the stated value, ±2% of the stated value, or ±1% of the stated value.

[0078] Although the present subject matter has been described in language specific to structural features, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features described. Rather, the specific features are disclosed as exemplary forms of implementing the claims. The invention as originally claimed in the present application is set forth below. [1] A head mounted display (HMD) system, comprising: The HMD, Housing and a pair of display panels mounted within the housing, the pair including a first display panel and a second display panel; the first display panel is oriented in a first orientation by rotating the first display panel in a first direction about a first axis perpendicular to a frontal plane of the first display panel relative to an upright panel orientation; The HMD system, wherein the second display panel is oriented in a second orientation by rotating the second display panel relative to the upright panel orientation in a second direction opposite to the first direction about a second axis perpendicular to the frontal plane of the second display panel. [2] one or more processors; stored in a memory of the HMD system; sending camera pose data associated with a pair of virtual cameras to an application configured to output frames, the camera pose data comprising: a first rotated camera orientation that specifies that a first virtual camera of the pair of virtual cameras is rotated in the second direction relative to an upright camera orientation; a second rotated camera orientation specifying that a second virtual camera of the pair of virtual cameras is rotated in the first direction relative to the upright camera orientation; receiving the frame from the application; resampling the frame by modifying pixel data of the frame to obtain modified pixel data of the resampled frame; The HMD system of [1], further comprising computer-executable instructions executable by the one or more processors to: output the modified pixel data to a frame buffer. [3] The HMD system according to [2], wherein the application is a video game application. [4] one or more processors; stored in a memory of the HMD system; sending camera pose data associated with a pair of virtual cameras to an application configured to output frames, the camera pose data comprising: a first upright camera orientation that specifies that a first virtual camera of the pair of virtual cameras is in an upright camera orientation; a second upright camera orientation specifying that a second virtual camera of the pair of virtual cameras is in the upright camera orientation; receiving the frame from the application; resampling the frame to obtain a resampled frame, said obtaining comprising: rotating a first subset of pixel data of the frame associated with the first virtual camera in the second orientation to obtain a first subset of modified pixel data of the resampled frame; and obtaining a second subset of pixel data of the frame associated with the second virtual camera by rotating the second subset of pixel data of the frame associated with the second virtual camera about the first orientation to obtain the second subset of modified pixel data of the resampled frame; The HMD system of [1], further comprising computer-executable instructions executable by the one or more processors to: output the modified pixel data to a frame buffer. [5] The HMD system of [1], wherein the first display panel is oriented in the first orientation and the second display panel is oriented in the second orientation by rotating each display panel by a substantially equal amount of rotation. [6] The HMD system of [5], wherein the substantially equal amount of rotation is within a range of 10 to 80 degrees relative to the upright panel orientation. [7] The HMD system described in [1], wherein the pair of display panels are in the same plane. [8] the first display panel is oriented in the first orientation by further tilting the first display panel about an axis perpendicular to a transverse plane of the upright panel orientation; The HMD system of claim 1, wherein the second display panel is oriented in the second orientation by further tilting the second display panel in a direction opposite to that of the first display panel about the axis perpendicular to the transverse plane of the upright panel orientation. [9] A head mounted display (HMD) system, comprising: HMD is, Housing and an HMD system comprising: a pair of display panels mounted within the housing, the pair of display panels including a first display panel and a second display panel adjacent to the first display panel, the first display panel and the second display panel each being oriented such that side edges of the first display panel and side edges of the second display panel are not parallel.

[10] the first display panel is oriented in a first orientation by rotating the first display panel in a first direction about a first axis perpendicular to a frontal plane of the first display panel by an amount of rotation of 10 degrees or greater relative to an upright panel orientation; The HMD system of [9], wherein the second display panel is oriented in a second orientation by rotating the second display panel by the rotation amount relative to the upright panel orientation in a second direction opposite to the first direction about a second axis perpendicular to the frontal plane of the second display panel.

[11] one or more processors; stored in a memory of the HMD system; sending camera pose data associated with a pair of virtual cameras to an application configured to output frames, the camera pose data comprising: a first rotated camera orientation indicating that a first virtual camera of the pair of virtual cameras is rotated in a first direction relative to an upright camera orientation; a second rotated camera orientation indicating that a second virtual camera of the pair of virtual cameras is rotated in a second direction opposite to the first direction relative to the upright camera orientation; receiving the frame from the application; resampling the frame by modifying pixel data of the frame to obtain modified pixel data of the resampled frame; The HMD system of [9], further comprising computer-executable instructions executable by the one or more processors to: output the modified pixel data to a frame buffer.

[12] The HMD system of

[11] , wherein the application is a video game application.

[13] one or more processors; stored in a memory of the HMD system; sending camera pose data associated with a pair of virtual cameras to an application configured to output frames, the camera pose data comprising: a first upright camera orientation indicating that a first virtual camera of the pair of virtual cameras is in an upright camera orientation; a second upright camera orientation indicating that a second virtual camera of the pair of virtual cameras is in the upright camera orientation; receiving the frame from the application; resampling the frame to obtain a resampled frame, said obtaining comprising: rotating a first subset of pixel data of the frame associated with the first virtual camera in a first orientation to obtain a first subset of modified pixel data of the resampled frame; obtaining a second subset of pixel data of the frame associated with the second virtual camera by rotating the second subset of pixel data of the frame in a second direction opposite to the first direction to obtain the second subset of modified pixel data of the resampled frame; The HMD system of [9], further comprising computer-executable instructions executable by the one or more processors to: output the modified pixel data to a frame buffer.

[14] The HMD system of

[13] , wherein the application is a video game application.

[15] The HMD system of [9], wherein the HMD is at least one of a virtual reality headset or an augmented reality headset.

[16] the first display panel is tilted in a first direction about an axis perpendicular to a transverse plane of an upright panel orientation; The HMD system of [9], wherein the second display panel is tilted in a second direction opposite to the first direction about the axis perpendicular to the transverse plane of the upright panel orientation.

[17] A head mounted display (HMD) system, comprising: HMD is, Housing and a pair of display panels mounted within the housing, the pair of display panels including a first display panel and a second display panel, the first display panel and the second display panel each oriented in an upright panel orientation; one or more processors; stored in a memory of the HMD system; sending camera pose data associated with a pair of virtual cameras to an application configured to output frames, the camera pose data comprising: a first rotated camera orientation indicating that a first virtual camera of the pair of virtual cameras is rotated in a first direction relative to an upright camera orientation; a second rotated camera orientation indicating that a second virtual camera of the pair of virtual cameras is rotated in a second direction opposite to the first direction relative to the upright camera orientation; receiving the frame from the application; resampling the frame to obtain a resampled frame, said obtaining comprising: rotating a first subset of pixel data of the frame associated with the first virtual camera in the second orientation to obtain a first subset of modified pixel data of the resampled frame; and obtaining a second subset of pixel data of the frame associated with the second virtual camera by rotating the second subset of pixel data of the frame associated with the second virtual camera about the first orientation to obtain the second subset of modified pixel data of the resampled frame; outputting the modified pixel data to a frame buffer; and computer-executable instructions executable by the one or more processors to perform the above steps.

[18] the first display panel is tilted about an axis perpendicular to a transverse plane of the upright panel orientation; the second display panel is tilted in a direction opposite to that of the first display panel about the axis perpendicular to the transverse plane of the upright panel orientation; the first rotated camera orientation further indicating that the first virtual camera is tilted; The HMD system of

[17] , wherein the second rotational camera orientation further indicates that the second virtual camera is tilted.

[19] 17. The HMD system of claim 17, wherein resampling the frame includes rotating each of the first subset of pixel data and the second subset of pixel data by a substantially equal amount of rotation.

[20] 20. The HMD system of claim 19, wherein the substantially equal amounts of rotation are within a range of 40 to 50 degrees relative to the upright camera orientation.

Claims

1. A head mounted display (HMD) system, comprising: An HMD that can be worn on a user's head; one or more processors; and computer-executable instructions stored in a memory of the HMD system and executable by the one or more processors; The HMD includes: Housing and a pair of display panels mounted within the housing, the pair including a first display panel and a second display panel, the first display panel being mounted at a position corresponding to a left eye of the user wearing the HMD on the head, and the second display panel being mounted at a position corresponding to a right eye of the user wearing the HMD on the head; When the user wearing the HMD on the head views the pair of display panels, the first display panel is oriented in a first orientation by rotating the first display panel in a clockwise direction about a first axis perpendicular to a frontal plane of the first display panel relative to an upright panel orientation; when the user wearing the HMD on the head views the pair of display panels, the second display panel is oriented in a second orientation by rotating the second display panel relative to the upright panel orientation in a counterclockwise direction about a second axis perpendicular to a frontal plane of the second display panel; The computer executable instructions include: sending camera pose data associated with a pair of virtual cameras to an application configured to output frames; The camera pose data is a first rotated camera orientation that specifies that a first virtual camera of the pair of virtual cameras is rotated in the counterclockwise direction relative to an upright camera orientation; a second rotated camera orientation specifying that a second virtual camera of the pair of virtual cameras is rotated in the clockwise direction relative to the upright camera orientation; receiving the frame from the application; resampling the frame by modifying pixel data of the frame to obtain modified pixel data of the resampled frame; outputting the modified pixel data to a frame buffer; and the one or more processors.

2. The HMD system of claim 1 , wherein the application is a video game application.

3. A head mounted display (HMD) system, comprising: An HMD that can be worn on a user's head; one or more processors; and computer-executable instructions stored in a memory of the HMD system and executable by the one or more processors; The HMD includes: Housing and a pair of display panels mounted within the housing, the pair including a first display panel and a second display panel, the first display panel being mounted at a position corresponding to a left eye of the user wearing the HMD on the head, and the second display panel being mounted at a position corresponding to a right eye of the user wearing the HMD on the head; When the user wearing the HMD on the head views the pair of display panels, the first display panel is oriented in a first orientation by rotating the first display panel in a clockwise direction about a first axis perpendicular to a frontal plane of the first display panel relative to an upright panel orientation; when the user wearing the HMD on the head views the pair of display panels, the second display panel is oriented in a second orientation by rotating the second display panel relative to the upright panel orientation in a counterclockwise direction about a second axis perpendicular to a frontal plane of the second display panel; The computer executable instructions include: sending camera pose data associated with a pair of virtual cameras to an application configured to output frames; The camera pose data is a first upright camera orientation that specifies that a first virtual camera of the pair of virtual cameras is in an upright camera orientation; a second upright camera orientation specifying that a second virtual camera of the pair of virtual cameras is in the upright camera orientation; receiving the frame from the application; resampling the frame to obtain a resampled frame; The obtaining includes: rotating a first subset of pixel data of the frame associated with the first virtual camera in the counterclockwise direction to obtain a first subset of modified pixel data of the resampled frame; and by rotating in the clockwise direction a second subset of the pixel data of the frame associated with the second virtual camera to obtain the second subset of the modified pixel data of the resampled frame; outputting the modified pixel data to a frame buffer. An HMD system executable by the one or more processors.

4. 2. The HMD system of claim 1, wherein the first display panel is oriented in the first orientation and the second display panel is oriented in the second orientation by rotating each display panel by an equal amount of rotation.

5. The HMD system of claim 4, wherein the equal amount of rotation is within a range of 10 to 80 degrees relative to the upright panel orientation.

6. The HMD system of claim 1 , wherein the pair of display panels are coplanar.

7. the first display panel is oriented in the first orientation by further tilting the first display panel about an axis perpendicular to a transverse plane of the upright panel orientation; 2. The HMD system of claim 1, wherein the second display panel is oriented in the second orientation by further tilting the second display panel about the axis perpendicular to the transverse plane of the upright panel orientation in a direction opposite to that of the first display panel.

8. A head mounted display (HMD) system, comprising: An HMD that can be worn on a user's head; one or more processors; and computer-executable instructions stored in a memory of the HMD system and executable by the one or more processors; The HMD includes: Housing and a pair of display panels mounted within the housing, the pair of display panels including a first display panel and a second display panel adjacent to the first display panel, the first display panel being mounted at a position corresponding to a left eye of the user wearing the HMD on the head, and the second display panel being mounted at a position corresponding to a right eye of the user wearing the HMD on the head; When the user wearing the HMD on the head views the pair of display panels, the first display panel is oriented in a first orientation by rotating the first display panel in a clockwise direction about a first axis perpendicular to a frontal plane of the first display panel relative to an upright panel orientation; when the user wearing the HMD on the head views the pair of display panels, the second display panel is oriented in a second orientation by rotating the second display panel relative to the upright panel orientation in a counterclockwise direction about a second axis perpendicular to a frontal plane of the second display panel; The computer executable instructions include: sending camera pose data associated with a pair of virtual cameras to an application configured to output frames; The camera pose data is a first rotated camera orientation indicating that a first virtual camera of the pair of virtual cameras is rotated in the counterclockwise direction relative to an upright camera orientation; a second rotated camera orientation indicating that a second virtual camera of the pair of virtual cameras is rotated in the clockwise direction relative to the upright camera orientation; receiving the frame from the application; resampling the frame by modifying pixel data of the frame to obtain modified pixel data of the resampled frame; outputting the modified pixel data to a frame buffer. An HMD system executable by the one or more processors.

9. the first display panel is oriented in the first orientation by rotating the first display panel in the clockwise direction by an amount of rotation of 10 degrees or greater relative to the upright panel orientation; 9. The HMD system of claim 8, wherein the second display panel is oriented in the second orientation by rotating the second display panel the rotational amount relative to the upright panel orientation in the counterclockwise direction.

10. The HMD system of claim 9 , wherein the application is a video game application.

11. A head mounted display (HMD) system, comprising: An HMD that can be worn on a user's head; one or more processors; and computer-executable instructions stored in a memory of the HMD system and executable by the one or more processors; The HMD includes: Housing and a pair of display panels mounted within the housing, the pair of display panels including a first display panel and a second display panel adjacent to the first display panel, the first display panel being mounted at a position corresponding to a left eye of the user wearing the HMD on the head, and the second display panel being mounted at a position corresponding to a right eye of the user wearing the HMD on the head; When the user wearing the HMD on the head views the pair of display panels, the first display panel is oriented in a first orientation by rotating the first display panel in a clockwise direction about a first axis perpendicular to a frontal plane of the first display panel relative to an upright panel orientation; when the user wearing the HMD on the head views the pair of display panels, the second display panel is oriented in a second orientation by rotating the second display panel relative to the upright panel orientation in a counterclockwise direction about a second axis perpendicular to a frontal plane of the second display panel; The computer executable instructions include: sending camera pose data associated with a pair of virtual cameras to an application configured to output frames; The camera pose data is a first upright camera orientation indicating that a first virtual camera of the pair of virtual cameras is in an upright camera orientation; a second upright camera orientation indicating that a second virtual camera of the pair of virtual cameras is in the upright camera orientation; receiving the frame from the application; resampling the frame to obtain a resampled frame; The obtaining includes: rotating a first subset of pixel data of the frame associated with the first virtual camera in the counterclockwise direction to obtain a first subset of modified pixel data of the resampled frame; by rotating in the clockwise direction a second subset of the pixel data of the frame associated with the second virtual camera to obtain the second subset of the modified pixel data of the resampled frame; outputting the modified pixel data to a frame buffer. An HMD system executable by the one or more processors.

12. The HMD system of claim 11 , wherein the application is a video game application.

13. The HMD system of claim 8 , wherein the HMD is at least one of a virtual reality headset or an augmented reality headset.

14. the first display panel is tilted in a first direction about an axis perpendicular to a transverse plane of the upright panel orientation; 9. The HMD system of claim 8, wherein the second display panel is tilted in a second direction opposite to the first direction about the axis perpendicular to the transverse plane of the upright panel orientation.

15. A head mounted display (HMD) system, comprising: An HMD including a pair of display panels and mountable on a user's head, the pair of display panels including a first display panel and a second display panel, the first display panel being attached to a position corresponding to a left eye of the user wearing the HMD on the head, and the second display panel being attached to a position corresponding to a right eye of the user wearing the HMD on the head; one or more processors; computer-executable instructions stored in a memory of the HMD system and executable by the one or more processors; The executable computer executable instructions include: sending camera pose data including a counter-rotation camera orientation to an application configured to output a frame; The camera pose data is a first rotated camera orientation in which, when the user wearing the HMD on the head looks at the pair of display panels, the first display panel is rotated counterclockwise with respect to an upright camera orientation; a second rotated camera orientation, the second display panel being rotated clockwise relative to the upright camera orientation when the user wearing the HMD on the head views the pair of display panels; receiving the frame from the application, the frame including pixel data representing a pair of images of a counter-rotated scene based on the counter-rotation camera orientation; resampling the frame to obtain a resampled frame of the scene in an upright orientation; The obtaining includes: rotating the first subset of pixel data associated with the first rotated camera orientation in the clockwise direction to obtain a first subset of modified pixel data of the resampled frame; and by rotating in a counterclockwise direction a second subset of the pixel data associated with the second rotated camera orientation to obtain the second subset of the modified pixel data of the resampled frame; outputting the modified pixel data to a frame buffer. The HMD system is executable by the one or more processors.

16. a first display panel of the pair of display panels is tilted about an axis perpendicular to a transverse plane of an upright panel orientation; a second display panel of the pair of display panels is tilted in a direction opposite to that of the first display panel about the axis perpendicular to the transverse plane of the upright panel orientation; the camera pose data further includes a tilted camera orientation; The HMD system according to claim 15.

17. The HMD system of claim 15 , wherein resampling the frame includes rotating each of the first subset of pixel data and the second subset of pixel data an equal amount of rotation.

18. 18. The HMD system of claim 17, wherein the equal amount of rotation is in the range of 40 to 50 degrees relative to the upright camera orientation.

Images