Stereoscopic fovea image generation

Abstract

In one implementation, a method of generating an image is performed by a device including one or more processors and a non-transitory memory. The method includes generating a first resolution function based on a formula having a set of variables, the set of variables having a first set of values. The method includes generating a first image based on the first content and a first resolution function. The method includes detecting a resolution constraint. The method includes generating a second resolution function based on a formula having a set of variables, the set of variables having a second set of values, where the second resolution function has a summed value that satisfies a resolution constraint. The method includes generating a second image based on the second content and a second resolution function.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 444,097, filed on February 8, 2023, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to image generation, and particularly to systems, methods, and apparatus for generating images with varying amounts of detail. Background Art

[0004] Rendering or otherwise processing an image can be computationally expensive. Therefore, to alleviate this computational burden, the fact that humans typically have relatively poor peripheral vision is exploited. Consequently, different portions of the image are presented at different resolutions on the display panel. For example, in various implementations, the portion corresponding to the user's fovea is presented at a higher resolution than the portion corresponding to the user's periphery. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] In order that the present disclosure may be understood by those skilled in the art, a more particular description will be given of various aspects with reference to certain exemplary implementations, some of which are illustrated in the accompanying drawings.

[0006] Figure 1 is a block diagram of an example operating environment according to some implementations.

[0007] Figure 2 An XR pipeline is illustrated that receives XR content and displays an image on a display panel based on the XR content, according to some implementations.

[0008] Figures 3A to 3D Various resolution functions in the first dimension according to various implementations are illustrated.

[0009] Figures 4A to 4D Various two-dimensional resolution functions are illustrated according to various implementations.

[0010] Figure 5A An example resolution function characterizing resolution in display space as a function of angle in warp space is illustrated according to some implementations.

[0011] Figure 5B Examples according to some specific implementations Figure 5A Example of the integral of the resolution function.

[0012] Figure 5C Examples according to some specific implementations Figure 5AAn example of the resolution function is the tangent of the integral of the inverse.

[0013] Figure 6A An example resolution function for performing static foveation according to some implementations is illustrated.

[0014] Figure 6B An example resolution function for performing dynamic foveation according to some implementations is illustrated.

[0015] Figure 7 is a flowchart representation of a method for rendering an image based on a resolution function according to some specific implementations.

[0016] Figure 8A Example image representations of XR content to be rendered in display space are illustrated according to some implementations.

[0017] Figure 8B Examples according to some specific implementations Figure 8A Distorted images of XR content.

[0018] Figure 9 is a flowchart representation of a method for satisfying resolution constraints according to some specific implementations.

[0019] 10A to 10D Example resolution functions for performing monocular and binocular resolution reduction are illustrated.

[0020] Figure 11 is a flowchart representation of a method of rendering an image according to some specific implementations.

[0021] Figure 12 is a block diagram of an example controller according to some specific implementations.

[0022] Figure 13 is a block diagram of an example electronic device according to some implementations.

[0023] In accordance with common practice, the various features illustrated in the accompanying drawings may not be drawn to scale. Therefore, the dimensions of various features may be arbitrarily expanded or reduced for clarity. Furthermore, some of the accompanying drawings may not depict all components of a given system, method, or apparatus. Finally, similar reference numerals may be used to denote similar features throughout the specification and accompanying drawings. Summary of the Invention

[0024] Various embodiments disclosed herein include devices, systems, and methods for generating images. In various embodiments, the method is performed by a device including a display, one or more processors, and non-volatile memory. The method includes obtaining a first resolution function and a second resolution function, wherein the second resolution function is different from the first resolution function. The method includes generating a first rendered image based on content and the first resolution function and generating a second rendered image based on the content and the second resolution function. The method includes simultaneously displaying a first display image based on the first rendered image on a first portion of the display and displaying a second display image based on the second rendered image on a second portion of the display.

[0025] According to some specific implementations, a device includes one or more processors, non-volatile memory, and one or more programs; the one or more programs are stored in the non-volatile memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing the execution of any of the methods described herein. According to some specific implementations, a non-volatile computer-readable storage medium has instructions stored therein that, when executed by one or more processors of the device, cause the device to perform or cause the execution of any of the methods described herein. According to some specific implementations, a device includes: one or more processors, non-volatile memory, and components for performing or causing the execution of any of the methods described herein. DETAILED DESCRIPTION

[0026] Numerous details are described to provide a thorough understanding of the example implementations shown in the accompanying drawings. However, the accompanying drawings illustrate only some example aspects of the present disclosure and, therefore, should not be considered limiting. One of ordinary skill in the art will appreciate that other effective aspects and / or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits are not described in detail in order to avoid obscuring more relevant aspects of the example implementations described herein.

[0027] As noted above, in various implementations, different portions of an image are presented on a display panel at different resolutions. However, in various implementations, when a pair of stereoscopic images is presented on a first portion of a display panel and a second portion of a display panel (or two separate display panels), the resolutions at corresponding locations on the first portion of the display panel and the second portion of the display panel may be different.

[0028] Figure 11 is a block diagram of an example operating environment 100 according to some implementations. Although relevant features are shown, those skilled in the art will recognize from this disclosure that various other features are not illustrated for the sake of brevity and to avoid obscuring more relevant aspects of the example implementations disclosed herein. To this end, as a non-limiting example, the operating environment 100 includes a controller 110 and an electronic device 120.

[0029] In some implementations, the controller 110 is configured to manage and coordinate the user's XR experience. In some implementations, the controller 110 includes a suitable combination of software, firmware, and / or hardware. Figure 12 Controller 110 is described in more detail. In some implementations, controller 110 is a computing device that is located locally or remotely relative to physical environment 105. For example, controller 110 is a local server located within physical environment 105. In another example, controller 110 is a remote server (e.g., a cloud server, a central server, etc.) located outside of physical environment 105. In some implementations, controller 110 is communicatively coupled to electronic device 120 via one or more wired or wireless communication channels 144 (e.g., Bluetooth, IEEE 802.11x, IEEE 802.16x, IEEE802.3x, etc.). In another example, controller 110 is included within a housing of electronic device 120. In some implementations, the functionality of controller 110 is provided by and / or combined with electronic device 120.

[0030] In some implementations, the electronic device 120 is configured to provide an XR experience to the user. In some implementations, the electronic device 120 includes a suitable combination of software, firmware, and / or hardware. According to some implementations, the electronic device 120 presents XR content to the user via the display 122 while the user is physically present within a physical environment 105, which includes a table 107 within the field of view 111 of the electronic device 120. Thus, in some implementations, the user holds the electronic device 120 in one or both of his / her hands. In some implementations, when providing XR content, the electronic device 120 is configured to display an XR object (e.g., an XR cylinder 109) and implement video pass-through of the physical environment 105 (e.g., including a representation 117 of the table 107) on the display 122. The following description with respect to Figure 13 The electronic device 120 is described in more detail.

[0031] According to some implementations, the electronic device 120 provides an XR experience to the user while the user is virtually and / or physically present within the physical environment 105 .

[0032] In some implementations, the user wears the electronic device 120 on his / her head. For example, in some implementations, the electronic device includes a head-mounted system (HMS), a head-mounted device (HMD), or a head-mounted enclosure (HME). Thus, the electronic device 120 includes one or more XR displays configured to display XR content. For example, in various implementations, the electronic device 120 surrounds the user's field of view. In some implementations, the electronic device 120 is a handheld device (such as a smartphone or tablet) configured to present XR content, and the user no longer wears the electronic device 120 but instead holds the device with the display facing the user's field of view and the camera facing the physical environment 105. In some implementations, the handheld device can be placed in a housing that can be worn on the user's head. In some implementations, the electronic device 120 is replaced with an XR cabin, enclosure, or room configured to present XR content, in which the user no longer wears or holds the electronic device 120.

[0033] In various implementations, the electronic device 120 includes an XR pipeline that renders XR content. Figure 2 An XR pipeline 200 is illustrated that receives XR content and displays an image on a display panel 240 based on the XR content.

[0034] The XR pipeline 200 includes a rendering module 210 that receives XR content (and eye tracking data from an eye tracker 260) and renders an image based on the XR content. In various implementations, the XR content includes definitions of the geometry of virtual objects, the colors and / or textures of virtual objects, images (such as pass-through images of the physical environment), and other information describing the content to be represented in the rendered image.

[0035] The image includes a pixel matrix, each pixel having a corresponding pixel value and a corresponding pixel position. In various specific implementations, the pixel values ​​range from 0 to 255. In various specific implementations, each pixel value is a color triplet, which includes three values ​​corresponding to three color channels. For example, in one specific implementation, the image is an RGB image, and each pixel value includes a red value, a green value, and a blue value. For another example, in one specific implementation, the image is a YUV image, and each pixel value includes a luminance value and two chrominance values. In various specific implementations, the image is a YUV444 image, wherein each chrominance value is associated with a pixel. In various specific implementations, the image is a YUV420 image, wherein each chrominance value is associated with a 2×2 pixel block (e.g., the chrominance value is downsampled). In some specific implementations, the image includes a tile matrix, each tile has a corresponding tile position and includes a pixel block with a corresponding pixel value. In some specific implementations, each tile is a 32×32 pixel block. While specific pixel values, image formats, and tile sizes are provided, it should be understood that other values, formats, and tile sizes may be used.

[0036] The image rendered by the rendering module 210 (e.g., the rendered image) is provided to a transmission module 220, which couples the rendering module 210 to the display module 230. The transmission module 220 includes a compression module 222 that compresses the rendered image (thereby generating a compressed image), a communication channel 224 that transmits the compressed image, and a decompression module 226 that decompresses the compressed image (thereby generating a decompressed image).

[0037] The decompressed image is provided to the display module 230, which converts the decompressed image into panel data. The panel data is provided to the display panel 240, which displays the displayed image as described by the panel data (e.g., according to the panel data). The display module 230 includes a lens compensation module 232 that compensates for the distortion caused by the eyepiece 242 of the electronic device 120. For example, in various specific implementations, the lens compensation module 232 pre-distorts the decompressed image in an inversely proportional relationship to the distortion caused by the eyepiece 242, so that when the user 250 observes through the eyepiece 242, the displayed image appears undistorted. The display module 230 also includes a panel compensation module 234 that converts the image data into panel data for reading by the display panel 240.

[0038] The display panel 240 includes a matrix of M×N pixels located at corresponding positions in a display space. The display panel 240 displays a displayed image as described by (eg, according to) the panel data by emitting light from each pixel.

[0039] In various implementations, the XR pipeline 200 includes an eye tracker 260 that generates eye tracking data indicating a gaze of the user 250. In various implementations, the eye tracking data includes data indicating a point of gaze of the user 250 on the display panel 240. In various implementations, the eye tracking data includes data indicating a gaze angle of the user 250, such as an angle between a current viewing axis of the user 250 and the viewing axis when the user 250 is looking at the center of the display panel 240.

[0040] In various implementations, to render an image to be displayed on display panel 240, rendering module 210 generates M×N pixel values ​​for each pixel of the M×N images. Thus, each pixel of the rendered image corresponds to a pixel of display panel 240 having a corresponding position in the display space. Thus, rendering module 210 generates pixel values ​​for the M×N pixel positions that are evenly spaced in a grid pattern in the display space.

[0041] Rendering M×N pixel values ​​can be computationally expensive. Additionally, as the size of the rendered image increases, the amount of processing required to compress the image at compression module 222, the amount of bandwidth required to transmit the compressed image across communication channel 224, and the amount of processing required to decompress the compressed image at decompression module 226 also increases.

[0042] In various implementations, in order to reduce the size of the rendered image without degrading the user experience, foveation (e.g., imaging of the fovea) is used. Foveation is a digital image processing technique in which the image resolution or amount of detail varies across the image. Thus, the image of the fovea has different resolutions at different parts of the image. Humans typically have relatively poor peripheral vision. According to one model, the user's resolvable resolution is greatest over the gaze zone (e.g., the area the user is looking at) and decreases in an inverse linear manner. Thus, in one implementation, the displayed image displayed by the display panel 240 is an image of the fovea, which has maximum resolution in the gaze zone and whose resolution decreases linearly inversely with distance from the gaze zone.

[0043] Because some portions of the image have a lower resolution, the M×N foveated image includes less information than the M×N non-foveated image. Therefore, in various implementations, the rendering module 210 generates the foveated image as the rendered image. The rendering module 210 can generate the M×N foveated image more quickly and with less processing power (and battery charge) than the rendering module 210 can generate the M×N non-foveated image. Additionally, the M×N foveated image can be represented using less data than the M×N non-foveated image. In other words, the size of the M×N foveated image file is smaller than the size of the M×N non-foveated image file. In various implementations, compressing the M×N foveated image using various compression techniques produces fewer bits than compressing the M×N non-foveated image.

[0044] The foveal ratio, R, can be defined as the amount of information in the M×N non-foveal image divided by the amount of information in the M×N foveal image. In various implementations, the foveal ratio is between 1.5 and 10. For example, in some implementations, the foveal ratio is 2. In some implementations, the foveal ratio is 3 or 4. In some implementations, the foveal ratio is constant across images. In some implementations, the foveal ratio is determined for the image to be rendered. For example, in various implementations, the amount of information that the XR pipeline 200 can pass through within a particular time period (e.g., a frame period of an image) may be limited. For example, in various implementations, the amount of information that the rendering module 210 can render within a frame period may be reduced due to a thermal event (e.g., when the process of calculating additional pixel values ​​causes the processor to overheat). As another example, in various implementations, the amount of information that the transmission module 220 can transmit within a frame period may be reduced due to a decrease in the signal-to-noise ratio of the communication channel 224.

[0045] In some implementations, to render an image to be displayed on the display panel 240, the rendering module 210 generates M / R×N / R pixel values ​​for each pixel of the M / R×N / R warped images. Each pixel of the warped images corresponds to an area larger than the pixel of the display panel 240 at the corresponding location in the display space. Therefore, the rendering module 210 generates a pixel value for each of the M / R×N / R locations in the display space that are not evenly distributed in the grid pattern. The corresponding area in the display space corresponding to each pixel value is defined by the corresponding location in the display space (rendering location) and the scaling factor (or a set of the horizontal scaling factor and the vertical scaling factor).

[0046] In various implementations, the rendering module 210 generates a warped image as the rendered image. In various implementations, the warped image includes a matrix of M / R×N / R pixel values ​​at M / R×N / R locations that are evenly spaced in a grid pattern in a warp space that is different from the display space. Specifically, the warped image includes a matrix of M / R×N / R pixel values ​​at M / R×N / R locations in the display space that are not evenly distributed in the grid pattern. Therefore, although the resolution of the warped image is uniform in the warp space, the resolution varies in the display space. Figure 8A and Figure 8B Describe this in more detail.

[0047] The rendering module 210 determines a rendering position and a corresponding scaling factor based on a resolution function, which generally characterizes the resolution of a rendered image in a display space.

[0048] In one implementation, the resolution function S(x) varies with the distance from the origin of the display space (which may correspond to the center of the display panel 240). In another implementation, the resolution function S(θ) varies with the angle between the viewing axis of the user 250 and the viewing axis when the user 250 is looking at the center of the display panel 240. Therefore, in one implementation, the resolution function S(θ) is expressed in pixels per degree (PPD).

[0049] Humans typically have relatively poor peripheral vision. According to one model, the user's resolvable resolution is greatest over the fixation zone and decreases in an inverse linear manner as the angle relative to the optical axis increases. Therefore, in one embodiment, the resolution function (in the first dimension) is defined as:

[0050]

[0051] Among them S max is the maximum value of the resolution function (e.g., about 60 PPD), S min is the asymptote of the resolution function, θ f Characterizes the size of the fixation zone, and w characterizes the width of the resolution function or the speed at which the resolution function falls outside the fixation zone as the angle relative to the optical axis increases.

[0052] Figure 3A A resolution function 310 (in the first dimension) is illustrated that decreases from the fixation region in an inverse linear manner. Figure 3B A resolution function 320 is illustrated (in the first dimension) that decreases linearly from the fixation region. Figure 3C A resolution function 330 (in a first dimension) that approximates a Gaussian function is illustrated. Figure 3D A resolution function 340 (in the first dimension) is illustrated that decreases in a circular step-wise manner.

[0053] Figures 3A to 3D Each of the resolution functions 310 to 340 is in the form of a peak comprising a peak height (e.g., a maximum value) and a peak width. The peak width can be defined in a variety of ways. In one embodiment, the peak width is defined as the size of the focus area (e.g., Figure 3A The width is 311 and Figure 3B In one embodiment, the peak width is defined as the full width at half maximum (e.g., Figure 3C In one embodiment, the peak width is defined as the distance between the two inflection points closest to the origin (e.g., Figure 3D In various implementations, the number of pixels in the rendered image is proportional to the integral of the resolution function within the field of view. Thus, the sum value is defined as the area under the resolution function within the field of view.

[0054] Although Figures 3A to 3D The resolution function in a single dimension is illustrated, but it should be understood that the resolution function used by the rendering module 210 may be a two-dimensional function. Figure 4A A two-dimensional resolution function 410 is illustrated, wherein the resolution function 410 has two dimensions (θ) in the horizontal dimension and the vertical dimension. China is independent. Figure 4B A two-dimensional resolution function 420 is illustrated, where the resolution function 420 is a function of a single variable (e.g., ). Figure 4C A two-dimensional resolution function 430 is illustrated, wherein the resolution function 430 has two dimensions (θ) in the horizontal dimension and the vertical dimension. It is different. Figure 4D A two-dimensional resolution function 440 based on a human vision model is illustrated.

[0055] As described in detail below, the rendering module 210 generates a resolution function based on multiple factors, including biological information related to human vision, eye tracking data, eye tracking metadata, XR content, and various constraints (such as constraints imposed by the hardware of the electronic device 120).

[0056] Figure 5A An example resolution function 510 (denoted as S(θ)) is illustrated that characterizes the resolution in display space as a function of angle in warp space. The resolution function 510 is within the fixation zone (between −θ f and +θ f between) is a constant (for example, S max ) and decreases in an inverse linear manner outside this window.

[0057] Figure 5B The example within the field of view (e.g., –θfov to +θ fov )of Figure 5A The integral 520 of the resolution function 510 (denoted as U(θ)). Therefore, The range of integral 520 is –θ fov 0 to +θ fov The maximum value under (denoted as U max ).

[0058] Figure 5C Illustrated Figure 5A The tangent 530 of the inverse of the integral 520 of the resolution function 510 (denoted as V(x R )). Therefore, V(x R )=tan(U -1 (x R )). Tangent 530 illustrates the R Rendering space in x D Direct mapping of the display space in the . According to the fovea indicated by the resolution function 510, uniform sampling points in the warped space (along x R Axis equally spaced) corresponds to non-uniform sampling points in display space (along x D The scaling factor can be determined by the distance between non-uniform sampling points in display space.

[0059] When static foveation is implemented, the rendering module 210 uses a resolution function that is not dependent on the user's gaze. However, when dynamic foveation is implemented, the rendering module 210 uses a resolution function that is dependent on the user's gaze. Specifically, when dynamic foveation is implemented, the rendering module 210 uses a resolution function that has a peak height at a location corresponding to the location in the display space where the user is looking (e.g., the user's gaze point as determined by the eye tracker 260).

[0060] Figure 6A The rendering module 210 may use a resolution function 610 when implementing static foveation. The rendering module 210 may also use a resolution function 610 when implementing dynamic foveation and the user is looking at the center of the display panel 240. Figure 6A The resolution function 610 is shown. Figure 6B The example shows that dynamic foveation is being performed and the user is moving at an angle (θ away from the center of the display panel 240 g ) resolution function 620 that can be used by the rendering module 210 when viewing.

[0061] Thus, in one embodiment, the resolution function (in the first dimension) is defined as:

[0062]

[0063] Figure 7 is a flowchart representation of a method 700 for rendering an image according to some implementations. In some implementations (and as described in detail below as an example), the method 700 is performed by a rendering module (such as Figure 2 In various implementations, the method 700 is performed by an electronic device (such as a Figure 1 electronic device 120) or a portion thereof (such as Figure 2 In various implementations, method 700 is performed by a device having one or more processors, non-transitory memory, and one or more XR displays. In some implementations, method 700 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some implementations, method 700 is performed by a processor executing instructions (e.g., code) stored in a non-transitory computer-readable medium (e.g., memory).

[0064] Method 700 begins at block 710, where the rendering module obtains XR content to be rendered into the display space. In various implementations, the XR content may include definitions of the geometry of virtual objects, the colors and / or textures of virtual objects, images (such as a pass-through image of the physical environment), or other information describing the content to be represented in the rendered image.

[0065] Method 700 continues at block 720 when the rendering module obtains a resolution function that defines a mapping between display space and warp space. Figures 3A to 3D and Figures 4A to 4D Various methods of generating resolution functions are further described below.

[0066] In various implementations, the resolution function generally characterizes the resolution of the rendered image in the display space. Thus, the integral of the resolution function provides a mapping between the display space and the warp space (e.g., Figures 5A to 5C ). In one implementation, the resolution function S(x) varies with the distance from the origin of the display space. In another implementation, the resolution function S(θ) varies with the angle between the user's viewing axis and the user's viewing axis when looking at the center of the display panel. Thus, the resolution function characterizes the resolution in the display space as a function of angle (in the display space). Therefore, in one implementation, the resolution function S(θ) is expressed in pixels per degree (PPD).

[0067] In various implementations, the rendering module performs dynamic foveation and the resolution function depends on the user's gaze. Thus, in some implementations, obtaining the resolution function includes, for example, Figure 2The eye tracker 260 of the embodiment of the present invention obtains eye tracking data indicating a gaze of a user, and generates a resolution function based on the eye tracking data. In various implementations, the eye tracking data includes at least one of data indicating an angle of gaze of the user or data indicating a point of gaze of the user. Specifically, in various implementations, generating the resolution function based on the eye tracking data includes generating a resolution function having a peak height at a location where the user is looking, as indicated by the eye tracking data.

[0068] Method 700 continues at block 730, where the rendering module generates a rendered image based on the XR content and the resolution function. The rendered image includes a warped image having a plurality of pixels at corresponding locations evenly spaced in a grid pattern in the warp space. The plurality of pixels are respectively associated with a plurality of corresponding pixel values ​​based on the XR content. The plurality of pixels are respectively associated with a plurality of corresponding scaling factors that define a region in the display space based on the resolution function.

[0069] An image that is said to be in display space has evenly spaced regions (e.g., pixels or groups of pixels) that map to evenly spaced regions (e.g., pixels or groups of pixels) of the display. An image that is said to be in warp space has evenly spaced regions (e.g., pixels or groups of pixels) that map to non-evenly spaced regions (e.g., pixels or groups of pixels) in display space. The relationship between evenly spaced regions in warp space and non-evenly spaced regions in display space is defined, at least in part, by a scaling factor. Thus, the plurality of corresponding scaling factors (as with the resolution function) defines a mapping between warp space and display space.

[0070] In various implementations, the rendering module sends the warped image including the plurality of pixel values ​​associated with the plurality of respective scaling factors. Thus, the warped image and scaling factors are propagated through the XR pipeline 200 rather than a foveated image that can be generated using this information.

[0071] Specifically, relative to Figure 2 Specifically, in various implementations, the rendering module 210 generates a warped image and a plurality of corresponding scaling factors that are sent by the rendering module 210. At various stages in the XR pipeline 200, the warped image (or a processed version of the warped image) and the plurality of corresponding scaling factors are received by the transport module 220 (and its compression module 222 and decompression module 226) (and used to process the warped image). At various stages in the XR pipeline 200, the warped image (or a processed version of the warped image) and the plurality of corresponding scaling factors are received by the display module 230 (and its lens compensation module 232 and panel compensation module 234) (and used to process the warped image).

[0072] In various implementations, the rendering module 210 generates a scaling factor based on a resolution function. For example, in some implementations, based on the above-described Figures 5A to 5C The scaling factor is generated based on the resolution function described. In various implementations, generating the scaling factor includes determining an integral of the resolution function. In various implementations, generating the scaling factor includes determining a tangent of an inverse of the integral of the resolution function. In various implementations, generating the scaling factor includes determining a corresponding scaling factor for each of the corresponding uniformly spaced locations in the grid pattern in the warped space based on the tangent of the inverse of the integral of the resolution function. Thus, for the uniformly spaced locations in the warped space, the non-uniformly spaced locations in the display space are represented by the scaling factor.

[0073] Figure 8A An image representation of XR content 810 to be rendered in display space is illustrated. Figure 8B Illustrated based on Figure 7 The warped image 820 is generated using the method 700. Different portions of the XR content 810 corresponding to non-uniformly spaced regions (e.g., different amounts of regions) in the display space are rendered into evenly spaced regions (e.g., the same amount of regions) in the warped image 820 according to the resolution function.

[0074] For example, a matrix consisting of K pixels (and K pixel values) Figure 8B The region in the distorted image 820 is represented by Figure 8A Similarly, the image representation of the XR content 810 is composed of K pixels (and K pixel values). Figure 8B The region in the distorted image 820 is represented by Figure 8A The image of the XR content 810 represents the area on the corner ( Figure 8A The area at the center is larger).

[0075] In various implementations, to provide a three-dimensional XR experience, the display panel 240 includes a first portion for displaying a first display image to a first eye of the user 250 (e.g., the left eye of the user 250) and a second portion for simultaneously displaying a second display image to a second eye of the user 250 (e.g., the right eye of the user 250). Therefore, in various implementations, the rendering module 210 renders the first rendered image and the second rendered image based on the XR content. In various implementations, the rendering module 210 uses the same resolution function to render the first rendered image and the second rendered image. However, in various implementations, the rendering module 210 uses different resolution functions to render the first rendered image and the second rendered image.

[0076] For example, in various implementations, the second resolution function has a lower maximum value or a lower summation value than the first resolution function to reduce computations in rendering the first rendered image and the second rendered image while minimizing degradation in viewing experience due to binocular suppression. Binocular suppression is a visual phenomenon whereby the perceived quality of two simultaneously presented images is not degraded when one of the images has a lower quality (e.g., the lower quality of the image is suppressed).

[0077] Figure 9 is a flowchart representation of a method 900 for satisfying a resolution constraint according to some implementations. In some implementations, the method 900 is performed by a rendering module (such as Figure 2 In various implementations, the method 900 is performed by an electronic device (such as a Figure 1 electronic device 120) or a portion thereof (such as Figure 2 In various implementations, method 900 is performed by a device having one or more processors, non-transitory memory, and one or more XR displays. In some implementations, method 900 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some implementations, method 900 is performed by a processor executing instructions (e.g., code) stored in a non-transitory computer-readable medium (e.g., memory).

[0078] Method 900 begins at block 910, where a device generates a first left rendered image using a first resolution function having a first maximum value and generates a first right image using the first resolution function having a first maximum value. In various implementations, the first left rendered image and the first right rendered image are generated based on first content (e.g., first XR content).

[0079] Figure 10A An example first resolution function 1010L for a first left rendered image and 1010R for a first right rendered image is illustrated. The first rendering resolution function has a maximum value S 1 max .

[0080] Method 900 continues at block 920, where the device detects a resolution constraint. In various implementations, the resolution constraint indicates a number of pixels. In various implementations, the resolution constraint indicates a summation value. In various implementations, the resolution constraint is detected based on user input. For example, in various implementations, a user activates a low-power mode, and in response, a resolution constraint is generated and / or detected by the device. In various implementations, the resolution constraint is detected based on the amount of available processing power. For example, when the device has little available processing power (due to little processing power or high utilization of processing power), the device may generate and / or detect the resolution constraint. As another example, when a thermal event occurs (e.g., when the processing for calculating additional pixel values ​​would cause the processor to overheat), the device may generate and / or detect the resolution constraint. In various implementations, the resolution constraint is generated based on the bandwidth of the communication channel. For example, in response to a decrease in the signal-to-noise ratio of the communication channel, the device may generate and / or detect the resolution constraint.

[0081] Method 900 continues at block 930, where the device determines whether to apply monocular or binocular resolution reduction. In various implementations, the device determines whether to apply monocular or binocular resolution reduction based on a user preference. For example, in various implementations, the user can activate or deactivate a setting that allows monocular resolution reduction. In various implementations, the device determines whether to apply monocular or binocular resolution reduction based on the content used to generate the rendered image. For example, in various implementations, when the content is three-dimensional content, the device determines that the device will apply monocular resolution reduction, and when the content is text or video, the device determines that the device will apply binocular resolution reduction.

[0082] If the device determines that the device will apply binocular resolution reduction, the method 900 continues at block 940, where the device generates a second left rendered image using a second resolution function having a second maximum value and generates a second right rendered image using a second resolution function having a second maximum value. In various implementations, the second maximum value is determined such that a doubled value of the sum of the second resolution function satisfies the resolution constraint. In various implementations, the second left rendered image and the second right rendered image are generated based on the second content (e.g., second XR content).

[0083] Figure 10B An example second resolution function 1020L for a second left rendered image and 1020R for a second right rendered image is illustrated. The second resolution function has a maximum value S 2 max , the maximum value of the second resolution function is less than the maximum value S of the first resolution function 1 maxIn various implementations, the second resolution function is a capped version of the first resolution function, where the second resolution function at each angle is equal to the lesser of: the first resolution function at that angle, and the second maximum value.

[0084] If the device determines that the device will apply monocular resolution reduction, method 900 continues in block 950, where the device further determines whether to reduce the resolution of the left eye or the right eye. In various implementations, the device determines whether to reduce the resolution of the left eye or the right eye based on user preference. For example, the user may explicitly select the left eye or the right eye for resolution reduction via a user interface. As another example, the user may select the left eye or the right eye for resolution reduction via a calibration procedure. In various implementations, the device determines whether to reduce the resolution of the left eye or the right eye based on a variable that alternates between sessions. For example, in various implementations, the selection of the left eye or the right eye for resolution reduction alternates between each session in which the user puts on, uses, and removes the device. In various implementations, the device determines whether to reduce the resolution of the left eye or the right eye based on the content used to generate the rendered image. For example, if the user is positioned so that the left eye's view is looking out a window, but the right eye's view is obscured by a wall, the device may determine that the device will reduce the resolution of the eye viewing the low-contrast content (e.g., the right eye viewing the wall). As another example, if the user is viewing monocularly with their left eye, the device may determine that the device will reduce the resolution of the right eye (which may be assumed to be closed or having its vision suppressed due to focus on the content being viewed monocularly).

[0085] If the device determines that the device will reduce the resolution of the left eye, method 900 continues at block 960, where the device generates a second left rendered image using a third resolution function having a third maximum value and generates a second right image using the first resolution function having the first maximum value. In various implementations, the third maximum value is determined such that a sum of a sum of the first resolution function and a sum of the third resolution function satisfies the resolution constraint.

[0086] Figure 10C An example third resolution function 1030L for a second left rendered image and an example first resolution function 1030R for a second right rendered image are illustrated. The third resolution function has a maximum value S 3 max , the maximum value of the third resolution function is less than the maximum value S of the first rendering function 1 max (and less than the maximum value S of the second resolution function 2 maxIn various implementations, the third resolution function is a capped version of the first resolution function, where the third resolution function at each angle is equal to the smaller of: the first resolution function at that angle, and the third maximum value.

[0087] If the device determines that the device will reduce the resolution for the right eye, method 900 continues in block 970 where the device generates a second left rendered image using a first resolution function having a first maximum value and generates a second right image using a third resolution function having a third maximum value.

[0088] Figure 10D An example first resolution function 1040L for a second left rendered image and an example third resolution function 1040R for a second right rendered image are illustrated.

[0089] although Figure 9 While a method is described in which two different resolution functions are used to generate rendered images to be presented simultaneously to a user's two eyes (after being transformed into corresponding display images) in order to satisfy a resolution constraint, various implementations may also generate rendered images to be presented simultaneously to a user's two eyes (after being transformed into corresponding display images) for various other reasons. For example, if one eye has poorer vision than the other eye, the resolution function used to generate the image for that eye may have a lower maximum value than the resolution function used to generate the image for the other eye. As another example, if the eyepiece positioned in front of one eye is of lower quality than the eyepiece positioned in front of the other eye, the resolution function used to generate the image for that eye may have a lower maximum value than the resolution function used to generate the image for the other eye. As another example, if eye tracking performed for one eye is less accurate than eye tracking performed for the other eye, the resolution function used to generate the image for that eye may have a lower maximum value (and / or a larger width) than the resolution function used to generate the image for the other eye.

[0090] In addition, although Figure 9 Methods are described in which two different resolution functions are used to generate rendered images to be presented simultaneously, but in various implementations, the rendered images to be presented simultaneously are also generated to meet different perceptual criteria. For example, in various implementations, the anti-aliasing algorithm applied to the image for one eye is more computationally intensive than the anti-aliasing algorithm applied to the image for the other eye (if applied at all). As another example, in various implementations, the blurring algorithm applied to the image for one eye (e.g., for a depth effect) is more robust than the blurring algorithm applied to the image for the other eye (if applied at all).

[0091] Figure 11FIG. 1 is a flowchart representation of a method 1100 for rendering an image using different resolution functions according to some implementations. In some implementations, the method 1100 is performed by a rendering module (such as Figure 2 In various implementations, the method 1100 is performed by an electronic device (such as a Figure 1 electronic device 120) or a portion thereof (such as Figure 2 In various implementations, method 1100 is performed by a device having one or more processors, non-transitory memory, and a display. In some implementations, method 1100 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some implementations, method 1100 is performed by a processor executing instructions (e.g., code) stored in a non-transitory computer-readable medium (e.g., memory).

[0092] Method 1100 begins at block 1110, where a device obtains a first resolution function and a second resolution function, where the second resolution function is different from the first resolution function. In various implementations, the maximum value of the second resolution function is different from the maximum value of the first resolution function. For example, Figure 10D The maximum value of the third resolution function 1040R for the right eye (eg, S 3 max ) is less than the maximum value of the first resolution function 1040L for the left eye (e.g., S 1 max In various implementations, the sum of the second resolution function is different from the sum of the first resolution function. Figure 10D In FIG, the sum of the third resolution function 1040R for the right eye is less than the sum of the first resolution function 1040L for the left eye. In various implementations, the second resolution function at each angle is equal to the smaller of: the first resolution function at that angle, and the maximum value of the second resolution function. For example, at Figure 10D , the third resolution function is a capped version of the first resolution function.

[0093] In various implementations, obtaining the first resolution function and the second resolution function includes: generating the first resolution function based on a formula having a set of variables, the set of variables having a first set of values; and generating the second resolution function based on a formula having a set of variables, the set of variables having a second set of values. In various cases, the formula (in the first dimension) is:

[0094]

[0095] Thus, in various implementations, the set of values ​​includes the maximum value (S max ), asymptote (Smin ), the first width (θ f ), the second width (w) and the gaze angle (θ g ). In various implementations, the set of variables includes at least one of a maximum value, a minimum value, an asymptote, a width, or a gaze angle. In various implementations, the second set of values ​​differs from the first set of values ​​by having a different maximum value. For example, in Figure 10D The maximum value of the third resolution function 1040R for the right eye (eg, S 3 max ) is less than the maximum value of the first resolution function 1040L for the left eye (e.g., S 1 max ). In various implementations, the second set of values ​​differs from the first set of values ​​by having different widths. For example, in Figure 10D , the width of the third resolution function 1040R for the right eye is greater than the width of the first resolution function 1040L for the left eye. In various implementations, at least one value in the first set of values ​​is the same as at least one value in the second set of values. For example, Figure 10D , the gaze angle of the third resolution function 1040R of the right eye is the same as the gaze angle of the first resolution function 1040L of the left eye.

[0096] Method 1100 continues at block 1120, where the device generates a first rendered image based on the content and the first resolution function and generates a second rendered image based on the content and the second resolution function. In various implementations, the device generates the first rendered image and the second rendered image as described above with respect to Figure 7 described.

[0097] Method 1100 continues at block 1130 where the device simultaneously displays a first display image based on the first rendered image on a first portion of the display and a second display image based on the second rendered image on a second portion of the display. In various implementations, the first portion of the display is positioned in front of a first eye of a user (e.g., the user's left eye) and the second portion of the display is positioned in front of a second eye of the user (e.g., the user's right eye).

[0098] In various implementations, the first display image and the second display image are foveated images based on the first rendered image and the second rendered image, respectively, and the first rendered image and the second rendered image are warped images. Therefore, in various implementations, method 1100 includes transforming the first rendered image and the second rendered image into the first display image and the second display image based on the first resolution function and the second resolution function (e.g., according to a first scaling factor based on the first resolution function and a second scaling factor based on the second resolution function).

[0099] In various implementations, method 1100 includes detecting a resolution constraint, wherein a sum of a first summed value of a first resolution function and a second summed value of a second resolution function satisfies the resolution constraint. In various implementations, obtaining the first resolution function and the second resolution function is performed in response to detecting the resolution constraint. In various implementations, obtaining the first resolution function and the second resolution function includes generating the first resolution function and the second resolution function to satisfy the resolution constraint. For example, Figure 9 In block 920 , the device detects the resolution constraint and obtains different resolution functions in block 960 (or block 970 ).

[0100] In various implementations, method 1100 includes determining that a device is to perform monocular resolution reduction, wherein obtaining the first resolution function and the second resolution function is performed in response to determining that the device is to perform monocular resolution reduction. Figure 9 In block 930, when the device determines that the device will perform monocular resolution reduction, the device obtains the different resolution functions in block 960 (or block 970). In various implementations, determining that the device will perform monocular resolution reduction is based on user preferences. In various implementations, determining that the device will perform monocular resolution reduction is based on content.

[0101] In various implementations, the method 1100 further includes selecting the first resolution function or the second resolution function to have a lower summation value, wherein obtaining the first resolution function and the second resolution function is performed in response to selecting the first resolution function or the second resolution function to have a lower summation value, and wherein the selected one of the first resolution function or the second resolution function has a lower summation value. Figure 9 In block 950, the device selects whether to reduce the resolution of the left eye or the right eye, and obtains a different resolution function in block 960 (or block 970). In various implementations, selecting the first resolution function or the second resolution function is based on user preference. In various implementations, selecting the first resolution function or the second resolution function is based on content. In various implementations, selecting the first resolution function or the second resolution function is based on a variable that alternates between sessions.

[0102] In various implementations, in addition to generating stereoscopic images based on different resolution functions, method 1100 also includes generating stereoscopic images based on the same resolution function. Figure 10A In FIG, the first resolution function 1010L for the left eye is the same as the first resolution function 1010R for the right eye. As another example, in Figure 10B, the second resolution function of the left eye 1020L is the same as the second resolution function of the right eye 1020R. Thus, in various implementations, the method 1110 includes generating a third rendered image based on the content and the third resolution function and generating a fourth rendered image based on the content and the third resolution function. The method 1110 includes simultaneously displaying a third display image based on the third rendered image on a first portion of the display and displaying a fourth display image based on the fourth rendered image on a second portion of the display. In various implementations, the third display image and the fourth display image are displayed before the first display image and the second display image. For example, the third display image and the fourth display image may correspond to Figure 9 The first left rendered image and the first right rendered image of block 910 in FIG. 1 and thus may be displayed before the resolution constraint is detected. In various implementations, the third display image and the second display image are displayed after the first display image and the second display image, for example, when the resolution constraint is lifted. In various implementations, the third display image and the second display image may correspond to Figure 9 The second left rendered image and the second right rendered image of block 940 may be displayed before or after the first display image or the second display image.

[0103] although Figure 11 A method is described in which two rendered images to be presented simultaneously have different resolutions, but in various specific implementations, the method includes generating two rendered images to be presented simultaneously with different perceptual qualities (such as resolution, color gamut, aliasing, frame rate, etc.).

[0104] Figure 12is a block diagram of an example of a controller 110 according to some implementations. While certain specific features are illustrated, those skilled in the art will recognize from this disclosure that various other features are not illustrated for the sake of brevity and so as not to obscure more relevant aspects of the implementations disclosed herein. To this end, as a non-limiting example, in some implementations, the controller 110 includes one or more processing units 1202 (e.g., a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), a central processing unit (CPU), a processing core, and / or the like), one or more input / output (I / O) devices 1206, one or more communication interfaces 1208 (e.g., a universal serial bus (USB), FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE 802.11x, IEEE 802.16x, Global System for Mobile Communications (GSM), code division multiple access (CDMA), time division multiple access (TDMA), global positioning system (GPS), infrared (IR), Bluetooth, ZIGBEE, and / or similar types of interfaces), one or more programming (e.g., I / O) interfaces 1210, a memory 1220, and one or more communication buses 1204 for interconnecting these and various other components.

[0105] In some implementations, the one or more communication buses 1204 include circuits that interconnect system components and control communications between the system components. In some implementations, the one or more I / O devices 1206 include at least one of a keyboard, a mouse, a touchpad, a joystick, one or more microphones, one or more speakers, one or more image sensors, one or more displays, and the like.

[0106] Memory 1220 includes high-speed random access memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), double data rate random access memory (DDR RAM), or other random access solid-state memory devices. In some embodiments, memory 1220 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 1220 optionally includes one or more storage devices remotely located from one or more processing units 1202. Memory 1220 includes non-transitory computer-readable storage media. In some embodiments, memory 1220 or a non-transitory computer-readable storage medium of memory 1220 stores the following programs, modules, and data structures, or a subset thereof, including an optional operating system 1230 and an XR experience module 1240.

[0107] The operating system 1230 includes processes for handling various basic system services and for performing hardware-related tasks. In some implementations, the XR experience module 1240 is configured to manage and coordinate single or multiple XR experiences for one or more users (e.g., a single XR experience for one or more users, or multiple XR experiences for corresponding groups of one or more users). To this end, in various implementations, the XR experience module 1240 includes a data acquisition unit 1242, a tracking unit 1244, a coordination unit 1246, and a data sending unit 1248.

[0108] In some implementations, the data acquisition unit 1242 is configured to at least Figure 1 The electronic device 120 acquires data (e.g., presentation data, interaction data, sensor data, location data, etc.). To this end, in various specific implementations, the data acquisition unit 1242 includes instructions and / or logic components for these instructions as well as heuristics and metadata for the heuristics.

[0109] In some implementations, the tracking unit 1244 is configured to map the physical environment 105 and to track at least the electronic device 120 relative to the physical environment 105. Figure 1 To this end, in various implementations, the tracking unit 1244 includes instructions and / or logic for these instructions as well as heuristics and metadata for the heuristics.

[0110] In some implementations, the coordination unit 1246 is configured to manage and coordinate the XR experience presented to the user by the electronic device 120. To this end, in various implementations, the coordination unit 1246 includes instructions and / or logic for these instructions, as well as heuristics and metadata for the heuristics.

[0111] In some implementations, the data sending unit 1248 is configured to send data (e.g., presentation data, location data, etc.) to at least the electronic device 120. To this end, in various implementations, the data sending unit 1248 includes instructions and / or logic components for these instructions, as well as heuristics and metadata for the heuristics.

[0112] Although the data acquisition unit 1242, the tracking unit 1244, the coordination unit 1246, and the data sending unit 1248 are shown as residing on a single device (e.g., the controller 110), it should be understood that in other specific implementations, any combination of the data acquisition unit 1242, the tracking unit 1244, the coordination unit 1246, and the data sending unit 1248 may be located in separate computing devices.

[0113] also, Figure 12It serves more as a functional description of various features that may be present in a particular implementation, rather than as a structural diagram of the implementations described herein. As one of ordinary skill in the art will recognize, items shown separately may be combined, and some items may be separated. For example, Figure 12 Some functional modules shown separately in the figure may be implemented in a single module, and the various functions of a single functional block may be implemented by one or more functional blocks in various implementations. The actual number of modules and the division of specific functions and how the features are distributed among them will vary depending on the specific implementation and, in some implementations, will depend in part on the specific combination of hardware, software, and / or firmware selected for a particular implementation.

[0114] Figure 13 is a block diagram of an example of an electronic device 120 according to some implementations. While certain specific features are illustrated, those skilled in the art will recognize from this disclosure that various other features are not illustrated for the sake of brevity and so as not to obscure more relevant aspects of the implementations disclosed herein. To this end, as a non-limiting example, in some implementations, the electronic device 120 includes one or more processing units 1302 (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, etc.), one or more input / output (I / O) devices and sensors 1306, one or more communication interfaces 1308 (e.g., USB, FIREWIRE, THUNDERBOLT, IEEE802.3x, IEEE 802.11x, IEEE 802.16x, GSM, CDMA, TDMA, GPS, IR, Bluetooth, ZIGBEE, and / or similar types of interfaces), one or more programming (e.g., I / O) interfaces 1310, one or more XR displays 1312, one or more optional internal-facing and / or external-facing image sensors 1314, memory 1320, and one or more communication buses 1304 for interconnecting these and various other components.

[0115] In some implementations, the one or more communication buses 1304 include circuits that interconnect system components and control communications between the system components. In some implementations, the one or more I / O devices and sensors 1306 include at least one of the following: an inertial measurement unit (IMU), an accelerometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., a blood pressure monitor, a heart rate monitor, a blood oxygen sensor, a blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, or one or more depth sensors (e.g., structured light, time of flight, etc.), etc.

[0116] In some implementations, one or more XR displays 1312 are configured to provide an XR experience to the user. In some implementations, the one or more XR displays 1312 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light-emitting field effect transistor (OLET), organic light-emitting diode (OLED), surface conduction electron emission display (SED), field emission display (FED), quantum dot light-emitting diode (QD-LED), microelectromechanical system (MEMS) and / or similar display types. In some implementations, the one or more XR displays 1312 correspond to diffraction, reflection, polarization, holographic, and other waveguide displays. For example, the electronic device 120 includes a single XR display. In another example, the electronic device includes an XR display for each eye of the user. In some implementations, the one or more XR displays 1312 are capable of presenting MR and VR content.

[0117] In some implementations, the one or more image sensors 1314 are configured to obtain image data corresponding to at least a portion of the user's face, including the user's eyes (and can be referred to as an eye-tracking camera). In some implementations, the one or more image sensors 1314 are configured to face forward so as to obtain image data corresponding to the physical environment that the user would see when the electronic device 120 is not present (and can be referred to as a scene camera). The one or more optional image sensors 1314 can include one or more RGB cameras (e.g., having a complementary metal oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor), one or more infrared (IR) cameras, and / or one or more event-based cameras, among others.

[0118] Memory 1320 includes high-speed random access memory, such as DRAM, SRAM, DDRRAM, or other random access solid-state memory devices. In some specific implementations, memory 1320 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 1320 optionally includes one or more storage devices remotely located from one or more processing units 1302. Memory 1320 includes non-transitory computer-readable storage media. In some specific implementations, memory 1320 or a non-transitory computer-readable storage medium of memory 1320 stores the following programs, modules, and data structures, or a subset thereof, including an optional operating system 1330 and an XR rendering module 1340.

[0119] The operating system 1330 includes processes for handling various basic system services and for performing hardware-related tasks. In some implementations, the XR rendering module 1340 is configured to present XR content to a user via one or more XR displays 1312. To this end, in various implementations, the XR rendering module 1340 includes a data acquisition unit 1342, a resolution function generation unit 1344, an XR rendering unit 1346, and a data transmission unit 1348.

[0120] In some implementations, the data acquisition unit 1342 is configured to at least Figure 1 The controller 110 acquires data (e.g., presentation data, interaction data, sensor data, location data, etc.). To this end, in various implementations, the data acquisition unit 1342 includes instructions and / or logic components for these instructions as well as heuristics and metadata for the heuristics.

[0121] In some implementations, the resolution function generation unit 1344 is configured to generate different resolution functions for rendering images for different eyes of the user. To this end, in various implementations, the resolution function generation unit 1344 includes instructions and / or logic for the instructions, as well as heuristics and metadata for the heuristics.

[0122] In some implementations, the XR rendering unit 1346 is configured to display the transformed image via one or more XR displays 1312. To this end, in various implementations, the XR rendering unit 1346 includes instructions and / or logic for instructions, as well as heuristics and metadata for the heuristics.

[0123] In some implementations, the data sending unit 1348 is configured to send data (e.g., presentation data, location data, etc.) to at least the controller 110. In some implementations, the data sending unit 1348 is configured to send authentication credentials to the electronic device. To this end, in various implementations, the data sending unit 1348 includes instructions and / or logic for these instructions, as well as heuristics and metadata for the heuristics.

[0124] Although the data acquisition unit 1342, the resolution function generation unit 1344, the XR rendering unit 1346, and the data sending unit 1348 are shown as residing on a single device (e.g., the electronic device 120), it should be understood that in other specific implementations, any combination of the data acquisition unit 1342, the resolution function generation unit 1344, the XR rendering unit 1346, and the data sending unit 1348 may be located in separate computing devices.

[0125] also, Figure 13It serves more as a functional description of various features that may be present in a particular implementation, as opposed to a schematic diagram of the structure of the implementation described herein. As one of ordinary skill in the art will recognize, items shown separately may be combined, and some items may be separated. For example, Figure 13 Some functional modules shown separately in the figure may be implemented in a single module, and the various functions of a single functional block may be implemented by one or more functional blocks in various implementations. The actual number of modules and the division of specific functions and how the features are distributed among them will vary depending on the specific implementation and, in some implementations, will depend in part on the specific combination of hardware, software, and / or firmware selected for a particular implementation.

[0126] Although various aspects of specific implementations within the scope of the appended claims are described above, it should be apparent that the various features of the above-mentioned specific implementations can be embodied in a variety of forms, and any specific structure and / or function described above is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that the aspects described herein can be implemented independently of any other aspects, and two or more of these aspects can be combined in various ways. For example, a device and / or a method can be implemented using any number of aspects set forth herein. In addition, in addition to or different from one or more aspects set forth herein, such a device and / or such a method can be implemented using other structures and / or functions.

[0127] It will also be understood that, although the terms "first," "second," etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description as long as all occurrences of "first node" are consistently renamed and all occurrences of "second node" are consistently renamed. A first node and a second node are both nodes, but they are not the same node.

[0128] The terms used herein are merely for describing specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms "a", "an", and "the" are intended to also cover the plural forms, unless the context clearly indicates otherwise. It will also be understood that the terms "and / or" used herein refer to and cover any and all possible combinations of one or more of the associated listed items. It will be further understood that the term "comprising" when used in this specification specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groupings.

[0129] As used herein, the term “if” may be interpreted to mean “when the precondition is true” or “when the precondition is true” or “in response to determining” or “upon determining” or “in response to detecting” that the precondition is true, depending on the context. Similarly, the phrase “if it is determined that [the precondition is true]” or “if [the precondition is true]” or “when [the precondition is true]” may be interpreted to mean “upon determining that the precondition is true” or “in response to determining” or “upon determining” that the precondition is true or “when detecting that the precondition is true” or “in response to detecting” that the precondition is true, depending on the context.

Claims

1. A method comprising: At a device comprising one or more processors, non-transitory memory, and a display: obtaining a first resolution function and a second resolution function, wherein the second resolution function is different from the first resolution function; generating a first rendered image based on content and the first resolution function and generating a second rendered image based on the content and the second resolution function; as well as Simultaneously, a first display image based on the first rendered image is displayed on a first portion of the display and a second display image based on the second rendered image is displayed on a second portion of the display. 2 . The method of claim 1 , wherein a maximum value of the second resolution function is different from a maximum value of the first resolution function.

3. The method according to claim 1 or 2, wherein the summed value of the second resolution function is different from the summed value of the first resolution function.

4. The method according to any one of claims 1 to 3, wherein the second resolution function at each angle is equal to the smaller of: the maximum value of the first resolution function and the second resolution function at that angle.

5. The method according to any one of claims 1 to 4, wherein obtaining the first resolution function and the second resolution function comprises: generating the first resolution function based on a formula having a set of variables, the set of variables having a first set of values; as well as The second resolution function is generated based on the formula having the set of variables having a second set of values.

6. The method of any one of claims 1 to 5, wherein the first portion of the display is positioned in front of a first eye of a user and the second portion of the display is positioned in front of a second eye of the user. 7 . The method according to claim 1 , further comprising detecting a resolution constraint, wherein a sum of a first summed value of the first resolution function and a second summed value of the second resolution function satisfies the resolution constraint. 8 . The method of claim 7 , wherein obtaining the first resolution function and the second resolution function is performed in response to detecting the resolution constraint.

9. The method of any one of claims 1 to 8, further comprising determining that the device is to perform monocular resolution reduction, wherein obtaining the first resolution function and the second resolution function is performed in response to determining that the device is to perform monocular resolution reduction.

10. The method of claim 9, wherein determining that the device is to perform monocular resolution reduction is based on a user preference.

11. The method of claim 9 or 10, wherein determining that the device is to perform monocular resolution reduction is based on the content.

12. The method according to any one of claims 1 to 11, further comprising selecting the first resolution function or the second resolution function to have a lower summation value, wherein obtaining the first resolution function and the second resolution function is performed in response to selecting the first resolution function or the second resolution function to have a lower summation value, and wherein the selected one of the first resolution function or the second resolution function has the lower summation value.

13. The method of claim 12, wherein selecting the first resolution function or the second resolution function is based on user preference.

14. The method of claim 12 or 13, wherein selecting the first resolution function or the second resolution function is based on the content.

15. The method of any one of claims 12 to 14, wherein selecting the first resolution function or the second resolution function is based on a variable that alternates between sessions.

16. The method according to any one of claims 1 to 15, further comprising: generating a third rendered image based on the content and a third resolution function and generating a fourth rendered image based on the content and the third resolution function; as well as Simultaneously, a third display image based on the third rendered image is displayed on the first portion of the display and a fourth display image based on the fourth rendered image is displayed on the second portion of the display.

17. A device comprising: monitor; non-transitory memory; and One or more processors configured to: obtaining a first resolution function and a second resolution function, wherein the second resolution function is different from the first resolution function; generating a first rendered image based on content and the first resolution function and generating a second rendered image based on the content and the second resolution function; as well as Simultaneously, a first display image based on the first rendered image is displayed on a first portion of the display and a second display image based on the second rendered image is displayed on a second portion of the display.

18. A device comprising: monitor; one or more processors; non-transitory memory; and One or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the apparatus to perform the method of any one of claims 1 to 16.

19. A non-transitory memory storing one or more programs which, when executed by one or more processors of a device comprising a display, cause the device to perform the method of any one of claims 1 to 16.

20. A device comprising: monitor; one or more processors; non-transitory memory; and Means for causing the apparatus to perform the method according to any one of claims 1 to 16.

Images