Optical artifact mitigation and content display enhancement

Abstract

The invention relates to optical artifact mitigation and content display enhancement. Various implementations disclosed herein include devices, systems, and methods for presenting content items (e.g., movies, television programs, homemade videos, etc.) on an electronic device, such as an HMD. Some implementations adjust content being displayed by an electronic device to mitigate optical module-based artifacts (e.g., ghosts). For example, in an HMD with a reflective refractive lens, the specular layer may leak some light, resulting in ghosting artifacts, which may be mitigated by adjusting brightness, dynamic range, contrast, light spillover, color, etc. Some implementations utilize content item perception-based adjustments (e.g., adjustments based on peak brightness of a scene in a movie being displayed within an augmented reality (XR) environment, etc.). Some implementations provide adjustments based on environmental perception (e.g., how dark ambient or transparent environment is) and / or optical module modeling. These adjustments may alter a particular content item and / or an environment in which the content item is presented.

Description

Technical Field

[0001] The present disclosure generally relates to systems, methods, and devices for mitigating optical artifacts that may occur in the presentation of content items (e.g., movies, television shows, etc.) within an extended reality (XR) environment and / or providing other content display enhancements on an electronic device such as a head mounted device (HMD). Background Art

[0002] Displaying content on an electronic device may result in various optical artifacts and other undesirable characteristics. For example, in an HMD with a catadioptric lens, the mirror layer may leak some light, thereby producing ghosting artifacts. Existing techniques may not be able to adequately, efficiently, or effectively mitigate these and other optical artifacts and undesirable characteristics. Summary of the invention

[0003] Various specific implementations disclosed herein include devices, systems, and methods for presenting content items (e.g., movies, TV shows, homemade videos, etc.) on electronic devices such as HMDs. Some specific implementations adjust the content being displayed by the electronic device to mitigate artifacts (e.g., ghosting) based on optical modules. For example, in an HMD with a reflective refractive lens, the mirror layer may leak some light, thereby generating ghosting artifacts, which can be mitigated by adjusting brightness, dynamic range, contrast, light overflow, color, etc. Some specific implementations utilize adjustments based on content item perception (e.g., adjustments based on the peak brightness of scenes in a movie being displayed in an extended reality (XR) environment, etc.). Some specific implementations provide adjustments based on environmental perception (e.g., how dark the virtual environment or transparent environment around the content item is). Some specific implementations provide adjustments based on optical module modeling. Some specific implementations provide adjustments based on viewing conditions (such as where the user is looking). Mitigation adjustments can change specific content items and / or the environment in which the content item is presented. The adjustment can be determined to maintain the artist's intent based on content type and / or using other criteria.

[0004] In some implementations, an electronic device has a processor (e.g., one or more processors) that executes instructions stored in a non-transitory computer-readable medium to perform a method. The method performs one or more steps or processes. In some implementations, the method involves obtaining a content item to be rendered within a view of an XR environment (e.g., a movie, television program, application, etc. that can be rendered in an immersive view or within a virtual or real environment).

[0005] The method may involve identifying visual characteristics of the content item (e.g., content type, scene type, peak brightness, minimum brightness, brightness histogram, etc.). The characteristics may be determined based on content metadata and / or environmental metadata specific to one or more frames or scenes, a histogram or spatial analysis of content brightness, etc. Such metadata may identify a location within the content item where a particular brightness feature occurs, e.g., an x / y location.

[0006] The method may involve determining display parameters of the view to mitigate the appearance of optical artifacts associated with the visual characteristics of the segment of the identified content item. The display parameters may correspond to adjusting brightness (e.g., reducing the brightness of highlights or hiding reflections by reducing the brightness in reflective areas), dynamic range (e.g., fitting / mapping to a specific range), black level, contrast, color, tinting, etc. of the content item and / or (if applicable) the environment surrounding the content item (e.g., light spill, tinting, etc.). The adjustments may be smoothed (e.g., across multiple frames) or timed to occur when the scene changes to avoid significant changes. The method also involves: presenting a view with the content item within the XR environment via a display according to the determined display parameters.

[0007] Some implementations adjust content being displayed on an electronic device such as an HMD to mitigate artifacts (e.g., ghosting) based on the optical module. The adjustments can be based on optical module modeling, including but not limited to modeling that takes into account the relative positioning of the display panel and / or the eyes of the viewing user. An optical stack model (e.g., a 3D model for tracing rays from the display location to the user's eye location) can be used to predict where and how intense artifacts may occur.

[0008] In some embodiments, the electronic device has a processor (e.g., one or more processors) that executes instructions stored in a non-transitory computer-readable medium to perform a method. The method performs one or more steps or processes. In some embodiments, the method involves obtaining a content item to be presented within a view of an XR environment. The method may involve predicting characteristics (e.g., saliency, brightness, size, position, etc.) of optical artifacts of the content item. The characteristics of the optical artifacts may be predicted based on modeling the light path using a 3D model of the optical module of the HMD and the eye position relative to the 3D model. The method may involve determining display parameters of the view based on the predicted characteristics to reduce the appearance of optical artifacts. The method may involve presenting a view with a content item within the XR environment via a display according to the determined display parameters.

[0009] In some implementations, the content item is presented in an XR environment, and one or more effects are applied to the XR environment, for example, to change the appearance of the content item and / or the appearance of the window or virtual screen around which the content item is presented. In some implementations, the content item is presented in a pass-through video modified with one or more enhancements or other effects. For example, an effect can be used to change the view of a live pass-through video, where the content item displayed on the virtual screen is positioned to appear at a position within the environment depicted by the pass-through video, and a light overflow effect can be applied to the pass-through video around the virtual screen. In a pass-through implementation, the effect can be added by changing the display properties of the pass-through content around the object. In some implementations, the light overflow effect can be added by modifying the display properties of a portion of the live pass-through content (e.g., a wall, floor, furniture, etc.) located around or adjacent to the content item. In other examples, the effect can utilize display parameters that adjust, in particular, color properties, shading properties, dimming properties, glow properties, etc.

[0010] In some implementations, a process implemented by a hardware mixing architecture (e.g., a mixing circuit) can be used to mix a pass-through video frame of live pass-through video content with the content of a virtual object. The mixing process can also be configured to achieve effects such as light overflow effects. In some implementations, the mixing process can include a shading / color mixing process using hardware-based logical pixel operations. The mixing process can include any type of mixing technology.

[0011] Some embodiments provide gaze-based contrast adjustment to provide an enhanced user experience. In some embodiments, the identified gaze direction can be used to identify a peripheral portion of a view, and the local contrast can be suppressed in the portion. In one example, the local contrast can be maintained in the foveal portion of the view, while the local contrast can be flattened in the portion of the view corresponding to the peripheral vision. In some embodiments, the electronic device has a processor (e.g., one or more processors) that executes instructions stored in a non-transitory computer-readable medium to perform a method. The method performs one or more steps or processes. In some embodiments, the method may involve obtaining a content item to be rendered within a view of an extended reality (XR) environment on an HMD. The method may involve identifying a gaze direction of a user of the HMD. The method may involve identifying a portion of a view associated with peripheral vision based on a gaze direction. The method may involve presenting a view with a content item within an XR environment via a display, wherein the portion of the view associated with the peripheral vision is adjusted according to a display parameter. In some embodiments, the display parameter specifies that the contrast within the portion of the view associated with the peripheral vision is flattened. In some embodiments, the display parameter specifies that the portion of the view associated with the peripheral vision is dimmed.

[0012] According to some specific implementations, a device includes one or more processors, non-volatile memory, and one or more programs; the one or more programs are stored in the non-volatile memory and are configured to be executed by one or more processors, and the one or more programs include instructions for performing or causing the execution of any of the methods described herein. According to some specific implementations, a non-volatile computer-readable storage medium stores instructions that, when executed by one or more processors of the device, cause the device to perform or cause the execution of any of the methods described herein. According to some specific implementations, a device includes: one or more processors, non-volatile memory, and means for performing or causing the execution of any of the methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] So that the present disclosure can be understood by those of ordinary skill in the art, a more detailed description may be obtained with reference to aspects of some exemplary implementations, some of which are illustrated in the accompanying drawings.

[0014] Figure 1 Example electronic devices are shown operating in a physical environment according to some implementations.

[0015] Figure 2 According to some specific implementations, Figure 1 The view of the XR environment provided by the device.

[0016] Figure 3 Shows Figure 2 Optical artifacts in the view of the XR environment.

[0017] Figure 4 According to some specific implementations Figure 3 Reduction of optical artifacts of the view.

[0018] Figure 5 Shows Figure 2 Optical artifacts in the view of the XR environment.

[0019] Figure 6 According to some specific implementations Figure 5 Reduction of optical artifacts of the view.

[0020] Figure 7 Shows Figure 2 A view of the XR environment.

[0021] Figure 8 According to some specific implementations, Figure 7 Enhancements to the view.

[0022] Fig. 9A lens double-bounce ghosting artifact produced by an exemplary optical module of an electrical device is shown.

[0023] Fig.10 Optical artifact prediction based on optical module modeling according to some specific implementations is shown.

[0024] Fig.11 Optical artifacts caused by double bounce of the lens are shown.

[0025] Fig.12 is a flow chart illustrating an exemplary method of mitigating optical artifacts according to some implementations.

[0026] Fig.13 is a flow chart illustrating another exemplary method of mitigating optical artifacts according to some implementations.

[0027] Fig.14 is a flow chart illustrating an exemplary method of display enhancement according to some implementations.

[0028] Fig.15 is a block diagram of an electronic device according to some specific implementations.

[0029] As a matter of practice, the various features shown in the drawings may not be drawn to scale. Therefore, for clarity, the sizes of the various features may be arbitrarily enlarged or reduced. In addition, some drawings may not depict all components of a given system, method, or device. Finally, throughout the specification and drawings, the same reference numerals may be used to represent the same features. DETAILED DESCRIPTION

[0030] In order to provide a thorough understanding of the example implementations shown in the drawings, many details are described. However, the drawings only illustrate some example aspects of the present disclosure and should not be considered limiting. It will be understood by those of ordinary skill in the art that other effective aspects and / or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits are not described in detail so as not to obscure more relevant aspects of the example implementations described herein.

[0031] Figure 1 An exemplary electronic device 105 is shown operating in the physical environment 100. Figure 1In the example of , the physical environment 100 is a room with a plant 120, a sofa 125, a door 130, and a table 135. The electronic device 105 includes one or more cameras, microphones, depth sensors, or other sensors, which can be used to capture and evaluate information about the physical environment 100 and objects therein, as well as information about the user 102 of the electronic device 105. The information about the physical environment 100 and / or the user 102 can be used to provide visual and audio content, and / or identify the current location of the physical environment 100 and / or the location of the user within the physical environment 100.

[0032] In some implementations, a view of an extended reality (XR) environment may be provided to one or more participants (e.g., user 102 via electronic device 105, and / or other participants not shown via other electronic devices not shown). Such an XR environment may be a 3D environment generated based on a camera image and / or a depth camera image of the physical environment 100, and a representation of the user 102 generated based on a camera image and / or a depth camera image of the user 102. Such an XR environment may include virtual content (e.g., one or more virtual content items) that are positioned at one or more 3D locations relative to a 3D coordinate system associated with the XR environment, which may correspond to the 3D coordinate system of the physical environment 100. In one example, the XR environment includes virtual content items displayed within a completely virtual surrounding environment. In one example, the XR environment includes virtual content items displayed within a completely real surrounding environment (e.g., via a pass-through portion or a see-through portion of an HMD). In one example, the XR environment includes virtual content items displayed within a surrounding environment that includes both real elements and virtual elements. In one example, the XR environment provides an immersive view of only virtual content items, i.e., without the surrounding environment. In a view of an XR environment, the appearance of content items and / or the surrounding environment may be configured or modified to present mitigations (e.g., light bloom, lighting effects, etc.) that mitigate the appearance of artifacts and / or otherwise enhance the view or experience.

[0033] In some implementations, the XR environment is presented using a pass-through video depicting a physical environment (e.g., physical environment 100). For example, the pass-through video can be provided based on receiving and presenting an image from an image sensor (e.g., an outward-facing camera) of a device (e.g., device 105). In some implementations, a virtual content item is presented with the pass-through video content, and one or more mitigations (e.g., display parameter selections or changes) are generated to change the appearance of the virtual content item and / or a portion of the view around or near the content item.

[0034] Optical artifact mitigation can take various forms. In some implementations, artifact mitigation is provided by configuring a light overflow effect. The light overflow effect can be configured with respect to size, position, brightness, color, etc. to change the appearance of the pass-through video, thereby reducing the appearance of optical artifacts. The light overflow effect may involve, for example, changing the appearance of a portion of the view (e.g., by shading and / or changing brightness) based on content displayed as a virtual content item (e.g., based on brightness properties, color, etc. of the content item). The light overflow effect can be presented in a realistic manner, for example, presenting a virtual light overflow similar to the light overflow that a real TV or the like might provide. In some implementations, the light overflow effect can be enabled relative to the pass-through video content using a constructed geometric mesh (e.g., providing surface normals or other environmental surface information that can be used to provide or enhance lighting effects). In some implementations, the bloom effect (e.g., tinting / color blending) is generated by changing the appearance of frames of a pass-through video (of the physical environment) captured by an outward-facing camera 205 connected to the display via a dedicated hardware path of an application-specific integrated circuit (ASIC).

[0035] The processor may be configured to process frames of the pass-through video at a relatively high frames per second (FPS) rate, such as, for example, a frame rate greater than 60 FPS. In some implementations, the ASIC retrieves each frame (of the pass-through video) from an outward-facing camera and alters each pass-through frame with frame-specific effects. The process may include an alpha blending process for combining each frame of the pass-through video (e.g., background content) with frame-specific virtual content. In some implementations, the alpha-blending values ​​associated with the pass-through video in areas corresponding to virtual content items and / or the surrounding environment may be adjusted. In some implementations, blending the pass-through frames with the virtual content using a hardware-based process may enable the process to, for example, quickly add virtual content items and optical artifact mitigation effects in a real-time live view, and / or use fewer resources than might otherwise be required.

[0036] Changing the portion of each frame of the pass-through video may be performed via the operation of a processor (e.g., an ASIC), and may be, but is not limited to, coloring, dimming, or changing the brightness of the corresponding portion of each frame. In some specific implementations, the modification process may include performing hardware-implemented logic operations via an ASIC.

[0037] Various techniques may be used to provide artifact mitigation. Some techniques are gaze and content independent. For example, such techniques may involve changing the viewing position or size of an area (e.g., a virtual screen) on which virtual content is positioned. This may involve changing the absolute position and / or size of the area / screen, or changing the position and / or size of the area screen relative to the user's viewpoint, for example, within an XR environment. In another example, such techniques involve changing the ambient brightness of the XR environment, for example, by increasing or decreasing the ambient illumination in a manner independent of the virtual content.

[0038] Some techniques for providing artifact mitigation rely on content. For example, based on the type and / or characteristics of the virtual content item, the tone mapping of the content can be adjusted to reduce artifact significance. In another example, based on the type and / or characteristics of the virtual content item, a light overflow effect is used to reduce artifact significance. As an example, this can involve controlling the amplitude of the light overflow and / or the degree of overflow (e.g., the affected spatial area) based on the virtual content. In another example, based on the type and / or characteristics of the virtual content item, the brightness of the surrounding environment of the XR environment can be adjusted. This can involve dynamically adjusting the illumination.

[0039] Some techniques for providing artifact mitigation rely on gaze. For example, gaze-based dimming can be employed to reduce artifact significance. For example, dimming can be selectively employed based on identifying an area around the user's gaze direction, such as a foveal gaze area. Some specific implementations involve structures that control temporal filtering.

[0040] Some techniques for providing artifact mitigation rely on gaze and content. For example, adjustments can be made based on both the virtual content item type / characteristics and gaze. This can involve, for example, foveal tone mapping, for example, tone mapping part or all of the content (e.g., only specific areas based on the user's gaze / foveal gaze area). This can involve changing the tone based on actual pixel values. Another example involves background subtraction based on virtual content item type / characteristics and gaze. Another example involves performing foveal average pixel level (APL) boosting based on virtual content item type / characteristics and gaze, which can involve applying a boosting function with amplitude and / or size based on gaze / foveal area and / or virtual content.

[0041] Some techniques for providing artifact mitigation rely on content and pupil location. For example, adjustments can be made based on a real-time computed image of undesirable artifacts based on both virtual content and user pupil location. This can involve, for example (ghosting cancellation) subtracting luminance from the background of an expected image (when available) that matches the unintended contribution (ghosting prediction), which utilizes hardware and implementation to provide a real-time lens model pupil size beam trajectory that provides a second-order point spread function (based on non-uniform projection geometry) for each pixel that is sufficient to correctly model the nonlinear lens dynamics, which is then coupled with adaptive multi-anisotropic texture sampling across multiple mip levels so that for each color channel, all contributing light is summed, also attenuated by the reflection / transmission efficiency (for the corresponding light path) provided by the ML model trained from polarized ray tracing simulations.

[0042] Some techniques for providing artifact mitigation rely on content, pupil location, and gaze.

[0043] Figure 2 The device (e.g., Figure 1 1. The view 200 of an exemplary XR environment provided by the device 105 of FIG. 1 includes a physical environment (e.g., Figure 1 100 ), and virtual content items 240 (e.g., movies, television shows, etc.) presented in a 2D display window 240 positioned at a 3D location within the XR environment. The pass-through video 205 includes a depiction 220 of the flower 120, a depiction 225 of the sofa 125, a depiction 230 of the door 130, a depiction 235 of the table 135, and the like. The virtual content items 240 are fixed in their positions relative to other objects in the XR environment. Thus, as the user moves the device 105 around the physical environment 100 to view the XR environment from different viewpoints, the virtual content items 240 may appear to remain in a fixed position, e.g., world-locked to appear to remain in a fixed position on a wall of the physical environment 100.

[0044] Figure 3 Shows Figure 2 305 of an XR environment. In this example, virtual content item 240 includes a depiction of bright flames 270, and optical artifact 310 (corresponding to bright flames 270) is visible within view 305 of the XR environment. In this example, the portion of view 305 surrounding virtual content item 240 is relatively dim or dark, which enhances the visibility / significance of optical artifact 310.

[0045] Figure 4 The reduction of optical artifact 310 is shown in view 405 (the view is Figure 3305 of the device). In this example, the device determines one or more characteristics of the optical artifact 310 based on the content item 240, the surrounding environment (e.g., the transmittal view characteristics), and / or the optical module used by the device. The one or more characteristics of the optical artifact may, but need not necessarily, include: the brightness of the artifact, the relative brightness of the artifact to nearby items, the size of the artifact, the location of the optical artifact in the view (e.g., relative to the content item 240 or the surrounding environment content), the significance of the artifact (e.g., on a numerical scale), other characteristics associated with the significance of the artifact, etc.

[0046] One or more characteristics of the optical artifact may be determined, for example, based on metadata identifying peak brightness, average brightness, a histogram representing the amount of pixels within various brightness ranges for the content item, or the like, brightness information about a current frame or current scene of a movie (in an example where the movie is content item 240).

[0047] One or more characteristics of the optical artifact can be determined based on information about the surrounding environment (e.g., based on the current capabilities and / or dynamic range utilization of the display, the lighting in the environment, etc.). Information about the surrounding environment can be determined by the device. For example, in the case of a transparent video, the device can analyze one or more frames of the transparent video to identify brightness characteristics. In addition, since the transparent video depicts the physical environment, information about the physical environment captured by other sensors such as ambient light sensors, motion sensors, depth sensors, thermal sensors, etc. (e.g., in addition to the camera that captures the transparent video) can be used to determine scene understanding, from which information about the surrounding environment and / or a view of the surrounding environment can be determined.

[0048] One or more characteristics of the optical artifacts may be determined based on information about the optical module used to view view 305, for example, using an optical module modeling technique that predicts the location, size, intensity, etc. of expected artifacts via 3D modeling. For example, a 3D model of the optical module (such as Fig. 9 An exemplary optical module of the invention can be used to model or simulate light reflections (e.g., expected reflections and unexpected artifact reflections) to determine characteristics of optical artifacts, such as where the artifacts may appear, how large and / or bright the artifacts will be, etc.

[0049] exist Figure 4 In the example of , based on one or more characteristics of the optical artifact, the device determines to provide mitigation. In other cases (e.g., less noticeable artifacts), the device may determine to omit providing mitigation. Figure 4In this example of , the mitigation is a light bloom effect 320 that masks or hides the appearance of optical artifact 310. In this example, light bloom effect 320 is provided by increasing the brightness and / or tinting in an area of ​​the view proximate to virtual content item 240. The amount of increase and / or tinting may decrease based on the distance away from virtual content item 240 within view 405. Light bloom effect 320 may give the appearance of virtual light emanating from virtual content item 240 to illuminate nearby portions of the depicted environment.

[0050] Mitigation, such as bloom effect 320, can be configured based on one or more characteristics of the optical artifact to hide or reduce the noticeability of the optical artifact. In this example, based on the optical artifact 310 being located proximate to the virtual content item (e.g., within a bloom mitigation threshold distance) and being relatively bright to surrounding portions of the view (e.g., brighter than surrounding portions by at least a threshold level) and / or having a particular color (e.g., being white, yellow, etc.), the device determines to apply a bloom effect having a particular intensity (e.g., strong enough to cause the size of the bloom to overlap with the optical artifact 310), brightness (e.g., bright enough to reduce the noticeability of the optical artifact 310), and / or having other configured properties (e.g., using tinting or coloring that is consistent with the content displayed in content item 270 and is also suitable for reducing the noticeability of the optical artifact).

[0051] In addition to this example, the characteristics of the optical artifacts can be used to configure mitigation in a variety of other ways.

[0052] Figure 5 Shows Figure 2 3. Optical artifact 510 in view of XR environment of FIG. 505. In this example, virtual content item 240 includes a depiction of bright flame 270 in dark content scene 520, and optical artifact 510 (corresponding to bright flame 270) is visible within view 505 of XR environment. In this example, the portion of content item 240 surrounding flame 270 is relatively dark, which enhances the visibility / significance of optical artifact 310 covering it.

[0053] Figure 6 The reduction of optical artifact 610 of view 605 is shown (the view is Figure 5In this example, the device determines one or more characteristics of the optical artifact 510 based on the content item 240, the surrounding environment (e.g., the characteristics of the see-through view), and / or the optical module used by the device. The one or more characteristics of the optical artifact 510 may, but need not necessarily, include: the brightness of the artifact, the relative brightness of the artifact to nearby items, the size of the artifact, the location of the optical artifact in the view (e.g., relative to the content item 240 or the surrounding environment content), the expected significance of the artifact (e.g., on a numerical scale), other characteristics associated with the significance of the artifact, etc. Figure 4 As described, one or more characteristics of an optical artifact, such as optical artifact 510, may be determined based on metadata identifying peak brightness, average brightness, a histogram representing the amount of pixels within various brightness ranges of the content item, or the like, brightness information about a current frame or current scene of a movie (in the example where the movie is content item 240). Similarly, as described with respect to Figure 4 As described, one or more characteristics of an optical artifact, such as optical artifact 510 , may be determined based on information about the surrounding environment and / or information about an optical module used to view the view, for example, modeled using the optical module.

[0054] exist Figure 6 In the example of , based on one or more characteristics of the optical artifact, the device determines to provide mitigation. In other cases (e.g., less noticeable artifacts), the device may determine to omit providing mitigation. Figure 6 In this example, mitigation changes the characteristics of content item 240 to mask or hide the appearance of optical artifact 510, e.g., to instead produce optical artifact 610 that is less bright than optical artifact 510. In this example, the peak brightness of content item 240 (e.g., flame 270) is reduced, resulting in less bright optical artifact 610.

[0055] Mitigation (such as content item changes) can be configured based on one or more characteristics of the optical artifact to hide or reduce the noticeability of the optical artifact. In this example, based on the location of optical artifact 510 within content item 240 and the relative brightness to the surrounding portions of content item 240 (e.g., being at least a threshold level brighter than the surrounding portions) and / or having a particular color (e.g., being white, yellow, etc.), the device determines to reduce the brightness of content item 240 (and / or entire view 605) to reduce the brightness of optical artifact 510 to the brightness of optical artifact 610. Note that view 505 never needs to be presented, i.e., the device can predict optical artifact 505 and adjust the content so that view 605 is presented instead of view 505.

[0056] In addition to this example, the characteristics of the optical artifacts can be used to configure mitigation in a variety of other ways.

[0057] Optical artifact mitigation can involve various adjustments. In the context of a virtual content item within a view presenting an XR environment, some adjustments involve content item adjustments (e.g., pixel-by-pixel adjustments of one or more frames of content item content), and some adjustments involve environment or view adjustments (e.g., adjustments to the dynamic range or other brightness characteristics used to provide the XR environment and / or view). The adjustments can change over time, for example, changing whether mitigation is provided and / or how such mitigation is provided based on the current virtual content item content (e.g., the current frame or scene of a movie presented as a virtual content item) and / or the current environment (e.g., the current brightness level of the view of the XR environment). Some specific implementations take into account specific hardware components that cause optical artifacts, for example, based on modeling optical components and light that travels from a display through the optical components and reaches a viewpoint / eye position.

[0058] Some implementations take into account the user's expected or actual perceptual state (e.g., eye adaptation to light or dark conditions, amount of pupil dilation, etc.) when determining whether and / or how to provide optical artifact mitigation. For example, optical artifacts involving bright reflections may be more noticeable when the user is dark-adapted. The device may recognize that an upcoming frame or scene is likely to produce such optical artifacts, and, prior to that frame or scene, alter the brightness characteristics of the view to gradually change the user's perceptual state to be less dark-adapted, and thus make the appearance of the resulting optical artifact less noticeable.

[0059] In some implementations, optical artifact adjustments involve hardware and / or software mitigation techniques. Hardware techniques can change the characteristics of the optical modules (e.g., adjusting the layers of the optical stack) to reduce the appearance of optical artifacts. Software adjustments can adjust the appearance of a view and / or content items within the view (e.g., Figures 3 to 8 shown).

[0060] Content-aware lightening and enhancement

[0061] Mitigation techniques may be based at least in part on an understanding of the content (e.g., content items and / or environment). Mitigation techniques may be static (e.g., consistent during display of a scene or content item) or dynamic (e.g., changing over time, such as based on user / gaze changes).

[0062] In some implementations, mitigation can involve changing the dynamic range and / or raising the black point used to display the view in order to mask one or more optical artifacts that may occur, for example, during a particular scene in a movie. Similarly, brightness can be reduced (e.g., an upper value of the dynamic range (e.g., an SDR bright value)) to limit optical artifacts caused by bright lights that cause glare. In some implementations, the dynamic range is adjusted in a linear manner to mitigate optical artifacts. In some implementations, the dynamic range is adjusted in a nonlinear manner to account for optical artifacts. In some implementations, the changes are applied to the entire view (e.g., the entire FOV of the HMD), while in other implementations, the changes are applied only to certain areas (e.g., at or around areas where optical artifacts are predicted to occur).

[0063] Some specific implementations involve dynamic range adjustment. The dynamic range used to display content can be compressed to achieve one or more benefits. For example, dynamic range compression can be performed based on determining the absolute brightness of the content, environment, or view to provide a better viewing experience.

[0064] like Figure 3 to Figure 4 Some of the embodiments shown provide light overflow-based mitigation to reduce the appearance or noticeability of optical artifacts. This can involve adjusting properties of the light overflow effect (e.g., glow) based on the optical artifact characteristics. For example, given an expected bright optical artifact, the glow of the light overflow effect can be enlarged to cover more of the user's field of view and / or better reduce global contrast. Conversely, when the virtual content item is darker, adjustments to light overflow, contrast, etc. may not be required.

[0065] Information about the content of the virtual content item can be used in various ways to predict artifacts and / or determine mitigation. Depending on the content, the artifact may be predicted to have a certain level of structure, or to have more or less structure than a threshold, for example, as defined or diffuse. More structure may require more mitigation. Therefore, in some embodiments, content mitigation is determined based on the brightness and / or structure of the content and / or associated optical artifacts. Some embodiments algorithmically calculate how much mitigation (e.g., light overflow) is required based on evaluating the content of the content item. If the scene of the content item is determined to be unlikely to be affected by optical artifacts, it may not be necessary to utilize system resources or otherwise change the viewing experience, that is, there is no need to address insignificant phenomena.

[0066] Some implementations determine optical artifact characteristics and / or corresponding mitigations based on a combination of: (a) content that a user is viewing (e.g., a specific content item); (b) content in the user's peripheral vision (e.g., the environment surrounding the content item); and / or (c) optical module behavior / understanding. In some implementations, mitigations (such as light bloom effects) are strategically positioned to account for the location of optical artifacts. For example, light bloom can be applied to only the floor, only the ceiling, only the side walls, etc. of an environment.

[0067] Some implementations use pass-through tinting to mitigate the appearance of optical artifacts. For example, the average color of the optical artifact, the strongest color, the dominant color, the average color of the brightest half of the content, etc. can be determined and used to apply a tint that will blur or otherwise make the optical artifact less noticeable.

[0068] Based on the content analysis, environmental analysis and / or modeling of the optical module, one or more changes to the view (e.g., image) may be applied. This may involve reducing the brightness or contrast of the content item, or changing the brightness or contrast of the surrounding environment, such as displaying content within or outside the user's field of view.

[0069] In some implementations, mitigation is selected based on criteria that seek to preserve the intent of the content item creator. For example, this may involve prioritizing mitigation external to the content item (e.g., environmental changes) over mitigation that changes the appearance of the content item itself. In some implementations, minimal changes are applied and only applied when necessary to mitigate artifacts such as ghosting, for example, when an artifact is determined to have characteristics such as brightness or contrast that differ by more than a threshold.

[0070] In some cases, the deeper the black provided by the display, the more noticeable the optical artifacts will be. Reducing the depth of the displayed black can help make the optical artifacts less noticeable. In some cases, the greater the dynamic range provided by the display, the more noticeable the optical artifacts will be. Reducing the dynamic range of the display can help make the optical artifacts less noticeable. Therefore, some mitigations reduce the black depth level and / or reduce the dynamic range to mitigate optical artifacts. Static mitigation can perform content-aware adaptation of the black level and / or extended dynamic range (EDR) used by the display over time.

[0071] In some implementations, artifact mitigation decisions are made based on metadata (e.g., of a content item or environment.) From such metadata, understanding that a small area or item within a content item (e.g., a lightsaber) will potentially produce a bright and well-defined ghost, appropriate mitigation can be determined, e.g., determining that reducing contrast in the periphery will be insufficient, and therefore determining to reduce the brightness of a small area or item within the content itself (e.g., a lightsaber).

[0072] In some implementations, artifact mitigation is based on metadata associated with a specific time period (e.g., frame, scene, etc.) of a content item. For example, brightness metadata for a scene may provide an average brightness value across all frames of the scene. Statistics about frame or scene brightness in movie metadata and / or statistics derived during playback can be used for adaptation. As an example, such information may include peak brightness, average brightness, a histogram of brightness value distribution, etc. A histogram may, for example, provide the number of pixels with the brightness (i.e., brightness within an associated level range) for each brightness level / brightness level range, thereby providing a signal distribution relative to how many pixels fall into a specific nit range. Information stored in or otherwise associated with a content item may provide information about the content item creator's intent relative to the brightness that the content item should have when experienced. In some implementations, information about a content item identifies brightness information for each spatial portion (e.g., tile, patch, segment, grid area, etc.) of the content item. Such metadata may specify, for example, brightness information for each pixel of a frame / scene, which is the average of brightness information for nearby pixels, such as the average brightness of all pixels within a threshold distance, the average brightness of the nearest 9 pixels, or the average brightness of 16 pixels in an associated surrounding rectangular area, etc.

[0073] The histogram of brightness information may be obtained based on pre-existing metadata and / or generated metadata (e.g., metadata generated using an analysis process such as a video decoder). Thus, a histogram or other brightness information about a content item or environment may be predetermined or determined in real time during playback, streaming, or other live processing involved in rendering a view of an XR environment.

[0074] In some implementations, the display of the device is configured to use a brightness range (e.g., a dynamic range), and the range is adjusted over time based on optical artifact characteristics or related considerations. A view of the XR environment can be displayed / mapped into an appropriate range at a given point in time. Thus, based on optical artifact characteristics and / or related considerations, the content of a given view can be mapped to an appropriate range for display, for example, to hide artifacts while still providing a user with sufficient dynamic range to have a desired view experience.

[0075] In some implementations, the view depicts a content item (e.g., a movie) within the XR environment. The mitigation of optical artifacts in such conditions can be based in part on the relative size of the content item within the view of the XR environment. For example, when the content item occupies a small portion of the view (e.g., less than a threshold percentage of the view or less than a threshold percentage of the user's field of view), mitigation can be applied to change the appearance of the environment. Conversely, in the case where the content item occupies more than a portion of the view (e.g., more than a threshold percentage of the view or more than a threshold percentage of the user's field of view), mitigation can be applied to additionally (or alternatively) change the appearance of the content item itself.

[0076] In some implementations, the process utilizes inputs regarding (a) the content item (e.g., regarding brightness and artist intent), (b) the surrounding environment (e.g., regarding brightness, area, complexity, contrast, scene understanding, etc.), and / or (c) the optical module to produce an output that specifies a mitigation or adjustment (e.g., removing highlights, adding light bloom, selecting a specific dynamic range, etc.). In some implementations, these inputs are used to determine the relative brightness of the optical artifact relative to the background portion of the view surrounding the artifact, for example, determining that the artifact will be 3 times brighter than the background. Based on the optical artifact exceeding a threshold (e.g., at least 2 times brighter than the background, at least 1.2 times brighter than the background, etc.), the process can determine whether to apply mitigation and the characteristics of the mitigation. In some implementations, the peak brightness of the scene of the content item is used to predict the expected brightness of the optical artifact. In some cases, the dynamic range of the display is adjusted to reduce the expected relative brightness between the optical artifact and the background, for example, using only 50%, 80%, etc. of the dynamic range to reduce the relative brightness of the optical artifact to less than 2 times the average brightness of the background.

[0077] Some implementations provide different viewing modes, the viewing modes including a first mode in which no (or relatively less) optical artifact mitigation is applied and a second mode in which at least some (or relatively more) optical artifact mitigation is applied. In some implementations, the mitigation mode used by the device is automatically determined (e.g., based on determining a type of use of the device, such as productivity, entertainment, meditation, etc.). In some implementations, the mitigation mode used by the device is manually selected by a user.

[0078] Gaze-based mitigation and enhancement

[0079] Dynamic mitigation techniques can use dynamic information, such as user gaze data, to determine whether and how to apply mitigation or otherwise enhance a view to provide a better user experience. Some artifacts may be caused by parts of the view that the user is not watching. By associating where the user is watching with reducing the amount of light in the selected area based on where the user is watching, the optical artifact significance can be reduced, for example, by reducing the total amount of light provided. In some specific implementations, dynamic mitigation may involve gaze-based dimming of the peripheral parts of the view to mitigate optical artifacts. This may involve adjusting the content displayed in the user's periphery in a way that makes the optical artifacts less noticeable. Bright portions of the view or bright portions of content items displayed within the view can be identified, and selective dimming is applied to make potential optical artifacts associated with such bright portions less noticeable. Some specific implementations determine whether the user is watching a relatively bright area, and modify the display characteristics of the peripheral area accordingly (e.g., dynamic range, peak brightness, HDR on or off, theater on or off, etc.).

[0080] In some implementations, the gaze-based dimming curve is adjusted to fit the content. This can involve dimming the periphery more aggressively when the content in the periphery is typically already dark. For example, in cinema mode, the surrounding environment may be relatively dark, so the dimming may be more noticeable compared to conditions such as XR passthrough showing a bright room environment, where the dimming is more likely to be objectionable or noticeable.

[0081] In some implementations, local contrast is additionally or alternatively suppressed in the periphery. For example, this can involve a two-stage process. The first stage can involve reducing the brightness of the pixel using a nonlinear curve defining an alpha multiplier, where the falloff is based on the distance from the fovea (e.g., alpha multiplied by pixel (x, y) = f (distance from gaze position)). The second stage can involve adjusting the alpha multiplier based on the brightness value of each pixel to flatten the contrast in the periphery (e.g., based on the HDR range that contributes most to ghosting), thereby increasing the alpha multiplier. Doing so can maintain local contrast in the foveal region while flattening the contrast in the periphery.

[0082] Some implementations also involve utilizing smooth transitions between dimming curves and / or contrast changes. For example, such transitions may be appropriate when the content changes significantly or changes between scenes.

[0083] Optical ghosting, glare, and other artifacts may be caused by the user's gaze or other eye behavior (e.g., based on the user's gaze moving around). In addition, eye tracking inaccuracies may make lightening (e.g., selective dimming, contrast changes, etc.) more noticeable or otherwise distract the user.

[0084] Temporal filtering can be used to reduce artifact noticeability. Temporal algorithms can be used to apply mitigation in order to compensate for inaccuracies in tracking eye behavior (e.g., rapid eye movement (REM) or blinking). During REM or blinking, there may be a delay between where the gaze is and where the gaze tracking system has determined the gaze is. To compensate for those scenarios, the temporal system can smooth out the artifact mitigation so that the user does not notice that parts of their field of view become dim / bright, etc. when their eyes are experiencing REM or blinking.

[0085] Some implementations take into account user differences, for example, user personalization. Users have different brightness perceptions, so predictions of necessary or desired dimming amounts for an average user may not be applicable to some users. User-personalized dimming can be determined based on identifying individual characteristics of a given user. For example, users can be classified based on whether the user is nearsighted and therefore potentially more sensitive to peripheral dimming. For another example, users can be classified based on detecting eye convergence difficulties and therefore potentially more sensitive to peripheral dimming. User categories or other user-specific attributes can be used to identify and / or configure appropriate mitigation and enhancement.

[0086] Figure 7 Shows Figure 2 The XR environment view 705, and Figure 8 Shown includes Figure 7 An enhanced view 805 of view 705. Figure 7 , the user is looking at content object 710 in gaze direction 720 and has a foveal gaze zone (FGZ) 725. Portions of view 705 outside of FGZ 725 are in the user's periphery and may therefore be perceived with less sharpness than portions of view 705 within FGZ 725.

[0087] exist Figure 8 In the embodiment of the present invention, enhancements are applied to change the appearance of portions of view 705 outside of FGZ 725 (in the user's periphery). For example, the appearance of such portions may be dimmed. In another example, the appearance of such portions may have a changed contrast, for example, modified by a contrast flattening technique. Such dimming and / or contrast changes can enhance the user's experience of viewing view 805. For example, such dimming and / or contrast changes can reduce the appearance of optical artifacts (such as optical artifact 715). Contrast flattening can reduce the occurrence or severity of adjacent view portions with significantly different brightness values. Noticeable brightness values ​​can draw the user's attention from the gaze / FGZ to the periphery. By reducing the occurrence of significant contrast differences in the periphery, the user can have a better experience viewing the content, for example, being able to better focus on the interesting portions of the content without being distracted by artifacts or other peripheral anomalies.

[0088] Artifacts based on optical modules and associated mitigation and enhancement

[0089] Some specific implementations take into account the characteristics of optical modules in optical artifact mitigation. This can involve, for example, modeling and predicting the optical module based on ray tracing, 3D modeling, etc., where optical artifacts are expected given the current content, the display panel position relative to the optical module, and / or the user's eye position relative to the optical module. For example, the optical module may include a panel with a polarizer on it. Polarized light can be emitted through the display panel and pass through a half-mirror so that 50% is removed and 50% passes through the half-mirror and passes through a quarter-wave plate to an advanced polarizing film (APF). In an ideal scenario, the light is reflected back from the polarizing film, travels back through the optical stack, and is reflected from the half-mirror mirror to the user's eye. In this example of a reflective-refractive lens (e.g., a flat / folded plate lens), the light takes a folded path through the optical stack to enable a significant optical lens effect with a very small form factor. However, this type of optical module configuration can result in optical artifacts, such as ghosting. Because the optical film is imperfect, ghosting may exist, and at any time when the light is not along the expected path, it may leak from elsewhere and cause artifacts such as ghosting. Direct ghosting can be the result of light passing through the APF layer and not bouncing off the APF layer properly (e.g., going straight through). Indirect ghosting can be the result of light taking an additional circuit through the lens, e.g., bouncing back to the APF again.

[0090] Fig. 9 A lens double bounce ghosting artifact is shown on an exemplary model 900 of an optical module of an electrical device. In this example, the model 900 of the optical module includes the 3D positions of multiple layers 912, 914, 916 of the optical module, as well as the 3D positions of a display 918 and an eye / viewpoint 910. The model can also model specific content on the display 918, for example, by dividing the model of the display into multiple rectangular sub-regions with different content (e.g., bright or dark) corresponding to actual content to be displayed on the display.

[0091] Model 900 can be used to model or simulate light paths (both expected and unexpected). For example, expected light path 920 can be modeled as traveling from modeled display 918 through layers 916 and 914, bouncing through layers 914 and 916, and then popping out to reach modeled eye / viewpoint 910. Similarly, unintended light path 930 can be modeled as traveling from modeled display 918 through layers 916 and 914, bouncing through layers 914 and 916, bouncing through layers 916 and 914, bouncing through layers 914 and 916 again, and then popping out to reach modeled viewpoint 910. Fig. 9 An example of modeling a lens double bounce ghosting artifact is provided, which may have (e.g. Fig.11 Shown) appearance 1110.

[0092] Modeling can generate predictions of artifact locations based on predicting unexpected light paths (e.g., light path 930 and other light paths similar thereto. Such predictions can be combined and represented as an image and / or provide 2D coordinate information relative to a 2D pixel grid associated with a view of the XR environment, the 2D coordinate information identifying the location within the view of one or more artifacts.

[0093] Fig.10 The optical module modeling based on the optical path is shown to model various optical artifacts. In these examples, the expected optical path will ideally (e.g., in the absence of defects in the optical module components) produce an object 1000 surrounded by a completely black environment. In the first example, the modeling shows optical artifacts 1010 that are evenly distributed around the object 1000. In the second example, the modeling shows optical artifacts 1020 only on the right side of the object 1000. In the third example, the modeling shows optical artifacts 1030 only on the left side of the object 1000.

[0094] The appearance of artifacts can be modeled in a manner that takes into account characteristics of the optical module (e.g., the position and characteristics of different layers, films, etc.), the displayed content (e.g., content item color / intensity, ambient color / intensity, relative characteristics between content item and environment, etc.) and / or user information (e.g., user pupil position, user gaze direction, user FGZ, etc.).

[0095] Example of ghosting artifacts

[0096] Ghosting artifacts can have various forms. Ghosting artifacts can include static localized ghosting, for example, conditions that cause a content item to have one or more small areas of content that trigger localized ghosting. Mitigation in such conditions can include, but is not limited to, using light blooming to hide or reduce the prominence of such ghosting and / or using dynamic local dimming. Maintaining consistent dimming (e.g., across a pass-through environment) can additionally or alternatively mitigate such ghosting.

[0097] Ghosting artifacts may include static global ghosting, for example, a condition that causes any content to have an overall foggy appearance when viewed on a significantly darker environment. Mitigation in such conditions may include, but is not limited to, using light bloom and / or changing brightness conditions. In a dim environment, some content conditions may appear very bright, and some environmental transition conditions may enhance the prominence of the artifact. Such problems may be addressed, for example, by increasing dimming and / or applying smart transition strategies to avoid changes that enhance the prominence of the artifact.

[0098] Ghosting can include dynamic local ghosting, for example, where a small area of ​​bright moving content creates ghosting motion across the field of view (FOV). Similarly, ghosting can include dynamic global ghosting, where large moving areas can create moving ghosts for certain content conditions.

[0099] Example Method

[0100] Fig.12 12 is a flow chart illustrating an exemplary method for mitigating optical artifacts. In some implementations, method 1200 is performed by a device (such as a mobile device, a desktop computer, a laptop computer, an HMD, or a server device). In some implementations, the device has a screen for displaying images and / or a screen for viewing stereo images, such as a head-mounted display (HMD, such as, for example, Figure 1 Device 105). In some implementations, method 1200 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some implementations, method 1200 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., memory). Each box in method 1200 can be enabled and executed in any order.

[0101] At block 1202, method 1200 involves obtaining a content item to be rendered within a view of an extended reality (XR) environment. As an example, this may involve obtaining a movie, television show, application, etc., which may be rendered in an immersive view or within a virtual or real environment.

[0102] At block 1204, method 1200 involves identifying visual characteristics of the content item (e.g., content type, scene type, peak brightness, minimum brightness, etc.). In some examples, the visual characteristics include or are based on a peak brightness of a segment and / or a minimum brightness of a segment, for example, where those characteristics for the segment correspond to averages for frames within the segment. The visual characteristics may depend on a current viewpoint for providing a view. The content item may include media frames ordered in time, and determining the visual characteristics includes evaluating the content item on a frame-by-frame or scene-by-scene basis.

[0103] In some examples, the visual characteristics include or are based on a histogram of brightness of multiple parts (e.g., pixels) of a frame of a segment, or the type of content item. In some examples, the visual characteristics include or are based on metadata stored in the content item, or metadata stored separately from the content item and associated with the content item. The characteristic can be determined based on content and / or environmental metadata specific to one or more frames or scenes. The visual characteristics can include or be based on a histogram or spatial analysis of content brightness. The visual characteristics can identify one or more locations, e.g., x / y locations, where specific brightness features occur within the content item.

[0104] At block 1206, method 1200 involves determining display parameters of the view to mitigate the appearance of optical artifacts associated with the identified visual characteristics of the segment of the content item. Determining the display parameters may include determining the brightness for at least a portion of the content item during the segment. Determining the display parameters may include determining the brightness for at least a portion of the XR environment separated from the content item during the segment. Determining the display parameters may include determining a mapping of the content item to a dynamic range for displaying the view. Determining the display parameters may include determining a change in the contrast used in the view during the segment, determining a change in the black level used in the view during the segment, and / or determining a change in the color of at least a portion of the content during the segment. Determining the display parameters may include determining a change in the coloring of the view during the segment. Determining the display parameters may include determining a light overflow effect to be applied in the view during the segment.

[0105] Determining display parameters can involve adjusting brightness (e.g., to reduce the brightness of highlights or hide reflections by reducing brightness in reflective areas), dynamic range (e.g., fitting / mapping to a specific range), black level, contrast, color, tinting, etc., of the content item and / or the environment surrounding the content item (e.g., light spill, tinting, etc.), and / or any other type of display adjustment.

[0106] Determining display parameters may involve determining one or more frame-to-frame adjustments, which may be smoothed or timed to occur when scenes change to avoid noticeable changes.

[0107] At box 1208, method 1200 involves presenting a view with a content item within an XR environment via a display according to the determined display parameters. Such presentation can mitigate the appearance of optical artifacts that may otherwise be more noticeable, and / or provide a view that would otherwise be enhanced. In some embodiments, method 1200 is configured to address optical artifacts that are the result of light leakage in a reflective-refractive lens of an optical module of the display. In some embodiments, the optical artifacts addressed are at least partially the result of clamping lenses. In some embodiments, the optical artifact is dichoptic including different optical artifacts for each eye, and corresponding different display parameters for each eye are determined to mitigate the appearance of different optical artifacts for each eye. In some embodiments, for example, a subset of the field of view associated with the view is identified based on one or both of objectionability / visual sensitivity and lens characteristics (e.g., lens design / optical path / polarization details, etc.), and mitigation is selectively applied only to a subset of the field of view. Objectionability / visual sensitivity can correspond to a measure of how objectionable the artifact is to a particular user or to users in general. In one example, the system can analyze different portions of the display / view (e.g., top, bottom, etc.) and / or mitigate in that portion of the display / view (e.g., only in the bottom portion) based on determining that people generally find the artifact more objectionable when it appears in that portion of the display / view.

[0108] In some implementations, method 1200 further involves identifying an environmental characteristic of the XR environment. Display parameters of the view may be determined additionally or alternatively based on the environmental characteristic. As an example, the environmental characteristic may be a peak brightness, a minimum brightness, an average brightness, or a combination thereof, of a portion of the view that is separate from the content item.

[0109] In some implementations, method 1200 further involves identifying optical module characteristics based on 3D modeling of the optical module of the display.Display parameters of the view can be additionally or alternatively determined based on the optical module characteristics.

[0110] Method 1200 can present content items within an XR environment in various ways. In some implementations, the content item occupies the entire view, and determining the display parameters includes adjusting the display of the content item. In some implementations, the content item is displayed at a 2D position within the XR environment, the view depicts the portion of the XR environment surrounding the content item based on the current 3D viewpoint within the XR environment, and determining the display parameters includes adjusting the display of the content item. In some implementations, the content item is displayed at a 2D position within the XR environment, the view depicts the portion of the XR environment surrounding the content item based on the current 3D viewpoint within the XR environment, and determining the display parameters includes adjusting the display of the portion of the XR environment surrounding the content item.

[0111] In some implementations, method 1200 involves determining different display parameters for a view during different segments based on different visual characteristics during different segments of the content item. For example, during a first segment corresponding to a dark scene, standard display parameters can be used without mitigation adjustments, and then, during a second segment corresponding to a brighter scene, display parameters can be used with optical artifact mitigation adjustments.

[0112] Fig.13 1300 is a flow chart illustrating another exemplary method of mitigating optical artifacts. In some implementations, method 1300 is performed by a device (such as a mobile device, a desktop computer, a laptop computer, an HMD, or a server device). In some implementations, the device has a screen for displaying images and / or a screen for viewing stereo images, such as a head mounted display (HMD, such as, for example, Figure 1 Device 105). In some implementations, method 1300 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some implementations, method 1300 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., memory). Each box in method 1300 can be enabled and executed in any order.

[0113] At block 1302, method 1300 involves obtaining a content item to be rendered within a view of an XR environment. As an example, this may involve obtaining a movie, television show, application, etc., which may be rendered in an immersive view or within a virtual or real environment.

[0114] At block 1304, method 1300 involves predicting characteristics (e.g., saliency, brightness, size, position, etc.) of optical artifacts of the content item. The characteristics of the optical artifacts may be predicted, for example, based on modeling the light path using a 3D model of an optical module of the HMD and eye position. As an example, the characteristics may correspond to one or more frames or one or more scenes.

[0115] At block 1306, method 1300 involves determining display parameters of the view to mitigate the appearance of optical artifacts based on the predicted characteristics. Determining the display parameters may include determining a brightness for at least a portion of the content item during the segment. Determining the display parameters may include determining a brightness for at least a portion of the XR environment separate from the content item during the segment. Determining the display parameters may include determining a mapping of the content item to a dynamic range for displaying the view. Determining the display parameters may include determining a change in the contrast used in the view during the segment, determining a change in the black level used in the view during the segment, and / or determining a change in the color of at least a portion of the content during the segment. Determining the display parameters may include determining a change in the coloring of the view during the segment. Determining the display parameters may include determining a light overflow effect to be applied in the view during the segment.

[0116] Determining display parameters can involve adjusting brightness (e.g., to reduce the brightness of highlights or hide reflections by reducing brightness in reflective areas), dynamic range (e.g., fitting / mapping to a specific range), black level, contrast, color, tinting, etc., of the content item and / or the environment surrounding the content item (e.g., light spill, tinting, etc.), and / or any other type of display adjustment.

[0117] Determining display parameters may involve determining one or more frame-to-frame adjustments, which may be smoothed or timed to occur when scenes change to avoid noticeable changes.

[0118] At box 1308, method 1200 involves presenting a view with a content item within an XR environment via a display according to the determined display parameters. Such presentation can mitigate the appearance of optical artifacts that may otherwise be more noticeable, and / or provide a view that would otherwise be enhanced. In some embodiments, method 1200 is configured to address optical artifacts that are the result of light leakage in a reflective-refractive lens of an optical module of the display. In some embodiments, the optical artifacts addressed are at least partially the result of clamping lenses. In some embodiments, the optical artifact is binocular anisotropy including different optical artifacts for each eye, and corresponding different display parameters for each eye are determined to mitigate the appearance of different optical artifacts for each eye. In some embodiments, for example, a subset of the field of view associated with the view is identified based on one or both of objectionability / visual sensitivity and lens characteristics (e.g., lens design / optical path / polarization details, etc.), and mitigation is selectively applied only to a subset of the field of view. Objectionability / visual sensitivity can correspond to a measure of how objectionable the artifact is to a particular user or to users in general. In one example, the system can analyze different portions of the display / view (e.g., top, bottom, etc.) and / or mitigate in that portion of the display / view (e.g., only in the bottom portion) based on determining that people generally find the artifact more objectionable when it appears in that portion of the display / view.

[0119] In some implementations, method 1300 also involves identifying content item characteristics and / or environmental characteristics XR. Additionally or alternatively, display parameters of a view can be determined based on visual characteristics and / or environmental characteristics.

[0120] Method 1300 may present a content item within an XR environment. In some implementations, the content item occupies the entire view, and determining the display parameters includes adjusting the display of the content item. In some implementations, the content item is displayed at a 2D location within the XR environment, the view depicts a portion of the XR environment surrounding the content item based on a current 3D viewpoint within the XR environment, and determining the display parameters includes adjusting the display of the content item. In some implementations, the content item is displayed at a 2D location within the XR environment, the view depicts a portion of the XR environment surrounding the content item based on a current 3D viewpoint within the XR environment, and determining the display parameters includes adjusting the display of the portion of the XR environment surrounding the content item.

[0121] In some implementations, method 1200 involves determining different display parameters for a view during different segments based on different visual characteristics during different segments of the content item. For example, during a first segment corresponding to a dark scene, standard display parameters can be used without mitigation adjustments, and then, during a second segment corresponding to a brighter scene, display parameters can be used with optical artifact mitigation adjustments.

[0122] Fig.14 1400 is a flow chart illustrating an exemplary method of display enhancement. In some implementations, method 1400 is performed by a device (such as a mobile device, a desktop computer, a laptop computer, an HMD, or a server device). In some implementations, the device has a screen for displaying images and / or a screen for viewing stereoscopic images, such as a head-mounted display (HMD, such as, for example, Figure 1 Device 105). In some implementations, method 1400 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some implementations, method 1400 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., memory). Each box in method 1400 can be enabled and executed in any order.

[0123] At block 1402, method 1400 involves obtaining a content item to be rendered within a view of an XR environment. As an example, this may involve obtaining a movie, television show, application, etc., which may be rendered in an immersive view or within a virtual or real environment.

[0124] At block 1404, method 1400 involves identifying a gaze direction of a user of the HMD. For example, the HMD may have one or more eye tracking systems including emitters and / or sensors that capture sensor data for determining eye pose (e.g., position and / or orientation relative to other components of the HMD). In one example, such a tracking system generates a plurality of IR flashes (e.g., in one or more patterns) that are captured in an IR image or otherwise detected and used to determine eye pose. Based on the eye pose (or otherwise), a gaze direction of the user is determined. The gaze direction may be determined based on determining a gaze direction of one or both eyes.

[0125] At block 1406, method 1400 involves identifying a portion of a view associated with peripheral vision based on a gaze direction. The gaze direction may be used to identify which portion (e.g., which pixel or group of pixels a user's gaze is directed toward). Identifying the portion of the view associated with peripheral vision may be based on determining a foveal gaze zone (FGZ) based on the gaze direction and identifying one or more portions of the view outside of the FGZ.

[0126] At block 1408, method 1400 involves presenting a view having a content item within an XR environment via a display. Wherein, a portion of the view associated with peripheral vision is adjusted according to a display parameter. In some implementations, the display parameter specifies flattening contrast within the portion of the view associated with peripheral vision. Flattening contrast within the portion of the view associated with peripheral vision may include suppressing local contrast within the portion of the view associated with peripheral vision. In some implementations, additionally or alternatively, the display parameter specifies reducing brightness of the portion of the view associated with peripheral vision.

[0127] A portion of a view associated with peripheral vision may be adjusted using a non-linear curve defining an alpha multiplier to reduce pixel brightness of pixels based on distance from a foveal region of the view. A portion of a view associated with peripheral vision may be adjusted using an alpha multiplier to flatten contrast. A portion of a view associated with peripheral vision may be adjusted using both: (a) a non-linear curve defining a first multiplier to reduce pixel brightness of pixels based on distance from a foveal region of the view; and a second multiplier to flatten contrast.

[0128] Some implementations adjust the view over time to provide a smooth transition between gaze-based dimming curves that adjust over time to change the amount of dimming applied to portions of the view associated with peripheral vision. Some implementations adjust the view over time to account for detected blink or saccade user activity.

[0129] Some implementations are customized based on user-specific characteristics. For example, method 1400 can involve adjusting the amount of contrast flattening based on user-specific characteristics. In one example, the user-specific characteristic is myopia. In another example, the user-specific characteristic is eye convergence inaccuracy.

[0130] Fig.15 is a block diagram of an example device 1500. Device 1500 is shown for Figure 1Although some specific features are shown, those skilled in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and in order not to obscure more relevant aspects of the specific implementations disclosed herein. To this end, as a non-limiting example, in some embodiments, the device 1500 includes one or more processing units 1502 (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, etc.), one or more input / output (I / O) devices and sensors 1506, one or more communication interfaces 1508 (e.g., USB, FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE802.11x, IEEE 802.14x, GSM, CDMA, TDMA, GPS, IR, BLUETOOTH, ZIGBEE, SPI, I2C, and / or similar type interfaces), one or more programming (e.g., I / O) interfaces 1510, output devices (e.g., one or more displays) 1512, one or more internal-facing and / or external-facing image sensor systems 1514, memory 1520, and one or more communication buses 1504 for interconnecting these and various other components.

[0131] In some implementations, the one or more communication buses 1504 include circuits that interconnect system components and control communications between system components. In some implementations, the one or more I / O devices and sensors 1506 include at least one of the following: an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., a blood pressure monitor, a heart rate monitor, a blood oxygen sensor, a blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, one or more depth sensors (e.g., structured light, time of flight, etc.), one or more cameras (e.g., an inward-facing camera and an outward-facing camera of the HMD), one or more infrared sensors, one or more thermal map sensors, and / or the like.

[0132] In some implementations, one or more displays 1512 are configured to present a view of a physical environment, a graphical environment, an extended reality environment, etc. to the user. In some implementations, one or more displays 1512 are configured to present content (determined based on the user / object position of the determined user within the physical environment) to the user. In some implementations, one or more displays 1512 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light emitting field effect transistor (OLET), organic light emitting diode (OLED), surface conduction electron emitter display (SED), field emission display (FED), quantum dot light emitting diode (QD-LED), microelectromechanical system (MEMS) and / or similar display types. In some implementations, one or more displays 1512 correspond to diffraction, reflection, polarization, holographic, etc. waveguide displays. In one example, device 1500 includes a single display. In another example, device 1500 includes a display for each eye of the user.

[0133] In some implementations, the one or more image sensor systems 1514 are configured to obtain image data corresponding to at least a portion of the physical environment 100. For example, the one or more image sensor systems 1514 include one or more RGB cameras (e.g., with complementary metal oxide semiconductor (CMOS) image sensors or charge coupled device (CCD) image sensors), monochrome cameras, IR cameras, depth cameras, event-based cameras, etc. In various implementations, the one or more image sensor systems 1514 also include an illumination source that emits light, such as a flash. In various implementations, the one or more image sensor systems 1514 also include an on-camera image signal processor (ISP) that is configured to perform multiple processing operations on the image data.

[0134] In some implementations, sensor data may be provided by a device (e.g., Figure 1The sensor data may include a 3D point cloud and a sequence of 2D images corresponding to views of the room captured during the scan of the room. In some implementations, the sensor data includes image data (e.g., from an RGB camera), depth data (e.g., a depth image from a depth camera), ambient light sensor data (e.g., from an ambient light sensor), and / or motion data from one or more motion sensors (e.g., an accelerometer, a gyroscope, an IMU, etc.). In some implementations, the sensor data includes visual inertial odometry (VIO) data determined based on the image data. The 3D point cloud may provide semantic information about one or more elements of the room. The 3D point cloud may provide information about the positioning and appearance of a surface portion within the physical environment. In some implementations, the 3D point cloud is obtained over time (e.g., during a scan of the room), and the 3D point cloud may be updated, and an updated version of the 3D point cloud is obtained over time. For example, the 3D representation may be obtained (and analyzed / processed) as it updates / adjusts over time (eg, as a user scans a room).

[0135] In some implementations, the sensor data may be positioning information, and some implementations include VIO to determine equivalent odometer information using sequence camera images (e.g., light intensity image data) and motion data (e.g., obtained from an IMU / motion sensor) to estimate the distance traveled. Alternatively, some implementations of the present disclosure may include a simultaneous localization and simultaneous mapping (SLAM) system (e.g., a positioning sensor). The SLAM system may include a multi-dimensional (e.g., 3D) laser scanning and range measurement system that is independent of GPS and provides real-time simultaneous positioning and mapping. The SLAM system may generate and manage data of a very accurate point cloud generated by reflections of laser scans from objects in the environment. Over time, the movement of any point in the point cloud is accurately tracked so that the SLAM system can use points in the point cloud as reference points for location, maintaining an accurate understanding of its position and orientation as it travels through the environment.

[0136] In some implementations, device 1500 includes an eye tracking system for detecting eye position and eye movement (e.g., eye gaze detection). For example, the eye tracking system can include one or more infrared (IR) light emitting diodes (LEDs), an eye tracking camera (e.g., a near infrared (NIR) camera), and an illumination source (e.g., a NIR light source) that emits light (e.g., NIR light) toward the user's eyes. In addition, the illumination source of device 1500 can emit NIR light to illuminate the user's eyes, and the NIR camera can capture images of the user's eyes. In some implementations, images captured by the eye tracking system can be analyzed to detect the position and movement of the user's eyes, or to detect other information about the eyes such as pupil dilation or pupil diameter. In addition, the gaze point estimated from the eye tracking image can enable gaze-based interaction with content shown on the near-eye display of device 1500.

[0137] Memory 1520 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid-state memory devices. In some specific implementations, memory 1520 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices or other non-volatile solid-state storage devices. Memory 1520 optionally includes one or more storage devices remotely located from one or more processing units 1502. Memory 1520 includes non-transitory computer-readable storage media.

[0138] In some implementations, the memory 1520 or a non-transitory computer-readable storage medium of the memory 1520 stores an optional operating system 1530 and one or more instruction sets 1540. The operating system 1530 includes processes for handling various basic system services and for performing hardware-related tasks. In some implementations, the instruction set 1540 includes executable software defined by binary information stored in the form of electrical charge. In some implementations, the instruction set 1540 is software that can be executed by one or more processing units 1502 to implement one or more of the techniques described herein.

[0139] Instruction set 1540 includes video content presentation instruction set 1542. Instruction set 1540 may be embodied as a single software executable file or multiple software executable files. Artifact mitigation and display enhancement instruction set 1542 is configured with instructions executable by a processor to analyze content and environmental characteristics, perform optical modeling, and / or otherwise determine mitigation and enhancement based on various considerations, as described herein.

[0140] Although the instruction set 1540 is shown as residing on a single device, it should be understood that in other implementations, any combination of elements may be located in separate computing devices. Fig.15Rather, it serves as a functional description of various features present in a particular implementation that are different from the structural schematics of the implementations described herein. As one of ordinary skill in the art will recognize, items shown separately may be combined, and some items may be separated. The actual number of instruction sets and how features are distributed among them will vary depending on the implementation and may depend in part on the specific combination of hardware, software, and / or firmware selected for a particular implementation.

[0141] Those of ordinary skill in the art will understandably describe well-known systems, methods, components, devices, and circuits so as not to obscure more relevant aspects of the example implementations described herein. In addition, other effective aspects and / or variations do not include all of the details in the specific details described herein. Therefore, several details are described in order to provide a thorough understanding of the example aspects shown in the accompanying drawings. In addition, the accompanying drawings only illustrate some example embodiments of the present disclosure and therefore should not be considered as limiting.

[0142] Although this specification includes many specific implementation details, these specific implementation details should not be understood as limitations on the scope of any invention or possible content to be protected, but should be understood as descriptions of features specific to a particular embodiment of a particular invention. Certain features described in the context of different embodiments in this specification may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in the form of any suitable sub-combination. In addition, although certain features may be described above as acting in certain combinations and even initially claimed in this way, one or more features of the claimed combination may be removed from the combination in some cases, and the claimed combination may involve a sub-combination or a variation of the sub-combination.

[0143] Similarly, although the operations are shown in a particular order in the accompanying drawings, this should not be understood as requiring such operations to be performed in a sequential order or in the particular order shown, or requiring the execution of all the operations shown to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the division of each system component in the above-mentioned embodiment should not be understood as requiring such division in all embodiments, and it should be understood that the program components and the system can be generally integrated together in a single software product or encapsulated in multiple software products.

[0144] Thus, specific embodiments of the subject matter have been described. Other embodiments are also within the scope of the following claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some specific implementations, multitasking and parallel processing may be advantageous.

[0145] The embodiments of the subject matter and operations described in this specification may be implemented in digital electronic circuits or in computer software, firmware or hardware (including the structures disclosed in this specification and their structural equivalents) or in a combination of one or more of them. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, for example, one or more modules of computer program instructions encoded on a computer storage medium, for execution by a data processing device or for controlling the operation of a data processing device. Alternatively or alternatively, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical signal, optical signal or electromagnetic signal, which is generated to encode information so as to be sent to a suitable receiver device for execution by a data processing device. The computer storage medium may be or be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. In addition, although the computer storage medium is not a propagation signal, the computer storage medium may be the source or destination of the computer program instructions encoded in the artificially generated propagation signal. The computer storage medium may also be or be included in one or more separate physical components or media (e.g., multiple CDs, disks or other storage devices).

[0146] The term "data processing device" covers all kinds of devices, equipment and machines for processing data, including, for example, programmable processors, computers, systems on chip, or multiple or combinations of the foregoing. The device may include a dedicated logic circuit (e.g., FPGA (field programmable gate array) or ASIC (application-specific integrated circuit)). In addition to hardware, the device may also include code for creating an execution environment for the computer program under consideration, such as code constituting a processor firmware, a protocol stack, a database management system, an operating system, a cross-platform operating environment, a virtual machine, or a combination of one or more of them. The device and the execution environment can implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures. Unless otherwise specifically stated, it should be understood that throughout the specification, discussions using terms such as "processing", "calculating", "calculating", "determining", and "identifying" refer to the actions or processes of a computing device, such as one or more computers or similar electronic computing devices, which manipulate or convert data represented as physical electronic quantities or magnetic quantities within a memory, register, or other information storage device, transmission device, or display device of a computing platform.

[0147] The one or more systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device may include any suitable arrangement of components that provide results conditioned on one or more inputs. Suitable computing devices include multi-purpose microprocessor-based computer systems that access stored software that programs or configures the computing system from a general-purpose computing device to a dedicated computing device that implements one or more specific implementations of the subject matter of the present invention. Any suitable programming, scripting, or other type of language or combination of languages ​​may be used to implement the teachings contained herein in software for programming or configuring a computing device.

[0148] The specific implementation of the method disclosed herein can be performed in the operation of such computing devices. The order of the boxes presented in the above examples can be changed, for example, the boxes can be reordered, combined and / or divided into sub-blocks. Some boxes or processes can be executed in parallel. The operations described in this specification can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0149] The use of "suitable for" or "configured to" herein is meant to be open and inclusive language that does not exclude devices that are suitable for or configured to perform additional tasks or steps. In addition, the use of "based on" is meant to be open and inclusive, as a process, step, calculation, or other action "based on" one or more of the stated conditions or values ​​may in practice be based on additional conditions or values ​​beyond those stated. The headings, lists, and numbers included herein are for ease of explanation only and are not intended to be limiting.

[0150] It will also be understood that, although the terms "first", "second", etc. may be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description, as long as all occurrences of the "first node" are consistently renamed and all occurrences of the "second node" are consistently renamed. Both the first node and the second node are nodes, but they are not the same node.

[0151] The terms used herein are only for the purpose of describing a specific implementation and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms of "one", "an" and "the" are intended to also cover the plural form, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and covers any and all possible combinations of one or more items in the associated listed items. It will also be understood that when used in this specification, the terms "include" and / or "comprise" specify the presence of specified features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groups.

[0152] As used herein, the term “if” may be interpreted to mean “when the antecedent is true” or “when the antecedent is true” or “in response to determining” or “upon determining” or “in response to detecting” that the antecedent is true, depending on the context. Similarly, the phrase “if it is determined that [the antecedent is true]” or “if [the antecedent is true]” or “when [the antecedent is true]” is interpreted to mean “upon determining that the antecedent is true” or “in response to determining” or “upon determining” that the antecedent is true or “when detecting that the antecedent is true” or “in response to detecting” that the antecedent is true, depending on the context.

Claims

1. A method, comprising: At a head mounted device (HMD) having a processor and a display: Obtaining a content item to be rendered within a view of an extended reality (XR) environment; identifying a visual characteristic of the content item; determining display parameters of the view to mitigate the appearance of optical artifacts associated with the identified visual characteristics; as well as The view with the content item is presented within the XR environment via the display according to the determined display parameters.

2. The method according to claim 1, wherein: The visual characteristics are: The peak brightness of the segment; or The minimum brightness of the segment.

3. The method according to claim 1, wherein: The visual characteristics are: A histogram corresponding to brightness of portions of the frame of the segment, or the type of the content item.

4. The method according to claim 1, wherein: The visual characteristics are determined based on metadata stored in the content item, or metadata stored separately from the content item and associated with the content item.

5. The method according to claim 1, wherein: The visual characteristics depend on the current viewpoint used to provide the view.

6. The method according to claim 1, wherein: The content item includes time-ordered media frames, and determining the visual characteristic includes evaluating the content item on a frame-by-frame or scene-by-scene basis.

7. The method according to claim 1, wherein: Determining the display parameter includes: A brightness for at least a portion of the content item during the segment is determined.

8. The method according to claim 1, wherein: Determining the display parameter includes: A brightness is determined for at least a portion of the XR environment separate from the content item during the segment.

9. The method according to claim 1, wherein: Determining the display parameter includes: A mapping of the content items to a dynamic range for displaying the view is determined.

10. The method according to claim 1, wherein: Determining the display parameter includes: determining a change to a contrast used in the view during the segment; determining a change to a black level used in the view during the segment; determining a change to a color of at least a portion of the content during the segment; or A change to the shading of the view during the segment is determined.

11. The method according to claim 1, wherein: Determining the display parameter includes: A bloom effect is determined to be applied in the view during the segment.

12. The method according to claim 1, further comprising: identifying environmental characteristics of the XR environment, The display parameter of the view is determined based on the environmental characteristic, wherein the environmental characteristic is a peak brightness, a minimum brightness, or an average brightness of a portion of the view separate from the content item.

13. The method according to claim 1, wherein: Optical artifacts are the result of light leakage in the catadioptric lenses of the display's optical module.

14. The method according to claim 13, wherein: The optical artifacts are at least partly a result of clamping the lens.

15. The method according to claim 1, wherein: The optical artifact is binocular anisotropy including different optical artifacts for each eye, and corresponding different display parameters for each eye are determined to mitigate the appearance of the different optical artifacts for each eye.

16. The method according to claim 1, further comprising: selecting a subset of fields of view associated with the views, the subset selected based on objectionability and lens characteristics; as well as Only the subset of the field of view is mitigated.

17. The method according to claim 1, further comprising: Identify the gaze direction; as well as identifying a portion of the view associated with peripheral vision based on the gaze direction; and Wherein the display parameters of the view are determined based on the portion of the view associated with peripheral vision, wherein the display parameters reduce brightness of the portion of the view associated with peripheral vision.

18. The method according to claim 1, further comprising: Identify the gaze direction; as well as identifying a portion of the view associated with peripheral vision based on the gaze direction; and Wherein the display parameters of the view are determined based on the portion of the view associated with peripheral vision, wherein the display parameters perform contrast flattening on the portion of the view associated with peripheral vision.

19. A system, comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium includes program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: Obtaining a content item to be rendered within a view of an extended reality (XR) environment; identifying a visual characteristic of the content item; determining display parameters of the view to mitigate the appearance of optical artifacts associated with the identified visual characteristics; and The view with the content item is presented within the XR environment via the display according to the determined display parameters.

20. A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising: Obtaining a content item to be rendered within a view of an extended reality (XR) environment; identifying a visual characteristic of the content item; determining display parameters of the view to mitigate the appearance of optical artifacts associated with the identified visual characteristics; as well as The view with the content item is presented within the XR environment via the display according to the determined display parameters.

Images