Cache architecture for image warp processing systems and methods

Abstract

An apparatus may include a display to display an image frame based on warped image data, and image processing circuitry to generate the warped image data by warping input image data to account for one or more distortions associated with displaying the image. The image processing circuitry may include a two-level cache architecture having a first cache and a second cache, and warp the input image data by generating mapping data indicative of a warp between an input image space and an output image space, and extracting the input image data to populate the first cache. The warping may also include populating the second cache with a packet of pixel values from the first cache, the pixel values selected according to a sliding window traversing the first cache based on the mapping data; and interpolating between the pixel values of the group to generate pixel values of the warped image data.

Description

Background Art

[0001] The present disclosure generally relates to the processing of displayed images, and more particularly, to image warping and cache architectures for image warping.

[0002] This section is intended to introduce the reader to various aspects of the art that may be related to various aspects of the present disclosure, which are described and / or claimed hereinafter. This discussion is believed to be helpful to provide the reader with background information to better understand the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this context and not as an admission of prior art.

[0003] Electronic devices typically use one or more electronic displays to present visual information, such as text, still images, and / or video, by displaying one or more images. For example, such electronic devices may include computers, mobile phones, portable media devices, tablets, televisions, virtual reality headsets, and vehicle dashboards. To display an image, an electronic display may control the light emission of its display pixels based at least in part on corresponding image data. Moreover, the image data may be processed to account for one or more physical or digital effects associated with displaying the image data. For example, the image data may be compensated for pixel aging (e.g., burn-in compensation), crosstalk between electrodes within the electronic device, transitions from previously displayed image data (e.g., pixel drive compensation), warping, contrast control, and / or other factors that may cause viewer-perceivable distortion or artifacts.

[0004] In particular, it may be desirable to change the amount or distribution of pixel values to account for different display scenarios. For example, the image data may be warped to account for the surrounding environment, display characteristics, the viewer's point of view (POV), and / or other factors that may cause image distortion perceivable by the viewer. Thus, prior to display, the image data may be processed to warp the image using a desired change in the amount or distribution of pixel values such that the perceived image is undistorted. However, it may be difficult to perform such warping efficiently and / or within bandwidth / timing constraints (e.g., for real-time operation). Summary of the Invention

[0005] A summary of certain embodiments disclosed herein is set forth below. It should be understood that presenting these aspects is merely to provide a concise summary of these particular embodiments to the reader and these aspects are not intended to limit the scope of the present disclosure. Indeed, the present disclosure may cover a variety of aspects that may not be set forth below.

[0006] The image processing circuit can warp one or more sets of input image data to account for input distortion (e.g., camera lens distortion), output distortion (e.g., lens effects associated with the shape of a display panel and / or its cover glass), processing distortion (e.g., POV change, shift, scale, foveation-related resolution change, etc.), and / or to implement a common image space for blending. For example, the image processing circuit (e.g., the warp block) can utilize configuration data associated with the desired warp effect to generate a mapping from the input image data to the warped image data. The configuration data can include or define mappings, algorithms, and / or parameters that indicate the warp to be done on the sets of input image data. Additionally, the configuration information can include static aspects and / or dynamic aspects to account for warp characteristics that do not change (e.g., display geometry) and things that are done (e.g., POV change, shift, scale, foveation-related resolution change, etc.). In other words, which input pixels map to which output pixel positions on the display panel (e.g., as achieved by warping the input image data) can change based on the parameters, algorithms, mappings, etc. captured in the configuration data.

[0007] Moreover, the image processing circuit can extract the input image data (e.g., from memory) and generate output pixel values based on the input image data using the mapping. Additionally, in some embodiments, the output pixel values can be interpolated from a set of multiple input pixel values selected based on the mapping. However, it can prove difficult to perform such warping while maintaining synchronization and / or within the timing constraints of the system, especially for real-time operations such as warping a camera feed. Thus, the image processing circuit can utilize a two-level cache architecture to efficiently extract and interpolate the input image data and / or within the bandwidth / timing limitations to produce the warped image data.

[0008] In some embodiments, the first cache is filled by the extractor with input image data. Also, the extractor may utilize a mapping (e.g., based on configuration data) to extract the input image data in an order associated with the mapping. For example, instead of extracting the input image data in a raster scan order, the input image data may be extracted in a tiled section along a virtual curve of the raster scan of the warped image data that indicates the mapping to the source image space. In other words, the extractor may request the input image data from the first cache in the expected usage order. Additionally, the second cache may be filled from the first cache according to a sliding window that follows a virtual curve in the source image space. Also, the sliding window may include pixel values around the pixel positions (in the source image space) that map to the pixel positions of the warped image data (in the output image space) to accommodate interpolation. Additionally, since the second cache includes multiple input pixel values, in some embodiments, multiple warped pixel values may be determined simultaneously (e.g., processed together from the second cache), which may improve efficiency and / or reduce processing time. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Aspects of the present disclosure may be better understood when the following detailed description is read in conjunction with the accompanying drawings, in which:

[0010] Figure 1 is a schematic diagram of an electronic device including an electronic display in accordance with an embodiment;

[0011] Figure 2 is an example of an electronic device in the form of a handheld device in accordance with an embodiment Figure 1 of;

[0012] Figure 3 is an example of an electronic device in the form of a tablet device in accordance with an embodiment Figure 1 of;

[0013] Figure 4 is an example of an electronic device in the form of a computer in accordance with an embodiment Figure 1 of;

[0014] Figure 5 is an example of an electronic device in the form of a watch in accordance with an embodiment Figure 1 of;

[0015] Figure 6 is an example of an electronic device in the form of a laptop in accordance with an embodiment Figure 1 of;

[0016] Figure 7 is a schematic diagram of an image processing circuit including a warping block in accordance with an embodiment Figure 1 of;

[0017] Figure 8 is of a warping block according to an embodiment Figure 7 Schematic diagram of the warping block;

[0018] Figure 9 is of a warping sub - block of a warping block according to an embodiment Figure 8 Schematic diagram of the warping sub - block of the warping block;

[0019] Figure 10 is a conceptual diagram of how a first cache according to an embodiment extracts input image data;

[0020] Figure 11 is a conceptual diagram of a sliding window that traverses a first cache to fill a second cache according to an embodiment;

[0021] Figure 12 is of a second cache filled with input image data from a first cache according to an embodiment and Figure 10 and Figure 11 Schematic diagram of the second cache filled with input image data from the first cache;

[0022] Figure 13 is a schematic diagram of a pruning correction associated with a second cache according to an embodiment; and

[0023] Figure 14 is a flowchart of an example process for warping input image data to generate warped image data according to an embodiment. DETAILED DESCRIPTION

[0024] One or more specific embodiments of the present disclosure will be described below. These described embodiments are merely examples of the presently disclosed technology. Additionally, an attempt is made to provide a brief description of these embodiments, and not all features of an actual specific implementation may be described in this specification. It should be understood that in the development of any such actual specific implementation, as in any engineering or design project, specific decisions specific to many specific implementations must be made to achieve the developer's specific goals, such as meeting system - related and business - related constraints that can vary from one specific implementation to another. Moreover, it should be appreciated that such development efforts can be complex and time - consuming, but for those of ordinary skill in the art who benefit from the present disclosure, such development efforts can still be routine tasks in design, fabrication, and manufacturing.

[0025] When introducing elements of the various embodiments of the present disclosure, the articles "a", "an", and "the" are intended to mean that there is one or more of the elements. The terms "comprising", "including", and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. Additionally, it should be understood that a reference to "one embodiment" or "an embodiment" of the present disclosure is not intended to be construed as excluding the existence of additional embodiments that also incorporate the recited features. Further, the phrase A "based on" B is intended to mean that A is at least partially based on B. Also, the term "or" is intended to be inclusive (e.g., logical or) and not exclusive (e.g., logical xor). In other words, the phrase A "or" B is intended to mean A, B, or both A and B.

[0026] Electronic devices typically use electronic displays to present visual information. Such electronic devices may include computers, mobile phones, portable media devices, tablets, televisions, virtual reality headsets, and vehicle dashboards, among others. To display an image, an electronic display controls the brightness (and thus the color) of its display pixels based on corresponding image data received at a particular resolution. For example, an image data source may provide image data as a pixel data stream, where the data for each pixel indicates the target brightness (e.g., luminance and / or color) of one or more display pixels located at the corresponding pixel position. In some embodiments, the image data may indicate the brightness of each color component, for example, via red component image data, blue component image data, and green component image data (collectively referred to as RGB image data (e.g., RGB, sRGB)). Additionally or alternatively, the image data may be indicated by a luminance channel and one or more chrominance channels (e.g., YCbCr, YUV, etc.), grayscale (e.g., grayscale levels), or other color bases. It should be understood that, as disclosed herein, the luminance channel may encompass linear luminance values, non-linear luminance values, and / or gamma-corrected luminance values.

[0027] Additionally, image data can be processed to account for one or more physical or digital effects associated with displaying the image data. For example, the image data can be compensated for pixel aging (e.g., burn-in compensation), crosstalk between electrodes within the electronic device, conversion from a previously displayed image data (e.g., pixel drive compensation), warping, contrast control, and / or other factors that may cause perceptible distortion or artifacts to the viewer. For example, in some scenarios, due to environmental effects, the performance of the display, the viewer's point of view (POV) perspective, image processing changes such as shifting and scaling, and / or other distortion factors, the image to be displayed may appear distorted (if unchanged) when perceived by the viewer. For example, the display can include a screen with curved edges and / or a lens effect, which may distort the image if displayed without correction. Additionally, the viewer's POV relative to the display can change the way the viewer perceives the image. For example, the viewer's gaze can be determined based on the position of the viewer relative to the display and / or eye tracking. Additionally, the display can be a foveated display such that different parts of the screen are displayed at different resolutions (e.g., depending on the viewer's gaze / focus on the display). Additionally or alternatively, image data can be received from a distortion source such as a camera, and the image data can be warped to account for the lens effect associated with the captured image. Thus, it may be necessary to change the amount (e.g., resolution) or distribution (e.g., shape, relative size, perspective, etc.) of the pixel values to account for different display scenarios and / or input image characteristics. Therefore, prior to display, the image data can be processed to warp the image using a desired change to the amount or distribution of the pixel values such that the perceived image is not distorted.

[0028] Moreover, in some embodiments, an image to be displayed can be generated based on multiple sets of image data that are blended together from one or more sources. Image blending (e.g., for virtual reality, mixed reality, and / or augmented reality) can be utilized to incorporate image data from multiple sources into a single image frame. For example, a generated object can be incorporated into an image capture of the real-life surrounding environment (e.g., via a camera), a portion of the captured image can be incorporated into a virtual surrounding environment, and / or a combination of both. Thus, image data from multiple sources can be blended together to form a single output image. In some embodiments, each set of image data can be warped to a common image space prior to blending.

[0029] As discussed herein, an image processing circuit can warp one or more sets of input image data to account for input distortion (e.g., camera lens distortion), output distortion (e.g., lens effects associated with the shape of a display panel and / or its cover glass), processing distortion (e.g., POV change, shift, scale, etc.), and / or to implement a common image space for blending. Further, the image processing circuit can include separate warping hardware (e.g., for parallel processing), and / or perform separate warping operations on different sets of input image data using the same hardware.

[0030] In some embodiments, the image processing circuit (e.g., warping block) can utilize configuration data associated with a desired warping effect to generate a mapping from the input image data to the warped image data. The configuration data can include mappings, algorithms, and / or parameters that indicate the warping to be done for a set of input image data. Additionally, the configuration data can include static aspects and / or dynamic aspects. For example, the configuration data can include a static mapping between a generated graphics image space and a display image space that accounts for unchanging distortion associated with an electronic display. Further, the configuration data can include a static mapping between a camera image space and a display image space that accounts for unchanging camera lens distortion and unchanging distortion associated with an electronic display. As should be appreciated, captured image data from a camera is given as an example set of input image data, and such data can be processed or partially processed or not processed prior to the warping block of the image processing circuit. Further, a camera can include multiple or variable lenses related to the dynamic portion of the configuration data. Additionally, dynamic aspects can be included in the configuration data to provide different mappings in different scenarios. For example, in a foveated display, the output resolution at different parts of the display panel can change based on the focus of the user's gaze (such as determined by eye tracking). In other words, which input pixels map to which output pixel locations on the display panel (e.g., as achieved by warping the input image data) can change based on additional input parameters captured in the configuration data.

[0031] Based on the configuration data, a mapping can be determined that relates the output pixel values of the warped image data to the pixel values of the input image data. As should be appreciated, the output image space can be associated with the physical pixel locations of a display panel (e.g., the display image space) or any desired image space. Also, the image processing circuit can extract the input image data (e.g., from memory) and generate the output pixel values based on the input image data using the mapping. Additionally, in some embodiments, the output pixel values can be interpolated from a set of multiple input pixel values selected based on the mapping. However, it may prove difficult to perform such warping while maintaining synchronization and / or within the timing constraints of the system, especially for real-time operations such as warping a camera feed. Thus, the image processing circuit can utilize a two-level cache architecture to efficiently extract and interpolate the input image data, either within the bandwidth / timing constraints, to produce the warped image data.

[0032] In some embodiments, the first cache is filled with the input image data by the extractor. Also, the extractor can use the mapping (e.g., based on the configuration data) to extract the input image data in an order associated with the mapping. For example, instead of extracting the input image data in raster scan order, the input image data can be extracted in tile-segment warps along a virtual curve that indicates the raster scan of the warped image data mapped to the source image space. In other words, the extractor can request the input image data from the first cache in the order of expected use. Additionally, the second cache can be filled from the first cache according to a sliding window that follows the virtual curve in the source image space. Also, the sliding window can include the pixel values around the pixel locations (in the source image space) that map to the pixel locations of the warped image data (in the output image space) to accommodate interpolation. Additionally, since the second cache includes multiple input pixel values, in some embodiments, multiple warped pixel values can be determined simultaneously (e.g., processed together from the second cache), which can improve efficiency and / or reduce processing time.

[0033] In view of the foregoing, Figure 1 is an example electronic device 10 having an electronic display 12 with independently controllable color component illuminators (e.g., projectors, backlights, etc.). As described in more detail below, the electronic device 10 can be any suitable electronic device, such as a computer, mobile phone, portable media device, tablet, television, virtual reality headset, wearable device (such as a watch), vehicle dashboard, etc. Thus, it should be noted that Figure 1 is only an example of a particular implementation and is intended to illustrate the types of components that may be present in the electronic device 10.

[0034] The electronic device 10 may include one or more electronic displays 12, an input device 14, an input / output (I / O) port 16, a processor core complex 18 having one or more processors or processor cores, a local memory 20, a main memory storage device 22, a network interface 24, a power supply 26, and an image processing circuit 28. Figure 1 The various components described in Figure 1 may include hardware elements (e.g., circuits), software elements (e.g., tangible non-transitory computer-readable media storing instructions), or a combination of hardware and software elements. It should be understood that the various components may be combined into fewer components or separated into additional components. For example, the local memory 20 and the main memory storage device 22 may be included in a single component. Also, the image processing circuit 28 (e.g., a graphics processing unit, a display image processing pipeline, etc.) may be included in the processor core complex 18 or implemented separately.

[0035] The processor core complex 18 is operatively coupled to the local memory 20 and the main memory storage device 22. Thus, the processor core complex 18 may execute instructions stored in the local memory 20 or the main memory storage device 22 to perform operations such as generating and / or transmitting image data for display on the electronic display 12. As such, the processor core complex 18 may include one or more general-purpose microprocessors, one or more application-specific integrated circuits (ASICs), one or more field-programmable logic arrays (FPGAs), or any combination thereof.

[0036] In addition to program instructions, the local memory 20 or the main memory storage device 22 may store data to be processed by the processor core complex 18. Thus, the local memory 20 and / or the main memory storage device 22 may include one or more tangible non-transitory computer-readable media. For example, the local memory 20 may include random access memory (RAM), and the main memory storage device 22 may include read-only memory (ROM), rewritable non-volatile memory such as flash memory, a hard disk drive, an optical disc, etc.

[0037] The network interface 24 may transfer data with another electronic device or network. For example, the network interface 24 (e.g., a radio frequency system) may enable the electronic device 10 to be communicatively coupled to a personal area network (PAN) (such as a Bluetooth network), a local area network (LAN) (such as an 802.11x Wi-Fi network), and / or a wide area network (WAN) (such as a 4G, Long-Term Evolution (LTE), or 5G cellular network).

[0038] The power supply 26 may provide power to operate the processor core complex 18 and / or other components in the electronic device 10. Thus, the power source 26 may include any suitable energy source, such as a rechargeable lithium polymer (Li-poly) battery and / or an alternating current (AC) power converter.

[0039] The I / O port 16 enables the electronic device 10 to interact with various other electronic devices. The input device 14 enables a user to interact with the electronic device 10. For example, the input device 14 can include buttons, a keyboard, a mouse, a touchpad, etc. Additionally or alternatively, the electronic display 12 can include a touch-sensing component that enables user input to the electronic device 10 by detecting the occurrence and / or location of an object touching its screen (e.g., the surface of the electronic display 12).

[0040] For example, the electronic display 12 can display a graphical user interface (GUI) of an (e.g., operating system or computer program), an application interface, text, a still image, and / or video content. The electronic display 12 can include a display panel having one or more display pixels to facilitate image display. Additionally, each display pixel can represent a subpixel that controls the luminance of a color component (e.g., red, green, or blue). As used herein, a display pixel can refer to a collection of subpixels (e.g., red, green, and blue subpixels) or can refer to a single subpixel.

[0041] As described above, the electronic display 12 can display an image by controlling the luminance output (e.g., light emission) of the subpixels based on corresponding image data. In some embodiments, the pixel or image data can be generated by an image source such as a processor core complex 18, a graphics processing unit (GPU), or an image sensor (e.g., a camera). Additionally, in some embodiments, image data can be received from another electronic device 10, for example, via the network interface 24 and / or the I / O port 16. Moreover, in some embodiments, the electronic device 10 can include multiple electronic displays 12 and / or can perform image processing (e.g., via the image processing circuit 28) on one or more external electronic displays 12 connected, for example, via the network interface 24 and / or the I / O port 16.

[0042] The electronic device 10 can be any suitable electronic device. For ease of illustration, an example of a suitable electronic device 10, particularly a handheld device 10A, is shown in Figure 2 . In some embodiments, the handheld device 10A can be a portable phone, a media player, a personal data manager, a handheld game platform, etc. For illustrative purposes, the handheld device 10A can be a smart phone, for example, any model available from Apple Inc.

[0043] The handheld device 10A may include a housing 30 (e.g., a casing) to, for example, protect internal components from physical damage and shield the internal components from electromagnetic interference. The housing 30 may at least partially surround the electronic display 12. In the depicted embodiment, the electronic display 12 displays a graphical user interface (GUI) 32 having an array of icons 34. For example, when an icon 34 is selected via the input device 14 or a touch-sensing component of the electronic display 12, an application may be launched.

[0044] The input device 14 may be accessible through an opening in the housing 30. Moreover, the input device 14 may enable a user to interact with the handheld device 10A. For example, the input device 14 may enable a user to activate or deactivate the handheld device 10A, navigate the user interface to the home screen, navigate the user interface to a user-configurable application screen, activate a voice recognition feature, provide volume control, and / or switch between a vibrate mode and a ring mode. Also, an I / O port 16 may also be open through the housing 30. Additionally, the electronic device may include one or more cameras 36 to capture pictures or videos. In some embodiments, the cameras 36 may be used in conjunction with virtual reality or augmented reality visualizations on the electronic display 12.

[0045] Figure 3 Another example of a suitable electronic device 10 is shown, specifically a tablet device 10B. The tablet device 10B may be any model available from Apple Inc. Figure 4 Another example of a suitable electronic device 10 is shown, specifically a computer 10C. For illustrative purposes, the computer 10C may be any or model available from Apple Inc. Figure 5 Another example of a suitable electronic device 10 is shown, specifically a watch 10D. For illustrative purposes, the watch 10D may be any APPLE model available from Apple Inc. As depicted, the tablet device 10B, the computer 10C, and the watch 10D each further include an electronic display 12, an input device 14, an I / O port 16, and a housing 30. The electronic display 12 may display the GUI 32. Here, the GUI 32 displays a visualization of a clock. When the visualization is selected via the input device 14 or a touch-sensing component of the electronic display 12, an application may be launched, such as transforming the GUI 32 to present the Figure 2 and Figure 3 icons 34 discussed in

[0046] See Figure 6 , the computer 10E may represent Figure 1Another embodiment of the electronic device 10. The computer 10E can be any suitable computer, for example, a desktop computer, a server, or a laptop computer, but can also be a standalone media player or a video game console. For example, the computer 10E can be an or other similar devices. It should be noted that the computer 10E can also represent a personal computer (PC) of another manufacturer. A similar housing 30 can be provided to protect and enclose the internal components of the computer 10E, such as the electronic display 12. In some embodiments, the user of the computer 10E can use various peripheral input devices 14 (such as a keyboard 14A or a mouse 14B that can be connected to the computer 10E) to interact with the computer 10E.

[0047] As described above, the electronic display 12 can display an image based on the image data. Before the corresponding image is displayed on the electronic display 12, the image data can be processed via the image processing circuit 28. The image processing circuit 28 can process the image data for display on one or more electronic displays 12. For example, the image processing circuit 28 can include a display pipeline, a memory-to-memory scaler and rotator (MSR) circuit, a warping compensation circuit, or additional hardware or software means for processing the image data. The image data can be processed by the image processing circuit 28 to reduce or eliminate image artifacts, compensate for one or more different software or hardware-related effects, and / or format the image data for display on one or more electronic displays 12. It should be understood that the present technology can be implemented in standalone circuits, software, and / or firmware, and can be considered part of, separate from, and / or parallel to the display pipeline or the MSR circuit.

[0048] For illustrative purposes, Figure 7 a portion of the electronic device 10 including the image processing circuit 28 is shown. The image processing circuit 28 can be implemented in the electronic device 10, the electronic display 12, or a combination thereof. Thus, the image processing circuit 28 can be included in the processor core complex 18, the timing controller (TCON) in the electronic display 12, or any combination thereof. It should be understood that although the image processing is discussed herein as being performed via multiple image data processing blocks, the embodiments can include hardware or software components to perform the techniques discussed herein.

[0049] The electronic device 10 may also include an image data source 38, a display panel 40, and / or a controller 42 that communicate with the image processing circuit 28. In some embodiments, the display panel 40 of the electronic display 12 may be a reflective technology display, a liquid crystal display (LCD), or any other suitable type of display panel 40. In some embodiments, the controller 42 may control the operation of the image processing circuit 28, the image data source 38, and / or the display panel 40. To facilitate control of the operation, the controller 42 may include a controller processor 44 and / or a controller memory 46. In some embodiments, the controller processor 44 may be included in the processor core complex 18, the image processing circuit 28, a timing controller in the electronic display 12, a separate processing module, or any combination thereof, and execute instructions stored in the controller memory 46. Additionally, in some embodiments, the controller memory 46 may be included in the local memory 20, the main memory storage device 22, a separate tangible non-transitory computer-readable medium, or any combination thereof.

[0050] Thus, the image processing circuit 28 may receive source image data 48 corresponding to a desired image to be displayed on the electronic display 12 from the image data source 38. The source image data 48 may indicate target characteristics (e.g., pixel data) corresponding to the desired image using any suitable source format (such as, RGB format, αRGB format, YCbCr format, etc.). Moreover, the source image data may be fixed or floating-point and may have any appropriate bit depth. Additionally, the source image data 48 may reside in a linear color space, a gamma-corrected color space, or any other suitable color space. As used herein, a pixel or pixel data may refer to a grouping of sub-pixels (e.g., individual color component pixels such as red, green, and blue) or the sub-pixels themselves.

[0051] As described above, the image processing circuit 28 is operable to process source image data 48 received from an image data source 38. The image data source 38 may include captured images (e.g., from one or more cameras 36), images stored in a memory, graphics generated by the processor core complex 18, or a combination thereof. Additionally, the image processing circuit 28 may include one or more sets of image data processing blocks 50 (e.g., circuits, modules, or processing stages), such as a warping block 52. It should be understood that multiple other processing blocks 54, such as a pixel contrast control (PCC) block, a color management block, a dithering block, a blending block, a burn-in compensation (BIC) block, a scaling / rotation block, etc., may also be incorporated into the image processing circuit 28 before and / or after the warping block 52. The image data processing blocks 50 may receive and process the source image data 48 and output display image data 58 in a format interpretable by the display panel 40 (e.g., digital format, image space, and / or resolution). Further, the functions (e.g., operations) performed by the image processing circuit 28 may be divided among the various image data processing blocks 50, and although the term "block" is used herein, there may or may not be a physical or logical separation between the image data processing blocks 50.

[0052] In some scenarios, due to environmental effects, the performance of the electronic display 12, the viewer's perspective (e.g., POV), image processing alterations such as shifting and scaling, and / or other distortion factors, the image to be displayed may (if unchanged) appear distorted when perceived by the viewer. Thus, in some embodiments, as Figure 8 shown, the warping block 52 may remap the input image data 60 such that the resulting warped image data 62 accounts for such distortions when displayed (e.g., on the display panel 40). As should be appreciated, the input image data 60 may include any suitable image data that needs to be transformed (e.g., warped). For example, the input image data 60 may include graphic image data 64 (e.g., a stored or generated digital image), captured image data 66 (e.g., a video image captured by the camera 36), and / or other image data 68, such as matte image data generated to represent an alpha value for an image blending process, image data received via the network interface 24 or the I / O port 16, etc. Thus, the warping block 52 may generate warped image data 62 (e.g., warped graphic image data 70, warped captured image data 72, warped other image data 74, etc.) to change the amount (e.g., resolution) or distribution (e.g., shape, relative size, perspective, etc.) of the pixel values of the input image data 60, thereby accounting for different display scenarios and / or input image characteristics.

[0053] For example, the warped image data 62 can account for the curved edges associated with the display panel 40 and / or lens effects (e.g., the curved edges and / or lens effects of the cover glass) and / or the POV of the viewer relative to the display panel 40 or relative to the image capture device (e.g., camera 36). Additionally, the electronic display 12 can be a foveated display such that different portions of the display panel 40 are displayed at different resolutions (e.g., depending on the viewer's gaze), and when determining the mapping between the input image data 60 and the warped image data 62, the warping block 52 can take into account the resolutions at different portions of the display panel 40. Additionally, the warping block 52 can also account for the distortion associated with the input image data 60 and / or the image data source 38. For example, the captured image data 66 can be warped to account for the lens effects associated with the captured image (e.g., camera lens distortion) and / or to account for the difference between the user's POV and the POV of the camera 36. As should be appreciated, the captured image data 66 is given as an example set of input image data 60 that can be warped for the distortion associated with the image data source 38, and any set of input image data 60 can be warped for the distortion associated with the corresponding image data source 38 and / or to obtain a common image space. Moreover, multiple warping operations (e.g., accounting for multiple distortion effects) can be implemented via a single warping (e.g., a single mapping accounting for multiple distortions) or successive warping. Thus, prior to display, the input image data 60 can be warped to change the amount or distribution of pixel values such that the perceived image has limited or no distortion.

[0054] In addition, in some embodiments, the warping block 52 may warp multiple different sets of input image data 60 (e.g., graphical image data 64, captured image data 66, other image data 68, etc.) simultaneously (e.g., in parallel) or sequentially for use alone or together. For example, an image may be generated by blending multiple sets of input image data 60 from one or more image data sources 38. However, in some scenarios, the image data to be blended may be warped to a common image space before blending, which may be done by the warping block 52. Image blending (e.g., for virtual reality, mixed reality, and / or augmented reality) may be utilized to incorporate multiple sets of warped image data 62 into a single image frame. For example, a generated object (e.g., warped graphical image data 70) may be incorporated into a captured image of the real-life surrounding environment (e.g., warped captured image data 72), and / or a portion of the captured image may be used as a separate blending layer in the foreground (e.g., based on warped matte image data) such that the generated object is between that portion in the foreground and the background portion of the captured image. Additionally or alternatively, a portion of the captured image (e.g., warped captured image data 72) may be incorporated into a virtual environment (e.g., warped graphical image data 70). Thus, the input image data 60 from one or more image data sources 38 may be blended together after being warped to a common image space via the warping block 52 to form a single output image.

[0055] As discussed above, the warping block 52 of the image processing circuit 28 may warp one or more sets of input image data 60 to account for input distortions (e.g., camera lens distortion), output distortions (e.g., lens effects associated with the shape of the display panel and / or its glass cover plate), processing distortions (e.g., POV change, shift, scale, etc.), and / or to implement a common image space for blending. Moreover, the image processing circuit may include separate warping hardware (e.g., for parallel processing), and / or perform separate warping operations on different sets of input image data using the same hardware. For example, in some embodiments, the warping block 52 may include a graphical warping sub-block 76, a captured warping sub-block 78, and / or other warping sub-blocks 80. As should be appreciated, the sub-blocks described herein are given as examples, and any suitable warping sub-block may utilize the features discussed herein to warp any suitable set of input image data 60 and generate warped image data 62.

[0056] In some embodiments, the warping block 52 may utilize configuration data 82 associated with a desired warping effect to generate a mapping from input image data 60 to warped image data 62. The configuration data 82 may include mappings, algorithms, and / or parameters that indicate the warping to be done for a set of input image data 60. Additionally, the configuration data 82 may include static aspects and / or dynamic aspects and may include different parameters / mappings for different sets of input image data 60. For example, the configuration data 82 may include a static mapping between a generated graphical image space (e.g., graphical image data 64) and a display image space (e.g., warped graphical image data 70) to account for unaltered distortion associated with the electronic display 12. Moreover, the configuration data 82 may include a static mapping between a camera image space (e.g., captured image data 66) and a display image space (e.g., warped captured image data 72) that accounts for unaltered camera lens distortion as well as unaltered distortion associated with the electronic display 12. As should be appreciated, the captured image data 66 from the camera 36 is given as an example set of input image data 60, and such data may or may not be processed or partially processed prior to the warping block 52 of the image processing circuit 28. Also, the camera 36 may include multiple or variable lenses related to the dynamic portion of the configuration data 82. The dynamic aspect of the configuration data may provide different mappings based on the scene at the time of warping (e.g., for the image frame being processed). For example, in a foveated display, the output resolution at different portions of the display panel may change based on the focus of the user's gaze (e.g., determined by eye tracking), which may alter the mapping. In other words, which input pixels of the input image data 60 map to which output pixel locations of the display panel 40 (e.g., as characterized by warping the warped image data 62) may be changed based on the parameters of the configuration data 82. As should be appreciated, the configuration data 82 may include any suitable information (e.g., parameters, tags, flags, algorithms, mappings, etc.) that characterizes the warping to be implemented for a particular set of input image data 60.

[0057] Based on the configuration data 82, mapping data 84 can be generated (e.g., via the mapping and interpolation sub-block 86) to correlate the output pixel values of the warped image data 62 with the pixel values of the input image data 60. As should be appreciated, the output image space can be associated with the physical pixel locations of the display panel 40 (e.g., the display image space) or any desired image space. Moreover, the warping block 52 (e.g., the graphics warping sub-block 76, the captured warping sub-block 78, another warping sub-block 80, etc.) can perform an extraction 88 of the input image data 60 from the relevant image data source 38 (e.g., the memory 20, the graphics generator of the processor core complex 18, other processing blocks 54, the network interface 24, the camera 36, etc.). Using the mapping data 84, the warping block 52 can generate the warped image data 62 based on the input image data 60.

[0058] Figure 9FIG. is a schematic diagram of a warping sub-block 90 of a warping block 52. As should be appreciated, the warping sub-block 90 may indicate a graphical warping sub-block 76, a captured warping sub-block 78, and / or other warping sub-blocks 80 for warping any set of input image data 60. Moreover, in some embodiments, the warping block 52 (or its warping sub-block 90) may include one or more separate sections 92 and / or section portions 94 for different components of a set of input image data 60. The different sections 92, 94 may perform warping operations in different ways depending on the warping to be implemented (e.g., based on mapping data 84). For example, in some embodiments, different components of the input image data 60 may have different resolutions (e.g., bit depths) and thus have different mappings between the input image data 60 and the warped image data 62. In other words, components of a set of input image data 60 that undergo similar warping operations (e.g., where the extracted portions of the components are the same) may be warped in a combined section 94, and each component may be processed through separate but parallel data paths. Also, components of a set of input image data 60 that undergo a single warping operation may be warped in separate sections 92. As a non-limiting example, in some embodiments, the gamma (e.g., Y) component and / or the alpha (e.g., α) component of the input image data 60 may be warped in a separate section 92, while the chrominance components (e.g., Cb and Cr) and / or the primary color components (e.g., red, green, and blue (RGB)) may be warped in a combined section 94. The sections 92, 94 may together generate warped image data 62 corresponding to the set of input image data 60. Additionally or alternatively, the warping sub-block 90 may utilize multiple separate sections 92 operating in parallel regardless of whether the components are similarly processed or differently processed, or may recursively utilize the same separate sections 92 for different components of the input image data 60. As should be appreciated, the sections 92, 94 for warping different components of the input image data 60 may vary based on the implementation. For example, the captured warping sub-block 78 may include an arrangement of sections 92, 94 different from that of the graphical warping sub-block 76 and / or other warping sub-blocks 80. Additionally, the warping block 52 may include multiple warping sub-blocks 90 for warping different sets of input image data 60 (e.g., in parallel) or a single warping sub-block 90. As discussed herein, the warped image data 62 from one or more warping sub-blocks 90 may be displayed or used in further image processing such as blending.

[0059] The warping sub-block 90 can receive the mapping data 84 and utilize the extractor 96 and / or the filter 98 to request (e.g., extract 88) portions of the input image data 60 to populate the two-level cache architecture 100. For example, the two-level cache architecture 100 can include a first cache 102 populated with the input image data 60 by the extractor 96, and a second cache 104 populated with portions of the input image data 60 from the first cache 102. The first cache 102 and the second cache 104 can be of different cache levels or have the same cache level. For example, the first cache 102 can be a level 1 (L1) cache, and the second cache 104 can be a level 0 (L0) cache. As should be appreciated, depending on the particular implementation, any level of cache can be used for the first cache 102 and the second cache 104. Also, the extractor 96 can utilize the mapping data 84 (e.g., based on the configuration data 82) to extract 88 the input image data 60 in an order associated with the mapping between the input image data 60 and the warped image data 62. In other words, the extractor 96 can request the input image data 60 from the first cache 102 in the expected usage order, rather than extracting the input image data 60 in raster scan order, as further discussed below. Also, the filter 98 can utilize one or more tags 106 to populate the second cache 104 from the first cache 102, where the one or more tags indicate the extracted image data to be utilized in the warping. Additionally, the resampler 108 can interpolate output pixel values from a set of the extracted image data in the second cache 104, and if utilized, the conversion buffer 110 can place the output pixel values in an output format indicative of the warped image data 62.

[0060] In some embodiments, as Figure 10 shown in the example first cache 102 of Figure 10The virtual curve 114 can indicate the first (e.g., top) row of the warped image data 62. Since the virtual curve 114 does not intersect the first section 116 of the tile 112, the pixel values of the tile 112 in the first section 116 may not be utilized in the warp and can thus be skipped during extraction 88. As should be appreciated, in some embodiments, the buffer around the virtual curve 114 can be considered to include pixel values that can be used in interpolating output pixel values, and the tile 112 that intersects the virtual curve 114 and / or the buffer can be extracted 88. Also, the sections 116, 118, 120 discussed herein are given for illustrative purposes and may or may not indicate a logical division by the warp block 52. Depending on the warp to be implemented (e.g., according to the mapping data 84), the virtual curve 114 can span multiple tiles 112 both vertically and horizontally. In Figure 10 the example, the virtual curve 114 spans the tiles 112 of the second section 118 and is three vertical tiles in height. As such, the extraction 88 can include the tiles 112 of the second section 118. In some scenarios, it may be beneficial (e.g., for memory access and / or extraction 88) to maintain a constant height (e.g., the number of vertical tiles 112) when extracting 88 the input image data 60. As such, in some embodiments, the tiles 112 of the third section 120 can be extracted 88 together with the second section 118 and can serve as part of a pre-extraction for subsequent warp operations (e.g., for subsequent output rows). Also, when generating additional output rows of the warped image data 62, the extraction 88 of the input image data 60 can continue. As should be appreciated, depending on the implementation, the tiles 112 extracted 88 can include an offset 122 that can increase the extraction 88 and / or cache size efficiency, or can be banded into rectangular sections of the tiles 112 that include the virtual curve 114.

[0061] The extractor 96 and / or the filter 98 can use the tag 106 to correlate a portion of the first cache 102 with the pixel coordinates of the input image data 60 (e.g., along the virtual curve 114). Also, the tag 106 can be used (e.g., by the filter 98) to define a sliding window 124 that includes the mapped input pixels 126 of the input image data 60 that are mapped to the output pixels of the warped image data 62, as Figure 11As shown in. Pixel values within the sliding window 124 can be used to populate the second cache 104 (e.g., based on the tag 106), and the populated second cache 104 can be utilized to generate one or more output pixel values of the warped image data 62. By leveraging the two-level cache architecture 100, more efficient extraction 88 can be achieved while concurrently determining output pixel values to ensure the maintenance of timing constraints (e.g., the output timing of the warped image data 62). Additionally, the two-level cache architecture can improve efficiency by retaining the extracted input image data 60 in the first cache 102 while populating the second cache 104 based on the sliding window 124 traversing the first cache 102. Thus, the input image data 60 can be extracted 88 from the image data source 38 (e.g., the memory 20, etc.) once, thereby reducing or eliminating duplicate extractions 88 of the same pixel values. Moreover, in some embodiments, the tag 106 can be used for both extracting 88 the pixel values of the sliding window 124 and populating the second cache 104 with these pixel values, thereby further enhancing efficiency and reducing redundant processing.

[0062] The sliding window 124 can traverse the first cache 102 (e.g., along the virtual curve 114) and populate the second cache 104 with pixel values including the mapped input pixels 126. In some embodiments, the second cache 104 can include multiple mapped input pixels 126 such that multiple output pixel values of the warped image data 62 are generated simultaneously (e.g., considered or calculated concurrently), which can also improve efficiency and / or shorten the processing time. Additionally, the multiple mapped input pixels 126 can be associated with a single or multiple virtual curves 114 to generate the output pixel values of one or more output rows of the warped image data 62. For example, Figure 12 is an example second cache 104 populated with pixel values 128 of the input image data 60 pulled from the first cache 102, which include the mapped input pixels 126 of two output rows (e.g., the virtual curve 114). Additionally, each mapped input pixel 126 can be associated with a set of support pixels 130. The support pixels 130 can be used for interpolation and / or other image processing techniques (e.g., dithering) when generating the corresponding output pixel values of the warped image data 62. For example, as should be understood, the output pixels may or may not be directly mapped to the mapped input pixels 126, but rather to pixel positions that may or may not have integer values (e.g., aligned with the pixel grid of the input image data 60). As such, the output pixel values can be interpolated based on the mapped input pixels 126 and the support pixels 130. Additionally, by considering multiple mapped input pixels 126 together, the support pixels 130 can overlap, thereby increasing efficiency through reduction while Figure 12Four mapped input pixels 126 corresponding to a 2×2 set of output pixels of the warped image data 62 are shown in the second cache 104. As should be appreciated, any number (e.g., 1, 2, 4, 6, 9, 16, etc.) of mapped input pixels 126 may be retained in the second cache 104 and processed together, depending on the particular implementation. Additionally, any number of support pixels 130 may be attributed to the mapped input pixels 126, depending on the particular implementation.

[0063] Depending on the desired warp to be implemented and / or the programmed boundaries of the warp process, the size of the second cache 104 may be set such that a bounding box 132 that includes the mapped input pixels 126 and the support pixels 130 fits within the second cache 104. In other words, the bounding box 132 may be less than or equal to the size of the second cache 104. The bounding box 132 may define a set of pixels for generating output pixels. For example, the pixels within the bounding box may be used by the resampler 108 to generate the warped image data 62. Also, the maximum size of the bounding box 132 may be set based on the input requirements of the resampler 108, and both may vary depending on the particular implementation. Using the pixel values 128 within the bounding box 132 of the second cache 104, the resampler 108 may interpolate the output pixel values of the warped image data 62.

[0064] As discussed above, the bounding box 132 may contain the mapped input pixels 126 and their support pixels 130. Also, the desired warp (e.g., the range of the mapped data 84) may be predetermined (e.g., based on the estimated or known extreme values of the virtual curve 114), and the size of the bounding box 132 and / or the second cache 104 may be based thereon. However, in some scenarios, it may be desirable to reference specific warp operations on outlier pixels 134, mapped input pixels 126, and / or support pixels 130 that are not within the bounding box 132, as Figure 13 shown. Such outlier pixels 134 may or may not be included in the second cache 104. However, to maintain synchronization and timing, the outlier pixels 134 may be trimmed, and a set of replacement pixels 136 may be used to replace the outlier pixels 134. In some embodiments, the set of replacement pixels 136 may be the set of pixels closest to the outlier pixels 134 enclosed within the bounding box 132. In this way, processing integrity (e.g., continue operation) and efficiency may be maintained for any suitable warp profile (e.g., the virtual curve 114 based on the mapped data 84).

[0065] Figure 14FIG. 140 is a flow chart of an example process for warping input image data 60 and generating warped image data 62. An image processing circuit 28, such as a warping block 52, may receive configuration data 82 that characterizes the warping to be implemented for a set of input image data 60 (process block 150). As should be appreciated, receiving configuration data 82 may include, but is not limited to: using static parameters, algorithms, etc. from a memory 20, and / or determining dynamic aspects, such as considering eye tracking data and / or relative positions for POV correction. Additionally, mapping data 84 (e.g., indicating a virtual curve 114) may be generated based on the configuration data 82 (process block 160). A portion of the input image data 60 (e.g., a tile 112) may be extracted from an image data source 38 based on the mapping data to populate a first cache 102 of a two-level cache architecture 100 (process block 170). Additionally, a second cache 104 of the two-level cache architecture 100 may be populated with pixel values 128 of a sliding window 124 that traverses the first cache 102 based on the mapping data 84 (process block 180). For example, the sliding window 124 may follow the virtual curve 114 and include mapped input pixels 126 corresponding to output pixel values of the warped image data 62. The pixel values 128 of the second cache (e.g., the mapped input pixels 126 and corresponding support pixels 130) may be interpolated (e.g., via a resampler 108) to generate the warped image data 62 (process block 190), and the warped image data 62 may be output (process block 200) for further processing (e.g., blending, compensation, etc.) and / or viewing on an electronic display 12.

[0066] In conjunction with the warping block 52, the two-level cache architecture 100 may provide synchronization and / or higher efficiency with respect to timing constraints to allow for a higher bandwidth (e.g., more input image data) for the warping block 52. Such efficiency may allow for real-time operations, such as warping a live camera feed for blending and / or viewing. Moreover, by saving processing time, there may be sufficient time to perform other image processing techniques (e.g., blending, compensation, etc.) while maintaining real-time operation. Additionally, although FIG. 140 is shown in a given order, in certain embodiments, process blocks / decision blocks may be reordered, altered, deleted, and / or occur simultaneously. Further, FIG. 140 is given as an illustrative tool and additional decision blocks and process blocks may also be added depending on the particular implementation.

[0067] The specific embodiments described above have been shown by way of example, and it should be understood that these embodiments are susceptible to various modifications and alternative forms. It should also be understood that the claims are not intended to be limited to the particular forms disclosed, but rather are intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure.

[0068] It is well known that the use of personally identifiable information should follow privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining user privacy. Specifically, personally identifiable information data should be managed and processed to minimize the risk of inadvertent or unauthorized access or use, and the nature of the authorized use should be clearly explained to users.

[0069] The technologies presented and claimed herein are recited and applied to specific examples of a physical and tangible nature that demonstrably improve the art and are thus not abstract, intangible, or purely theoretical. Additionally, if any claim appended to the end of this specification contains one or more elements designated as "means for [performing] [function]..." or "step for [performing] [function]...", such elements will be construed in accordance with 35 U.S.C. 112(f). However, for any claim containing elements designated in any other manner, it is intended that such elements shall not be construed in accordance with 35 U.S.C. 112(f).

Claims

1. A device, the device comprising: an electronic display configured to display an image based on warped image data; and an image processing circuit configured to generate the warped image data by warping input image data to account for one or more distortions associated with displaying the image, wherein the image processing circuit includes a two-level cache architecture including a first cache and a second cache, and wherein warping the input image data includes: generating mapping data indicative of a warp between an input image space of the input image data and an output image space of the warped image data; extracting the input image data to fill the first cache; filling the second cache with groups of pixel values of the input image data from the first cache, wherein the groups of pixel values are selected according to a sliding window traversing the first cache based on the mapping data; and using the second cache to interpolate between input pixel values of the groups of pixel values to generate one or more output pixel values of the warped image data.

2. The device according to claim 1, wherein the mapping data is related to a virtual curve indicative of a row of output pixel positions in the input image space of the output image space, wherein the sliding window traverses the first cache along the virtual curve, and the one or more output pixel values correspond to one or more respective pixel positions of the row of output pixel positions.

3. The device according to claim 1, wherein extracting the input image data to fill the first cache comprises: extracting the input image data in order from an image data source, wherein the order is based on the mapping data.

4. The device according to claim 1, wherein the input image data includes graphic image data and captured image data, and wherein the image processing circuit includes a graphic warp sub-block and a captured warp sub-block, the graphic warp sub-block being configured to warp the graphic image data, and the captured warp sub-block being configured to warp the captured image data configured to operate in parallel.

5. The device according to claim 4, wherein the image processing circuit includes a mixing circuit configured to combine warped graphic image data and warped captured image data for a single image frame.

6. The device according to claim 1, wherein the one or more distortions include a lens effect associated with glass of the electronic display.

7. The device according to claim 1, wherein the image processing circuit includes a hardware pipeline having a dedicated warp circuit configured to generate the warped image data.

8. The device according to claim 1, wherein the mapping data is based on a viewer's viewing point relative to the electronic display.

9. The apparatus according to claim 8, wherein the electronic display includes a foveal display, and wherein the viewer's viewing point includes the viewer's focus on the foveal display.

10. An image processing circuit, the image processing circuit comprising: a two-level cache architecture including a first cache and a second cache; an extractor configured to extract input image data and populate the first cache; a filter configured to select a portion of the input image data from the first cache to populate the second cache based on mapping data, wherein the mapping data indicates a warp between an input image space of the input image data and an output image space of warped image data; and a resampler configured to interpolate between pixel values of the input image data in the second cache to generate one or more pixel values of the warped image data.

11. The image processing circuit according to claim 10, wherein the filter is configured to select the portion of the input image data from the first cache according to a sliding window that traverses the first cache based on the mapping data.

12. The image processing circuit according to claim 10, wherein the extractor is configured to populate the first cache in order based on the mapping data.

13. The image processing circuit according to claim 12, wherein the extractor is configured to populate the first cache in order based on one or more tags associated with pixel positions along a virtual curve, the virtual curve indicating a row of output pixel positions in the input image space of the output image space, wherein the virtual curve is based on the mapping data.

14. The image processing circuit according to claim 13, wherein the filter is configured to select the portion of the input image data from the first cache based on the one or more tags.

15. The image processing circuit according to claim 10, wherein the extractor is configured to extract the input image data from an image data source such that each portion of the input image data is only extracted from the image data source once.

16. The image processing circuit according to claim 10, the image processing circuit including a mapping and interpolation sub-block configured to generate the mapping data based on a plurality of parameters characterizing physical distortion effects of a camera, an electronic display, or both.

17. A non-transitory machine-readable medium including instructions, wherein when executed by one or more processors, the instructions cause the one or more processors to control operations of an image processing circuit, the operations including: generating mapping data indicating a warp between an input image space of input image data and an output image space of warped image data; extracting the input image data to populate a first cache of a two-level cache architecture; Filling a second cache of the two - level cache architecture with groups of pixel values of the input image data from the first cache, wherein the groups of pixel values are selected according to a sliding window that traverses the first cache based on the mapping data; And Using the second cache to interpolate between the input pixel values of the groups of pixel values to generate one or more output pixel values of the warped image data.

18. The non - transitory machine - readable medium according to claim 17, wherein the mapping data is related to a virtual curve indicating rows of output pixel positions in the output image space within the input image space.

19. The non - transitory machine - readable medium according to claim 18, wherein the rows of output pixel positions correspond to one or more mapped input pixel positions of the input image data, and wherein the operation Comprises: In response to determining that a mapped input pixel position of the one or more mapped input pixel positions is outside a bounding box, during the interpolation of the input pixel values, replacing a first pixel value associated with the mapped input pixel position with a second pixel value within the bounding box, wherein the bounding box comprises a continuous set of pixel values within the second cache.

20. The non - transitory machine - readable medium according to claim 18, wherein extracting the input image data to fill the first cache Comprises: Extracting the input image data from an image data source in an order based on the virtual curve.

21. A system, the system Comprises: Means for filling a second cache of a two - level cache architecture with groups of pixel values of input image data from a first cache, wherein the groups of pixel values are selected according to a sliding window that traverses the first cache based on mapping data; And Means for using the second cache to interpolate between the input pixel values of the groups of pixel values to generate one or more output pixel values of warped image data.

22. The system according to claim 21, the system Comprises: Means for generating mapping data indicating a warp between an input image space of the input image data and an output image space of the warped image data; And Means for extracting the input image data to fill the first cache of the two - level cache architecture.

23. An electronic device, the electronic device Comprises: One or more electronic displays configured to display warped image data in a warped display space; And Image processing circuitry configured to generate the warped image data, the image processing circuitry comprising: A two - level cache architecture comprising a first cache and a second cache; An extractor configured to extract input image data and fill the first cache; A filter configured to select a portion of the input image data from the first cache to populate the second cache based on mapping data that indicates a warp between an input image space of the input image data and an output image space of the warped image data; and A resampler configured to interpolate between pixel values of the input image data in the second cache to generate one or more pixel values of the warped image data.

24. The electronic device according to claim 23, the electronic device including an image source configured to generate the input image data in an input image data space that is not suitable for display on the electronic display.

25. The electronic device according to claim 23, wherein the one or more electronic displays include a plurality of electronic displays.

Images