Cache-based warping engine

Abstract

The present invention relates to an image warping system and method capable of performing image warping quickly at low cost by using a cache memory. The image warping system is provided to generate a transformed image by warping an input image with the help of a cache-based warping engine. The warping engine accesses the input image and loads a portion of the image to a cache memory in order to speed up the engine process. The warping engine performs interpolation on the input image to generate an output image without distortion. The obtained output image is then stored in a DDR SDRAM of an electronic device.

Description

TECHNICAL FIELD

[0001] The present invention relates to an image warping system using cache memory and a method thereof. More particularly, the present invention relates to an image warping system and method capable of performing image warping at a low cost using cache memory. BACKGROUND

[0002] Image warping refers to transforming the geometry of an original image by resampling the original image according to certain rules. As an example, rectifying lens distortion of a captured image to generate an image corrected for lens distortion is exemplified. Image warping can also be used for matching image correction of stereo images or for composing a panoramic image by synthesizing multiple images.

[0003] Image warping is performed by the process of pixel coordinate conversion between an original image and a warped image. The transformation rule between the coordinates (x, y) of the pixels of the warped image can be represented by a predetermined mapping function. There are many existing technologies that discuss warping techniques available.

[0004] One of the many existing technologies that disclose image warping techniques is US Patent No. 6756993B2 assigned to the University of North Carolina at Chapel Hill. The invention provides a method and apparatus for rendering an image, and more particularly, to a method and apparatus for rendering an image using 3D warping techniques. The patent discusses a method of bilinearly interpolating an image to generate an interpolated output image. The disclosed warping technique removes distortion from a warped image and produces a distortion-free image. However, the described technique is only capable of removing distortion from a warped image and cannot perform geometric transformation, panorama generation, and blending of two images.

[0005] Another US Patent No. 8649558B2 assigned to the Wisconsin Alumni Research Foundation relates to processing image data, and more particularly, to an apparatus, system, and method for warping video data. Video data is processed by resizing each of a plurality of source video frames to warp (e.g., resize) the video data for display. The patent overcomes the shortcomings of the above-mentioned patent by introducing features such as geometric transformation, panorama generation, and blending. However, the patent does not include a cache-based warping engine to store images for their later use. Therefore, the disclosed image warping technique becomes an expensive and time-consuming technique.

[0006] To overcome the limitations of the above-mentioned patents, Korean Patent No. 101540300B1 assigned to Robot Co., Ltd. relates to an image warping system using cache memory and a method thereof. However, the disclosed technique only supports one image format and only performs distortion correction.

[0007] Accordingly, to overcome the drawbacks of the prior art, a cache-based warp engine has been disclosed which performs distortion correction, panorama generation, and perspective correction on input images. Moreover, the cache-based warp engine utilizes very little cache memory to speed up the engine process, thereby significantly reducing warp cost.

[0008] It is now apparent that many methods and systems have been developed in the prior art to address various purposes. Moreover, while these inventions can be suitable for their intended purpose, they would not be suitable for the purposes of the present invention as described above. SUMMARY

[0009] It is an object of the present invention to provide an image warping system that can rapidly perform image warping at low cost by utilizing cache memory. Embodiments of the present invention provide a solution to the above and other problems of systems and methods in the art for warping panoramic images using a cache-based warp engine (WARP engine). Specifically, the disclosed systems and methods generate an enhanced panoramic image by warping a panoramic digital image having a skewed boundary to fit a predetermined shape using a cache-based warp engine that can support multiple image formats without considering image content. The cache-based warp engine supports multiple image formats.

[0010] It is a primary object of the present invention to provide a cache-based warp engine that performs image distortion correction, geometric transformation, panorama generation, and blending of two or more images together. The cache-based warp engine reduces DDR SDRAM access time and DDR SDRAM bandwidth requirements, and also reduces local SRAM requirements. In addition, the engine unifies multiple image format processing speed, thereby making the engine processing speed independent of image format. The different types of image formats supported by the warp-based engine include RGB888, RGB888-Planar, YUV422, NV12, and NV21, and YUV420 (Planar).

[0011] According to one aspect of the present invention, an image warping system is provided for generating a transformed image comprising a plurality of pixel groups by warping an original image comprising a plurality of original image tiles. In a primary embodiment, the original image is stored in a DDR SDRAM of an electronic device. The cache memory of the device is implemented in SRAM (static RAM) to store input image data for access by a cache-based warping engine. The cache memory typically comprises SRAM, which is more expensive than DDR but can provide much higher bandwidth. The input image is stored in DDR SDRAM and only a portion of the image data is loaded into the cache memory to speed up the engine process. The cache memory of the device is comprised of a plurality of cache lines that store input image data. The present invention typically supports 32k byte caches; however, the size of the cache memory is configurable. Further, the size of the individual cache lines is configurable.

[0012] Another object of the present invention is to provide a cache lookup table stored within the cache memory for storing order information of the original image tiles to be loaded into the cache memory in sequence. The cache lookup table comprises a plurality of coordinates defining information related to the input image. The coordinates of the output image comprise 2 bytes for the X coordinate and 2 bytes for the Y coordinate. The image warping system can use the image lookup table to perform image warping on the original image to generate a warped image. The image lookup table can be a lookup table that stores mapping information between the pixels included in the transformed image and the pixels included in the original image.

[0013] The cache-based warping engine of the present invention will use the coordinates to map the input image to the output image. The coordinates of the output image are fixed from left to right and from top to bottom. The engine will follow the output sequence of the coordinates and look at the warp table to determine which portion of the input data the engine should grab. The warp engine uses the same cache line and the same cache size to support different image formats. The only difference between the individual image formats supported is the bandwidth requirement, different image formats require different bandwidth to achieve the same frame rate.

[0014] The warp engine then performs the warping operation by bilinearly or bicubicly interpolating the input image based on the plurality of coordinates of the input image. The cache-based warp engine supports both bilinear and bicubic image interpolation techniques. Bicubic interpolation provides better image quality than the image quality achieved by bilinear interpolation. The processor of the warp engine computes the output image by a raster scan method.

[0015] According to another aspect of the present application, there is provided an image warping system comprising a processor and a memory for storing a computer program executed by the processor, wherein the computer program, when executed by the processor, implements the above method. There is provided an image warping system for causing the above method to be performed.

[0016] Other objects and aspects of the application will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the features of embodiments in accordance with the application.

[0017] To the accomplishment of the foregoing and related objects, the application can take the form of implementations illustrated in the accompanying drawings, attention being called to the fact, however, that the drawings illustrate only typical implementations and are therefore not to be considered limiting of its scope, as the application can admit to other equally effective implementations.

[0018] While the application has been described above by reference to various example embodiments and implementations, it will be understood that the various features, aspects and functionality described in one or more separate embodiments are not limited in their applicability to the particular embodiment with which they are described, and they can be applied individually or in various combinations to one or more other embodiments of the application, whether or not they are described in combination with such embodiments. The scope of the application is therefore not to be limited by any of the above example embodiments.

[0019] In some cases, the presence of broadening words and phrases such as "one or more," "at least," "but not limited to," or other like phrases, indicates that the subject term includes the listing of identifier followed by such broadening phrase, and does not foreclose the addition of other, identical or non-identical identifiers that are not recited. BRIEF DESCRIPTION OF DRAWINGS

[0020] The objects and features of the present application will become more fully apparent from the following description and appended claims taken in conjunction with the accompanying drawings. Understanding that these drawings depict only typical embodiments of the application and are not therefore to be considered to be limiting of its scope, the principles and features of this application can be better understood by reference to the drawings, wherein:

[0021] Figure 1 A method of warping an input image using a cache-based warping engine is illustrated;

[0022] Figure 2 A system for warping an input image in accordance with the present application is illustrated;

[0023] Figure 3 An input device for a system in accordance with the present application is illustrated;

[0024] Figure 4 A cache-based warping engine for a system in accordance with the present application is illustrated;

[0025] Figure 5 Coordinates of a distorted input image and a non-distorted output image according to the present application are illustrated;

[0026] Figure 6A A cache tag according to the present application is illustrated;

[0027] Figure 6B A twisted cache line access according to the present application is illustrated; and

[0028] Figure 6C A twisted cache bank for accessing a cache line according to the present application is illustrated. DETAILED DESCRIPTION

[0029] Content (e.g., visual content) can be captured by one or more image capture devices (e.g., cameras) as more images or video frames. An image capture device includes one or more lenses, image sensors, image signal processors, encoders, etc. A lens can receive light and focus the light on an image sensor. An image sensor can sample the light and generate an electronic image signal. An image signal processor (ISP) can receive the image signal from the image sensor and can process the image signal to generate an image, picture, or frame. The generated image can be stored in, for example, a memory of the image capture device.

[0030] Some image capture devices use lenses (e.g., fisheye or spherical lenses) that significantly distort the captured images. An image signal processor can apply a twist transform to correct for lens distortion and other distortions associated with capturing an image with one or more image sensors (e.g., electronic rolling shutter correction, binocular parallax correction, image stitching, electronic image stabilization, etc.). Because some of these distortions can be significant, the twist transform that corrects for the distortions can significantly move portions (e.g., pixels or blocks of pixels) within the image(s).

[0031] Figure 1 A method for twisting an input image using a cache-based twist engine is illustrated. The twist engine performs image distortion correction, geometric transformation, panorama generation, and blending of two or more images together. The cache-based twist engine reduces DDR SDRAM access time and DDR SDRAM bandwidth requirements, and also reduces local SRAM requirements. In addition, the engine unifies multiple image format processing speeds, such that the engine processing speed is independent of the image format.

[0032] A method for warping an input image using a cache-based warp engine includes accessing an input image 102 from a DDR SDRAM of a device. Next, the method includes loading a portion of the input image to a cache memory 104 of the device. The cache memory of the device is implemented in SRAM (static RAM). The cache memory of the device is comprised of a plurality of cache lines that store input image data. The present invention generally supports a 32k byte cache; however, the size of the cache memory is configurable. Additionally, the size of the individual cache lines is configurable. Accessing a portion of the input image from the cache memory, rather than the DDR SDRAM, reduces local storage (SRAM) requirements. This further reduces DDR SDRAM access time and DDR SDRAM bandwidth requirements.

[0033] In the next step 106, the pre-loading of the lookup table stored in the cache memory with some preliminary information is completed. The LUT (lookup table) includes a plurality of coordinates of the input image. The cache lookup table includes a plurality of coordinates that define information related to the input image. The image warp system can use the image lookup table to perform image warping on the original image to generate a warped image. The coordinates of the output image include 2 bytes for the X coordinate and 2 bytes for the Y coordinate. In the next step, the warp engine follows the plurality of coordinates, where the engine uses the coordinates to map the input image to the output image. The cache-based warp engine of the present invention will use the coordinates to map the input image to the output image. The coordinates of the output image are fixed from left to right and from top to bottom. The engine will follow the output sequence of the coordinates and look at the warp table to determine which portion of the input data the engine should grab.

[0034] After following the coordinates, the input image is used to form an interpolated image 108 by mapping (using different coordinates by bilinearly or bicubicly interpolating the input image) the input image. Bilinear interpolation includes a resampling method that estimates new pixel values using a distance-weighted average of the four nearest pixel values. The four cell centers from the input raster that are closest to the cell center of the output processing cell will be weighted based on distance and then averaged. Bicubic interpolation is a 2D system that sharpens and enlarges digital images using cubic splines or other polynomial techniques. In the next step, the output image ready for display to the user is calculated by the warp engine 110 using a raster scan method. The output image can be stored in the DDR SDRAM of the electronic device 112.

[0035] Figure 2A system for warping an input image is illustrated. In one embodiment, the original image is stored in the DDR SDRAM 204 of the input device 202. Double Data Rate Synchronous Dynamic Random-Access Memory (formally known as DDR SDRAM) is a type of memory integrated circuit used in computers that is of the Double Data Rate (DDR) Synchronous Dynamic Random-Access Memory (SDRAM) class. In the primary embodiment, the input device 202 is a desktop computer. In alternative embodiments, the device can be a portable device (e.g., a laptop computer, a tablet computer, or a handheld device). The device can have a touchpad and a touch-sensitive display (also known as a "touchscreen" or "touchscreen display").

[0036] The cache memory 206 of the device is implemented in SRAM (Static RAM) to store the input image data for access by the cache-based warp engine 210. The cache memory 206 typically includes SRAM, which is more expensive than DDR SDRAM but can provide much higher bandwidth. The cache memory 206 of the device 202 is composed of multiple cache lines that store the input image data. The present invention typically supports 32k byte caches; however, the size of the cache memory 206 is configurable. Additionally, the size of the individual cache lines is configurable.

[0037] The cache memory 206 is a special very high-speed memory. The cache memory 206 is used for acceleration and synchronization of high-speed CPUs. The cache memory 206 is used to reduce the average time to access data from the main memory. A cache is a smaller and faster memory that stores copies of data from frequently used memory locations of the main memory. There are various different independent caches in a CPU that store instructions and data. When a processor needs to read or write a location in the main memory, the processor first checks the corresponding entry in the cache. There are three different types of mapping for the purpose of the cache memory 206, as follows: direct mapping, associative mapping, and set-associative mapping.

[0038] The cache memory 206 is divided into multiple lines that store data including input images to be warped to obtain output images. The cache memory 206 of the electronic device accesses a small portion of the input images to speed up the engine 210 process. This enables the warp engine 210 to access the input images directly from the cache memory 206 instead of from the DDR SDRAM 204. Accessing a portion of the input images from the cache memory 206 instead of the DDR SDRAM reduces the local memory (SRAM) requirement. This further reduces the DDR SDRAM access time and DDR SDRAM bandwidth requirement.

[0039] The cache memory 206 includes a cache LUT or cache lookup table 208 that stores some information about the input images. The cache lookup table 208 stores the input image coordinates that are utilized by the warp engine 210 (described in subsequent paragraphs) to obtain the output images. The cache lookup table 208 is an 8 x 8 interval lookup table 208 that contains 2 bytes for the x coordinate and 2 bytes for the y coordinate. The cache lookup table 208 also supports up to 4095 x 4095 image size. The lookup table 208 does not require a large amount of memory space within the cache memory 206. A very small cache size (a few kilobytes) is required to store the cache lookup table 208. The cache-based warp engine 210 of the present invention will use the coordinates to map the input image to the output image. The coordinates of the output image are fixed from left to right and from top to bottom. The engine 210 will follow the output sequence of the coordinates and look at the warp table 208 to determine which portion of the input data the engine 210 should grab.

[0040] In addition, the present invention discloses a cache-based warp engine 210 that performs a warp operation on an input image and provides a warped output image with no distortion. The cache-based warp engine 210 includes a preloader 212 for loading a set of coordinates to a cache lookup table. The set of coordinates is based on the coordinates of the input image. The warp engine 210 also includes a follower that follows the set of coordinates of the input image and an image interpolator 214. The image interpolator 214 bilinearly or bicubicly interpolates the input image. The interpolation method of the present invention uses simple linear equations to obtain the interpolated pixel values, thereby significantly reducing the amount of computation, which facilitates hardware implementation and reduces cost.

[0041] The cache-based warp engine 210 of the present application can support bilinear and bicubic interpolation techniques. Bicubic interpolation provides better image quality than bilinear interpolation. The warp engine 210 supports a number of image formats for different applications, including RGB888 (planar and non-planar), YUV444, YUV422 (planar), and NV12. The warp engine 210 uses the same cache line and the same cache size to support these image formats. The only difference between the various image format support is the bandwidth requirement, with different image formats requiring different bandwidths to achieve the same frame rate. The warp engine 210 performs image distortion correction, geometric transformation, panorama generation, and blending of two or more images together. The cache-based warp engine 210 reduces DDR SDRAM access time and DDR SDRAM bandwidth requirements, and reduces local SRAM requirements. In addition, the engine 210 unifies the multiple image format processing speed, such that the engine 210 processing speed is independent of the image format.

[0042] A processor 216 for calculating an output image by raster scan interpolation of an image is installed inside the cache-based warp engine. In the raster scan method, an electron beam scans the screen one line at a time from top to bottom. The cache-based warp engine also includes a memory element 218 for storing the output image calculated by the processor 216. The stored image is then loaded to the DDR SDRAM of the electronic device.

[0043] Figure 3 An input device 300 according to the present application is illustrated. The warping is implemented by a physical system that provides a user with a system that acts between the input image and the end user display device 300. In these contexts, image processing / transformations and related operations can be performed on the image display device 300. The image warp engine outputs warped image data that is provided for output to a display (e.g., a resized display), with handheld devices and flat panel displays (e.g., for computers or televisions) shown by way of example. In a primary embodiment, the input device 300 is a desktop computer. In alternative embodiments, the device is portable (e.g., a laptop computer, a tablet computer, or a handheld device). The device can have a touchpad and a touch-sensitive display (also known as a "touch screen" or "touch screen display").

[0044] Additionally, each of the various components can be executed in software-implemented modules, in logic circuits, processors, or computers. These software-implemented modules can be executed partially or entirely in the handheld device 300, such as a mobile phone device or a tablet device. In certain embodiments, aspects of these modules are executed at a location remote from the display device 300, such as a mobile phone, and can be executed based on known screen size constraints at the handheld device 300 to which the video is streamed.

[0045] The input device 300 of the present invention includes a DDR SDRAM 302 for storing the input image. In one embodiment, the original image is stored in the DDR SDRAM 302 of the electronic device 300. Double Data Rate Synchronous Dynamic Random-Access Memory (formally abbreviated as DDR SDRAM) is a type of memory integrated circuit used in computers that adheres to a double data rate (DDR) Synchronous Dynamic Random-Access Memory (SDRAM) class.

[0046] The input device 300 also includes a cache memory 304. In accordance with the present invention, a cache memory 304 is provided that includes a plurality of cache blocks and a plurality of cache blocks that are to be loaded into the cache memory 304 sequentially. The cache memory 304 includes a cache LUT or cache lookup table 306 that stores some information about the input image. The cache lookup table 306 stores the input image coordinates that are utilized by the warping engine (described in subsequent paragraphs) to obtain the output image. The cache lookup table 306 is an 8x8 interval lookup table that contains 2 bytes for the x coordinate and 2 bytes for the y coordinate. The cache lookup table 306 also supports image sizes up to 4095x4095.

[0047] The lookup table 306 does not require a large amount of storage space within the cache memory 304. A very small cache size (a few kilobytes) is required to store the cache lookup table 306. The cache-based warping engine of the present invention will use the coordinates of the output image to map the input image to the output image. The coordinates of the output image are fixed from left to right and from top to bottom. The engine will follow the output sequence of coordinates and look at the warp table 306 to determine which portion of the input data the engine should grab.

[0048] Figure 4A cache-based warping engine according to the present application is illustrated. The cache-based warping engine 400 includes a pre-loader 402 that loads a set of coordinates of an output image to be obtained based on the coordinates of an input image to a cache lookup table. The loading of the coordinates is done with the help of a coordinate loading unit 404 that stores the coordinates of the input image within the pre-loader 402. The cache-based warping engine 400 of the present application will use the coordinates of the output image to map the input image to the output image. The coordinates of the output image are fixed from left to right and from top to bottom. The engine will follow the output sequence of the coordinates and look at the warp table to determine which part of the input data the engine should grab.

[0049] The cache lookup table is an 8x8 interval lookup table that contains 2 bytes for the x coordinate and 2 bytes for the y coordinate. The cache lookup table also supports image sizes up to 4095x4095. The lookup table does not require a large amount of storage space within the cache. A very small cache size (a few kilobytes) is required to store the cache lookup table.

[0050] Additionally, the cache-based warping engine includes an image interpolator 406 for interpolating the input image. The cache-based warping engine supports bilinear and bicubic interpolation. In order for the interpolator to perform the interpolation of the input image, the interpolator includes a bilinear interpolation unit 408 and a bicubic interpolation unit 410. The interpolation method of the present application uses simple linear equations to obtain the interpolated pixel values, thereby significantly reducing the amount of computation, which facilitates hardware implementation and reduces cost.

[0051] Bilinear interpolation includes a resampling method that estimates a new pixel value using a distance-weighted average of four nearest pixel values. The four cell centers from the input raster that are closest to the cell center of the output processing cell will be weighted based on distance and then averaged. Bicubic interpolation is a 2D system that uses cubic splines or other polynomial techniques to sharpen and enlarge digital images. In the next step, the output image ready to be displayed to the user is computed by the warping engine 400 using a raster scan method. The output image can be stored in the DDR SDRAM of the electronic device. A processor 412 including an output image calculator 414 is installed inside the cache-based warping engine 400 to calculate the output image after scanning the output image using the raster scan method. The engine 400 also has a memory 416 for storing the obtained output image. The memory 416 loads the output image to the DDR SDRAM of the electronic device.

[0052] Figure 5Coordinates of a distorted input image and a non-distorted output image according to the present application are illustrated. A distorted input image 502 and a non-distorted output image 504 are illustrated in diagram 500. As shown, the input image 502 is shown in the diagram without uniform borders. The input image 502 has two areas marked in green and red, which are shown to be distorted. The coordinates of the red area 502b of the input image are shown as a0, a1, a2, a3. The coordinates are outside the border and overlap with the gray area 502a in the diagram, which makes it distorted.

[0053] The area 502c shown in green in the input image 502 has coordinates b0, b1, b2, b3. The coordinates are outside the border and overlap with the red area 502b in the diagram, which makes it distorted. To remove the distortion, the warp engine employs a warp technique to make the borders of the input image 502 uniform. The pixels or coordinates of the input image 502, including the red area 502b and the green area 502c, are stored in a cache lookup table within the cache memory and utilized during the image interpolation process. The process of preloading the lookup table begins before the interpolation operation. Once all the required input pixels are loaded from the DDR SDRAM to the cache memory, the warp engine is set to proceed with the interpolation process.

[0054] First, the red area 502b of the input image 502 is loaded into the cache memory, after which the processing of the red area begins. While the processing of the red area 502b continues, the green area 502c of the input image 502 is loaded into the cache memory. Once the loading of the green area 502c is complete, the warp engine begins processing the green area 502c of the input image 502. Once all the required input pixels are loaded from the DDR SDRAM to the cache memory, the warp engine is set to proceed with the interpolation process. The warp will process and output 16 blocks in the first row, then process and output 16 blocks in the second row, after which the warp will process the last 16 blocks in the first row horizontally, after processing and outputting the first 16 blocks vertically. This will minimize the cache size regardless of the image size, distortion, and transformation type. The warp engine supports bilinear and bicubic interpolation methods to interpolate the input image 502 and generate the output image 504.

[0055] After the process of interpolating the input image 502 by the cache-based warping engine, the output image 504 is obtained with zero distortion and uniform boundaries throughout. As can be seen in the figure, the output image 504 is obtained with the same coordinates as the input image 502. The distortion of the red region 502b and the green region 502c has been removed, resulting in an 8x8 pixel red region 504a and an 8x8 pixel green region 504b. The 8x8 red block 504a of the output image 504 is interpolated from the red region 502b of the input image 502, while the 8x8 green block 504b of the output image 504 is interpolated from the green region 502c of the input image 502.

[0056] Figure 6A A cache tag according to the present application is illustrated. The cache has two tags; a horizontal tag depending on the cache line size and a vertical tag depending on the number of cache lines. The tags 600a are kept to allow the cache to translate from a cache address (tag, index and offset) to a unique CPU address. A cache hit means that the CPU tries to access an address and a matching cache block (index, offset and matching tag) is available in the cache.

[0057] Each cache block is associated with at least a modified and valid bit, and a tag address. The valid bit indicates whether the cache block is in use (has valid data) or not. The modified bit only makes sense when the valid bit is set. The modified bit indicates whether the data in the cache block is different from the RAM (modified) or the same as the RAM. The tag is called the matching tag if, after decoding the CPU address from a pending read or write, the tag field of the CPU address matches the tag associated with the cache block at the cache address given by (index, offset).

[0058] If the cache is n-way set associative, then the cache address (index, offset) not only specifies one cache block, but a set of n cache blocks. When reading from or writing to the cache, if any cache block in the set has a matching tag, then this is a cache hit and the cache block is used. If no cache block in the set has a matching tag, then this is a cache miss and one cache block of the set is selected for replacement. The data from the cache block is written back to the RAM to make room for the new data, as in the case of a tag miss in a state transition.

[0059] Figure 6B A warped cache line access according to the present application is illustrated. The warp engine of the present application supports bicubic and bilinear interpolation techniques. In the worst case scenario, the bicubic interpolation technique requires accessing 8 cache lines to output 1 pixel.

[0060] In the worst case scenario for bilinear interpolation techniques, bilinear interpolation techniques require access to 4 cache lines to output 1 pixel. Statistically, the likelihood of accessing 8 cache lines for bicubic is approximately 2.4%, 4 cache line accesses will be good enough for 1 pixel output / clock.

[0061] Figure 6C A twisted cache bank for accessing cache lines is illustrated. As shown Figure 6B A cache bank for accessing 4 cache lines simultaneously is described. A memory cache bank prediction unit is provided for a processor having a plurality of memory cache banks. The memory cache bank prediction unit has an input port to receive an instruction. The memory cache bank prediction unit also has an evaluation unit coupled to the input port that predicts which of the plurality of memory cache banks is associated with the instruction.

[0062] The entire cache has 4 banks. In Figure 6B The bank organization for bilinear and bicubic interpolation is shown. On the left is the bank organization 602c for bilinear interpolation, while on the right is the bank organization 604c for the bicubic interpolation technique.

[0063] While various embodiments of the application have been described above, it should be understood that they have been presented by way of example only, and not limitation. Likewise, the illustrative drawings can depict example architectures or other configurations, which can be utilized in accordance with the present application, and that each of the illustrated features can be employed alone or in combination with other features in accordance with the application. The present application is not limited to the illustrative architectures or configurations described, but rather, can utilize alternative architectures and configurations in accordance with the principles of the present application.

[0064] While the present application has been described above in terms of various exemplary embodiments and implementations, it should be understood that the various features, aspects and functionality described herein can be implemented in various combinations of hardware and / or software, and that in some embodiments, various components can be implemented in software (including firmware, resident software, micro-code, etc.) and / or hardware. It should also be understood that the various features, aspects and functionality described herein can be implemented alone or in any combination of one or more

[0065] In some cases, the presence of expansible words and phrases such as "one or more," "at least," "but not limited to," or other similar phrase, should not be construed to imply that there are fewer or a narrower scope of possibilities for the described embodiments than appear in the patenting application.

Claims

1. A method for warping an input image using a cache-based warping engine, wherein, The method comprises: accessing the input image from a DDR SDRAM of a device; loading a portion of the input image to a cache memory of the device; pre-loading a lookup table stored in the cache memory, wherein the lookup table comprises a plurality of coordinates of the image; mapping the input image by bicubicly interpolating the plurality of coordinates to generate an interpolated image; computing an output image by raster scanning the interpolated image; and storing the output image in the DDR SDRAM of the device; wherein the warping operation comprises distortion correction and perspective correction of the input image; the lookup table utilizes the cache memory of the order of kilobytes in size; the warp engine is capable of supporting different formats for the input image; the size of the cache memory is configurable; an enhanced panoramic image is generated by warping a panoramic digital image having skewed boundaries to fit a predetermined shape using the warp engine; the cache memory comprises horizontal tags dependent on cache line size and vertical tags dependent on number of cache lines; during the interpolation process, image blocks in adjacent rows are processed and output vertically first and then in the same row horizontally.

2. The method of claim 1, wherein, The warp engine reduces DDR SDRAM access time and DDR SDRAM bandwidth requirements.

3. The method of claim 1, wherein, The plurality of coordinates of the output image comprises 2 bytes for X coordinates.

4. The method of claim 1, wherein, The plurality of coordinates of the output image comprises 2 bytes for Y coordinates.

5. The method of claim 1, wherein, The lookup table is an 8x8 interval lookup table.

6. The method of claim 1, wherein, The lookup table supports up to 4095x4095 image size.

7. The method of claim 1, wherein, The device can be any one of a tablet computer, a PC, a laptop computer, a mobile phone.

8. The method of claim 1, wherein, The cache memory comprises SRAM.

9. A system for warping an input image, wherein, The system comprises: an input device, wherein the input device comprises: a DDR SDRAM for storing the input image; a cache memory having a plurality of cache lines, wherein the plurality of cache lines access a portion of the image from the DDR SDRAM; a cache lookup table, wherein the cache lookup table comprises a plurality of coordinates of a portion of the input image; and a cache-based warp engine, wherein the cache-based warp engine comprises: a pre-loader, wherein the pre-loader loads a set of coordinates based on the plurality of coordinates of the input image to the cache lookup table; an image interpolator, wherein the image interpolator bicubicly interpolates the input image to generate an interpolated image; a processor for computing an output image by raster scanning the interpolated image; and a memory for storing the output image on the DDR SDRAM; wherein the warped input image includes distortion correction and perspective correction of the input image; the cache lookup table utilizes the cache memory of size in the order of kilobytes; the warp engine is capable of supporting different formats for the input image; the size of the cache memory is configurable; an enhanced panoramic image is generated by warping a panoramic digital image with skewed boundaries to fit a predetermined shape using the warp engine; the cache memory includes horizontal tags dependent on cache line size and vertical tags dependent on number of cache lines; in an interpolation process, image blocks in adjacent rows are processed and output vertically first and then image blocks in the same row are processed and output horizontally.

10. The system of claim 9, wherein, The cache memory includes SRAM.

11. The system of claim 9, wherein, The cache memory includes cache tags.

12. The system of claim 11, wherein, The cache-based warp engine follows a sequence of coordinates.

13. A computer usable medium having computer program logic for enabling at least one processor in a computer system to warp an image using a cache-based warp engine, the computer program logic comprising: accessing an input image from a DDR SDRAM of a device; loading a portion of the input image into a cache memory of the device; preloading a lookup table stored in the cache memory, wherein the lookup table includes a plurality of coordinates of the input image; mapping the input image by bicubicly interpolating the plurality of coordinates to generate an interpolated image; computing an output image by raster scanning the interpolated image; and storing the output image in the DDR SDRAM of the device; wherein the warped image includes distortion correction and perspective correction of the input image; the lookup table utilizes the cache memory of size in the order of kilobytes; the warp engine is capable of supporting different formats for the input image; the size of the cache memory is configurable; an enhanced panoramic image is generated by warping a panoramic digital image with skewed boundaries to fit a predetermined shape using the warp engine; the cache memory includes horizontal tags dependent on cache line size and vertical tags dependent on number of cache lines; in an interpolation process, image blocks in adjacent rows are processed and output vertically first and then image blocks in the same row are processed and output horizontally.

Images