Apparatus and method for processing a single instruction for image transformation from non-integer positions

Abstract

A processor pipeline circuit within a processor for non-integer transformation of an image using a single instruction is disclosed. The processor pipeline circuit includes a data fetch circuit configured to receive a memory address of an input image and fetch multiple pixels of the input image. The processor pipeline circuit further includes a weight access circuit configured to receive elements of an array of offsets and an interpolation type parameter. The weight access circuit is configured to determine weights to apply to the multiple pixels of the input image. The processor pipeline circuit further includes a multiply-accumulate circuit configured to calculate an output pixel of the transformed image by multiplying the multiple pixels of the input image by the weights and summing each resulting product.

Description

[Technical Field]

[0001] Priority application

[0001] This application claims priority to U.S. patent application Ser. No. 17 / 823,711, filed Aug. 31, 2022, entitled "APPARATUSES AND METHODS FOR PROCESSING SINGLE INSTRUCTION FOR IMAGE TRANSFORMATION FROM NON-INTEGRAL LOCATIONS," the entire contents of which are incorporated herein by reference. [Background technology]

[0002] I. Areas of Disclosure

[0002] The technology disclosed herein generally relates to processors configured to execute Single Instruction Multiple Data (SIMD) instructions, and more particularly to processors configured to execute SIMD instructions for transforming an image from its non-integer position.

[0003] II. Background

[0003] Images may need to be transformed for various reasons. Transformations within an image may include downscaling, upscaling, distortion reduction, or a combination of scaling and distortion reduction. In one example, transforming an image captured by a fisheye lens camera downscales and removes distortion. Fisheye lens cameras are widely used in security and autonomous driving applications. Fisheye lens cameras are useful for expanding the camera's field of view, but as a result, portions of the captured image are distorted. In order to identify specific objects in the image and, in some cases, make automated decisions, images captured from fisheye lens cameras need to have their distortion removed. When performing this transformation, the original image is typically downscaled, resulting in fewer pixels in the output image than in the input image. Therefore, there is no direct scalar mapping from one pixel in the input image to one pixel in the output image.

[0004] To determine an output pixel for an output image, the transformation process begins with a theoretical x,y coordinate location of the pixel in the input image on which the output is based. At least one of the x,y coordinate axes has floating-point values. In other words, the desired pixel of the input image to be transformed into a particular output image theoretically exists and is located at a non-integer coordinate in the input image. Because the actual input pixel is located at a scalar value of the x,y coordinate, the transformation process must select actual input pixels that are near the theoretical floating-point coordinate to approximate the theoretical pixel location. Specifically, each output pixel is determined by a combination of input pixels near or surrounding the floating-point location, as well as weights associated with those input pixels and applied to the combination of input pixels.

[0005] Traditionally, image transformations are performed by software code running on a Single Instruction Multiple Data (SIMD) processor. However, SIMD processor programmers code these non-integer transformation algorithms on a pixel-by-pixel basis. When developing code to perform this transformation, current programmers must utilize a series of base SIMD instructions for each pixel to determine how many pixels from the input image should be loaded into the processor pipeline, load the number of input pixels, load weights associated with the number of input pixels, determine how to calculate the output pixel based on some formula associated with the number of input pixels and the weights, and calculate the output pixel. As a result, current software code for this image processing is rather inefficient in terms of power consumption and throughput. Summary of the Invention

[0006] Aspects disclosed in the detailed description include an apparatus and method for processing a single instruction for image transformation from non-integer positions of an input image. The instruction includes a memory address of the input image, a memory address of the output image, an array of offsets associated with the output image, and an interpolation type parameter. Using a single instruction to perform image transformation from non-integer positions of the input image frees the programmer from having to decompose each operation, per output pixel, and use a series of base instructions to determine which and how many pixels of the input image should be loaded into the processor pipeline, determine how much weight to assign to each loaded pixel, load and store interim calculations, and calculate the output pixel on a selected selection of those. Additionally, processing a single instruction allows the processor pipeline to avoid loading and storing interim calculations, thereby saving power and increasing system throughput.

[0007] In this regard, one exemplary aspect provides an apparatus for processing a single instruction for non-integer transformation of an image. The apparatus includes a processor pipeline circuit configured to receive a single instruction including a memory address of an input image, a second memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter. The processor pipeline circuit is further configured to write output pixels of the transformed image. The processor pipeline circuit further includes a data fetch circuit configured to receive the memory address of the input image, the data fetch circuit further configured to fetch multiple pixels of the input image. The processor pipeline circuit further includes a weight access circuit configured to receive elements of the array of offsets and the interpolation type parameter, the weight access circuit configured to determine weights to apply to the multiple pixels of the input image. The processor pipeline circuit further includes a multiply-accumulate circuit configured to calculate an output pixel of the transformed image by multiplying the multiple pixels of the input image by the weights and summing each resulting product. The processor pipeline circuit further includes a data output circuit configured to receive a second memory address of the output image and the output pixels from the multiply-accumulate circuit, the data output circuit configured to write the output pixels of the transformed image to the memory.

[0008] In another exemplary aspect, a method for processing a single instruction for a non-integer transform of an image includes receiving a single instruction containing a memory address of an input image, a second memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter. The method further includes fetching a plurality of pixels of the input image from memory based on the memory address of the input image. The method further includes determining weights to apply to the plurality of pixels of the input image based on elements of the array of offsets and the interpolation type parameter. The method further includes calculating output pixels of the transformed image by multiplying the plurality of pixels of the input image by the weights and summing each resulting product, and writing the output pixels of the transformed image to memory.

[0009]

[0009] In another exemplary aspect, a non-transitory computer-readable storage medium storing instructions executable by a processor that, when executed by the processor, cause the processor to perform an image transform single instruction, multiple data (SIMD) operation. The non-transitory computer-readable storage medium comprises the SIMD instruction, the SIMD instruction specifying a memory address of an input image, a memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter. The non-transitory computer-readable storage medium further comprises: code for fetching a plurality of pixels of the input image based on the memory address of the input image; code for determining weights to apply to the plurality of pixels of the input image based on the elements of the array of offsets and the interpolation type parameter; code for calculating an output pixel of the transformed image by multiplying the plurality of pixels of the input image by the weights and summing each resulting product; and code for writing the output pixel of the transformed image to memory. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a high-level block diagram of an exemplary conversion from a 6×6 image to a 3×3 image. [Figure 2]

[0011] FIG. 2 is a block diagram of a processor pipeline circuit within a processor for converting one or more pixels of an input image into a single output pixel. [Figure 3]

[0012] FIG. 1 is a block diagram of multiple processor pipeline circuits within a processor(s) for transforming an input image into an output image in response to execution of a single instruction. [Figure 4]

[0013] 1 is a flowchart illustrating an exemplary process by which a processor transforms an input image into an output image in response to execution of a single instruction. [Figure 5]

[0014] FIG. 4 is a block diagram of an exemplary processor-based system that may include a processor including a processor pipeline circuit configured to convert one or more pixels of an input image into a single output pixel based on execution of a single instruction, including, but not limited to, the processor pipeline circuitry of FIGS. 1-3. DETAILED DESCRIPTION OF THE INVENTION

[0011]

[0015] Some exemplary aspects of the present disclosure will now be described with reference to the drawings. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

[0012]

[0016] Utilizing a single instruction to perform image transforms from non-integer positions in an input image frees the programmer from having to decompose each operation for each output pixel and utilize a series of multiple base instructions to determine which and how many pixels of the input image should be loaded into the processor pipeline, determine how much weight to assign to each loaded pixel, load and store interim values, and calculate the output pixel on a selected selection of those. Additionally, processing a single instruction allows the processor pipeline to avoid loading and storing interim values, thereby saving power and increasing system throughput.

[0013]

[0017] In this regard, FIG. 1 is a high-level block diagram 100 of an exemplary conversion of a 6×6 input image 102 to a 3×3 output image 104 by a processor pipeline circuit 106A receiving a single instruction 107, conveniently referred to as “TransformImage.” The processor pipeline circuit 106A receives the single instruction 107, which includes an array of offsets 108 and an interpolation type parameter 110, for transforming the input image 102 to the output image 104. The single instruction 107 also includes the address of the input image 102 and the address of the output image 104. The number of offsets in the array of offsets 108 is equal to the number of pixels in the output image 104. For the example shown in FIG. 1 , there are nine elements in the array of offsets 108. Each element of the array of offsets 108 is a pair of at least one floating-point number representing the x and y coordinates of a theoretical pixel in the input image 102 that would be transformed into an output pixel in the output image 104. For example, element 1 of array of offsets 108 contains coordinates (2.5, 4.7), which map to theoretical point 112 in input image 102. Because one of the 36 pixels in input image 102 is not directly at theoretical point 112, processor pipeline circuit 106A will use actual pixels in input image 102 near theoretical point 112 to generate pixel 1 in output image 104. Interpolation type parameter 110 will indicate the pixels in input image 102 needed to generate pixel 1 in output image 104. Interpolation type parameter 110 may indicate various interpolations, including bilinear 1×2 interpolation, bilinear 2×1 interpolation, bilinear 2×2 interpolation, bilinear 3×3 interpolation, or any other multilinear interpolation.

[0014]

[0018] In the example shown in FIG. 1 , the interpolation type parameter 110 is a bilinear 1×2 interpolation, which means that two pixels have the same y-axis but different x-axis and are near the theoretical point shown in the corresponding element in the array of offsets 108 and will be used in the transformation process. In FIG. 1 , pixel Pa and pixel Pb are fetched from the input image 102 and will be processed by the processor pipeline circuit 106A. Bilinear 2×1 interpolation means that two pixels with the same x-index but different y-indexes will be used from the input image 102. Bilinear 2×2 interpolation means that four pixels near the theoretical point will be used from the input image 102. Bilinear 3×3 interpolation means that nine pixels near the theoretical point will be used from the input image 102 for the transformation process. The interpolation type parameter 110 may include any of these types of bilinear interpolation, in addition to other types of interpolation. Optionally, each element in the array of offsets 108 may include a set of weights for each of the pixels in the input image 102 to be subsequently used in the transformation process performed by the processor pipeline circuit 106A.

[0015]

[0019] Additionally, processor pipeline circuit 106A will iterate through array of offsets 108 in order, processing the nearest pixels of input image 102 corresponding to coordinates contained within array of offsets 108 according to interpolation type parameter 110 to generate pixels 2-9, and write output image 104 to memory. This iteration will be performed in parallel by multiple processor pipeline circuits, as will be described with respect to FIG. 3.

[0016]

[0020] 2 is a block diagram of a processor pipeline circuit 106A in a processor 200 for converting one or more pixels of an input image 102 into a single output pixel. The processor pipeline circuit 106A includes a data fetch circuit 202, a weight access circuit 204, an optional multiplexer circuit 206, a multiply-accumulate circuit 208, and a data output circuit 210. The processor pipeline circuit 106A receives data and writes it to a memory 212.

[0017]

[0021] The data fetch circuit 202 receives the address of the input image and fetches pixels of the input image 102 from the memory 212. In one embodiment, the data fetch circuit 202 also receives the first element of the array of offsets 108 and retrieves a set of pixels of the input image 102 that encompass the non-integer coordinates represented in the first element of the array of offsets 108. For example, the data fetch circuit 202 may fetch nine pixels that encompass the non-integer coordinates. In this example, the retrieved pixels would be sent to a multiplexer circuit 206, which would receive an interpolation type parameter 110 as an input. The multiplexer circuit 206 would activate a subset of the n lines to the multiply-accumulate circuit 208 based on the interpolation type parameter 110. The activated subset of lines to the multiply-accumulate circuit 208 corresponds to pixels of the input image 102 that will be used for the transformation process in the multiply-accumulate circuit 208. For example, if the interpolation type indicated bilinear 1x2 interpolation, even if nine pixels are fetched, n=3 lines would be activated between the multiplexer circuit 206 and the multiply-accumulate circuit 208. In another example, if the interpolation type indicated bilinear 2x2 interpolation, n=4 lines would be activated. In another example, if the interpolation type indicated 3x3 interpolation, n=9 lines would be activated.

[0018]

[0022] In an alternative embodiment, the data fetch circuit 202 would receive the interpolation type parameter 110 and fetch the required amount of pixels from the memory 212 as specified by the interpolation type parameter 110. In this example, that would be three pixels. In this alternative embodiment, the multiplexer circuit 206 would not be needed. The activated subset of lines from the data fetch circuit 202 to the multiply-accumulate circuit 208 corresponds to pixels of the input image 102 that will be used for the transformation process in the multiply-accumulate circuit 208.

[0019]

[0023] Note that the examples are described to address grayscale images for simplicity and clarity. However, the present disclosure applies to multidimensional images, multi-channel images, or both. In the case of multidimensional images, such as tensors, pixel coordinates, including those held in the array of offsets 108, will have a z-axis to represent depth coordinates. In the case of multi-channel images, each pixel fetched and generated will have multiple channels. For example, a pixel in a red, green, blue (RGB) image will include three channels: a red channel, a green channel, and a blue channel.

[0020]

[0024] The weight access circuit 204 receives the first element of the array of offsets 108 and the interpolation type parameter 110. In one embodiment, the weight access circuit 204 calculates the weights. In this embodiment, when the interpolation type parameter 110 is bilinear 1×2 or 2×1 interpolation, the weight access circuit calculates the weights as follows:

[0025] Wa=Ceil(offset[1])-offset[1] (1)

[0026] Wb=offset[1]-Floor(offset[1]) (2)

[0021]

[0027] Wa is the weight of input pixel Pa, Wb is the weight of input pixel Pb, offset[1] is the first element in the array of offsets 108, Ceil(x) is a function that calculates the smallest integer greater than or equal to x, and Floor(x) is a function that calculates the largest integer less than or equal to x.

[0022]

[0028] If the interpolation type parameter 110 is 2x2 bilinear interpolation, the weight access circuit 204 calculates the weights as follows:

[0029] Wa=Wx * Wy

[0030] Wb=Wx * (1-Wy)

[0031] Wc=(1-Wx) * Wy

[0032] Wd=(1-Wx) * (1-Wy)

[0033] where

[0034] Wx=Ceil(offsetX[1]-offsetX[1])

[0035] Wy=Ceil(offsetY[1]-offsetY[1])

[0023]

[0036] Wa is the weight of input pixel Pa, Wb is the weight of input pixel Pb, Wc is the weight of input pixel Pc, and Wd is the weight of input pixel Pd. offsetX[1] is the x-axis coordinate of the first element in offset array 108, and offset[1] is the y-axis coordinate of the first element in offset array 108.

[0024]

[0037] If the interpolation type parameter 110 is 3x3 bilinear interpolation, there will be several different ways to weight the nine input pixels to generate one pixel. Any combination of W1...W9 such that Sum(W1)=1 and W1>=0 is a valid candidate for interpolation.

[0025]

[0038] In an alternative embodiment, the weights are predetermined and stored in memory 212. In this embodiment, weight access circuit 204 accesses a memory or register file to retrieve the weights.

[0026]

[0039] In all aspects, the weight access circuit 204 transfers to the sum-of-products circuit 208 the weights corresponding to the pixels received at the sum-of-products circuit 208 .

[0027]

[0040] The multiply-accumulate circuit 208 performs the following calculations to convert the pixels of the input image 102 into one output pixel, output[0], based on the fetched input pixels and the calculated weights in a bilinear 1x2 interpolation:

[0041] output[1]=(Pa * Wa)+(Pb * Wb) (3)

[0028]

[0042] As shown in equation (3), the output[1] pixel is calculated by multiplying multiple pixels of the input image by weights and summing each resulting product.

[0029]

[0043] FIG. 3 is a block diagram 300 of multiple processor pipeline circuits 106A-106I for transforming a 6×6 input image 102 into an output image 104 using a single instruction. The processor pipeline circuits 106A-106I may be deployed in a SIMD machine. Each processor pipeline circuit 106A-106I will receive a respective element of the array of offsets 108. For example, the processor pipeline circuit 106A will receive the first element of the array of offsets 108, offset[1], while the processor pipeline circuit 106I will receive the ninth element, offset[9]. The processor pipeline circuits 106B-106I will each retrieve pixels from the input image 102 according to their respective individual elements of the array of offsets 108 and perform the same operations in the same manner as described for the processor pipeline circuit 106A as described with respect to FIG. 2. The number of processor pipeline circuits is determined by the number of output pixels in the transformed image. Although the example describes and shows nine pixels for illustrative purposes, there could be hundreds of processor pipeline circuits deployed in a SIMD machine.

[0030]

[0044] 4 is a flowchart 400 illustrating an exemplary method for transforming an input image into an output image using a single instruction. At block 402, the method begins by receiving a single instruction containing a memory address of an input image, a memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter. At block 404, the method proceeds to fetch a plurality of pixels of the input image from memory based on the memory address of the input image. At block 406, the method proceeds to determine weights to apply to the plurality of pixels of the input image based on the array of offsets and the interpolation type parameter. At block 408, the method proceeds to calculate an output pixel of the transformed image by multiplying the plurality of pixels of the input image by the determined weights and summing each resulting product. At block 410, the method concludes by writing the output pixel of the transformed image to memory.

[0031]

[0045] A single instruction for image transformation from a non-integer position according to aspects disclosed herein may be provided within or integrated into any processor-based device, including, but not limited to, a set-top box, an entertainment unit, a navigation device, a communication device, a fixed position data unit, a mobile position data unit, a global positioning system (GPS) device, a mobile phone, a cellular phone, a smartphone, a session initiation protocol (SIP) phone, a tablet, a phablet, a server, a computer, a portable computer, a mobile computing device, a wearable computing device (e.g., a smart watch, a health or fitness tracker, eyewear, etc.), a desktop computer, a personal digital assistant (PDA), a monitor, a computer monitor, a television, a tuner, a radio, a satellite radio, a music player, a digital music player, a portable music player, a digital video player, a video player, a digital video disc (DVD) player, a portable digital video player, an automobile, and a vehicle component.

[0032]

[0046] In this regard, FIG. 5 illustrates an example of a processor-based system 500 that may employ a processor pipeline circuit 502, such as the processor pipeline circuits 106A-106I described with respect to FIGS. 1-3. In this example, the processor-based system 500 includes a processor 504 having one or more central processing units (CPUs) 506. The CPU(s) 506 may have a cache memory 516 coupled to the CPU(s) 506 for fast access to temporarily stored data. The CPU(s) 506 are coupled to a system bus 510, which may interconnect devices included within the processor-based system 500. As is well known, the CPU(s) 506 communicate with these other devices by exchanging address, control, and data information via the system bus 510. For example, the CPU(s) 506 may communicate bus transaction requests to a memory controller 512. Although not shown in FIG. 5, multiple system buses 510 may be provided, with each system bus 510 constituting a different fabric.

[0033]

[0047] Other devices may also be connected to the system bus 510. As shown in FIG. 5, these devices may include, by way of example, a memory system 514, one or more input devices 518, one or more output devices 520, one or more network interface devices 522, and one or more display controllers 524. The input device(s) 518 may include any type of input device, including, but not limited to, input keys, switches, audio processors, etc. The output device(s) 520 may include any type of output device, including, but not limited to, audio, video, other visual indicators, etc. The network interface device(s) 522 may be any device configured to enable the exchange of data with a network 526. The network 526 can be any type of network, including, but not limited to, a wired or wireless network, a private or public network, a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a BLUETOOTH™ network, and the Internet. The network interface device(s) 522 can be configured to support any type of communication protocol desired. The memory system 514 can include one or more memory arrays 516.

[0034]

[0048] The CPU(s) 506 may also be configured to access display controller(s) 524 via the system bus 510 to control information sent to one or more displays 528. The display controller(s) 524 send information to be displayed to the display(s) 528 via one or more video processors 530, which process the displayed information into a format suitable for the display(s) 528. The display(s) 528 may include any type of display, including, but not limited to, a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, a light emitting diode (LED) display, etc.

[0035]

[0049] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithms described in connection with aspects disclosed herein may be implemented as electronic hardware, as instructions stored in a memory or another computer-readable medium, any such instructions executed by a processor or other processing device, or a combination of both. The devices described herein may be employed in, by way of example, any circuit, hardware component, integrated circuit (IC), or IC chip. The memories disclosed herein may be of any type and size and may be configured to store any type of information desired. To clearly illustrate this interchangeability, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. How such functionality is implemented depends on the specific application, design choices, and / or design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0036]

[0050] The various illustrative logic blocks, modules, and circuits described in connection with aspects disclosed herein may be implemented or performed using a processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. A processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration).

[0037]

[0051] Aspects disclosed herein may be embodied in hardware or in instructions stored in the hardware and that may reside in, for example, a Random Access Memory (RAM), a flash memory, a Read Only Memory (ROM), an Electrically Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a register, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a remote station. Alternatively, the processor and the storage medium may reside as discrete components in a remote station, a base station, or a server.

[0038]

[0052] It should also be noted that the operational steps described in any of the exemplary aspects herein are described to provide examples and explanations. The described operations may be performed in many different sequences other than the sequence shown. Furthermore, an operation described in a single operational step may actually be performed in several different steps. Furthermore, one or more operational steps described in the exemplary aspects may be combined. It should be understood that numerous different modifications may be made to the operational steps depicted in the flowchart diagrams, as would be readily apparent to one of ordinary skill in the art. Those skilled in the art will also appreciate that information and signals may be represented using any of a wide variety of technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0039]

[0053] The above description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other variations. Thus, the disclosure is not intended to be limited to the embodiments and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0040]

[0054] Example implementations are described in the following numbered aspects / clauses. 1. An apparatus for non-integer transformation of an image, comprising: a processor pipeline circuit configured to receive a single instruction comprising a memory address of an input image, a second memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter, the processor pipeline circuit configured to write output pixels of a transformed image, the processor pipeline circuit comprising: a data fetching circuit configured to receive a memory address of an input image, the data fetching circuit further configured to fetch a plurality of pixels of the input image; a weight access circuit configured to receive an element of the array of offsets and an interpolation type parameter, the weight access circuit configured to determine weights to be applied to a plurality of pixels of the input image; a multiply-accumulate circuit configured to calculate an output pixel of the transformed image by multiplying a plurality of pixels of the input image by the weights and summing each resulting product; a data output circuit configured to receive a second memory address of the output image and the output pixels from the multiply-accumulate circuit, the data output circuit configured to write the output pixels of the transformed image to the memory; 1. An apparatus comprising a processor pipeline circuit, 2. The apparatus of clause 1, wherein the array of offsets includes x,y coordinates with at least one axis having floating-point values. 3. The apparatus of clause 1 or 2, wherein the weight access circuit calculates the weights based on elements of the array of offsets and an interpolation type parameter. 4. The apparatus of clause 1 or 2, wherein the weight access circuit determines the weights by retrieving the weights from memory based on elements of an array of offsets. 5. The apparatus of clause 1, 2, 3, or 4, wherein the output pixel includes three channels, the three channels including a red channel, a green channel, and a blue channel. 6. The apparatus of clause 1, further comprising a multiplexer circuit coupled to the data fetch circuit with n lines, the multiplexer circuit configured to receive an interpolation type parameter and select an active subset of the n lines for transmitting a plurality of pixels of the input image to the multiply-accumulate circuit, each active line representing one pixel of the plurality of pixels of the input image. 7. The apparatus of clause 1, 2, 3, 4, 5, or 6, wherein the interpolation type parameter is bilinear 1x2 interpolation, bilinear 2x1 interpolation, bilinear 2x2 interpolation, bilinear 3x3 interpolation, or multilinear interpolation. 8. The apparatus of clause 6, wherein the interpolation type parameter is bilinear 1x2 interpolation and the active subset of n lines is 2 lines. 9. The apparatus of clause 6, wherein the interpolation type parameter is bilinear 2x2 interpolation and the active subset of n lines is 4 lines. 10. The apparatus of clause 6, wherein the interpolation type parameter is bilinear 3x3 interpolation and the active subset of n lines is 9 lines. 11. A method for non-integer transformation of an image, comprising: receiving a single instruction comprising a memory address of an input image, a second memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter; fetching a plurality of pixels of the input image from a memory based on a memory address of the input image; determining weights to be applied to a plurality of pixels of the input image based on elements of the array of offsets and the interpolation type parameter; calculating an output pixel of the transformed image by multiplying a plurality of pixels of the input image by weights and summing each resulting product; writing output pixels of the transformed image to a second memory address; A method comprising: 12. The method of clause 11, wherein the array of offsets includes x,y coordinates with at least one axis having floating point values. 13. The method of clause 11 or 12, wherein the interpolation type parameter is bilinear 1x2 interpolation, bilinear 2x1 interpolation, bilinear 2x2 interpolation, bilinear 3x3 interpolation, or multilinear interpolation. 14. The method of clause 11 or 12, wherein the interpolation type parameter is bilinear 1x2 interpolation. 15. The method of clause 11 or 12, wherein the interpolation type parameter is bilinear 2x2 interpolation. 16. A non-transitory computer-readable storage medium comprising processor-executable instructions that, when executed by the processor, cause the processor to perform an image transform utilizing single instruction multiple data (SIMD) operations, the non-transitory computer-readable storage medium comprising: A SIMD instruction, The memory address of the input image, a second memory address of the output image; an array of offsets associated with the output image; an interpolation type parameter; code for fetching a plurality of pixels of an input image based on a memory address of the input image; code for determining weights to be applied to pixels of the input image based on elements of the array of offsets and an interpolation type parameter; code for calculating an output pixel of the transformed image by multiplying a plurality of pixels of the input image by weights and summing each resulting product; and code for writing the output pixels of the transformed image to a second memory address. 1. A non-transitory computer-readable storage medium comprising: 17. The non-transitory computer-readable storage medium of clause 16, wherein the interpolation type parameter is bilinear 1x2 interpolation, bilinear 2x1 interpolation, bilinear 2x2 interpolation, bilinear 3x3 interpolation, or multilinear interpolation. 18. The non-transitory computer-readable storage medium of clause 16, wherein the interpolation type parameter is bilinear 1x2 interpolation. 19. The non-transitory computer-readable storage medium of clause 16, wherein the interpolation type parameter is bilinear 2x2 interpolation.

Claims

1. 1. An apparatus for non-integer transformation of an image, comprising: a processor pipeline circuit configured to receive a single instruction comprising a memory address of an input image, a second memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter, the processor pipeline circuit configured to write output pixels of a transformed image, the processor pipeline circuit comprising: a data fetching circuit configured to receive the memory address of the input image, the data fetching circuit further configured to fetch a plurality of pixels of the input image; a weight access circuit configured to receive elements of the array of offsets and the interpolation type parameter, the weight access circuit configured to determine weights to be applied to the plurality of pixels of the input image; a product-sum circuit configured to calculate the output pixel of the transformed image by multiplying the pixels of the input image by the weights and summing each resulting product; a data output circuit configured to receive the second memory address of the output image and the output pixels from the multiply-accumulate circuit, the data output circuit configured to write the output pixels of the transformed image to a memory; 1. An apparatus comprising a processor pipeline circuit,

2. 2. The apparatus of claim 1, wherein the array of offsets comprises x,y coordinates with at least one axis having floating point values.

3. The apparatus of claim 1 , wherein the weight access circuitry calculates the weights based on the elements of the array of offsets and the interpolation type parameter.

4. 2. The apparatus of claim 1, wherein the weight access circuitry determines the weights by retrieving the weights from a memory based on the elements of the array of offsets.

5. 10. The apparatus of claim 1, wherein the output pixel comprises three channels, the three channels comprising a red channel, a green channel, and a blue channel.

6. 2. The apparatus of claim 1, further comprising: a multiplexer circuit coupled to the data fetch circuit with n lines, the multiplexer circuit configured to receive the interpolation type parameter and to select an active subset of the n lines for transmitting the plurality of pixels of the input image to the multiply-accumulate circuit, each active line representing one pixel of the plurality of pixels of the input image.

7. The apparatus of claim 1 , wherein the interpolation type parameter is bilinear 1×2 interpolation, bilinear 2×1 interpolation, bilinear 2×2 interpolation, bilinear 3×3 interpolation, or multilinear interpolation.

8. 7. The apparatus of claim 6, wherein the interpolation type parameter is bilinear 1x2 interpolation and the active subset of the n lines is 2 lines.

9. 7. The apparatus of claim 6, wherein the interpolation type parameter is bilinear 2x2 interpolation and the active subset of n lines is four lines.

10. 7. The apparatus of claim 6, wherein the interpolation type parameter is bilinear 3x3 interpolation and the active subset of n lines is 9 lines.

11. 1. A method for non-integer transformation of an image, comprising: receiving a single instruction containing a memory address of an input image, a second memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter; fetching a plurality of pixels of the input image from memory based on the memory address of the input image; determining weights to be applied to the pixels of the input image based on elements of the array of offsets and the interpolation type parameter; calculating an output pixel of a transformed image by multiplying the pixels of the input image by the weights and summing each resulting product; writing the output pixels of the transformed image to the second memory addresses; A method comprising:

12. The method of claim 11 , wherein the array of offsets comprises x,y coordinates with at least one axis having floating point values.

13. The method of claim 11 , wherein the interpolation type parameter is bilinear 1×2 interpolation, bilinear 2×1 interpolation, bilinear 2×2 interpolation, bilinear 3×3 interpolation, or multilinear interpolation.

14. The method of claim 11 , wherein the interpolation type parameter is bilinear 1×2 interpolation.

15. The method of claim 11 , wherein the interpolation type parameter is bilinear 2×2 interpolation.

16. 1. A non-transitory computer-readable storage medium comprising processor-executable instructions that, when executed by the processor, cause the processor to perform an image transform utilizing single instruction multiple data (SIMD) operations, the non-transitory computer-readable storage medium comprising: SIMD instructions, The memory address of the input image, a second memory address of the output image; an array of offsets associated with the output image; an interpolation type parameter; code for fetching a plurality of pixels of the input image based on the memory address of the input image; code for determining weights to be applied to the pixels of the input image based on elements of the array of offsets and the interpolation type parameter; code for calculating an output pixel of a transformed image by multiplying the pixels of the input image by the weights and summing each resulting product; and code for writing the output pixels of the transformed image to the second memory address.

1. A non-transitory computer-readable storage medium comprising:

17. 17. The non-transitory computer-readable storage medium of claim 16, wherein the interpolation type parameter is bilinear 1x2 interpolation, bilinear 2x1 interpolation, bilinear 2x2 interpolation, bilinear 3x3 interpolation, or multilinear interpolation.

18. 17. The non-transitory computer-readable storage medium of claim 16, wherein the interpolation type parameter is bilinear 1x2 interpolation.

19. 17. The non-transitory computer-readable storage medium of claim 16, wherein the interpolation type parameter is bilinear 2x2 interpolation.

Images