Image scaling

Abstract

A video processor comprising: an input buffer for receiving an input image, wherein the input image is a single frame of a video stream compatible with a high-resolution multimedia interface (HDMI); a cutting circuit for dividing the input image into multiple N vertical slices; N parallel image scalers, each scaler being configured in hardware to scale one of the N vertical slices line by line according to an image scaling algorithm; and an output multiplexer for combining the scaled vertical slices into a combined scaled output image.

Description

Technical field of disclosure

[0001] This disclosure relates generally to the field of data processing and in particular, but not exclusively, to a system and method for video processing. background

[0002] Modern video systems can receive and reproduce high-quality digital video signals via interfaces such as the high-resolution multimedia interface (HDMI), DisplayPort, the universal serial bus type C (USB-C) and the digital video interface (DVI), to name just a few non-limiting examples. List of characters

[0003] The present disclosure is best understood from the following detailed description in conjunction with the accompanying figures. It is emphasized that, in accordance with standard industry practice, various features are not necessarily drawn to scale and are used for illustrative purposes only. Where a scale is shown explicitly or implicitly, it provides only an illustrative example. In other embodiments, the dimensions of the various features may be arbitrarily enlarged or reduced for the sake of simplicity. Fig. 1a and Fig. Figure 1b shows representations of different embodiments of a video system according to the teachings of the present description. Fig. Figure 2 is a representation of the vertical image division according to the principles of the present description. Fig.Figure 3 is a block diagram representation of how image scaling is performed according to the principles outlined in this description. Fig. Figure 4 is a block diagram that represents a scaling operation according to the teachings of the present description. Fig. Figure 5 is a block diagram of an illustrative scaler according to the teachings of the present description. Fig. Figure 6 is a block diagram illustrating pipeline formation according to the principles of this description. Fig. Figure 7 is a flowchart of a procedure for upscaling video according to the teachings of the present description. Fig. Figure 8 is a block diagram of an HDMI system according to the teachings of the present description. Fig. 9a and Fig. Figure 9b is a block diagram of a single-chip system according to the teachings of the present description. Summary

[0004] In one example, a video processor is disclosed comprising: an input buffer for receiving an input image; a slicer circuit for dividing the input image into several N vertical slices; N parallel image scalers, each scaler being configured in hardware to scale one of the N vertical slices line by line according to an image scaling algorithm; and an output multiplexer for combining the scaled vertical slices into a combined scaled output image. Implementations of the revelation

[0005] The following disclosure provides many different embodiments or examples for implementing various features of the present disclosure. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, only examples and are not intended to be limiting. Furthermore, the present disclosure may repeat reference numerals and / or letters in the various examples or, in some cases, in different figures. This repetition serves the purpose of simplicity and clarity and does not, in itself, prescribe a specific relationship between the various embodiments and / or configurations discussed. Different embodiments may have different advantages, and no particular advantage is necessarily required for any of the embodiments.

[0006] Many contemporary display systems surpass their data sources in terms of display resolution, color depth, and frame rate. Newer televisions, for example, are capable of ultra-high-definition 4K and 8K display (4K and 8K UHD display). These display technologies are able to test the limits of human vision. However, these displays are often driven by previous-generation display sources that cannot fully utilize these capabilities. For example, many consumers own a first-generation Blu-ray Disc player. Most common Blu-ray Disc players display video in high definition (HD) (e.g., 720p) or so-called Full HD (1080p). The "p" in 720p and 1080p refers to an HD signal format with progressive scanning (or "non-interlaced").720p HD displays 1280 pixels horizontally across the screen and 720 horizontal pixels vertically down the screen, resulting in 720 horizontal lines with a 16:9 aspect ratio. 1080p Full HD displays 1920 pixels horizontally across the screen and 1080 horizontal pixels vertically down the screen, resulting in 1080 horizontal lines with a 16:9 aspect ratio. Compared to the informal but commonly used designations 720i and 1080i, where the "i" refers to an interlaced HD signal format, 720p and 1080p reduce the need to prevent flicker by transmitting all lines, rather than only alternating lines, in a single frame.

[0007] Although Blu-ray Disc players can deliver 720p HD and 1080p Full HD, they can also play older digital video discs (DVDs). Many users still have a large collection of existing DVDs that they don't want to replace with Blu-ray Discs. DVDs are even more limited, with a maximum resolution of 480p (720 × 480 pixels).

[0008] Compare these resolutions to a 4K UHD TV, which offers a resolution of 3840 × 2160, or an 8K UHD TV, which offers a resolution of 7680 × 4320.

[0009] These resolutions are so high that a standard Blu-ray disc lacks the data capacity to store 4K or 8K video. Therefore, in the current situation, 4K and 8K are generally considered video streaming technologies. However, modern consumers expect videos to stream in real time and play back on their TVs with little to no buffering. 4K and 8K videos may not stream successfully if the user does not have a very high-quality internet connection, such as a fiber optic connection. Consumers with 4K and 8K UHD-capable TVs thus have a high-performance display but are faced with a large number of video sources that cannot drive this display at its full resolution.

[0010] Many existing televisions are capable of supporting video streams from older, lower-resolution devices such as 480p DVDs or 720p and 1080p Blu-ray Disc players. However, consumers may be dissatisfied with watching grainy, low-resolution videos on a high-definition television for which they paid a premium price.

[0011] To provide consumers with a premium viewing experience, many televisions now feature video upscaling capabilities. For example, these televisions might receive a 1080p input stream and upscale it to 4K UHD resolution. However, as the resolution increases to 8K and beyond, such upscaling can exceed the capabilities of the types of microprocessors, digital signal processors (DSPs), or hardware such as application-specific integrated circuits (ASICs) used to perform the upscaling.

[0012] For example, processing 8K UHD video at 60 frames per second requires processing approximately 2.4 gigapixels per second. However, common DSPs used for such processing operate at around 700 megapixels per second. Designing even larger or faster DSPs, or even larger or faster conversion hardware, is a non-trivial task.

[0013] The present description discloses a system and a method for providing upscaling or downscaling of a video to an HD format by dividing the video stream into multiple N vertical slices and processing the N vertical slices in parallel on N parallel scaler hardware elements or DSPs. Downscaling may be supported if the source content has a high resolution (e.g., 8K) and the source does not have a downscaling function, but the display only supports a lower resolution (e.g., 1080p or 4K). In this case, a downscaler may need to be placed between the source and the display.

[0014] This description discusses examples relating to a high-resolution multimedia interface receiver (HDMI receiver) that upscales video to 8K using four parallel scalers. All these examples are to be understood as non-limiting. Other embodiments may employ different video standards and other intermediate connections. For example, video could be transmitted via a digital transmission system such as the internet or via other interfaces such as the Universal Serial Bus (USB), USB-C, DisplayPort, or the like. Furthermore, the teaching of video upscaling in this description should not be understood as being an exclusive function of any video signal processor described herein.A video signal processor can offer many types of video signal processing, including other services such as color space conversion, cropping, interlacing, and frame rate conversion, as an illustrative and non-limiting example.

[0015] Embodiments of the present description may include an ASIC or a system-on-a-chip (SoC) with the N parallel scalers taught herein, as well as other elements such as derasterization buffers, rerasterization buffers, HDMI controllers, double data rate (DDR) controllers, physical and logical interfaces, and other suitable elements. The scalers discussed herein may, in some examples, be provided as blocks of intellectual property (IP blocks). An IP block containing a scaler may contain only the scaler or support elements specifically designed to adapt the scaler for use in the teachings of this description. For example, an IP block may contain not only the scaler but also support elements such as a pixel converter, a derasterization buffer, a rerasterization buffer, and / or an output multiplexer.In some cases, the N scalers can be provided monolithically as a parallel scaling unit. In this case, the scaler can be provided as an IP block, a separate ASIC, or part of an integrated circuit. Although custom-designed circuits are used as an example of a scaler in this description, scalers of the type described here can also be built using DSPs, graphics processing units (GPUs), central processing units (CPUs), or other suitable processing elements.

[0016] The parallel scaler described here, as a non-limiting and illustrative example, features four independent scalers operating at approximately 600 to 700 MHz. These four independent scalers together provide a total image processing performance of 2.4 gigapixels per second, enabling the real-time upscaling of videos, including 480p, 720p, 1080p, and 4K, up to 8K. Furthermore, a system as described here, in conjunction with a cooperative interlacing unit, can also upscale other video formats such as 480i, 720i, and 1080i.

[0017] An illustrative embodiment of the scaler described herein provides four parallel scalers, each offering essentially the same image scaling algorithm. The image scaling algorithm can be a polynomial interpolation algorithm that provides high-quality image scaling by interpolating a curve based on four points of input data, rather than simply calculating a linear approximation between two points. Each point of input data can be a single pixel. Thus, the scaler can take a row of input pixels and divide that row into four parallel pieces, each containing approximately 2000 pixels.These 2000-pixel chunks are then processed sequentially by the four parallel scalers, with each scaler receiving a block from the next row after outputting its interpolated pixels to a FIFO buffer (first-in, first-out). The output images can be multiplexed, rerasterized, and then displayed at full 8K UHD resolution on a video display.

[0018] To ensure that the FIFO buffers for each scaler do not run out, the image scalers can be arranged in a pipeline. All disk scalers operate simultaneously in parallel. At the beginning of the image, they start with quarter-line offsets, assuming the input is in a raster format. This quarter-line offset delay serves to optimize the derasterization and rerasterization buffer sizes more closely. If, however, the input were stored as a file in a file system, all disk scalers could start together at the beginning of the single frame. For real-time operation, however, the rerasterization buffer size is better if the scalers start with quarter-line offsets.

[0019] If the image is sliced ​​into four vertical slices and each slice is processed independently by its own scaler, there is a risk that the image will be displayed as four discrete images with a line between the slices. This can be considered a suboptimal viewing experience from the end user's perspective. In many cases, it is preferable to provide a continuous image that looks as if it were originally encoded as a single 8K UHD image. To achieve this, each scaler can receive as its first input for a given quartile on a given line one or more preceding pixels from the previous quartile. For example, if bilinear interpolation is used, the scaler would 2 The first input pixel is the last pixel of the quartile. 1received. Thus, the scaler would be able to interpolate a weighted average based on the distance of a new pixel from the two surrounding pixels. For a better display experience, a polynomial interpolation, such as bicubic interpolation, can be used instead of bilinear interpolation. Bicubic interpolation requires four pixels as input and calculates a weighted average from them, instead of a simple linear weighted average among two pixels. Because bicubic interpolation requires four input pixels, each quartile can receive the last two pixels of the previous quartile as its first two pixels. Similarly, each quartile can 2Pixels from the next quartile are received to calculate the scaled output pixels at the end of the quartile. This gives the scaler four pixels with which to perform a bicubic interpolation. If other interpolation algorithms are used, a different number of previous pixels from the preceding quartile can be provided to ensure a smooth transition between vertical slices.

[0020] Once the scalers have successfully calculated a complete output line, this output line can be rerasterized and then displayed. To provide a satisfactory user experience, the scalers can be trained to process approximately 60 frames per second, ensuring that the display can be driven at the desired 60 frames per second.

[0021] A system and method for providing image scaling are now described with particular reference to the accompanying figures. It should be noted that certain reference numerals may be repeated across the figures to indicate that a particular device or block is wholly or substantially consistent throughout. This is not intended to imply any special relationship between the various disclosed embodiments. In certain examples, reference may be made to a genus of elements by a specific reference numeral (“Object 10”), while reference may be made to individual species or examples of the genus by a hyphenated reference numeral (“first specific Object 10-1” and “second specific Object 10-2”).

[0022] Fig. 1a and Fig. 1b are representations of different embodiments of a video system 100according to the teachings of the present description.

[0023] In the Fig. The embodiment shown in 1a provides an HDMI source. 104 a signal. The HDMI source 104 This could be, for example, a VHS player, a DVD player, a Blu-ray disc player, or a digital media content delivery system such as online video or video stored on local storage. In this example, the HDMI source provides... 104 The output video has a lower resolution such as 480p or HD with 720p or 1080p. The HDMI source 104 delivers its output video signal to the HDMI sink 108 .

[0024] The HDMI sink 108 It receives the incoming video at its native resolution, for example, 480p, 720p, 1080p, or another resolution. The HDMI sink 108 However, it is trained to communicate with a display such as a television set. 116to connect. The television 116 is at a higher resolution than the native resolution of the HDMI source 104 capable. For example, the television set 116 be capable of resolutions such as 4K or 8K UHD. This is necessary for a user or viewer of the television. 116 To provide a first-class video experience, a video processor can 112 between the HDMI sink 108 and the television 116 be arranged. The video processor 112 may be able to output Ultra HD using the full resolution capacity of the television. 116 to deliver. In particular, the HDMI processor can 112 be trained to receive a lower resolution HDMI signal from the HDMI sink 108 to receive, upscale this signal to a higher resolution such as 4K or 8K UHD, and send the upscaled image to the television. 116 to transmit. The television set 116The video processor then receives the UHD HDMI output. 112 and displays the video on his screen.

[0025] It should be noted that the video system 100-1 The physical relationship between the various elements is not necessarily specified. In this embodiment, the HDMI source 104 , the HDMI sink 108 , the HDMI processor 112 and the television 116 shown in a block diagram configuration representing separate logical functions. However, it should be noted that these elements can be arranged in any suitable configuration. For example, the HDMI source 104 , the HDMI sink 108 , the HDMI processor 112 and the television 116 all of them could be a physically packaged unit. In particular, the television set could be one example. 116 have the ability to output digital video from an HDMI source 104to receive the video at an HDMI sink 108 to forward the video to the video processor 112 to process and display the video on a television screen 116 to display. Alternatively, the HDMI source could be used. 104 a completely separate device, such as a video player or a network connection, and the HDMI sink 108 It could also be provided as a separate HDMI receiver for the system. The video processor 112 could function as a separate logical device or as a standalone accessory that acts as an HDMI sink 108 provided for video upscaling functionality.

[0026] The video processor 112 Its upscaled output signal could then be sent to a UHD-capable display such as a television. 116This shows only two of many possible embodiments, and in general any logical grouping or physical configuration and arrangement could be used.

[0027] Fig. 1b is a block diagram of the video system 100-2 , in which certain elements are arranged in a system-on-a-chip (SoC) 150. The SoC 150 can be a monolithic device based on a silicon substrate or another semiconductor substrate, with logic elements as well as electrical and electronic elements provided therein via a manufacturing process such as silicon wafer fabrication. In this embodiment, the video signal processor can 124 analogous to the HDMI processor 112 from Fig. 1a. The serializer / deserializer receiver (SERDES receiver) 120 can be analogous to the HDMI sink 108 from Fig. It should be top-notch. The SERDES transmitter 128can be part of the video processor 112 from Fig. 1a or similarly considered.

[0028] In this embodiment, the SoC 150 is equipped with a dynamic direct access memory (DRAM) for communication purposes. 132 coupled. The DRAM 132 Provides operating memory for the SoC 150. The SERDES receiver 120 The SERDES receiver can be any suitable type of SERDES and can receive signals in one or more formats, including, for example, HDMI, Direct Media Interface (DMI), USB-C, Ethernet (with embedded video in Ethernet packets), or other suitable video transmission mechanisms. 120 delivers its video signal to the video signal processor 124 .

[0029] The video signal processor 124It can offer any number of video signal processing functions for its input video. These can include, in an illustrative and non-limiting example, color space conversion (or "gamut" conversion, where the number of displayed colors can be increased or decreased), image cropping (e.g., from 16×9 to 4×3 or other cropping), interlaced video decoding, scaling (including upscaling, e.g., from a format such as 480p, 480i, 720p, 1080p, or the like to 4K or 8K UHD), and frame rate conversion (e.g., interpolating and adding or removing frames to convert between formats such as 24 frames per second and 60 frames per second).

[0030] It should be noted that the various functions of the video signal processor 124These functions can be executed by various assigned or cooperative logic blocks. In some cases, these functions can be inserted into a pipeline, so that, for example, an image is cropped or unbundled before scaling is performed. In some embodiments, the various elements of the video signal processor can be 124 This can be provided by an IP block that can act as a "black box" with defined inputs, defined outputs, and a defined processing function that is executed on the inputs to generate the outputs. With such IP blocks, system developers can advantageously sell these functions as standalone units.

[0031] Once the video signal processor 124 Once the conversion and processing of his video is complete, he delivers the original video to the SERDES transmitter. 128, which sends the output signal to a display such as the UHD-capable display 116 from Fig. 1a, outputs to a computer monitor or commercial display (e.g. a jumbotron or large conference room display) or other display technology.

[0032] Fig. Figure 2 is a representation of the vertical image division according to the principles of the present description.

[0033] Fig. 2 shows an original single image 204 , which can be a single frame extracted from a video stream. For example, the original single frame 204This could be a single video frame displayed on a screen, for example, for 1 / 24 or 1 / 60 of a second. It should be noted that HDMI uses video compression, such as Display Stream Compression (DSC). Because HDMI uses DSC and other video formats offer compression, individual frames may not initially be available as monolithic images. Rather, compression technologies often use a delta between the previous frame and the current frame, modifying only the pixels that differ between frames. However, some embodiments of the present description gain advantages by processing the video in an uncompressed, frame-by-frame format. To preserve the original single frame 204 To obtain the full frame from the video stream, a DSC function may therefore be required to construct the full frame from the source data. Once the single frame is 204If it is properly constructed, it can be processed according to the principles outlined in this description.

[0034] As an illustrative example, a scaler can be used to resize the original single image. 204 to upscale to 8K resolution based on a 7680x4320 pixel bit map with 48-bit color depth. It should be noted that 8K video with 48-bit color depth is given here as a non-limiting example, and other video resolutions can be used. In some embodiments, a method for converting from a lower color depth, such as 24-bit color depth, by generating random bits on the LSB side of each color component in a pixel through a process typically referred to as up-dithering may also be provided.

[0035] It is important to note that dithering and upscaling can be separate processes or performed together. Upscaling can take the form of linear interpolation, where a point is interpolated between two other points, or where a pixel is interpolated between two other pixels and the new pixel is given a color based on a linear weighted average of the colors of the two nearest pixels. Other implementations may use more complex algorithms to provide better interpolation and thus better image scaling. For example, a scaler might use an interpolation algorithm where the newly sampled pixel is interpolated from a number of neighboring pixels by weighting them with a suitable impulse response, which is determined by the type of kernel used for scaling.For example, in an interpolation method like bicubic interpolation, four pixels are used to interpolate the value of the new pixel. The new value is interpolated from two pixels in each dimension, horizontally and vertically (i.e., two pixels horizontally on each side of the new pixel, and two above and two below vertically). This type of interpolation results in both a better position and a better color for the new pixel.

[0036] At a given process node (such as 28 nm), existing scalers typically reach a maximum of approximately 650 to 700 MHz, providing 650 to 700 megasamples per second for interpolating a new image. This may be sufficient for interpolating, for example, 1080p Full HD to 4K UHD. However, 700 megasamples per second is insufficient for interpolation if the scaler's input or output rate exceeds 4K at 60 Hz, such as 8K at 60 Hz. Providing very high resolutions like 8K UHD upscaling may require additional processing power.

[0037] Since designing a larger and faster scaler IP block is not trivial, the present description truncates the original single image. 204 vertically into the cut single image 212 The cut single image 212 contains a number of vertical discs 208, namely disc 208-1 , disc 208-2 , disc 208-3 and disc 208-4 Using 8K resolution as an example, the cropped single frame shows 206 4320 Rows, with each row subdivided into quartiles of slightly less than 2000 pixels. Each quartile or slice 208 can be assigned to a dedicated scaler in a parallel scaling architecture, so that, for example, a disk 208- 1 is assigned to a first scaler, disk 208-2 is assigned to a second scaler, disk 208-3 is assigned to a third scaler and disk 208-4 is assigned to a fourth scaler. These scalers can interpolate pixels row by row, as indicated by the row. 212 The image shows a row containing 7680 pixels. This row is divided into four quartiles or slices. 208-1 , disc 208-2 , disc 208-3 and disc 208-4subdivided. If a first scaler pixels for the disc 208-1 Interpolation allows the scaler to use an interpolation algorithm in each operation to interpolate a new pixel from four input pixels in each dimension (i.e., vertically and horizontally). This output pixel is then presented as a new pixel positioned between the two preceding and the two following pixels. The scaler then increments by one pixel, takes a new set of pixels, and interpolates one or more new pixels from the center of these four pixels.

[0038] However, it should be noted that when the discs 208 They are treated as completely independent units, the output looks like four related but distinct images with visible lines between the four disks. Therefore, the disks receive 208-2 , 208-3 and 208-4Each disk uses the last three pixels of the previous disk as its first three pixels, allowing the scalers working on these three disks to interpolate the starting boundary pixels between the disks. Similarly, the disks receive... 208-1 , 208-2 and 208-3 The first three pixels of each subsequent disk are used as its last three pixels, allowing the scalers working on these three disks to interpolate the end-boundary pixels between the disks. The horizontal offset value can be modulated at the beginning of each disk except the first.

[0039] This ensures that when the cut single frame 206 When upscaled to 8K, the original video appears as a single, seamless image without lines. Thus, as shown in line 212 The last three pixels are shown. 220 the disc 208-1 a second scaler than the first three pixels of the disc208-2 provided. Thus, the discs 208-1 and 208-4 1923 pixels, while the discs 208-2 and 208-3 They have 1926 pixels. This use of the last and first pixels of the preceding and following disks helps to ensure uniform scaling across vertical disk boundaries.

[0040] Fig. Figure 3 is a block diagram representation of how image scaling is performed according to the principles outlined in this description.

[0041] Starting with the opening image 304 The image is divided into four vertical slices. 308-1 , 308-2 , 308-3 and 308-4 cut each slice 308 represents approximately one quarter of the opening image 304 , although, as shown in the previous figure, the discs 308-2 , 308-3 and 308-4may contain one or more pixels from the previous image (e.g., one previous pixel if linear interpolation is used, or three previous pixels if higher-order interpolation (e.g., bicubic) is used).

[0042] Four independent scalers then scale the disks. 308 on upscaled discs 312 up. The disc 308-1 It will be applied to the disc 312-1 upscaled. The disc 308-2 is applied to the disc 312-2 upscaled. The disc 308-3 is applied to the disc 312-3 upscaled. The disc 308-4 is applied to the disc 312-4Upscaled. In this description, linear interpolation and higher-order interpolation are used as examples of image scaling techniques that can be employed. However, it should be noted that many image scaling algorithms are known, and the teachings of this description do not necessarily require a specific image scaling method. Rather, the teachings contained herein are compatible with any image scaling algorithm.

[0043] The scaled discs 312 are then achieved by horizontally recombining the different disks 312 and by multiplexing the result to create a highly scaled image 316 multiplexed together. For example, the original image 304 It will be a 1080p full HD image, while the upscaled image 316 It could be a 4K or 8K UHD image.

[0044] Fig.Figure 4 is a block diagram that represents a scaling operation according to the teachings of the present description.

[0045] In the representation of Fig. 4 should be an input 404 will be scaled up. To facilitate reference, pixels within the input are used. 404These are designated as pixels A, E, I, M, and Q. To illustrate, the pixels may need to be scaled up horizontally by a factor of four. For example, when converting between 1080p and 8K UHD, three additional horizontal pixels must be interpolated from each source pixel. It might seem counterintuitive that going from 1080p to 8K means quadrupling the resolution, but this is indeed the case. In 1080p, the number 1080 refers to the number of vertical lines in a resolution of 1920x1080, while in 8K, the number 8K refers to the number of horizontal pixels per line in a resolution of 7680x4320. Thus, 8K is approximately four times larger than 1080p in every dimension and contains roughly 16 times more total pixels.

[0046] Thus, pixel A of the input should 404 in the edition 408 to be interpolated to pixels ABCD. The pixel E of the input 404 should be included in the edition 408The pixels EFGH are interpolated. The pixel I of the input 404 should be included in the edition 408 The pixels IJKL are interpolated. The pixel M of the input 404 should be included in the edition 408 The pixels are interpolated to MNOP.

[0047] As mentioned previously, interpolating to an 8K resolution requires approximately 2.4 megapixels per second of processing power. Since existing scalers are often limited to a capacity of 700 megapixels per second due to implementation constraints such as available MIPS in an embedded system or maximum speed at a given processing node, these scalers may be insufficient to interpolate 1080p to 8K in real time. Therefore, this example uses four independent scalers to process the four quartiles of the input. 404to interpolate. Specifically, a first scaler interpolates three additional pixels starting from pixel A, a second scaler interpolates three additional pixels starting from pixel E, a third scaler interpolates three additional pixels starting from pixel I, and a fourth scaler interpolates three additional pixels starting from pixel M. As shown in the block diagram, the interpolation is performed in real time. Each disk scaler operates on a vertical disk whose row width is 1 / N of the input row width and scales the row with a slower time duration equal to one row time of the actual input. This is different from multiple parallel scaling operations that compute A, B, C, and D at the time E arrives.

[0048] To ensure that the output buffers do not run out, the scalers can operate in a pipeline-like manner, as shown. For example, the scaler interpolates. 412-1First, the pixels ABCD are processed and written to an output buffer. During the scaling process... 412-1 The scaler interpolates when working on the same quartile of the next row. 412-2 The pixel EFGH is read and written to its output buffer.

[0049] During the scaler 412-2 The scaler works on a second quartile of another row, interpolating the data. 412-3 The pixel IJKL is processed and written to its output buffer. During the scaler... 412-3 The scaler interpolates the third quartile of another row. 412-4 The pixel MNOP is written to its output buffer. Once the scalers have written the complete interpolated line to their output buffers, the four quartiles can be multiplexed together and written to an output buffer to draw this line of output onto the scaled image.

[0050] In cases where linear interpolation is used, each pixel can be interpolated as a simple weighted average of its two nearest points in each dimension (horizontal and vertical), with weighting depending on proximity to a point. When bicubic interpolation is used, the pixel can be interpolated from two points above and below, and two points to the left and right, resulting in a cubic interpolation in each dimension.

[0051] Fig. Figure 5 is a block diagram of an exemplary scaler according to the teachings of the present description.

[0052] As previously discussed, the scaler 500 four independent scalers, namely S1 512-1 until S4 512-4 Although these scalers operate independently, the parallel scaler is 500trained to operate these scalers in order to deliver a single scaled output image.

[0053] The parallel scaler 500 receives a video input 502 The video input 502 As previously explained, it is divided into four vertical disks. In particular, the first disk 1923 Pixel (with 3 pixels from the start of the second disk), a second disk 1926 pixels (including the last three pixels from the first disk and the first three pixels from the third disk), a third disk has 1926 pixels (including the last three pixels from the second disk and the first three pixels from the fourth disk) and a fourth disk has 1923 pixels (including the last three pixels from the third disk).

[0054] One purpose of descreening buffers 504The process involves slowing down the input pixels so that the disk scaler can operate at the slower rate corresponding to its maximum capability. For example, each disk scaler can only produce 1920 scaled pixels from 480 input pixels in a line time. However, for an 8K output, 7680 pixels must be produced in a line time, which requires processing 1920 input pixels in a line time. Since each disk scaler can only process 480 input pixels in a line time, the 1920 input pixels are slowed down to a rate of 480 input pixels per line. The input rate into the descreening buffer is the actual input rate (1920 pixels in a line time), and the output from this buffer is one-quarter of that speed (e.g., 480 input pixels per line time).

[0055] The first disc is attached to the de-rasterizing buffer. 504-1 supplied. The second disc is attached to the de-rasterizing buffer.504-2 delivered. The third disc is attached to the de-rasterizing buffer. 504-3 delivered. The fourth disc is attached to the de-rasterizing buffer. 504-4 supplied. De-rasterizing buffer. 504 They take the rasterized inputs and generate an output at a rate equal to the line rate of the image.

[0056] The derasting buffer 504-1 delivers its lines to the converter. 508-1 , the derasting buffer 504-2 delivers its output to the converter 508-2 , the derasting buffer 504-3 delivers its output to the converter 508-3 and the derasting buffer 504-4 delivers its output to the converter 508-4 .

[0057] Four-pixel-to-one-pixel converters 508-1 collect the four pixels to be used for interpolation into one pixel. Since the system transports and processes four pixels in one clock cycle (e.g., driven by a clock frequency of 600 to 700 MHz with a requirement to convert 2400 megapixels per second), it may be necessary to convert four pixels into one. Each disk scaler then works on one pixel as if it were a single-pixel processing scaler. The pixel converters 508 They deliver their output (e.g., a single pixel each) to the scalers. In particular, the converter delivers 508-1 its output to the scaler S1 512-1 , the converter 508-2 delivers its output to the scaler. S2 512-2 , the converter 508-3 delivers its output to the scaler. S3 512-3 and the converter 508-4 delivers its output to the scaler.S4 512-4 .

[0058] The scalers 512 perform the most computationally intensive operation of the parallel scaler 500 through. In the illustrative example, the scalers take 512 The four pixels in each dimension supplied by the derasterization buffers are interpolated from the input pixels to create one or more interpolated pixels in the center of these input pixels. Since the scalers 512 Since dedicated hardware (such as in an ASIC or SoC) can be used, it can be designed to perform the interpolation operation in a single clock cycle. Thus, a scaler operating at 700 MHz delivers 512 700Millions of samples per second interpolation. It should be noted that the number of pixels used depends on the scaler kernel employed. For example, two input pixels are used for bilinear interpolation and four pixels for bicubic interpolation; a different number of pixels may be used for other models.

[0059] The scalers 512 deliver their output to the rerasterization buffers 516 The scaler S1 512-1 delivers its output to the rerasterization buffer. 516-1 , the scaler S2 512-2 delivers its output to the rerasterization buffer. 516-2 , the scaler S3 512-3 delivers its output to the rerasterization buffer. 516-3 and the scaler S4 512-4 delivers its output to the rerasterization buffer. 516-4 .

[0060] The rerasterization buffers 516They then deliver their newly rasterized outputs to an output multiplexer. 520 , which combines the four vertical slices from the four quartiles of the image into a single output row of the image. The output multiplexer 520 This line is then output as video output. 524 .

[0061] Fig. Figure 6 is a block diagram illustrating pipeline formation according to the principles outlined in this description. As previously explained, pipeline formation can be used to ensure that output buffers are not left empty, thus preventing buffer underflows.

[0062] As it is in Fig.As shown in Figure 6, an input row can be divided into four quartiles. These are designated L1Q1 for the first quartile of row 1, L1Q2 for the second quartile of row 1, L1Q3 for the third quartile of row 1, and L1Q4 for the fourth quartile of row 1. Each quartile can be assigned to a specific scaler, such as the scaler 512 from Fig. 5. For example, the scaler could S1 512-1 Q1 , the scaler S2 512-2 Q2 , the scaler S3 512-3 Q3 and the scaler S4 512-4 Q4 are assigned. Thus, each scaler consistently works on the same quartile of its input row. To provide pipeline formation, the slice scalers work on the same input row, but in a time-distributed manner. As described in Fig. As shown in section 6, it works S1completely during line 1; S2 S3 works 3 / 4 of its time on the current line (e.g., for line 1) and 1 / 4 of its time at the beginning of the next line (e.g., for line 2); S3 works half of its time on the current line (e.g., for line 1) and half of its time at the beginning of the next line (e.g., for line 2). 2 ); S4 works 1 / 4 of its time on the current line (e.g. for line 1 ) and 3 / 4 of its time at the beginning of the next line (e.g. for line 2).

[0063] Assuming the scaler starts with the first line of the first frame in a video stream, L1Q1 is scaled at time = t0 at 1 / 4 of the pixel rate. This means the first scaler receives 1 / 4 of the pixels from L1 and operates at approximately one-quarter of the desired pixel rate. For example, upscaling to 8K requires approximately 2.4 gigasamples per second, while each scaler can operate at approximately 600 to 700 megasamples per second. At this point, the re-rasterization buffer for scaler S1 is... 512-1 3 / 16 of a line deep.

[0064] The scaler S1 The scaler starts working on L1Q1 and operates on L1Q1 between time t0 and time t3. At all times, the scaler writes to L1Q1. S1 512-1 its output into the scaled L1Q1 output.

[0065] The scaling process begins at time t1. S2 512-2with the work on L1Q2. As with L1Q1, L1Q2 is scaled at 1 / 4 pixel rate and the rerasterization buffer for the scaler. S2 512-2 is 3 / 16 of a line deep.

[0066] The scaling process begins at time t2. S3 512-3 with the scaling of L1Q3. As with L1Q1 and L1Q2, L1Q3 is scaled at 1 / 4 pixel rate and the rerasterization buffer for the scaler. S3 512-3 is 3 / 16 of a line deep.

[0067] At the current time = t4, the scaler has S1 512-1 It finishes its work on L1Q1 and delivers a scaled L1Q1 output to its output buffer. Meanwhile, the scaler begins. S4 512-4 with the work on L1Q4. As with L1Q1, L1Q2 and L1Q3, L1Q4 is scaled at 1 / 4 pixel rate and the reraster buffer of the scaler. S4 512-4 is 3 / 16 of a line deep.

[0068] In the next cycle, the scaler begins. S1 512-1At time t0, the scaler begins working on L2Q1, while the other scalers continue their work on their respective quartiles of L1. At time t1 of the new cycle, the scaler begins... S2 512-2 with work on L2Q2. At time t2 of the new cycle, the scaler begins. S3 512-3 The scaler begins working on L1Q3 of the new cycle at time t3 of the new cycle. S4 512-4 with the work on L2Q4. The outputs can then be multiplexed.

[0069] The table below shows the state of the rerasterization buffers at various output times. This table shows only one example; the implementation may be the same or different from the one described elsewhere in this document. For each quartile of a row: 1. 1 / 4 of the output line is taken from the Q1, Q2, Q3 or Q4 rerasterization buffer. 2. 1 / 16 of the output line is generated by each scaler. 3. 1 / 4 input line is the maximum that a given derasterization buffer can hold, depending on the availability of the required input and the space in that buffer. De-rasterizing buffer Rerasterization buffer Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Output video 1 / 4 (7,4, 8,1, 8,2, 8,3) 1 / 16 (7,8) 1 / 16 (7,12) 1 / 16 (7,16) 3 / 16 3 / 16 3 / 16 3 / 16 This pattern repeats itself 1 / 4 - 1 / 16 + 1 / 16 = 1 / 4 (8,1 - 8,4) 1 / 16 (7,8) 1 / 16 (7,12) 1 / 16 (7,16) 3 / 16 + 1 / 16 - 1 / 4 = 0 3 / 16 3 / 16 3 / 16 1 / 4 1 / 4 - 1 / 16 3 / 16 1 / 16 - 1 / 16 + 1 / 4 = 1 / 4 1 / 16 (7,12) 1 / 16 (7,16) 0 + 1 / 16 = 1 / 16 3 / 16 + 1 / 16-1 / 4 = 3 / 16 3 / 16 1 / 4 3 / 16 - 1 / 16 2 / 16 (8,3, 8,4) 1 / 4 - 1 / 16 3 / 16 (8.6 to 8.8) 1 / 16 - 1 / 16 + 1 / 4 = (8,9 - 8,12) 1 / 16 (7,16) 1 / 16 + 1 / 16 = 2 / 16 0 + 1 / 16 1 / 16 3 / 16 + 1 / 16 - 1 / 4 = 0 3 / 16 1 / 4 2 / 16 - 1 / 16 1 / 16 (8,4) 3 / 16 - 1 / 16 = 1 / 8 (8,7, 8,8) 1 / 4 - 1 / 16 = 3 / 16 (8.10 to 8.12) 1 / 16 - 1 / 16 + 1 / 4 (8,13 8,16) 2 / 16 + 1 / 16 = 3 / 16 1 / 16 + 1 / 16 = 2 / 16 0 + 1 / 16 1 / 16 3 / 16 + 1 / 16 - 1 / 4 = 0 1 / 4 1 / 16 - 1 / 16 + 1 / 4 = 1 / 4 (9.1 to 9.4) 2 / 16 - 1 / 16 (8,8) 3 / 16 - 1 / 16 = (8,11, 8,12) 1 / 4 - 1 / 16 = 3 / 16 (8.14 to 8.16) 3 / 16 + 1 / 16 - 1 / 4 = 0 2 / 16 + 1 / 16 = 3 / 16 1 / 16 + 1 / 16 = 2 / 16 0+ 1 / 16 = 1 / 16 1 / 4

[0070] By providing this parallel and pipelined scaling, the system described here implements a 12-bit process with 4:4:4 color encoding data. 4:4:4 is a color format in which the luma and chroma components are preserved at full bandwidth. The color can be represented in a color space such as YUV (Y = luma, UV = chrominance). In other embodiments, other color spaces such as red, green, blue (RGB) or cyan, magenta, yellow, black (CMYK) can be used.

[0071] Advantageously, this method can be used with directional interpolation, which provides sharpness, detail and image enhancement, and also offers support for user-controllable horizontal and vertical resolutions as well as aspect ratio conversion.

[0072] Fig. 7 is a flowchart of a process 700 for upscaling video according to the principles outlined in this description.

[0073] The procedure 700 can be performed by any suitable hardware or combination of hardware, software, and / or firmware. For ease of reference, the device that performs the procedure will be referred to as... 700 The device that executes the process is referred to as a scaler. This includes any hardware, software, and / or firmware that provides the necessary functionality.

[0074] In block 704The scaler receives the original input image. This original input image can be, for example, a video in any suitable format, such as, for illustrative and non-limiting reasons, 480p, 720p, 1080p, or 4K UHD. It should be noted that these formats are only examples, and other resolutions may be supported according to the requirements of a specific implementation.

[0075] In block 708 The scaler divides the input image into N vertical slices. In the preceding examples, figures show that the image is divided into four vertical slices, each of which is called a quartile. This is useful, for example, when upscaling a video to 8K using a 700 MHz scaler. However, other implementations may require a different number of slices or different types of hardware operating at different clock speeds.

[0076] In block712 The scaler operates several individual per-disk scalers in parallel to upscale the vertical disks and interpolate intermediate points between pixels for each image. This interpolation can take the form of, for example, a linear interpolation between two points in each dimension, a higher-order interpolation (e.g., a bicubic interpolation) between four points in each dimension, another polynomial interpolation, or another suitable upscaling or downscaling algorithm compatible with the teachings of the present description.

[0077] In block 716Upscaling the vertical slices results in a smoothing of the boundaries between the slices. The scaler can achieve this, for example, by allocating one, three, or another number of pixels from the previous slice to the next slice. This enables smooth interpolation between the slices. In other embodiments, particularly those using an algorithm other than linear interpolation or higher-order interpolation, other suitable algorithms can be used to smooth the boundaries.

[0078] The result is N individual upscaled disks.

[0079] In block 720The scaler multiplexes the scaled disks together. This allows the scaled disks to be output as a single image. For example, the multiplexer can have a series of input lines, with each individual scaler providing a quartile or, alternatively, 1 / N input line. The multiplexer combines these quartiles or 1 / N quantiles together to produce a single, contiguous output line.

[0080] In block 724 Once all lines of an output image have been collected, this output image can be delivered to the screen for 1 / k second, where k is the frame rate (e.g. 24 frames per second or 60 frames per second, for illustration).

[0081] In block 798 The process is complete.

[0082] Fig. Figure 8 is a block diagram of an HDMI system according to the principles outlined in this description. In this example, the HDMI system assigns the HDMI source800 and the HDMI sink 802 on.

[0083] The HDMI source 800 This can be provided, for example, by a DVD player, a Blu-ray disc player, a computer, or another video source. The HDMI source 800 receives video and audio signals from the HDMI transmitter 804 to be sent. The HDMI transmitter 804 It features three physically separate communication channels: the Data Display Channel (DDC), the Transition Minimised Differential Signaling (TMDS) signal, and the Consumer Electronics Control Channel (CEC) channel. The HDMI source 800 It can also provide an HDMI Ethernet and Audio Return Channel (HEAC), which is an optional channel for Ethernet and audio return. Furthermore, the HDMI source can 800Provide an optional hot-plug detection channel (HPD) line that is either high or low and enables the detection of a connection.

[0084] The different channels and functions of an HDMI source 800 and an HDMI sink 802are provided by the HDMI standard, which has various embodiments and iterations. The representation in this figure is merely an illustrative example of an Ethernet controller pair and should not be understood as a limiting example or as restricting the teachings of this description to any particular version of the HDMI standard. In general, the teachings in this figure should be understood as an illustrative framework within which the teachings of this description can be practiced. In particular, in embodiments where HDMI is used as the transmission standard for audio and video data, embodiments of the teachings of this description may include elements such as those in the HDMI source. 800 and the HDMI sink 802 shown.

[0085] The TMDS channel between the HDMI transmitter 804 and the HDMI receiver 808It interweaves video, audio, and auxiliary data using three different packet types. These are referred to as the video data period, data island period, and control period.

[0086] During the video data period, the pixels of the output video are transferred between the HDMI transmitter and the HDMI transmitter. 804 and the HDMI receiver 808 The data island period, which can occupy horizontal or vertical blanking intervals, is used to transmit audio and auxiliary data between the HDMI transmitter and the receiver. 804 and the HDMI receiver 808 to transmit. This audio and auxiliary data can be transmitted as a sequence of packets, similar to data transmission over a more conventional communication medium such as Ethernet or the Universal Serial Bus (USB).

[0087] The control period occurs between the video period and the data island period and is used to transmit control signals. HDMI uses the TMDS channels to transmit characters encoded with 8b / 10b encoding. It's worth noting that HDMI is backward compatible with the older Digital Visual Interface (DVI) standard, as DVI also uses TMDS for video data transmission. However, HDMI uses a smaller connector and also provides the ability to include the data island period and the control period between the horizontal and vertical blanking intervals of the video data period.

[0088] The display data channel provides communication based on the Inter-integrated Circuit (I2C) bus specification. The HDMI standard requires its devices to implement the DDC or Extended Display Data Channel (EDDC) provided by the HDMI source. 800for reading Extended Display Identification Data (EDID) or Extended EDID from the HDMI sink 802 EDID is used. 810 inform the HDMI source 800 about which audio and / or video formats the HDMI sink supports 802 The HDMI standard requires that the DDC support a bandwidth of at least 100 kb per second, and this can be scaled up to even faster speeds. In some implementations, the DDC is also used to provide High Bandwidth Digital Content Protection (HDCP), which is used to manage digital rights (DRM) for content transmitted between the HDMI source and the source device. 800 and the HDMI sink 802 to enforce the transmission. This helps ensure that a consumer cannot extract high-quality digital video from the video stream using an intermediary device.

[0089] In the representation of Fig. 8 communicate Tx CEC 812 and Rx CEC 816 via a CEC connection. A CEC connection allows an end user to command and control CEC-enabled devices. Current iterations of the HDMI standard allow the control of up to 15 different CEC-enabled devices via the HDMI connection. Thus, a user can control a television, DVD player, Blu-ray player, or other devices with just a single remote control, such as a TV remote, a separate device remote, a DVD player remote, or any other remote that controls a single device. Advantageously, CEC allows individual CEC-enabled devices to command and control each other without user intervention.

[0090] CEC offers a bidirectional serial single-wire bus based on the standard AV.link protocol of the European Committee for Electrotechnical Standardization (CENELEC).

[0091] Where provided for, a HEAC supply line can enable communication between a HEAC transmitter and a transmitter. 820 and a HEAC receiver 824 The HDMI Ethernet and Audio Return Channel provides both the Audio Return Channel (ARC) and the HDMI Ethernet Channel (HEC). The shared HEAC was included in the HDMI standard. 1.4 introduced and offers directed high-speed data communication for HEC and the ability to send audio data upstream via ARC.

[0092] ARC is an audio connection that replicates other cables between the TV and a receiver or speaker system. This can be used when the TV, rather than other devices, generates or receives a video stream. For example, this can be used when video is provided via an application on a smart TV (such as the streaming service Netflix), but the audio is delivered by other devices, such as external speakers. Without ARC, the audio output from the TV would have to be routed to the speaker system via a separate cable. With ARC, however, the audio can be sent directly from the TV to the speaker. The HEC combines video, audio, and data streams into a single HDMI cable and enables IP-based applications over HDMI. This provides bidirectional Ethernet communication at 100 MB / s or more.Physically, the HEC is a hybrid that simultaneously transmits and receives 100Base-T signals over a single twisted pair of quality-of-charge data wire.

[0093] Fig. 9a and Fig. 9b is a block diagram of a single-chip system. 900 according to the teachings of the present description.

[0094] It should be noted that the SoC 900 is provided as an illustrative and non-limiting example of a form factor that can be used in the teachings of this description. In this case, the SoC 900 monolithically provides a set of video processing functions on an HDMI receiver and delivers the processed video as an HDMI output. Such a SoC was known as the SoC 150 in Fig. 1b is shown. The lessons of SoC 150 from Fig. 1b can be used on the SoC 900 from Fig. 9a and Fig.9b will be applied if they are compatible. However, it is understood that in some examples the SoC 900 is a separate or alternative embodiment to the SoC 150. Fig. It may be 1b. In other examples, SoC 900 can be a more detailed presentation of the lessons of SoC 150. Fig. 1b can be viewed.

[0095] In Fig. Figure 9a shows that the SoC 900 is equipped with a direct access memory such as DDR. 3 / 4 948 This is a dynamic, double-speed direct-access memory (DRAM) which, in some embodiments, is not part of the SoC 900 but is provided as a separate module. However, this should not be interpreted as precluding the use of on-chip DDR with the SoC 900. Rather, this is simply a common embodiment where the DDR 948 as a separate, modular element provided by the SoC 900.

[0096] With the exception of the GDR 948 are the others in Fig. 9a and Fig. The elements shown in Figure 9b are typically included in a SoC such as the SoC 900. However, this should not be interpreted as meaning that an SoC 900 must have all the elements shown here, or that it cannot have any elements other than those shown. Instead, the SoC 900 is provided as an illustration of a commonly used configuration.

[0097] For the sake of simplicity, the SoC 900 is divided into three blocks, namely the blocks 901 and 902 from Fig. 9a and the block 903 from Fig. 9b. The blocks 901 and 903 They can operate at approximately 700 MHz or 700 megasamples per second. In contrast, Block 902Real-time video operations may need to be performed at a much faster speed, such as 2.4 gigasamples per second. This higher speed may be necessary, for example, to enable real-time processing and upscaling of 8K video.

[0098] In block 901 A PHY HDMI receiver 904 A physical interface is provided for receiving video data from a video source. The video source can be any of the sources discussed in this description or any other video source suitable for the teachings described here. For example, in addition to HDMI, DisplayPort, or an analog video interface such as CVBS (compound video baseband signal, also known as color, video, blanking, and synchronization).

[0099] The HDMI receiver PHY 904The received input video is processed by a digital HDMI receiver circuit. 906 processed. The digital HDMI receiver circuit 906 It features the logic for processing an HDMI input according to the HDMI standard. It should be noted that the digital HDMI receiver circuit... 906 also HDCP capability 908 can exhibit the HDCP 908 offers as in Fig. 8 shows DRM for HDMI content.

[0100] The digital HDMI receiver circuit 906 Its output is sent to a link data decompression circuit. 910 The connection data decompression circuit 910It may be necessary to convert the compressed HDMI video signal into a sequence of individual frames, with each frame being its own bitcard. This may be required because, according to industry standards, HDMI provides a compressed video format in which video frames are not provided as a series of bitcards, but rather as a series of deltas relative to a reference frame.

[0101] The decompressed video can be sent to the image buffer compressor. 912 The component supplied provides the image buffer compression and stores the result in memory via the DDR3 / 4 memory controller 940. The memory controller 940 communicates with DDR 3 / 4 PHY 944 , regarding communication with the GDR 948 provides. As mentioned above, the GDR must 948 It is not necessarily part of the SoC 900, but in some examples may instead be provided as a separate modular unit. The DDR 3 / 4 PHY 994It may contain compression codecs.

[0102] The image buffer decompressor 914 It receives the compressed information from memory and then delivers the decompressed video to a decoupling / scaling machine. 916 The decoupling / scaling device 916 It can offer several functions. For example, if video is provided in an interlaced format instead of a progressive scan format, it may first need to be converted to the progressive scan format before it can be scaled. Thus, the unbundling function of the unbundle / scaler can 916 Provide this decoupling. The decoupling / scaling tool. 916 can also be a parallel scaler such as the scaler 500 from Fig. Provide 5. This can scale images up or down according to the principles outlined in this description.

[0103] As soon as the decoupling / scaling 916Once the image has been processed, unbundled, and scaled up or down, the video images can be sent to the connection data compressor. 918 to deliver. The connection data compressor 918 It compresses the video according to the HDMI standard, so that it is no longer a sequence of individual bitcards of individual images, but a compressed video format based on differences between individual images is provided.

[0104] In Fig. 9b is delivered by the DSC compressor 920 its compressed HDMI-compatible signal to the digital HDMI transmitter circuit 924 As with the digital HDMI receiver circuit 906 The digital HDMI transmitter circuit 924 the logic for providing the HDMI signal and can handle HDCP 928 exhibiting DRM.

[0105] The digital HDMI transmitter circuit 924 delivers its data to the HDMI transmitter PHY 932, which sends the HDMI signal to a display such as a UHD-capable television 116 from Fig. outputs 1. Example implementations

[0106] The following examples are provided for illustration.

[0107] In one example, a video processor is disclosed comprising: an input buffer for receiving an input image; a cutting circuit for dividing the input image into several N vertical slices; N parallel image scalers, each scaler being configured in hardware to scale one of the N vertical slices line by line according to an image scaling algorithm; and an output multiplexer for combining the scaled vertical slices into a combined scaled output image.

[0108] Furthermore, a video processor is disclosed which further comprises logic and a circuit arrangement for receiving an input video stream and for extracting the input image from the input video stream.

[0109] Furthermore, a video processor is disclosed in which each vertical disk is essentially a 1 / N quantile of the input image.

[0110] Furthermore, a video processor is disclosed in which N = 4.

[0111] Furthermore, a video processor is disclosed in which the input image is a full frame of a video stream and in which the output image is scaled to 7680 horizontal pixels × 4320 vertical pixels.

[0112] Furthermore, a video processor is disclosed which includes logic and a circuit arrangement for receiving an input video stream with 8K UHD resolution and for downscaling the image to a lower resolution via the N parallel image scalers.

[0113] Furthermore, a video processor is disclosed which includes logic and a circuit arrangement for receiving an input video stream with a resolution selected from 480p, 480i, 720p, 720i, 1080p, 1080i and 4K Ultra High Definition (UHD), and for scaling the input video stream via the N parallel image scalers into an output video stream with 8K UHD resolution.

[0114] Furthermore, a video processor is disclosed in which the image scaling algorithm features bilinear interpolation.

[0115] Furthermore, a video processor is disclosed in which the image scaling algorithm features higher-order interpolation.

[0116] Furthermore, a video processor is disclosed in which the image scaling algorithm features higher-order interpolation.

[0117] Furthermore, a video processor is disclosed in which vertical disks representing essentially a first, second, and third quartile contain three first pixels from a subsequent disk, and vertical disks representing a second, third, and fourth quartile contain three last pixels from a preceding disk; and the first, second, and third disks also contain the three first pixels of the subsequent disk.

[0118] Furthermore, a video processor is disclosed in which at least some of the vertical disks contain one or more last pixels from a preceding or subsequent disk.

[0119] Furthermore, a video processor is disclosed in which at least some of the vertical disks contain the last three pixels from a preceding or subsequent disk.

[0120] Furthermore, a video processor is disclosed in which the N image scalers are arranged in pipelines to prevent buffer underflows.

[0121] Furthermore, a video processor is disclosed in which the output image is a single frame of a video stream compatible with a high-resolution multimedia interface (HDMI).

[0122] Furthermore, a video processor is disclosed in which the input image is a single frame of a video stream compatible with a high-resolution multimedia interface (HDMI).

[0123] Furthermore, a block of intellectual property (IP block) is disclosed which includes the video processor of one of the preceding examples.

[0124] It is also disclosed a system-on-a-chip (SoC) comprising: a circuit arrangement for receiving an input image; an image cutter for dividing the input image into multiple N slices; N image scalers designed to scale the N slices in parallel and comprising an interpolator circuit for interpolating intermediate pixels between pixels in the input image; and an output multiplexer for multiplexing the scaled vertical slices into a combined scaled output image.

[0125] Furthermore, a SoC is disclosed which further comprises: N input derasterization buffers for receiving the input image at 1 / N of the pixel rate; and a rerasterization buffer for collecting the N scaled disks and outputting them at a pixel rate of 1.

[0126] Furthermore, a SoC is disclosed which further comprises a circuit arrangement for receiving an input video stream and for extracting the input image from the input video stream.

[0127] Furthermore, a SoC is disclosed in which each vertical disk is essentially a 1 / N quantile of the input image.

[0128] Furthermore, a SoC is disclosed where N = 4.

[0129] Furthermore, a SoC is disclosed in which the input image is a single frame of a video stream and in which the output image is scaled to 7680 horizontal pixels × 4320 vertical pixels.

[0130] Furthermore, a SoC is disclosed which includes a circuit arrangement for receiving an input video stream with a resolution selected from 480p, 480i, 720p, 720i, 1080p, 1080i and 4K Ultra High Definition (UHD), and for scaling the input video stream via the N parallel image scalers into an output video stream with 8K UHD resolution.

[0131] Furthermore, a SoC is disclosed in which the interpolator circuit has a linear interpolator.

[0132] Furthermore, a SoC is disclosed in which the interpolator circuit has a polynomial interpolator.

[0133] Furthermore, a SoC is disclosed in which the interpolator circuit has a higher-order interpolator and in which at least some of the vertical disks contain three last pixels from a previous disk.

[0134] Furthermore, a SoC is disclosed in which the interpolator circuit has a higher-order interpolator and in which at least some of the vertical disks contain three last pixels from a subsequent disk.

[0135] Furthermore, a SoC is disclosed in which at least some of the vertical disks contain one or more last pixels from a previous disk.

[0136] Furthermore, a SoC is disclosed in which at least some of the vertical disks contain one or more last pixels from a subsequent disk.

[0137] Furthermore, a SoC is disclosed in which at least some of the vertical disks contain the last three pixels from a previous disk.

[0138] Furthermore, a SoC is disclosed in which at least some of the vertical disks contain the last three pixels from a subsequent disk.

[0139] Furthermore, a SoC is disclosed in which the N image scalers are arranged in pipelines to prevent buffer underflows.

[0140] It is further disclosed that the SoC of several of the above examples further comprises: a high-resolution multimedia interface receiver (HDMI receiver) for receiving an input HDMI video stream; a DSC decompressor for decompressing the HDMI video input and extracting the input image; a DSC compressor for receiving the output image and encoding the output image into a scaled output HDMI video stream; and an HDMI transmitter for sending the scaled output HDMI video stream.

[0141] Furthermore, a SoC is disclosed which also features a codec for image buffer compression.

[0142] Furthermore, a SoC has been revealed which also features an interface for dynamic random access memory.

[0143] Furthermore, a SoC is disclosed which also features an interlaced deinterlacer.

[0144] Furthermore, a SoC is disclosed which also includes a color space converter.

[0145] Furthermore, a SoC is revealed which also features an image cropper.

[0146] Furthermore, a SoC is disclosed which also features a frame rate converter.

[0147] Furthermore, a method for scaling an image is disclosed, comprising: receiving an input image; dividing the input image into several N slices; parallel raster-based scaling of the N slices according to an image scaling algorithm; and combining the scaled vertical slices to form a combined scaled output image.

[0148] Furthermore, a method is disclosed which also includes receiving and extracting the input image from an input video stream.

[0149] Furthermore, a method is disclosed in which each disk is essentially a 1 / N quantile of the input image.

[0150] Furthermore, a method is disclosed in which N = 4.

[0151] Furthermore, a method is disclosed in which the input image is a single frame of a video stream and in which the output image is scaled to 7680 horizontal pixels × 4320 vertical pixels.

[0152] Furthermore, a method is disclosed which also includes upscaling an image to an 8K UHD resolution or downscaling the image from an 8K UHD resolution.

[0153] Furthermore, a method is disclosed in which the image scaling algorithm features linear interpolation.

[0154] Furthermore, a method is disclosed in which the image scaling algorithm features polynomial interpolation.

[0155] Furthermore, a method is disclosed in which the image scaling algorithm features quadratic interpolation.

[0156] Furthermore, a method is disclosed in which at least some of the vertical disks contain one or more last pixels from a previous disk.

[0157] Furthermore, a method is disclosed in which at least some of the vertical disks contain three last pixels from a previous disk.

[0158] Furthermore, a method is disclosed which also includes the integration of parallel line-wise image scaling into a pipeline.

[0159] Furthermore, a method is disclosed in which the output image is a single frame of a video stream compatible with a high-resolution multimedia interface (HDMI).

[0160] Furthermore, a method is disclosed in which the input image is a single frame of a video stream compatible with a high-resolution multimedia interface (HDMI).

[0161] Furthermore, the method of a number of the above examples is disclosed, which further includes providing image buffer compression and decompression.

[0162] Furthermore, a SoC has been revealed which also features an interface for dynamic random access memory.

[0163] Furthermore, the method of a number of the above examples is disclosed, which also includes the provision of interlacing (deinterlacing).

[0164] Furthermore, the procedure of a number of the above examples is disclosed, which also includes providing a color space conversion.

[0165] Furthermore, the method of a number of the above examples is disclosed, which also includes providing an image cropping function.

[0166] Furthermore, a device is disclosed which includes a means for carrying out the method of a number of the above examples.

[0167] Furthermore, a device is disclosed in which the means comprise a single-chip system.

[0168] Furthermore, a device is disclosed in which the means include an auxiliary box.

[0169] Furthermore, an intelligent television set is disclosed which has a device according to one or more of the above examples.

[0170] One or more specific non-volatile, computer-readable media are also disclosed on which instructions are stored to command a programmable device to execute the procedure or at least to realize part of the device of a number of the above examples.

[0171] One or more specific non-volatile, computer-readable media are also disclosed, the commands of which include hardware description language instructions.

[0172] The foregoing outlines features of one or more embodiments of the subject matter disclosed herein. These embodiments are provided to enable those skilled in the art to better understand various aspects of the present disclosure. Reference may be made to certain well-understood concepts as well as underlying technologies and / or standards without describing them in detail. It is expected that those skilled in the art will have, or access to, background knowledge or information in these technologies and standards in order to put the lessons of this description into practice.

[0173] Those skilled in the art will recognize that they can easily use the present disclosure as a basis for designing or modifying other processes, structures, or variations to achieve the same purposes and / or the same advantages as the embodiments introduced herein. They will also recognize that such equivalent designs do not deviate from the concept and scope of protection of the present disclosure and that they can make various changes, substitutions, and modifications to it without deviating from the concept and scope of the present disclosure.

[0174] It should be noted that the activities discussed above with reference to the figures are applicable to any integrated circuit that incorporates signal processing (for example, gesture signal processing, video signal processing, audio signal processing, analog-to-digital conversion, digital-to-analog conversion), especially those capable of executing specialized software programs or algorithms, some of which may be associated with processing digitized real-time data. Specific embodiments may relate to multi-DSP, multi-ASIC, or multi-SoC signal processing, floating-point processing, signal / control processing, fixed-function processing, microcontroller applications, etc.In certain contexts, the features discussed herein may be applicable to medical systems, scientific instruments, wireless and wired communications, radar, industrial process control, audio and video equipment, power measurement, instruments (which may be high-precision), and other digitally processing-based systems. Furthermore, certain embodiments discussed above may be used in digital signal processing technologies for medical imaging, patient monitoring, medical instrumentation, and home healthcare. This could include, for example, lung monitors, accelerometers, heart rate monitors, or pacemakers, along with their peripheral devices. Other applications may include automotive technologies for safety systems (e.g., stability control systems, driver assistance systems, braking systems, infotainment, and interior applications of all kinds).Furthermore, powertrain systems (e.g., in hybrid and electric vehicles) can utilize high-precision data conversion, rendering, and display products for battery monitoring, control systems, message control, maintenance activities, and other purposes. In yet other exemplary scenarios, the teachings of this disclosure may be applicable to industrial markets encompassing process control systems that contribute to increased productivity, energy efficiency, and reliability. In consumer applications, the teachings of the signal processing circuits discussed above may be used for image processing, autofocus, and image stabilization (e.g., for digital still cameras, camcorders, etc.). Other consumer applications may include audio and video processors for home theater systems, DVD recorders, and HD televisions. Further consumer applications may include advanced touchscreen controllers (e.g., smartphones, smartphones, etc.).for any type of portable media device). Therefore, such technologies could easily be part of smartphones, tablets, security systems, PCs, gaming technologies, virtual reality, simulation training, etc.

[0175] The foregoing outlines features of several embodiments so that those skilled in the art may better understand the aspects of the present disclosure. Those skilled in the art should recognize that they can readily use the present disclosure as a basis for designing or modifying other processes and structures to achieve the same purposes and / or the same advantages as the embodiments introduced herein. They should also recognize that such equivalent designs do not deviate from the concept and scope of protection of the present disclosure and that they can make various changes, substitutions, and modifications to it without deviating from the concept and scope of protection of the present disclosure.

[0176] The specific embodiments of the present disclosure may readily include a single-chip system-on-a-chip central processing unit (SoC-CPU assembly). An SoC is an integrated circuit (IC) that integrates components of a computer or other electronic system onto a single chip. It may have digital, analog, mixed-signal, and radio-frequency capabilities, all of which may be provided on a single chip substrate. Other embodiments may include a multi-chip module (MCM) with several chips arranged in a single electronic assembly and configured to interact closely with one another via the electronic assembly.Each module, function, or block element of an ASIC or SoC can optionally be provided in a reusable block of intellectual property (IP block, "black box") that can be distributed separately without disclosing the logical details of the IP block. In various other embodiments, the digital signal processing functionalities can be implemented in one or more silicon cores in application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and other semiconductor chips.

[0177] In some cases, the teachings of this description may be encoded in one or more specific non-volatile, computer-readable media on which executable instructions are stored. These instructions, when executed, instruct a programmable device (such as a processor or DSP) to perform the methods or functions disclosed herein. In cases where the teachings herein are at least partially embodied in a hardware device (such as an ASIC, an IP block, or a SoC), a non-volatile medium could contain a hardware device programmed with logic to perform the methods or functions disclosed herein. The teachings could also be practiced in the form of the Register Transfer Layer (RTL) or another hardware description language such as VHDL or Verilog, which can be used to program a manufacturing process for producing the disclosed hardware elements.

[0178] In exemplary implementations, at least some parts of the processing activities outlined herein may also be implemented in software. In some embodiments, one or more of these features may be implemented in hardware provided outside the elements of the disclosed figures, or combined in any suitable way to achieve the intended functionality. The various components may include software (or response software) that can coordinate to achieve the operations described herein. In still other embodiments, these elements may include any suitable algorithms, hardware, software, components, modules, interfaces, or objects that facilitate operations by them.

[0179] Additionally, some of the components associated with the described microprocessors can be removed or consolidated in other ways. In general, the arrangements depicted in the figures may be more logical in their representations, whereas a physical architecture may exhibit various permutations, combinations, and / or hybrids of these elements. It is essential to note that countless possible design configurations can be used to achieve the operational goals described herein. Accordingly, the associated infrastructure exhibits a multitude of substitute arrangements, design options, fixture possibilities, hardware configurations, software implementations, equipment options, and so on.

[0180] Any suitably trained processor component can execute any type of instruction associated with the data to achieve the operations set forth herein. Any processor disclosed here could transform an element or object (for example, data) from one state or item to another state or item. In another example, some of the activities outlined here can be implemented with fixed logic or programmable logic (e.g., software and / or computer instructions executed by a processor), and the elements identified herein could be a type of programmable processor, programmable digital logic (e.g.,An FPGA, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an ASIC with digital logic, software, code, electronic instructions, flash memory, optical disks, CD-ROMs, DVD-ROMs, magnetic or optical cards, other types of machine-readable media suitable for storing electronic instructions, or any suitable combination thereof. In operation, processors may store information in any suitable type of non-volatile memory medium (e.g., random-access memory (RAM), read-only memory (ROM), FPGA, EPROM, electrically erasable programmable ROM (EEPROM), etc.), software, hardware, or any other suitable component, device, element, or object, provided this is appropriate and based on specific requirements.Furthermore, the information tracked, sent, received, or stored in a processor could be made available in any database, register, table, cache, queue, control list, or memory structure, based on specific requirements and implementations, all of which could be accessed at any suitable time. Each of the memory elements discussed here should be understood as falling under the general term "memory." Similarly, each of the potential processing elements, modules, and machines described here should be understood as falling under the general term "microprocessor" or "processor."Furthermore, in various embodiments, the processors, memory, network cards, buses, storage devices, associated peripherals and other hardware elements described herein can be implemented by a processor, memory and other associated devices configured by software or firmware to emulate or virtualize the functions of these hardware elements.

[0181] Computer program logic that implements all or part of the functionality described herein is embodied in various forms, including, but not limited to, source code, computer-executable, hardware description, and various intermediate forms (e.g., topographies or forms created by an assembler, compiler, linker, or locator). For example, the source code contains a set of computer program instructions implemented in various programming languages, such as object code, assembly language, or a high-level language like OpenCL, RTL, Verilog, VHDL, Fortran, C, C++, Java, or HTML for use in different operating systems or operating environments. The source code may define and use various data structures and communication messages. The source code may be in computer-executable form (e.g.,the source code is available via an interpreter) or the source code can be converted into a computer-executable form (e.g. via a translator, assembler or compiler).

[0182] In the discussions of the above embodiments, the capacitors, buffers, graphics elements, interconnect boards, clock generators, DDRs, camera sensors, converters, inductors, resistors, amplifiers, switches, digital cores, transistors, and / or other components can be easily replaced, exchanged, or otherwise modified to meet specific circuit requirements. Furthermore, it should be noted that the use of complementary electronic devices, hardware, non-volatile software, etc., provides an equally viable option for implementing the teachings of this disclosure.

[0183] In an exemplary embodiment, any number of electrical circuits of the figures can be implemented on a circuit board of an associated electronic device. The circuit board can be a general-purpose printed circuit board that can accommodate various components of the electronic device's internal electronic system and also provide connections for other peripheral devices. In particular, the circuit board can provide the electrical connections through which the other components of the system can communicate electrically. Any suitable processors (including digital signal processors, microprocessors, supporting chipsets, etc.), memory elements, etc., can be appropriately coupled to the circuit board based on specific configuration requirements, processing requirements, computer designs, etc.Other components, such as external storage, additional sensors, audio / video display controllers, and peripherals, can be connected to the circuit board as plug-in cards, via cables, or integrated directly into the board itself. In another exemplary embodiment, the electrical circuits of the figures can be implemented as standalone modules (e.g., a device with associated components and circuits configured to perform a specific application or function) or as plug-in modules in application-specific hardware of electronic devices.

[0184] It should be noted that in the numerous examples provided herein, the interaction may be described with respect to two, three, four, or more electrical components. This has been done, however, only for the sake of clarity and illustration. It is understood that the system can be simplified in any suitable way. Along similar design alternatives, any of the components, modules, and elements of the figures can be combined in various possible configurations, all of which are clearly within the broad scope of this description. In certain cases, it may be simpler to describe one or more of the functions of a particular set of processes by referring to only a limited number of electrical elements.It is understood that the electrical circuits in the figures and their teachings are easily scalable and can incorporate a large number of components as well as more intricate / complex arrangements and configurations. Accordingly, the examples provided should not limit the scope of protection or hinder the general teachings of electrical circuits as they may be applied to a variety of other architectures.

[0185] Numerous other modifications, replacements, variations, alterations and modifications may be known to those skilled in the art, and it is intended that the present disclosure includes all such modifications, replacements, variations, alterations and modifications that fall within the scope of protection of the attached claims.To assist the United States Patent and Trademark Office (USPTO) and, furthermore, all readers of a patent granted for this application in interpreting the accompanying claims, the applicant wishes to state that the applicant has no intention of: (a) The accompanying claims are not intended to rely on 35 USC § 112(f) as it exists on the filing date, unless the words “means for” or “steps for” are expressly used in the respective claims; and (b) No statement in the description is intended to limit this disclosure in any way not otherwise provided in the accompanying claims.

[0186] According to one aspect, an example discloses a video processor comprising: an input buffer for receiving an input image; a cutting circuit for dividing the input image into several N vertical slices; N parallel input buffers for derasterization; N parallel image scalers, each scaler being configured in hardware to scale one of the N vertical slices into a raster shape according to an image scaling algorithm; N parallel output buffers for rerasterization; and an output multiplexer for combining the scaled vertical slices into a combined scaled output image.

Claims

[1] Video processor comprising the following features: an input buffer to receive an input image; a cutting circuit for dividing the input image into several N vertical slices; N parallel image scalers, each scaler being hardware-configured to scale one of the N vertical disks row-wise according to an image scaling algorithm; and an output multiplexer to combine the scaled vertical slices into a combined scaled output image. [2] Video processor according to claim 1, further comprising logic and a circuit arrangement for receiving an input video stream and for extracting the input image from the input video stream. [3] Video processor according to any of the preceding claims, wherein each vertical disk is essentially a quartile of the input image. [4] Video processor according to any of the preceding claims, comprising logic and a circuit arrangement for receiving an input video stream with a resolution selected from 480p, 480i, 720p, 720i, 1080p, 1080i and 4K Ultra High Definition (UHD), and for scaling the input video stream via the N parallel image scalers into an output video stream with 8K UHD resolution. [5] Video processor according to any of the preceding claims, comprising logic and a circuit arrangement for receiving an input video stream with 8K UHD resolution and for downscaling the image to a lower resolution via the N parallel image scalers. [6] Video processor according to any of the preceding claims, wherein the image scaling algorithm comprises linear interpolation. [7] Video processor according to any of the preceding claims, wherein the image scaling algorithm comprises higher order interpolation. [8] Video processor according to any of the preceding claims, wherein vertical disks representing essentially a first, second and third quartile contain three first pixels from a subsequent disk, and vertical disks representing a second, third and fourth quartile contain three last pixels from a preceding disk. [9] Video processor according to any of the preceding claims, wherein the N image scalers are included in pipelines. [10] Video processor according to any of the preceding claims, wherein the output image is a single frame of a video stream compatible with a high-resolution multimedia interface (HDMI). [11] Video processor according to any of the preceding claims, wherein the input image is a single frame of a video stream compatible with a high-resolution multimedia interface (HDMI). [12] Block of intellectual property (IP block) that includes the video processor of one of the preceding examples. [13] System-on-a-chip (SoC) comprising the following: a circuit arrangement for receiving an input image; an image cutter for dividing the input image into several N slices; N image scalers designed to scale the N disks in parallel, and which include an interpolator circuit for interpolating intermediate pixels between pixels in the input image; and an output multiplexer for multiplexing the scaled vertical disks to a combined scaled output image. [14] SoC according to claim 13, further comprising: N input derasterization buffers for receiving the input image at 1 / N of the pixel rate; and a rerasterization buffer for collecting the N scaled disks and outputting them at a pixel rate of 1. [15] SoC according to claim 13 or 14, wherein the interpolator circuit comprises a higher order interpolator and wherein at least some of the vertical disks contain three last pixels from a subsequent disk. [16] SoC according to any one of claims 13 to 15, further comprising: a high-resolution multimedia interface receiver (HDMI receiver) for receiving an input HDMI video stream; a DSC decompressor for decompressing the HDMI video input and extracting the input image; a DSC compressor for receiving the output image and encoding the output image into a scaled output HDMI video stream; and an HDMI transmitter to send the scaled output HDMI video stream. [17] Method for scaling an image that has the following features: Receiving an input image; Dividing the input image into several N slices; parallel raster-based scaling of the N disks according to an image scaling algorithm; and Combining the scaled vertical slices into a combined scaled output image. [18] Method according to claim 17, further comprising receiving and extracting the input image from an input video stream. [19] Method according to claim 17 or 18, further comprising upscaling an image to an 8K UHD resolution or downscaling the image from an 8K UHD resolution. [20] Method according to any one of claims 17 to 19, wherein the image scaling algorithm has quadratic interpolation, and wherein at least some of the vertical slices contain one or more last pixels from a previous slice.

Images