Scale processing method for high resolution display, and display device

Abstract

A display device of the present disclosure may comprise: a system on chip (SOC) including an image processing processor for generating an image of a first resolution by processing an input image; a second scaler for performing up-scaling on the image; and a display. The display device can generate, by using the system-on-chip, an image having a first resolution with respect to a first input image, and then increase, by using a first scaler of the system-on-chip, the size of the image in the vertical direction by a vertical resolution value difference between the first resolution and a second resolution, increase, by using a second scaler, the size of the image in the horizontal direction by a horizontal resolution value difference between the first resolution and the second resolution with respect to the first input image, and output, through the display, the first input image converted to have a second resolution size.

Description

Scale processing method for high-resolution displays and display device

[0001] The present disclosure relates to a scale processing method and a display device for a high-resolution display.

[0002] Display technology is evolving with continuous improvements in size and clarity (resolution). Through LCD and LED technologies, more pixels can be packed into a smaller space, significantly enhancing screen clarity. As a result, displays are now capable of delivering high-definition content more vividly across various devices, including TVs, smartphones, and monitors. Furthermore, next-generation display technologies like OLED and QLED are dramatically improving color reproduction and contrast ratios to create even more lifelike images. This reflects design innovations that allow for efficient space utilization while increasing display size and maximizing immersion. For instance, advancements are moving in a direction where the user's visual experience is not obstructed, even as the screen size increases, by reducing bezel thickness or adopting curved displays.

[0003] Image processing processors are also advancing upscaling technology suitable for high-resolution displays. Upscaling, which converts low-resolution video to fit high-resolution screens, is evolving into a method that utilizes sophisticated algorithms to minimize image degradation and restore detail. In particular, image processing technology incorporating artificial intelligence (AI) and machine learning plays a significant role in improving image quality while simultaneously increasing resolution.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] A display device according to one embodiment of the present disclosure may include a system on chip (SoC) comprising an image processing processor that processes an input image to generate an image of a first resolution; a second scaler that performs upscaling of the image; and a display that outputs an image of a second resolution larger than the first resolution. After generating an image of the first resolution for a first input image through the system on chip, the size of the image in the vertical direction is increased by the difference in vertical resolution values ​​between the first resolution and the second resolution using the first scaler of the system on chip and transmitted to the second scaler. Through the second scaler, the size of the image in the horizontal direction is increased by the difference in horizontal resolution values ​​between the first resolution and the second resolution for the first input image and transmitted to the display. The first input image converted to the size of the second resolution may be output through the display.

[0006] A method for generating a high-resolution display image of a display device according to another embodiment of the present disclosure may include: generating an image having a first resolution by performing image processing on a first input image using an image processing processor that generates an image of a first resolution; increasing the size of the first input image by q in the vertical direction using a first scaler of the image processing processor and transmitting it to a second scaler through a first high-speed data transmission interface; increasing the size of the first input image by p in the horizontal direction using the second scaler; transmitting the first input image to a display through a second high-speed data transmission interface by the second scaler; and the display outputting the first input image of a second resolution that is increased by p in the horizontal direction and q in the vertical direction from the first resolution. However, the problem to be solved by the present disclosure is not limited to the problem mentioned above, and may be determined in various ways without departing from the subject matter and scope of the present disclosure.

[0007] FIG. 1 is a block diagram including hardware components of a display device according to one embodiment of the present disclosure.

[0008] FIG. 2 is an example of an upscaling process for an input image using a display device according to one embodiment of the present disclosure.

[0009] FIG. 3 is a flowchart illustrating a scale processing operation for a high-resolution display in a display device according to one embodiment of the present disclosure.

[0010] FIG. 4 is a block diagram including a logic component including an on-the-fly scaler of a display device according to one embodiment of the present disclosure.

[0011] FIG. 5 is a block diagram including a logic component including a main / subscaler of a display device according to one embodiment of the present disclosure.

[0012] FIG. 6 is an example of a raster size change according to a panel size change in a display device according to one embodiment of the present disclosure.

[0013] FIG. 7 is an example of a display device according to various types of high-resolution displays according to one embodiment of the present disclosure.

[0014] In the following description, the attached drawings are referenced, and specific examples of implementation are illustrated within the drawings. Additionally, other examples may be used and structural modifications may be made without departing from the scope of the various examples.

[0015] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the technical features of this disclosure. For example, a component expressed in the singular form should be understood as a concept including singular or plural components unless the context clearly indicates only the singular form.

[0016] In the present disclosure, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may include any one of the items listed with the corresponding phrase, or all possible combinations thereof. The term “and / or” as used in the present disclosure should be understood to encompass any possible combination by one or more of the plurality of items listed with the corresponding term. Terms such as “first,” “second,” “first,” or “second” as used in the present disclosure may be used merely to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order).

[0017] Where it is stated that any (e.g., 1st) component is “coupled,” “connected,” “linked,” “coupled,” “supported,” “connected,” or “contacted” with or without the terms “functionally” or “communicationly,” it includes not only cases where the component is directly coupled, connected, linked, coupled, supported, or contacted with the other component, but also cases where it is indirectly coupled, connected, linked, coupled, supported, or contacted through a third component.

[0018] Terms such as "include" or "have" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this disclosure, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. When a component is described as being located "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where another component exists between the two components.

[0019] The expression “configured to” as used in this disclosure may be appropriately substituted, depending on the context, for example, “suitable for,” “capable of,” “designed to,” “modified to,” “made to,” or “capable of.” The term “configured to” does not necessarily mean only that which is “specially designed” in hardware. Instead, in some situations, the expression “device configured to” may mean that the device is “capable of” together with other devices or components. For example, the phrase “device configured (or set) to perform A, B, and C” may mean a device dedicated to performing the said operation, or a general-purpose device capable of performing various operations including said operation.

[0020] Terms used in this disclosure, such as "upper side," "lower side," and "front-rear direction," are defined based on the drawings, and the shape and position of each component are not limited by these terms.

[0021] Although the description in this disclosure is centered on specific embodiments, this disclosure is not limited to such specific embodiments and should be understood to encompass all various modifications, equivalents, and / or substitutions of the various embodiments described in this disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0022] Image processing processors designed to generate video signals for high-resolution displays can be designed and mass-produced to produce video at a fixed resolution (e.g., 8K). For instance, since an image processing processor for an 8K resolution display is optimized to generate video with a maximum resolution of 8K, it is fundamentally difficult to process video for higher resolution displays (e.g., 9K). As display resolution increases, the image processing processor must process more pixels, requiring corresponding bandwidth, memory, and computational power. Therefore, if an image processing processor is designed for a specific resolution, it lacks the algorithms and processing power necessary to upgrade to higher resolutions, making it impossible to fully implement video at the new resolution. Even if GPU performance is enhanced and AI-based upscaling technology is added, fundamental hardware improvements tailored to the resolution difference are required. However, as high-resolution displays continue to develop, implementing a separate chip containing the image processing processor can waste hardware resources and ultimately increase the production cost of display devices.

[0023] FIG. 1 is a block diagram including hardware components of a display device according to one embodiment of the present disclosure.

[0024] A display device (101) according to one embodiment may include a system on chip (SoC) (110), a scaler (120), and a display (130) designed to perform one or more image processing operations on an input image. The system on chip (SoC) (110) may include at least a portion of a processor (CPU), a graphics processor (GPU), memory, a communication circuit, various image and signal processing units, or a power management circuit. Since the system on chip (SoC) (110) is a term expressing a hardware type and substantially performs the role of an image processing processor, it may be referred to as an image processing processor (110) for convenience of explanation in this disclosure. The scaler (120) is implemented as a separate chip from the system on chip (110) and may perform the role of increasing the horizontal size of an input image. For convenience of explanation in this disclosure, the scaler (120) may be referred to as a horizontal scaler or an H scaler below.

[0025] A display device (101) according to one embodiment can perform image processing on an input image to match the resolution of a display (130) through an image processing processor (110). For example, the image processing processor (110) can analyze an input image signal to perform scaling to match the display resolution or improve image quality by optimizing color or screen detail elements. The SoC (110) can be manufactured as a semiconductor integrated circuit (field-programmable gate array; FPGA) that includes a logic circuit optimized to generate an image for a display of a predetermined resolution according to a predetermined profile. For example, an image processing processor (110) in the form of an SoC can be mass-produced by optimizing it to generate an image for an NxM resolution display.

[0026] An SoC (110) according to one embodiment can generate an NxM size image by undergoing preprocessing processes such as noise removal on an input image and image processing processes such as color and detail processing. Although the SoC (110) is optimized to generate an image suitable for NxM resolution, it can increase the size of the image in the vertical direction without hardware modification. Based on the order of operations according to the raster scan direction, the SoC (110) can increase the vertical size of the image using a poly-phase scaler. The SoC (110) can perform additional operations to perform vertical scaling up on the NxM resolution image by using the line memory or DDR memory (double data rate memory) designed to generate the NxM resolution image as is.

[0027] According to one embodiment, the SoC (110) can generate an NxM resolution image, increase the size of the image by q in the vertical direction, and then transmit the Nx(M+q) resolution image to the scaler (120).

[0028] A scaler (120) according to one embodiment can increase the horizontal size of a received Nx(M+q) resolution image. The scaler (120) can be manufactured by applying a poly-phase method and can use only line memory since it supports scaling up in one direction (horizontal direction) of vertical or horizontal. The scaler (120) can be designed relatively simply than performing scaling up in vertical and horizontal directions and can increase the horizontal size of an input image by a predetermined size. In one embodiment, the scaler (120) can increase the size of an input image by p in the horizontal direction. The scaler (120) can increase the horizontal size of an Nx(M+q) resolution image by p to output an (N+p)x(M+q) resolution image. The scaler (120) can transmit the (N+p)x(M+q) resolution image to a display (130).

[0029] A display (130) according to one embodiment can receive an image of the maximum outputtable resolution and output an optimized image. The display (130) can be manufactured to output an image of (N+p)x(M+q) resolution. The display (130) can output an image of (N+p)x(M+q) resolution received from an H scaler (120).

[0030] According to one embodiment, a display device (101) may include a system on chip (SoC) (110) comprising an image processing processor that processes an input image to generate an image of a first resolution; a second scaler (120) that performs upscaling of the image; and a display (130) that outputs an image of a second resolution greater than the first resolution. A display device (101) can generate an image having the first resolution for a first input image through the system-on-chip (110), increase the size of the image in the vertical direction by the difference in vertical resolution values ​​between the first resolution and the second resolution using the first scaler of the system-on-chip (110), transmit it to the second scaler (120), increase the size of the image in the horizontal direction by the difference in horizontal resolution values ​​between the first resolution and the second resolution through the second scaler (120), transmit it to the display (130), and output the first input image converted to the second resolution size through the display (130).

[0031] According to one embodiment, the system-on-chip (110) may include the first scaler of the on-the-fly type, which performs an operation to increase the vertical size using a line memory buffer. The system-on-chip (110) may use the first scaler to increase the size of the image in the vertical direction by the difference between the vertical resolution values ​​of the first resolution and the second resolution for the first input image.

[0032] According to one embodiment, the system-on-chip (110) may change the raster size of the first input image according to the ratio of changing the vertical panel size of the first resolution of the first input image to the vertical panel size of the second resolution, and perform the operation of the first scaler using the line memory buffer based on the changed raster size and panel size.

[0033] According to one embodiment, the system-on-chip (110) completes one or more image processing operations for the first input image before generating an input image having a horizontal resolution value of the first resolution and a vertical resolution value of the second resolution for the first input image, and the one or more image processing operations include at least some of noise removal, contour removal, scaling for screen composition, image color adjustment, image detail adjustment, frame rate conversion, or on-screen display (OSD) mix, and the system-on-chip (110) may include a third scaler for scaling for screen composition that is separate from the first scaler.

[0034] According to one embodiment, the system-on-chip (110) may add an operation to increase the image size of the first input image according to the raster scan direction in the first scaler.

[0035] According to one embodiment, the first scaler of the system-on-chip (110) can increase the size of the image in a vertical direction with respect to the first input image according to a poly-phase method.

[0036] According to one embodiment, the H scaler (120) may be included in the display device (101) in the form of a chip or semiconductor integrated circuit (field-programmable gate array, FPGA) separate from the system-on-chip (110).

[0037] According to one embodiment, the second scaler (120) can increase the size of the image in the horizontal direction with respect to the first input image according to a poly-phase method.

[0038] According to one embodiment, the system-on-chip (110) performs one or more image processing operations on the first input image, and the one or more image processing operations on the first input image include at least some of noise removal, contour removal, scaling for screen composition, image color adjustment, image detail adjustment, frame rate conversion, or on-screen display (OSD) mix, and the system-on-chip (110) may further include an operation to generate an input image having a horizontal resolution value of the first resolution and a vertical resolution value of the second resolution for the first input image in a scaling operation for screen composition using the first scaler.

[0039] According to one embodiment, the system-on-chip (110) may perform an additional operation corresponding to the difference in vertical resolution values ​​between the first resolution and the second resolution in at least one operation related to the vertical direction of the first input image, for at least one operation corresponding to the scaling operation among one or more image processing operations for the first input image.

[0040] According to one embodiment, the system-on-chip (110) can perform an operation to increase the scale in the vertical direction for the first input image using a frame-unit DDR memory (double data rate memory) buffer in the first scaler.

[0041] FIG. 2 is an example of an upscaling process for an input image using a display device according to one embodiment of the present disclosure.

[0042] A display device according to one embodiment (e.g., the display device (101) of FIG. 1) can sequentially generate an input image of a predetermined size, perform vertical scaling up and horizontal scaling up, and then generate an image of the final resolution (N+p)x(M+q) of the display (e.g., the display (130) of FIG. 1). A display device (101) according to one embodiment can first generate an NxM resolution image, then sequentially increase it by a vertical scale of q and increase it by a horizontal scale of p. The display device (101) can change the raster size according to the increase in panel size during the process of generating an image for a high-resolution display. Referring to FIG. 2, the panel size is indicated by a dotted line, and the raster size is indicated by a solid line.

[0043] For example, when the display module size is 552x312 (pixels), the panel size and raster size of the display can be determined according to the number of modules as shown in Table 1.

[0044] RP_FRCRP_VSCLNext HSCLNxN module num10111213141414Output frame rate(Hz)120120120120120120120Panel Size(H)5,5206,0726,6247,1767,6807,6807,728Panel Size(V)3,1203,4323,7444,0564,3204,3684,368Rast Size(H)8,8008,8008,8008,8008,8008,7048,704Rast Size(V)4,5004,5004,5004,5004,5004,5504,550

[0045] With reference to FIG. 2 and Table 1 above, an example of generating an image for an 8.1K display (130) using an 8K SoC (110) and an H scaler (120) implemented as a separate chip is described. Referring to the first image (210), a display device (101) according to one embodiment can generate a first image (210) with a panel size of 7680x4320 and a raster size of 8800x4500 through frame rate conversion (FRC) to match the resolution (8K) according to the case where the number of modules is 14x14. Referring to the second image (220), a display device (101) according to one embodiment can increase the size of the vertical image to 7680x4368 using a first scaler (or referred to as a V scaler) within the SoC (110). V Scaler can use the existing line memory buffer as is by changing the raster size according to the scaffolding relationship between the panel size and the raster size. V Scaler can use an additional memory buffer if the panel size and the raster size do not match the proportional relationship. Since V Scaler performs vertical scaling through additional operations based on the raster scan direction, using the existing line memory buffer or an additional memory buffer does not require changing the computational process for memory access.

[0046] The V scaler can convert the raster size as the panel size increases. The raster size can be changed according to the change in panel size to include the horizontal / vertical blanks required during the computation process for the panel-sized image. In the V scaler, if the vertical size of the panel is increased by 48 lines, the raster vertical size (x) to be changed can be calculated as Equation 1 using the proportional relationship between the panel size and the raster size.

[0047]

[0048] According to mathematical formula 1, the raster vertical length can be 4550. The display device (101) can perform upscaling according to the on-the-fly scaler method. At this time, in order to use the line memory buffer during the upscaling operation, HxV must be maintained identically before and after the raster size change, and if the raster vertical length increases, the raster horizontal length decreases. From the initial raster size of 8800x4500, if the raster vertical length becomes 4550 due to vertical upscaling, the horizontal length can become 8704. The V scaler can increase the vertical panel size to 4368 to generate a second image (220) with a panel size of 7680x4368 and a raster size of 8704x4550. In one embodiment, if the raster size increase ratio does not match the increase in panel size, the operation may be performed using an additional memory buffer.

[0049] Referring to the third image (230), an electronic device (101) according to one embodiment can perform horizontal upscaling on an input image using a second scaler (e.g., the scaler (120) of FIG. 1) (hereinafter referred to as the H scaler). When an image (220) with a panel size of 7680x4368 is generated within the SoC (110), it can be transmitted to the H scaler (120) using a high-speed data transmission interface (e.g., V1X). The H scaler (120) can perform horizontal upscaling on the input image to match the resolution level of the display (130). Referring to FIG. 2, the panel width of the input image is 7680, and the H scaler (120) can increase the panel length to 7728. The H scaler (120) can perform horizontal upscaling using a line memory buffer, in which case a change in the last size is not required. The H scaler (120) can finally generate an image (230) with a panel size of 7728x4368 and a raster size of 8704x4550.

[0050] FIG. 3 is a flowchart illustrating a scale processing operation for a high-resolution display in a display device according to one embodiment of the present disclosure.

[0051] A display device according to one embodiment (e.g., the display device (101) of FIG. 1) can perform upscaling for a high-resolution display. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0052] In operation 310, the image processing processor (110) of the display device (101) according to one embodiment can receive an input image. The image processing processor (110) can receive the input image in various ways such as HDMI (high-definition multimedia interface), USB (universal serial bus), CODEC (compressor-decompressor), and OTT (over-the-top). The size (resolution) of the input image is variable. The image processing processor (110) can store the input image in DDR memory.

[0053] In operation 320, an image processing processor (110) according to one embodiment can generate an image of a first resolution (e.g., NxM). The image processing processor (110) can be designed to optimize bandwidth, memory, and computational power to generate an image of a predetermined resolution. That is, the SoC (110) can be implemented to efficiently utilize hardware resources based on a predetermined resolution. For example, the SoC (110) can be designed to include an image processing processor optimized to generate an 8K resolution image.

[0054] An image processing processor (110) according to one embodiment can generate an image having a first resolution by performing one or more image processing operations on an input image. In one embodiment, one or more image processing operations may include noise reduction, decontouring, scaling for screen composition, image color adjustment, image detail adjustment, frame rate conversion (FRC), and on-screen display (OSD) mixing for processing the quality of the input image. When the image processing is completed, the input image can be changed to an image of the first resolution.

[0055] In operation 330, the image processing processor (110) of the display device (101) according to one embodiment can increase the image resolution by q in the vertical direction using a first scaler (hereinafter referred to as a vertical scaler or V scaler).

[0056] The V scaler can increase the size of images by applying an on-the-fly scaler method using a line memory buffer and a poly-phase scaler method.

[0057] In one embodiment, the poly-phase method may refer to values ​​at regular intervals to adjust the scale pointers by a ratio when the positions between scale pointers do not fall into integers. For example, the poly-phase method may calculate the position of the scale pointer to be calculated as a decimal point relative to the input, thereby adjusting the reference weight of a certain number of taps (e.g., 8 taps or 12 taps). A certain number of taps may represent the number of pixels referenced before and after the current position of the scale pointer to be calculated.

[0058] The V scaler can generate an image of size Nx(M+q) by performing additional operations q times on an image of size NxM to increase the resolution. Since the image processing processor (110) processes operations in the raster scan direction, the line memory buffer can be used as is for additional operations in the vertical direction. The V scaler can increase the vertical panel size, change the raster vertical size according to the ratio before and after the change, and change the raster horizontal size so that the raster horizontal / vertical product before and after the change is constant. Through such raster size adjustment, the V scaler can simply upscale the vertical resolution by using the line memory buffer as is.

[0059] In operation 340, the image processing processor (110) of the display device (101) according to one embodiment can transmit the input image of the first resolution to an H scaler (120) after vertical upscaling. The image processing processor (110) can compress the input image of size Nx(M+q) for high-speed data transmission. The image processing processor (110) can transmit the compressed image to a separate H scaler (e.g., the scaler (120) of FIG. 1) through a high-speed data transmission interface (e.g., Vx1).

[0060] In operation 350, the H scaler (120) of the display device (101) according to one embodiment can increase the image resolution of an image of size Nx(M+q) by p in the horizontal direction. The H scaler (120) can be implemented in the form of a separate chip or an FGPA. By manufacturing the system-on-chip (110) including an image processing processor and the H scaler (120) as separate hardware modules, the usability of each module can be increased. Since the H scaler (120) performs only one-way upscaling of the input image, it can be simply designed to use only a line memory buffer. Like the V scaler, the H scaler (120) can upscale the resolution of the image by applying a polyphase scaler. The H scaler (120) can be designed to perform horizontal upscaling by the difference between the horizontal length (horizontal) of the first resolution that the SoC (110) can generate and the horizontal length of the second resolution of the display (e.g., the display (130) of FIG. 1).

[0061] In operation 360, the H scaler (120) of the display device (101) according to one embodiment can transmit an image of size (N+p)x(M+q) to the display (130). The H scaler (120) can transmit an input image of resolution (N+p)x(M+q) to the display (130) through a high-speed data transmission interface (e.g., Vx1). The display (130) can output a second resolution, and (N+p)x(M+q) can be the second resolution.

[0062] In operation 370, the display (130) of the display device (101) according to one embodiment can receive and output an input image having a second resolution. The display device (101) can increase the vertical resolution scale in the image processing processor (110) and increase the horizontal resolution scale using the H scaler (120) to finally generate an image of the maximum output resolution (second resolution) of the display (130). The display device (101) can receive and output an input image of size (N+p)x(M+q) from the H scaler (120).

[0063] FIG. 4 is a block diagram including a logic component including an on-the-fly scaler of a display device according to one embodiment of the present disclosure.

[0064] A display device according to one embodiment (e.g., the display device (101) of FIG. 1) includes an image processing processor (e.g., the image processing processor (110) of FIG. 1 or the system-on-chip (110)) for generating an image for a high-resolution display, and the image processing processor (110) may include one or more logic components. The image processing processor (110) may include an input unit (401), a preprocessing unit (PRE) (402), a scaler (SCALER) (403), a DP (display processor) / VE (video engine) (404), a frame rate conversion unit (FRC) (405), an OSD mixer (on screen display mix) (406), a V scaler (V scaler) (407), a compression unit (compression, Comp) (408), and a high-speed data transmission interface (Vx1) (409). The display device (101) may include a connection terminal (410) for transmitting an input image to an image processing processor (110), one or more memories (420, 430, 440), and an H scaler (120). FIG. 4 describes functional components for explaining the operation of a display device as an example. In addition to the components shown in FIG. 4, additional components may be included, at least one of the shown components may be omitted, or replaced with components that perform substantially the same function but are called by different names. Each component is divided into functional units, and multiple logical components may be executed at once, and a single logical component may be executed in several discontinuous stages.

[0065] A display device (101) according to one embodiment can receive input video through various connection terminals (410). For example, the connection terminal (410) may be an HDMI wired cable.

[0066] In one embodiment, the input unit (401) can receive an input image and store it in memory (e.g., DDR (420)). The DDR memory (420) is a memory capable of storing and processing large amounts of data, and can be used when data can be read from any location and processing of the entire image data or multiple frames is required.

[0067] In one embodiment, the preprocessing unit (PRE) (402) performs a preprocessing process on the input image, specifically, noise removal and contour removal to improve the image quality of the input image. The preprocessing unit (PRE) (402) can perform preprocessing operations by referencing the DDR (420) memory and performing calculations on a frame-by-frame basis.

[0068] In one embodiment, the scaler (403) can perform scaling adjustments for screen configuration (e.g., multi-view configuration). The scaler (403) can analyze the input image and perform scaling operations necessary for screen configuration. For example, the scaler (403) can upscale the image displayed in some areas of the screen or downscale the image displayed in other areas.

[0069] In one embodiment, the DP (display processor) / VE (video engine) (404) can improve the quality of the input image by analyzing the input image and performing color adjustment or detail processing of the image. The DP (404) can perform frame-by-frame computation processing by referring to the DDR memory (430). The DP (404) can generate an image of a finally determined resolution (first resolution) while performing various image processing.

[0070] In one embodiment, the frame rate conversion unit (FRC) (405) can convert the frame rate of the input video to a frame rate according to the display (e.g., the display (130) of FIG. 1). For example, the FRC (405) can adjust the frame rate through frame duplication, deletion, or interpolation operations. The frame rate conversion unit (405) can change the input video output from the DP (404) to 120Hz for high-resolution output if the input video is 60Hz.

[0071] In one embodiment, the OSD mixer (on screen display mix) (406) can process graphical user interface (GUI) elements that can be displayed on the display (130) along with the video. The OSD mixer (406) can mix various information, such as text, menus, icons, volume indicators, and channel numbers displayed at the top of the screen, with the input video so that it is displayed on the display (130). The OSD mixer (406) can perform frame-by-frame video processing by referring to the DDR memory (430). One or more video processing operations on the input video correspond to frame-by-frame operations and can be operated by referring to the DDR memory (420, 430).

[0072] In one embodiment, the V scaler (407) completes one or more image processing operations on the input image and can perform upscaling of the input image having a first resolution to the vertical length of the maximum resolution (second resolution) that the display (130) can support. Since the V scaler (407) does not require frame-by-frame operations on the input image, it can perform upscaling operations by referencing only the line memory (440). The V scaler (407) can add operations for increasing the vertical resolution to the raster scan direction operations while referencing the line memory buffer (440) using an on-the-fly scaler. For example, if the difference between the vertical resolution value of the first resolution and the vertical resolution value of the second resolution is q, the V scaler (407) can perform vertical upscaling by q on the input image.

[0073] In one embodiment, the compression unit (Comp) (408) can perform compression on the input image for high-speed data transmission. The operation of the compression unit (408) is independent of the resolution of the input image, and the vertical length increased by the V scaler (407) of the input image does not affect the compression operation.

[0074] In one embodiment, the high-speed data transmission interface (Vx1) (409) is an interface for rapidly transmitting high-resolution images (high-capacity data) to other components within the display device (101). The image processing processor (110) can transmit an input image to an H scaler (120) through the high-speed data transmission interface (409). At this time, the resolution of the input image transmitted has a panel with a horizontal length of the first resolution and a panel with a vertical length of the second resolution.

[0075] The H scaler (120) can receive an input image and perform horizontal upscaling. The H scaler (120) can adjust the horizontal scale of the input image by the horizontal length of the maximum resolution (second resolution) that the display (130) can support.

[0076] FIG. 5 is a block diagram including a logic component including a main / subscaler of a display device according to one embodiment of the present disclosure.

[0077] A display device according to one embodiment (e.g., the display device (101) of FIG. 1 or the display device (101) of FIG. 4) can perform vertical upscaling while generating an input image of a predetermined resolution. The display device (101) can generate an image for a display (130) that outputs a second resolution (e.g., (N+p)x(M+q)) larger than the first resolution (e.g., NxM) by using an SoC (110) that generates an image of a first resolution. FIG. 5 describes an embodiment in which the V scaler component of the display device of FIG. 4 described above is included in a scaler (403) that performs scale adjustment for screen configuration of an input image within an image processing processor (110), thereby performing a vertical upscaling operation of the first resolution. The logic component included in the image processing processor (110) of FIG. 5 can perform at least some of the operations of the logic component of the same name included in the image processing processor (110) of FIG. 4. The input section (401) and preprocessing section (402) performed prior to vertical upscaling can be considered substantially identical to those of FIG. 4.

[0078] An image processing processor (110) of a display device (101) according to one embodiment may include a scaler (503) that analyzes an input image and performs scale adjustment for screen composition. The image processing processor (110) may include one or more scalers, and each scaler may be referred to as a main / sub scaler. There may be one or more sub scalers. After performing scale adjustment for screen composition using the scaler (503), the image processing processor (110) may perform upscaling in the vertical direction by the difference in vertical resolution values ​​between the first resolution and the second resolution. The scaler (503) may perform upscaling operations by referring to a DDR memory (420) capable of frame-unit operations. The scaler (503) may increase the resolution by applying a poly-phase upscaling method. The scaler (503) may be divided into a main scaler that performs scale adjustment for screen composition and a sub scaler that increases the vertical scale to match the display resolution. The scaler (503) may further include a subscaler for various scale adjustments through analysis of the input image.

[0079] An image processing processor (110) according to one embodiment may include additional operations to perform processing on an input image of size Nx(M+q) in the next image processing operation as the vertical length of the first resolution of the input image increases while performing image processing on the input image. According to one embodiment, vertical operations of the operation process on the input image may be added by q in the operation of DP / VE (504), FRC (505), and OSD mix (506). For example, DP (504) may perform vertical operations of M+q in the operation process for performing color processing on an input image of size Nx(M+q). Since the image processing processor (110) performs operations on the input image based on the raster scan direction, the addition of vertical operations on the input image does not affect memory references, etc.

[0080] In the image processing processor (110) according to one embodiment, there is no need to consider additional operations for operations that are independent of the resolution size among operations performed after the operation of the scaler (503). For example, the compression unit (408) and the high-speed data transmission interface (Vx1) (409) can be performed substantially the same as in FIG. 4. The compression unit (408) and Vx1 (409) are steps applied after the operation of the V scaler (407) in FIG. 4 and are not affected by the increase in the vertical size of the input image.

[0081] An image processing processor (110) according to one embodiment can transmit an input image (Nx(M+q)) having a horizontal length of a first resolution and a vertical length of a second resolution to an H scaler (120) through a high-speed data transmission interface (Vx1) (409).

[0082] FIG. 6 is an example of a raster size change according to a panel size change in a display device according to one embodiment of the present disclosure.

[0083] According to one embodiment, a display device (e.g., the display device (101) of FIG. 1, the display device (101) of FIG. 4) can change the raster size according to the increase in panel size in order to perform an upscaling operation of the input device by referencing a line memory buffer using an on-the-fly scaler.

[0084] According to one embodiment, the display device (101) can increase the vertical size of the panel for a first resolution (e.g., 7680x4320 of 8K) by 48 lines. Referring to the first image (610), the panel size is 7680x4320, and the raster size (h rast xv rast) is 8800x4500. The display device (101) can increase the vertical size of the first image (610) by 48 lines to match the vertical length of the second resolution (e.g., 7728x4368 of 8.1K). In processing operations on the input image, a time interval is required to perform the next line operation after the operation for one line in the raster direction is actually finished. When all pixels are active, it is difficult to perform the next operation continuously, so a blank time is required, and the difference in size between the panel size and the raster size can create the blank time.

[0085] Since the display device (101) performs calculations in the raster scan direction, the raster size is made larger than the panel size, thereby inevitably including a horizontal blank (c+d) required for the calculation process of the input image. The display device (101) can maintain the blank (c+d) by adjusting the raster horizontal length when the raster vertical length changes according to the change in panel size, so that the line memory buffer can be used as is without changing the memory reference for V scaler operation. At this time, the vertical blank (a+b) does not affect the calculation process.

[0086] In the first image (610), the panel size can be increased to 7680x4368 as vertical upscaling of the panel size increases. The vertical size of the raster also increases in proportion to the increase in the vertical size of the panel and changes to 4550. At this time, in order to use the line memory buffer as is within the same clock, the vertical length of the raster size can be changed so that the horizontal / vertical product of the raster size before and after the raster size change is constant.

[0087] In the second image (620), h' can be changed to 8704 so that h*v is equal to h'*v'. Finally, the second image can have a panel size of 7680x4368 and a raster size of 8704x4550.

[0088] FIG. 7 is an example of a display device according to various types of high-resolution displays according to one embodiment of the present disclosure.

[0089] A display device according to one embodiment (e.g., the display device (101) of FIG. 1 or the display device (101) of FIG. 4) can generate an image for displaying a second resolution larger than the first resolution by using a system-on-chip (e.g., the SoC (110) of FIG. 1) that generates an image of the first resolution. The second resolution may be one or more and may be determined within the maximum bit size that can be computed by the SoC (110) (e.g., 13 bits for an 8K SoC). The second resolution may be computed within the same frequency while maintaining a constant horizontal / vertical product before and after a change in raster size. If the second resolution does not have a constant horizontal / vertical product before and after a change in raster size, the frequency for image processing of the input image may need to be increased. For example, the 8K SoC (110) may include an additional separate H scaler to generate images for high-resolution displays such as 7872x4416, 7740x4320, 7920x4410, 7728x4368.

[0090] According to one embodiment, the display device (101) can generate a display image that outputs an image of various resolutions using an SoC (110) that generates an image of a first resolution, and can configure a V scaler and an H scaler according to the output resolution of the display.

[0091] As an example, the first display device (710) includes an 8K SoC (711) that generates an 8K image and an 8K display (713), and can generate an 8K image (7680x4320) through the 8K SoC (711) and output it on the 8K display (713).

[0092] As an example, the second display device (720) may include a V scaler in an 8K SoC (721) that generates an 8K image, an H scaler (722), and an 8.1K display (713). The 8K SoC (721) can perform upscaling of the 8K image by 48 lines in the vertical direction through additional computation. The H scaler (722) can perform upscaling of the 8K image by 48 lines in the horizontal direction. An 8.1K image (7728x4368) can be generated through the 8K SoC (721) and the H scaler (722) and output on the 8.1K display (723).

[0093] As an example, a third display device (730) may include a V scaler in an 8K SoC (731) that generates an 8K image, an H scaler (732), and an 8.3K display (733). The 8K SoC (731) can perform upscaling by q lines in the vertical direction on the 8K image through additional computation. The H scaler (732) can perform upscaling by p lines in the horizontal direction. An 8.3K image can be generated through the 8K SoC (731) and the H scaler (732) and output on an 8.3K display (723).

[0094] The embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" each may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0095] The term “module” as used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0096] One embodiment of the present document may be implemented as software comprising one or more instructions stored in a storage medium readable by a machine. For example, the processor of the machine may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by a machine may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0097] According to one embodiment, the method according to the embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0098] According to one embodiment, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to one embodiment, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a display device, A system on chip (SoC) including an image processing processor that processes an input image to generate an image of a first resolution; A second scaler that performs upscaling of the image; and It includes a display that outputs an image of a second resolution larger than the first resolution, and After generating an image having the first resolution for a first input image through the system-on-chip, the size of the image is increased in the vertical direction by the difference in vertical resolution values ​​between the first resolution and the second resolution using the first scaler of the system-on-chip, and transmitted to the second scaler. Through the second scaler, the size of the image in the horizontal direction is increased for the first input image by the difference in horizontal resolution values ​​between the first resolution and the second resolution, and transmitted to the display, and A display device that outputs the first input image converted to the second resolution size through the above display.

2. In Paragraph 1, The above system-on-chip includes the first scaler of the on-the-fly type, which performs an operation to increase the vertical size using a line memory buffer, and A display device that uses the first scaler to increase the size of an image in the vertical direction by the difference in vertical resolution values ​​between the first resolution and the second resolution for the first input image.

3. In Paragraph 1 or 2, The system-on-chip changes the raster size of the first input image according to the ratio of changing the vertical panel size of the first resolution for the first input image to the vertical panel size of the second resolution, and A display device that performs the operation of the first scaler using the line memory buffer based on the above-mentioned changed raster size and panel size.

4. In any one of paragraphs 1 through 3, The system-on-chip completes one or more image processing operations for the first input image before generating an input image having a horizontal resolution value of the first resolution and a vertical resolution value of the second resolution for the first input image. The above one or more image processing operations include at least some of noise removal, contour removal, scaling for screen composition, image color adjustment, image detail adjustment, frame rate conversion, or on-screen display (OSD) mix, and A display device comprising a system-on-chip including a third scaler for scaling the screen configuration, which is separate from the first scaler.

5. In any one of paragraphs 1 through 4, The above system-on-chip is a display device that adds an operation to increase the image size of the first input image according to the raster scan direction in the first scaler.

6. In any one of paragraphs 1 through 5, A display device in which the first scaler increases the size of the image in a vertical direction with respect to the first input image according to a poly-phase method.

7. In any one of paragraphs 1 through 6, The display device, wherein the second scaler is included in the display device in the form of a chip or semiconductor integrated circuit (field-programmable gate array, FPGA) separate from the system-on-chip.

8. In any one of paragraphs 1 through 7, The above second scaler is a display device that increases the size of the image in the horizontal direction with respect to the first input image according to a poly-phase method.

9. In any one of paragraphs 1 through 3, The above system-on-chip performs one or more image processing operations on the first input image, and One or more image processing operations for the first input image include at least some of noise removal, contour removal, scaling for screen composition, image color adjustment, image detail adjustment, frame rate conversion, or on-screen display (OSD) mixing, and The above system-on-chip is a display device that further includes an operation of generating an input image having a horizontal resolution value of the first resolution and a vertical resolution value of the second resolution for the first input image in a scale adjustment operation for screen composition using the first scaler.

10. In Paragraph 9, A display device wherein the system-on-chip described above performs an additional operation corresponding to the difference in vertical resolution values ​​between the first resolution and the second resolution in at least one operation related to the vertical direction of the first input image, for at least one operation corresponding to the scaling operation among one or more image processing operations for the first input image.

11. In Paragraph 9, The above system-on-chip is a display device that performs an operation to increase the scale in the vertical direction for the first input image using a frame-unit DDR memory (double data rate memory) buffer in the first scaler.

12. A method for generating a high-resolution display image of a display device, An operation to generate an image having the first resolution by performing image processing on a first input image using an image processing processor that generates an image of the first resolution; An operation of increasing the size of the first input image in the vertical direction by q using the first scaler of the image processing processor and transmitting it to the second scaler through the first high-speed data transmission interface; An operation to increase the size of the first input image in the horizontal direction by p using the second scaler; The above second scaler performs the operation of transmitting the first input image to a display through a second high-speed data transmission interface; and A method comprising the operation of the display outputting the first input image of a second resolution increased by p in the horizontal direction and q in the vertical direction from the first resolution.

13. In Paragraph 12, The above image processing processor is included in the display device in the form of a system-on-chip (SoC) designed to generate an image of the first resolution, and A method in which the second scaler is included in the display device in the form of a chip or semiconductor integrated circuit (FPGA) separate from the system-on-chip.

14. In Paragraph 12 or 13, The operation of increasing the size of the first input image by q in the vertical direction using the first scaler of the image processing processor is a method of increasing the first input image by q in the vertical direction using a line memory buffer according to an on-the-fly method.

15. In any one of paragraphs 12 through 14, A method wherein the first scaler changes the raster size of the first input image according to the ratio of changing the vertical panel size of the first resolution to a panel size increased by q, and performs a scale-up operation using the line memory buffer based on the changed raster size and panel size.

Images