Image processing device, display device, image processing method, and image processing program

Abstract

In display devices where color shifts may occur due to the arrangement structure of subpixels, the present invention provides an image processing apparatus, a display device, an image processing method, and an image processing program that can effectively reduce fringing while maintaining the high-frequency components of the image. [Solution] The image processing device outputs image data to a display unit in which a single unit pixel is composed of multiple subpixels of different colors, and the centroid positions of each subpixel within the unit pixel are spaced apart from each other. The image processing device comprises a sampling value calculation unit, a fringe detection unit, and a subpixel value determination unit. The sampling value calculation unit assumes that the subpixel value of the first color is output at the second centroid position of the second color subpixel, based on the relationship between the pixel value of the first color subpixel included in the image data, the first centroid position of the first color subpixel, and the second centroid position of the second color subpixel adjacent to the first color, and generates a sampling value at the first centroid position of the first color subpixel based on this assumption. The fringe detection unit generates a fringe detection value indicating the amount of fringe in the second color subpixel based on the pixel value of the second color subpixel. The sub-pixel value determination unit determines the output pixel value of the first color sub-pixel by combining the original pixel value and the sampled value of the first color sub-pixel at a composite ratio such that the proportion of the sampled value increases as the amount of fringing indicated by the fringe detection value increases.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a display device, an image processing method, and an image processing program.

Background Art

[0002] In recent years, self-emitting displays such as organic EL (OLED: Organic Light Emitting Diode) panels and micro LED panels have been widely used as display devices for smartphones, computer monitors, etc. In a conventional liquid crystal display device (LCD), generally, a structure that extracts light from a backlight through a color filter is adopted, and each sub-pixel of red (R), green (G), and blue (B) that constitutes one pixel is regularly arranged in close proximity to each other (for example, stripe arrangement, etc.).

[0003] On the other hand, in a self-emitting display such as an OLED panel, due to various design reasons such as improving manufacturing yield, absorbing differences in the light emission efficiency of each color, equalizing the lifespan, and reducing the thermal influence between sub-pixels, the area of sub-pixels may be non-uniform, or the distance between sub-pixels may be physically separated and arranged (for example, S-Stripe arrangement, pentile arrangement, etc.).

[0004] In such a pixel structure, the light emission center of each sub-pixel may deviate significantly from the geometric center of the unit pixel to which the sub-pixel belongs. As a result, particularly when displaying the outline (edge) of characters or graphics, color separation may be visually recognized in a portion that should originally be mixed and recognized as white or a specific color, and there is a problem that image quality degradation called so-called fringe (color fringe, color bleeding, false color, or chromatic aberration) is likely to occur.

[0005] Conventionally, as a fringe removal technique for an LCD panel or a plasma display panel (PDP), a technique for reducing color deviation by performing interpolation processing (sampling) on an input signal is known (for example, refer to Patent Document 1). [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2005-331603 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, if the conventional fringe removal techniques described above are applied directly to OLED panels and the like, where the distance between subpixels is relatively large, the following problems arise. As mentioned above, the subpixels of an OLED panel are often located at a greater distance from the center of the unit pixel compared to the subpixels of an LCD panel. Therefore, if wide-area sampling (interpolation with smoothing) is performed to correct the positional misalignment of the subpixels, fine information contained in the image, i.e., high-frequency components, tends to be lost.

[0008] Specifically, there was a trade-off: applying strong correction to completely eliminate fringing blurred the outlines of characters and fine textures (reduced resolution), while maintaining resolution resulted in residual fringing. In other words, it was difficult to effectively reduce fringing while maintaining the high-frequency components (sharpness) of the image in display devices with spaced-out subpixels.

[0009] The present invention has been made in view of these circumstances, and aims to provide an image processing apparatus, a display device, an image processing method, and an image processing program that can effectively reduce fringing while maintaining the high-frequency components of an image in a display device where color shift may occur due to the arrangement structure of subpixels. [Means for solving the problem]

[0010] According to the present invention, an image processing apparatus with the following configuration is provided. [1] An image processing device that outputs image data to a display unit in which a single unit pixel is composed of multiple subpixels of different colors, and the centroid positions of each subpixel within the unit pixel are spaced apart from each other, comprising a sampling value calculation unit, a fringe detection unit, and a subpixel value determination unit, wherein the sampling value calculation unit determines the pixel value of a first color subpixel included in the image data, and based on the relationship between the first centroid position of the first color subpixel and the second centroid position of a second color subpixel adjacent to the first color, the first color An image processing apparatus that assumes that subpixel values ​​are output, generates a sampling value at the first centroid position of the subpixel of the first color based on this assumption, the fringe detection unit generates a fringe detection value indicating the amount of fringe in the subpixel of the second color based on the pixel value of the subpixel of the second color, and the subpixel value determination unit determines the output pixel value of the subpixel of the first color by combining the original pixel value of the subpixel of the first color and the sampling value at a composite ratio such that the proportion of the sampling value increases as the fringe detection value increases.

[0011] When determining the output value of a subpixel (e.g., blue), the image processing device processes it based on its relationship to the centroid position of a subpixel of a different color (e.g., red) located at a physically distant position. Specifically, it performs adaptive synthesis processing using the following procedure. 1. Estimation of values ​​at the ideal position (sampling value calculation): Assuming that the pixel data of the first subpixel of the first color (target for correction, e.g., blue) is output from the second centroid position of the adjacent second color (reference, e.g., red), the ideal pixel value (sampling value) at the first centroid position of the first color is calculated based on the relationship between the first centroid position of the first color and the second centroid position of the second color. This generates correction data to eliminate positional misalignment between colors. 2. Edge component intensity determination (fringe detection): Simultaneously, a fringe detection value is generated based on the difference between the pixel value at the centroid position of the second color (high frequency component) and the estimated value of the second color at the centroid position of the first color (low frequency component). This serves as an indicator to determine whether it is an edge area where color bleeding is easily noticeable or a flat area where correction is not necessary. 3. Adaptive synthesis (sub-pixel value determination): The "original pixel value" of the first color and the "sampling value" corrected for positional shift are synthesized. In this process, the synthesis ratio is dynamically adjusted so that the proportion of the sampling value is increased when the fringe detection value (fringe amount) is large, and the original pixel value is prioritized when it is small. The above configuration solves the conventional problem of "reduced resolution due to uniform correction" and simultaneously achieves the following effects. 1. Effective removal of fringe (color bleeding): In areas where high-frequency components (amount of fringe) are strong and fringe is easily visible, such as the outlines of letters and shapes, sample values ​​corrected for positional shifts are preferentially used, resulting in spatially consistent colors and effective removal of color bleeding. 2. Preservation of high-frequency components (sharpness) of the image: In flat or textured areas where fringing is not noticeable, the effect of interpolation (including sampling and smoothing) is reduced, and the original pixel values ​​are prioritized. This prevents the entire image from becoming blurred due to correction processing, and preserves fine details and resolution without loss.

[0012] The following are examples of various embodiments of the present invention. The embodiments shown below can be combined with each other. [2] The image processing apparatus according to [1] or [2], wherein the fringe detection unit generates the fringe detection value indicating the amount of fringe based on the relationship between the pixel value of the subpixel of the second color and the centroid position of the subpixel of the first color and the second color. [3] The image processing apparatus according to [1] or [2], wherein the sampling value calculation unit performs the sampling in only one direction, either horizontal or vertical, for each subpixel. [4] The image processing apparatus according to any one of [1] to [3], wherein the sampling value calculation unit further generates a second sampling value based on the centroid position of a third color subpixel different from the second color subpixel, the fringe detection unit further generates a second fringe detection value indicating a second fringe amount in the third color subpixel based on the pixel value of the third color subpixel, and the subpixel value determination unit compares the fringe amount indicated by the fringe detection value with the second fringe amount indicated by the second fringe detection value, selects a sampling value generated based on the subpixel of the color having a larger fringe amount, and uses it to determine the output pixel value. [5] The image processing apparatus according to any one of [1] to [4], wherein the centroid position of each subpixel is the centroid of the output amount with respect to the physical output area of ​​each subpixel. [6] The image processing apparatus according to any one of [1] to [5], wherein the subpixel of the second color used as a reference in the sampling value calculation unit is the subpixel that is closest in distance from the centroids of the other subpixels among the subpixels constituting the unit pixel, or the subpixel that is closest to the centroid of the unit pixel. A display device comprising an image processing device described in any of [7][1] to [6], and a display unit that displays based on the output pixel values ​​output from the image processing device. [8] An image processing method for outputting image data to a display unit in which a single unit pixel is composed of a plurality of subpixels of different colors, and the centroid positions of each subpixel within the unit pixel are spaced apart from each other, comprising the steps of: assuming that the subpixel value of the first color is output at the second centroid position of the second color subpixel based on the relationship between the pixel value of the first color subpixel included in the image data and the first centroid position of the first color subpixel and the second centroid position of the second color subpixel adjacent to the first color, and generating a sampling value at the first centroid position of the first color subpixel based on this assumption; generating a fringe detection value indicating the amount of fringe in the second color subpixel based on the pixel value of the second color subpixel; and determining the output pixel value of the first color subpixel by combining the original pixel value of the first color subpixel and the sampling value at a synthesis ratio such that the proportion of the sampling value increases as the fringe detection value increases. [9] An image processing program that causes a computer to perform the image processing method described in [8].

[10] An image processing apparatus that outputs image data to a display unit in which a single unit pixel is composed of subpixels of multiple colors, and the centroid positions of each subpixel within the unit pixel are spaced apart from each other, comprising a sampling value calculation unit, a fringe detection unit, and a subpixel value determination unit, wherein the sampling value calculation unit generates a sampling value corresponding to the centroid position of the first color subpixel based on the relationship between the pixel value of the first color subpixel included in the image data and the centroid positions of the first and second color subpixels, the fringe detection unit generates a fringe detection value indicating the amount of fringe in the second color subpixel based on the pixel value of the second color subpixel and the relationship between the centroid positions of the first and second color subpixels, and the subpixel value determination unit determines the output pixel value of the first color subpixel by combining the original pixel value of the first color subpixel and the sampling value at a composite ratio such that the proportion of the sampling value increases as the fringe detection value increases.

[11] The fringe detection unit calculates a low-frequency component based on the pixel value of the sub-pixels of the second color and the relationship of the centroid positions of the sub-pixels of the first color and the second color, and generates the fringe detection value indicating the fringe amount based on the pixel value of the sub-pixels of the second color and the low-frequency component, the image processing apparatus according to any one of [1] to [6],

[10] .

[12] The fringe detection unit calculates a high-frequency component from the absolute value of the difference between the pixel value of the sub-pixels of the second color and the low-frequency component of the second color, and uses the high-frequency component as the fringe detection value indicating the fringe amount, the image processing apparatus according to any one of [1] to [6],

[10] . [Effect of the Invention]

[0013] According to the present invention, in a display device in which color misregistration may occur due to the arrangement structure of sub-pixels, fringes can be effectively reduced while maintaining the high-frequency components of an image. [Brief Description of the Drawings]

[0014] [Figure 1] It is a diagram showing an example of the arrangement structure of sub-pixels in a display unit, FIG. 1A shows a stripe arrangement, and FIG. 1B shows an example of a special arrangement (S-Stripe arrangement, etc.) assumed in the present embodiment. [Figure 2] It is a block diagram showing the overall configuration of a display device provided with the image processing apparatus according to the present embodiment. [Figure 3] It is a diagram for explaining the concept of the arrangement of sub-pixels and sampling processing, FIG. 3A shows an example of the definition of the centroid coordinates of each sub-pixel within a unit pixel, and FIG. 3B shows the relationship between the horizontal centroid distance and the sampling reference position between adjacent pixels. [Figure 4] It is a diagram showing a specific example of the change in pixel value by image processing, FIG. 4A shows the pixel value of the input image (original image) before processing, and FIG. 4B shows the pixel value of the output image after processing. [Figure 5] It is a flowchart showing an example of the image processing procedure executed by the image processing apparatus according to the present embodiment. [Figure 6]It is a flowchart showing an image processing procedure (power saving mode) with reduced calculation amount according to Modification 21.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the embodiments shown below are examples for embodying the technical idea of the present invention, and there is no intention to limit the present invention to the following. Each feature shown in the embodiments can be combined with each other. Also, an invention can be established independently for each feature.

[0016] [1. Details of Problems and Generation Mechanisms in the Background Art] First, the generation mechanism of "fringe (color bleeding)", which is the problem to be solved by the present invention, will be described in detail in association with the pixel structure of the display device and human visual characteristics. FIG. 1 is a diagram showing an example of the arrangement structure of sub-pixels in the display unit 5 (display device). FIG. 1A shows a stripe arrangement (Stripe Arrangement) generally adopted in a conventional liquid crystal display device (LCD) or the like.

[0017] In the stripe arrangement shown in FIG. 1A, each sub-pixel of red (Red: R), green (Green: G), and blue (Blue: B) is arranged in a vertically long rectangular shape in a regular horizontal row without gaps. Conventionally, it has been considered that the centroid shift can be ignored in this arrangement. However, due to the recent rise of self-emitting panels, the cases where the centroids of sub-pixels are arranged further apart within the unit pixel region are increasing. Furthermore, even in the stripe arrangement that is common in liquid crystal (LCD), as a result of adjusting the aperture ratio of each color for high brightness and power saving, the centroid position of the sub-pixel (for example, green) arranged in the middle does not coincide with the geometric center of the unit pixel due to the difference in the light emission efficiency (difference in aperture area) of the sub-pixels. That is, even in the stripe arrangement, there are cases where the centroids are strictly separated.

[0018] Figure 1B shows examples of special arrangements (e.g., S-Stripe arrangement, PenTile arrangement, diamond arrangement, etc.) used in self-emissive display devices such as organic EL (OLED) panels and microLED panels, which are the main targets of this embodiment. In self-emissive devices such as OLED panels, there is a need to compensate for manufacturing process constraints (e.g., the limit of aperture accuracy of the fine metal mask when depositing organic light-emitting materials) and differences in luminous efficiency (brightness lifetime) of each color light-emitting material to achieve uniform lifetime.

[0019] For these reasons, as shown in Figure 1B, the shape and area of ​​subpixels are often non-uniform for each color (for example, blue pixels with a short lifespan are larger, while green pixels with high visual sensitivity are smaller), and the distance (pitch) between subpixels is often physically large. In such a pixel structure, the physical light emission point of each subpixel, i.e., the centroid of the light energy, is significantly deviated (shifted) from the geometric center coordinates of the unit pixel to which that subpixel should logically belong.

[0020] For example, in the example in Figure 1B, the red (R) and blue (B) subpixels are positioned diagonally, horizontally, and vertically apart from the green (G) subpixels. When attempting to display images with extreme differences in brightness and high contrast, such as black and white text or the outlines of shapes, on such a display device, visual problems arise.

[0021] In areas where red, green, and blue light should be visually integrated (mixed) to produce "white" or "gray," the separation of the light-emitting points for each color results in incomplete mixing, causing each color to appear spatially separated. Specifically, this can manifest as a reddish tint bleeding around the edges of white text, or a bluish tint extending beyond the text. This is a phenomenon known as "fringing" (color fringing, color bleeding, false color, or chromatic aberration), a major cause of reduced text readability, especially on high-resolution displays.

[0022] One possible method for removing this fringing is to apply digital image processing to the input image signal and spatially sample (interpolate) the pixel values ​​to correct the color misalignment. For example, this technique involves mixing the blue component of the pixel to the left of a blue subpixel that is physically shifted to the right, thereby generating pixel values ​​that computationally shift the center of gravity of the emission to the left (to the correct position, or the position of another color).

[0023] However, simply applying strong sampling (position correction) uniformly across the entire screen can lead to serious side effects. Sampling (interpolation) is essentially a process of weighting and averaging the values ​​of surrounding pixels and mixing them together (low-pass filtering), which can blur and smooth the edges of the image. In particular, with uniform interpolation processes like those in existing patented technologies, if the centroid distance between subpixels is large, it is necessary to shift the pixel information by a large amount (mix strongly) by that distance. As a result, a new problem has emerged: the sampling process itself, which is intended to improve fringing, is causing the loss of high-frequency components in the image, leading to a more pronounced degradation of image quality (blurring).

[0024] In other words, in display devices where subpixels are spaced apart, "fringe removal (correction of positional misalignment)" and "maintaining resolution (preservation of high-frequency components)" are in a trade-off relationship. If you try to eliminate the fringe, the image becomes blurry, and if you try to make the image sharp, the fringe remains. Achieving both of these at a high level has been extremely difficult with conventional technology. To solve this problem, the image processing apparatus 3 according to the present invention employs a configuration that adaptively switches processing for each pixel according to the local characteristics of the image, in particular "edge strength (amount of high-frequency components)".

[0025] [2. Overall configuration of the display device] Figure 2 is a block diagram showing the overall configuration of a display device 1 equipped with an image processing device 3 according to this embodiment. The display device 1 comprises an image processing device 3 and a display unit 5 (display device). The display device 1 is an electronic device such as a personal computer, tablet terminal, smartphone, television receiver, in-car display, smartwatch, head-mounted display (HMD), or digital signage.

[0026] The display unit 5 is a display panel (display) for displaying images. In this embodiment, the display unit 5 is a device (display device) in which the centroid positions of multiple color subpixels constituting a single unit pixel are spaced apart from each other. Specifically, the display unit 5 can be an organic EL (OLED) panel, a microLED panel using inorganic LEDs, a quantum dot (QD) display, or an LCD panel (liquid crystal display device). Each unit pixel is composed of, for example, three subpixels of red (second color), green (third color), and blue (first color), but the types and number of colors are not limited to these, and may be a four-color configuration including white (W) and yellow (Y).

[0027] The image processing device 3 is a device that applies image processing to the input image data (video signal) to reduce fringing, and outputs the processed image data to the display unit 5. The image processing device 3 comprises, as its internal functional blocks, an image data acquisition unit 10, a sampling value calculation unit 12, a fringe detection unit 14, and a sub-pixel value determination unit 16. These are functional blocks and do not necessarily need to be physically independent circuits.

[0028] The image processing device 3 can be implemented as a display driver IC (DDIC) that drives the display unit 5, a timing controller (T-CON), or an image processing unit (IPU) within an application processor (AP). It may also be implemented as a dedicated hardware circuit (ASIC or FPGA), or as a software module that functions when a processor such as a CPU, GPU, or DSP executes an image processing program stored in memory.

[0029] Here, we clarify the correspondence between each constituent element of the patent claims and each functional block in this embodiment. The "image processing device" described in claim 1, etc., corresponds to the image processing device 3 in Figure 2. The "sampling value calculation unit" described in claim 1, etc., corresponds to the sampling value calculation unit 12 in Figure 2. The "fringe detection unit" described in claim 1, etc., corresponds to the fringe detection unit 14 in Figure 2. The "sub-pixel value determination unit" described in claim 1, etc., corresponds to the sub-pixel value determination unit 16 in Figure 2. The "display unit" described in claim 1, etc., corresponds to the display unit 5 (display device) in Figure 2. The “display device” described in claim 9 corresponds to the display device 1 in Figure 2.

[0030] The following describes the detailed functions of each part. The image data acquisition unit 10 is an interface for acquiring video signals (image data) input from an external host system. This image data is digital data containing the R, G, and B gradation values ​​(pixel values) of each unit pixel. The image data acquisition unit 10 temporarily holds the acquired image data in an internal memory such as a line buffer and supplies it to the subsequent sampling value calculation unit 12, fringe detection unit 14, and sub-pixel value determination unit 16 in synchronization with the pixel clock.

[0031] The sampling value calculation unit 12 is a calculation unit that generates sampling values ​​for a subpixel of a certain color (e.g., blue: first color) contained in the image data, based on the relationship between the first centroid position of the first color subpixel and the second centroid position of a subpixel of another color (e.g., red: second color) adjacent to the first color. Specifically, it assumes that the subpixel value of the first color is output at the second centroid position of the second color subpixel, and calculates the sampling value at the first centroid position of the first color subpixel based on this assumption. This process generates data to shift the phase of the blue image to the adjacent red position.

[0032] The fringe detection unit 14 is a calculation unit that, based on the pixel value of a subpixel of another reference color (for example, red: second color), detects whether the image is changing abruptly at that location, that is, whether it is a high-frequency region where fringing (color bleeding) is likely to occur, and generates a "fringe detection value" indicating the degree of the change. This fringe detection value is a numerical representation of edge sharpness and contrast strength, and serves as an indicator used to determine the weighting (blending ratio) during the synthesis process described later.

[0033] The sub-pixel value determination unit 16 is a calculation unit that determines the final output pixel value by combining the "sampling value (position-corrected value)" calculated by the sampling value calculation unit 12 and the "original pixel value (uncorrected value)" included in the original image data. At this time, the combination ratio is dynamically changed based on the fringe detection value output from the fringe detection unit 14. Specifically, in areas with strong high-frequency components (where fringing is noticeable), the proportion of the sampling value is increased to actively correct the positional shift, and in flat areas, the proportion of the original pixel value is increased to maintain the sharpness of the original image. This makes it possible to pinpoint and remove fringing only where necessary, while maintaining the resolution in other areas.

[0034] [3. Pixel Coordinate Systems and the Concept of Sampling] Next, the coordinate system and basic concepts of the image processing that are prerequisites for this embodiment will be explained in detail with reference to Figure 3. Figure 3A shows an example of the definition of the centroid coordinates of each subpixel within a unit pixel. Here, the region of one unit pixel is normalized and represented in a Cartesian coordinate system with its geometric center as the origin (0,0). In this example, it is assumed that the centroid position of each subpixel is defined as follows. Green: (-0.1, -0.25) ... Located in the upper left when viewed from the center. Red: (-0.3, 0.2) - Located in the lower left when viewed from the center. • Blue: (0.25, 0.25) ... Located in the lower right when viewed from the center.

[0035] In this context, the "center of gravity" is preferably not limited to the physical center of the subpixel, but rather refers to the center of gravity of the luminescence distribution (amount of light emitted) in the subpixel's light-emitting region (aperture). This is because the human eye perceives the center of gravity of light energy as the position of the point light source. As can be seen from Figure 3A, red (R) and blue (B) are separated by a large distance of 0.55 (more than half the pixel pitch) in the horizontal direction, with distances of "-0.3" and "0.25" respectively. This is the main cause of color shift.

[0036] Figure 3B shows in detail the horizontal positional relationship of each subpixel when three adjacent pixels (P10, P11, and P12 from left to right) are lined up. When the image processing device 3 determines the value of a certain subpixel (the one to be corrected), it samples using the position of a "subpixel of another color (reference)" that is physically distant as a target. The numbers shown at the bottom of Figure 3B indicate the physical distance (pitch ratio) between the centroids of each subpixel. In the actual interpolation calculation, the inverse ratio of these distances is used as a coefficient.

[0037] For example, consider the case where the blue subpixel of pixel P11 in the center of Figure 3B is the target of correction. The horizontal centroid coordinate of the blue subpixel of P11 is 0.25 (hereinafter referred to as the first centroid position). On the other hand, the centroid coordinate of the red subpixel within the same pixel P11 is -0.3 (hereinafter referred to as the second centroid position). Here, in order to match the blue emission position to the red emission position (align the phase), it is necessary to estimate "what value would it be if the blue subpixel were outputting from the second centroid position of the red (coordinate -0.3)?"

[0038] In this case, the interpolation calculation uses two pixel data points that straddle the target position. As shown in Figure 3B, in order to determine the value of the blue (first centroid position 0.25) of P11 which is to be corrected, this embodiment utilizes the "arrangement of the second centroid position of the red, which serves as the phase reference". Specifically, the blue (first centroid position 0.25) of P11 is located between the red (second centroid position -0.3) of P11 and the red (second centroid position 0.7) of P12 which is to its right.

[0039] Therefore, the interpolation coefficient is determined based on the distance relationship between the centroid positions of these two red points and the centroid position of the blue point of P11. Distance L1: From the blue point (0.25) of P11 to the red point (-0.3) of P11 on the left = 0.55 • Distance L2: From the blue point (0.25) at P11 to the red point (0.7) at P12 on the right = 0.45 In linear interpolation, the coefficients are set so that the closer pixel has a stronger influence (they are inversely proportional in terms of distance). Since the red pixels on the P12 side (right side) are closer, a larger weight (0.55) is given to the corresponding blue pixel value at P12, and a smaller weight (0.45) is given to the blue pixel value at the farther P11. This allows for the calculation of the optimal estimated blue value based on the arrangement of the red centroids.

[0040] Here, we will elaborate on the selection of the second color subpixel (red in this example) used as the reference in the sampling value calculation unit 12. It is preferable to select the subpixel closest in distance from the centroids of the other subpixels (the first color being corrected) among the multiple subpixels constituting the unit pixel, or the subpixel closest to the centroid of the unit pixel. By fixing the subpixel used as the correction reference (partner) to the one closest in physical distance or close to the pixel center, the phase shift (position shift) of the image due to the correction can be minimized. Using a distant pixel as the reference carries the risk of the image appearing shifted, but by selecting the closest pixel, only the fringe can be removed while maintaining a natural positional relationship.

[0041] In this embodiment, one of the features is that instead of sampling both colors (e.g., red and blue) that cause fringing, only "one of them" is sampled depending on the situation. For example, if the positions of red and blue are misaligned, instead of moving both and aligning them in the middle, one (e.g., red) is fixed and only the other (e.g., blue) is sampled to match the position of red. This minimizes image modification due to interpolation processing and prevents a decrease in image quality.

[0042] [4. Technical meaning and detailed calculation process of each calculation step] From here, we will explain the "technical meaning (why such calculations are performed)" and the "detailed calculation process" using specific numerical values ​​for the three main steps performed by the image processing device 3 (sampling value generation, fringe detection value generation, and output pixel value determination). To make the explanation more concrete, we will assume a 3x3 pixel block (P00~P22) as shown in Figure 4A and explain the process of determining the output value of the blue subpixel of pixel P11, which is at the center of the block, as an example.

[0043] (Prerequisite) As shown in Figure 4A, in the input image (original image), only the central pixel P11 emits "white (R=255, G=255, B=255)", while all other surrounding pixels (P00~P10, P12~P22) are "black (0,0,0)". This is a so-called impulse input, where there is one white dot in a black background. Such an image has extremely strong high-frequency components (fringe) and sharp edges, making the positional shifts between colors most noticeable and representing a typical worst-case scenario where fringe is likely to occur. We will verify how the present invention works with this input.

[0044] <Step 1: Generating Sampling Values> The sampling value calculation unit 12 generates a "sampling value" by estimating the pixel value of the subpixel of the first color (target for correction: blue) at the centroid position of the subpixel of the second color (reference: red) contained in the image data.

[0045] (Technical meaning: Why do we perform this calculation?) The fundamental cause of fringing (color bleeding) is that red, green, and blue light, which should ideally be emitted in the same location, are actually emitting light in physically different locations (at distant coordinates, as shown in Figure 3A). Therefore, we simulate the question, "If the blue subpixel were in the same location (coordinate) as the red subpixel, what brightness (pixel value) should it emit?" This is the sampling value.

[0046] This calculation spatially "shifts" the blue light emission information to the position of the red light. This virtually aligns the centroids of the physically separated red and blue light emission, fundamentally eliminating color bleeding caused by phase (position) misalignment. This is a type of image resampling process, and its technical significance lies in improving apparent resolution by performing positional correction at the sub-pixel level.

[0047] (Specific calculation process) First, for the blue (B) element of P11, which is the target of correction, we calculate the value if it were at the red (R) position (sampling value 1) and the value if it were at the green (G) position (sampling value 2). We use the interpolation coefficient based on the distance relationship shown in Figure 3B. In this embodiment, considering the balance between computational load and image quality, we show an example of execution using the most basic interpolation method, "linear interpolation."

[0048] (1-A: Calculation of the blue sampling value (S_BR) at the red position) Here, the parameter "S_BR" means "the sampled value of blue at the red position". As mentioned above, we assume that the blue pixel value of P11 is output at the second centroid position of red (-0.3) of pixel P11, and the blue pixel value of P12 is output at the second centroid position of red (-0.3) of P12. Based on this assumption and the relationship with the first centroid position of blue in P11, we perform sampling by linear interpolation at the first centroid position of blue in P11. Specifically, the distance from the second centroid position of red in P11 to the first centroid position of blue in P11 is 0.55, and the distance from the second centroid position of red in P12 to the first centroid position of blue in P11 is 0.45. Therefore, we perform a weighted average using the inverse ratio of the distances. That is, we assign a larger weight (0.55) to the closer P12 and a smaller weight (0.45) to the farther P11. S_BR = P11.B × 0.45 + P12.B × 0.55 Here, P11.B is the blue value of the current pixel, and P12.B is the blue value of the pixel to its right.

[0049] (Supplementary explanation: Regarding the basis for setting the coefficients) Here, let's elaborate on the assignment of coefficients "0.45" and "0.55". In linear interpolation, there is a principle that "the closer a pixel is to the target position, the stronger its influence (greater weight) on its pixel value." In this embodiment, the distance from the second centroid position of the blue pixel in P11 (target: coordinate 0.25) to the second centroid position of the red pixel in P11 on the left (source 1: coordinate -0.3) is "0.55", and the distance from the second centroid position of the red pixel in P12 on the right (source 2: coordinate 0.7) is "0.45". Since the second centroid position of the red pixel in P12 is closer, a larger coefficient (0.55) is assigned to the blue pixel value in P12 assumed to be at that position, and a smaller coefficient (0.45) is assigned to the blue pixel value in P11 assumed to be at the second centroid position of the red pixel in P11 which is further away. In this way, by making the ratio of the distances between the first and second centroid positions and the ratio of the coefficients in an "inverse ratio (cross multiplication)" relationship, accurate luminance values ​​corresponding to the centroid position are calculated.

[0050] Substitute specific values. The blue on P11 is 255, and the blue on P12 is 0. S_BR = 255 × 0.45 + 0 × 0.55 S_BR = 114.75 + 0 S_BR=114.75 This value "114.75" is an estimate of the ideal brightness value at the original centroid position of the blue color, based on the blue pixel value considered to be at the centroid position of the red color. The reason it is smaller than the original value of 255 is that it was affected by the adjacent pixel P12 (pixel value 0) due to interpolation calculation (weighted average), and physically, it is the result of some of the light energy held by the original pixel P11 being distributed (shifted) to the leftmost pixel (P10 side) due to phase shift correction.

[0051] (1-B: Calculation of the blue sampling value (S_BG) at the green position) The parameter "S_BG" means "the sample value of blue at the green position." Similarly, assuming that blue is output at the centroid position of the green pixel P11 (-0.1) and blue is output at the centroid position of the green pixel P12, sampling is performed by linear interpolation at the centroid position of the blue pixel P11. The distance from the centroid position of the green pixel P11 to the centroid position of the blue pixel P11 is 0.35, and the distance from the centroid position of the green pixel P12 to the centroid position of the blue pixel P11 is 0.65. Therefore, a weighted average is performed using the inverse ratio of the distances. That is, a larger weight (0.65) is assigned to the closer P11 and a smaller weight (0.35) is assigned to the farther P12. S_BG = P11.B × 0.65 + P12.B × 0.35 S_BG = 255 × 0.65 + 0 × 0.35 S_BG = 165.75 + 0 S_BG = 165.75 This is the blue sampling value estimated at the green position.

[0052] <Step 2: Generating fringe detection values> The fringe detection unit 14 generates a "fringe detection value" that indicates the amount of fringe (strength of high-frequency components) in a subpixel of the second color (reference: red) based on the pixel value of that subpixel. Similarly, it generates a value for the third color (reference: green).

[0053] (Technical meaning: Why examine the reference "red" instead of the "blue" being corrected?) The most technically important point here is that when determining the presence or absence of fringing (edge ​​strength), the pixel values ​​of the "red (or green)" image used as the alignment reference are used, rather than the pixel values ​​of the "blue" image being corrected. This is because this method is a process of "aligning blue to the position of red," and whether or not edges exist in the red image, which is the "target (reference) for alignment," determines whether correction is necessary.

[0054] For example, if the reference red image is flat (such as a blue sky), there will be no visual difference even if the blue is not precisely aligned to it. However, if the red image has sharp edges (such as the outline of text), and the blue is not precisely superimposed on the position of those edges, it will be clearly visible as a color shift. Therefore, the "edge intensity of the reference color (red)" is the indicator of the risk of fringing. Furthermore, a bandpass filter method is used to extract edge components by subtracting low-frequency components (blurred image) from the original image.

[0055] (Specific calculation process) Fringe (high-frequency components, edge strength) can be extracted by subtracting the "low-frequency components (blurred values)" from the "original values." This is referred to as the fringe detection value.

[0056] (2-A: Calculation of fringe detection value (F_R) based on red) The parameter "F_R" represents the "red-based fringe detection value (edge ​​strength)." First, we calculate the "red low-frequency component (LowPass_R)" which serves as the basis for this detection value. This corresponds to the "smooth red value" obtained by blurring the edges of the image. The following formula is used for calculation. LowPass_R=P11.R×0.45+P12.R×0.55 Here, P11.R is the red value of the current pixel, and P12.R is the red value of the pixel to its right.

[0057] (Technical meaning of the formula: Why use the same coefficients as in sampling?) In this calculation, the same coefficients (0.45 and 0.55) used in the blue sampling calculation earlier are applied to the red values ​​at P11 and P12. This is no mere coincidence. The value calculated by this weighted average estimates "what the brightness would be like if red were at the centroid position of blue (the low-frequency component of red)." In other words, fringe detection takes the difference between "the estimated red value at the blue position (low frequency)" and "the value at the original red position (high frequency)." By perfectly matching the "spatial positional relationship (phase)" in the correction calculation and the detection calculation, edge detection that accurately reflects the amount of positional displacement becomes possible.

[0058] Substitute specific values. The red value for P11 is 255, and the red value for P12 is 0. LowPass_R = 255 × 0.45 + 0 × 0.55 LowPass_R=114.75 This corresponds to a value that slightly blurs the red image. Next, we calculate the absolute difference between the original red value (P11.R) and the calculated low-frequency component (LowPass_R). F_R = |Original pixel value - Low-frequency component| F_R=|P11.R-LowPass_R| F_R=|255-114.75| F_R=140.25 This large value of "140.25" indicates that at this location (the red and blue positions on P11), the pixel value deviates significantly from 255 to 114.75, meaning there is a "very strong edge (sharp change)." In other words, it is judged to be a "dangerous location where fringing is likely to occur."

[0059] (2-B: Calculation of fringe detection value (F_G) based on the green standard) The parameter "F_G" represents the "fringe detection value relative to the green." Similarly, the low-frequency component of the green (LowPass_G) and the fringe detection value (F_G) are calculated using a distance ratio of 0.35:0.65. LowPass_G=P11.G×0.65+P12.G×0.35 LowPass_G = 255 × 0.65 + 0 × 0.35 LowPass_G=165.75 F_G=|P11.G-LowPass_G| F_G=|255-165.75| F_G=89.25

[0060] <Step 3: Determining output pixel values ​​(adaptive synthesis)> The sub-pixel value determination unit 16 combines the original pixel value of the first color (blue) with the sampled value obtained in step 1. In this process, the larger the fringe detection value (fringe amount, strength of high-frequency components) obtained in step 2, the higher the proportion of the sampled value. If there are multiple reference colors (red, green), the one with the larger fringe detection value is adopted as the reference.

[0061] (Technical meaning: Why do we perform this calculation?) This is the core of the present invention and the logic for resolving the trade-off between "fringe removal" and "resolution maintenance." First, let's explain the selection of the reference color (red or green). Comparing the fringe detection value for red (140.25) and the fringe detection value for green (89.25), the value for red is higher. This means that "red edges are sharper (more drastic)." Since fringes are more noticeable the sharper the edges, correcting the blue color to match the more noticeable red edges results in a higher visual fringe removal effect. Therefore, red is selected as the reference color.

[0062] Next, we will explain blending. Here, the fringe detection value (edge ​​intensity) is used directly as the blending ratio. Case A: When the fringe detection value is large (when there is a strong edge). →The color shift is very noticeable, making the text blurry and difficult to read. In this case, even if it means sacrificing some resolution (the sharpness of the original pixel values), correcting the positional shift should be the top priority. Therefore, the percentage of the "sampling value" after positional correction should be increased (close to 100%) to forcibly shift the blue to the red position. Case B: When the fringe detection value is small (flat areas or gradual changes) →Color shift is barely noticeable to the human eye. If sampling (position correction) is forced in this state, only the calculated "blur" remains, and the image quality deteriorates. Therefore, in this case, the proportion of the "original pixel value" is increased (close to 100%) to maintain the sharpness of the original image.

[0063] Thus, this composite calculation is performed to smoothly (analogously) switch on each pixel whether to "correct or not" based on the local features (edge ​​strength) of the image. Switching in a binary manner (0 or 1) would result in unnatural boundaries (artifacts) at the transition points, so alpha blending, which provides a continuous change, is optimal.

[0064] (Specific calculation process) First, we select a base color. Since F_R(140.25) > F_G(89.25), we adopt "Red" as the base color. Next, we calculate the blend ratio (α: alpha). Since the fringe detection value (F_R) takes values ​​between 0 and 255, we divide it by 255 to normalize it to a range of 0.0 to 1.0. Formula:α=F_R / 255 Calculation: α=140.25 / 255 Result: α = 0.55 This means "apply position correction (sampling value) with 55% strength." The remaining 45% retains the original pixel value.

[0065] Finally, the output value is calculated using the following alpha blending formula. Formula: Output value = Original pixel value × (1-α) + Sampling value × α (1-α) is the proportion that prioritizes the original pixel values, and α is the proportion that prioritizes the sampled values. Substitute specific values. The original blue value (P11.B) is 255, and the sampled value (S_BR) is 114.75. Output value = 255 × (1 - 0.55) + 114.75 × 0.55 Output value = 255 × 0.45 + 114.75 × 0.55 Output value = 114.75 + 63.1125 Output value ≈ 177.86

[0066] As a result, the blue value of pixel P11 changes (decreases) from the original "255" to "177.86". On the other hand, if we perform the same calculation on the adjacent pixel P10, the blue value of P10 increases from "0" to approximately "77.14" (see P10 in Figure 4B). In other words, some of the blue light that was in P11 has moved towards P10. From a physical perspective, this means that the center of gravity of the blue light in P11 has moved to the left (towards the direction of the red light), and this eliminates the positional misalignment with the red light.

[0067] (Example of vertical processing: Green on page 11) Next, as an example of how to handle vertical misalignment, we will explain the process for determining the output value of the green subpixel of pixel P11. As shown in Figure 3A, the centroid of green (y=-0.25) is located at the top of the pixel, while red (y=0.2) and blue (y=0.25) are located at the bottom. Therefore, to align the green with the positions of red and blue, vertical interpolation calculation is necessary. Here, we show an example where the calculation is performed based on the positional relationship between P11 (luminance 255) and the adjacent pixel above it, P01 (luminance 0).

[0068] <Step 1V: Generate vertical sampling values> First, we calculate the sampling values ​​when the green (G) element, which is the target of correction, is vertically shifted to the position of the reference red element and the position of the blue element. (1) Sampling based on the centroid of red (S_GR) The ratio of vertical distances derived from the center of gravity is set to "0.55:0.45". S_GR=255(P11)×0.55+0(P01)×0.45=140.25 (2) Sampling based on the centroid of the blue (S_GB) The vertical distance ratio to the blue position is set to "0.5:0.5". S_GB=255(P11)×0.5+0(P01)×0.5=127.5 These values ​​are estimated luminance values ​​when the green light source is aligned with the height of the red and blue light sources, respectively.

[0069] <Step 2V: Generate vertical fringe detection values> Next, vertical fringes (high-frequency components) are detected for the reference red and blue signals. (1) Red-based fringe detection (F_Rv) The vertical low-frequency component at the red position of P11 is calculated using the same weighting as the sampling. Low-frequency component = 255(P11) × 0.55 + 0(P01) × 0.45 = 140.25 F_Rv=|255-140.25|=114.75 (2) Blue-based fringe detection (F_Bv) Similarly, the low-frequency component in the vertical direction at the blue position is calculated. Low-frequency component = 255(P11) × 0.5 + 0(P01) × 0.5 = 127.5 F_Bv=|255-127.5|=127.5

[0070] <Step 3V: Determining the output pixel value> Finally, select a base color and perform the blending process. (1) Selection of a reference color We compare the vertical fringe detection values. Since the red fringe (114.75) < the blue fringe (127.5), we adopt the "Blue" fringe, which has a stronger edge (larger value), as the baseline. As a result, the sampling value used for synthesis is determined to be "S_GB(127.5)" based on the blue fringe.

[0071] (2) Blending The blend ratio is calculated using the adopted blue-based fringe detection value (F_Bv). Blend ratio α = F_Bv / 255 α = 127.5 / 255 = 0.5 Using this blend ratio, the original green value (255) and the blue-based sampling value (S_GB=127.5) are combined. Combined value = 255 × (1 - 0.5) + 127.5 × 0.5 Combined value = 127.5 + 63.75 = 191.25

[0072] As a result, the output pixel value for the green of pixel P11 is "191.25". By performing the same calculation for the blue of the adjacent pixel P10 and the green of pixel P21, the final output result shown in Figure 4B is obtained. For example, the change in the green value of P21 from "0" to "63.75" indicates that some of the green energy from P11 was moved to the lower P21 through vertical interpolation.

[0073] [5. Image Processing Procedure (Flowchart)] Next, with reference to Figure 5, the overall processing flow performed by the image processing device 3 will be described. Figure 5 is a flowchart showing the procedure of the image processing method according to this embodiment. This processing is performed sequentially or in parallel for each unit pixel or each subpixel of the input image.

[0074] (Step S100: Image data acquisition step) First, the image data acquisition unit 10 acquires a video signal (image data) input from an external source. This image data is digital data that includes the RGB gradation values ​​of each pixel.

[0075] (Step S110: Sampling value generation step) Next, the sampling value calculation unit 12 calculates a sampling value for the first color to be corrected (for example, blue) based on its relationship to the centroid positions of several reference candidate colors (for example, red and green). Specifically, it assumes that the blue pixel data is output from the centroid position of red and calculates an estimated value at the original centroid position of blue. Similarly, it also calculates an estimated value when green is used as the reference.

[0076] (Step S120: Fringe detection value generation step) The fringe detection unit 14 calculates the amount of fringe (edge ​​intensity) for each candidate color (red, green) used as a reference, and generates fringe detection values. Specifically, it quantifies the edge intensity of red by calculating the absolute difference between the estimated value of red at the centroid position of blue (low-frequency component) and the pixel value at the actual centroid position of red (high-frequency component). The same procedure is followed for green.

[0077] (Step S130: Output pixel value determination step) The sub-pixel value determination unit 16 first selects the color with the largest fringe detection value from among several candidate colors as the "optimal reference color." For example, if the red edge is stronger than the green edge, red is determined to be the reference. Next, the sampling value corresponding to the selected reference color is combined with the original pixel value. At this time, the fringe detection value of the selected reference color is used as the blend ratio (alpha value). The larger the fringe detection value, the higher the contribution rate of the sampling value, and the smaller the value, the higher the contribution rate of the original pixel value. As a result, adaptive pixel values ​​are determined that prioritize position correction in edge areas and resolution in flat areas.

[0078] (Step S140: Output step) The determined output pixel values ​​for each subpixel are output to the display unit 5 and displayed on the screen. Through this process, a clear image without color shift is displayed.

[0079] [6. Configuration and operation / effect of each claim corresponding to the embodiment] The following provides a detailed explanation of which configuration of this embodiment each claim of the invention described in the patent claims corresponds to and what functions and effects it produces.

[0080] (Claim 1) In this embodiment, the image processing device 3 comprises a sampling value calculation unit 12, a fringe detection unit 14, and a sub-pixel value determination unit 16. The sampling value calculation unit 12 generates a sampling value based on the relationship between the pixel value of a first-color sub-pixel included in the image data and the first centroid position of the first-color sub-pixel and the second centroid position of a second-color sub-pixel adjacent to the first color. Specifically, it assumes that the first-color sub-pixel value is output at the second centroid position of the second-color sub-pixel, and calculates the sampling value at the first centroid position of the first-color sub-pixel based on this assumption. The fringe detection unit 14 generates a fringe detection value indicating the amount of fringe in the second-color sub-pixel based on the pixel value of the second-color sub-pixel. The sub-pixel value determination unit 16 determines the output pixel value of the first-color sub-pixel by combining the original pixel value and the sampling value of the first-color sub-pixel at a composite ratio such that the proportion of the sampling value increases as the fringe detection value increases. According to the above configuration, the calculation logic becomes clear: assuming that the first color is ideally output at the pixel position of the second color (reference) which is physically separated, the luminance value at the first centroid position of the first color is estimated. This makes it possible to obtain sampling values ​​with high accuracy to theoretically correct the phase shift between subpixels. Furthermore, in display devices with physically separated subpixels, by switching the strength of the correction for each pixel according to the edge strength, it is possible to effectively correct only the areas where fringing is noticeable while maintaining the resolution (high-frequency components) in other areas.

[0081] (Claim 2) In this embodiment, the fringe detection unit 14 generates fringe detection values ​​based on the relationship between the pixel value of the second color subpixel and the centroid positions of the first and second color subpixels. With the above configuration, edge determination can be performed using the same "distance relationship (phase relationship) between centroids" used in sampling (position correction). As a result, the spatial position to be corrected and the spatial position for detecting edges strictly coincide, preventing erroneous corrections and correction omissions due to deviations in the detection position, and enabling highly accurate adaptive processing.

[0082] (Claim 3) In this embodiment, the sampling value calculation unit 12 performs sampling in only one direction, either horizontal or vertical, for each subpixel. With this configuration, compared to performing two-dimensional interpolation, it is possible to prevent the image from becoming excessively blurred and to maintain maximum resolution. Furthermore, depending on the subpixel array structure, the shift in a specific direction (e.g., the horizontal direction) is often dominant, and by limiting the processing to that direction, efficient and effective correction becomes possible.

[0083] (Claim 4) In this embodiment, the sampling value calculation unit 12 and the fringe detection unit 14 perform calculations based not only on the centroid position of the second color (e.g., red) but also on the centroid position of the third color (e.g., green), and the sub-pixel value determination unit 16 selects based on the color with the larger fringe detection value. With the above configuration, the correction criterion can be dynamically switched according to the color that is causing the most color shift, depending on the content of the image (e.g., whether it is the edge of a red object or the edge of a green object). As a result, the optimal fringe removal effect can always be obtained for any image with any color scheme.

[0084] (Claim 5) In this embodiment, the centroid position is the centroid of the output amount relative to the physical output area of ​​each subpixel. With the above configuration, even if the shape of the pixel is complex, such as L-shaped or rectangular, the calculation is performed based on the central position of the output (light) that the human eye actually perceives, making it possible to perform visually more accurate and natural correction.

[0085] (Claim 6) In this embodiment, the reference subpixel is the subpixel closest in distance to the centroids of multiple other subpixels, or the subpixel closest to the centroid of a unit pixel. With the above configuration, by setting the position used as the reference for correction to a physically close position, the amount of image movement (phase shift) due to correction can be minimized. This prevents the harmful effects of distortion or displacement of the entire image due to correction.

[0086] (Claim 7) In this embodiment, the display device 1 comprises an image processing device 3 and a display unit 5. With this configuration, the benefits of high image quality can be enjoyed not only as a standalone image processing chip, but also as a final product such as a smartphone or monitor.

[0087] (Claim 8) In this embodiment, the image processing method includes a sampling value generation step, a fringe detection value generation step, and an output pixel value determination step. In particular, in the sampling value generation step, it is assumed that the subpixel value of the first color is output at the second centroid position of the second color, based on the relationship between the first centroid position of the first color subpixel and the second centroid position of the adjacent second color subpixel, and a sampling value at the first centroid position is generated based on this assumption. With the above configuration, the technology of the present invention can be implemented in various forms, not limited to hardware implementation, but also including software processing and cloud-based processing.

[0088] (Claim 9) In this embodiment, the image processing program is a program that causes a computer (e.g., a processor) to execute the above-described image processing method. With the above configuration, it is possible to retrospectively add the image quality enhancement function of the present invention to an existing device by updating the firmware or the like.

[0089] (Claim 10) In this embodiment, the fringe detection unit 14 generates fringe detection values ​​not only based on the pixel values ​​of the reference second color, but also based on the "relationship of centroid positions" between the first and second color subpixels that are to be corrected. Specifically, it calculates the estimated luminance (low-frequency component) of the second color at the centroid position of the first color using the same geometric relationship (inverse ratio of distances, etc.) that the sampling value calculation unit 12 uses for position correction. With the above configuration, instead of simply detecting the edges of the image, it is possible to evaluate how the luminance of the reference color (second color) changes at the physical position of the correction target (first color) while perfectly matching the spatial phase. This makes it possible to accurately extract only the fringe components caused by physical positional misalignment, eliminates the risk of mistakenly correcting areas without positional misalignment, and enables more precise and artifact-free high-image-quality enhancement.

[0090] (Claim 11) In this embodiment, the fringe detection unit 14 calculates the low-frequency component based on the relationship between the pixel value of the second color and the centroid positions of the first and second colors. With the above configuration, by performing a weighted average using the same centroid distance (weighting coefficient) as in the sampling calculation, it is possible to accurately generate an estimated value of the second color at the centroid position of the first color (ideal low-frequency component). This makes it possible to acquire a reference signal for extracting the fringe component in subsequent processing without phase shift.

[0091] (Claim 12) In this embodiment, the fringe detection unit 14 calculates the high-frequency component from the absolute value of the difference between the pixel value of the second color (original value) and the calculated low-frequency component, and uses this as the fringe detection value. With the above configuration, it is possible to accurately extract visually prominent edge components while using computationally intensive methods such as low-pass filtering and its difference. This enables highly accurate adaptive processing according to the local characteristics of the image without increasing the circuit size.

[0092] [7. Variant] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of its technical idea. The following are possible modifications to ensure that the scope of the present invention is not limited to these embodiments.

[0093] (Variation 1: Diversification of interpolation algorithms) The sampling in the sampling value calculation unit 12 is not limited to the linear interpolation shown in the above embodiment. Depending on the application, required image quality, and available computing resources, sampling may be performed using cubic interpolation, Lanczos interpolation, spline interpolation, Gaussian interpolation, or Mitchell-Netravari interpolation. For example, using cubic interpolation (bicubic method) increases the computational load, but it can improve the accuracy of the estimated values ​​and produce a more natural corrected image with less jaggedness. Also, using a Lanczos filter can suppress ringing while preserving high-frequency components. The algorithm may be dynamically switched according to the processing power and conditions of the device, such as using a simple interpolation similar to the nearest neighbor method when hardware resources are limited, or using higher-order interpolation when a high-performance GPU is available.

[0094] (Variation 2: Division of labor between hardware and software) In addition to a configuration in which all functions of the image processing device 3 are implemented with a dedicated circuit (ASIC), it is also possible to configure it so that some or all of the functions are executed by software on a general-purpose CPU, GPU, or DSP. For example, a hybrid configuration is conceivable in which high-speed fringe detection (difference calculation) is performed by a hardware line buffer and difference circuit, while sampling calculations requiring complex floating-point operations are performed by software.

[0095] (Variation 3: Changing the color space) Although the embodiment was described using the RGB color space, processing may also be performed using the YUV (luminance-chrominance) color space, YCbCr color space, or XYZ color space. For example, by utilizing the characteristic that the human eye is sensitive to changes in luminance (Y) but insensitive to changes in chrominance (U, V), it is possible to reduce the amount of computation while maintaining visual image quality by performing high-precision position correction only on the luminance component (Y) and performing simple correction or no correction at all on the chrominance components (U, V).

[0096] (Modification 4: Thresholding for fringe detection) A "threshold" may be set for fringe detection values. For example, if the detected value is below a certain level (noise floor), it is considered sensor noise or compression noise rather than image edge noise, and the blend ratio is fixed at 0 (no correction). Only if the value is above that level is correction performed according to that value. This prevents the phenomenon of image graininess (increased noise perception) caused by the emphasis of minute noise in flat areas. This threshold may be dynamically changed according to the camera's ISO sensitivity and ambient brightness.

[0097] (Modification 5: Nonlinear transformation processing for fringe detection) A nonlinear transformation process may be applied to the fringe detection values. For example, a sigmoid function can be applied to the detection values ​​to determine the blending ratio. This allows processing to be suppressed where the detection values ​​are small, treating them as noise, and then strong correction can be applied where the detection values ​​exceed a certain amount, treating them as fringes. Unlike simple thresholding (ON / OFF), the correction amount changes continuously, preventing unnaturalness at the application boundary of the correction and achieving continuous yet adaptive correction.

[0098] (Variation 6: Application of 2D sampling) In the embodiment, unidirectional sampling (horizontal or vertical) was recommended, but depending on the arrangement of subpixels, diagonal misalignment may be significant. In such cases, two-dimensional sampling (two-dimensional filtering) including both horizontal and vertical directions, or diagonal directions, may be performed. This allows for a uniform correction effect to be obtained for edges in all directions.

[0099] (Variation 7: Application to different panel arrangements) Although an S-Stripe array is used as an example in this embodiment, the present invention is not limited to this. It can also be applied to displays with sub-pixel arrays such as pentile arrays, diamond arrays, delta arrays, or hexagonal close-packed arrays. By pre-measuring or simulating the centroid coordinates in each array and adjusting the interpolation coefficients accordingly, the present invention can be applied to any display, including LCD panels (liquid crystal display devices), not just self-emissive displays such as organic EL displays.

[0100] (Modification 8: Consideration of the time direction) This method can be applied not only to still images but also to videos. In this case, information from the previous and subsequent frames (correlation in the temporal direction) can be used in addition to the information from the current frame. For example, by performing temporal smoothing (temporal filtering) by referring to the sampling value or fringe detection value of the previous frame, it becomes possible to achieve stable correction that suppresses the flicker and temporal noise specific to videos.

[0101] (Variation 9: Processing switching for each region) Instead of processing the entire screen uniformly, the processing can be varied for each area within the screen. For example, strong correction can be applied to areas displaying text information (text areas) or UI components, while weaker correction can be applied to areas displaying natural images (photos or videos). This allows for the provision of optimal image quality tailored to the characteristics of the content. Area determination can be performed using metadata, layer information, or image analysis.

[0102] (Modification 10: Use of a Lookup Table (LUT)) Complex calculations in sampling and blending ratio calculations can be performed using a lookup table (LUT) instead of an arithmetic circuit. By pre-calculating output values ​​corresponding to input pixel values ​​and distance coefficients and storing them in memory, and simply referencing them during execution, processing speed can be increased and power consumption reduced.

[0103] (Modification 11: Advanced edge direction detection) In the fringe detection unit 14, not only the amount of fringe (strength of high-frequency components) but also the "direction of the edge (horizontal, vertical, or diagonal)" may be detected. The correction direction is adaptively switched according to the orientation of the edge, such as performing vertical correction when the edge runs horizontally and horizontal correction when the edge runs vertically. This suppresses the occurrence of jagged edges (staircase-like jaggedness) in diagonal lines and enables smoother contour representation.

[0104] (Modification 12: Interoperability with power saving mode) In mobile devices and the like, when the battery level is low or in power-saving mode, the processing of the present invention may be skipped (bypassed) to reduce power consumption, and the processing may be enabled only in normal mode or high-definition mode. Alternatively, a switch may be made to use simple interpolation (linear interpolation) in power-saving mode and high-precision interpolation (cubic interpolation) in high-definition mode.

[0105] (Example 13: Adjustment by user settings) The sensitivity (gain) of fringe detection and the upper limit of the blend ratio could be adjusted by the user to suit their preferences and eyesight. For example, a customization feature could be provided that strengthens the correction for users who want to see text clearly, and weakens the correction for users who prefer a more natural image quality.

[0106] (Variation 14: Application to different colors) In the embodiment, an example was shown in which blue (B) is corrected based on red (R) or green (G), but conversely, red (R) or green (G) may be corrected based on blue (B). Depending on the pixel arrangement and the characteristics of the light-emitting material, the most visually effective color combination can be selected.

[0107] (Variation 15: Application of machine learning) Machine learning (such as deep learning) may be used to determine sampling coefficients and fringe detection parameters. By training the system with a large number of pairs of high-quality and degraded images, it is possible to automatically acquire optimal correction parameters for complex textures that were difficult to achieve with conventional methods, thereby further improving image quality.

[0108] (Modification 16: Control according to viewing distance) The device can use its built-in camera or sensors to measure the distance to the viewer and adjust the processing intensity accordingly. For example, when the viewer is close to the screen, fringing is more noticeable, so the correction can be strengthened, while when they are further away, the correction can be weakened.

[0109] (Variation 17: Integration with anti-burn function) This can be integrated with pixel shifting (a process that periodically shifts the entire pixel slightly) used to prevent burn-in in OLEDs. If the center of gravity changes due to pixel shifting, updating the sampling coefficient in real time to match this change will maintain an optimal fringe removal state at all times.

[0110] (Variation 18: In combination with subpixel rendering) This method can be used in conjunction with existing subpixel rendering techniques (such as font anti-aliasing). By applying device-specific physical position correction using this method to the results of subpixel rendering performed by the OS, higher-resolution character display becomes possible.

[0111] (Modification 19: Correction according to brightness level) The appearance of fringing also changes depending on the screen's brightness. Therefore, the strength and parameters of the correction may be dynamically adjusted according to the APL (Average Pixel Level) and ambient light sensor values. For example, since fringing is more noticeable at high brightness levels, the correction could be strengthened.

[0112] (Modification 20: Application to RGB stripe array) The present invention is effective not only for special pixel arrangements but also for display devices with subpixels arranged in a general R, G, B stripe pattern. In particular, in high-resolution LCD panels, the subpixel shape may be designed to improve the aperture ratio, or the effective center of emission may appear shifted due to viewing angle characteristics. The application of the present invention improves the sharpness of characters. In a stripe arrangement, for example, if the central green (G) is used as a reference, color shift can be eliminated by sampling the left red (R) or right blue (B) in one direction (horizontally) to align with the green position. Specifically, sampling calculations are performed such that the red (R) is shifted to the right to align with the green (G) position, and the blue (B) is shifted to the left to align with the green (G) position. In this way, by sampling only one of the two colors that improve fringing, either blue (B) or red (R), in one direction, fringing can be effectively suppressed while maintaining resolution.

[0113] (Modification 21: Logic for selecting reference pixels based on distance) In this embodiment, the reference color was selected based on the fringe detection value (edge ​​intensity), but the reference centroid position may be selected based on the physical arrangement (distance) of the subpixels. The sampling value calculation unit 12 may limit the selection of which subpixel to use as the reference for the centroid position to the selection of a subpixel that satisfies any of the following conditions. (Condition 1) Select a pixel that is physically closer to the subpixel to be corrected than other subpixels. (Condition 2) Select the pixel closest to the geometric center of the unit pixel. For example, if the color to be corrected is "blue," and within a single pixel, "green" is located closer to "blue" than "red" (Condition 1), then "green" is always used as the reference for selection. Alternatively, if "green" is in the center of the single pixel (Condition 2), then "green" is used as the reference for selection. This minimizes the amount of pixel movement due to sampling, and physically minimizes image blurring associated with interpolation.

[0114] (Variation 22: Optimization of processing steps and reduction of computational complexity) The flowchart of the embodiment shown in Figure 5 employs a procedure in which sampling values ​​and fringe detection values ​​are generated for all candidate colors (e.g., red and green), and then the optimal one is selected or combined. This procedure is advantageous because it allows for flexible and advanced implementation, such as further recombining the combined results based on multiple calculated reference points as needed (e.g., blending the red-based result and the green-based result 50:50). On the other hand, if reducing the amount of computation is the priority, the processing order may be changed (flowchart 2). Specifically, first, only the fringe detection values ​​for multiple candidate colors are calculated, and these are compared to determine the "optimal reference color" with the largest fringe detection value. Then, sampling values ​​are calculated only for that determined reference color, and the combination process is executed. According to this procedure, sampling calculations for reference colors that are not ultimately adopted can be omitted (skipped), making it possible to significantly reduce the computational load and power consumption of the image processing device 3 while maintaining image quality.

[0115] Figure 6 is a flowchart showing the image processing procedure according to a modified version of this embodiment. In the procedure shown in Figure 5, sampling values ​​and fringe detection values ​​were generated for all candidate colors before selection, but in this modified version, the procedure is optimized to reduce the computational load.

[0116] (Step S200: Fringe detection step for candidate reference point color) First, the fringe detection unit 14 calculates only the fringe detection values ​​(F_R, F_G) for all candidate reference colors (e.g., red, green). At this stage, it does not calculate the output sampling value (interpolation value) for the correction target (first color).

[0117] (Step S210: Step to determine the reference color) Next, the sub-pixel value determination unit 16 compares the calculated fringe detection values ​​and determines the color with the largest value (strongest edge) as the "optimal reference color". For example, if F_R > F_G, red is selected as the reference color.

[0118] (Step S220: Sampling step for the selected color) The sampling value calculation unit 12 calculates the sampling value of the first color corresponding to the "reference color" selected in step S210. It does not perform the sampling calculation for the first color corresponding to the color that was not selected (for example, green) (it skips this step).

[0119] (Step S230: Output pixel value determination step) Finally, the sub-pixel value determination unit 16 uses the sampled value of the selected reference color and the fringe detection value to perform alpha blending with the original pixel value and determine the output value. This procedure eliminates the need to perform sampling calculations for reference colors that are not ultimately adopted, thereby significantly reducing the computational load and power consumption of the image processing device 3 while maintaining image quality. This is particularly effective in battery-powered mobile devices. [Explanation of Symbols]

[0120] 1:Display device 3: Image processing device 5: Display unit (display device) 10: Image data acquisition unit 12: Sampling Value Calculation Unit 14: Fringe detection area 16: Sub-pixel value determination unit P00~P22: Unit pixels

Claims

1. An image processing device that outputs image data to a display unit in which a single unit pixel is composed of multiple subpixels of different colors, and the centroid positions of each subpixel within the unit pixel are spaced apart from each other, It comprises a sampling value calculation unit, a fringe detection unit, and a sub-pixel value determination unit. The sampling value calculation unit assumes that the subpixel value of the first color is output at the second centroid position of the second color subpixel, based on the relationship between the pixel value of the first color subpixel included in the image data, the first centroid position of the first color subpixel, and the second centroid position of the second color subpixel adjacent to the first color, and generates a sampling value at the first centroid position of the first color subpixel based on this assumption. The fringe detection unit generates a fringe detection value indicating the amount of fringe in the second color subpixel based on the relationship between the pixel value of the second color subpixel and the centroid positions of the first and second color subpixels. The sub-pixel value determination unit determines the output pixel value of the sub-pixel of the first color by combining the original pixel value of the sub-pixel of the first color and the sampling value at a composite ratio such that the proportion of the sampling value increases as the fringe detection value increases. Image processing device.

2. The sampling value calculation unit generates the sampling value for each subpixel in only one direction, either horizontally or vertically. The image processing apparatus according to claim 1.

3. The sampling value calculation unit assumes that the subpixel value of the first color is output at the third centroid position of the third color subpixel, based on the relationship between the pixel value of the first color subpixel included in the image data and the first centroid position of the first color subpixel and the third centroid position of the third color subpixel which is different from the second color subpixel, and further generates a second sampling value at the first centroid position of the first color subpixel based on this assumption. The fringe detection unit further generates a second fringe detection value indicating a second amount of fringe in the third color subpixel, based on the relationship between the pixel value of the third color subpixel and the centroid positions of the first color and the third color subpixels. The sub-pixel value determination unit compares the amount of fringe indicated by the fringe detection value with the second amount of fringe indicated by the second fringe detection value, selects a sampling value generated based on the sub-pixel of the color with the larger amount of fringe, and uses it to determine the output pixel value. The image processing apparatus according to claim 1.

4. The centroid position of each subpixel is the centroid of the output amount relative to the physical output area of ​​each subpixel. The image processing apparatus according to claim 1.

5. The subpixel of the second color used as a reference in the sampling value calculation unit is the subpixel that is closest in distance to the centroid of the other subpixels among the multiple subpixels constituting the unit pixel, or the subpixel that is closest to the centroid of the unit pixel. The image processing apparatus according to claim 1.

6. The image processing apparatus according to claim 1, A display unit that displays based on the output pixel values ​​output from the image processing device, A display device equipped with the following features.

7. An image processing method for outputting image data to a display unit in which a single unit pixel is composed of multiple subpixels of different colors, and the centroid positions of each subpixel within the unit pixel are spaced apart from each other, The steps include: assuming that the subpixel value of the first color is output at the second centroid position of the second color subpixel based on the relationship between the pixel value of the first color subpixel included in the image data, the first centroid position of the first color subpixel, and the second centroid position of the second color subpixel adjacent to the first color, and generating a sampling value at the first centroid position of the first color subpixel based on this assumption; A step of generating a fringe detection value indicating the amount of fringe in the subpixel of the second color based on the relationship between the pixel value of the subpixel of the second color and the centroid position of the subpixels of the first and second colors, The steps include determining the output pixel value of the subpixel of the first color by combining the original pixel value of the subpixel of the first color and the sampling value at a synthesis ratio such that the proportion of the sampling value increases as the fringe detection value increases, Image processing methods including [specific details omitted].

8. An image processing program for causing a computer to perform the image processing method described in claim 7.

9. The fringe detection unit calculates an estimated value which is a weighted average of the pixel values ​​of the subpixels of the second color based on the relationship between the centroid positions of the subpixels of the first and second colors, and generates a fringe detection value indicating the amount of fringe based on the pixel values ​​of the subpixels of the second color and the estimated value. The image processing apparatus according to claim 1.

10. The fringe detection unit calculates the absolute difference between the pixel value of the subpixel of the second color and the estimated value, and uses this absolute value as the fringe detection value indicating the amount of fringe. The image processing apparatus according to claim 9.

Images