Programs, computer devices, and methods

Abstract

This provides a novel program for anti-aliasing processing. [Solution] A program for performing anti-aliasing on an image, wherein a computer device functions as: edge detection means for detecting edge information relating to the edges of regions composed of pixels of the same or similar color included in the image; pattern identification means for identifying edge patterns; weight calculation means for calculating interpolation weights for each pixel; blend color identification means for identifying colors to be blended in interpolation; and blend means for identifying blended colors for each pixel based on the calculated interpolation weights and identified blend colors, wherein the pattern identification means identifies patterns where the edges are T-shaped, cross-shaped, extend diagonally, form corners of a predetermined size or larger, and / or surround a single pixel.

Description

[Technical Field]

[0001] The present invention relates to a program, a computer device, and a method. [Background technology]

[0002] Various anti-aliasing techniques are used to correct aliasing (jaggies) that occurs when displaying images on a screen.

[0003] For example, MLAA (Morphological Anti-Aliasing) is a technique that detects edges in an image that form L-shaped, U-shaped, and / or Z-shaped patterns, and blends them using interpolation weights corresponding to the patterns to reduce the appearance of jagged edges (see Non-Patent Literature 1). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Alexander Reshetov. 2009. Morphological Antialiasing. In High-Performance Graphics. The Eurographics Association, 109‐116. [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] The object of the present invention is to provide a novel program for anti-aliasing processing. [Means for solving the problem]

[0006] The problem of the present invention is, [1] A program for performing anti-aliasing on an image, wherein a computer device is configured to function as: edge detection means for detecting edge information relating to the edges of regions in an image composed of pixels of the same or similar color; pattern identification means for identifying edge patterns based on the detected edge information; weight calculation means for calculating interpolation weights for each pixel based on the identified patterns; blend color identification means for identifying colors to be blended in interpolation based on the identified patterns; and blend means for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend colors, wherein the pattern identification means identifies patterns in which edges form a T-shape, patterns in which edges form a cross-shape, patterns in which edges extend diagonally, patterns in which edges form angles of a predetermined size or larger, and / or patterns in which edges surround a single pixel; [2] The program according to [1], wherein the pattern identification means identifies a higher-level pattern of an edge based on edge information in a pixel group of a predetermined size, and then identifies a lower-level pattern of an edge based on edge information relating to an edge formed by pixels different from the pixel group of the predetermined size that is continuous with the edge included in the pixel group of the predetermined size; [3] When the pattern identification means identifies a pattern in which the edges form a T-shape and / or a pattern in which the edges form a cross-shape, the weight calculation means calculates interpolation weights in different ways depending on the shape and / or direction of the edges located near the ends of the T-shape and / or cross-shaped edges that connect to the T-shape and / or cross-shaped edges, or depending on the length of the portion of the T-shape and / or cross-shaped edges perpendicular to the edges in a predetermined direction, according to the program described in [1] or [2] above; [4] A program according to any one of [1] to [3] above, wherein, when the pattern identification means identifies that the edge is a pattern forming a T-shape, the weight calculation means calculates interpolation weights such that the color of pixels in one or less of the three regions adjacent to the T-shaped edge is interpolated; [5] A program according to any one of [1] to [4], wherein, when the pattern identification means identifies that the edge is a pattern forming a cross shape, the weight calculation means calculates interpolation weights in different ways according to the color combination of the four regions adjacent to the cross-shaped edge; [6] A program according to any one of [1] to [5], wherein, when the pattern identification means identifies that the edge is a pattern that extends diagonally, the weight calculation means calculates interpolation weights in different ways according to the number of pixels that form the diagonal edge; [7] A program according to any one of [1] to [6], wherein, if the pattern identification means identifies that the edge is a pattern that extends diagonally, the weight calculation means calculates interpolation weights in different ways depending on the length and / or direction of the end of the diagonal edge; [8] A program according to any one of [1] to [7], wherein a computer device is further configured to function as a formula identification means for identifying a formula corresponding to a reconstruction line identified based on the shape of the edge, and a weight calculation means calculates interpolation weights based on the formula identified by the formula identification means, and when the pattern identification means identifies that the edge is a pattern that extends diagonally, the formula identification means identifies a formula corresponding to a reconstruction line that passes through the intersection of two provisional reconstruction lines; [9] A program according to any one of [1] to [8], wherein the pattern identification means further identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, and when the pattern identification means identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, the weight calculation means calculates interpolation weights in different ways according to the length of the portion of the edge perpendicular to the edge in a predetermined direction among the L-shaped, Z-shaped, and / or U-shaped edges;

[10] A program according to any one of [1] to [9] above, wherein a computer device is further configured to function as a formula identification means for identifying a formula corresponding to a reconstruction line identified based on the shape of the edge, a weight calculation means for calculating interpolation weights based on the formula identified by the formula identification means, a pattern identification means for further identifying a pattern in which the edge forms a U-shape, and when the pattern identification means identifies a pattern in which the edge forms a U-shape, the formula identification means identifies a formula for the curve;

[11] A program according to any one of [1] to

[10] , wherein, when the pattern identification means identifies that an edge is a pattern that forms a corner of a predetermined size or larger, the weight calculation means calculates interpolation weights in different ways according to the size of the corner for the pixel located at the tip of the corner and for three pixels adjacent to the pixel on either side of the edge;

[12] A program according to any one of [1] to

[11] , wherein, when the pattern identification means identifies that an edge is a pattern surrounding a single pixel, the weight calculation means calculates interpolation weights for the pixel and eight adjacent pixels;

[13] A program according to any one of [1] to

[12] , wherein a computer device further functions as a formula identification means for identifying a formula corresponding to a reconstruction line identified based on the shape of the edge, and a weight calculation means calculates interpolation weights based on the formula identified by the formula identification means, and when the pattern identification means identifies a pattern in which the edge forms a T-shape and / or a pattern in which the edge forms a cross-shape, the blend color identification means identifies the same color for all pixels that intersect with one reconstruction line;

[14] A program according to any of [1] to

[13] , wherein the pattern identification means further identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, and when the pattern identification means identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, the blend color identification means identifies the same color for all pixels that intersect with one reconstruction line;

[15] A program according to any one of [1] to

[14] , which further causes a computer device to function as a weight change receiving means that receives input for changing information about a criterion for calculating interpolation weights;

[16] The program described in

[15] further comprises a computer device which functions as a formula identification means for identifying formulas corresponding to reconstruction lines identified based on the shape of the edges, a weight calculation means which calculates interpolation weights based on the formulas identified by the formula identification means, and a weight change receiving means which receives input for changing information for identifying the slope of the reconstruction lines;

[17] A program according to any one of [1] to

[16] above, wherein a computer device is further configured to function as a color space conversion means for converting the color space of an image, the color space conversion means capable of converting the color space of the portion of the image excluding pixels of a predetermined color to a linear color space, and the blending means identifies the blended color in the linear color space;

[18] A computer device for performing anti-aliasing on an image, comprising: edge detection means for detecting edge information relating to the edges of regions in the image composed of pixels of the same or similar color; pattern identification means for identifying edge patterns based on the detected edge information; weight calculation means for calculating interpolation weights for each pixel based on the identified patterns; blend color identification means for identifying colors to be blended in the interpolation based on the identified patterns; and blend means for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend colors, wherein the pattern identification means identifies patterns in which edges form a T-shape, patterns in which edges form a cross-shape, patterns in which edges extend diagonally, patterns in which edges form angles of a predetermined size or larger, and / or patterns in which edges surround a single pixel;

[19] A method for performing anti-aliasing on an image, performed on at least one computer device, comprising: an edge detection step for detecting edge information relating to the edges of regions in the image composed of pixels of the same or similar color; a pattern identification step for identifying edge patterns based on the detected edge information; a weight calculation step for calculating interpolation weights for each pixel based on the identified patterns; a blend color identification step for identifying colors to be blended in the interpolation based on the identified patterns; and a blend step for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend colors, wherein the pattern identification step identifies patterns in which edges form a T-shape, patterns in which edges form a cross-shape, patterns in which edges extend diagonally, patterns in which edges form angles of a predetermined size or larger, and / or patterns in which edges surround a single pixel;

[20] A program for performing anti-aliasing on an image, wherein a computer device is configured to function as: edge detection means for detecting edge information relating to the edges of regions composed of pixels of the same or similar color included in the image; pattern identification means for identifying edge patterns based on the detected edge information; weight calculation means for calculating interpolation weights for each pixel based on the identified patterns; blend color identification means for identifying colors to be blended in interpolation based on the identified patterns; and blend means for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend colors, wherein the pattern identification means identifies patterns in which edges form an L-shape, patterns in which edges form a Z-shape, and / or patterns in which edges form a U-shape, and when the pattern identification means identifies a pattern in which edges form an L-shape, a Z-shape, and / or a U-shape, the weight calculation means calculates interpolation weights in different ways according to the length of the edges of the portion of the L-shaped, Z-shaped, and / or U-shaped edges that is perpendicular to the edges in a predetermined direction;

[21] A computer device for performing anti-aliasing on an image, comprising: edge detection means for detecting edge information relating to the edges of regions in the image composed of pixels of the same or similar color; pattern identification means for identifying edge patterns based on the detected edge information; weight calculation means for calculating interpolation weights for each pixel based on the identified pattern; blend color identification means for identifying colors to be blended in interpolation based on the identified pattern; and blend means for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend color, wherein the pattern identification means identifies patterns in which edges form an L-shape, patterns in which edges form a Z-shape, and / or patterns in which edges form a U-shape, and when the pattern identification means identifies a pattern in which edges form an L-shape, a Z-shape, and / or a U-shape, the weight calculation means calculates interpolation weights in different ways according to the length of the edges of the portion of the L-shaped, Z-shaped, and / or U-shaped edges that is perpendicular to the edge in a predetermined direction;

[22] A method for performing anti-aliasing on an image, performed on at least one computer device, comprising: an edge detection step for detecting edge information relating to the edges of a region in the image composed of pixels of the same or similar color; a pattern identification step for identifying an edge pattern based on the detected edge information; a weight calculation step for calculating interpolation weights for each pixel based on the identified pattern; a blend color identification step for identifying a color to be blended in the interpolation based on the identified pattern; and a blend step for identifying a blended color for each pixel based on the calculated interpolation weights and the identified blend color, wherein the pattern identification step identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, and when the pattern identification step identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, the weight calculation step calculates interpolation weights in different ways according to the length of the portion of the edge perpendicular to the edge in a predetermined direction among the L-shaped, Z-shaped, and / or U-shaped edges;

[23] A program for performing anti-aliasing on an image, wherein a computer device is configured to function as: edge detection means for detecting edge information relating to the edges of regions composed of pixels of the same or similar color included in the image; pattern identification means for identifying edge patterns based on the detected edge information; formula identification means for identifying formulas corresponding to reconstruction lines identified based on the shape of the edges; weight calculation means for calculating interpolation weights for each pixel based on the identified pattern and the identified formula; blend color identification means for identifying colors to be blended in interpolation based on the identified pattern; and blend means for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend colors, wherein the pattern identification means identifies a pattern in which the edges form a U-shape, and when the pattern identification means identifies a pattern in which the edges form a U-shape, the formula identification means identifies a formula for the curve;

[24] A computer device for performing anti-aliasing on an image, comprising: edge detection means for detecting edge information relating to the edges of regions in the image composed of pixels of the same or similar color; pattern identification means for identifying edge patterns based on the detected edge information; formula identification means for identifying formulas corresponding to reconstruction lines identified based on the shape of the edges; weight calculation means for calculating interpolation weights for each pixel based on the identified pattern and the identified formula; blend color identification means for identifying colors to be blended in interpolation based on the identified pattern; and blend means for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend colors, wherein the pattern identification means identifies a pattern in which the edges form a U-shape, and when the pattern identification means identifies a pattern in which the edges form a U-shape, the formula identification means identifies a formula for the curve;

[25] A method for performing anti-aliasing on an image, performed on at least one computer device, comprising: an edge detection step for detecting edge information relating to the edges of regions in the image composed of pixels of the same or similar color; a pattern identification step for identifying edge patterns based on the detected edge information; a formula identification step for identifying formulas corresponding to reconstruction lines identified based on the shape of the edges; a weight calculation step for calculating interpolation weights for each pixel based on the identified pattern and the identified formula; a blend color identification step for identifying colors to be blended in interpolation based on the identified pattern; and a blend step for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend colors, wherein the pattern identification step identifies a pattern in which the edges form a U-shape, and if the pattern identification step identifies a pattern in which the edges form a U-shape, the formula identification step identifies a formula for the curve;

[26] A program for performing anti-aliasing on an image, wherein a computer device is configured to function as: edge detection means for detecting edge information relating to the edges of regions in the image composed of pixels of the same or similar color; pattern identification means for identifying edge patterns based on the detected edge information; formula identification means for identifying formulas corresponding to reconstruction lines identified based on the shape of the edges; weight calculation means for calculating interpolation weights for each pixel based on the identified patterns and formulas; blend color identification means for identifying colors to be blended in interpolation based on the identified patterns; blend means for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blend colors; and weight change receiving means for receiving input for changing information relating to a criterion for calculating interpolation weights, wherein the weight change receiving means receives input for changing information relating to identifying the slope of the reconstruction lines;

[27] A computer device for performing anti-aliasing processing on an image, comprising: an edge detection means for detecting edge information regarding an edge of a region composed of pixels of the same or similar color included in the image; a pattern identification means for identifying an edge pattern based on the detected edge information; a formula identification means for identifying a formula corresponding to a reconstruction line specified based on the shape of the edge; a weight calculation means for calculating an interpolation weight for each pixel based on the identified pattern and the identified formula; a blend color identification means for identifying a color to be blended in interpolation based on the identified pattern; a blend means for identifying a blended color for each pixel based on the calculated interpolation weight and the identified blend color; and a weight change reception means for receiving an input for changing information regarding a reference for calculating the interpolation weight, wherein the weight change reception means receives an input for changing information for identifying the slope of the reconstruction line; a computer device;

[28] A method for performing anti-aliasing processing on an image, executed in at least one computer device, comprising: an edge detection step for detecting edge information regarding an edge of a region composed of pixels of the same or similar color included in the image; a pattern identification step for identifying an edge pattern based on the detected edge information; a formula identification step for identifying a formula corresponding to a reconstruction line specified based on the shape of the edge; a weight calculation step for calculating an interpolation weight for each pixel based on the identified pattern and the identified formula; a blend color identification step for identifying a color to be blended in interpolation based on the identified pattern; a blend step for identifying a blended color for each pixel based on the calculated interpolation weight and the identified blend color; and a weight change reception step for receiving an input for changing information regarding a reference for calculating the interpolation weight, wherein the weight change reception step receives an input for changing information for identifying the slope of the reconstruction line; a method;

[29] A program for performing anti-aliasing processing on an image, which causes a computer device to function as color space conversion means for converting the color space of the image and blend means for specifying a color obtained by blending the color of a pixel located at an edge of a region composed of pixels of the same or similar colors included in the image with another color, wherein the color space conversion means can convert the color space of a portion excluding pixels of a predetermined color into a linear color space, and the blend means specifies a color blended in the linear color space;

[30] A computer device for performing anti-aliasing processing on an image, comprising color space conversion means for converting the color space of the image and blend means for specifying a color obtained by blending the color of a pixel located at an edge of a region composed of pixels of the same or similar colors included in the image with another color, wherein the color space conversion means can convert the color space of a portion excluding pixels of a predetermined color into a linear color space, and the blend means specifies a color blended in the linear color space;

[31] A method for performing anti-aliasing processing on an image, which is executed on at least one computer device, the method having a color space conversion step for converting the color space of the image and a blend step for specifying a color obtained by blending the color of a pixel located at an edge of a region composed of pixels of the same or similar colors included in the image with another color, wherein the color space conversion step can convert the color space of a portion excluding pixels of a predetermined color into a linear color space, and the blend step specifies a color blended in the linear color space; can be achieved by.

Advantages of the Invention

[0007] According to the present invention, a novel program for anti-aliasing processing can be provided.

Brief Description of the Drawings

[0008] [Figure 1] It is a block diagram showing the hardware configuration of a computer device according to an embodiment of the present invention. [Figure 2] This is a flowchart of the anti-aliasing process according to an embodiment of the present invention. [Figure 3] This is a flowchart of the conversion input processing according to an embodiment of the present invention. [Figure 4] This is a diagram illustrating edge information according to an embodiment of the present invention. [Figure 5] This is a flowchart of the smoothing process according to an embodiment of the present invention. [Figure 6] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 7] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 8] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 9] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 10] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 11] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 12] This is a diagram illustrating the smoothing process according to an embodiment of the present invention. [Figure 13] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 14] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 15] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 16] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 17] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 18] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 19] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 20] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 21] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 22] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 23] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 24] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 25] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 26] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 27] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Figure 28] This is a diagram illustrating a smoothing process according to an embodiment of the present invention. [Modes for carrying out the invention]

[0009] The following describes embodiments of the present invention, but the present invention is not limited to the following embodiments unless it contradicts the spirit of the present invention. Furthermore, the description of effects is only one aspect of the effects of the embodiments of the present invention and is not limited to those described herein. The order of each process constituting the flowchart described below is not limited to any order insofar as it does not cause contradictions or inconsistencies in the processing content. It is also possible to omit some of the processes constituting the flowchart or to add new processes to each process constituting the flowchart, as long as it does not contradict the spirit of the present invention. In addition, the device that is the main entity that executes each process constituting the flowchart can be changed to another device unless it contradicts the spirit of the present invention. In this case, it is possible to change the processing content so as not to cause contradictions or inconsistencies in the processing content.

[0010] Figure 1 is a block diagram showing the hardware configuration of a computer device according to an embodiment of the present invention. The computer device 1 comprises a control unit 11, RAM 12, storage unit 13, input unit 14, display unit 15, and communication interface 16, each connected by a bus.

[0011] The control unit 11 consists of a CPU and ROM. The control unit 11 executes programs stored in the storage unit 13 and controls the computer device 1. The RAM 12 is the work area of ​​the control unit 11. The storage unit 13 is a storage area for saving programs and data. In other words, the storage unit 13 functions as a recording medium that stores programs. The control unit 11 performs calculations based on the programs and data read from the RAM 12, as well as the data input at the input unit 14.

[0012] The display unit 15 has a display screen. The control unit 11 outputs a video signal for displaying an image on the display screen according to the result of the calculation processing. Here, the display screen of the display unit 15 may be a touch panel equipped with a touch sensor. In this case, the touch panel functions as an input unit 14. The display unit 15 and / or the input unit 14 may be connected externally to the computer device 1. The input unit 14 may include a keyboard, a pointing device (mouse, pen tablet, etc.), etc.

[0013] The communication interface 16 can be connected to a communication network wirelessly or via a wired connection, and can send and receive data with other computer devices via the communication network. Data received via the communication interface 16 is loaded into the RAM 12, and calculation processing is performed by the control unit 11.

[0014] The program (program product) according to the embodiment of the present invention may be pre-installed in the computer device 1, or it may be distributed from an external computer device and installed in the computer device 1. The program only needs to be capable of causing the computer device 1 to perform a predetermined function. Furthermore, the program may function by being plugged into other applications, or it may function as an independent application.

[0015] Furthermore, the program (program product) may be stored on a recording medium such as a CD-ROM. In this case, the program stored on the recording medium may be installed on the computer device 1, causing the computer device 1 to execute a predetermined function.

[0016] Furthermore, while this example shows all processing being performed on computer device 1, at least a portion of the processing may be performed on a server device or other computer device. In other words, the method of the present invention can be performed on at least one computer device. In this case, information used for processing may be transmitted and received between computer device 1 and the server device or other computer device as appropriate.

[0017] The following describes an example of performing anti-aliasing on an image using the program of the present invention. The type of image to be subjected to anti-aliasing is not particularly limited and can be designed as appropriate. For example, the type of image may be an image used for animation production, an image used as a key visual, or an image displayed as a game screen.

[0018] Figure 2 is a flowchart of the anti-aliasing process according to an embodiment of the present invention. First, the computer device 1 receives an image input (step S101).

[0019] In step S101, the method for receiving image input is not particularly limited and can be designed as appropriate. For example, image input may be performed by receiving image data transmitted from another device, or by identifying image data stored in the storage unit 13 of the computer device 1.

[0020] Note that the "image input" in step S101 can be any input that allows for the identification of the image to be subjected to anti-aliasing. For example, in step S101, an image may be displayed on the display screen of computer device 1, and an instruction to perform anti-aliasing on the displayed image may be input.

[0021] The image format to be subjected to anti-aliasing is preferably raster format (bitmap format). When the program of the present invention is used for animation production, the input image may be image data including line drawings before coloring (so-called animation), image data in which the area inside the outline of the line drawing is colored (so-called cel image), or image data in which the foreground and background are combined (so-called composite image).

[0022] Computer device 1 performs a conversion input process on the input image (hereinafter also referred to as the input image) (step S102). The conversion input process may be a process that performs various conversions on the image data. The specific content of the conversion input process is not particularly limited and can be designed as appropriate. For example, the conversion input process may be as follows:

[0023] Figure 3 is a flowchart of the conversion input processing according to an embodiment of the present invention. First, the computer device 1 converts the pixel format of the input image (step S201).

[0024] The method for converting the pixel format in step S201 is not particularly limited and can be designed as appropriate. The pixel format may include the arrangement of color channels, bit depth, numerical encoding format, etc. For example, in step S201, the pixel format of the input image may be converted to a format in which the four color channels of RGBA (red, green, blue, and alpha) are represented in floating-point format.

[0025] Next, the computer device 1 performs amplimultiply processing (step S202). If the input image is a premultiplied image, the RGB channels are multiplied by the alpha value, so by performing amplimultiply processing and dividing the RGB values ​​by the alpha value, the RGB values ​​can be made into the correct values ​​(straight values).

[0026] Next, the computer device 1 performs a color filtering process (step S203). The color filtering process sets the alpha value of pixels other than a predetermined color to "0". The predetermined color (RGB value) may be selectable by user input. For example, by setting the alpha value of pixels other than black to "0", everything except the outlines in the input image may be made transparent.

[0027] Furthermore, computer device 1 performs color keying (step S204). Color keying is a process that sets the alpha value of pixels of a predetermined color to "0". The predetermined color (RGB value) may be selectable by user input. For example, the background of the input image may be made transparent by setting the alpha value of white pixels to "0".

[0028] Computer device 1 converts the color space of the input image (step S205). For example, if the color space of the input image is sRGB, it may be converted to a linear color space.

[0029] Colors obtained by blending two or more colors tend to look better in a linear color space than in an sRGB color space. For example, when blending blue and red in equal proportions, the resulting color will be a dull, blackish color in an sRGB color space, but in a linear color space, it will be a beautiful color without any blackness.

[0030] On the other hand, when white and black are blended in an equal manner, the resulting color in linear color space will be a lighter gray than when the color space is sRGB. Therefore, for example, if you blend colors in linear color space for an input image that contains black outlines, the outlines may become fainter.

[0031] According to the present invention, it is possible to convert to a linear color space after setting the alpha values ​​of pixels other than a predetermined color, and / or the alpha values ​​of pixels of a predetermined color, to "0" by color filtering and / or color keying. Since the color space of the portion with an alpha value of "0" is not converted to a linear color space, for example, by setting the alpha value of black pixels to "0" by color keying and then converting to a linear color space, it is possible to convert only the color space of the portion excluding the black pixels that constitute the outline to a linear color space. In other words, according to the present invention, it is possible to convert the color space of the portion excluding pixels of a predetermined color to a linear color space. Furthermore, according to the present invention, it is possible to specify the color space of a pixel according to the color of the pixel. It may be possible for the user to select which color pixels' color space should be a linear color space and which color pixels' color space should be another color space (e.g., sRGB).

[0032] Steps S201 to S205 complete the conversion input processing. Note that one or more of the above steps S201 to S205 may be omitted. In addition, processes other than those in steps S201 to S205 may be performed as part of the conversion input processing.

[0033] Returning to the explanation of Figure 2, the computer device 1 then detects information about the edges of regions composed of pixels of the same or similar color included in the converted input image (hereinafter also referred to as edge information) (step S103).

[0034] An "edge" may be the outer perimeter of a region in the input image that is composed of pixels of the same or similar color. One region may be composed of at least one pixel.

[0035] "Identical colors" may refer to colors whose numerical values ​​for the parameters that define the color (e.g., RGB, luminance, etc.) are the same. "Similar colors" may refer to colors that are not identical but are considered to constitute the same color range. For example, if the difference in the numerical values ​​of the parameters that define two colors is within a predetermined threshold, the two colors may be considered similar. Hereafter, "identical or similar colors" will also be simply referred to as "the same color." Similarly, "colors that are not identical or similar" will also be simply referred to as "different colors."

[0036] The types and number of parameters defining color are not particularly limited and can be designed as appropriate. For example, it may be one or more parameters from red (R), green (G), blue (B), and luminance, or it may be the parameter with the largest difference among red (R), green (G), blue (B), and luminance. A predetermined threshold may be selectable by user input.

[0037] The edge information detected in step S103 is not particularly limited and can be designed as appropriate. For example, the edge information may be information indicating whether or not a side or vertex of a pixel included in the input image corresponds to an edge. Hereinafter, "a side or vertex of a pixel corresponds to an edge" will also be referred to as "an edge exists on the side or vertex of a pixel."

[0038] Whether or not a pixel edge or vertex in an input image is an edge can be determined, for example, by comparing the color of a certain pixel (hereinafter referred to as the "pixel of interest") with the color of adjacent pixels that share an edge or vertex with the pixel of interest. Specifically, for example, if the color of the pixel of interest is the same as the color of adjacent pixels that share an edge or vertex with the pixel of interest, the edge or vertex may be determined not to be an edge. Alternatively, for example, if the color of the pixel of interest is different from the color of adjacent pixels that share an edge or vertex with the pixel of interest, the edge or vertex may be determined to be an edge.

[0039] For example, if the color of the pixel of interest is different from the color of the pixel to its right, it can be detected that the right-hand edge of the pixel of interest is an edge. In this case, the region composed of the pixel of interest and the region composed of the pixel to its right are considered to be two adjacent and distinct regions. Note that "up, down, left, and right" may refer to the top, bottom, left, and right of the input image when viewed from the front. Furthermore, below, "vertical direction" refers to the direction parallel to the vertical direction, "horizontal direction" refers to the direction parallel to the left and right direction, and "diagonal direction" refers to the direction that is not parallel to either the vertical or horizontal direction. The vertical and horizontal directions may be parallel or perpendicular to the grid lines that constitute the pixels (hereinafter also called "grid"), and the diagonal direction may be parallel to the line connecting two opposite vertices among the four vertices that the pixel has.

[0040] Hereafter, an edge that lies on the edge of a pixel will also be called an "edge line." Furthermore, a pixel on which an edge lies on an edge or vertex will also be called an "edge pixel." An edge pixel can also be described as a pixel that is tangent to an edge. Additionally, an edge pixel can be described as a pixel that possesses an edge. Moreover, an edge can be described as being formed by edge pixels.

[0041] The number of adjacent pixels compared to the pixel of interest when detecting edge information is not particularly limited and can be designed as appropriate. For example, the number of adjacent pixels compared to the pixel of interest may be four, eight, or any other number.

[0042] Furthermore, the positional relationship of adjacent pixels compared to the pixel of interest is not particularly limited and can be designed as appropriate. One or more adjacent pixels with any positional relationship can be selected from those located in the up, down, left, right, and diagonal directions relative to the pixel of interest. If adjacent pixels located in the up, down, left, or right directions relative to the pixel of interest are selected, edge information for the edges of the pixel of interest is detected. If adjacent pixels located diagonally to the pixel of interest are selected, edge information for the vertices of the pixel of interest is detected. In order to determine whether the region formed by the pixel of interest may be connected diagonally, it is preferable to detect edge information for the vertices of the pixel of interest.

[0043] Furthermore, when two diagonally adjacent pixels have the same color, it is possible that the two pixels constitute the same region, or that the two pixels constitute different regions but happen to be diagonally adjacent.

[0044] Figure 4 is a diagram illustrating edge information according to an embodiment of the present invention. Image 100 shown in Figure 4 consists of a total of 100 pixels, with 10 pixels vertically and 10 pixels horizontally. Image 100 contains black pixels and white pixels.

[0045] If we consider the black pixel 101a as the pixel of interest, the colors of the pixels above, diagonally to the upper right, to the right, diagonally to the lower right, and below the pixel of interest are different from the color of the pixel of interest. Therefore, the top edge, the top right vertex, the right edge, the bottom right vertex, and the bottom edge of pixel 101a are considered edges. Also, the colors of the pixels diagonally to the lower left, to the left, and diagonally to the upper left of the pixel of interest are the same as the color of the pixel of interest. Therefore, the bottom left vertex, the left edge, and the top left vertex of pixel 101a are not considered edges. Thus, for example, if we represent the top left vertex, top edge, top right vertex, right edge, bottom right vertex, bottom edge, bottom left vertex, and left edge of the pixel of interest as "1" if an edge or vertex is an edge, and as "0" if it is not an edge, the edge information will be (0, 1, 1, 1, 1, 1, 0, 0).

[0046] Furthermore, if we consider the black pixel 101b as the pixel of interest, the colors of the pixels to its left, diagonally upper left, above, right, diagonally lower right, and below the pixel of interest are different from the color of the pixel of interest. Therefore, the left edge, top left vertex, top edge, right edge, bottom right vertex, and bottom edge of pixel 101b are considered edges. Also, the colors of the pixels diagonally lower left and diagonally upper right of the pixel of interest are the same as the color of the pixel of interest. Therefore, the bottom left vertex and top right vertex of pixel 101b are not considered edges. Thus, for example, if we represent the top left vertex, top edge, top right vertex, right edge, bottom right vertex, bottom edge, bottom left vertex, and left edge of the pixel of interest as "1" if an edge or vertex is an edge, and as "0" if it is not an edge, the edge information will be (1, 1, 0, 1, 1, 1, 0, 1).

[0047] The computer device 1 may store the edge information detected in step S103 in the storage unit 13, associating it with information that can identify a pixel and information that can identify the edges or vertices of the pixel.

[0048] For example, edge information may be stored using the color channels of pixels at positions corresponding to pixels in the input image, within a layer of the same size and number of pixels as the input image. Specifically, for example, computer device 1 may write edge information for the left edge of the pixel of interest to the R color channel of the pixel of interest, edge information for the top edge of the pixel of interest to the G color channel, edge information for the top-left vertex of the pixel of interest to the B color channel, and edge information for the top-right vertex of the pixel of interest to the A color channel. As edge information, for example, if it does not correspond to an edge, the value "0" may be written, and if it corresponds to an edge, a value other than "0" (for example, the value "1") may be written.

[0049] Since the edges and vertices of a pixel of interest are shared with adjacent pixels, some of the edge information of the pixel of interest is shared with some of the edge information of adjacent pixels. By detecting and storing edge information in the four directions described above for all pixels in the input image, it is possible to detect and store edge information for the edges and vertices of all pixels in the input image.

[0050] Returning to the explanation of Figure 2, the computer device 1 then performs a smoothing process (step S104).

[0051] Figure 5 is a flowchart of the smoothing process according to an embodiment of the present invention. In step S103, the computer device 1 identifies the edge pattern contained in the image based on the detected edge information (step S301). Based on the identified pattern, the computer device 1 calculates the interpolation weight for each pixel (step S302). The computer device 1 also identifies the color to be blended in the interpolation based on the identified pattern (step S303). The computer device 1 stores the interpolation weight calculated in step S302 and the color to be blended identified in step S303 in the storage unit 13, associating them with information that can identify a pixel (e.g., pixel coordinates, pixel number, etc.) (step S304).

[0052] The term "edge pattern" may refer to the type of edge classified for the purpose of calculating interpolation weights. Computer device 1 can identify the shape of the edge line, the arrangement of edge pixels, and the similarity or difference in color of diagonally adjacent pixels from the edge information detected in step S103. In step S301, computer device 1 can identify which pattern the edges included in the image belong to, based on the shape of the edge line, the arrangement of edge pixels, and the similarity or difference in color of diagonally adjacent pixels.

[0053] The types of edge patterns identified in step S301 are not particularly limited and can be designed as appropriate. For example, the computer device 1 may identify patterns in which edges form a T-shape, patterns in which edges form a cross-shape, patterns in which edges extend diagonally, patterns in which edges form angles of a predetermined size or larger, and / or patterns in which edges surround a single pixel. The computer device 1 may also identify patterns in which edges form an L-shape, patterns in which edges form a Z-shape, and / or patterns in which edges form a U-shape.

[0054] The storage unit 13 of the computer device 1 may store a method for calculating interpolation weights for each edge pattern. In step S302, the computer device 1 can calculate the interpolation weights for each pixel according to the method stored in association with the edge pattern identified in step S301.

[0055] In step S301, the method for identifying the edge pattern is not particularly limited and can be designed as appropriate. For example, the computer device 1 may identify the edge pattern in steps. For example, the computer device 1 may identify the upper-level edge pattern based on edge information in a pixel group of a predetermined size, and then identify the lower-level edge pattern based on edge information relating to an edge formed by pixels different from the pixel group of the predetermined size, which is continuous with the edge included in the pixel group of the predetermined size.

[0056] The “group of pixels of a predetermined size” may be a group of pixels having a predetermined shape and consisting of a predetermined number of pixels. For example, the “group of pixels of a predetermined size” may be a group of pixels in the shape of a rectangle with 3 pixels horizontally and 3 pixels vertically, a group of pixels in the shape of a rectangle with 3 pixels horizontally and 5 pixels vertically, or a group of pixels in the shape of a circle with a diameter of 10 pixels.

[0057] "Continuous edges" means that edge lines existing on the edges of two or more adjacent pixels are connected in the same direction. Here, "same direction" may be vertical, horizontal, or diagonal. Continuous edges can be considered as a single edge even if they have the length of multiple pixels. "Connected edges" means that two edge lines are touching. In cases where two edge lines appear to be touching, the edge lines may be considered connected regardless of whether the vertex of the pixel located at the point of contact is actually an edge.

[0058] Furthermore, "identifying the higher-level patterns of the edges, and then identifying the lower-level patterns of the edges" may mean, when the edge patterns are structured hierarchically, identifying the patterns that belong to a higher level of the hierarchy, and then identifying the patterns that belong to a lower level. The higher-level patterns may be patterns that are common to multiple lower-level patterns. The method for calculating the interpolation weights may be stored in association with the patterns that belong to the lowest level of the hierarchical structure.

[0059] Edge patterns may be classified into a hierarchical structure of two or more levels. Furthermore, the hierarchical structure may have different numbers of levels depending on the type of pattern. Note that some types of patterns do not need to have a hierarchical structure. In other words, the computer device 1 may be able to determine a method for calculating interpolation weights at the time it identifies which higher-level pattern an edge belongs to.

[0060] The following describes an example in which, based on edge information in a set of pixels shaped like a rectangle of 3 pixels horizontally and 3 pixels vertically, the higher-level edge patterns are identified, then the lower-level edge patterns are identified as appropriate, and the interpolation weights for each pixel are calculated based on the identified edge patterns.

[0061] Figure 6 is a diagram illustrating a smoothing process according to an embodiment of the present invention. In the table in Figure 6, the left column shows the names of the higher-level patterns (hereinafter also referred to as higher-level pattern names), and the right column shows at least some examples of pixel groups with a rectangular shape of 3 pixels horizontally and 3 pixels vertically (hereinafter also referred to as 3x3 patterns) corresponding to each higher-level pattern.

[0062] The pixels in the 3x3 pattern shown in Figure 6 are black and white, and an edge exists on the adjacent edge or vertex of a black pixel and a white pixel. In Figure 6, the light gray dashed lines represent edges. In Figure 6, the edge line on the edge of the pixel of interest located in the center of the 3x3 pattern is designated as the edge of interest, and this edge of interest is represented by a light gray dashed line.

[0063] In Figure 6, the left column shows the names of six major patterns: "linear," "diagonal," "mixed linear and diagonal," "hard polygonal," "single pixel," and "no interpolation."

[0064] A 3x3 pattern that falls under the "linear" category may be a pattern in which the edge of interest located on the upper edge of the pixel of interest has a linear length of two pixels or more. In this case, in the 3x3 pattern, two or more edge pixels are adjacent to each other vertically or horizontally, and the edge lines located on the edges of these two or more edge pixels are linearly continuous and connected.

[0065] In Figure 6, four pixel groups are shown as examples of 3x3 patterns that fall under the "linear" category. Of the 3x3 pattern examples shown in Figure 6, the leftmost 3x3 pattern 200 is a pattern in which the edge 202 (the edge of interest) located on the upper edge of the pixel of interest 201 has a horizontal length of 3 pixels.

[0066] As shown in Figure 6, in a 3x3 pattern corresponding to the "linear type," the direction in which the edge lines extend may be vertical or horizontal. Furthermore, in a 3x3 pattern corresponding to the "linear type," edges may exist on two opposing edges within the edge of the pixel of interest. In other words, one pixel of interest may have two edges of interest.

[0067] The higher-level patterns of the "linear type" may include, as described below, lower-level patterns such as patterns in which the edges form an L-shape, patterns in which the edges form a Z-shape, patterns in which the edges form a U-shape, patterns in which the edges form a T-shape, and patterns in which the edges form a cross-shape.

[0068] A 3x3 pattern that corresponds to a "diagonal type" may be a pattern in which an edge exists on two adjacent edges within the edge of the pixel of interest, and that edge connects to an edge of a pixel of the same color that is diagonally adjacent to the pixel of interest. In this case, in the 3x3 pattern, two or more edge pixels are arranged diagonally, and the edge lines existing on the edges of these two or more edge pixels are connected in a step-like manner in a diagonal direction. In this case, it is considered that one region is formed by pixels of the same color arranged diagonally.

[0069] Furthermore, the width and height of each step in the staircase shape formed by the edge line are each the length of one pixel. In raster images, colors are represented by square pixels, so the edges appear staircase-like, but it is thought that the staircase-like edges were originally straight or curved edges extending diagonally. Therefore, in this specification, edges that extend in a staircase-like manner are also referred to as "diagonal edges." Patterns corresponding to "diagonal type" correspond to "patterns where the edges extend diagonally."

[0070] In Figure 6, four pixel groups are shown as examples of 3x3 patterns that correspond to the "diagonal type". Of the 3x3 pattern examples shown in Figure 6, the leftmost 3x3 pattern 210 is a pattern in which the edges 212 (the edge of interest) located on the bottom and right edges of the black pixel of interest 211 are connected to the edge located on the bottom edge of the black pixel adjacent to the upper right of the pixel of interest 211, and to the edge located on the right edge of the black pixel adjacent to the lower left of the pixel of interest 211. The 3x3 pattern 210 can also be described as a pattern in which diagonal edges exist below the edge pixels arranged diagonally. Note that in Figure 6, the diagonal edges are shown diagonally so as to overlap with the pixels, rather than on the edges of the pixels.

[0071] As shown in Figure 6, in a 3x3 pattern corresponding to the "diagonal type," the direction in which the edge lines extend may be from the bottom left to the top right, or from the top left to the bottom right. Also, in a 3x3 pattern corresponding to the "diagonal type," diagonal edges may exist below the edge pixels arranged diagonally, or diagonal edges may exist above the edge pixels arranged diagonally. Furthermore, in a 3x3 pattern corresponding to the "diagonal type," edges may exist on two or more pairs of edges within a pair of adjacent edges in the pixel of interest. In other words, one pixel of interest may have two or more edges of interest.

[0072] A 3x3 pattern that falls under the "mixed type of straight lines and diagonals" may be a pattern in which both an edge extending linearly for two or more pixels and an edge extending diagonally for two or more pixels exist on the pixel of interest.

[0073] In Figure 6, four pixel groups are shown as examples of 3x3 patterns that fall under the "mixed type of straight lines and diagonals." Of the 3x3 pattern examples shown in Figure 6, the leftmost 3x3 pattern 220 is a pattern in which an edge 222a (edge ​​of interest) extends linearly on the upper edge of the pixel of interest 221, an edge 222b (edge ​​of interest) extends linearly on the lower edge of the pixel of interest 221, and a diagonal edge 222c (edge ​​of interest) extends diagonally downwards to the right above the edge pixels that include the pixel of interest 221 and are aligned diagonally.

[0074] As shown in Figure 6, in a 3x3 pattern corresponding to the "mixed linear and diagonal type," the direction in which the linear edge lines extend may be vertical or horizontal. Also, the linear edge lines may exist on two opposing edges within the edge of the pixel of interest. Furthermore, in a 3x3 pattern corresponding to the "mixed linear and diagonal type," the direction in which the diagonal edge lines extend may be from the bottom left to the top right or from the top left to the bottom right. Also, the diagonal edge lines may be located below the edge pixels arranged diagonally or above the edge pixels arranged diagonally. In addition, in a 3x3 pattern corresponding to the "diagonal type," edges may exist on two or more pairs of adjacent edges within the pixel of interest. In a 3x3 pattern corresponding to the "mixed linear and diagonal type," one pixel of interest has two or more edges of interest.

[0075] A 3x3 pattern that corresponds to a "hard, angular shape" may be a pattern in which edges exist on two adjacent edges within the edge of the pixel of interest, and each of these edges has a linear length of two pixels. In this case, a 90° angle (right angle) is formed in the 3x3 pattern by a linear edge extending in the vertical direction and a linear edge extending in the horizontal direction. If the length of the linear edges forming the angle is long, it is highly likely that it is an edge in a region representing a sharp, hard shape. A pattern that corresponds to a "hard, angular shape" is equivalent to "a pattern in which edges form angles of a predetermined size or larger."

[0076] In Figure 6, four pixel groups are shown as examples of 3x3 patterns corresponding to "hard rectangular" shapes. Of the 3x3 pattern examples shown in Figure 6, the leftmost 3x3 pattern 230 is a pattern in which an edge 232a (the edge of interest) extends vertically in a straight line on the right edge of the pixel of interest 231, and an edge 232b (the edge of interest) extends horizontally in a straight line on the bottom edge of the pixel of interest 221.

[0077] As shown in Figure 6, in a 3x3 pattern corresponding to a "hard rectangular" shape, the pixel of interest may be diagonally adjacent to a pixel of the same color on the outside of the corner with an edge. Also, on the inside of the corner, pixels diagonally adjacent to the pixel of interest may be pixels of a different color. In a 3x3 pattern corresponding to a "hard rectangular" shape, one pixel of interest has two edges of interest.

[0078] A 3x3 pattern that corresponds to a "single-pixel type" can be a pattern in which edges exist on the four sides and four vertices of the pixel of interest. In this case, in the 3x3 pattern, the color of the pixel of interest and the colors of the eight pixels adjacent to the pixel of interest are different. A pattern that corresponds to a "single-pixel type" is equivalent to a "pattern in which an edge surrounds a single pixel."

[0079] In Figure 6, one pixel group is shown as an example of a 3x3 pattern that corresponds to a "single-pixel type". The 3x3 pattern 240 is a pattern in which the area of ​​interest pixel 241 is surrounded by edges 242.

[0080] A 3x3 pattern that falls under the "no interpolation" category may be any pattern that does not fall under any of the following categories: "linear," "diagonal," "mixed linear and diagonal," "hard polygonal," or "single pixel."

[0081] In Figure 6, four pixel groups are shown as examples of 3x3 patterns that fall under the "no interpolation" category. Of the 3x3 pattern examples shown in Figure 6, the leftmost 3x3 pattern 250 is a pattern in which all pixels in a quadrilateral-shaped group of 3 pixels horizontally and 3 pixels vertically have the same color.

[0082] As shown in Figure 6, in the 3x3 pattern corresponding to the "no interpolation" type, even if a group of pixels in the shape of a rectangle of 3 pixels horizontally and 3 pixels vertically contains pixels of a different color from the pixel of interest, these pixels do not need to be in adjacent positions.

[0083] The following describes how to calculate the interpolation weights for each higher-level pattern. Depending on the type of higher-level pattern, the interpolation weights may be calculated by first identifying lower-level patterns and then using a method appropriate to those identified. According to the program of the present invention, the interpolation weights can be calculated using different methods for each edge pattern.

[0084] A pixel from which interpolation weights can be calculated, that is, a pixel that can be interpolated, may be at least one pixel that includes a pixel forming a corner formed by the edge of interest, or a pixel that forms a corner formed by the edge of interest and an edge orthogonal to the edge of interest. Furthermore, a pixel that can be interpolated may be a pixel tangent to an edge.

[0085] First, we will explain how to calculate interpolation weights for edges that fall under the higher-level "linear" pattern. If an edge is identified as "linear" in a 3x3 pattern, the computer device 1 may then search for the end of the edge of interest and identify the main pattern of the edge based on the shape of the edge at the end of the edge of interest.

[0086] The "end of the edge of interest" can be any point where a linearly continuous edge of interest is thought to end. The "end of the edge of interest" is the location where a pixel that could be interpolated is located, and it can be the point where the edge of interest connects with an edge orthogonal to it.

[0087] Furthermore, the "end of the edge of interest" may refer to the vicinity of the end of the edge of interest. For example, the "end of the edge of interest" may refer to a predetermined range centered on the vertex where the end of the edge of interest is located. The predetermined range is not particularly limited and can be designed as appropriate. For example, the predetermined range may be the range of 4 pixels adjacent to the end of the edge of interest, or it may be the range of 16 pixels, which is the sum of these 4 pixels and the 12 pixels adjacent to them.

[0088] The method for finding the edge of the target edge is not particularly limited and can be designed as appropriate.

[0089] Figure 7 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 7 shows an example of a method for finding the ends of linear edges extending in the horizontal direction for a 3x3 pattern 200 corresponding to the "linear type" shown in Figure 6.

[0090] Computer device 1 may, in order to find the left edge, start with a set of two pixels flanking edge 202 (the pixel of interest 201 and the pixel adjacent to the pixel of interest 201 in the upward direction), as shown in the figure, and search for the left edge by shifting the position of the set of pixels to be searched one pixel at a time to the left. In Figure 7, the set of pixels that will serve as the starting point in the 3x3 pattern 200 is enclosed by a gray line. Computer device 1 can determine whether or not the edge is continuous based on the edge information of the pixels included in the set of pixels.

[0091] If the color combination and arrangement of two pixels included in the set of pixels being searched are the same as the two pixels at the starting point, the computer device 1 may determine that the edge between the two pixels being searched is continuous with the starting edge 202. In this case, the computer device 1 may refer to the edge information of the set of pixels located one pixel to the left.

[0092] If the color combination and arrangement of two pixels included in the set of pixels to be searched are not the same as the two pixels at the starting point, the computer device 1 may determine that the continuity from the starting edge 202 is broken. The computer device 1 may also identify the leftmost edge of the edge that was last determined to be continuous with the starting edge 202 as the leftmost edge of the target edge. Alternatively, a pixel adjacent to the leftmost edge and constituting the same region as the target pixel may be identified as the pixel located at the leftmost edge (hereinafter also referred to as the leftmost edge pixel).

[0093] The right side of Figure 7 shows an example of a group of pixels in the shape of a rectangle, with 2 pixels horizontally and 2 pixels vertically, including the left edge (hereinafter also referred to as a 2x2 pattern). The pixels in the 2x2 pattern 203 (203a~203i) shown in Figure 7 are black, dark gray, light gray, and white, and an edge exists on an adjacent edge or vertex where pixels of different colors are adjacent. In Figure 7, edges are represented by light gray dashed lines. A gray circle is placed in the center of the leftmost pixel of an edge. In Figure 7, the edge of interest is the edge located on the edge above the leftmost pixel of an edge.

[0094] In the 2x2 pattern 203a, a vertical edge exists above the left edge of the edge of interest (the top-left vertex of the leftmost pixel of the edge), and connecting the edge of interest with the vertical edge forms an L-shape. Similarly, in the 2x2 pattern 203b, a vertical edge exists below the left edge of the edge of interest, and connecting the edge of interest with the vertical edge forms an L-shape.

[0095] In 2x2 pattern 203c, vertical edges exist above and below the left edge of the focus edge, and connecting the focus edge with the vertical edges forms a T-shape. Similarly, in 2x2 pattern 203d, a vertical edge exists above the left edge of the focus edge, and a horizontal edge exists to the left, and connecting the focus edge with both the vertical and horizontal edges forms a T-shape. Furthermore, in 2x2 pattern 203e, a vertical edge exists below the left edge of the focus edge, and a horizontal edge exists to the left, and connecting the focus edge with both the vertical and horizontal edges forms a T-shape.

[0096] In the 2x2 patterns 203f to 203i, vertical edges exist above and below the left edge of the focus edge, and horizontal edges exist to the left of it. Connecting the focus edge with the vertical and horizontal edges results in a cross shape.

[0097] For information on how to find the rightmost edge, refer to the above description to the extent necessary. An example of a 2x2 pattern including the rightmost edge of the edge of interest may be a horizontal mirror image of 2x2 patterns 203a to 203i. The rightmost edge of the edge of interest may be L-shaped, T-shaped, or cross-shaped. The length from the leftmost to the rightmost edge of the edge of interest may be the length of the edge of interest.

[0098] After identifying the left and right ends of the edge of interest, the computer device 1 may identify the main pattern to which the edge of interest belongs based on the number and direction of the edges connected to the left and / or right ends of the edge of interest. In other words, the computer device 1 may identify the main pattern of the edge according to the shape of the edges at the left and / or right ends of the edge of interest.

[0099] The "main pattern of the edge" may be a lower-level pattern relative to the higher-level pattern of the edge. The type of main pattern of the edge is not particularly limited and can be designed as appropriate. The main pattern of the edge may be for the shape of the entire edge, including both ends of the edge of interest, or it may be for the shape of one end of the edge of the edge of interest.

[0100] Figure 8 is a diagram illustrating the smoothing process according to an embodiment of the present invention. Figure 8 shows nine different patterns as examples of the main edge patterns. Note that in Figures 8(D) to (I), pixels marked with the symbol "*" can be pixels of any color.

[0101] Figure 8(A) shows an example of a pattern in which edges form an L-shape (hereinafter also referred to as the "L-shaped pattern"). Figure 8(A) shows an example in which the left end of the edge of interest has a downward L-shape. Note that the "downward L-shape" can be defined as a shape in which, if the edge of interest is considered to correspond to the vertical bar of an L, the edge corresponding to the horizontal bar of the L extends downward. In Figure 8(A), the two vertically aligned squares on the right, marked with the symbol "×", correspond to the outside of the input image. In other words, the right end of the edge is located at the edge of the input image. If one end of the edge of interest has an L-shape, and the edges of the other end do not have an L-shape, then the edge of one end of the edge of interest corresponds to the L-shaped pattern.

[0102] Figure 8(B) shows an example of a pattern in which edges form a Z-shape (hereinafter also referred to as the "Z-shaped pattern"). An edge is considered to have a Z-shape when two L-shaped edges share an edge corresponding to the vertical bar, and the edges corresponding to the horizontal bar of the L-shape extend in opposite directions at both ends. Figure 8(B) shows an example in which the left end of the edge of interest has a downward L-shape, and the right end of the edge of interest has an upward L-shape.

[0103] Figure 8(C) shows an example of a pattern in which edges form a U-shape (hereinafter also referred to as the "U-shaped pattern"). An edge is considered to have a U-shape when two L-shaped edges share an edge corresponding to the vertical bar, and the edges corresponding to the horizontal bar of the L extend in the same direction at both ends. Figure 8(C) shows an example in which the left end of the edge of interest has a downward L-shape, and the right end of the edge of interest has a downward L-shape.

[0104] Figure 8(D) shows the first pattern in which the edges form a T-shape (hereinafter referred to as "T 1 This is an example of a "pattern" (also called a "trick"). 1 The pattern may be a pattern in which the edges form a T-shape, and the edge of interest corresponds to the horizontal bar of the T. If the end of the edge of interest is connected to an edge extending in the same direction as the edge of interest, and to an edge extending in a direction perpendicular to the edge of interest, then the end of that edge is T 1 It fits the pattern.

[0105] Figure 8(D) shows an example where the right end of the edge of interest has a T-shape. The upper part of Figure 8(D) shows an example of a T-shape where a horizontal edge extends to the right and a vertical edge extends downward from the right end of the edge of interest, while the lower part of Figure 8(D) shows an example of a T-shape where a horizontal edge extends to the right and a vertical edge extends upward from the right end of the edge of interest.

[0106] Figure 8(E) shows the second pattern in which the edges form a T-shape (hereinafter referred to as "T 2 This is an example of a "pattern" (also called a "trick"). 2 The pattern may be a pattern in which the edges form a T-shape, and the edge of interest corresponds to the vertical bar of the T. If the end of the edge of interest is connected to two edges extending in a direction perpendicular to the edge of interest, then the end of the edge is T 2 It fits the pattern.

[0107] Figure 8(E) shows an example where the right end of the edge of interest has a T-shape. Figure 8(E) shows an example where a vertical edge extends upwards and downwards from the right end of the edge of interest, creating a T-shape.

[0108] Below, T 1 Pattern and T 2 When combined with other patterns, it is also called a "T-shaped pattern."

[0109] Figures 8(F) to 8(I) show examples of patterns in which edges form a cross shape. The upper part of Figures 8(F) to 8(I) shows an example where the right end of the edge of interest forms a cross shape, and the lower part of Figures 8(F) to 8(I) shows an example where the left end of the edge of interest forms a cross shape. In all the patterns shown in Figures 8(F) to 8(I), the end of the edge of interest is connected to an edge that extends in the same direction as the edge of interest, and to two edges that extend in a direction perpendicular to the edge of interest.

[0110] When an edge forms a cross shape, the main pattern may be determined according to the color combination of the four regions adjacent to the cross-shaped edge. For example, the computer device 1 may determine the main pattern of the edges at the ends of the cross shape depending on whether the colors of the diagonally adjacent pairs of pixels are the same or different in the four pixels that share the vertex located at the point where the edges intersect, which corresponds to the center of the cross. The pairs of diagonally adjacent pixels across the cross-shaped edge include the pair of pixels at the bottom left and top right of the vertex located at the point where the edges intersect (hereinafter also referred to as the first pair), and the pair of pixels at the top left and bottom right of the vertex located at the point where the edges intersect (hereinafter also referred to as the second pair).

[0111] Figure 8(F) shows a pattern in which two pixels in the first pair have the same color, and two pixels in the second pair have the same color (hereinafter also referred to as the "double-connected cross pattern"). If the colors of both pairs of pixels that are diagonally adjacent across the cross-shaped edge are the same, then it is possible that both regions that are diagonally adjacent across the center of the cross are connected.

[0112] Figure 8(G) shows a pattern in which two pixels in the first pair have the same color, and two pixels in the second pair have different colors (hereinafter also referred to as the "right-connected cross pattern"). If the bottom-left and top-right pairs of pixels, which are diagonally adjacent across the cross-shaped edge, have the same color, then there is a possibility that the bottom-left and top-right regions, which are diagonally adjacent across the center of the cross, are connected.

[0113] Figure 8(H) shows a pattern in which the two pixels in the first pair have different colors, and the two pixels in the second pair have the same color (hereinafter also referred to as the "left-connected cross pattern"). If the top-left and bottom-right pairs of pixels that are diagonally adjacent across the cross-shaped edge have the same color, then there is a possibility that the top-left and bottom-right regions that are diagonally adjacent across the center of the cross are connected.

[0114] Figure 8(I) shows a pattern in which the two pixels in the first pair have different colors, and the two pixels in the second pair also have different colors (hereinafter also referred to as the "disconnected cross pattern"). When the colors of both pairs of pixels that are diagonally adjacent across a cross-shaped edge are different, it can be considered that both of the diagonally adjacent regions across the center of the cross are not connected.

[0115] Hereinafter, the cross pattern with both connections, the cross pattern with the right connection, the cross pattern with the left connection, and the unconnected cross pattern will also be referred to as the "cross-shaped pattern."

[0116] When interpolating edges that are "linear," the computer device 1 may identify a mathematical formula corresponding to the reconstruction line identified based on the edge shape, and calculate the interpolation weight for each pixel based on the identified formula.

[0117] Figure 9 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 9(A) is a diagram illustrating a method for identifying a mathematical formula corresponding to a reconstruction line, and Figure 9(B) is a diagram illustrating a method for calculating the weight of each pixel based on the identified mathematical formula.

[0118] The reconstruction line may be an edge line assuming that the edges of the region have been reconstructed to represent their original shape. When interpolating edges that fall under the "linear" category, the reconstruction line may be composed of line segments connecting a point on the edge of interest and a point on an edge perpendicular to the edge of interest, among L-shaped, Z-shaped, U-shaped, T-shaped, and / or cross-shaped edges. The length of the "edge perpendicular to the edge of interest" used when identifying the formula corresponding to the reconstruction line may be the length of one pixel. The reconstruction line does not need to lie on the edge of a pixel. Furthermore, the reconstruction line may be a straight line, a polyline, or a curve. Hereinafter, "identifying the formula corresponding to the reconstruction line" will also be simply referred to as "identifying the reconstruction line." The formula corresponding to the reconstruction line may be identified, for example, by the method shown in Figure 9(A).

[0119] In a portion of the image shown in Figure 9(A), 300 pixels (hereinafter also referred to as the partial image), pixels 301a to 301g are black, and pixels 301h to 301n are white. In the partial image 300, pixels marked with the symbol "*" can be of any color. Between the black pixels 301a to 301g and the white pixels 301h to 301n, there is an edge 302 (302a to 302c) corresponding to a Z-shaped pattern. In the partial image 300, edge 302 is represented by a light gray dashed line. In the partial image 300, edge 302b is the edge of interest, edge 302a is an edge extending from the left end of the edge of interest in a direction perpendicular to the edge of interest, and edge 302c is an edge extending from the right end of the edge of interest in a direction perpendicular to the edge of interest.

[0120] For example, the reconstruction line 303 may be a line segment connecting the midpoint of edge 302a and the midpoint of edge 302c. In this case, the reconstruction line 303 passes through the midpoint of edge 302b. Hereafter, the intersection points of the reconstruction line and the edges will also be referred to as points of interest. In Figure 9(A), the point of interest 304a on edge 302a is represented by a circle with the letter A in the center, the point of interest 304b on edge 302b is represented by a circle with the letter B in the center, and the point of interest 304c on edge 302c is represented by a circle with the letter C in the center.

[0121] If the pixels constituting the partial image 300 are squares with side length 1, and the point of interest 304b is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the length L of edge 302b is "5". Therefore, the coordinates (x, y) of point of interest 304b are (0, 0), the coordinates (x, y) of point of interest 304a are (-2.5, -0.5), and the coordinates (x, y) of point of interest 304c are (2.5, 0.5). Hereafter, the reconstruction line connecting point of interest 304a and point of interest 304b will also be called line segment AB, and the reconstruction line connecting point of interest 304b and point of interest 304c will also be called line segment BC.

[0122] In step S302, interpolation weights may be calculated for pixels that intersect with the reconstruction line. When calculating interpolation weights based on the formula corresponding to the reconstruction line shown in Figure 9(A), interpolation weights are calculated for the black pixels 301a to 301c that intersect with line segment AB, and the white pixels 301h to 301i that intersect with line segment BC, as shown in Figure 9(B). The interpolation weights may be determined based on the area of ​​the region enclosed by the reconstruction line and the edge. Note that the "area enclosed by the reconstruction line and the edge" may be the "area enclosed by the reconstruction line, the edge, and the grid," that is, the "area sandwiched between the reconstruction line and the edge." The interpolation weights may, for example, be equal to the ratio of the area enclosed by the reconstruction line and the edge to the area of ​​the pixel.

[0123] For example, if the pixels constituting the partial image 300 are squares with side length 1, the area of ​​the white pixel 301j located above the point of interest 304b will be 1, and the area of ​​the triangular portion within pixel 301j enclosed by the reconstruction line 303 and edge 302 will be "0.025". Therefore, the interpolation weight calculated for the white pixel 301j may be "0.025".

[0124] Furthermore, for example, if the pixels constituting the partial image 300 are squares with side length 1, the area of ​​the black pixel 301a located to the right of the point of interest 304a will be 1, and the area of ​​the trapezoidal portion within pixel 301a enclosed by the reconstruction line 303 and edge 302 will be "0.4". Therefore, the interpolation weight calculated for the black pixel 301a may be "0.4".

[0125] "Interpolating pixel color" may refer to changing the color of a pixel that forms a corner adjacent to the edge of a region, so that aliasing becomes less noticeable. In this case, the color of the pixel may be changed to a color blended with the color of other pixels located near the edge. Hereinafter, the pixel selected to identify the color to be blended will also be called the "pixel to be blended". Computer device 1 can select a pixel that is in a predetermined positional relationship with the edge to be interpolated as the pixel to be blended, and can identify the color of that pixel as the color to be blended.

[0126] When interpolating pixel colors using reconstruction lines, the resulting pixel color may be the original pixel color mixed with the color of the pixel being blended, in proportion to the calculated interpolation weight. For example, if a white pixel is interpolated to blend with black using an interpolation weight of "0.1", the resulting pixel may be a light gray, with a ratio of 90% white and 10% black.

[0127] If the pattern includes a Z-shaped edge, then for pixels that intersect with a reconstruction line, the pixels to be blended may be pixels that intersect with the reconstruction line, are tangent to an edge perpendicular to the edge of interest, and do not intersect the reconstruction line. Note that "tangent to an edge perpendicular to the edge of interest" may mean that the pixel has an edge perpendicular to the edge of interest on its edge.

[0128] For example, for pixels that intersect line segment BC, the pixel to be blended may be a black pixel 301g that is tangent to the edge where point of interest 304c is located and does not intersect line segment BC. In Figure 9(A), the black pixel 301g is marked with a gray circle. Also, for example, for pixels that intersect line segment AB, the pixel to be blended may be a white pixel 301n that is tangent to the edge where point of interest 304a is located and does not intersect line segment AB. In Figure 9(A), the white pixel 301n is marked with a gray circle.

[0129] As shown in Figure 9(B), for the white pixel 301j, the interpolation weight is calculated to be "0.025", and the color to be blended can be identified as black, which is the color of pixel 301g. Similarly, for the black pixel 301a, the interpolation weight is calculated to be "0.4", and the color to be blended can be identified as white, which is the color of pixel 301n.

[0130] If the edge pattern is one in which the edges form an L-shape, a Z-shape, a U-shape, a T-shape, and / or a cross-shape, and interpolation is performed, the blended color may be determined by identifying the formula corresponding to the reconstruction line, calculating the interpolation weights, and identifying the color to be blended, as described above.

[0131] However, depending on the edge shape, interpolation can sometimes cause the edge pixels to become unnecessarily blurred. In typical 3DCG games, there are many finely detailed objects on the screen, so even if interpolation is performed on all edges, such as L-shaped, U-shaped, and Z-shaped edges, it is unlikely to cause any discomfort to the viewer. However, in typical animation, a wide area is painted with a single color, so if interpolation is performed on all edges, it is highly likely to cause discomfort to the viewer. Therefore, for images including line-drawing-based illustrations such as those in animation, it is necessary to perform interpolation appropriately according to the overall edge shape, pixel arrangement, etc. According to the program of the present invention, it is possible to determine whether or not to perform interpolation based on the edge pattern, and if interpolation is performed, to determine how to calculate the interpolation weights.

[0132] Computer device 1 may, after identifying the main pattern of the edge, further identify the subpattern of the edge, decide whether or not to perform interpolation based on the identified subpattern, and if interpolation is performed, decide how to calculate the interpolation weights. Alternatively, depending on the identified main pattern of the edge, it may decide whether or not to perform interpolation based on the identified main pattern without identifying the subpattern, and if interpolation is performed, decide how to calculate the interpolation weights. The "subpattern of the edge" may be a lower-level pattern relative to the main pattern of the edge. The type of subpattern of the edge is not particularly limited and can be designed as appropriate.

[0133] For example, an L-shaped pattern may contain two subpatterns. The subpatterns of an L-shaped pattern may be identified based on the length of the portion of the edge perpendicular to the edge in a given direction within the L-shaped edge. The "length of the portion of the edge perpendicular to the edge in a given direction within the L-shaped edge" may be the length of the linearly continuous portion of the edge extending from the end of the edge of interest in a direction perpendicular to the edge of interest.

[0134] Figure 10 is a diagram illustrating a smoothing process according to an embodiment of the present invention. A portion of the image shown in Figure 10 (hereinafter also referred to as the partial image) 400 contains black pixels and white pixels, and between the black pixels and white pixels there are edges 402 (402a and 402b) corresponding to an L-shaped pattern. In the partial image 400, the edges 402 are represented by light gray dashed lines. In the partial image 400, edge 402b is the edge of interest, and edge 402a is an edge extending from the left end of the edge of interest in a direction perpendicular to the edge of interest. In the partial image 400, the portion marked with the symbol "×" is outside the input image.

[0135] The computer device 1 can determine, based on the length of edge 402a, which subpattern of partial image 400a or 400b an edge 402 included in partial image 400 corresponds to.

[0136] As shown in partial image 400a, if an edge 402a perpendicular to the edge of interest has a length of two pixels or more in a straight line, interpolation may not be performed on edge 402 which corresponds to the L-shaped pattern. Note that "not performing interpolation on the edge" may mean "not performing interpolation on the pixels that form the edge."

[0137] As shown in partial image 400b, if the edge 402a in a direction orthogonal to the edge of interest does not have a length of two pixels or more in a straight line, interpolation may be performed on the edge 402 that corresponds to the L-shaped pattern. In partial image 400b, the parts marked with the symbol "*+" can be pixels of any color as long as they form a different pattern from partial image 400a.

[0138] In this case, the line segment connecting the point of interest 404a located on edge 402a and the point of interest 404b located on edge 402b may be defined as the reconstruction line 403. The length of edge 402a used to identify the reconstruction line 403 may be the length of one pixel. In Figure 10, the point of interest 404a on edge 402a is represented by a circle with the letter A in the center, and the point of interest 404b on edge 402b is represented by a circle with the letter B in the center.

[0139] In Figure 10, point 404a is located at the midpoint of edge 402a, and point 404b is located at the midpoint of edge 402b. If the pixels constituting the partial image 400 are squares with side length 1, and point 404b is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the length L of edge 402b is "3", so the coordinates (x, y) of point 404b are (0, 0) and the coordinates (x, y) of point 404a are (-1.5, -0.5).

[0140] In this case, the computer device 1 may calculate interpolation weights for the black pixels 401a and 401b that intersect the reconstruction line 403 based on the area of ​​the portion enclosed by the reconstruction line 403 and the edge 402. Furthermore, the pixel to be blended with pixels 401a and 401b may be identified as pixel 401c, which is tangent to the edge where point of interest 404b is located and does not intersect the reconstruction line 403. It is preferable that the computer device 1 identifies the same color as the blending color for all pixels that intersect the reconstruction line 1 with the edge corresponding to the L-shaped pattern.

[0141] In the above example, the point of interest is located at the midpoint of the edges corresponding to the vertical and horizontal bars of an L-shape. However, the position of the point of interest may be changed according to user input. Since the position of the point of interest is information for identifying the slope of the reconstruction line, it can be said that the computer device 1 can accept input to change the information for identifying the slope of the reconstruction line. Furthermore, the position of the point of interest is also information related to the criteria for calculating the interpolation weights.

[0142] The information regarding the criteria for calculating interpolation weights is not particularly limited and can be designed as appropriate. For example, the information regarding the criteria for calculating interpolation weights may be information indicating the strength of the interpolation. This information may be a numerical value between 0 and 100, where a larger value indicates stronger interpolation. Alternatively, the information indicating the strength of the interpolation may be a string that conveys the degree of interpolation strength, such as "weak," "medium," or "strong." A stronger interpolation strength may result in a larger interpolation weight. A larger interpolation weight results in smoother edges.

[0143] Furthermore, the information used to identify the slope of the reconstruction line is not particularly limited and can be designed as appropriate. For example, the information used to identify the slope of the reconstruction line may be information used to identify the position of a point of interest. The information used to identify the position of a point of interest may be a numerical value between 0 and 100. Alternatively, the position of the point of interest may be identified such that a larger numerical value indicates a stronger interpolation. Or, the information used to identify the position of a point of interest may be a string of characters such as "weak," "medium," or "strong," which indicates the degree of interpolation strength corresponding to the position of the point of interest.

[0144] Furthermore, the computer device 1 may be capable of receiving inputs of information for identifying the position of a point of interest in the vertical direction and information for identifying the position of a point of interest in the horizontal direction, respectively, as information for identifying the position of a point of interest. Here, "vertical direction" and "horizontal direction" may refer to the vertical and horizontal directions in the input image, or they may refer to the direction in which the edges corresponding to the vertical bar of an L-shape extend and the direction in which the edges corresponding to the horizontal bar of an L-shape extend. In other words, the computer device 1 may be capable of receiving inputs of information for identifying the position of a point of interest in a first predetermined direction and information for identifying the position of a point of interest in a second predetermined direction. The first predetermined direction is preferably a direction perpendicular to the second predetermined direction.

[0145] Here, computer device 1 accepts a numerical input called a smoothness parameter as information for identifying the position of a point of interest in the direction in which the edge corresponding to the horizontal bar of the L extends, and accepts a numerical input called an extra smoothness parameter as information for identifying the position of a point of interest in the direction in which the edge corresponding to the vertical bar of the L extends. The smoothness parameter and the extra smoothness parameter are each defined by a numerical value between 0 and 100.

[0146] The computer device 1 can change the position of the point of interest according to the input values ​​of the smoothness parameter and the extra smoothness parameter.

[0147] Figure 11 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 11 illustrates the changes in the positions of points of interest 404a and 404b and the inclination of the reconstruction line 403 when the values ​​of the smoothness parameter and the extra smoothness parameter are changed in the partial image 400b shown in Figure 10. In Figure 11, the gray dashed lines indicating edges 402a and 402b are omitted, but edges 402a and 402b exist in the same positions as in Figure 10.

[0148] Figures 11(A) to (C) show the positions of points of interest 404a and 404b and the slope of the reconstruction line 403 when the value of the smoothness parameter is changed between 0 and 100. In Figures 11(A) to (C), the value of the extra smoothness parameter is "0", and point of interest 404b is located at the midpoint of edge 402b.

[0149] As shown in Figure 11(A), when the value of the smoothness parameter is "0", point of interest 404a is located at the intersection of point of interest (edge ​​402b) and edge 402a. In this case, since the reconstruction line 403 is located on edge 402b, there are no pixels that intersect with the reconstruction line 403, and therefore no interpolation is performed.

[0150] Furthermore, as shown in Figure 11(B), when the value of the smoothness parameter is "50", the point of interest 404a lies on edge 402a, between the intersection of the edge of interest (edge ​​402b) and edge 402a, and the midpoint of edge 402a. In this case, the reconstruction line 403 intersects pixels 401a and 401b, so the interpolation weights for the two pixels 401a and 401b are calculated based on the area of ​​the region enclosed by the reconstruction line 403 and edge 402.

[0151] Furthermore, as shown in Figure 11(C), when the value of the smoothness parameter is "100", point of interest 404a is located at the midpoint of edge 402a. In this case, the reconstruction line 403 intersects pixels 401a and 401b, so the interpolation weights for the two pixels 401a and 401b are calculated based on the area enclosed by the reconstruction line 403 and edge 402. As shown in the figure, when the value of the smoothness parameter is "100", the interpolation weights for each pixel are larger than when the value of the smoothness parameter is "50".

[0152] Thus, the position of point 404a may be determined between the intersection of the edge of interest (edge ​​402b) and edge 402a, and the midpoint of edge 402a, such that the position becomes closer to the midpoint of edge 402a as the value of the smoothness parameter increases. Also, the numerical value of the interpolation weight for each pixel may increase as the value of the smoothness parameter increases.

[0153] Figures 11(D) to (F) show the positions of points of interest 404a and 404b and the slope of the reconstruction line 403 when the value of the extra smoothness parameter is changed between 0 and 100. In Figures 11(D) to (F), the value of the smoothness parameter is "100", and point of interest 404a is located at the midpoint of edge 402a.

[0154] As shown in Figure 11(D), when the value of the extra smoothness parameter is "0", point of interest 404b is located at the midpoint of edge 402b. In this case, the reconstruction line 403 intersects pixels 401a and 401b, and therefore the interpolation weights for the two pixels 401a and 401b are calculated based on the area enclosed by the reconstruction line 403 and edge 402.

[0155] Furthermore, as shown in Figure 11(E), when the value of the extra smoothness parameter is "50", the point of interest 404b lies on edge 402b, between the midpoint of edge 402b and the edge of edge 402b located at the edge of the image (hereinafter also referred to as the image edge). In this case, the reconstruction line 403 intersects pixels 401a to 401c, so the interpolation weights for the three pixels 401a to 401c are calculated based on the area enclosed by the reconstruction line 403 and edge 402. As shown in the figure, when the value of the extra smoothness parameter is "50", the interpolation weights for each pixel are larger than when the value of the extra smoothness parameter is "0".

[0156] Furthermore, as shown in Figure 11(F), when the value of the extra smoothness parameter is "100", point of interest 404a is located at the edge of the image, edge 402b. In this case, the reconstruction line 403 intersects pixels 401a to 401c, and therefore, the interpolation weights for the three pixels 401a to 401c are calculated based on the area enclosed by the reconstruction line 403 and edge 402. As shown in the figure, when the value of the extra smoothness parameter is "100", the interpolation weights for each pixel are larger than when the value of the extra smoothness parameter is "50".

[0157] Thus, the position of point of interest 404b may be determined between the midpoint of edge 402b and the image edge of edge 402b, such that as the value of the extra smoothness parameter increases, it becomes closer to the image edge of edge 402b. Also, as the value of the extra smoothness parameter increases, the value of the interpolation weight may increase.

[0158] Next, we will describe an example where the main pattern is a Z-shaped pattern. For example, a Z-shaped pattern may contain four subpatterns. The subpatterns of a Z-shaped pattern may be identified based on the length of the portion of the Z-shaped edge that is perpendicular to the edge in a given direction. The "length of the portion of the Z-shaped edge that is perpendicular to the edge in a given direction" may be the length of the linearly continuous portion of the edge that extends from the end of the edge of interest in a direction perpendicular to the edge of interest.

[0159] Figure 12 is a diagram illustrating a smoothing process according to an embodiment of the present invention. A portion of the image shown in Figure 12 (hereinafter also referred to as a partial image) 500 contains black pixels and white pixels, and between the black pixels and white pixels there are edges 502 (502a to 502c) corresponding to a Z-shaped pattern. In the partial image 500, the edges 502 are represented by light gray dashed lines. In the partial image 500, edge 502b is the edge of interest, edge 502a is an edge extending downward from the left end of the edge of interest in a direction perpendicular to the edge of interest, and edge 502c is an edge extending upward from the right end of the edge of interest in a direction perpendicular to the edge of interest.

[0160] The computer device 1 can determine, based on the length of edge 502a and the length of edge 502c, which of the subpatterns 500a to 500d the edge 502 included in the partial image 500 corresponds to.

[0161] As shown in partial image 500a, if the edge 502a in the direction orthogonal to the edge of interest has a linear length of 2 pixels or more, and the edge 502c in the direction orthogonal to the edge of interest also has a linear length of 2 pixels or more, interpolation may not be performed on the edge 502 corresponding to the Z-shaped pattern.

[0162] As shown in partial image 500b, if edge 502a in a direction orthogonal to the edge of interest has a linear length of 2 pixels or more, and edge 502c in a direction orthogonal to the edge of interest does not have a linear length of 2 pixels or more, interpolation may be performed on edge 502 corresponding to the Z-shaped pattern. In partial image 500b, the parts marked with the symbol "*+" can be pixels of any color as long as they form a pattern different from any of partial images 500a, 500c, and 500d.

[0163] In this case, the line segment connecting the point of interest 504b located on edge 502b and the point of interest 504c located on edge 502c may be defined as the reconstruction line 503b. The length of edge 502c used to identify the reconstruction line 503b may be the length of one pixel. In Figure 12, the point of interest 504b on edge 502b is represented by a circle with the letter B in the center, and the point of interest 504c on edge 502c is represented by a circle with the letter C in the center.

[0164] Computer device 1 may calculate interpolation weights for white pixels 501a to 501c that intersect with reconstruction line 503b based on the area of ​​the region enclosed by reconstruction line 503b and edge 502. Furthermore, the pixel to be blended from pixels 501a to 501c may be identified as pixel 501d, which is tangent to the edge where point of interest 504c is located and does not intersect with reconstruction line 503.

[0165] As shown in partial image 500c, if the edge 502c in the direction orthogonal to the edge of interest has a linear length of 2 pixels or more, and the edge 502a in the direction orthogonal to the edge of interest does not have a linear length of 2 pixels or more, interpolation may be performed on the edge 502 corresponding to the Z-shaped pattern. In partial image 500c, the part marked with the symbol "*+" can be any color pixel as long as it forms a pattern different from any of the partial images 500a, 500b, and 500d.

[0166] In this case, the line segment connecting the point of interest 504b located on edge 502b and the point of interest 504a located on edge 502a may be defined as the reconstruction line 503a. The length of edge 502a used to identify the reconstruction line 503a may be the length of one pixel. In Figure 12, the point of interest 504a on edge 502a is represented by a circle with the letter A in the center.

[0167] Computer device 1 may calculate interpolation weights for black pixels 501e to 501g that intersect with reconstruction line 503a based on the area of ​​the region enclosed by reconstruction line 503a and edge 502. Furthermore, the pixel to be blended among pixels 501e to 501g may be identified as pixel 501h, which is tangent to the edge where point of interest 504a is located and does not intersect with reconstruction line 503.

[0168] As shown in partial image 500d, if the edge 502a in the direction orthogonal to the edge of interest does not have a linear length of more than two pixels, and the edge 502c in the direction orthogonal to the edge of interest does not have a linear length of more than two pixels, interpolation may be performed on the edge 502 that corresponds to the Z-shaped pattern. In partial image 500d, the parts marked with the symbol "*+" can be pixels of any color as long as they form a pattern different from any of the partial images 500a, 500b, and 500c.

[0169] In this case, the line segment connecting the point of interest 504a located on edge 502a and the point of interest 504b located on edge 502b may be defined as reconstruction line 503a, and the line segment connecting the point of interest 504b located on edge 502b and the point of interest 504c located on edge 502c may be defined as reconstruction line 503b. The length of edge 502a used to identify reconstruction line 503a, and the length of edge 502c used to identify reconstruction line 503b, may be the length of one pixel. Reconstruction line 503, which connects reconstruction line 503a and reconstruction line 503b, may correspond to the line segment connecting the point of interest 504a located on edge 502a and the point of interest 504c located on edge 502c.

[0170] Computer device 1 may calculate interpolation weights for black pixels 501e to 501g that intersect with reconstruction line 503a based on the area of ​​the region enclosed by reconstruction line 503a and edge 502. Furthermore, the pixel to be blended among pixels 501e to 501g may be identified as pixel 501h, which is tangent to the edge where point of interest 504a is located and does not intersect with reconstruction line 503.

[0171] Furthermore, the computer device 1 may calculate interpolation weights for white pixels 501a to 501c that intersect with the reconstruction line 503b, based on the area of ​​the portion enclosed by the reconstruction line 503b and the edge 502. In addition, the pixel to be blended with pixels 501a to 501c may be identified as pixel 501d, which is tangent to the edge where point of interest 504c is located and does not intersect with the reconstruction line 503.

[0172] As described above, an edge corresponding to a Z-shaped pattern can be understood as two edges corresponding to an L-shaped pattern joined in opposite directions, sharing the vertical bar of the L. Furthermore, the reconstruction lines identified for an edge corresponding to a Z-shaped pattern can be understood as two reconstruction lines identified for two edges corresponding to L-shaped patterns. In other words, it is preferable for computer device 1 to identify the edges corresponding to a Z-shaped pattern as a color that blends the same color for all pixels that intersect with the reconstruction lines of 1.

[0173] In Figure 12, point of interest 504a is located at the midpoint of edge 502a, point of interest 504b is located at the midpoint of edge 502b, and point of interest 504c is located at the midpoint of edge 502c. If the pixels constituting the partial image 500 are squares with side length 1, and point of interest 504b is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the length L of edge 502b is "5", so the coordinates (x, y) of point of interest 504b are (0, 0), the coordinates (x, y) of point of interest 504a are (-2.5, -0.5), and the coordinates (x, y) of point of interest 504c are (2.5, 0.5).

[0174] Even in the reconstruction lines identified for edges corresponding to a Z-shaped pattern, the position of the point of interest may be changed according to user input. An example of changing the position of the point of interest in the reconstruction lines identified for edges corresponding to a Z-shaped pattern according to the input values ​​of the smoothness parameter and the extra smoothness parameter will be described.

[0175] Figure 13 is a diagram illustrating the smoothing process according to an embodiment of the present invention. Figure 13 illustrates the change in the position of point of interest 504 and the slope of the reconstruction line 503 when the values ​​of the smoothness parameter and the extra smoothness parameter are changed in the partial image 500d shown in Figure 12. In Figure 13, the gray dashed lines and reference numerals indicating edges 502a to 502c are omitted, but edges 502a to 502c exist in the same positions as in Figure 12. Also, in Figure 13, the reference numeral for pixel 501 is omitted.

[0176] Figures 13(A) to (C) show the position of point of interest 504 and the slope of the reconstruction line 503 when the value of the smoothness parameter is changed between 0 and 100. In Figures 13(A) to (C), the value of the extra smoothness parameter is "0", and point of interest 504b is located at the midpoint of edge 502b.

[0177] As shown in Figure 13(A), when the value of the smoothness parameter is "0", point of interest 504a is located at the intersection of point of interest (edge ​​502b) and edge 502a. Also, point of interest 504c is located at the intersection of point of interest (edge ​​502b) and edge 502c. In this case, since the reconstruction line 503 is located on edge 502b, there are no pixels that intersect with the reconstruction line 503, and therefore no interpolation is performed.

[0178] Furthermore, as shown in Figure 13(B), when the value of the smoothness parameter is "50", point of interest 504a lies on edge 502a and is located between the intersection of the edge of interest (edge ​​502b) and edge 502a and the midpoint of edge 502a. Also, point of interest 504c lies on edge 502c and is located between the intersection of the edge of interest (edge ​​502b) and edge 502c and the midpoint of edge 502c. In this case, for the three black pixels 501 that intersect with reconstruction line 503a, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 503a and edge 502, and for the three white pixels 501 that intersect with reconstruction line 503b, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 503b and edge 502.

[0179] Furthermore, as shown in Figure 13(C), when the value of the smoothness parameter is "100", point of interest 504a is located at the midpoint of edge 502a. Also, point of interest 504c is located at the midpoint of edge 502c. In this case, for the three black pixels 501 that intersect with reconstruction line 503a, the interpolation weight is calculated based on the area enclosed by reconstruction line 503a and edge 502, and for the three white pixels 501 that intersect with reconstruction line 503b, the interpolation weight is calculated based on the area enclosed by reconstruction line 503b and edge 502. As shown in the figure, when the value of the smoothness parameter is "100", the interpolation weight for each pixel is larger than when the value of the smoothness parameter is "50".

[0180] Thus, the position of point of interest 504a may be determined between the intersection of the edge of interest (edge ​​502b) and edge 502a and the midpoint of edge 502a, such that as the value of the smoothness parameter increases, it approaches the midpoint of edge 502a. Similarly, the position of point of interest 504c may be determined between the intersection of the edge of interest (edge ​​502b) and edge 502c and the midpoint of edge 502c, such that as the value of the smoothness parameter increases, it approaches the midpoint of edge 502c. As the value of the smoothness parameter increases, the numerical value of the interpolation weight for each pixel may increase.

[0181] Figures 13(D) to (F) show the position of point of interest 504 and the slope of the reconstruction line 503 when the value of the extra smoothness parameter is changed between 0 and 100. In Figures 13(D) to (F), the value of the smoothness parameter is "100", point of interest 504a is located at the midpoint of edge 502a, and point of interest 504c is located at the midpoint of edge 502c.

[0182] As shown in Figure 13(D), when the value of the extra smoothness parameter is "0", point of interest 504b is located at the midpoint of edge 502b. In this case, for the three black pixels 501 that intersect with reconstruction line 503a, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 503a and edge 502, and for the three white pixels 501 that intersect with reconstruction line 503b, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 503b and edge 502.

[0183] Furthermore, as shown in Figure 13(E), when the value of the extra smoothness parameter is "50", point of interest 504b, which corresponds to the intersection of the reconstruction line 503a and edge 502b, lies on edge 502b and is located between the midpoint of edge 502b and the right end of edge 502b. Also, point of interest 504d, which corresponds to the intersection of the reconstruction line 503b and edge 502b, lies on edge 502b and is located between the midpoint of edge 502b and the left end of edge 502b. In Figure 13, point of interest 504d is represented by a circle with the letter D in the center.

[0184] In this case, for the four black pixels 501 that intersect with reconstruction line 503a, the interpolation weights are calculated based on the area enclosed by reconstruction line 503a and edge 502, and for the four white pixels 501 that intersect with reconstruction line 503b, the interpolation weights are calculated based on the area enclosed by reconstruction line 503b and edge 502. As shown in the figure, when the value of the extra smoothness parameter is "50", the interpolation weights for each pixel are greater than when the value of the extra smoothness parameter is "0".

[0185] Furthermore, as shown in Figure 13(F), when the value of the extra smoothness parameter is "100", point of interest 504b is located at the right end of edge 502b, and point of interest 504d is located at the left end of edge 502b.

[0186] In this case, for the five black pixels 501 that intersect with reconstruction line 503a, the interpolation weights are calculated based on the area enclosed by reconstruction line 503a and edge 502, and for the five white pixels 501 that intersect with reconstruction line 503b, the interpolation weights are calculated based on the area enclosed by reconstruction line 503b and edge 502. As shown in the figure, when the value of the extra smoothness parameter is "100", the interpolation weights for each pixel are larger than when the value of the extra smoothness parameter is "50".

[0187] Thus, the position of point of interest 504b may be determined between the midpoint of edge 502b and the right edge of edge 502b, such that as the value of the extra smoothness parameter increases, it moves closer to the right edge of edge 502b. Similarly, the position of point of interest 504d may be determined between the midpoint of edge 502b and the left edge of edge 502b, such that as the value of the extra smoothness parameter increases, it moves closer to the left edge of edge 502b.

[0188] Point of interest 504d is considered to be in the same position as point of interest 504b when the value of the extra smoothness parameter is "0". Also, the larger the value of the extra smoothness parameter, the larger the interpolation weight should be.

[0189] Next, we will describe an example where the main pattern is a U-shaped pattern. For example, a U-shaped pattern may contain four subpatterns. The subpatterns of a U-shaped pattern may be identified based on the length of the portion of the edge perpendicular to the edge in a given direction within the U-shaped edge. The "length of the portion of the edge perpendicular to the edge in a given direction within the U-shaped edge" may be the length of the linearly continuous portion of the edge extending from the end of the edge of interest in a direction perpendicular to the edge of interest.

[0190] Figure 14 is a diagram illustrating a smoothing process according to an embodiment of the present invention. A portion of the image shown in Figure 14 (hereinafter also referred to as the partial image) 600 contains black pixels and white pixels, and between the black pixels and white pixels there are edges 602 (602a to 602c) corresponding to a U-shaped pattern. In the partial image 600, the edges 602 are represented by light gray dashed lines. In the partial image 600, edge 602b is the edge of interest, edge 602a is an edge extending downward from the left end of the edge of interest in a direction perpendicular to the edge of interest, and edge 602c is an edge extending downward from the right end of the edge of interest in a direction perpendicular to the edge of interest.

[0191] The computer device 1 can determine, based on the length of edge 602a and the length of edge 602c, which of the subpatterns 600a to 600d the edge 602 included in the partial image 600 corresponds to.

[0192] As shown in the partial image 600a, if the edge 602a in the direction orthogonal to the edge of interest has a linear length of 2 pixels or more, and the edge 602c in the direction orthogonal to the edge of interest has a linear length of 2 pixels or more, interpolation may not be performed on the edge 602 corresponding to the U-shaped pattern.

[0193] As shown in partial image 600b, if edge 602a in a direction orthogonal to the edge of interest has a linear length of 2 pixels or more, and edge 602c in a direction orthogonal to the edge of interest does not have a linear length of 2 pixels or more, interpolation may be performed on edge 602 corresponding to the U-shaped pattern. In partial image 600b, the parts marked with the symbol "*+" can be pixels of any color as long as they form a pattern different from any of partial images 600a, 600c, and 600d.

[0194] In this case, the line segment connecting the point of interest 604b located on edge 602b and the point of interest 604c located on edge 602c may be defined as the reconstruction line 603b. The length of edge 602c used to identify the reconstruction line 603b may be the length of one pixel. In Figure 14, the point of interest 604b on edge 602b is represented by a circle with the letter B in the center, and the point of interest 604c on edge 602c is represented by a circle with the letter C in the center.

[0195] Computer device 1 may calculate interpolation weights for black pixels 601c to 601e that intersect with reconstruction line 603b based on the area of ​​the region enclosed by reconstruction line 603b and edge 602. Furthermore, the pixel to be blended from pixels 601c to 601e may be identified as pixel 601f, which is tangent to the edge where point of interest 604c is located and does not intersect with reconstruction line 603.

[0196] As shown in partial image 600c, if edge 602c in a direction orthogonal to the edge of interest has a linear length of 2 pixels or more, and edge 602a in a direction orthogonal to the edge of interest does not have a linear length of 2 pixels or more, interpolation may be performed on edge 602 corresponding to the U-shaped pattern. In partial image 600c, the part marked with the symbol "*+" can be any color pixel as long as it forms a pattern different from any of partial images 600a, 600b, and 600d.

[0197] In this case, the line segment connecting the point of interest 604b located on edge 602b and the point of interest 604a located on edge 602a may be defined as the reconstruction line 603a. The length of edge 604a used to identify the reconstruction line 603a may be the length of one pixel. In Figure 14, the point of interest 604a on edge 602a is represented by a circle with the letter A in the center.

[0198] Computer device 1 may calculate interpolation weights for black pixels 601a to 601c that intersect with reconstruction line 603a, based on the area of ​​the region enclosed by reconstruction line 603a and edge 602. Furthermore, the pixel to be blended from pixels 601a to 601c may be identified as pixel 601g, which is tangent to the edge where point of interest 604a is located and does not intersect with reconstruction line 603.

[0199] As shown in partial image 600d, if the edge 602a in the direction orthogonal to the edge of interest does not have a linear length of more than two pixels, and the edge 602c in the direction orthogonal to the edge of interest does not have a linear length of more than two pixels, interpolation may be performed on the edge 602 corresponding to the U-shaped pattern. In partial image 600d, the parts marked with the symbol "*+" can be pixels of any color as long as they form a pattern different from any of the partial images 600a, 600b, and 600c.

[0200] In this case, the line segment connecting point 604a located on edge 602a and point 604b located on edge 602b may be defined as reconstruction line 603a, and the line segment connecting point 604b located on edge 602b and point 604c located on edge 602c may be defined as reconstruction line 603b. The length of edge 602a used to identify reconstruction line 603a, and the length of edge 602c used to identify reconstruction line 603b, may be the length of one pixel. Reconstruction line 603, which connects reconstruction line 603a and reconstruction line 603b, may be a curve passing through the three points 604a, 604b, and 604c. Furthermore, reconstruction line 603b and reconstruction line 603a in the aforementioned partial images 600b and 600c may also be curves.

[0201] The type of curve and the method for identifying the curve's formula can be designed as appropriate. For example, the reconstruction line for an edge corresponding to a U-shaped pattern may be part of a parabola with point 604b as its vertex, or part of an ellipse with point 604b as its vertex. The reconstruction line for an edge corresponding to a U-shaped pattern may be symmetrical with respect to a straight line passing through point 604b and perpendicular to edge 602b. The reconstruction line for an edge corresponding to a U-shaped pattern may be a curve such that the interpolation weight in the central part of the reconstruction line (near the vertex) is smaller than the interpolation weight calculated by the straight reconstruction line.

[0202] Computer device 1 may calculate interpolation weights for black pixels 601a to 601c that intersect with reconstruction line 603a, based on the area of ​​the region enclosed by reconstruction line 603a and edge 602. Furthermore, the pixel to be blended from pixels 601a to 601c may be identified as pixel 601g, which is tangent to the edge where point of interest 604a is located and does not intersect with reconstruction line 603a.

[0203] Furthermore, the computer device 1 may calculate interpolation weights for black pixels 601c to 601e that intersect with the reconstruction line 603b, based on the area of ​​the portion enclosed by the reconstruction line 603b and the edge 602. In addition, the pixel to be blended from pixels 601c to 601e may be identified as pixel 601f, which is tangent to the edge where point of interest 604c is located and does not intersect with the reconstruction line 603b.

[0204] Furthermore, the interpolation weight for pixel 601c that intersects both reconstruction line 603a and reconstruction line 603b may be calculated based on the sum of the area enclosed by reconstruction line 603a and edge 602 and the area enclosed by reconstruction line 603b and edge 602.

[0205] As described above, an edge corresponding to a U-shaped pattern can be considered as two edges corresponding to an L-shaped pattern that share the vertical bar of the L and meet in the same direction. Furthermore, the reconstruction line identified for an edge corresponding to a U-shaped pattern can be considered as two reconstruction lines identified for two edges corresponding to L-shaped patterns. In other words, it is preferable for computer device 1 to identify the same color as the color that blends for all pixels that intersect with the reconstruction line of 1, even for edges corresponding to a U-shaped pattern.

[0206] In Figure 14, point of interest 604a is located at the midpoint of edge 602a, point of interest 604b is located at the midpoint of edge 602b, and point of interest 504c is located at the midpoint of edge 602c. If the pixels constituting the partial image 600 are squares with side length 1, and point of interest 604b is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the length L of edge 602b is "5". Therefore, the coordinates (x, y) of point of interest 604b are (0, 0), the coordinates (x, y) of point of interest 604a are (-2.5, -0.5), and the coordinates (x, y) of point of interest 604c are (2.5, 0.5).

[0207] Even in the reconstruction lines identified for edges corresponding to a U-shaped pattern, the position of the point of interest may be changed according to user input. An example of changing the position of the point of interest in the reconstruction lines identified for edges corresponding to a U-shaped pattern according to the input values ​​of the smoothness parameter and the extra smoothness parameter will be described.

[0208] Figure 15 is a diagram illustrating the smoothing process according to an embodiment of the present invention. Figure 15 illustrates the change in the position of point of interest 604 and the slope of the reconstruction line 603 when the values ​​of the smoothness parameter and the extra smoothness parameter are changed in the partial image 600d shown in Figure 14. In Figure 15, the gray dashed lines and reference numerals indicating edges 602a to 602c are omitted, but edges 602a to 602c exist in the same positions as in Figure 14. Also, in Figure 15, the reference numeral for pixel 601 is omitted.

[0209] Figures 15(A) to (C) show the position of point of interest 604 and the slope of the reconstruction line 603 when the value of the smoothness parameter is changed between 0 and 100. In Figures 15(A) to (C), the value of the extra smoothness parameter is "0", and point of interest 604b is located at the midpoint of edge 602b.

[0210] As shown in Figure 15(A), when the value of the smoothness parameter is "0", point of interest 604a is located at the intersection of point of interest (edge ​​602b) and edge 602a. Also, point of interest 604c is located at the intersection of point of interest (edge ​​602b) and edge 602c. In this case, since the reconstruction line 603 is located on edge 602b, there are no pixels that intersect with the reconstruction line 603, and therefore no interpolation is performed.

[0211] Furthermore, as shown in Figure 15(B), when the value of the smoothness parameter is "50", point of interest 604a lies on edge 602a and is located between the intersection of the edge of interest (edge ​​602b) and edge 602a and the midpoint of edge 602a. Also, point of interest 604c lies on edge 602c and is located between the intersection of the edge of interest (edge ​​602b) and edge 602c and the midpoint of edge 602c. In this case, for the five black pixels 601 that intersect with reconstruction lines 603a and 603b, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 603 and edge 602.

[0212] Furthermore, as shown in Figure 15(C), when the value of the smoothness parameter is "100", point of interest 604a is located at the midpoint of edge 602a. Also, point of interest 604c is located at the midpoint of edge 602c. In this case, for the five black pixels 601 that intersect with reconstruction lines 603a and 603b, the interpolation weights are calculated based on the area enclosed by reconstruction line 603 and edge 602. As shown in the figure, when the value of the smoothness parameter is "100", the interpolation weights for each pixel are larger than when the value of the smoothness parameter is "50".

[0213] Thus, the position of point 604a may be determined between the intersection of edge 602b and edge 602a and the midpoint of edge 602a, such that as the value of the smoothness parameter increases, it approaches the midpoint of edge 602a. Similarly, the position of point 604c may be determined between the intersection of edge 602b and edge 602c and the midpoint of edge 602c, such that as the value of the smoothness parameter increases, it approaches the midpoint of edge 602c. As the value of the smoothness parameter increases, the numerical value of the interpolation weight for each pixel may increase.

[0214] Figures 15(D) to (F) show the position of point of interest 604 and the slope of the reconstruction line 603 when the value of the extra smoothness parameter is changed between 0 and 100. In Figures 15(D) to (F), the value of the smoothness parameter is "100", point of interest 604a is located at the midpoint of edge 602a, and point of interest 604c is located at the midpoint of edge 602c.

[0215] As shown in Figure 15(D), when the value of the extra smoothness parameter is "0", point of interest 604b is located at the midpoint of edge 602b. In this case, for the five black pixels 601 that intersect with reconstruction line 603a and reconstruction line 603b, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 603 and edge 602.

[0216] Furthermore, as shown in Figure 15(E), when the value of the extra smoothness parameter is "50", point of interest 604b, which corresponds to the intersection of the reconstruction line 603a and edge 602b, lies on edge 602b and is located between the midpoint of edge 602b and the right end of edge 602b. Also, point of interest 604d, which corresponds to the intersection of the reconstruction line 603b and edge 602b, lies on edge 602b and is located between the midpoint of edge 602b and the left end of edge 602b. In Figure 15, point of interest 604d is represented by a circle with the letter D in the center.

[0217] In this case, for the three black pixels 601 that intersect with reconstruction line 603a, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 603a and edge 602, and for the three black pixels 601 that intersect with reconstruction line 603b, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 603b and edge 602.

[0218] Furthermore, the interpolation weight for pixel 601c that intersects both reconstruction line 603a and reconstruction line 603b may be calculated based on the sum of the area enclosed by reconstruction line 603a and edge 602, and the area enclosed by reconstruction line 603b and edge 602. As shown in the figure, when the value of the extra smoothness parameter is "50", the interpolation weight for each pixel is larger than when the value of the extra smoothness parameter is "0".

[0219] Furthermore, as shown in Figure 15(F), when the value of the extra smoothness parameter is "100", point of interest 604b is on edge 602b to the right of the position shown in Figure 15(E), and is located between the midpoint of edge 602b and the right end of edge 602b. Also, point of interest 604d is on edge 602b to the left of the position shown in Figure 15(E), and is located between the midpoint of edge 602b and the left end of edge 602b.

[0220] In this case, for the four black pixels 601 that intersect with reconstruction line 603a, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 603a and edge 602, and for the four black pixels 601 that intersect with reconstruction line 603b, the interpolation weights are calculated based on the area of ​​the region enclosed by reconstruction line 603b and edge 602.

[0221] Furthermore, the interpolation weights for pixels 601b to d that intersect both reconstruction line 603a and reconstruction line 603b may be calculated based on the sum of the area enclosed by reconstruction line 603a and edge 602, and the area enclosed by reconstruction line 603b and edge 602. As shown in the figure, when the value of the extra smoothness parameter is "100", the interpolation weights for each pixel are larger than when the value of the extra smoothness parameter is "50".

[0222] Thus, the position of point of interest 604b may be determined between the midpoint of edge 602b and a point closer to the right edge of edge 602b than the midpoint of edge 602b, such that as the value of the extra smoothness parameter increases, it moves closer to the right edge of edge 602b. Similarly, the position of point of interest 604d may be determined between the midpoint of edge 602b and a point closer to the left edge of edge 602b than the midpoint of edge 602b, such that as the value of the extra smoothness parameter increases, it moves closer to the left edge of edge 602b.

[0223] Point 604d is considered to be in the same position as point 604b when the value of the extra smoothness parameter is "0". Also, the larger the value of the extra smoothness parameter, the larger the interpolation weight should be.

[0224] Thus, if the pattern is identified as one in which the edges form an L-shape, one in which the edges form a Z-shape, and / or one in which the edges form a U-shape, the interpolation weights may be calculated in different ways depending on the length of the portion of the edge perpendicular to the edge in a given direction (the edge of interest) within the L-shape, Z-shape, and / or U-shape.

[0225] Also, when it is specified that the pattern is such that the edge forms an L-shaped pattern, the edge forms a Z-shaped pattern, and / or the edge forms a U-shaped pattern, if the length of the edge of the portion perpendicular to the edge in a predetermined direction (the edge of interest) among the L-shaped, Z-shaped, and / or U-shaped edges is longer than a predetermined length or is equal to or greater than the predetermined length, interpolation may not be performed from the center of the edge in the predetermined direction to the edge of the perpendicular portion that is longer than the predetermined length or is equal to or greater than the predetermined length. "Interpolation is not performed from the center of the edge in the predetermined direction to the edge of the perpendicular portion that is longer than the predetermined length or is equal to or greater than the predetermined length" may mean that "a reconstruction line is not specified from the center of the edge in the predetermined direction to the edge of the perpendicular portion that is longer than the predetermined length or is equal to or greater than the predetermined length."

[0226] Also, when it is specified that the pattern is such that the edge forms an L-shaped pattern, the edge forms a Z-shaped pattern, and / or the edge forms a U-shaped pattern, if the length of the edge of the portion perpendicular to the edge in a predetermined direction (the edge of interest) among the L-shaped, Z-shaped, and / or U-shaped edges is shorter than a predetermined length or is equal to or less than the predetermined length, interpolation may be performed from the center of the edge in the predetermined direction to the edge of the perpendicular portion that is shorter than the predetermined length or is equal to or less than the predetermined length. "Interpolation is performed from the center of the edge in the predetermined direction to the edge of the perpendicular portion that is shorter than the predetermined length or is equal to or less than the predetermined length" may mean that "a reconstruction line is specified from the center of the edge in the predetermined direction to the edge of the perpendicular portion that is longer than the predetermined length or is equal to or greater than the predetermined length."

[0227] Next, an example where the main pattern is the T 1 pattern will be described. For example, the T 1 pattern may include four sub-patterns. The T 1Subpatterns of a pattern are edges located near the ends of a T-shaped edge and may be identified based on the shape and / or direction of edges that connect to the T-shaped edge. "Edges located near the ends of a T-shaped edge and that connect to the T-shaped edge" may be edges extending from the ends of the edges corresponding to the vertical bars of the T.

[0228] Also, T 1 Edges corresponding to subpatterns of a pattern may have their interpolation weights calculated in different ways based on the shape and / or direction of the edges at the ends of the edges that are continuous with the edges located at the ends of the T-shaped edges. "Edges that are continuous with the edges located at the ends of the T-shaped edges" may refer to edges that are linearly continuous with the edge of interest, corresponding to the horizontal bar of the T, in the same direction as the edge of interest. The ends of such edges will be described later.

[0229] Figure 16 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 16(A) is an example of a subpattern identified based on the shape and / or direction of the edges extending from the ends of the edges corresponding to the vertical bars of the T, and Figure 16(B) is an example of an interpolation method identified based on the shape and / or direction of the ends of the edges corresponding to the horizontal bars of the T.

[0230] The portion of the image shown in Figure 16(A), 700 pixels (hereinafter also referred to as the partial image), contains black pixels, white pixels, and dark gray pixels, and between pixels of different colors, T 1 There are edges 702 (702a to 702c) that correspond to the pattern. In partial image 700, edges 702 are represented by a light gray dashed line. Also in partial image 700, edge 702a is the edge of interest, edge 702b is an edge that extends downward from the right end of the edge of interest and is perpendicular to the edge of interest, and edge 702c is an edge that extends from the right end of the edge of interest in the same direction as the edge of interest.

[0231] The computer device 1 can determine which of the subpatterns 700a to 700d an edge 702b in partial image 700 corresponds to, based on the shape of the edge 702b extending from the opposite end of the intersection point with edge 702a.

[0232] As shown in partial image 700a, if there is an edge 702d that extends laterally from the end of edge 702b toward the side where the edge of interest (edge ​​702a) exists, then T 1 Interpolation may be performed on the edge 702 corresponding to the pattern. In this case, edges 702a, 702b, and 702d form a U-shaped edge. Furthermore, with respect to the region containing the edge of interest, edge 702d is considered to extend inward.

[0233] In this case, the line segment connecting the point of interest 704a located at the midpoint of edge 702a and the point of interest 704b located at the midpoint of edge 702b may be defined as the reconstruction line 703. In Figure 16, the point of interest 704a on edge 702a is represented by a circle with the letter A in the center, and the point of interest 704b on edge 702b is represented by a circle with the letter B in the center.

[0234] In Figure 16, if the pixels constituting the partial image 700a are squares with side length 1, and if the point of interest 704a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 702a is L, then the coordinates (x, y) of the point of interest 704a are (0, 0), and the coordinates (x, y) of the point of interest 704b are (L / 2, -0.5).

[0235] Computer device 1 may calculate interpolation weights for black pixels 701a and 701b that intersect with reconstruction line 703 based on the area of ​​the region enclosed by reconstruction line 703b and edge 702. Furthermore, the pixel to be blended between pixels 701a and 701b may be identified as pixel 701c, which is tangent to the edge where point of interest 704 is located and does not intersect with reconstruction line 703. Computer device 1 may calculate interpolation weights for black pixels 701a and 701b that intersect with reconstruction line 703. 1 For edges corresponding to the pattern, it is preferable to identify all pixels that intersect with the reconstruction line 1 as a color to blend with the same color.

[0236] As shown in partial image 700b, if there is an edge 702d extending laterally from the end of edge 702b toward the side where the edge of interest (edge ​​702a) exists, and an edge 702e extending laterally toward the side where edge 702c exists, then T 1 Interpolation may be performed on edge 702 corresponding to the pattern. In this case, edges 702b, 702d, and 702e form a T-shaped edge. Furthermore, with respect to the region containing the edge of interest, edge 702d is considered to extend inward, and edge 702e is considered to extend outward. In this case, the interpolation weights may be calculated based on the shape and / or direction of the end of edge 702c using the method shown in partial image 700b-1 or partial image 700b-2 of Figure 16(B). The position of the point of interest on edge 702b differs between the method shown in partial image 700b-1 and the method shown in partial image 700b-2.

[0237] Furthermore, as shown in partial image 700c, if there are edges 702d extending laterally from the end of edge 702b toward the side where the edge of interest (edge ​​702a) exists, edge 702e extending laterally toward the side where edge 702c exists, and edge 702f extending in the same direction as edge 702b, then T 1Interpolation may be performed on edge 702 corresponding to the pattern. In this case, edges 702b, 702d, 702e, and 702f form a cross-shaped edge. Furthermore, with respect to the region containing the edge of interest, edge 702d is considered to extend inward, and edge 702e is considered to extend outward. In addition, edge 702f can be considered as an extended version of edge 702b. In this case, the interpolation weights may be calculated based on the shape and / or direction of the end of edge 702c using the method shown in partial image 700c-1 or partial image 700c-2 of Figure 16(B). The position of the point of interest on edge 702b differs between the method shown in partial image 700c-1 and the method shown in partial image 700c-2.

[0238] The "end of edge 702c" can be any point where a linearly continuous edge 702c is considered to have ended. The end of edge 702c may be defined by a different criterion than the end of the edge of interest.

[0239] Furthermore, "the end of edge 702c" may refer to the vicinity of the end of edge 702c. For example, "the end of edge 702c" may refer to a predetermined range centered on the vertex where the end of edge 702c is located. The predetermined range is not particularly limited and can be designed as appropriate. For example, the predetermined range may be the range of 4 pixels adjacent to the end of edge 702c, or it may be the range of 16 pixels, which is the sum of these 4 pixels and the 12 pixels adjacent to them.

[0240] As shown in Figure 16(B), the computer device 1 may search for the right edge of edge 702c by starting with a set of two pixels flanking edge 702c (pixel 701c and pixel 701d adjacent to pixel 701c in the upward direction), and shifting the position of the set of pixels to be searched one pixel at a time to the right. The computer device 1 can determine whether the edge is continuous or not based on the edge information of the pixels included in the set of pixels. Note that in Figure 16(B), the codes of some pixels 701 and edge 702 have been omitted.

[0241] If the color combination and arrangement of two pixels included in the set of pixels to be searched are the same as the two pixels at the starting point, or if the color of the pixel located below the two pixels included in the set of pixels to be searched is the same as the color of the lower pixel 701c in the set of pixels at the starting point, and the color of the upper pixel is different from both the color of the lower pixel 701c and the upper pixel 701d in the set of pixels at the starting point, then the computer device 1 may determine that the edge between the two pixels being searched is continuous with the starting point edge 702c. In this case, the computer device 1 may refer to the edge information of the set of pixels located one pixel to the right.

[0242] For example, edge 702c shown in partial image 700e is determined to be continuous with the starting edge 702c, and the edge information of the set of pixels located one pixel to the right is referenced. Also, for example, edge 702c shown in partial image 700f is orthogonal to edge 702g, but because the lower pixel in the set of pixels is the same dark gray as pixel 701c, and the upper pixel is a different light dark gray than pixel 701d, it is determined to be continuous with the starting edge 702c, and the edge information of the set of pixels located one pixel to the right is referenced.

[0243] On the other hand, in the pixel group of 2 horizontal and 2 vertical pixels shown in partial image 700g, the color of the two vertically adjacent pixels to the right is both dark gray, which is the same as the color of pixel 701c located at the bottom of the starting pixel set. Therefore, it is determined that the center of the pixel group shown in partial image 700e is the right end of edge 702c. Furthermore, the right end of edge 702c is in contact with edge 702h, which extends upward from the right end and is perpendicular to edge 702c. In other words, at the right end of edge 702c, an L-shape is formed by edge 702c and edge 702h. Also, the direction in which edge 702h, which corresponds to the horizontal bar of the L-shape, extends is opposite to the direction of edge 702b, which corresponds to the vertical bar of the T-shape. If the right end of edge 702c has an L-shape, as shown in partial images 700b-1 / 700c-1, the line segment connecting point 704a, which is located at the midpoint of edge 702a, and point 704b, which is located at the midpoint of edge 702b, may be defined as the reconstruction line 703.

[0244] In Figure 16(B), if the pixels constituting the partial images 700b-1 and 700c-1 are squares with side length 1, and if the point of interest 704a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 702a is L, then the coordinates (x, y) of the point of interest 704a are (0, 0), and the coordinates (x, y) of the point of interest 704b are (L / 2, -0.5).

[0245] Furthermore, in partial image 700h, the portion marked with the symbol "*+" can be any color pixel, as long as it has a pattern different from any of the partial images 700e to 700g. The rightmost edge of edge 702c shown in partial image 700h is determined to be the rightmost edge of edge 702c. In this case, as shown in partial images 700b-2 / 700c-2, the line segment connecting point 704a, which is located at the midpoint of edge 702a, and point 704b, which is located between the intersection of edge 702a and edge 702b and the midpoint of edge 702b, may be defined as the reconstruction line 703.

[0246] In Figure 16(B), if the pixels constituting the partial images 700b-2 and 700c-2 are squares with side length 1, and if the point of interest 704a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 702a is L, then the coordinates (x, y) of the point of interest 704a are (0, 0), and the coordinates (x, y) of the point of interest 704b are (L / 2, -0.25).

[0247] Computer device 1 may calculate interpolation weights for black pixels 701a and 701b that intersect with reconstruction line 703 based on the area of ​​the region enclosed by reconstruction line 703b and edge 702. Furthermore, the pixel to be blended between pixels 701a and 701b may be identified as pixel 701c, which is tangent to the edge where point of interest 704b is located and does not intersect with reconstruction line 703. Computer device 1 may calculate interpolation weights for black pixels 701a and 701b that intersect with reconstruction line 703. 1 For edges corresponding to the pattern, it is preferable to identify all pixels that intersect with the reconstruction line 1 as a color to blend with the same color.

[0248] Furthermore, the edge patterns shown in partial images 700b-1 / 700c-1 and 700b-2 / 700c-2 may be considered a type of subpattern. In that case, T 1 The pattern can be thought of as containing six subpatterns.

[0249] Furthermore, as shown in the partial image 700d of Figure 16(A), if the shape of the edge extending from the end of edge 702b is different from any of the partial images 700a to 700c, then T 1 Interpolation is not required for edge 702 that corresponds to the pattern. If the shape of the edge extending from the end of edge 702b is different from any of the partial images 700a to 700c, this includes cases such as when there is no edge extending horizontally from the end of edge 702b toward the edge of interest (edge ​​702a). In partial image 700d, the part marked with the symbol "*+" can be any color pixel as long as it forms a pattern different from any of the partial images 700a to 700c.

[0250] T 1 Even in the reconstruction lines identified for edges corresponding to the pattern, the position of the point of interest may be changed according to user input. For example, computer device 1 may, in accordance with the input values ​​of the smoothness parameter and the extra smoothness parameter, T 1 The position of the points of interest in the reconstruction lines identified for edges corresponding to the pattern may be changed. The methods for changing the position of the points of interest in the reconstruction lines can be adopted to the extent necessary, as described above.

[0251] The assignment of the numerical values ​​of the smoothness parameter and the extra smoothness parameter to the positions of the points of interest on each edge is not particularly limited and can be designed as appropriate. Furthermore, the way in which the positions of the points of interest are changed according to the numerical values ​​of the smoothness parameter or the extra smoothness parameter is not particularly limited and can be designed as appropriate. Computer device 1 may change the positions of the points of interest such that the numerical values ​​of the interpolation weights increase as the numerical values ​​of the smoothness parameter and / or the extra smoothness parameter increase.

[0252] Next, the main pattern is T 2 Let's explain an example of a pattern. For example, T 2 The pattern does not need to include subpatterns. Also, T 2 Interpolation is not required for edges that match the pattern.

[0253] In a general MLAA, an edge corresponding to a T-shaped pattern is processed as a combination of two L-shaped edges. Therefore, among the three regions adjacent to the edge, the colors of the pixels in two regions are interpolated, and the edge becomes blurred more than necessary. On the other hand, according to the program of the present invention, when an edge is identified as a pattern forming a T-shaped shape, weights can be calculated so that the colors of the pixels in one or less regions among the three regions adjacent to the edge are interpolated. Therefore, the edge does not become blurred more than necessary.

[0254] Also, in a general MLAA, for pixels adjacent to an edge corresponding to the horizontal bar of the L-shape among the pixels intersecting the reconstruction line, the colors of the adjacent pixels sandwiching the edge corresponding to the horizontal bar of the L-shape are blended, and for pixels adjacent to an edge corresponding to the vertical bar of the L-shape, the colors of the adjacent pixels sandwiching the edge corresponding to the vertical bar of the L-shape are blended. Therefore, the pixels adjacent to the T-shaped edge are not blended in a gradient-like manner, but are interpolated as if the colors are unnaturally mixed. On the other hand, according to the program of the present invention, since the same color can be specified as the color to be blended for all pixels intersecting one reconstruction line, the colors of the pixels adjacent to the T-shaped edge can be interpolated to be a natural color in a gradient-like manner.

[0255] Next, an example in which the main pattern is a double-connected cross pattern will be described. For example, the double-connected cross pattern may include 25 sub-patterns. The sub-pattern of the double-connected cross pattern may be an edge located in the vicinity of the end of the cross-shaped edge and may be specified based on the shape and / or direction of the edge connected to the cross-shaped edge. The "edge located in the vicinity of the end of the cross-shaped edge and connected to the cross-shaped edge" may be an edge extending from the end of the edge corresponding to the vertical bar of the cross. The edge corresponding to the vertical bar of the cross may be an edge orthogonal to the target edge among the cross-shaped edges.

[0256] As for the vertical bars of the cross, there are a vertical bar extending upward and a vertical bar extending downward. For example, there are five sub-patterns specified based on the shape and / or direction of the edge extending from the end of the edge corresponding to the vertical bar extending upward of the cross, and there are five sub-patterns specified based on the shape and / or direction of the edge extending from the end of the edge corresponding to the vertical bar extending downward of the cross. Twenty-five sub-patterns may be formed by the combination of each set of five sub-patterns. The pixels that can be interpolated based on the sub-pattern specified based on the shape and / or direction of the edge extending from the end of the edge corresponding to the vertical bar extending upward of the cross may be the pixels adjacent to the upper side of the target edge. Also, the pixels that can be interpolated based on the sub-pattern specified based on the shape and / or direction of the edge extending from the end of the edge corresponding to the vertical bar extending downward of the cross may be the pixels adjacent to the lower side of the target edge.

[0257] Hereinafter, referring to FIG. 17, the sub-pattern specified based on the shape and / or direction of the edge extending from the end of the edge corresponding to the vertical bar extending upward of the cross will be described. The sub-pattern specified based on the shape and / or direction of the edge extending from the end of the edge corresponding to the vertical bar extending downward of the cross may be a vertically mirror-inverted image of the example shown in FIG. 17.

[0258] FIG. 17 is a diagram for explaining the smoothing process according to an embodiment of the present invention. A part of the image shown in FIG. 17 (hereinafter also referred to as a partial image) 800 includes black pixels and white pixels, and between the black pixels and the white pixels, there are edges 802 (802a to 802d) corresponding to a cross pattern of both connections. In the partial image 800, the edges 802 are represented by thin gray broken lines. Also, in the partial image 800, the edge 802a is the target edge, the edge 802b is an edge in a direction orthogonal to the target edge extending upward from the right end of the target edge, the edge 802c is an edge in a direction orthogonal to the target edge extending downward from the right end of the target edge, and the edge 802d is an edge extending in the same direction as the target edge from the right end of the target edge. In the partial image 800, the pixels marked with the symbol "*" may be pixels of any color.

[0259] The computer device 1 can determine which of the subpatterns 800a to 800e an edge 802 included in the partial image 800 corresponds to, based on the shape of the edge 802b extending from the opposite end of the intersection point with edge 802a.

[0260] As shown in partial image 800a, if edge 802e extends from the upper end of edge 802b in the same direction as edge 802d, interpolation may be performed on the pixels in edge 802 corresponding to the cross pattern of both connections that are tangent to the upper side of the edge of interest (802a). In this case, edges 802b, 802d, and 802e form a U-shaped edge. Furthermore, with respect to the region containing the edge of interest, edge 802e is considered to extend outward.

[0261] Furthermore, as shown in partial image 800b, if edge 802e extends from the upper end of edge 802b in the same direction as edge 802d, and edge 702f extends in the same direction as edge 702b, interpolation may be performed on the pixels in edge 802 corresponding to the cross pattern of both connections that are tangent to the upper side of the edge of interest (802a). In this case, edges 802b, 802e, and 802f form a T-shaped edge. Also, with respect to the region containing the edge of interest, edge 802e is considered to extend outwards. Furthermore, edge 802f can be considered as an extended version of edge 802b.

[0262] In the partial images 800a and 800b, the line segment connecting the point of interest 804a located at the midpoint of edge 802a and the point of interest 804b located at the midpoint of edge 802b may be defined as the reconstruction line 803. In Figure 17, the point of interest 804a on edge 802a is represented by a circle with the letter A in the center, and the point of interest 804b on edge 802b is represented by a circle with the letter B in the center.

[0263] In Figure 17, if the pixels constituting the partial images 800a and 800b are squares with side length 1, and if the point of interest 804a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 802a is L, then the coordinates (x, y) of the point of interest 804a are (0, 0), and the coordinates (x, y) of the point of interest 804b are (L / 2, 0.5).

[0264] Computer device 1 may calculate interpolation weights for white pixels 801a and 801b that intersect with reconstruction line 803, based on the area of ​​the portion enclosed by reconstruction line 803 and edge 802. Furthermore, the pixel to be blended between pixels 801a and 801b may be identified as pixel 801c, which is tangent to the edge where point of interest 804b is located and does not intersect with reconstruction line 803.

[0265] As shown in partial image 800c, if edge 802e extends from the upper end of edge 802b in the same direction as edge 802d, and edge 802g extends in the same direction as the edge of interest (edge ​​802a), interpolation may be performed on the pixels in edge 802 corresponding to the cross pattern of both connections that are tangent to the upper side of the edge of interest (802a). In this case, edges 802b, 802e, and 802g form a T-shaped edge.

[0266] Furthermore, as shown in partial image 800d, if edge 802e extends from the upper end of edge 802b in the same direction as edge 802d, edge 802f extends in the same direction as edge 802b, and edge 802g extends in the same direction as the edge of interest (edge ​​802a), interpolation may be performed on the pixels in edge 802 corresponding to the cross pattern of both connections that are tangent to the upper side of the edge of interest (802a). In this case, edges 802b, 802e, 802f, and 802g form a cross-shaped edge. Also, with respect to the region containing the edge of interest, edge 802e is considered to extend outwards, and edge 802g is considered to extend inwards. Furthermore, edge 802f can be considered to be an extended version of edge 802b.

[0267] In the case shown in partial images 800c and 800d, the line segment connecting point 804a, which is located at the midpoint of edge 802a, and point 804b, which is located between the intersection of edge 802a and edge 802b and the midpoint of edge 802b, may be defined as the reconstruction line 803.

[0268] In Figure 17, if the pixels constituting the partial images 800c and 800d are squares with side length 1, and if the point of interest 804a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 802a is L, then the coordinates (x, y) of the point of interest 804a are (0, 0), and the coordinates (x, y) of the point of interest 804b are (L / 2, 0.25).

[0269] Computer device 1 may calculate interpolation weights for white pixels 801a and 801b that intersect with reconstruction line 803 based on the area of ​​the portion enclosed by reconstruction line 803 and edge 802. Furthermore, the pixel to be blended with pixels 801a and 801b may be identified as pixel 801c, which is tangent to the edge where point of interest 804b is located and does not intersect with reconstruction line 803. It is preferable that computer device 1 also identifies the edges corresponding to the cross pattern of both connections as the same color to be blended for all pixels that intersect with reconstruction line 1.

[0270] Furthermore, as shown in partial image 800e, if the shape of the edge extending from the upper end of edge 802b is different from any of the partial images 800a to 800d, interpolation may not be performed on the pixels adjacent to the upper side of the edge of interest (802a) in edge 802, which corresponds to the cross pattern of both connections. If the shape of the edge extending from the upper end of edge 802b is different from any of the partial images 800a to 800d, this includes, for example, cases where there is no edge 802e extending from the upper end of edge 802b in the same direction as edge 802d. In partial image 800e, the parts marked with the symbol "*+" can be pixels of any color as long as they form a pattern different from any of the partial images 800a to 800d.

[0271] The subpattern identified based on the shape of the edge extending from the lower end of edge 802c, which corresponds to the vertical bar extending from the bottom of the cross, may be a vertical mirror image of partial images 800a to 800e. Furthermore, if the subpattern identified based on the shape of the edge extending from the lower end of edge 802c corresponds to a vertical mirror image of partial images 800a to 800d, interpolation may be performed on the pixels adjacent to the lower side of the edge of interest (802a). The method for calculating the interpolation weights can be the one described above, to the extent necessary. Furthermore, if the subpattern identified based on the shape of the edge extending from the lower end of edge 802c corresponds to a vertical mirror image of partial image 800e, interpolation may not be performed on the pixels adjacent to the lower side of the edge of interest (802a).

[0272] Even in the reconstruction lines identified for edges corresponding to the cross pattern of both connections, the position of the point of interest may be changed according to user input. For example, computer device 1 may change the position of the point of interest in the reconstruction lines identified for edges corresponding to the cross pattern of both connections according to the input values ​​of the smoothness parameter and the extra smoothness parameter. The method for changing the position of the point of interest in the reconstruction lines can be adopted to the extent necessary, as described above.

[0273] The assignment of the numerical values ​​of the smoothness parameter and the extra smoothness parameter to the positions of the points of interest on each edge is not particularly limited and can be designed as appropriate. Furthermore, the way in which the positions of the points of interest are changed according to the numerical values ​​of the smoothness parameter or the extra smoothness parameter is not particularly limited and can be designed as appropriate. Computer device 1 may change the positions of the points of interest such that the numerical values ​​of the interpolation weights increase as the numerical values ​​of the smoothness parameter and / or the extra smoothness parameter increase.

[0274] Next, we will describe an example where the main pattern is a right-connected cross pattern. For example, a right-connected cross pattern may contain two subpatterns. The subpatterns of a right-connected cross pattern may be identified based on the length of the edge portion of the cross-shaped edge that is perpendicular to the edge in a given direction. The "length of the edge portion of the cross-shaped edge that is perpendicular to the edge in a given direction" may be the length of the linearly continuous portion of the edge that extends from the end of the edge of interest in a direction perpendicular to the edge of interest. In this case, the "edge perpendicular to the edge of interest" may be the edge of the two edges corresponding to the vertical bar of the cross that is in contact with the region that may be connected to the region that is in contact with the edge of interest. In the right-connected cross pattern shown in Figure 18, the subpatterns may be identified based on the length of the edge corresponding to the vertical bar extending to the top of the cross.

[0275] Figure 18 is a diagram illustrating a smoothing process according to an embodiment of the present invention. A portion of the image shown in Figure 18 (hereinafter also referred to as the partial image) 900 contains black pixels, white pixels, and dark gray pixels, and between pixels of different colors there are edges 902 (902a to 902d) corresponding to a right-connected cross pattern. In the partial image 900, the edges 902 are represented by light gray dashed lines. In the partial image 900, edge 902a is the edge of interest, edge 902b is an edge extending upward from the right end of the edge of interest in a direction perpendicular to the edge of interest, edge 902c is an edge extending downward from the right end of the edge of interest in a direction perpendicular to the edge of interest, and edge 902d is an edge extending from the right end of the edge of interest in the same direction as the edge of interest. In the partial image 900, pixels marked with the symbol "*" can be pixels of any color.

[0276] The computer device 1 can determine, based on the length of edge 902b, which subpattern of partial image 900a or 900b an edge 902 included in partial image 900 corresponds to.

[0277] As shown in partial image 900a, when edge 902b has a length of two or more pixels in a straight line, interpolation may not be performed on edge 902 corresponding to the right-connected cross pattern. At this time, it can be said that edge 902b extends upward.

[0278] As shown in partial image 900b, when edge 902b does not have a length of two or more pixels in a straight line, interpolation may be performed on edge 902 corresponding to the right-connected cross pattern. In partial image 900b, the portion marked with the symbol "*+" may be pixels of any color as long as they are pixels with a pattern different from that of partial image 900a.

[0279] In this case, a line segment connecting two points, i.e., the focus point 904a located at the midpoint of edge 902a and the focus point 904b located at the midpoint of edge 902b, may be used as the reconstruction line 903. In FIG. 18, the focus point 904a on edge 902a is represented by a circle with the letter "A" in the center, and the focus point 904b on edge 902b is represented by a circle with the letter "B" in the center.

[0280] In FIG. 18, when the pixels constituting partial image 900b are squares with a side length of 1, taking the focus point 904a as the origin, the horizontal grid as the x-axis, the vertical grid as the y-axis, and the length of edge 902a as L, the coordinates (x, y) of focus point 904a are (0, 0), and the coordinates (x, y) of focus point 904b are (L / 2, 0.5).

[0281] Computer device 1 may calculate interpolation weights for white pixels 901a and 901b that intersect with reconstruction line 903 based on the area of ​​the portion enclosed by reconstruction line 903 and edge 902. Furthermore, the pixel to be blended with pixels 901a and 901b may be identified as pixel 901c, which is tangent to the edge where point of interest 904b is located and does not intersect with reconstruction line 903. It is preferable that computer device 1 also identifies the same color as the blend color for all pixels that intersect with reconstruction line 1, including edges corresponding to the right-connected cross pattern.

[0282] Even in the reconstruction lines identified for edges corresponding to the right-connected cross pattern, the position of the point of interest may be changed according to user input. For example, computer device 1 may change the position of the point of interest in the reconstruction lines identified for edges corresponding to the right-connected cross pattern according to the input values ​​of the smoothness parameter and the extra smoothness parameter. The method for changing the position of the point of interest in the reconstruction lines can be adopted to the extent necessary, as described above.

[0283] The assignment of the numerical values ​​of the smoothness parameter and the extra smoothness parameter to the positions of the points of interest on each edge is not particularly limited and can be designed as appropriate. Furthermore, the way in which the positions of the points of interest are changed according to the numerical values ​​of the smoothness parameter or the extra smoothness parameter is not particularly limited and can be designed as appropriate. Computer device 1 may change the positions of the points of interest such that the numerical values ​​of the interpolation weights increase as the numerical values ​​of the smoothness parameter and / or the extra smoothness parameter increase.

[0284] Next, we will describe an example where the main pattern is a left-connected cross pattern. For example, a left-connected cross pattern may contain two subpatterns. The subpatterns of a left-connected cross pattern may be identified based on the length of the edge portion of the cross-shaped edge that is perpendicular to the edge in a given direction. The "length of the edge portion of the cross-shaped edge that is perpendicular to the edge in a given direction" may be the length of the linearly continuous portion of the edge that extends from the end of the edge of interest and is perpendicular to the edge of interest. In this case, the "edge perpendicular to the edge of interest" may be the edge of the two edges corresponding to the vertical bars of the cross that is in contact with the region that may be connected to the region that is in contact with the edge of interest. In the right-connected cross pattern shown in Figure 18, the subpatterns may be identified based on the length of the edge corresponding to the vertical bar extending from the bottom of the cross.

[0285] Figure 19 is a diagram illustrating a smoothing process according to an embodiment of the present invention. A portion of the image shown in Figure 19 (hereinafter also referred to as a partial image) 1000 contains black pixels, white pixels, and dark gray pixels, and between pixels of different colors there are edges 1002 (1002a to 1002d) corresponding to a left-connected cross pattern. In the partial image 1000, the edges 1002 are represented by light gray dashed lines. In the partial image 1000, edge 1002a is the edge of interest, edge 1002b is an edge extending upward from the right end of the edge of interest in a direction perpendicular to the edge of interest, edge 1002c is an edge extending downward from the right end of the edge of interest in a direction perpendicular to the edge of interest, and edge 1002d is an edge extending from the right end of the edge of interest in the same direction as the edge of interest. In the partial image 1000, pixels marked with the symbol "*" can be pixels of any color.

[0286] The computer device 1 can determine, based on the length of edge 1002c, which subpattern of partial image 1000a or 1000b an edge 1002 included in partial image 1000 corresponds to.

[0287] As shown in partial image 1000a, if edge 1002c has a length of two pixels or more in a straight line, interpolation may not be performed on edge 1002, which corresponds to the left-connected cross pattern. In this case, edge 1002c can be said to extend downwards.

[0288] As shown in partial image 1000b, if edge 1002c does not have a length of two pixels or more in a straight line, interpolation may be performed on edge 1002 which corresponds to a left-connected cross pattern. In partial image 1000b, the parts marked with the symbol "*+" can be pixels of any color as long as they form a different pattern from partial image 1000a.

[0289] In this case, the line segment connecting point 1004a, located at the midpoint of edge 1002a, and point 1004b, located at the midpoint of edge 1002c, may be defined as the reconstruction line 1003. In Figure 19, point 1004a on edge 1002a is represented by a circle with the letter A in the center, and point 1004b on edge 1002c is represented by a circle with the letter B in the center.

[0290] In Figure 19, if the pixels constituting the partial image 1000b are squares with side length 1, and if the point of interest 1004a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 1002a is L, then the coordinates (x, y) of point of interest 1004a are (0, 0), and the coordinates (x, y) of point of interest 1004b are (L / 2, -0.5).

[0291] Computer device 1 may calculate interpolation weights for black pixels 1001a and 1001b that intersect with reconstruction line 1003 based on the area of ​​the portion enclosed by reconstruction line 1003 and edge 1002. Furthermore, the pixel to be blended with pixels 1001a and 1001b may be identified as pixel 1001c, which is tangent to the edge where point of interest 1004b is located and does not intersect with reconstruction line 1003. It is preferable that computer device 1 also identifies the same color as the blend color for all pixels that intersect with reconstruction line 1, including edges corresponding to the left-connected cross pattern.

[0292] Even in the reconstruction lines identified for edges corresponding to the left-connected cross pattern, the position of the point of interest may be changed according to user input. For example, computer device 1 may change the position of the point of interest in the reconstruction lines identified for edges corresponding to the left-connected cross pattern according to the input values ​​of the smoothness parameter and the extra smoothness parameter. The method for changing the position of the point of interest in the reconstruction lines can be adopted to the extent necessary, as described above.

[0293] The assignment of the numerical values ​​of the smoothness parameter and the extra smoothness parameter to the positions of the points of interest on each edge is not particularly limited and can be designed as appropriate. Furthermore, the way in which the positions of the points of interest are changed according to the numerical values ​​of the smoothness parameter or the extra smoothness parameter is not particularly limited and can be designed as appropriate. Computer device 1 may change the positions of the points of interest such that the numerical values ​​of the interpolation weights increase as the numerical values ​​of the smoothness parameter and / or the extra smoothness parameter increase.

[0294] Next, we will describe an example where the main pattern is a disconnected cross pattern. For example, a disconnected cross pattern may contain 16 subpatterns. The subpatterns of a disconnected cross pattern are edges located near the ends of the cross-shaped edges and may be identified based on the shape and / or direction of the edges that connect to the cross-shaped edges. "Edges located near the ends of the cross-shaped edges and that connect to the cross-shaped edges" may be edges extending from the ends of the edges corresponding to the vertical bars of the cross. The edges corresponding to the vertical bars of the cross may be edges within the cross-shaped edges that are perpendicular to the edge of interest.

[0295] The vertical bars of the cross can be vertical bars extending upwards and vertical bars extending downwards. For example, there may be four subpatterns identified based on the shape and / or direction of the edges extending from the ends of the edges corresponding to the vertical bars extending upwards of the cross, and four subpatterns identified based on the shape and / or direction of the edges extending from the ends of the edges corresponding to the vertical bars extending downwards of the cross. Combinations of these four subpatterns may constitute 16 subpatterns. Pixels that can be interpolated based on the subpatterns identified based on the shape and / or direction of the edges extending from the ends of the edges corresponding to the vertical bars extending upwards of the cross may be pixels tangent to the upper side of the edge of interest. Similarly, pixels that can be interpolated based on the subpatterns identified based on the shape and / or direction of the edges extending from the ends of the edges corresponding to the vertical bars extending downwards of the cross may be pixels tangent to the lower side of the edge of interest.

[0296] Furthermore, for edges corresponding to unconnected subpatterns of a cross pattern, the interpolation weights may be calculated in a different way based on the shape and / or direction of the edges at the ends of the edges that are continuous with the edges located at the ends of the cross-shaped edges. "Edges that are continuous with the edges located at the ends of the cross-shaped edges" may refer to edges that are linearly continuous with the edges in the same direction as the edge of interest, corresponding to the horizontal bars of the cross. The ends of such edges will be described later.

[0297] The following describes the subpatterns identified based on the shape of the edges extending from the ends of the edges corresponding to the vertical bars extending from the bottom of the cross, with reference to Figure 20. The subpatterns identified based on the shape of the edges extending from the ends of the edges corresponding to the vertical bars extending from the top of the cross may be the example shown in Figure 20, mirrored vertically.

[0298] Figure 20 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 20(A) is an example of a subpattern identified based on the shape and / or direction of the edges extending from the ends of the edges corresponding to the vertical bars extending to the lower side of the cross, and Figure 20(B) is an example of an interpolation method identified based on the shape and / or direction of the ends of the edges corresponding to the horizontal bars of the cross.

[0299] The partial image 1100 shown in Figure 20 (hereinafter also referred to as the partial image) contains black pixels, white pixels, dark gray pixels, and light gray pixels, and between pixels of different colors there are edges 1102 (1102a to 1102d) that correspond to an unconnected cross pattern. In the partial image 1100, edges 1102 are represented by light gray dashed lines. In the partial image 1100, edge 1102a is the edge of interest, edge 1102b is an edge that extends downward from the right end of the edge of interest and is perpendicular to the edge of interest, edge 1102c is an edge that extends from the right end of the edge of interest in the same direction as the edge of interest, and edge 1102d is an edge that extends upward from the right end of the edge of interest and is perpendicular to the edge of interest. In the partial image 1100, pixels marked with the symbol "*" can be any color.

[0300] The computer device 1 can determine which of the subpatterns 1100a to 1100d the edge 1102 included in the partial image 1100 corresponds to, based on the shape of the edge 1102b extending from the end opposite to the intersection with edge 1102a.

[0301] As shown in partial image 1100a, if there is an edge 1102e extending laterally from the end of edge 1102b toward the side where the edge of interest (edge ​​1102a) exists, interpolation may be performed on edge 1102, which corresponds to an unconnected cross pattern. In this case, edges 1102a, 1102b, and 1102e form a U-shaped edge. Furthermore, using the region adjacent to the lower side of the edge of interest as a reference, edge 1102e is considered to extend inward.

[0302] In this case, the line segment connecting the point of interest 1104a located at the midpoint of edge 1102a and the point of interest 1104b located at the midpoint of edge 1102b may be defined as the reconstruction line 1103. In Figure 20, the point of interest 1104a on edge 1102a is represented by a circle with the letter A in the center, and the point of interest 1104b on edge 1102b is represented by a circle with the letter B in the center.

[0303] In Figure 20, if the pixels constituting the partial image 1100a are squares with side length 1, and if the point of interest 1104a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 1102a is L, then the coordinates (x, y) of the point of interest 1104a are (0, 0), and the coordinates (x, y) of the point of interest 1104b are (L / 2, -0.5).

[0304] Computer device 1 may calculate interpolation weights for black pixels 1101a and 1101b that intersect with reconstruction line 1103, based on the area of ​​the region enclosed by reconstruction line 1103 and edge 1102. Furthermore, the pixel to be blended between pixels 1101a and 1101b may be identified as pixel 1101c, which is tangent to the edge where point of interest 1104b is located and does not intersect with reconstruction line 1103.

[0305] As shown in partial image 1100b, if there is an edge 1102e extending laterally from the end of edge 1102b toward the edge of interest (edge ​​1102a), and an edge 1102f extending laterally toward the edge of edge 1102c, interpolation may be performed on edge 1102, which corresponds to an unconnected cross pattern. In this case, edges 1102b, 1102e, and 1102f form a T-shaped edge. Furthermore, using the region adjacent to the lower side of the edge of interest as a reference, 1102e is considered to extend inward, and 1102f is considered to extend outward. In this case, the interpolation weights may be calculated based on the shape and / or direction of the end of 1102c using the method shown in partial image 1100b-1 or partial image 1100b-2 of Figure 20(B). The method shown in partial image 1100b-1 and the method shown in partial image 1100b-2 have different positions for the point of interest on edge 1102b.

[0306] Furthermore, as shown in partial image 1100c, if edges 1102e, 1102f, and 1102g extend from the end of edge 1102b in the same direction as the edge of interest (edge ​​1102a), respectively, in the same direction as edge 1102c, then interpolation may be performed on edge 1102, which corresponds to an unconnected cross pattern. In this case, edges 1102b, 1102e, 1102f, and 1102g form a cross-shaped edge. Also, using the region adjacent to the lower side of the edge of interest as a reference, 1102e is considered to extend inward, and 1102f is considered to extend outward. Furthermore, edge 1102g can be considered as an extended version of edge 1102b. In this case, the interpolation weights may be calculated based on the shape and / or direction of the end of 1102c using the method shown in partial image 1100c-1 or partial image 1100c-2 of Figure 20(B). The position of the point of interest on edge 1102b differs between the method shown in partial image 1100c-1 and the method shown in partial image 1100c-2.

[0307] For the end of edge 1102c and the end of edge 1102c, T 1 Descriptions of the edge of edge 702c and the end of edge 702c in the pattern may be adopted to the extent necessary.

[0308] For how to find the right edge of edge 1102c, see T 1 The description of how to find the rightmost edge of edge 702c in the pattern can be adopted to the extent necessary.

[0309] For example, the edge 1102c shown in partial image 1100e and the edge 1102c shown in partial image 1100f of Figure 20(B) may be determined to be continuous with the starting edge 1102c.

[0310] Furthermore, for example, the center of the pixel group shown in partial image 1100g in Figure 20(B) may be determined to be the right end of edge 1102c. At the right end of edge 1102c, an L-shape is formed by edge 1102c and edge 1102g. Also, the direction in which edge 1102g, which corresponds to the horizontal bar of the L-shape, extends is opposite to the direction of edge 1102b, which corresponds to the vertical bar extending downwards of the cross. When the right end of edge 1102c has such an L-shape, as shown in partial images 1100b-1 / 1100c-1, the line segment connecting point 1104a, which is located at the midpoint of edge 1102a, and point 1104b, which is located at the midpoint of edge 1102b, may be defined as the reconstruction line 1103.

[0311] In Figure 20(B), if the pixels constituting the partial images 1100b-1 and 1100c-1 are squares with side length 1, and if the point of interest 1104a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 1102a is L, then the coordinates (x, y) of the point of interest 1104a are (0, 0), and the coordinates (x, y) of the point of interest 1104b are (L / 2, -0.5).

[0312] Furthermore, for example, the rightmost point of edge 1102c in partial image 1100h of Figure 20(B) may be determined to be the rightmost point of edge 1102c. In this case, as shown in partial images 1100b-2 / 1100c-2, the line segment connecting point 1104a, which is located at the midpoint of edge 1102a, and point 1104b, which is located between the intersection of edge 1102a and edge 1102b and the midpoint of edge 1102b, may be defined as the reconstruction line 1103.

[0313] In Figure 20(B), if the pixels constituting the partial images 1100b-1 and 1100c-1 are squares with side length 1, and if the point of interest 1104a is the origin, the horizontal grid is the x-axis, the vertical grid is the y-axis, and the length of edge 1102a is L, then the coordinates (x, y) of the point of interest 1104a are (0, 0), and the coordinates (x, y) of the point of interest 1104b are (L / 2, -0.25).

[0314] Computer device 1 may calculate interpolation weights for black pixels 1101a and 1101b that intersect with reconstruction line 1103 based on the area of ​​the portion enclosed by reconstruction line 1103 and edge 1102. Furthermore, the pixel to be blended with pixels 1101a and 1101b may be identified as pixel 1101c, which is tangent to the edge where point of interest 1104b is located and does not intersect with reconstruction line 1103. It is preferable that computer device 1 also identifies the same color as the blend color for all pixels that intersect with reconstruction line 1, even for edges corresponding to unconnected cross patterns.

[0315] Furthermore, the edge patterns shown in partial images 1100b-1 / 1100c-1 and 1100b-2 / 1100c-2 may be considered a type of subpattern. In that case, the unconnected cross pattern may be considered to contain 36 subpatterns.

[0316] Furthermore, as shown in partial image 1100d of Figure 20(A), if the shape of the edge extending from the end of edge 1102b is different from any of the partial images 1100a to 1100c, interpolation may not be performed on edge 1102, which corresponds to an unconnected cross pattern. Cases where the shape of the edge extending from the end of edge 1102b is different from any of the partial images 1100a to 1100c include, for example, cases where there is no edge extending horizontally from the end of edge 1102b toward the edge of interest (edge ​​1102a). In partial image 1100d, the parts marked with the symbol "*+" can be pixels of any color as long as they form a pattern different from any of the partial images 1100a to 1100c.

[0317] The subpattern identified based on the shape and / or direction of the edge extending from the upper end of edge 1102d, which corresponds to the vertical bar extending to the upper side of the cross, may be a vertical mirror image of partial images 1100a to 1100d. Furthermore, if the subpattern identified based on the shape and / or direction of the edge extending from the upper end of edge 1102d corresponds to a vertical mirror image of partial images 1100a to 1100c, interpolation may be performed on the pixels touching the upper side of the edge of interest (1102a). The method for calculating the interpolation weights can be found by referring to the above description to the extent necessary. Furthermore, if the subpattern identified based on the shape and / or direction of the edge extending from the upper end of edge 1102d corresponds to a vertical mirror image of partial image 1100d, interpolation may not be performed on the pixels touching the upper side of the edge of interest (1102a).

[0318] Subpatterns identified based on the shape and / or direction of edges extending from the ends of edges corresponding to the vertical bars extending on the upper side of the cross, and subpatterns identified based on the shape and / or direction of edges extending from the ends of edges corresponding to the vertical bars extending on the lower side of the cross, are respectively T 1 It is similar to a subpattern of the pattern. The edges corresponding to the unconnected cross pattern are T 1The pattern can be seen as two edges sharing a T-shaped horizontal bar and joining together in opposite directions.

[0319] Even in the reconstruction lines identified for edges corresponding to the unconnected cross pattern, the position of the point of interest may be changed according to user input. For example, computer device 1 may change the position of the point of interest in the reconstruction lines identified for edges corresponding to the unconnected cross pattern according to the input values ​​of the smoothness parameter and the extra smoothness parameter. The method for changing the position of the point of interest in the reconstruction lines can be adopted to the extent necessary, as described above.

[0320] The assignment of the numerical values ​​of the smoothness parameter and the extra smoothness parameter to the positions of the points of interest on each edge is not particularly limited and can be designed as appropriate. Furthermore, the way in which the positions of the points of interest are changed according to the numerical values ​​of the smoothness parameter or the extra smoothness parameter is not particularly limited and can be designed as appropriate. Computer device 1 may change the positions of the points of interest such that the numerical values ​​of the interpolation weights increase as the numerical values ​​of the smoothness parameter and / or the extra smoothness parameter increase.

[0321] In conventional MLAA, edges corresponding to a cross-shaped pattern are treated as a combination of four L-shaped edges, resulting in interpolation of the pixel colors in the four regions adjacent to the edge, causing the edge to blur unnecessarily. On the other hand, according to the program of the present invention, when an edge is identified as a pattern containing a cross shape, weights can be calculated so that the pixel colors of three or fewer of the four regions adjacent to the edge are interpolated, thus preventing the edge from blurring unnecessarily. Furthermore, according to the program of the present invention, subpatterns of edges can be identified based on the continuity of colors in the diagonal direction surrounding the cross-shaped edge, and interpolation weights can be calculated in different ways depending on the identified subpattern, so weights can be calculated in different ways depending on the color combination of the four regions adjacent to the edge.

[0322] Furthermore, in typical MLAA, for pixels that intersect a reconstruction line, the color of the adjacent pixels on either side of the horizontal edge of the L-shape is blended for pixels touching the horizontal edge, and the color of the adjacent pixels on either side of the vertical edge of the L-shape is blended for pixels touching the vertical edge of the L-shape. As a result, pixels touching the cross-shaped edges are not blended in a gradient-like manner, but are interpolated in an unnatural way, with the colors mixed together. On the other hand, according to the program of the present invention, the same color can be identified as the color to blend for all pixels that intersect a reconstruction line, so that the color of pixels touching the cross-shaped edges can be interpolated in a natural, gradient-like manner.

[0323] Next, we will explain how to calculate interpolation weights for edges that correspond to the higher-level "diagonal" pattern. In a 3x3 pattern, if an edge is identified as "diagonal," the computer device 1 may then search for the ends of diagonal edges that are continuous with the edge of interest. The method for searching for the ends of the edge of interest is not particularly limited and can be designed as appropriate.

[0324] Figure 21 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 21 shows an example of a method for finding the ends of diagonally extending edges for a 3x3 pattern 210 corresponding to the "diagonal type" shown in Figure 6.

[0325] Computer device 1 may, in order to find the lower left end of an edge, start with a set of two pixels flanking edge 212 (the pixel of interest 211 and a pixel adjacent to the pixel of interest 211 in the downward direction), as shown in the figure, and search for the lower left end of the edge by shifting the position of the set of pixels one pixel at a time in the downward left direction. In Figure 21, the set of pixels that serves as the starting point in the 3x3 pattern 210 is enclosed by a gray line. Computer device 1 can determine whether or not the edges are connected based on the edge information contained in the set of pixels.

[0326] If the color combination and arrangement of two pixels included in the set of pixels being searched are the same as the two pixels at the starting point, the computer device 1 may determine that the edge between the two pixels being searched is continuous with the starting edge 212. In this case, the computer device 1 may also refer to the edge information of the set of pixels located one pixel to the lower left.

[0327] If the color combination and arrangement of two pixels in the set of pixels being searched are not the same as the two pixels at the starting point, the computer device 1 may determine that the continuity of the staircase-shaped edge from the starting point edge 212 is broken. In this case, if the color of the upper pixel in the set of two pixels is the same as the color of the upper pixel (the pixel of interest) in the set of pixels at the starting point, the lower end of the vertical edge on the right edge of the upper pixel may be the lower left end of the edge. Also, if the color of the upper pixel in the set of two pixels is not the same as the color of the upper pixel (the pixel of interest) in the set of pixels at the starting point, the left end of the horizontal edge between the two sets of pixels that were last determined to be connected from the starting point edge 212 may be the lower left end of the edge. In other words, the end of the last edge connecting in a staircase shape (a shape that extends alternately in the vertical and horizontal directions) may be the lower left end of the edge.

[0328] The pixel identified as being located at the lower left end of an edge (hereinafter also referred to as the lower left edge pixel) is the pixel that forms the edge that was last determined to be continuous with the starting edge 212, and may be a pixel that constitutes the same region as the pixel of interest, or it may be a pixel that forms a vertically extending edge that connects to the edge that was last determined to be continuous with the starting edge 212, and may be a pixel that constitutes the same region as the pixel of interest. In other words, the last pixel lined up diagonally may be the lower left edge pixel.

[0329] In the case of the "diagonal type," the "end of the edge" may refer to the vicinity of the edge, just as in the "linear type."

[0330] On the right side of Figure 21, an example of a group of pixels in the shape of a rectangle (2 pixels horizontally and 3 pixels vertically) including the lower left corner of the edge (hereinafter also referred to as a 2x3 pattern) is shown. The pixels in the 2x3 pattern 213 (213a~213e) shown in Figure 21 are black, dark gray, light gray, and white, and an edge exists on the side or vertex where pixels of different colors are adjacent. In Figure 21, the edge located below the black pixels lined up diagonally is the edge of interest. Finally, among the pixels that form the edge determined to be connected from the starting edge 212, a gray circle is placed in the center of the white pixel located at the bottom.

[0331] In 2x3 patterns 213a and 213b, the pixel adjacent to the left of the white pixel marked with a gray circle may be the bottom-left edge pixel located at the bottom-left end of the diagonally arranged black pixels. In 2x3 patterns 213c to 213e, the pixel adjacent above the white pixel marked with a gray circle may be the bottom-left edge pixel located at the bottom-left end of the diagonally arranged black pixels.

[0332] In patterns that fall under the "diagonal" category, two adjacent regions separated by a diagonal edge are touching on the upper and lower sides of the edge and have different colors.

[0333] For information on how to find the upper right corner of an edge, refer to the above description to the extent necessary. An example of a 2x3 pattern containing the upper right corner of the edge of interest may be a 180° rotation of 2x3 patterns 213a to 213e.

[0334] After identifying the lower-left and upper-right ends of the edge of interest, the computer device 1 may identify the main pattern of the edge based on the number of diagonally aligned pixels that form the diagonal edge. Since the method of calculating interpolation weights changes depending on the main pattern of the edge, the computer device 1 may calculate the interpolation weights in different ways depending on the number of pixels that form the diagonal edge when the edge is identified as having a diagonal pattern.

[0335] The type of main edge pattern is not particularly limited and can be designed as appropriate. Here, we will explain using two types of main patterns as examples: the "small corner pattern," in which the edges form small corners, and the "diagonal line pattern," in which the edges form diagonal lines.

[0336] For example, if the number of pixels arranged diagonally in the computer device 1 is less than or equal to a predetermined number, the diagonal edge may be defined as a small corner pattern. The predetermined number is not particularly limited and can be designed as appropriate. For example, the predetermined number may be four, three, or two.

[0337] Figure 22 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 22 shows an example of a method for calculating interpolation weights when there are three pixels arranged diagonally and the diagonal edges are identified as small corner patterns. In Figure 22, pixels marked with the symbol "*" can be pixels of any color.

[0338] A portion of the image shown in Figure 22 (hereinafter also referred to as the partial image) 1200 contains black pixels 1201a to 1201c and white pixels 1201d to 1201f. Between the black and white pixels, there is a stepped edge 1202, which corresponds to a small corner pattern. In the partial image 1200, the edge 1202 is represented by a light gray dashed line. If the black pixel 1201b is the pixel of interest, then the edge 1202 is the edge of interest.

[0339] If the computer device 1 identifies that the stepped edge 1202 corresponds to a small corner pattern, it may calculate interpolation weights by treating each edge forming each step of the staircase as a corner having a length of 1 pixel horizontally and 1 pixel vertically. In this case, interpolation weights may be calculated for the pixel located at the tip of the corner and for the three pixels adjacent to that pixel across the edge.

[0340] For example, in Figure 22, if the black pixel 1201b is treated as a corner and interpolated, interpolation weights may be calculated for the pixel 1201b that constitutes the corner, as well as for the pixels 1201f, 1201e, and 1201g that are adjacent to pixel 1201b across edge 1202. Alternatively, for example, as shown on the right side of Figure 22, the interpolation weight for pixel 1201b may be calculated as "0.875", the interpolation weight for pixel 1201f as "0.05", the interpolation weight for pixel 1201e as "0.05", and the interpolation weight for pixel 1201g as "0.025".

[0341] In this case, the pixel to be blended with pixels 1201f, 1201e, and 1201g may be pixel 1201b, which constitutes the corner. In other words, the computer device 1 may specify that the color to be blended with pixels 1201f, 1201e, and 1201g is the color of pixel 1201b, which is black.

[0342] Furthermore, the pixels to be blended with pixel 1201b may be 1201f or 1201e, which are adjacent to it across a diagonal edge. In other words, the computer device 1 may specify that the color to be blended with pixel 1201b is white, which is the color of pixel 1201f or 1201e.

[0343] The method for calculating the interpolation weight for each pixel is not particularly limited and can be designed as appropriate. For example, the interpolation weight for each pixel in a small angular pattern may be calculated according to a predetermined formula including a smoothness parameter and / or an extra smoothness parameter. In other words, different numerical values ​​may be calculated for the interpolation weight of each pixel in a small angular pattern depending on the user-inputted information regarding the criteria for calculating the interpolation weight. Alternatively, the interpolation weight for each pixel in a small angular pattern may be configured to be calculated at a predetermined value, regardless of user input.

[0344] Furthermore, for example, if the number of pixels arranged diagonally in the computer device 1 is greater than or equal to a predetermined number, the diagonal edges may be defined as diagonal line patterns. The predetermined number is not particularly limited and can be designed as appropriate. For example, the predetermined number may be four, three, or two.

[0345] If an edge corresponds to a diagonal line pattern, the edge subpattern may be identified based on the pattern of the edge end. The edge end pattern may be identified based on the length and / or direction of the diagonal edge end. The types of edge end patterns and edge subpatterns are not particularly limited and can be designed as appropriate.

[0346] Figure 23 is a diagram illustrating a smoothing process according to an embodiment of the present invention. In the table in Figure 23, the left column shows the names of the edge end patterns (hereinafter also referred to as edge pattern names), and the right column shows at least some examples of pixel groups with a rectangular shape of 3 pixels horizontally and 4 pixels vertically (hereinafter also referred to as 3x4 patterns) corresponding to each edge pattern.

[0347] The pixels in the 3x4 pattern shown in Figure 23 are black, dark gray, and white, and edges exist on adjacent sides or vertices where pixels of different colors are adjacent. In Figure 23, edges are represented by light gray dashed lines. Edges that are not necessary for explanation are omitted from the dashed lines. Also, in Figure 23, pixels marked with the symbol "*" can be of any color. Furthermore, in Figure 23, squares marked with the symbol "×" represent the outside of the input image. Also, in Figure 23, pixels labeled "NW" are pixels of a color other than white.

[0348] In Figure 23, the left column shows four types of edge pattern names: "horizontal type," "horizontal type," "vertical type," and "vertical type." If the lower left end of an edge is the left end of a horizontal edge, it may be classified as "horizontal type" or "horizontal type," and if the lower left end of an edge is the bottom end of a vertical edge, it may be classified as "vertical type" or "vertical type."

[0349] In Figure 23, one pixel group is shown as an example of a 3x4 pattern corresponding to the "horizontal type". In the 3x4 pattern 1300 shown in Figure 23, the left end of edge 1302, located on the lower edge of the lower left pixel 1301, becomes the lower left end of the diagonal edge. Edge 1302 extends linearly to the left and has a length of 2 pixels. The end of the diagonal edge corresponding to the "horizontal type" may extend horizontally by more than 2 pixels.

[0350] If the lower left end of an edge corresponds to a "horizontally elongated" pattern, the point of interest may be located on an edge that has a length of 2 pixels horizontally and includes the lower left end of the diagonal edge.

[0351] In Figure 23, two pixel groups are shown as examples of 3x4 patterns corresponding to the "horizontal" type. In 3x4 pattern 1310, the left end of edge 1312, located on the lower edge of the lower-left pixel 1311, becomes the lower-left end of the diagonal edge. Edge 1312 has a length of 1 pixel. In 3x4 pattern 1320, the left end of edge 1322, located on the lower edge of the lower-left pixel 1321, becomes the lower-left end of the diagonal edge. Edge 1322 has a length of 1 pixel.

[0352] If the lower left end of an edge corresponds to a "horizontal" pattern, the point of interest may be located on an edge that has a horizontal length of 1 pixel and includes the lower left end of a diagonal edge.

[0353] In Figure 23, one pixel group is shown as an example of a 3x4 pattern corresponding to the "vertically elongated" type. In the 3x4 pattern 1330 shown in Figure 23, the lower end of edge 1332, located on the right edge of the lower left pixel 1331, becomes the lower left end of the diagonal edge. Edge 1332 extends linearly downwards and has a length of 2 pixels. The ends of the diagonal edges corresponding to the "vertically elongated" type may extend vertically by more than 2 pixels.

[0354] If the lower left end of an edge corresponds to a "vertically elongated" pattern, the point of interest may be located on an edge that has a vertical length of 2 pixels and includes the lower left end of a diagonal edge.

[0355] In Figure 23, three pixel groups are shown as examples of 3x4 patterns corresponding to the "vertical" orientation. In 3x4 pattern 1340, the bottom-left pixel 1341 is located at the edge of the image. In this case, an edge 1342 exists on the left edge of the adjacent pixel below the bottom-left pixel 1341, and the bottom end of edge 1342 may be considered the bottom-left end of the diagonal edge. Edge 1342 has a length of 1 pixel. In 3x4 pattern 1350, the bottom end of edge 1352, which is located on the right edge of the bottom-left pixel 1351, becomes the bottom-left end of the diagonal edge. Edge 1352 has a length of 1 pixel. In 3x4 pattern 1360, the bottom-left pixel 1361 is located at the edge of the image. In this case, the bottom end of edge 1362, which is located on the right edge of the bottom-left pixel 1361, becomes the bottom-left end of the diagonal edge. Edge 1362 has a length of 1 pixel.

[0356] If the lower left end of an edge corresponds to a "vertical" pattern, the point of interest may be located on an edge that has a vertical length of 1 pixel and includes the lower left end of a diagonal edge.

[0357] In Figure 23, the pattern at the lower left end of the diagonal edge is shown as an example, but the pattern at the upper right end of the diagonal edge may be a 3x4 pattern 1300-1360 rotated by 180°.

[0358] Furthermore, the subpattern of an edge may be identified by a combination of the pattern at the lower left end and the pattern at the upper right end. For example, if there are four types of patterns at the ends of the edge, there may be 16 types of subpatterns, which are combinations of the four types of patterns at the lower left end and the four types of patterns at the upper right end. The computer device 1 can identify the subpattern of an edge based on the combination of patterns at both ends of the edge.

[0359] If the edge is identified as a diagonal line pattern, the computer device 1 may identify a mathematical formula corresponding to the reconstruction line connecting the points of interest located at each end of the edge, and calculate the interpolation weights for each pixel based on the identified formula.

[0360] Figure 24 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 24(A) is a diagram illustrating a method for identifying a mathematical formula corresponding to a reconstruction line, and Figure 24(B) is a diagram illustrating a method for calculating the weight of each pixel based on the identified mathematical formula. In Figure 24, pixels marked with the symbol "*" may be pixels of any color.

[0361] A portion of the image shown in Figure 24(A), 1400a (hereinafter also referred to as the partial image), contains both black and white pixels. Although not shown in the illustration, edges exist between the black and white pixels. The edge 1402a located below the four black pixels arranged diagonally corresponds to a "diagonal line pattern".

[0362] Since the pattern at the lower left end of edge 1402a is "horizontal," point of interest 1404a is located on a horizontal edge that exists on the lower edge of the lower left end pixel 1401a. Also, since the pattern at the upper right end of edge 1402a is "horizontal," point of interest 1404b is located on a horizontal edge that exists on the lower edge of the upper right end pixel 1401b. In Figure 24(A), point of interest 1404a is represented by a circle with the letter A in the center, and point of interest 1404b is represented by a circle with the letter B in the center. Reconstruction line 1403a may be a line segment connecting point of interest 1404a and point of interest 1404b.

[0363] If the pixels constituting the partial image 1400a are squares with side length 1, and the left edge of the partial image 1400a is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the coordinates (x, y) of point of interest 1404a are (1.5, 1), and the coordinates (x, y) of point of interest 1404b are (4.5, 4).

[0364] In step S302, interpolation weights may be calculated for pixels that intersect with the reconstruction line. For edges corresponding to diagonal line patterns, both pixels of the two colors opposite each other on either side of the edge intersect with the reconstruction line, so interpolation weights may be calculated for both pixels of the two colors opposite each other on either side of the edge.

[0365] The partial image 1400b shown in Figure 24(B) contains both black and white pixels. Although not shown, edges exist between the black and white pixels. The edge 1402b located below the five black pixels arranged diagonally corresponds to a diagonal line pattern.

[0366] Since the pattern at the lower left end of edge 1402b is "elongated vertically", point of interest 1404c is located on the right edge of the lower left end pixel 1401c, on a vertical edge that is two pixels long. Similarly, since the pattern at the upper right end of edge 1402b is "elongated vertically", point of interest 1404d is located on the right edge of the upper right end pixel 1401g, on a vertical edge that is two pixels long. In Figure 24(B), points of interest 1404c and 1404d are represented by gray circles. The reconstruction line 1403b may be a line segment connecting points of interest 1404c and 1404d.

[0367] As shown in the figure, the pixels that intersect with the reconstruction line 1403b are the four black pixels 1401d to 1401g and the four white pixels 1401h to 1401k. The interpolation weights may be determined based on the area of ​​the region enclosed by the reconstruction line and the edge. For example, the computer device 1 may calculate for each pixel the ratio of the area of ​​the region enclosed by the reconstruction line 1403b and the stepped edge 1402b to the area of ​​the pixel, and use the calculated ratio as the interpolation weight.

[0368] Computer device 1 may identify the color of adjacent pixels separated by a diagonal edge as the color to be blended for each pixel. For example, for black pixels 1401d to 1401g, the adjacent white pixels 1401h to 1401k separated by edge 1402b may be the pixels to be blended. Similarly, for white pixels 1401h to 1401k, the adjacent black pixels 1401d to 1401g separated by edge 1402b may be the pixels to be blended.

[0369] Computer device 1 may take points of interest at different locations depending on the edge subpattern and identify formulas corresponding to the reconstruction lines. Since the edge subpattern is identified by the length and / or direction of the ends of the diagonal edges, if computer device 1 identifies that the edge is a pattern containing diagonal edges, it may calculate interpolation weights in different ways depending on the length and / or direction of the ends of the diagonal edges. Below, with reference to Figure 25, an example of how to identify the position of the point of interest and identify formulas corresponding to the reconstruction lines when the lower left end of the edge corresponds to a "horizontal" type will be described.

[0370] Figure 25 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 25(A) is an example of an edge where the lower left end of the edge corresponds to a "horizontal" subpattern and the upper right end of the edge corresponds to a "horizontal" subpattern; Figure 25(B) is an example of an edge where the lower left end of the edge corresponds to a "horizontal" subpattern and the upper right end of the edge corresponds to a "horizontally elongated" subpattern; Figure 25(C) is an example of an edge where the lower left end of the edge corresponds to a "horizontal" subpattern and the upper right end of the edge corresponds to a "vertical" subpattern; and Figure 25(D) is an example of an edge where the lower left end of the edge corresponds to a "horizontal" subpattern and the upper right end of the edge corresponds to a "vertically elongated" subpattern.

[0371] The 1500 pixels in a portion of the image shown in Figure 25 (hereinafter also referred to as the partial image) contain both black and white pixels, and there are edges corresponding to diagonal line patterns between the black and white pixels. Note that in Figure 25, the dashed lines and symbols indicating edges are omitted. In Figure 25, there are four black pixels aligned diagonally. Also, in Figure 25, pixels marked with the symbol "*" can be of any color.

[0372] As shown in the partial image 1500a of Figure 25(A), when the lower left end of an edge corresponds to a "horizontal" pattern and the upper right end of an edge corresponds to a subpattern that is also "horizontal", the lower left end pixel of the diagonal edge is pixel 1501a, and the upper right end pixel of the diagonal edge is pixel 1501b. In this case, the computer device 1 may designate the midpoint of an edge that includes the lower left end of the diagonal edge and has a horizontal length of one pixel as point of interest 1504a, and the midpoint of an edge that includes the upper right end of the diagonal edge and has a horizontal length of one pixel as point of interest 1504b. In Figure 25(A), point of interest 1504a is represented by a circle with the letter A in the center, and point of interest 1504b is represented by a circle with the letter B in the center.

[0373] If the pixels constituting the partial image 1500a are squares with side length 1, and the lower left corner of the partial image 1500a is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the coordinates (x, y) of point of interest 1504a are (1.5, 1), and the coordinates (x, y) of point of interest 1504b are (4.5, 4).

[0374] The reconstruction line 1503a may be the line segment connecting point 1504a and point 1504b. In Figure 25(A), the reconstruction line 1503b is represented by a solid gray line.

[0375] As shown in the partial image 1500b of Figure 25(B), when the lower left end of an edge corresponds to a "horizontal" subpattern and the upper right end of an edge corresponds to a "horizontally elongated" subpattern, the lower left end pixel of the diagonal edge is pixel 1501c, and the upper right end pixel of the diagonal edge is pixel 1501d. In this case, the computer device 1 may designate the midpoint of the edge that includes the lower left end of the diagonal edge and has a length of 1 pixel horizontally as point of interest 1504c, and the midpoint of the edge that includes the upper right end of the diagonal edge and has a length of 2 pixels horizontally as point of interest 1504d. In Figure 25(B), point of interest 1504c is represented by a circle with the letter A in the center, and point of interest 1504d is represented by a circle with the letter B in the center.

[0376] If the pixels constituting the partial image 1500b are squares with side length 1, and the lower left corner of the partial image 1500b is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the coordinates (x, y) of point of interest 1504c are (1.5, 1), and the coordinates (x, y) of point of interest 1504d are (5, 4).

[0377] The reconstruction line 1503b may be the line segment connecting point 1504c and point 1504d. In Figure 25(B), the reconstruction line 1503b is represented by a solid gray line.

[0378] As shown in the partial image 1500c of Figure 25(C), when the lower left end of an edge corresponds to a "horizontal" subpattern and the upper right end of an edge corresponds to a "vertical" subpattern, the lower left end pixel of the diagonal edge is pixel 1501e, and the upper right end pixel of the diagonal edge is pixel 1501f. In this case, the computer device 1 may designate the midpoint of the edge that includes the lower left end of the diagonal edge and has a horizontal length of 1 pixel as point of interest 1504e, and the midpoint of the edge that includes the upper right end of the diagonal edge and has a vertical length of 1 pixel as point of interest 1504f. In Figure 25(C), point of interest 1504e is represented by a circle with the letter A in the center, and point of interest 1504f is represented by a circle with the letter C in the center.

[0379] In this case, a mathematical formula corresponding to the provisional reconstruction line may be specified in order to make the reconstruction line a curve or polyline connecting point 1504e and point 1504f.

[0380] The provisional reconstruction line may be a line segment connecting the point of interest and the provisional point of interest. The provisional point of interest may be used to identify the mathematical formula corresponding to the provisional reconstruction line. The position of the provisional point of interest is not particularly limited and can be designed as appropriate. For example, the provisional point of interest may be located at the end of an extended edge, assuming that the edge on which the point of interest is located is extended by one pixel in a step-like manner. "Extending the edge by one pixel in a step-like manner" may mean adding an edge of one pixel length from the end of the edge in a direction different from that edge. For example, if it is a horizontal edge, it may mean adding and extending a vertical edge from the end of the edge, and if it is a vertical edge, it may mean adding and extending a horizontal edge from the end of the edge. In this case, for example, the provisional point of interest may be located at the vertex of a pixel adjacent to the lower left end pixel of the edge in the diagonal downward direction to the left, and at the vertex of a pixel adjacent to the upper right end pixel of the edge in the diagonal upward direction to the right.

[0381] In partial image 1500c-1, assuming that the edge where point of interest 1504e is located is extended downward by one pixel in a step-like manner, the hypothetical point of interest 1506a is located at the end of the extended edge. The hypothetical point of interest 1506a is located at the bottom right vertex of pixel 1501g, which is adjacent to the lower left of the bottom left edge pixel 1501e.

[0382] Furthermore, in partial image 1500c-1, assuming that the edge where point of interest 1504f is located is extended to the right by one pixel in a step-like manner, the hypothetical point of interest 1506b is located at the end of the extended edge. The hypothetical point of interest 1506b is located at the lower right vertex of pixel 1501h, which is adjacent to the upper right and diagonally above the upper right edge pixel 1501f.

[0383] In Figure 25(C), the provisional point of interest 1506a is represented by a circle with the letter D in the center, and the provisional point of interest 1506b is represented by a circle with the letter E in the center.

[0384] Computer device 1 may define the line segment connecting point of interest 1504f and provisional point of interest 1506a as provisional reconstruction line 1505a, and the line segment connecting point of interest 1504e and provisional point of interest 1506b as provisional reconstruction line 1505b. The combination of point of interest and provisional point of interest that forms the end of the provisional reconstruction line should be determined such that the two line segments connecting the point of interest and provisional point of interest intersect.

[0385] Computer device 1 can designate the intersection of provisional reconstruction line 1505a and provisional reconstruction line 1505b as point of interest 1504g. In Figure 25(C), point of interest 1504g is represented by a circle with the letter B in the center. Also in Figure 25(C), provisional reconstruction lines 1505a and 1505b are represented by gray dashed lines.

[0386] Computer device 1 may define the reconstruction line 1503c as a line passing through the three points of interest 1504e, 1504f, and 1504g, as shown in partial image 1500c-2. The reconstruction line 1503c may be a curve or a polyline. In Figure 25(C), the reconstruction line 1503c is represented by a solid gray line.

[0387] If the pixels constituting the partial image 1500c are squares with side length 1, and the lower left corner of the partial image 1500c is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the coordinates (x, y) of point of interest 1504e are (1.5, 1), the coordinates (x, y) of point of interest 1504f are (5, 4.5), the coordinates (x, y) of provisional point of interest 1506a are (1, 0), and the coordinates (x, y) of provisional point of interest 1506b are (6, 5).

[0388] As shown in the partial image 1500d of Figure 25(D), when the lower left end of an edge corresponds to a "horizontal" subpattern and the upper right end of an edge corresponds to a "vertical" subpattern, the lower left end pixel of the diagonal edge is pixel 1501i, and the upper right end pixel of the diagonal edge is pixel 1501j. In this case, the computer device 1 may designate the midpoint of the edge that is the lower left end of the diagonal edge and has a horizontal length of 1 pixel as point of interest 1504h, and the midpoint of the edge that is the upper right end of the diagonal edge and has a vertical length of 2 pixels as point of interest 1504i. In Figure 25(D), point of interest 1504h is represented by a circle with the letter A in the center, and point of interest 1504i is represented by a circle with the letter C in the center.

[0389] In this case, a mathematical formula corresponding to the provisional reconstruction line may be specified in order to make the reconstruction line a curve or polyline connecting point 1504h and point 1504i.

[0390] In partial image 1500d-1, assuming that the edge where point of interest 1504h is located is extended downward by one pixel in a step-like manner, the hypothetical point of interest 1506c is located at the end of the extended edge. The hypothetical point of interest 1506c is located at the bottom right vertex of pixel 1501k, which is adjacent to the lower left and slightly below pixel 1501i at the bottom left end of the edge.

[0391] Furthermore, in partial image 1500d-1, assuming that the edge where point of interest 1504i is located is extended to the right by one pixel in a step-like manner, the hypothetical point of interest 1506d is located at the end of the extended edge. The hypothetical point of interest 1506d is located at the lower right vertex of pixel 1501l, which is adjacent to the upper right and diagonally above the upper right edge pixel 1501j.

[0392] In Figure 25(D), the provisional point of interest 1506c is represented by a circle with the letter D in the center, and the provisional point of interest 1506d is represented by a circle with the letter E in the center.

[0393] Computer device 1 may define the line segment connecting point of interest 1504i and provisional point of interest 1506c as a provisional reconstruction line 1505c, and the line segment connecting point of interest 1504h and provisional point of interest 1506d as a provisional reconstruction line 1505d.

[0394] Computer device 1 can designate the intersection of the provisional reconstruction line 1505c and provisional reconstruction line 1505d as point of interest 1504j. In Figure 25(D), point of interest 1504j is represented by a circle with the letter B in the center. Also in Figure 25(D), provisional reconstruction lines 1505c and 1505d are represented by gray dashed lines.

[0395] Computer device 1 may define the reconstruction line 1503d as a line passing through the three points of interest 1504h, 1504i, and 1504j, as shown in partial image 1500d-2. The reconstruction line 1503d may be a curve or a polyline. In Figure 25(D), the reconstruction line 1503d is represented by a solid gray line.

[0396] If the pixels constituting the partial image 1500d are squares with side length 1, and the lower left corner of the partial image 1500d is the origin, with the horizontal grid as the x-axis and the vertical grid as the y-axis, then the coordinates (x, y) of point of interest 1504h are (1.5, 1), the coordinates (x, y) of point of interest 1504i are (5, 5), the coordinates (x, y) of provisional point of interest 1506d are (1, 0), and the coordinates (x, y) of provisional point of interest 1506e are (6, 5).

[0397] As described above, in the case of diagonal line patterns, if the combination of subpatterns where the direction of the edges, including the edge ends, is the same, such as "horizontal" and "horizontal," or "horizontal" and "horizontally elongated," a straight reconstruction line may be identified. Also, if the combination of subpatterns where the direction of the edges, including the edge ends, is different, such as "horizontal" and "vertical," or "horizontal" and "vertically elongated," a curved or polyline reconstruction line may be identified. In this case, the computer device 1 may identify a mathematical formula corresponding to the reconstruction line passing through the intersection of the two provisional reconstruction lines.

[0398] For other subpatterns corresponding to the diagonal line pattern, the above description can be adopted to the extent necessary for identifying the positions of the points of interest and the provisional points of interest, as well as for identifying the formulas corresponding to the reconstruction lines and provisional reconstruction lines.

[0399] Computer device 1 can calculate interpolation weights based on the area of ​​the region enclosed by the reconstruction lines and edges. The interpolation weights may be equal to the ratio of the area of ​​the region enclosed by the reconstruction lines and edges to the area of ​​the pixel, or they may be a ratio obtained by multiplying the ratio of the area of ​​the region enclosed by the reconstruction lines and edges to the area of ​​the pixel by a predetermined value. The predetermined value may include, for example, numerical values ​​of the smoothness parameter and / or extra smoothness parameter. In other words, even in diagonal line patterns, different numerical values ​​may be calculated for the interpolation weight of each pixel depending on the information about the criteria for calculating the interpolation weights entered by the user.

[0400] Next, we will explain how to calculate interpolation weights for edges that fall under the higher-level pattern of "mixed straight and diagonal type". In a 3x3 pattern, if an edge is identified as "mixed straight and diagonal type", the computer device 1 may perform interpolation for all of the target edges present on the edge of the target pixel, or it may choose not to perform interpolation for one or more of the target edges present on the edge of the target pixel.

[0401] For example, if the computer device 1 has both a linear edge and a diagonal edge on the edge of the pixel of interest, it may identify and interpolate only the linear edge, or it may identify and interpolate only the diagonal edge, or it may identify and interpolate both the linear and diagonal edges of interest.

[0402] When interpolation is performed on both linear and diagonal edges of interest, the interpolation weights may be calculated in a different way than when interpolation is performed on only one of the edges of interest.

[0403] For example, when the computer device 1 performs interpolation on both edges of interest, it may calculate the interpolation weights so that the values ​​of the interpolation weights calculated for each edge of interest are smaller than the values ​​when interpolation is performed on only one of the edges of interest. Specifically, for example, when interpolation is performed on both edges of interest, the interpolation value calculated for the linear edge of interest may be the value obtained by dividing the interpolation value calculated only for the linear edge of interest by a predetermined number, and the interpolation value calculated for the diagonal edge of interest may be the value obtained by dividing the interpolation value calculated only for the diagonal edge of interest by a predetermined number.

[0404] When interpolating both target edges, calculating the interpolation weights for each target edge so that they are smaller than when interpolating only one of the target edges prevents the target edge's color from blending excessively with other colors when blending the pixel's color using the sum of the multiple interpolation weights calculated for the target pixel.

[0405] There are no particular limitations on which edge pixels and edge arrangements are used for interpolation, which edges of interest are interpolated, and how the interpolation weights are calculated when interpolation is performed; these can be designed as appropriate.

[0406] Next, we will explain how to calculate interpolation weights for edges that fall under the higher-level pattern of "hard angular." In a 3x3 pattern, if an edge is identified as "hard angular," the computer device 1 may then search for the ends of vertical and horizontal edges that are continuous with the edge of interest. For the method of searching for the ends of edges, the description of the method for searching for the ends of edges that fall under the "linear" pattern can be adopted to the extent necessary. Vertical and horizontal edges that are continuous with the edge of interest correspond to edges that form corners.

[0407] Computer device 1 may search for the edges and determine the lengths of the longitudinal and transverse edges that are continuous with the edge of interest. Computer device 1 may also determine subpatterns of the edges based on the lengths of the identified edges. A longer edge length indicates a larger angle formed by the edge. Therefore, computer device 1 may calculate weights in different ways depending on the size of the angle.

[0408] The types of edge subpatterns are not particularly limited and can be designed as appropriate. Here, we will explain using three types of subpatterns as examples: a "2-pixel length pattern" where the edge length is 2 pixels, a "3-pixel length pattern" where the edge length is 3 pixels, and a "4-pixel or longer pattern" where the edge length is 4 pixels or more. Note that the edge length referenced when identifying a subpattern may be the shorter of the vertical edge length and the horizontal edge length, the longer of the vertical edge length and the horizontal edge length, or the average of the vertical edge length and the horizontal edge length.

[0409] Figure 26 is a diagram illustrating a smoothing process according to an embodiment of the present invention. A portion of the image shown in Figure 26 (hereinafter also referred to as the partial image) 1600 contains black pixels and light gray pixels. Between the black pixels and the light gray pixels, there are edges 1602 (1602a and 1602b) that correspond to "hard rectangular shapes". In the partial image 1600, the edges 1602 are represented by light gray dashed lines. Note that in Figure 26, pixels marked with the symbol "*" can be of any color.

[0410] Computer device 1 may calculate interpolation weights for edges corresponding to "hard corners" for the pixel located at the tip of the corner and for the three pixels adjacent to that pixel across the edge. In partial image 1600, the pixel located at the tip of the corner may be pixel 1601a. ​​Also, in partial image 1600, the three pixels adjacent to pixel 1601a across the edge may be pixels 1601b to 1601d.

[0411] If edge 1602 is identified as corresponding to a 2-pixel length pattern, then, for example, as shown on the lower left of Figure 26, the interpolation weights for pixel 1601a, pixel 1601b, and pixel 1601c may be calculated as follows: "0.5", "0.2", "0.2", and "0.1".

[0412] Furthermore, if edge 1602 is identified as corresponding to a 3-pixel length pattern, then, for example, as shown in the center of the lower row of Figure 26, the interpolation weights for pixel 1601a, pixel 1601b, and pixel 1601c may be calculated as follows: "0.875", "0.05", "0.05", and "0.025".

[0413] Furthermore, if edge 1602 is identified as a pattern with a length of 4 pixels or more, the interpolation weights for pixel 1601a, pixel 1601b, pixel 1601c, and pixel 1601d may be calculated as follows, for example, as shown on the right side of the lower panel of Figure 26: "1", "0", "0", "0", and "0".

[0414] The interpolation weights calculated here may be the proportion to which the color of the pixel located at the tip of the corner is blended. In other words, for pixels 1601b to 1601d, the pixel to be blended may be specifically identified as pixel 1601a.

[0415] For example, if a pattern is identified as having a 2-pixel length, the blended color of pixel 1601c may be identified as dark gray, which is the original color of pixel 1601c (light gray) mixed with 20% black, which is the color of pixel 1601a. ​​Alternatively, if a pattern is identified as having a 4-pixel length or longer, the blended color of pixel 1601c may be identified as light gray (the original color of pixel 1601c), which is the original color of pixel 1601c (light gray) mixed with 0% black, which is the color of pixel 1601a.

[0416] Furthermore, for pixels located at the tip of a corner, the original color of the pixel may be blended with the color of the adjacent pixel across the edge. In other words, for pixel 1601a, the pixels to be blended may be specified as pixels 1601b or 1601c.

[0417] For example, if a pattern is identified as having a 2-pixel length, the blended color of pixel 1601a may be identified as dark gray, obtained by mixing 50% of the original color of pixel 1601a (black) with 50% of the light gray color of pixel 1601b or 1601c. Alternatively, if a pattern is identified as having a 4-pixel length or longer, the blended color of pixel 1601a may be identified as black (the original color of pixel 1601a), obtained by mixing 100% of the original color of pixel 1601a (black) with 0% of the light gray color of pixel 1601b or 1601c.

[0418] In other words, if a pattern is identified as being 4 pixels or longer, interpolation does not need to be performed on edges that correspond to a "hard, angular" shape. The length of the edges forming a corner is considered to indicate a region that has been intentionally given an angular appearance. Therefore, if the length of the edges forming a corner is greater than a predetermined length, the color of the pixels constituting the corner does not need to be interpolated.

[0419] Furthermore, the computer device 1 may calculate interpolation weights in different ways depending on the arrangement of edges and / or pixels located inside the corners formed by edges corresponding to "hard corners".

[0420] For example, if there is an edge forming another smaller corner inside a corner formed by an edge corresponding to a "hard corner," the computer device 1 may calculate the interpolation weights in a different way than when there is no edge forming another smaller corner inside the corner. Cases where there is an edge forming another smaller corner inside a corner include, for example, the case in partial image 1600 where pixel 1601a and the adjacent pixel 1601e to the upper left are pixels other than black.

[0421] In this case, for example, the computer device 1 may calculate an interpolation weight for edge 1602 by dividing the value of the interpolation weight calculated when there is no edge that forms another small corner inside the corner by a predetermined number.

[0422] The method for calculating the interpolation weight for each pixel is not particularly limited and can be designed as appropriate. The interpolation weight for each pixel may be calculated, for example, according to a predetermined formula that includes a smoothness parameter and / or an extra smoothness parameter. In other words, for edges corresponding to "hard polygons," different numerical values ​​may be calculated for the interpolation weight of each pixel depending on the information about the criteria for calculating the interpolation weight that is entered by the user. Alternatively, for edges corresponding to "hard polygons," a predetermined numerical value may be calculated for the interpolation weight of each pixel, regardless of user input.

[0423] Next, we will explain how to calculate interpolation weights for edges that correspond to the higher-level patterns of the "single-pixel type". In a 3x3 pattern, if an edge is identified as a "single-pixel type", interpolation weights may be calculated for the pixels enclosed by the edge and the eight pixels adjacent to those pixels.

[0424] Figure 27 is a diagram illustrating a smoothing process according to an embodiment of the present invention. Figure 27(A) is a diagram illustrating the arrangement of pixels corresponding to the "single-pixel type," and Figure 27(B) is a diagram illustrating a method for calculating interpolation weights in the "single-pixel type."

[0425] In the partial image 1700 shown in Figure 27(A), the pixels marked with the symbol "≠" can be any color pixel, as long as they are a different color from the central pixel 1701a in the 3x3 pattern. An edge exists between the central pixel 1701a and the adjacent pixels 1701b to 1701i. Therefore, although not shown in the diagram, the area around pixel 1701a is surrounded by this edge.

[0426] In this case, for example, as shown in Figure 27(B), the interpolation weight for the central pixel 1701a may be calculated as "0.5", the interpolation weights for pixels 1701b to 1701e adjacent to pixel 1701a in all directions (up, down, left, and right) may be calculated as "0.1", and the interpolation weights for pixels 1701f to 1701i diagonally adjacent to pixel 1701a may be calculated as "0.025".

[0427] The weight calculated here may be the ratio to which the colors of pixels surrounded by edges are blended. In other words, for pixels 1701b to 1701i, the pixel to be blended may be specifically identified as pixel 1701a.

[0428] For example, the blended color of pixel 1701b may be defined as the original color of pixel 1701b mixed with the color of pixel 1701a at a ratio of 10%.

[0429] Furthermore, for pixels surrounded by an edge, the original color of the pixel may be blended with the color of the adjacent pixel across the edge. In other words, for pixel 1701a, the pixel to be blended may be any of pixels 1701b to 1701i.

[0430] For example, the blended color of pixel 1701a may be defined as a color obtained by mixing 50% of the original color of pixel 1701a with 50% of any of the colors of pixels 1701b to 1701i.

[0431] The method for calculating the interpolation weight for each pixel is not particularly limited and can be designed as appropriate. The interpolation weight for each pixel may be calculated, for example, according to a predetermined formula that includes a smoothness parameter and / or an extra smoothness parameter. In other words, for edges corresponding to the "single-pixel type," different numerical values ​​may be calculated for the interpolation weight of each pixel depending on the information about the criteria for calculating the interpolation weight that is entered by the user. Alternatively, for edges corresponding to the "single-pixel type," a predetermined numerical value may be calculated for the interpolation weight of each pixel, regardless of user input.

[0432] For pixel groups or edges corresponding to the higher-level patterns of the "no interpolation" type, interpolation weights do not need to be calculated. Alternatively, similar to the "hard rectangular" patterns of 4 pixels or longer, the interpolation weight for the pixel of interest may be calculated as "1," and the interpolation weights for pixels adjacent to the pixel of interest may be calculated as "0." In this case, the color of the pixel of interest may be specified as the color to be blended. In this case, the color of the pixel of interest will not be blended with other pixels.

[0433] In the above, for other subpatterns identified as "not interpolated," the description of the "no interpolation type" can be adopted to the extent necessary.

[0434] In the above explanation, we used partial images containing edges extending in a specific direction as examples of edges corresponding to each pattern. However, edges extending in other directions included in images obtained by rotating (90° rotation, 180° rotation, 270° rotation, etc.) or flipping (vertical flip, horizontal flip, etc.) the above partial images can also be processed by appropriately applying the above descriptions. In this case, the descriptions regarding direction should be appropriately interpreted and applied as needed.

[0435] Returning to the explanation of Figure 5, the computer device 1 identifies the blended color for each pixel based on the interpolation weights and the colors to be blended, which were stored in step S304 (step S305), and stores the identified colors in the storage unit 13 (step S306). Steps S301 to S306 complete the smoothing process.

[0436] Steps S301 to S304 store a list of colors to blend and interpolation weights for each pixel in the input image. If a pixel has one edge, the number of interpolation weights stored for that pixel may be one or less. If a pixel has two or more edges, the number of interpolation weights stored for that pixel may be two or more.

[0437] In step S305, the computer device 1 may identify the color obtained by mixing the color to be blended with the original color of the pixel in proportion to the interpolation weight as the blended color (hereinafter also referred to as the color after blending). If two or more interpolation weights are stored for a single pixel, and the blended color corresponding to each interpolation weight is the same, the interpolation weight for that color may be the sum of the two or more interpolation weights, or it may be the average of the two or more interpolation weights.

[0438] The method for mixing two or more colors is not particularly limited and can be designed as appropriate. For example, linear interpolation may be performed for each RGBA color channel. That is, the values ​​of each RGBA color channel of the mixed color may be determined by multiplying the value of each color channel of the color to be mixed by a value that represents the proportion corresponding to the interpolation weight for each color, and then summing these values ​​for each color channel.

[0439] Figure 28 is a diagram illustrating a smoothing process according to an embodiment of the present invention. A portion of the image shown in Figure 28 (hereinafter also referred to as a partial image) 1800 contains black pixels and white pixels, and edges exist between the black pixels and white pixels. When the lower left corner of partial image 1800 is taken as the origin, the edges on the left and top edges of the black pixel 1801 at pixel coordinates (x, y) = (4, 4) correspond to a diagonal line pattern, and the edges on the top and right edges of pixel 1801 correspond to an L-shaped pattern. In Figure 28, a gray circle is placed in the center of pixel 1801.

[0440] As shown in the diagram, for pixel 1801, the interpolation weight determined based on the area enclosed by the reconstructed line 1802a, which corresponds to a diagonal line pattern, and the stepped edge is set to "0.12". Similarly, for pixel 1801, the interpolation weight determined based on the area enclosed by the reconstructed line 1802b, which corresponds to an L-shaped pattern, and the L-shaped edge is set to "0.4". The blending color corresponding to these interpolation weights is white in both cases.

[0441] When determining the blended color of pixel 1801, if we assume that the interpolation weight for white is the sum of the interpolation weights calculated for each pattern, then the interpolation weight for white becomes "0.52". Also, when determining the blended color of pixel 1801, the interpolation weight for black, which is the original color of pixel 1801, becomes "0.48", which is "1" minus "0.52". Therefore, the values ​​of each color channel of pixel 1801 after blending may be the sum of the value representing white multiplied by "0.52" and the value representing black multiplied by "0.48".

[0442] In this case, the computer device 1 may identify the blended color in a linear color space. For example, the blended color may be identified using linear RGBA values.

[0443] In step S306, the computer device 1 may store the blended color identified in step S305 in the storage unit 13, associating it with information that can identify a pixel (e.g., pixel coordinates, pixel number, etc.).

[0444] Returning to the explanation of Figure 2, the computer device 1 then outputs the image that has been smoothed in step S104 (step S105). Steps S101 to S105 complete the anti-aliasing process.

[0445] The image output in step S105 is an image in which the color of the pixels subjected to interpolation has been changed to the blended color identified in step S305. Computer device 1 may output the image after changing the color of the pixels contained in the image to the blended color stored in association with information that can identify each pixel.

[0446] If an image conversion is performed in step S102, the reverse-converted image may be output in step S105. For example, the computer device 1 may perform a pre-multiply process on the smoothed image, convert the color space from linear color space to sRGB, convert the pixel format, and then output the final image.

[0447] According to the program of the present invention, it is possible to determine whether or not to interpolate a pixel, and if so, how to calculate the interpolation weights, based on various edge information, including not only the shape of the edges in the vicinity of the pixel to be interpolated, but also the shape of the edges in the extended portion of those edges. Therefore, it is possible to display a digitized image in a manner close to the expression that the artist originally intended to express, assuming the image had not been digitized.

[0448] The program of the present invention can be used for various images displayed on a display, but it is particularly suitable for use with images for animation production. In other words, the program of the present invention can be used as a program for animation production.

[0449] Furthermore, any part of the above-mentioned functions performed by the program of the present invention can be extracted and executed independently. In other words, any part of the above invention can be considered an independent invention.

[0450] For example, an invention that calculates interpolation weights in different ways depending on the length of the portion of an edge perpendicular to a given direction within an L-shaped, Z-shaped, and / or U-shaped edge may be an independent invention.

[0451] Furthermore, for example, if an edge is identified as a pattern forming a U-shape, the invention of identifying a mathematical formula for a curve corresponding to a reconstruction line may be an independent invention.

[0452] Furthermore, for example, an invention that accepts input for changing information to identify the slope of a reconstruction line may be an independent invention.

[0453] Furthermore, for example, an invention that converts the color space of the portion of the color excluding pixels of a predetermined color into a linear color space, and then identifies the blended color in the linear color space during interpolation, may be an independent invention.

[0454] Thus, a program for performing anti-aliasing on an image can provide a novel program for anti-aliasing by having a computer device function as: edge detection means for detecting edge information relating to the edges of regions composed of pixels of the same or similar color included in the image; pattern identification means for identifying edge patterns based on the detected edge information; weight calculation means for calculating interpolation weights for each pixel based on the identified patterns; blend color identification means for identifying colors to be blended in interpolation based on the identified patterns; and blend means for identifying blended colors for each pixel based on the calculated interpolation weights and the identified blended colors. The pattern identification means identifies patterns in which edges form a T-shape, patterns in which edges form a cross-shape, patterns in which edges extend diagonally, patterns in which edges form angles of a predetermined size or larger, and / or patterns in which edges surround a single pixel. Furthermore, it becomes possible to interpolate the color of pixels adjacent to edges that fall under patterns where the edges form a T-shape, a cross-shaped shape, an obliquely extending edge, an edge forming a corner of a predetermined size or larger, and / or an edge surrounding a single pixel.

[0455] Furthermore, in this manner, the pattern identification means can identify the upper-level pattern of an edge based on edge information in a pixel group of a predetermined size, and then identify the lower-level pattern of an edge based on edge information relating to an edge formed by pixels different from the pixel group of the predetermined size that is continuous with the edge included in the pixel group of the predetermined size, thereby enabling the identification of edge patterns in a stepwise manner.

[0456] Furthermore, by calculating interpolation weights in different ways depending on the shape and / or direction of edges located near the ends of T-shaped and / or cross-shaped edges that connect to T-shaped and / or cross-shaped edges, or depending on the length of edges in the portion of T-shaped and / or cross-shaped edges perpendicular to the edges in a predetermined direction, the weight calculation means makes it possible to interpolate the color of pixels tangent to T-shaped and / or cross-shaped edges according to the shape of edges near the ends of T-shaped and / or cross-shaped edges.

[0457] Furthermore, by calculating interpolation weights such that the weight calculation means interpolates the color of pixels in one or fewer of the three regions touching the T-shaped edge, it becomes possible to interpolate the color of pixels in one or fewer of the three regions touching the T-shaped edge.

[0458] Furthermore, by having the weight calculation means calculate interpolation weights in different ways depending on the color combination of the four regions touching the cross-shaped edge, it becomes possible to interpolate pixels touching the cross-shaped edge in different ways depending on the color combination of the four regions touching the cross-shaped edge.

[0459] Furthermore, by having the weight calculation means calculate the interpolation weights in different ways depending on the number of pixels forming the diagonal edge, it becomes possible to interpolate pixels tangent to the diagonal edge in different ways depending on the number of pixels forming the diagonal edge.

[0460] Furthermore, by having the weight calculation means calculate interpolation weights in different ways depending on the length and / or direction of the end of the diagonal edge, it becomes possible to interpolate pixels tangent to the diagonal edge in different ways depending on the length and / or direction of the end of the diagonal edge.

[0461] Furthermore, the computer device is made to function as a formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape, and the weight calculation means calculates interpolation weights based on the formula identified by the formula identification means. When the pattern identification means identifies that the edge is a pattern that extends diagonally, the formula identification means identifies a formula corresponding to a reconstruction line passing through the intersection of two provisional reconstruction lines, making it possible to interpolate pixels tangent to the diagonally extending edge using the reconstruction line passing through the intersection of the two provisional reconstruction lines.

[0462] Furthermore, the pattern identification means further identifies patterns in which edges form an L-shape, a Z-shape, and / or a U-shape. When the pattern identification means identifies a pattern in which edges form an L-shape, a Z-shape, and / or a U-shape, the weight calculation means calculates interpolation weights in different ways according to the length of the portion of the edge perpendicular to the edge in a predetermined direction within the L-shape, Z-shape, and / or U-shape edge. This makes it possible to interpolate pixels tangent to the L-shape, Z-shape, and / or U-shape edge in different ways according to the length of the portion of the edge perpendicular to the edge in a predetermined direction within the L-shape, Z-shape, and / or U-shape edge.

[0463] Furthermore, if the pattern identification means identifies that the edge forms a U-shaped pattern, the formula identification means can identify the formula of the curve, making it possible to interpolate the pixels tangent to the U-shaped edge using the curve reconstruction line.

[0464] Furthermore, when the pattern identification means identifies an edge as a pattern forming a corner of a predetermined size or larger, the weight calculation means calculates interpolation weights for the pixel located at the tip of the corner and the three adjacent pixels on either side of the edge in different ways according to the size of the corner. This makes it possible to interpolate the pixel located at the tip of the corner and the three adjacent pixels on either side of the edge in different ways according to the size of the corner.

[0465] Furthermore, when the pattern identification means identifies an edge as a pattern surrounding a single pixel, the weight calculation means calculates interpolation weights for that pixel and the eight pixels adjacent to it, thereby enabling interpolation for the pixel surrounded by the edge and the eight pixels adjacent to it.

[0466] Furthermore, by having the blend color identification means identify the same color for all pixels that intersect with reconstruction line 1, it becomes possible to interpolate all the colors of the pixels that intersect with reconstruction line 1 so as to blend them with color 1.

[0467] Furthermore, by having the computer device function as a weight change receiving means that accepts input to change information regarding the criteria for calculating interpolation weights, it becomes possible to change the information regarding the criteria for calculating interpolation weights according to user input.

[0468] Furthermore, by having the weight change receiving means accept input to change the information used to identify the slope of the reconstruction line, it becomes possible to change the information used to identify the slope of the reconstruction line according to the user's input.

[0469] Furthermore, the computer device can be made to function as a color space conversion means for converting the color space of an image. The color space conversion means can convert the color space of the portion of the image excluding pixels of a predetermined color into a linear color space, and the blending means can identify the blended color in the linear color space, thereby interpolating the color of the pixels using the blended color in the linear color space. [Explanation of Symbols]

[0470] 1. Computer device 11 Control Unit 12 RAM 13 Storage Section 14 Input section 15 Display 16 Communication Interfaces 100 images 101 pixels 200 3x3 patterns 201 Featured Pixels 202 Edge 203 2x2 pattern 210 3x3 patterns 211 Featured Pixels 212 Edge 213 2x3 patterns 220 3x3 patterns 221 Notable Pixels 222 Edge 230 3x3 patterns 231 Featured Pixels 232 Edge 240 3x3 patterns 241 Notable Pixels 242 Edge 250 3x3 patterns 300 partial images 301 pixels 302 Edge 303 Reconstruction Line 304 Points of Interest 400 partial images 401 pixels 402 Edge 403 Reconstruction Line 404 Points of Interest 500 partial images 501 pixels 502 Edge 503 Reconstruction Line 504 Points of Interest 600 partial images 601 pixels 602 Edge 603 Reconstruction Line 604 Points of Interest 700 partial images 701 pixels 702 Edge 703 Reconstruction Line 704 Points of Interest 800 partial images 801 pixels 802 Edge 803 Reconstruction Line 804 Points of Interest 900 partial images 901 pixels 902 Edge 903 Reconstruction Line 904 Points of Interest 1000 partial images 1001 pixels 1002 Edge 1003 Reconstruction Line 1004 Points of Interest 1100 Partial Images 1101 pixels 1102 Edge 1103 Reconstruction Line 1104 Points of Interest 1200 partial images 1201 pixels 1202 Edge 1300 3x4 pattern 1301 Bottom left pixel 1302 Edge 1310 3x4 pattern 1311 Bottom left pixel 1312 Edge 1320 3x4 pattern 1321 Bottom left pixel 1322 Edge 1330 3x4 pattern 1331 Bottom left pixel 1332 Edge 1340 3x4 pattern 1341 Bottom left pixel 1342 Edge 1350 3x4 pattern 1351 Bottom left pixel 1352 Edge 1360 3x4 pattern 1361 Bottom left pixel 1362 Edge 1400 partial images 1401 pixels 1402 Edge 1403 Reconstruction Line 1404 Points of Interest 1500 partial images 1501 pixels 1502 Edge 1503 Reconstruction Line 1504 Points of Interest 1505 Provisional Reconstruction Line 1506 Provisional points of interest 1600 partial images 1601 pixels 1602 Edge 1700 partial images 1701 pixels 1800 partial image 1801 pixels 1802 Reconstruction Line

Claims

1. A program for performing anti-aliasing on an image, Computer equipment, An edge detection means for detecting edge information relating to the edges of regions in an image that are composed of pixels of the same or similar color, A pattern identification means that identifies the edge pattern based on the detected edge information, A weight calculation means that calculates the interpolation weight for each pixel based on the identified pattern, A blend color identification means that identifies the colors to be blended in interpolation based on an identified pattern, A blending means that identifies the blended color for each pixel based on the calculated interpolation weights and the identified blended color. To make it function as, The pattern identification means identifies patterns in which edges form a T-shape, patterns in which edges form a cross-shape, patterns in which edges extend diagonally, patterns in which edges form angles of a predetermined size or larger, and / or patterns in which edges surround a single pixel. program.

2. The pattern identification means identifies the upper-level pattern of an edge based on edge information in a pixel group of a predetermined size, and then identifies the lower-level pattern of an edge based on edge information relating to an edge formed by pixels different from the pixel group of the predetermined size, which is continuous with the edge included in the pixel group of the predetermined size. The program according to claim 1.

3. When the pattern identification means identifies a pattern in which the edges form a T-shape and / or a pattern in which the edges form a cross-shape, the weight calculation means calculates interpolation weights in different ways depending on the shape and / or direction of the edges located near the ends of the T-shaped and / or cross-shaped edges that connect to the T-shaped and / or cross-shaped edges, or depending on the length of the portion of the T-shaped and / or cross-shaped edges perpendicular to the edges in a predetermined direction. The program according to claim 1.

4. When the pattern identification means identifies that the edge forms a T-shaped pattern, the weight calculation means calculates interpolation weights so as to interpolate the color of pixels in one or fewer of the three regions tangent to the T-shaped edge. The program according to claim 1.

5. If the pattern identification means identifies that the edge forms a cross shape, the weight calculation means calculates interpolation weights in different ways depending on the color combination of the four regions adjacent to the cross-shaped edge. The program according to claim 1.

6. If the pattern identification means identifies that the edge is a pattern that extends diagonally, the weight calculation means calculates interpolation weights in different ways depending on the number of pixels that form the diagonal edge. The program according to claim 1.

7. If the pattern identification means identifies that the edge is a pattern that extends diagonally, the weight calculation means calculates interpolation weights in different ways depending on the length and / or direction of the end of the diagonal edge. The program according to claim 1.

8. The computer device, further, Formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape. To make it function as, The weight calculation means calculates the interpolation weights based on the mathematical formula identified by the mathematical formula identification means. If the pattern identification means identifies that the edge is a pattern that extends diagonally, the formula identification means identifies a formula corresponding to the reconstruction line that passes through the intersection of the two provisional reconstruction lines. The program according to claim 1.

9. The pattern identification means further identifies patterns in which the edges form an L-shape, patterns in which the edges form a Z-shape, and / or patterns in which the edges form a U-shape. When the pattern identification means identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, the weight calculation means calculates interpolation weights in different ways according to the length of the portion of the edge perpendicular to the edge in a predetermined direction among the L-shaped, Z-shaped, and / or U-shaped edges. The program according to claim 1.

10. The computer device, further, Formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape. To make it function as, The weight calculation means calculates the interpolation weights based on the mathematical formula identified by the mathematical formula identification means. The pattern identification means further identifies patterns in which the edges form a U-shaped form, If the pattern identification means identifies that the edge forms a U-shaped pattern, the formula identification means identifies the formula of the curve. The program according to claim 1.

11. When the pattern identification means identifies an edge as a pattern that forms a corner of a predetermined size or larger, the weight calculation means calculates interpolation weights for the pixel located at the tip of the corner and the three pixels adjacent to that pixel across the edge, using different methods according to the size of the corner. The program according to claim 1.

12. When the pattern identification means identifies that an edge is a pattern surrounding a single pixel, the weight calculation means calculates interpolation weights for that pixel and eight adjacent pixels. The program according to claim 1.

13. The computer device, further, Formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape. To make it function as, The weight calculation means calculates the interpolation weights based on the mathematical formula identified by the mathematical formula identification means. When the pattern identification means identifies a pattern in which the edges form a T-shape and / or a pattern in which the edges form a cross-shape, the blend color identification means identifies the same color for all pixels that intersect with one reconstruction line. The program according to claim 1.

14. The pattern identification means further identifies patterns in which the edges form an L-shape, patterns in which the edges form a Z-shape, and / or patterns in which the edges form a U-shape. When the pattern identification means identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, the blend color identification means identifies the same color for all pixels that intersect with one reconstruction line. The program according to claim 1.

15. The computer device, further, A weight change receiving mechanism that accepts input to change information regarding the criteria for calculating interpolation weights. To make it function as The program according to any one of claims 1 to 14.

16. The computer device, further, Formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape. To make it function as, The weight calculation means calculates the interpolation weights based on the mathematical formula identified by the mathematical formula identification means. The weight change receiving mechanism accepts input to change information for identifying the slope of the reconstruction line. The program according to claim 15.

17. The computer device, further, Color space conversion means for converting the color space of the aforementioned image To make it function as, The color space conversion means can convert the color space of the portion of the color excluding pixels of a predetermined color into a linear color space. The blending means identifies the blended color in an image obtained by converting the color space to a linear color space using the color space conversion means. The program according to any one of claims 1 to 14.

18. A computer device for performing anti-aliasing on an image, An edge detection means for detecting edge information relating to the edges of regions in an image that are composed of pixels of the same or similar color, A pattern identification means that identifies the edge pattern based on the detected edge information, A weight calculation means that calculates the interpolation weight for each pixel based on the identified pattern, A blend color identification means that identifies the colors to be blended in interpolation based on an identified pattern, A blending means that identifies the blended color for each pixel based on the calculated interpolation weights and the identified blended color. Equipped with, The pattern identification means identifies patterns in which edges form a T-shape, patterns in which edges form a cross-shape, patterns in which edges extend diagonally, patterns in which edges form angles of a predetermined size or larger, and / or patterns in which edges surround a single pixel. Computer device.

19. A method for performing anti-aliasing on an image, which is performed on at least one computer device, An edge detection step for detecting edge information relating to the edges of regions in an image that consist of pixels of the same or similar color, A pattern identification step that identifies the edge pattern based on the detected edge information, A weight calculation step that calculates the interpolation weights for each pixel based on the identified pattern, A blend color identification step that identifies the colors to be blended in interpolation based on the identified pattern, A blending step that identifies the blended color for each pixel, based on the calculated interpolation weights and the identified colors to blend. It has, The pattern identification step identifies patterns in which edges form a T-shape, patterns in which edges form a cross-shape, patterns in which edges extend diagonally, patterns in which edges form angles of a predetermined size or larger, and / or patterns in which edges surround a single pixel. method.

20. A program for performing anti-aliasing on an image, Computer equipment, An edge detection means for detecting edge information relating to the edges of regions in an image that are composed of pixels of the same or similar color, A pattern identification means that identifies the edge pattern based on the detected edge information, A weight calculation means that calculates the interpolation weight for each pixel based on the identified pattern, A blend color identification means that identifies the colors to be blended in interpolation based on an identified pattern, A blending means that identifies the blended color for each pixel based on the calculated interpolation weights and the identified blended color. To make it function as, The pattern identification means identifies patterns in which the edges form an L-shape, patterns in which the edges form a Z-shape, and / or patterns in which the edges form a U-shape. When the pattern identification means identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, the weight calculation means calculates interpolation weights in different ways according to the length of the portion of the edge perpendicular to the edge in a predetermined direction among the L-shaped, Z-shaped, and / or U-shaped edges. program.

21. A computer device for performing anti-aliasing on an image, An edge detection means for detecting edge information relating to the edges of regions in an image that are composed of pixels of the same or similar color, A pattern identification means that identifies the edge pattern based on the detected edge information, A weight calculation means that calculates the interpolation weight for each pixel based on the identified pattern, A blend color identification means that identifies the colors to be blended in interpolation based on an identified pattern, A blending means that identifies the blended color for each pixel based on the calculated interpolation weights and the identified blended color. Equipped with, The pattern identification means identifies patterns in which the edges form an L-shape, patterns in which the edges form a Z-shape, and / or patterns in which the edges form a U-shape. When the pattern identification means identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, the weight calculation means calculates interpolation weights in different ways according to the length of the portion of the edge perpendicular to the edge in a predetermined direction among the L-shaped, Z-shaped, and / or U-shaped edges. Computer device.

22. A method for performing anti-aliasing on an image, which is performed on at least one computer device, An edge detection step for detecting edge information relating to the edges of regions in an image that consist of pixels of the same or similar color, A pattern identification step that identifies the edge pattern based on the detected edge information, A weight calculation step that calculates the interpolation weights for each pixel based on the identified pattern, A blend color identification step that identifies the colors to be blended in interpolation based on the identified pattern, A blending step that identifies the blended color for each pixel, based on the calculated interpolation weights and the identified colors to blend. It has, The pattern identification step identifies patterns in which the edges form an L-shape, patterns in which the edges form a Z-shape, and / or patterns in which the edges form a U-shape. If the pattern identification step identifies a pattern in which the edges form an L-shape, a pattern in which the edges form a Z-shape, and / or a pattern in which the edges form a U-shape, the weight calculation step calculates interpolation weights in different ways according to the length of the portion of the edge perpendicular to the edge in a given direction within the L-shape, Z-shape, and / or U-shape. method.

23. A program for performing anti-aliasing on an image, Computer equipment, An edge detection means for detecting edge information relating to the edges of regions in an image that are composed of pixels of the same or similar color, A pattern identification means that identifies the edge pattern based on the detected edge information, A formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape, A weight calculation means that calculates the interpolation weight for each pixel based on a specified pattern and a specified mathematical formula, A blend color identification means that identifies the colors to be blended in interpolation based on an identified pattern, A blending means that identifies the blended color for each pixel based on the calculated interpolation weights and the identified blended color. To make it function as, The pattern identification means identifies a pattern in which the edges form a U-shaped shape. If the pattern identification means identifies that the edge forms a U-shaped pattern, the formula identification means identifies the formula of the curve. program.

24. A computer device for performing anti-aliasing on an image, An edge detection means for detecting edge information relating to the edges of regions in an image that are composed of pixels of the same or similar color, A pattern identification means that identifies the edge pattern based on the detected edge information, A formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape, A weight calculation means that calculates the interpolation weight for each pixel based on a specified pattern and a specified mathematical formula, A blend color identification means that identifies the colors to be blended in interpolation based on an identified pattern, A blending means that identifies the blended color for each pixel based on the calculated interpolation weights and the identified blended color. Equipped with, The pattern identification means identifies a pattern in which the edges form a U-shaped shape. If the pattern identification means identifies that the edge forms a U-shaped pattern, the formula identification means identifies the formula of the curve. Computer device.

25. A method for performing anti-aliasing on an image, which is performed on at least one computer device, An edge detection step for detecting edge information relating to the edges of regions in an image that consist of pixels of the same or similar color, A pattern identification step that identifies the edge pattern based on the detected edge information, A formula identification step that identifies a formula corresponding to a reconstruction line identified based on the edge shape, A weight calculation step that calculates the interpolation weight for each pixel based on the identified pattern and identified mathematical formula, A blend color identification step that identifies the colors to be blended in interpolation based on the identified pattern, A blending step that identifies the blended color for each pixel, based on the calculated interpolation weights and the identified colors to blend. It has, The pattern identification step identifies a pattern in which the edges form a U-shape. If the pattern identification step identifies that the edge forms a U-shaped pattern, the formula identification step identifies the formula of the curve. method.

26. A program for performing anti-aliasing on an image, Computer equipment, An edge detection means for detecting edge information relating to the edges of regions in an image that are composed of pixels of the same or similar color, A pattern identification means that identifies the edge pattern based on the detected edge information, A formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape, A weight calculation means that calculates the interpolation weight for each pixel based on a specified pattern and a specified mathematical formula, A blend color identification means that identifies the colors to be blended in interpolation based on an identified pattern, A blending means that identifies the blended color for each pixel based on the calculated interpolation weights and the identified blended color, A weight change receiving mechanism that accepts input to change information regarding the criteria for calculating interpolation weights. To make it function as, The weight change receiving mechanism accepts input to change information for identifying the slope of the reconstruction line. program.

27. A computer device for performing anti-aliasing on an image, An edge detection means for detecting edge information relating to the edges of regions in an image that are composed of pixels of the same or similar color, A pattern identification means that identifies the edge pattern based on the detected edge information, A formula identification means for identifying a formula corresponding to a reconstruction line identified based on the edge shape, A weight calculation means that calculates the interpolation weight for each pixel based on a specified pattern and a specified mathematical formula, A blend color identification means that identifies the colors to be blended in interpolation based on an identified pattern, A blending means that identifies the blended color for each pixel based on the calculated interpolation weights and the identified blended color, A weight change receiving mechanism that accepts input to change information regarding the criteria for calculating interpolation weights. Equipped with, The weight change receiving mechanism accepts input to change information for identifying the slope of the reconstruction line. Computer device.

28. A method for performing anti-aliasing on an image, which is performed on at least one computer device, An edge detection step for detecting edge information relating to the edges of regions in an image that consist of pixels of the same or similar color, A pattern identification step that identifies the edge pattern based on the detected edge information, A formula identification step that identifies a formula corresponding to a reconstruction line identified based on the edge shape, A weight calculation step that calculates the interpolation weight for each pixel based on the identified pattern and identified mathematical formula, A blend color identification step that identifies the colors to be blended in interpolation based on the identified pattern, A blending step is performed to identify the blended color for each pixel, based on the calculated interpolation weights and the identified blended color. Weight change request step that accepts input to change information about the criteria for calculating interpolation weights. It has, The weight change acceptance step accepts input to change the information used to identify the slope of the reconstruction line. method.

29. A program for performing anti-aliasing on an image, Computer equipment, A color space conversion means for converting the color space of an image, A blending means identifies a color obtained by blending the color of pixels touching the edges of regions composed of pixels of the same or similar color, which are included in an image converted from a color space to a linear color space, with other colors. To make it function as, The color space conversion means can convert the color space of the portion of the color excluding pixels of a predetermined color into a linear color space. program.

30. A computer device for performing anti-aliasing on an image, A color space conversion means for converting the color space of an image, A blending means identifies a color obtained by blending the color of pixels touching the edges of regions composed of pixels of the same or similar color, which are included in an image converted from a color space to a linear color space, with other colors. Equipped with, The color space conversion means can convert the color space of the portion of the color excluding pixels of a predetermined color into a linear color space. Computer device.

31. A method for performing anti-aliasing on an image, which is performed on at least one computer device, A color space conversion step that converts the color space of an image, A blending step identifies a color obtained by blending the color of pixels touching the edges of regions composed of pixels of the same or similar color in the image converted from a color space to a linear color space with other colors. It has, The color space conversion step makes it possible to convert the color space of the portion of the color excluding pixels of a predetermined color to a linear color space. method.

Images