Image processing apparatus, method, and program

Abstract

To provide an image processing apparatus that achieves appropriate decoration processing according to the size of an object area.SOLUTION: An image processing apparatus comprises: setting means that sets, to an image, an area to which decoration processing is applied according to decoration data; first acquisition means that acquires texture data representing a texture image of a predetermined size; second acquisition means that acquires illumination data representing the light and dark contrast of an area of a predetermined size; first processing means that processes the texture data acquired by the first acquisition means to be matched with the size of the area set by the setting means; second processing means that processes the illumination data acquired by the second acquisition means to be matched with the size of the area set by the setting means; and application means that generates decoration data from the texture data processed by the first processing means and the illumination data processed by the second processing means, and applies the decoration data to the area set by the setting means.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, method, and program for performing decoration processing.

Background Art

[0002] There is known a decoration process of synthesizing texture information such as metal, cloth, and canvas texture into image information to give texture to the image. In particular, in expressing the texture of metal, not only texture information but also illumination information indicating the brightness distribution may be added. Patent Document 1 describes a technique for expressing a more realistic metal reflection by synthesizing texture information reflecting illumination information into image information.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, Patent Document 1 does not mention processing illumination information indicating the brightness distribution according to the size of the object to which the texture information is applied. Therefore, when the size of the object to which the decoration process is applied changes, the way the brightness contrast due to the illumination information is applied changes, and as a result, the texture of the metal imparted may change.

[0005] An object of the present invention is to provide an image processing apparatus, method, and program that realize appropriate decoration processing according to the size of a target area.

Means for Solving the Problems

[0006] In order to solve the above problems, an image processing apparatus according to the present invention includes: setting means for setting an area to which a decoration process is applied to an image using decoration data; first acquisition means for acquiring texture data representing a texture image of a predetermined size; second acquisition means for acquiring illumination data representing the light and dark contrast of an area of a predetermined size; first processing means for processing the texture data acquired by the first acquisition means so as to match the size of the area set by the setting means; second processing means for processing the illumination data acquired by the second acquisition means so as to match the size of the area set by the setting means; application means for generating the decoration data from the texture data processed by the first processing means and the illumination data processed by the second processing means, and applying the decoration data to the area set by the setting means.

Effect of the Invention

[0007] According to the present invention, it is possible to realize an appropriate decoration process according to the size of the target area.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Best Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are given the same reference numerals, and duplicate explanations are omitted.

[0010] [First Embodiment] FIG. 1 is a block diagram for explaining an example of the hardware configuration of the image processing apparatus according to the present embodiment. The CPU (Central Processing Unit) 100 comprehensively controls the image processing apparatus 113. For example, the CPU 100 executes the decoration process described in the present embodiment according to a program. The ROM 101 is a non-volatile memory and stores, for example, programs executed by the CPU 100. The RAM 102 is a volatile memory and is used as a memory for temporarily storing various information when the program is executed by the CPU 100. The secondary storage device 103 such as a hard disk is a storage medium for storing, for example, image files used in the present embodiment. The display 104 (display unit) displays various user interface screens and presents, for example, the processing result of the decoration process to the user. The display 104 may be configured to be able to receive user operations by providing a touch panel function. The control bus / data bus 109 interconnects the above-described respective units and the CPU 100. The mouse 105 and the keyboard 106 are operation units for receiving user operations and receive, for example, an execution instruction for the decoration process from the user.

[0011] The wireless LAN (Local Area Network) interface (IF) 108 is an interface for connecting the image processing apparatus 113 to an external network 111 via a wireless LAN. In FIG. 1, the Internet is shown as an example of the network 111, and the image processing apparatus 113 can acquire image data from an external server 112 connected to the Internet, for example. The IF 107 is an interface for connecting the image processing apparatus 113 to an external output device 110.

[0012] The output device 110 is a device for outputting images and the like, and is, for example, an inkjet recording device including a data transfer unit, a printer control unit (including a CPU, a ROM, and a RAM), a printing unit, and the like. The output device 110 is, for example, an inkjet recording device that performs recording on a recording medium based on the decoration data acquired from the image processing device 113. In the present embodiment, the output device 110 will be described as an inkjet recording device, but the output device 110 may be a recording device using another recording method, for example, an electrophotographic method. Further, the output device 110 is not limited to a recording device, and may be a display or a projector that performs display output. Note that the output device 110 is connected to the Internet 111, and the image processing device 113 can transmit and receive recording data to be processed by the output device 110 via the wireless LAN IF 108.

[0013] The image processing device 113 is, for example, a general-purpose PC or a mobile terminal such as a smartphone. The block configuration of the image processing device 113 is not limited to that shown in FIG. 1, and may appropriately include other configurations according to the functions that the image processing device 113 can implement. For example, when the image processing device 113 is a mobile terminal, it may include an imaging unit such as a camera.

[0014] FIG. 2 is a diagram showing an example of a user interface (UI) screen of software in the present embodiment. The software is, for example, a decoration application for adding texture information such as metal, cloth, or canvas fabric to image information to give the image a texture (decoration effect), and is stored in the ROM 101 or the secondary storage device 103. The screen of FIG. 2 is displayed by the execution of the decoration application by the user. Region 200 represents the entire image that is the output target of the output device 110. Region 201 is a list of objects that can be arranged within Region 200. Examples of objects included in Region 201 include character strings and figures. The user can select a desired object from Region 201 and arrange it at a desired position in Region 200. FIG. 2 shows an example in which the character string "Metalic" is arranged as object 202. The user can edit the font and size of the selected character string by selecting a desired character string and specifying a desired font and size in Region 203. Region 204 is a list of decoration patterns that can be applied to the object. The user can give a metallic feeling to the object by selecting a desired object and a desired decoration pattern from Region 204. Button 205 is a button for receiving a print instruction.

[0015] FIG. 3 is a block diagram showing an example of the configuration of the decoration application in the present embodiment. The target area setting unit 300 acquires the coordinate information of the object selected by the user as the target of the decoration process on the UI screen and calculates the size of the object. The texture data acquisition unit 301 acquires the texture data associated with the decoration pattern selected by the user on the UI screen. The lighting data acquisition unit 302 acquires the lighting data associated with the decoration pattern selected by the user on the UI screen. The lighting data will be described later.

[0016] The texture pattern generation unit 303 generates a texture pattern according to the object size calculated by the target area setting unit 300, using the texture data acquired by the texture data acquisition unit 301. The illumination pattern generation unit 304 generates an illumination pattern according to the object size calculated by the target area setting unit 300, using the illumination data acquired by the illumination data acquisition unit 302. Thus, in this embodiment, the processes of generating the texture pattern and generating the illumination pattern are each executed according to the object size.

[0017] The decoration pattern generation unit 305 synthesizes the illumination pattern generated by the illumination pattern generation unit 304 with the texture pattern generated by the texture pattern generation unit 303 to generate a decoration pattern. The decoration pattern application unit 306 applies the decoration pattern generated by the decoration pattern generation unit 305 to the object selected by the user as the target of the decoration process on the UI screen. Thereby, the object to which the decoration pattern is applied can obtain a metallic feeling. The output unit 307 causes an image including the object to which the decoration pattern has been applied by the decoration pattern application unit 306 to be printed by the output device 110, which is an inkjet printer, in response to the user pressing the button 205 on the UI screen.

[0018] Here, the characteristics of the texture data used in this embodiment will be described. As a characteristic of a metallic substance, there is a strong metallic luster generated by the plasmon resonance of free electrons in the substance and the electromagnetic wave of the illumination light. In a person's perception of the texture of a metal, perceiving the luster due to this metallic luster is one of the important elements.

[0019] The perception of metallic texture based on the above-mentioned glossiness is the same in the perception of the texture of substances projected onto a two-dimensional image. The human brain can perceive the metallic texture of the substance projected onto the image by using the statistic in the image that has a high correlation with the above-mentioned glossiness. It is known that the skewness of the luminance histogram contributes to the apparent glossiness of the substance in the image. Skewness is a statistic representing the bias of the histogram in an image, and is calculated by Equation (1) using the number of pixels n in the image, pixel values xi (i: 1, 2, ···, n), the average value x(-) of each pixel value, and the standard deviation s.

[0020] TIFF0007698503000001.tif25150 In the case of a symmetric distribution as shown in Fig. 4(b), for example, in the case of a normal distribution, the skewness is 0. In the case of a distribution with a long left tail as shown in Fig. 4(a), the skewness takes a negative value, and in the case of a distribution with a long right tail as shown in Fig. 4(c), the skewness takes a positive value. It is known that the greater the positive skewness of the luminance histogram in the image, the higher the perceived apparent glossiness.

[0021] As described above, the glossiness of the substance is an extremely important factor in perceiving the metallic texture. That is, using texture data with a large positive skewness of the luminance histogram in the image leads to an improvement in the apparent glossiness, that is, an improvement in the perceived metallic texture.

[0022] In perceiving the metallic texture from the texture data, the frequency characteristics of the texture data are also an important factor. Fig. 5 is a diagram showing the frequency response characteristics of human vision with the horizontal axis being the frequency and the vertical axis being the contrast sensitivity. The frequency response characteristics of vision are sensitivity characteristics showing the contrast that a person can visually recognize at a certain frequency. When the coarseness of the texture changes due to scaling etc. of the texture data, its frequency characteristics change, so the metallic texture perceived by a person changes. When creating the texture data, it is desirable to make the texture have a coarseness such that the skewness of the above-mentioned luminance histogram is the highest in consideration of the frequency response characteristics of vision. Hereinafter, an example of a method for generating luminance contrast data considering the frequency response characteristics of vision will be described.

[0023] First, convert the texture data into luminance contrast data. When the input texture data is RGB data, convert it to YCbCr using Expressions (2), (3), and (4). Note that the conversion formula from RGB to YCbCr is just an example, and other conversion formulas may also be used.

[0024] Y = 0.299×R + 0.587×G + 0.114×B ···(2) Cb = -0.169×R - 0.331×G + 0.5×B ···(3) Cr = 0.5×R - 0.419×G - 0.081×B ···(4) Next, a method for simulating the perceived luminance contrast data by applying the frequency response characteristics of vision to the luminance contrast data will be described. As the frequency response characteristics of vision (VTF), for example, the following Dooley's approximation formula can be used.

[0025] VTF = 5.05×exp(-0.138×πlf / 180)×(1 - exp(-0.1×πlf / 180)) ···(5) Here, l is the observation distance [mm], and f is the frequency [cycle / mm]. The assumed observation distance of the output image may be set for l. In the above, an example using Dooley's approximation formula was shown, but the frequency response characteristics of vision are not limited to this. Any sensitivity characteristic that shows the contrast that can be visually recognized according to the frequency may be used.

[0026] Subsequently, convert the luminance contrast data into frequency data. For the conversion into frequency data, known techniques such as two-dimensional Fourier transform (FFT: Fast Fourier Transform) can be used. The frequencies included in the data can be calculated based on the number of pixels of the texture data and the size after printing. For example, when the size after printing of the texture data is Size [mm], the highest frequency f [cycle / mm] included in the texture data can be calculated by Expression (6) assuming the number of pixels of the texture data is n [pix].

[0027] f = n / (2 × Size) ···(6) For each frequency in the texture data calculated based on the number of pixels of the texture data and the size after printing as described above, the frequency response characteristics of vision are multiplied, and the obtained frequency data is inverse-transformed into luminance contrast data. As a result, perceptual luminance contrast data simulating the contrast perceived by humans can be calculated. In the present embodiment, as an example, a case where the texture data is rectangular is assumed, but the texture data is not necessarily rectangular. In that case, the frequency in the texture data may be calculated based on the horizontal width according to the shape of the texture data, or the frequency in the texture data may be calculated based on the vertical width. Further, the frequency in the texture data may be calculated based on the average value of the vertical width and the horizontal width.

[0028] As described above, in order to express a realistic metallic feeling, in addition to the characteristics of the texture data, the light and dark contrast by the lighting data is also important. Here, an example of the method for calculating the light and dark contrast in the present embodiment will be described.

[0029] First, as a first example, the standard deviation or variance of the luminance histogram of the lighting data can be mentioned. The standard deviation Std is calculated by the following formula (7) using the number of pixels n in the image, the pixel values xi (i: 1, 2, ···, n), and the average value x(-) of each pixel value. Also, the variance is obtained by 2 it.

[0030] TIFF0007698503000002.tif39143Next, as a second example, the difference between the maximum value Ymax and the minimum value Ymin of the luminance values in the lighting data can be mentioned. Or, the ratio of Ymax to Ymin may also be used.

[0031] Further, as a third example, the Michelson contrast calculated by the following formula (8) may be used.

[0032] contrast = (Y_max - Y_min) / (Y_max + Y_min) ···(8) In the following, an example in which the standard deviation of the luminance histogram of illumination data is used as the value of the light and dark contrast will be described.

[0033] FIG. 6 is a flowchart showing the decoration process in the present embodiment. The process of FIG. 6 is realized, for example, by the CPU 100 reading and executing a program stored in the ROM 101. In the following, each process will be mainly described with reference to each block of FIG. 3 realized by the CPU 100.

[0034] In S601, the target area setting unit 300 designates the target area for the decoration process and acquires the size of that area. For example, the target area setting unit 300 acquires the coordinate information of the object selected by the user as the target for the decoration process on the UI screen and calculates the size of that object.

[0035] In S602, the texture data acquisition unit 301 acquires the texture data for decorating the area designated in S601. The texture data acquisition unit 301 acquires the texture data associated with the decoration pattern selected by the user on the UI screen. Also, together, the texture data acquisition unit 301 acquires the skewness Sr of the luminance histogram of the perceptual luminance contrast data obtained by applying the above-mentioned frequency response characteristics of human vision to the texture data. Sr is calculated in advance using Equation (1) for each texture data and is held in association with the texture data. Also, the skewness Sr may be calculated when the texture data acquisition unit 301 acquires the texture data. The skewness Sr here is a reference skewness value. It is preferable that the skewness value of the texture in the image of the decoration pattern displayed to the user on the UI screen is also Sr.

[0036] In S603, the lighting data acquisition unit 302 acquires lighting data used for decorating the area specified in S601. Specifically, the lighting data acquisition unit 302 acquires lighting data associated with the decoration pattern selected by the user on the UI screen. Also, together, the lighting data acquisition unit 302 also acquires the brightness contrast Cr of the lighting data. Cr is calculated in advance for each lighting data using Equation (7) and is held associated with the lighting data. Also, the brightness contrast Cr may be calculated when the lighting data acquisition unit 302 acquires the lighting data. The brightness contrast Cr here is a reference contrast value. Note that it is preferable that the contrast value of the lighting in the image of the decoration pattern displayed to the user on the UI screen is also Cr.

[0037] In S604, the texture pattern generation unit 303 generates a texture pattern based on the size of the area calculated in S601. The texture pattern generation unit 303 uses the texture data acquired by the texture data acquisition unit 301 to generate a texture pattern corresponding to the object size calculated by the target area setting unit 300.

[0038] FIG. 7 is a flowchart showing the generation process of the texture pattern in S604. In S701, the texture pattern generation unit 303 determines whether the object size calculated in S601 is equal to the size of the texture image represented by the texture data selected in S602. If it is determined to be equal, the texture pattern generation unit 303 outputs the texture data as a texture pattern and ends the process of FIG. 7. On the other hand, if it is determined that they are not equal, the process proceeds to S702.

[0039] In S702, the texture pattern generation unit 303 determines whether the size of the texture image represented by the texture data selected in S602 is larger than the object size calculated in S601. If it is determined to be larger, the process proceeds to S704. On the other hand, if it is determined not to be larger, that is, if the size of the texture image represented by the texture data is smaller than the object size, the process proceeds to S703. In S703, the texture pattern generation unit 303 generates a texture pattern by tiling the texture data until it reaches a size equal to or larger than the object size. After S703, the process proceeds to S704.

[0040] In S704, the texture pattern generation unit 303 aligns the texture pattern with the object size and then clips the texture pattern to the object size. Examples of methods for aligning the texture pattern with the object size include aligning the upper left coordinates, aligning the lower left coordinates, aligning the upper right coordinates, aligning the lower right coordinates, aligning the center coordinates, and so on. If it is determined in S702 that the size of the texture data is larger than the object size, the texture pattern generation unit 303 treats the clipped texture data as the texture pattern. The texture pattern generation unit 303 outputs the texture pattern clipped in S704 and ends the process of FIG. 7. The texture pattern generation unit 303 generates perceptual luminance contrast data by applying the aforementioned frequency response characteristics of human vision to the generated texture pattern. Furthermore, the texture pattern generation unit 303 calculates the skewness S of the histogram of the perceptual luminance contrast data using Equation (1).

[0041] Referring to FIG. 6 again, in S605, the illumination pattern generation unit 304 generates an illumination pattern based on the size of the region calculated in S601. The illumination pattern generation unit 304 uses the illumination data acquired by the illumination data acquisition unit 302 to generate an illumination pattern corresponding to the object size calculated by the target region setting unit 300. The generation of the illumination pattern will be described with reference to FIG. 8.

[0042] Figure 8 is a flowchart showing the process of generating the illumination pattern of S605. In S801, the illumination pattern generation unit 304 determines whether the size of the object calculated in S601 is equal to the size of the area of the luminance distribution represented by the illumination data acquired in S603. If it is determined to be equal, the illumination pattern generation unit 304 outputs the illumination data as the illumination pattern and ends the process of Figure 8. On the other hand, if it is determined to be not equal, the process proceeds to S802.

[0043] In S802, the illumination pattern generation unit 304 determines whether the size of the area of the luminance distribution represented by the illumination data acquired in S603 is larger than the size of the object calculated in S601. If it is determined to be larger, the process proceeds to S803. If it is not larger, that is, if it is determined that the size of the area of the luminance distribution represented by the illumination data is smaller than the size of the object, the process proceeds to S804.

[0044] In S803, the illumination pattern generation unit 303 generates an illumination pattern by reducing the illumination data so that the size of the area of the luminance distribution represented by the illumination data acquired in S603 becomes the same as the size of the object calculated in S601. Also, the illumination pattern generation unit 303 calculates the light and dark contrast C of the generated illumination pattern. Then, the illumination pattern generation unit 303 checks whether the values of Sr, Cr, S, and C acquired or calculated in S602, S603, S604, and S605 satisfy the following formula (9) to perform adjustment for imparting a metallic feeling closer to the reference.

[0045] C / S ≧ Cr / Sr ···(9) As described above, in order to impart a more realistic metallic feel, the skewness of the luminance histogram of the metal texture and the light and dark contrast of the illumination are required. Therefore, in order to impart a metallic feel regardless of the object size, it is preferable that the ratio C / S of the contrast C of the illumination pattern to the skewness S of the texture pattern is equal to or greater than the reference Cr / Sr. The reference Cr / Sr is a value representing the impression of metallic feel when the user selects a decorative pattern on the UI screen. Since the texture pattern is generated by tiling or trimming texture data, their skewnesses S and Sr are substantially equal values. That is, the confirmation by Equation (9) means that it is confirmed whether the metallic feel decreases because the light and dark contrast C of the generated illumination pattern becomes smaller than the reference Cr depending on the object size.

[0046] If the relationship (condition) of Equation (9) is not satisfied, assuming that the light and dark contrast C of the generated illumination pattern is small, the size of the illumination pattern is adjusted by expanding or contracting it by ±x% (x ≤ 10) with respect to the object size. When expanding the illumination pattern during the adjustment of the illumination pattern, the adjusted illumination pattern is clipped with the object size. By expanding the illumination pattern, the ratio of the number of pixels in the bright part and the number of pixels in the dark part of the illumination within the object changes, and the light and dark contrast may increase. When contracting the illumination pattern during the adjustment of the illumination pattern, since the adjusted illumination pattern becomes smaller than the object size, the area insufficient with respect to the object size is filled with the minimum value of the luminance within the illumination pattern. By increasing the number of pixels in the dark part of the illumination within the object, the ratio of the number of pixels in the bright part and the number of pixels in the dark part of the illumination within the object changes, and the light and dark contrast increases. The illumination pattern generation unit 303 calculates the light and dark contrast value C again for the adjusted illumination pattern, and repeats the adjustment by increasing the value of x until the relationship of Equation (9) is satisfied. If Equation (9) cannot be satisfied even after processing up to x = 10, the value of x at which the left side of Equation (9) becomes the largest is adopted. By doing so, a metallic feel closer to the reference can be imparted.

[0047] In the above, an example of adjusting the light and dark contrast value C of illumination by scaling the size of the illumination pattern has been described. However, the light and dark contrast value C may also be adjusted by expanding the luminance range of the illumination pattern. Further, the light and dark contrast value C may be adjusted by combining the scaling of the size of the illumination pattern and the expansion of the luminance range. After S803, the process of FIG. 8 ends.

[0048] In S804, the illumination pattern generation unit 303 determines whether or not the region indicating the luminance distribution represented by the illumination data acquired in S603 is a target for expansion. If it is determined that it is a target for expansion, the process proceeds to S805. If it is determined that it is not a target for expansion, the process proceeds to S806.

[0049] In S804, the illumination pattern generation unit 303 determines whether the illumination data is to be enlarged based on the gradient of a predetermined luminance change of the illumination pattern and a reference value. FIG. 13 is a diagram showing an example of illumination data, and shows illumination data 1301 and 1303. Also, luminance change 1302 shows the luminance change in the horizontal direction of illumination data 1301, and luminance change 1304 shows the luminance change in the horizontal direction of illumination data 1303. As shown in FIG. 13, the illumination data 1301 has a relatively gentle (below the reference value) gradient of light and dark contrast. On the other hand, the illumination data 1303 has a relatively steep (greater than the reference value) gradient of light and dark contrast. For illumination data with a gentle gradient of light and dark contrast like illumination data 1301, even if it is scaled according to the object size, the impression of the result does not change much. On the other hand, when enlarging illumination data with a steep gradient of light and dark contrast like illumination data 1303 according to the object size, the gradient of light and dark contrast becomes gentle, and as a result, there may be a case where the intended impression of light and dark contrast cannot be given. In such a case, it is better to perform tiling processing without changing the scale of the illumination data, as the change in the gradient of light and dark contrast is smaller, and as a result, the change in the impression of light and dark contrast given to the user can be reduced. Thus, in the present embodiment, the method of generating the illumination pattern is changed according to the type of illumination data and the intended effect. Note that, in order to switch the processing according to the type of illumination data, information regarding a processing method suitable for each illumination data may be added in advance, and the illumination pattern generation unit 303 may refer to that information during processing.

[0050] In S805, the illumination pattern generation unit 303 enlarges the illumination data so that the size of the region of the luminance distribution represented by the illumination data acquired in S603 becomes the same as the object size calculated in S601, and generates an illumination pattern. Here too, for the generated illumination pattern, similar to the explanation in S803, the illumination pattern generation unit 303 performs confirmation according to Equation (9) and adjustment of light and dark contrast. After S805, the processing of FIG. 8 ends.

[0051] On the one hand, in S806, the illumination pattern generation unit 304 tiles the illumination data until it reaches a size equal to or larger than the object size, and generates an illumination pattern. In S807, the illumination pattern generation unit 304 aligns the illumination pattern with the object size and then clips the illumination pattern by the object size. Examples of the method for aligning the illumination pattern with the object size include aligning the upper left coordinates, aligning the lower left coordinates, aligning the upper right coordinates, aligning the lower right coordinates, aligning the center coordinates, and so on. The illumination pattern generation unit 304 outputs the illumination pattern clipped in S807 and ends the process of FIG. 8.

[0052] Also, in the above, an example of processing assuming that the illumination data is in raster format has been described, but the illumination data may be in vector format. When the illumination data is held in vector format, the relative coordinates of the representative points and the color information at those coordinates are held in the illumination data. Then, the illumination pattern generation unit 303 may rasterize the vector-format illumination data by the object size. At that time, the color at coordinates other than between the representative points may be calculated by interpolation from the color information at the representative points.

[0053] Note that when it is determined in S802 that it is not large, that is, the size of the region showing the luminance distribution represented by the illumination data is smaller than the object size, the process of S805 may be executed without executing the processes of S804, S806, and S807. Also, it is not always necessary to scale one piece of illumination data to match the object size, and the illumination data used for each object size may be switched. Specifically, as shown in FIG. 14, the range of the object size and appropriate illumination data may be associated and held in advance, and the illumination data to be used may be switched according to the object size actually calculated in S601. Note that when the selected illumination data is larger than the object size, it is clipped to match the object size. Also, the illumination data may be held as a bitmap image or as vector data.

[0054] Also, in the above description, when the gradient of the light and dark contrast is relatively steep, it was explained that in S804, it is determined that the illumination data is not an enlargement target, and tiling processing is performed in S806. However, in S806, other processing may be performed instead of tiling processing. For example, the region of the luminance distribution represented by the illumination data may be positioned at the center of the target region of the decoration process, and pixels may be supplemented around the target region so as to maintain the luminance distribution. Even with such a configuration, it is possible to reduce the change in the impression of light and dark contrast given to the user.

[0055] Again, referring to FIG. 6. In S606, the decoration pattern generation unit 305 synthesizes the texture pattern generated in S604 and the illumination pattern generated in S605 to generate a decoration pattern. The decoration pattern generation unit 305 performs a synthesis process so that the contrast of the texture pattern generated by the texture pattern generation unit 303 is increased by the illumination pattern generated by the illumination pattern generation unit 304. The synthesis process may use soft light or overlay, which are known layer synthesis techniques. Assuming that the pixel value of the texture pattern is a, the pixel value of the illumination pattern is b, and the pixel value of the decoration pattern is c, the result of the soft light process is calculated by Equation (10), and the result of the overlay process is calculated by Equation (11).

[0056] TIFF0007698503000003.tif20150TIFF0007698503000004.tif23150In S607, the decoration pattern application unit 306 applies the decoration pattern generated by the decoration pattern generation unit 305 to the object selected by the user as the target of the decoration process on the UI screen. When the object is a figure or a character other than a rectangle, the decoration pattern is clipped and synthesized according to the shape.

[0057] In S608, the output unit 307 transmits the image data including the object to which the decoration pattern has been applied by the decoration pattern application unit 306 to the output device 110, which is an inkjet printer, for printing. Then, the process of FIG. 6 ends.

[0058] FIG. 9 is a flowchart showing a printing process in the output device 110 which is an inkjet printer. The process in FIG. 9 starts when the output device 110 receives image data including an object to which the decoration pattern transmitted in S608 is applied. Here, the image data will be described as RGB image data.

[0059] In S901, the printer control unit of the output device 110 inputs RGB image data that is the original document to be printed. Next, in S902, the printer control unit performs color correction processing to convert the RGB colors of the original document into RGB values suitable for printing. A known suitable process may be used for this color correction processing. In S903, the printer control unit performs color separation processing to convert the RGB values into the usage amounts of each ink. A known suitable process may be used as the method of color separation processing. In S904, the printer control unit performs quantization processing to convert the usage amount of each color ink of the recording head into the presence or absence of dots to be actually recorded. As the quantization processing, methods such as known error diffusion processing and dither processing may be used. When the quantized dot data is sent to the recording head and the preparation of dot data for one scan is completed, the printer control unit performs actual recording using the recording head on the recording paper. In S905, the printer control unit determines whether the processing has been completed for all the pixels of the image data. If it is determined that the processing has been completed for all the pixels, the process in FIG. 9 ends. On the other hand, if it is determined that the processing has not been completed for all the pixels, the process from S901 is repeated.

[0060] As described above, in this embodiment, the texture data and the illumination data are held separately, and after performing respective processes according to the size of the target area, they are synthesized to generate a decoration pattern. The effects thereof will be described below.

[0061] Figure 10 is a diagram for explaining a case where the area of illumination data is not changed according to the object size. Image 1002 is an image after reflecting the illumination data on Image 1001. Due to the illumination data, a light and dark contrast like that of Image 1003 is added within Image 1001. Graph 1004 is a graph representing the horizontal luminance change of Image 1003. Due to this light and dark contrast, Image 1002 can express a more realistic metallic feeling than Image 1001.

[0062] Image 1005 in Figure 10 is an image to which a metallic feeling is to be imparted. Two objects 1006 and 1007 are arranged in Image 1005. The size of object 1006 is relatively small, and the size of object 1007 is relatively large. By synthesizing Image 1002 with Image 1005, a metallic feeling is imparted to these two objects 1006 and 1007. Image 1008 is an image obtained by synthesizing Image 1002 with Image 1005. Object 1009 corresponds to object 1006, and object 1100 corresponds to object 1007. Light and dark contrast 1011 represents the light and dark contrast due to the illumination data within object 1009, and light and dark contrast 1012 represents the light and dark contrast due to the illumination data within object 1010. It can be seen that between objects 1009 and 1010, since the size and position within the image are different, the way the light and dark contrast due to the illumination data is applied is different. That is, the light and dark contrast due to the illumination information, which is an important element for expressing the metallic feeling more realistically, changes depending on the size and position within the image of the object to which the metallic feeling is to be imparted.

[0063] FIG. 11 is a diagram for explaining a case where texture data and illumination data are processed without separation. Image 1100 is one image having both light and dark contrast due to texture and illumination. A case of imparting a metallic feeling to objects of different sizes using image 1100 will be described. Image 1101 shows a state where 1100 is reduced and applied to a relatively small object. On the other hand, image 1102 shows a state where image 1100 is enlarged and applied to a relatively large object. The light and dark contrast due to illumination is approximately the same value in both images 1101 and 1102. On the other hand, due to the influence of scaling, the coarseness of the texture (the frequency of the texture image) is different between images 1101 and 1102. Therefore, when comparing images 1101 and 1102, the impression due to the light and dark contrast of illumination is approximately the same, but the impression due to the distortion of the texture considering the frequency characteristics is different. Therefore, in the method of scaling and applying one image having both light and dark contrast due to texture and illumination to an object, if the size of the object is different, the impression of the metallic feeling imparted to the object will be different.

[0064] Image 1103 is an object of the same size as Image 1101. Image 1103 is in a state where a part of Image 1100 is trimmed and applied. On the other hand, the thick dotted line of Image 1104 is an object of the same size as Image 1102. Image 1104 is in a state where multiple copies of the same image as Image 1100 are arranged and applied. Image 1105 is in a state where it is clipped along the contour of the object from the state of Image 1104. Since the frequency components of the textures of Image 1103 and 1105 are almost the same, the skewness of the luminance histogram considering the frequency characteristics becomes almost the same value. However, due to the influence of trimming and tiling, the value of the light and dark contrast due to illumination is different between Image 1103 and 1105. Therefore, when comparing Image 1103 and 1105, the impression due to the skewness of the texture considering the frequency characteristics is almost the same, but the impression due to the light and dark contrast of the illumination is different. Therefore, in the method of applying to an object without scaling an image having both texture and light and dark contrast due to illumination, if the size of the object is different, the impression of the metallic feeling given to the object will be different.

[0065] FIG. 12 is a diagram for explaining the effect of separately processing texture data and lighting data. Using Image 1200 and Image 1201, the case of imparting a metallic feeling to objects of different sizes will be described. The upper part of FIG. 12 represents the processing for relatively small objects. Image 1202 represents a state where Image 1200 is trimmed according to the object size. Image 1203 represents a state where Image 1201 is reduced according to the object size. Image 1204 is the result of synthesizing Image 1202 and Image 1203 and applying it to the object. On the other hand, the lower part of FIG. 12 represents the processing for relatively large objects. The thick dotted lines of Images 1205, 1206, and 1207 represent the object. Image 1205 represents a state where Image 1200 is tiled according to the object size. Image 1206 represents a state where Image 1201 is enlarged according to the object size. The result of clipping the tiling result of Image 1205 according to the object size and synthesizing it with Image 1206 and applying it to the object is Image 1207. Regarding the texture data, by processing so as not to substantially change the frequency, the impression by the texture of Images 1204 and 1207 is made not to change. Also, regarding the lighting data, by processing so as not to substantially change the light and dark contrast, the impression by the lighting of Images 1204 and 1207 is made not to change.

[0066] That is, according to the present embodiment, compared with the method of scaling and applying to an object a single image having light and dark contrast due to texture and illumination, the difference in the distortion degree of the texture between two objects of different sizes is reduced. Also, compared with the method of applying to an object a single image having light and dark contrast due to texture and illumination without scaling, the difference in the light and dark contrast of illumination between two objects of different sizes is reduced. Expressing this as the ratio of the ratio of the distortion degree of the texture and the light and dark contrast due to illumination in each object, it is as follows. First, it is calculated from the ratio in the process of the present embodiment. If the distortion degree of the texture in Image 1204 is S1 and the light and dark contrast due to illumination is C1, their ratio is C1 / S1. Next, if the distortion degree of the texture in Image 1207 is S2 and the light and dark contrast due to illumination is C2, their ratio is C2 / S2. Therefore, the ratio of the ratio of the distortion degree of the texture and the light and dark contrast due to illumination in Image 1204 and the ratio of the distortion degree of the texture and the light and dark contrast due to illumination in Image 1207 is (S1·C2) / (C1·S2). Similarly, the ratio in the conventional scaling method is (S3·C4) / (C3·S4) when the texture distortion degree in Image 1101 is S3, the light and dark contrast due to illumination is C3, the texture distortion degree in Image 1100 is S4, and the light and dark contrast due to illumination is C4. Similarly, the ratio in the conventional non-scaling method is (S5·C4) / (C5·S4) when the texture distortion degree in Image 1103 is S5 and the light and dark contrast due to illumination is C5. Thereby, the operating range in the present embodiment is as shown in the following formula (12).

[0067] TIFF0007698503000005.tif23148 Also, in the present embodiment, due to these actions, compared with the above two conventional methods, it is possible to reduce the decrease in metallic feeling or the difference in the impression of metallic feeling caused by the size and position of the object. Also, it is possible to reduce the difference in the impression of metallic texture between a plurality of objects having different sizes and positions.

[0068] As described above, according to the present embodiment, by generating an illumination pattern according to the size of the object to be processed, it is possible to reduce the reduction in metallic feeling or the difference in the impression of metallic feeling caused by the size and position of the object. In addition, it is possible to reduce the difference in the impression of metallic texture between a plurality of objects having different sizes and positions. In the present embodiment, luminance is used when calculating the distortion degree of the texture pattern and the light and dark contrast of the illumination pattern, but other numerical values indicating brightness such as lightness may be used.

[0069] [Second Embodiment] In the first embodiment, it was explained that by generating an illumination pattern according to the size of the region to be processed, it is possible to reduce the reduction in metallic feeling or the difference in the impression of metallic feeling caused by the size and position of that region. Hereinafter, the second embodiment will be described with respect to the points different from the first embodiment.

[0070] Region 1501 in Fig. 15(a) is the region to which the decoration process in the present embodiment is applied. Region 1501 contains an object of a two-line character string due to line breaks. Image 1502 is a decoration pattern generated by the decoration process in the first embodiment and applied to region 1501. Light and dark contrast 1503 indicates the light and dark contrast of the illumination pattern in image 1502. Image 1504 is the result of applying image 1502 to region 1501. Light and dark contrast 1505 indicates the light and dark contrast of the illumination pattern in image 1504. Region 1501 is composed of a two-line character string, and the highlight portion of image 1502 is located between the lines. Therefore, the light and dark contrast of the illumination in the decoration pattern applied to the actual character string in region 1501 becomes smaller than light and dark contrast 1503, as indicated by light and dark contrast 1505.

[0071] In this embodiment, additional processing is performed when setting the target area for the decoration process. In this embodiment, the process of FIG. 16(a) is performed instead of the process of S601 in FIG. 6. FIG. 16(a) is a flowchart showing the process of setting the target area in this embodiment. The process of FIG. 16(a) is realized, for example, by the CPU 100 reading and executing the program stored in the ROM 101. Hereinafter, each process will be described mainly with reference to each block of FIG. 3 realized by the CPU 100.

[0072] In S1601, the target area setting unit 300 acquires the coordinate information of the area including the object selected by the user as the target of the decoration process on the UI screen. In S1602, the target area setting unit 300 calculates the size of the area including the object based on the coordinate information acquired in S1601.

[0073] In the processes after S1603, the area including the object is divided according to the presence or absence of line breaks in the area including the object, and each divided area is extracted. In S1603, the target area setting unit 300 sets a parameter i for dividing the area including the object, and sets i = 1 as the initial value.

[0074] In S1604, the target area setting unit 300 determines whether there is a line break in the area including the object. The determination of the presence or absence of a line break is, for example, when a line of RGB pixel values (255, 255, 255) is repeated and then a pixel value other than white is detected, it is determined that there is a line break. As a result of the determination in S1604, if there is no line break, the process proceeds to S1610 and the process of setting the target area in FIG. 16(a) ends. If it is determined in S1604 that there is a line break, the process proceeds to S1605.

[0075] In S1605, the target area setting unit 300 acquires the size of the string in the first line. In S1606, the target area setting unit 300 divides and extracts the string in the first line as the first object area. The target area setting unit 300 sets the size of the first object area based on the size of the string acquired in S1605.

[0076] In S1607, the target area setting unit 300 sets the coordinates of the area including the strings from the second line onward. The target area setting unit 300 updates the coordinate information acquired in S1601 based on the size of the first object area acquired in S1605. In S1608, the target area setting unit 300 calculates the size of the area including the strings from the second line onward based on the coordinate information acquired in S1607. The target area setting unit 300 calculates the size of the area including the strings from the second line onward based on the size of the area including the object calculated in S1602 and the size of the first object area calculated in S1605.

[0077] Increment i in S1609 and proceed to S1604. Thereafter, by repeating the processing of S1604 to S1609, the strings separated by line breaks are divided and extracted as separate object areas.

[0078] In S602 and subsequent steps in FIG. 6, for each object divided in FIG. 16(a), a decoration pattern corresponding to each object size is generated and applied to each object.

[0079] FIG. 17(a) is a diagram for explaining the effect of this embodiment. The area 1701 in FIG. 17(a) shows the state where the processing of this embodiment is applied to the area 1501. By the processing of target area setting in FIG. 16(a), "Met" in the first line and "alic" in the second line are set as different objects.

[0080] Image 1702 is a decoration pattern for application to area 1701, generated by the decoration process of this embodiment. Luminance contrast 1703 indicates the luminance contrast of the illumination pattern in image 1702. Image 1704 is the result of applying image 1702 to image 1701. Luminance contrast 1705 indicates the luminance contrast of the illumination pattern in image 1704. In this embodiment, in order to generate and apply a decoration pattern to each string in each row of image 1701, the highlight part of the illumination of image 1702 is applied to each string. The luminance contrast of the illumination in the decoration pattern applied to the actual string of image 1701 can be set to a value approximately equal to luminance contrast 1703, as shown by luminance contrast 1705.

[0081] FIG. 15(b) is a diagram for explaining another case where the effect of reducing the reduction in metallic feeling or the difference in the impression of metallic feeling caused by the size and position of the object cannot be sufficiently obtained.

[0082] Area 1506 in FIG. 15(b) is an area to which the decoration process is applied. Area 1506 contains two characters, "M" and "-". Image 1507 is a decoration pattern for application to image 1506. Luminance contrast 1508 indicates the luminance contrast of the illumination pattern in image 1507. Image 1509 is the result of applying decoration pattern 1507 to object 1506. Luminance contrast 1510 indicates the luminance contrast of the illumination pattern in "M" of image 1509. Luminance contrast 1511 indicates the luminance contrast of the illumination pattern in "-" of image 1509. While luminance contrast 1510 becomes approximately the same value as luminance contrast 1508, luminance contrast 1511 becomes smaller compared to luminance contrast 1508. That is, due to the difference in the shape of the characters, the way the luminance contrast by the illumination pattern is attached varies for each character.

[0083] In this embodiment, for such an object, the process of FIG. 16(b) is performed instead of the process of S601 in FIG. 6. The process of FIG. 16(b) is realized, for example, by the CPU 100 reading and executing a program stored in the ROM 101.

[0084] In S1611, the target area setting unit 300 extracts a character area from an object selected by the user as a decoration target on the UI screen. As the extraction method, for example, an existing method such as OCR may be used. In S1612, the target area setting unit 300 calculates the size of each character as an individual object area based on the character area information obtained in S1611. In S602 and subsequent steps of FIG. 6, for each object area extracted in S601, a decoration pattern corresponding to each object size is generated and applied to each object.

[0085] FIG. 17(b) is a diagram for explaining the effect of the process of FIG. 16(b). Areas 1706 and 1707 in FIG. 17(b) show the state where the process of FIG. 16(b) is applied to area 1506. By the process of FIG. 16(b), the first character 'M' and the second character '-' are extracted as different object areas.

[0086] Images 1708 and 1709 are decorative patterns for applying to Images 1706 and 1707 respectively. Graph 1710 represents the luminance distribution of the lighting pattern in Image 1708, and Graph 1711 represents the luminance distribution of the lighting pattern in Image 1709. Light and dark contrast 1712 is the light and dark contrast of Graphs 1710 and 1711. Image 1713 is the result of applying the decorative pattern of Image 1708 to Image 1706. Image 1714 is the result of applying the decorative pattern of Image 1709 to Image 1707. Light and dark contrast 1715 indicates the light and dark contrast of the lighting pattern in Images 1713 and 1714. For each character in Images 1706 and 1707, a decorative pattern is generated and applied, so that the light and dark contrast of the lighting in Images 1713 and 1714 after applying the decorative pattern becomes approximately the same value as indicated by light and dark contrast 1715.

[0087] As described above, according to this embodiment, when the area to be processed includes a plurality of objects, for example, character objects, the area is divided into areas that make up the objects, and a decorative pattern is generated according to the size of each divided area. Thereby, the difference in the impression of the metallic texture between objects can be reduced.

[0088] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0089] The invention is not limited to the above-described embodiment, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of Reference Numerals

[0090] 100 CPU: 101 ROM: 102 RAM: 113 Image processing device

Claims

1. Setting means for setting an area to which a decoration process is to be applied to an image using decoration data; First acquisition means for acquiring texture data representing a texture image of a predetermined size; Second acquisition means for acquiring illumination data representing the light and dark contrast of an area of a predetermined size; First processing means for processing the texture data acquired by the first acquisition means so as to match the size of the area set by the setting means; Second processing means for processing the illumination data acquired by the second acquisition means so as to match the size of the area set by the setting means; Application means for generating the decoration data from the texture data processed by the first processing means and the illumination data processed by the second processing means, and applying the decoration data to the area set by the setting means; An image processing apparatus comprising the same.

2. The image processing apparatus according to claim 1, wherein in the processing by the first processing means, the frequency characteristics of the texture image are not changed.

3. The image processing apparatus according to claim 2, wherein the first processing means performs clipping on the texture image or performs tiling using the texture image.

4. The image processing apparatus according to claim 3, wherein when the size of the area set by the setting means is larger than a predetermined size of the texture image, the first processing means performs tiling using the texture image so as to match the size of the area set by the setting means.

5. The image processing apparatus according to claim 4, wherein after the tiling using the texture image, the first processing means performs clipping on the texture image on which the tiling has been performed.

6. The image processing apparatus according to any one of claims 3 to 5, wherein when the size of the area set by the setting means is smaller than a predetermined size of the texture image, the first processing means performs clipping on the texture image.

7. The image processing apparatus according to any one of claims 1 to 6, wherein when the size of the area set by the setting means is larger than a predetermined size of the area of the illumination data, the second processing means enlarges the area of the illumination data.

8. The image processing apparatus according to claim 7, wherein the second processing means enlarges the area of the illumination data when the size of the area set by the setting means is larger than a predetermined size of the area of the illumination data and the change in light and dark contrast represented by the illumination data is smaller than a reference value.

9. The image processing apparatus according to claim 8, wherein the second processing means performs tiling using the illumination data instead of enlarging the area of the illumination data when the size of the area set by the setting means is larger than a predetermined size of the area of the illumination data and the change in light and dark contrast represented by the illumination data is larger than a reference value.

10. The image processing apparatus according to claim 9, wherein the second processing means performs clipping on the area of the illumination data on which tiling has been performed so as to match the size of the area set by the setting means after performing tiling using the illumination data.

11. The image processing apparatus according to claim 8, wherein the second processing means performs pixel replenishment around the area of the illumination data instead of enlarging the area of the illumination data when the size of the area set by the setting means is larger than a predetermined size of the area of the illumination data and the change in light and dark contrast represented by the illumination data is larger than a reference value.

12. The image processing apparatus according to any one of claims 1 to 11, further comprising adjustment means for adjusting the light and dark contrast of the illumination data processed by the second processing means based on the skewness of the luminance histogram of the texture image.

13. The image processing apparatus according to any one of claims 1 to 12, wherein the second acquisition means acquires the illumination data based on the size of the area set by the setting means.

14. When the area set by the setting means includes a plurality of objects, the image processing apparatus further comprises extraction means for extracting a plurality of areas from the area set by the setting means, and each of the plurality of areas extracted by the extraction means is processed as the area set by the setting means. The image processing apparatus according to any one of claims 1 to 13, characterized in that.

15. The image processing apparatus according to claim 14, wherein the extraction means extracts, as the plurality of regions, regions corresponding to the plurality of objects respectively.

16. The image processing apparatus according to claim 14, wherein the region extracted by the extraction means includes a plurality of objects.

17. The image processing apparatus according to any one of claims 14 to 16, wherein the object included in the region set by the setting means is a character.

18. The image processing apparatus according to any one of claims 1 to 17, further comprising control means for causing a printing apparatus to print data to which the decoration data has been applied by the application means.

19. A method executed in an image processing apparatus, comprising: a setting step of setting a region to which a decoration process is to be applied to an image with decoration data; a first acquisition step of acquiring texture data representing a texture image of a predetermined size; a second acquisition step of acquiring illumination data representing the light and dark contrast of a region of a predetermined size; a first processing step of processing the texture data acquired in the first acquisition step so as to match the size of the region set in the setting step; a second processing step of processing the illumination data acquired in the second acquisition step so as to match the size of the region set in the setting step; an application step of generating the decoration data from the texture data processed in the first processing step and the illumination data processed in the second processing step, and applying the decoration data to the region set in the setting step; A method characterized by comprising the steps.

20. A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 18.

Images