A method for generating realistic colored pencil drawings

By editing the brightness and saturation components in pencil drawing generation and extracting the outlines in combination with local extreme filtering technology, the problems of unsatisfactory pencil drawings and incorrect outline colors in the existing technology are solved. The generated color pencil drawings are natural and have high brightness. Low saturation and the outline color is consistent with the foreground object.

CN113962851BActive Publication Date: 2024-11-01GUILIN GUANGXUN CHIP TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111261449.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-28
Publication Date
2024-11-01
Estimated Expiration
2041-10-28

AI Technical Summary

Technical Problem

In the prior art, the pencil drawing tone simulation effect is not ideal, especially the outline lines are incorrect, making it difficult to achieve consistency with the hand-painted style of professionals.

Method used

By converting the original image to the HSI color space, editing the brightness and saturation components, generating high-brightness and low-saturation components, and extracting the contours in combination with local maximum and local minimum values, blending the tone image and color contours to generate realistic Colorful pencil drawing.

Benefits of technology

The image is colorful and natural, meets the high brightness requirements of pencil drawings, reduces saturation, avoids abnormal gray areas, and ensures that the outline color is consistent with the color of the foreground object.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113962851B_ABST
    Figure CN113962851B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for generating realistic colored pencil drawings, comprising: converting an original image into an HSI color space to obtain a hue component, a saturation component and a brightness component; editing the brightness component to generate a high brightness component; editing the saturation component to generate a low saturation component, merging the hue component, the high brightness component and the low saturation component and converting them into an RGB color space to obtain a hue image; converting the original image into a grayscale image, extracting contours using local maximum and local minimum values ​​for the grayscale image, fusing the extracted contours to obtain a final image contour; determining the color contour of the pencil drawing according to the original image and the final image contour; and obtaining the final colored pencil drawing by fusing the hue image and the color contour. The color image generated by the method of the present invention meets the hue requirements of high brightness and low saturation for pencil drawings, and can achieve that the color of the generated contour line is consistent with the foreground object.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to a method for generating realistic colored pencil drawings. Background Art

[0002] Tone mapping and pencil line simulation are two important links in pencil drawing generation. The tone of pencil drawing has the characteristics of high brightness and low saturation. To simulate this effect, Gao et al. converted the image from RGB color space to YUV color space and used a local maximum filtering technique to generate a new Y brightness component. The value of each pixel of this component is not lower than the brightness value of the original corresponding pixel, so as to obtain an image with a high brightness component. Then the image is fused with the original image. Lu et al. observed that the brightness of pencil drawing consists of three layers: low, medium and high, and they satisfy Gaussian distribution, uniform distribution and Laplace distribution respectively. Accordingly, the brightness histogram of natural images is decomposed and the parameter models of three distribution functions are established to simulate the tone of pencil drawing. Jin et al. proposed geometric information constraints such as T-junction, convexity, continuity and approximation to simulate the tone effect of pencil drawing by mining the perceptible clues of the extracted contours at different levels when coloring the internal area of ​​the contour image. The existing tone mapping technology is not ideal in terms of generation effect and universality, especially in terms of saturation.

[0003] Pencil lines outline the contours of objects in the scene, with the characteristics of positive correlation between grayscale intensity and image edge intensity and line color corresponding to the color of foreground objects. Existing pencil line generation techniques mainly include gradient-based and Gaussian function difference-based. Zou et al. proposed an effective gradient estimation algorithm to automatically generate contour lines similar to pencil drawing structures from personal photos. Son et al. proposed two models of line extraction and drawing to simulate hand-drawn line drawings, and generated various hand-drawn line drawings by controlling the details, focus areas and styles of two-dimensional images. An optimization framework is proposed to unify the ideas of multiple gradient domain edge extraction into a single optimization model. Lu et al. proposed a gradient-based convolution framework model to generate the effect of piecewise linear pencil sketch lines. Winnemoller et al. used iterative Gaussian function difference to extract the contour of the image for image abstraction. Kang et al. proposed a flow-induced anisotropic Gaussian function difference filtering technology to extract smooth and consistent image line structures. Winnemoller et al. used Gaussian function difference to generate images with various styles and artistic effects. Spicker et al. proposed a depth-aware consistent line drawing generation technology. Currently, the main purpose of pencil drawing line simulation is to achieve the similar effect of hand-drawn pencil drawing lines, but no effective solution has been provided for the problem of incorrect color of colored contour lines. Summary of the invention

[0004] The purpose of the present invention is to provide a method for generating realistic colored pencil drawings, so as to solve the problems of unsatisfactory tone simulation effect and incorrect contour line color in the prior art, so that the tone and contour of the generated colored pencil drawings are as consistent as possible with the hand-drawn style of professionals.

[0005] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0006] A method for generating realistic colored pencil drawings comprises the following steps:

[0007] The original image is converted into the HSI color space to obtain a hue component, a saturation component and a brightness component; the brightness component is edited to generate a high brightness component; the saturation component is edited to generate a low saturation component, and the hue component, the high brightness component and the low saturation component are combined and converted into the RGB color space to obtain a hue image;

[0008] Convert the original image into a grayscale image, extract the contour of the grayscale image using the local maximum and local minimum, fuse the extracted contours, and obtain the final image contour;

[0009] Determining a color outline of the pencil drawing according to the original image and the final image outline;

[0010] The final colored pencil drawing is obtained by fusing the tone image with the colored outline.

[0011] Further, the brightness component is edited to generate a high brightness component, using the following expression:

[0012]

[0013] Where T I is the high brightness component, I I is the brightness component, σ1 is the adaptive control parameter, and its value is determined by expression 2:

[0014] σ1=α*I m *(1-f(I I )) (2)

[0015] Where f(I I )=exp(I I 2 -1 / (I I 2 +ε)), ε is a small number to prevent the denominator from being 0; α is the weight coefficient, I m Represents the mean brightness of the original image.

[0016] Further, the editing of the saturation component to generate the low saturation component includes:

[0017] According to its brightness component I I and saturation component I s Calculate the weight coefficient λ in expression (3):

[0018] λ.*I s +I I =1 (3)

[0019] Using the calculated weight coefficient λ and the edited high brightness component T I , calculate the desired low saturation component T s for:

[0020] T s =(1-T I ). / (λ+ε) (4)

[0021] Furthermore, the process of converting the original image into a grayscale image and extracting contours of the grayscale image using local maximum and local minimum respectively includes:

[0022] The original image I is converted into a grayscale image d, and the object contour g1 extracted based on the local maximum can be expressed as:

[0023] g1=f max ([d p ])-d p (5)

[0024] d p is the gray value of pixel p in gray image d, f max (·) is the maximum value definition, [·] is the selected local neighborhood centered on pixel p;

[0025] The object contour g2 extracted based on the local minimum is defined as:

[0026] g2=d p -f min ([d p ]) (6)

[0027] where f min (·) is defined as the minimum value.

[0028] Furthermore, the extracted contours are fused to obtain the final image contour, and the expression used is as follows:

[0029] g=M.*g1+(1-M).*g2 (7)

[0030] g is the final image contour; M can be generated by image segmentation method or interactively. By default, M is an all-1 matrix.

[0031] Furthermore, determining the color outline of the pencil drawing according to the original image and the final image outline includes:

[0032] Assume that the colored pencil drawing is drawn on white paper, and let the pure white image with the same size as the original image I be Then the colored contour C of the pencil drawing can be expressed as:

[0033]

[0034] Among them, g is the final image contour.

[0035] Furthermore, the final colored pencil drawing is obtained by fusing the tone image with the color outline, which is expressed as follows:

[0036] P=C.*T (9)

[0037] Among them, P is the final colored pencil drawing, C is the colored outline, and T is the toned image.

[0038] Compared with the prior art, the present invention has the following technical features:

[0039] 1. The present invention constructs a parameter adaptive function that combines the overall brightness and local brightness of the image, effectively improving the overall brightness of the image without changing the relative relationship between the original brightness within the brightness range. The image generated by this method is colorful and natural, meeting the high brightness requirements of pencil drawings.

[0040] 2. Based on the observation that there is a complementary relationship between the saturation component and the brightness component of an image pixel, the present invention establishes a mathematical model between them according to the original image, and generates a new low-saturation component with the help of the adjusted high-brightness component. This method can effectively reduce the image saturation while maintaining the original image content and avoiding the appearance of abnormal gray areas.

[0041] 3. The present invention provides an extreme value filtering edge extraction technology, which can control the extracted edge pixels to belong to the local foreground object rather than the background area, so that the generated contour color corresponds to the corresponding local foreground object color. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the process of processing an input image by the method of the present invention;

[0043] Figure 2 This is a comparison diagram of the effects of one embodiment of the present invention and several existing methods. DETAILED DESCRIPTION

[0044] The method of the present invention firstly provides an adaptive brightness optimization model, which effectively improves the overall brightness of the image without changing the relative relationship between the original brightness within the brightness range; then based on the observation that the saturation component and the brightness component of the image pixel have a complementary relationship, this relationship of the original natural image is transferred to the pencil drawing as a constraint, and a low saturation component that meets the style of the pencil drawing is generated according to the adjusted high brightness component; and an extreme value filtering technology is proposed to achieve the consistency of the extracted pencil drawing contour shape, intensity and color with the edge features of the corresponding object in the original image.

[0045] See attached Figure 1 The present invention provides a method for generating realistic colored pencil drawings. The method first converts an original image into an HSI color space to obtain a hue component, a brightness component and a saturation component; edits the brightness component to generate a high brightness component; edits the saturation component to generate a low saturation component, merges the hue component, the high brightness component and the low saturation component and converts them into an RGB color space to obtain a hue image; converts the original image into a grayscale image, extracts contours of the grayscale image using local maximum and local minimum, fuses the extracted contours to obtain a final image contour; determines the color contour of the pencil drawing according to the original image and the final image contour; and obtains the final colored pencil drawing by fusing the hue image with the color contour.

[0046] The specific steps of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0047] Step 1: Edit the tone of the original image I to generate a tone image T that meets the characteristics of high brightness and low saturation of pencil drawings, as follows:

[0048] Step 1.1 Convert the original image I to the HSI color space to obtain the hue component I H , saturation component I s and brightness component I I .

[0049] Step 1.2, considering the smoothness, boundedness, and monotonicity of the exponential function, this scheme establishes the brightness value T I and I I The relationship between them is shown in Formula 1.

[0050] Use formula 1 to calculate the brightness component I I Edit to generate the high brightness component T I :

[0051]

[0052] Among them, σ1 is an adaptive control parameter, and the value of σ1 has a direct impact on the result. When the brightness of pixel I in the original image is low, a larger σ1 value is assigned to suppress the brightness from increasing too quickly, while when the brightness is high, a smaller σ1 value is assigned to prevent the brightness from decreasing. At the same time, the overall brightness value of the original image is taken into account. When the overall brightness value is low, a larger adjustment is made, and when the overall brightness value is high, a smaller adjustment is made.

[0053] Assume that the overall brightness value of the original image I is the average brightness value of the entire image I m The adaptive control parameters that take into account the whole are determined by the following formula 2:

[0054] σ1=α*I m *(1-f(I I )) (2)

[0055] Where f(I I )=exp(I I 2 -1 / (I I 2 +ε)), ε is a relatively small number to prevent the denominator from being 0. In this embodiment, ε=10 -5 ; α is the weight coefficient.

[0056] Step 1.3, for the saturation component I s Edit to generate low saturation component T s .

[0057] Since the brightness component I I and saturation component I s All belong to [0,1]. In this scheme, this feature is understood as I s and 1-I I Proportional, written as an expression:

[0058] λ.*I s +I I =1 (3)

[0059] Where λ is the weight coefficient, and the parameter subscript "." represents the dot product, the same below. s (i)=0 and I I When (i)≠1, expression (3) does not hold. To avoid this situation, a relatively small positive number ε is artificially introduced into the saturation component of the image, that is, I s =I s +ε.

[0060] Pencil drawings have the characteristics of high brightness and low saturation, which still satisfy the relational expression (3). That is, for natural images, when their brightness components are increased, a low saturation component can be generated by using the expression (3), which satisfies the requirements of the hue of pencil drawings. For this reason, this scheme extends the relationship between the saturation component and the brightness component of natural images in the same scene to pencil drawings, that is, in the original natural image, according to its brightness component I I and saturation component I s Calculate the weight coefficient λ in expression (3), and use the weight coefficient λ and the high brightness component T generated by editing I , calculate the desired low saturation component T s for:

[0061] T s =(1-T I ). / (λ+ε) (4)

[0062] Step 1.4: convert the hue component I in step 1.1 H , step 1.2 generates the high brightness component T I And step 1.3 generates the low saturation component T s The three components are combined and converted to RGB color space to obtain the tone image T.

[0063] Step 2: Generate a colored pencil outline C based on the original image I.

[0064] In order to extract effective object contours, this scheme proposes an extreme value contour extraction algorithm. The idea of ​​this algorithm is derived from the following points: 1) Compared with the difference with adjacent pixels, the contrast between the extreme values ​​of the current pixel and the pixel in the local neighborhood can better express the strength of the object contour; 2) Statistics show that in most cases, the grayscale value of the local background area in the image is higher than the corresponding foreground grayscale value, and under the influence of factors such as shadows, the contour should be located at a position with a small local grayscale value, that is, the greater the difference between the current pixel grayscale value and the maximum grayscale value in the local neighborhood, the greater the possibility that the pixel belongs to the contour; 3) When the grayscale value of the local foreground area is greater than that of the corresponding background area, in order to keep the color of the extracted contour line consistent with the color of the foreground object, the contour line should be set at a position with a large local grayscale value.

[0065] Step 2.1, grayscale contour line g is generated.

[0066] This scheme proposes an extreme contour extraction algorithm to extract the grayscale contour line of an image, which includes two cases based on local maximum and local minimum.

[0067] The original image I is converted into a grayscale image d, and the object contour g1 extracted based on the local maximum can be expressed as:

[0068] g1=fmax ([d p ])-d p (5)

[0069] d p is the gray value of pixel p in gray image d, f max (·) is the maximum value definition, [·] is the selected local neighborhood centered on pixel p, such as 5×5; p ∈[0,1], so g1∈[0,1].

[0070] The object contour g2 extracted based on the local minimum is defined as:

[0071] g2=d p -f min ([d p ]) (6)

[0072] where f min (·) is defined as the minimum value.

[0073] This solution optimizes the combination of the contour g1 extracted based on the local maximum value and the contour g2 extracted based on the local minimum value to obtain the final image contour g of the image, that is:

[0074] g=M.*g1+(1-M).*g2 (7)

[0075] M can be generated by image segmentation method or interactively. By default, M is a matrix of all 1s.

[0076] Step 2.2, color contour line C is generated.

[0077] Assume that the colored pencil drawing is drawn on white paper, and let the pure white image with the same size as the original image I be Then the colored contour C of the pencil drawing can be expressed as:

[0078]

[0079] Step 3: The tonal image is fused with the color contour to generate the pencil drawing image P.

[0080] Similar to the colored pencil drawing, the final pencil drawing image P is generated by fusing the hue image T generated in step 1 and the color contour C generated in step 2. The final generated colored pencil drawing P can be expressed as:

[0081] P=C.*T (9)

[0082] Example:

[0083] See also Figure 2 , which is a comparison chart of the effects of the method of the present invention and the existing method. Figure 2In the figure, group (a) is the original image, group (f) is the result of the method of the present invention generating colored pencil drawings from the original image, and groups (b), (c), and (d) are the results of the existing methods generating colored pencil drawings from the original image, which are:

[0084] (b)C.Lu,L.Xu,J.Jia:Combining sketch and tone for pencil drawing production.Proceedings of the Symposium on Non-Photorealistic Animation andRendering(Annecy,France,2012),pp.65-73.

[0085] (c)C.Gao,M.Tang M,X.Liang,Z.Su Z,C.Zou:PencilArt:a chromaticpenciling style generation framework.Computer Graphics Forum,37(6):395-409,2018.

[0086] (d)Y.Li,C.Fang,A.Hertzmann,E.Shechtman,M.Yang:Im2Pencil:ControllablePencil Illustration From Photographs.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CA,USA,2019),pp.1525-1534.

[0087] It can be clearly seen from the effect comparison that the method of the present invention is more realistic in detail processing, and the generated color image meets the high-brightness and low-saturation tone requirements of pencil drawings, and the color of the generated contour line can be consistent with the foreground object.

[0088] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for generating realistic colored pencil drawings, characterized in that: The following steps are involved: Convert the original image to the HSI color space to obtain the hue component, saturation component and brightness component; The luminance component is edited to generate a high luminance component; The saturation component is edited to generate a low saturation component, and the hue component, the high brightness component and the low saturation component are combined and converted into an RGB color space to obtain a hue image; Convert the original image into a grayscale image, extract the contour of the grayscale image using the local maximum and local minimum, fuse the extracted contours, and obtain the final image contour; Determining the color outline of the pencil drawing according to the original image and the final image outline; By fusing the tone image with the color outline, a final colored pencil drawing is obtained; The brightness component is edited to generate a high brightness component, using the following expression: Where T I is the high brightness component, I I is the brightness component, σ1 is the adaptive control parameter, and its value is determined by expression (2): σ1=α*I m *(1-f(I I )) (2) Where f(I I )=exp(I I 2 -1 / (I I 2 +ε)),ε=10 -5 ; α is the weight coefficient, I m Represents the mean brightness of the original image; The step of editing the saturation component to generate a low saturation component comprises: According to its brightness component I I and saturation component I s Calculate the weight coefficient λ in expression (3): λ.*I s +I I =1 (3) The parameter subscript ". " represents the dot product; the calculated weight coefficient λ and the edited high brightness component T are used. I , calculate the desired low saturation component T s for: T s =(1-T I ). / (λ+ε) (4) The method of converting the original image into a grayscale image and extracting contours of the grayscale image using local maximum and local minimum values ​​respectively includes: The original image I is converted into a grayscale image d, and the object contour g1 extracted based on the local maximum is expressed as: g1=f max ([d p ])-d p (5) d p is the gray value of pixel p in the gray image d, f max (·) is the maximum value definition, [·] is the selected local neighborhood centered on pixel p; The object contour g2 extracted based on the local minimum is defined as: g2=d p -f min ([d p ]) (6) where f min (·) is defined as the minimum value; The extracted contours are fused to obtain the final image contour, and the expression used is as follows: g=M.*g1+(1-M).*g2 (7) g is the final image contour; M is generated by image segmentation method or interactive method. By default, M is a matrix of all 1s; Determining the color outline of the pencil drawing according to the original image and the final image outline includes: Assume that the colored pencil drawing is drawn on white paper, and let the pure white image with the same size as the original image I be Then the colored contour C of the pencil drawing is expressed as: Among them, g is the final image contour; The final colored pencil drawing is obtained by fusing the tone image with the color outline, which is expressed as follows: P=C.*T (9) Among them, P is the final colored pencil drawing, C is the colored outline, and T is the toned image.

Citation Information

Patent Citations

  • Compact dosage unit for buccal administration of a pharmacologically active agent

    WO2000042959A1

  • Method for automatically drawing two-dimensional color pencil drawings

    CN103489204A