An automatic coloring method and device for artistic line drawings and a storage medium

Abstract

This application discloses an automatic coloring method, apparatus, and storage medium for artistic line drawings. The automatic coloring method for artistic line drawings includes the following steps: acquiring a line drawing generated from a user-input photo or a line drawing directly input by the user; preprocessing the line drawing; converting the preprocessed line drawing into a grayscale image and performing normalization processing to obtain a target image; determining whether to perform regional coloring based on the user's instructions; if so, generating a corresponding parsing mask based on the photo; extracting the area to be colored based on the parsing mask and performing normalization processing to obtain the target area; if not, skipping this step; acquiring a template converted to a first color space; selecting a coloring method to color the template, target image, and / or target area based on the depth of the template's background color and the characteristics of the line drawing; and fusing the colored template, target image, and / or target area in the second color space to obtain a coloring result.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an automatic coloring method, apparatus, and storage medium for artistic line drawings. Background Technology

[0002] Artistic line drawing is an important way to express artworks, possessing extremely high and unique aesthetic value. Nowadays, people's pursuit of artistic line drawing is increasing, and it is no longer simply limited to a single photographic format. Therefore, whether applying line drawing to various practical products or to personalized wallpapers, avatars, and emojis, it is particularly important.

[0003] At the technical level, a common fusion method involves calculating the weights between the template image and the line drawing, such as the `addweighted` function in the OpenCV library of Python. Currently, there are methods for fusing color images, primarily using line masks. However, this method is suitable for foreground images (lines) with clear and distinct edges; otherwise, the results will have harsh edges and an overall uncoordinated image. Since line drawings cannot obtain a complete mask representing the lines, this method is not suitable for line drawings with grayscale values. Another method is manual Photoshop editing, but this requires the user to have certain computer processing skills. The most obvious problem with manual editing is that it is time-consuming and labor-intensive, and the fusion result often loses some grayscale values ​​from the line drawing. In practical product applications, most line coloring and fusion processes use binarized lines. While this method can change the color and texture of line drawings and apply them to various templates, a significant problem is that binarization loses countless grayscale information from the lines, which greatly reduces the quality of the final processed image.

[0004] Many image processing tasks can be performed based on HSV. Currently, there are techniques for fusing grayscale and color images in HSV space, such as remote sensing image fusion. However, this method simply replaces the V luminance space, limiting its applicability. For example, color multi-focus image fusion in HSV space can effectively improve image contrast, saturation, and tonal change speed. However, existing methods do not utilize linear operations in HSV space to achieve line coloring and template fusion. Overall, there is currently no complete, user-selectable method for automatically coloring and adding templates to line drawings that adequately addresses the aforementioned problems. Summary of the Invention

[0005] The purpose of this application is to provide an automatic coloring method, apparatus, and storage medium for artistic line drawings, in order to solve the problem that there is no complete set of processing methods in the prior art that allows users to choose their own automatic coloring and template addition for line drawings, which can be applied to the fusion of most templates and various line drawings, and can completely preserve line information.

[0006] To achieve the above objectives, this application provides an automatic coloring method for artistic line drawings, comprising the following steps: obtaining a line drawing generated from a user-inputted photo or a line drawing directly input by the user; preprocessing the line drawing to adapt it to the mask position of a template; converting the preprocessed line drawing into a grayscale image and performing normalization processing to obtain a target image; determining whether to perform region-based coloring based on user instructions; if so, generating a corresponding parsing mask based on the photo; extracting the region to be colored based on the parsing mask and performing normalization processing to obtain the target region; if not, skipping this step; obtaining a template converted to a first color space. Based on the depth of the template background color and the characteristics of the line drawing, a coloring method is selected to color the template, the target image, and / or the target area. The template, the target image, and / or the target area in the first color space after coloring are converted back to the second color space. The first color space includes LAB space, HSV space, or HSI space, and the second color space includes RGB space. The coloring method includes solid color coloring, texture extraction coloring, and / or template adaptive coloring. The template, the target image, and / or the target area in the second color space after coloring are blended to obtain the coloring result.

[0007] Optionally, the solid color coloring method includes:

[0008] When the lines of the target image or the target area are rendered black on the template, the following formula is used: Obtain the value of the luminance channel;

[0009] When the lines of the target image or the target area are rendered white on the template, the following formula is used: and Obtain the values ​​for the saturation channel and the brightness channel;

[0010] in, This refers to the value of the luminance channel obtained using the described solid color coloring method. The value of the luminance channel of the template in HSV color space, I gray This refers to the lines in the target image or the grayscale values ​​of the target region. This refers to the saturation channel value obtained using the described solid color coloring method. This refers to the value of the saturation channel of the template in the HSV color space.

[0011] Optionally, the texture extraction-based coloring method includes:

[0012] Using the formula: and Obtain the values ​​of the saturation channel and the brightness channel.

[0013] in, The value of the luminance channel obtained using the texture extraction-based coloring method. This refers to the saturation channel value obtained using the texture extraction-based coloring method described above. The value of the saturation channel of the template in HSV color space, I gray The grayscale value refers to the lines in the target image or the grayscale value of the target region.

[0014] Optionally, the template adaptive coloring method includes:

[0015] Using the formula: and Obtain the values ​​of the saturation channel, hue channel, and brightness channel.

[0016] Among them, 0 <c<1, This refers to the value of the tone channel obtained using the template adaptive coloring method. This refers to the value of the luminance channel obtained using the template adaptive coloring method. This refers to the saturation channel value obtained using the template adaptive coloring method. This refers to the value of the hue channel of the template in the HSV color space. This refers to the saturation channel value of the template in the HSV color space. The value of the luminance channel of the template in HSV color space, I gray The grayscale value refers to the lines in the target image or the grayscale value of the target region.

[0017] Optionally, after performing the texture extraction-based colorization process and obtaining the template, the target image, and / or the target region converted back to the second color space, the method further includes:

[0018] Coloring is performed using a blending coloring method, which includes:

[0019] Using the formula: Obtain the coloring result.

[0020] in, Refers to the coloring result in the RGB space, I colorThe term "target image" refers to the target image or target region in the RGB space after being colored using the texture extraction-based coloring method, and "T" refers to the module in the RGB space.

[0021] Optionally, the preprocessing method includes:

[0022] Trim the excess white borders around the line drawing. Scale the line drawing according to its aspect ratio after trimming, so that it fits the mask position of the template.

[0023] Optionally, the method for generating the corresponding parsing mask based on the photo includes:

[0024] The photo is cropped, and the cropped photo is input into the corresponding image segmentation network to obtain the multi-channel parsing mask.

[0025] Optionally, the method for fusing the template, the target image, and / or the target region in the second color space after colorization includes:

[0026] The target regions after being colored in different areas are merged to obtain a fusion result. The fusion result is then merged with the lines of the colored target image and the template to obtain the coloring result.

[0027] To achieve the above objectives, this application also provides an automatic coloring device for artistic line drawings, comprising: a memory; and

[0028] A processor connected to the memory, the processor being configured to:

[0029] Obtain a line drawing generated based on a photo input by the user or a line drawing directly input by the user; preprocess the line drawing to adapt it to the mask position of the template; convert the preprocessed line drawing into a grayscale image and perform normalization processing to obtain the target image.

[0030] Based on the user's instructions, determine whether to perform regional coloring. If yes, generate a corresponding parsing mask based on the photo, extract the region to be colored based on the parsing mask, and perform normalization processing to obtain the target region. If no, skip this step.

[0031] A template converted to a first color space is obtained. Based on the depth of the template's background color and the characteristics of the line drawing, a coloring method is selected to color the template, the target image, and / or the target area. The template, the target image, and / or the target area in the first color space after coloring are converted back to a second color space. The first color space includes LAB space, HSV space, or HSI space, and the second color space includes RGB space. The coloring method includes solid color coloring, texture extraction coloring, and / or template adaptive coloring.

[0032] The template, the target image, and / or the target region in the second color space after coloring are fused together to obtain the coloring result.

[0033] To achieve the above objectives, this application also provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a machine, implements the steps of the method described above.

[0034] The embodiments of this application have the following advantages:

[0035] 1. This application provides an automatic coloring method for artistic line drawings, including the following steps: obtaining a line drawing generated from a user-inputted photo or a line drawing directly input by the user; preprocessing the line drawing to adapt it to the mask position of a template; converting the preprocessed line drawing into a grayscale image and performing normalization processing to obtain a target image; determining whether to perform region-based coloring based on user instructions; if so, generating a corresponding parsing mask based on the photo; extracting the region to be colored based on the parsing mask and performing normalization processing to obtain the target region; otherwise, skipping this step; obtaining a template converted to a first color space; and, based on the user's instructions... The template, target image, and / or target area are colored using a coloring method selected based on the depth of the template background color and the characteristics of the line drawing. The colored template, target image, and / or target area in the first color space are then converted back to a second color space. The first color space includes LAB, HSV, or HSI space, and the second color space includes RGB space. The coloring method includes solid color coloring, texture extraction coloring, and / or template adaptive coloring. The colored template, target image, and / or target area in the second color space are then blended to obtain the final coloring result.

[0036] By employing the methods described above and performing different linear processing calculations in the HSV space, users can freely change line textures or colors, blend them with templates in various texture forms, and achieve a more natural blend while preserving more line information. This satisfies both the visual display needs of online platforms and users' application requirements for line drawings on various personalized products. Attached Figure Description

[0037] To more clearly illustrate the embodiments of this application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0038] Figure 1 A flowchart illustrating an automatic coloring method for artistic line drawings provided in this application embodiment;

[0039] Figure 2 A schematic diagram illustrating the process of merging line drawings with templates in an automatic coloring method for artistic line drawings provided in this application embodiment;

[0040] Figure 3 A schematic diagram illustrating a pure white coloring example of an automatic coloring method for artistic line drawings provided in this application embodiment;

[0041] Figure 4 This is a schematic diagram illustrating a solid-color method for automatically coloring artistic line drawings, where the lines are rendered in black, as provided in an embodiment of this application.

[0042] Figure 5 A schematic diagram illustrating a texture extraction-based coloring example of an automatic coloring method for artistic line drawings provided in this application embodiment;

[0043] Figure 6 A schematic diagram illustrating an example of template adaptive coloring for an automatic coloring method for artistic line drawings provided in this application embodiment;

[0044] Figure 7 A schematic diagram illustrating a fusion-style coloring example of an automatic coloring method for artistic line drawings provided in this application embodiment;

[0045] Figure 8 This is a schematic diagram illustrating the target area coloring of an automatic coloring method for artistic line drawings provided in an embodiment of this application.

[0046] Figure 9 A block diagram of an automatic coloring device for artistic line drawings provided in this application embodiment. Detailed Implementation

[0047] The following specific embodiments illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0049] One embodiment of this application provides a method for automatically coloring artistic line drawings, see reference. Figure 1 , Figure 1 The flowchart provided in one embodiment of this application illustrates an automatic coloring method for artistic line drawings. It should be understood that the method may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of this application is not limited in this respect.

[0050] In step 101, the line drawing generated based on the user-input photo or the line drawing directly input by the user is obtained. The line drawing is preprocessed to adapt to the mask position of the template. The preprocessed line drawing is converted into a grayscale image and normalized to obtain the target image.

[0051] In some embodiments, the preprocessing method includes: trimming the excess white borders around the line drawing; and scaling the line drawing according to its aspect ratio after trimming, so that the line drawing can fit the mask position of the template.

[0052] Specifically, based on the line drawing generated from the input photo or the line drawing directly input by the user, the excess white borders around the line drawing are trimmed. The trimmed line drawing is then scaled according to its aspect ratio to fit the template's mask position. Finally, the line drawing is converted to grayscale and normalized. For the template, it needs to be converted from RGB color space to a primary color space, such as HSV color space.

[0053] Cropping data: Users can input photo I p Online line drawing generator I s Or users can directly upload line drawings. s At this point, the width P of the left, right, top, and bottom white borders can be obtained from the pixels of the line drawing. l ,P r ,P u ,P d To ensure the line drawing is centered, we process P...l =P r =MIN(P l ,P r According to the above parameters, process and trim I. p and I s The final pre-processed photo and line drawing are denoted as I. P and I S .

[0054] Scaling data: In this application, the template is denoted as T, and the mask corresponding to the template is denoted as T. m First, we need to determine T. m Middle mask area The aspect ratio, if Then I S I P Scale proportionally to height and Consistent. Otherwise, I will... S I P Scale proportionally to width and Consistent.

[0055] Convert to grayscale and normalize: Scale the I S Convert RGB to grayscale and normalize the values ​​to between 0 and 1, denoted as .

[0056] Template color space conversion: Convert the input template T from RGB to HSV color space T. hsv The three channels under HSV represent hue, saturation, and brightness, respectively denoted as . Hue represents the spectral hue of the entire image; saturation represents how close a color is to the spectral color, with higher values ​​indicating higher saturation and lower values ​​indicating a color closer to white; brightness represents the lightness or darkness of a color, with higher values ​​indicating brighter colors.

[0057] In step 102, based on the user's instruction, it is determined whether to perform regional colorization. If so, a corresponding parsing mask is generated according to the photo, and the region to be colored is extracted according to the parsing mask and normalized to obtain the target region. If not, this step is skipped.

[0058] Specifically, the following embodiments use a user-uploaded facial photo as an example for illustration. (Reference) Figure 2 Based on the user's instructions, it determines whether to perform regional coloring. If so, it generates a corresponding face parsing mask based on the face photo uploaded by the user, and then extracts the area to be colored, i.e. the target area, by fine-tuning the mask of each part of the face, for subsequent coloring.

[0059] In some embodiments, the method for generating the corresponding parsing mask based on the photo includes: cropping the photo, inputting the cropped photo into a corresponding image segmentation network, and obtaining the multi-channel parsing mask.

[0060] Specifically, the processed photo I p The image is input into the corresponding image segmentation network to obtain the corresponding multi-channel analytical mask P. In this application, a face photo is used as an example: the face photo is input into a finely tuned BisenetV2 model to obtain a 16-channel analytical mask P. To make the final fusion result more natural at the boundaries, post-processing of the skin and hair regions is required beforehand: a minimum filter with Kernal = 5×5 is used for the skin region P. skin Processing is performed; the hair region P is processed using the erosion operation in image processing. hair The data is processed, with Kernal = 3 × 3 and the number of iterations being 5. Finally, the face parsing data is normalized to the range of 0-1, denoted as... .

[0061] In step 103, a template converted to a first color space is obtained. Based on the depth of the template's background color and the characteristics of the line drawing, a coloring method is selected to color the template, the target image, and / or the target area. The template, the target image, and / or the target area in the first color space after coloring are converted back to a second color space. The first color space includes LAB space, HSV space, or HSI space, and the second color space includes RGB space. The coloring method includes solid color coloring, texture extraction coloring, and / or template adaptive coloring.

[0062] Specifically, there are four coloring methods: solid color coloring, texture extraction coloring, template adaptive coloring, and blending coloring. A suitable coloring method is selected based on the template's background color depth and the characteristics of the line drawings, and then the template and line drawings are colored. In the following embodiments, coloring processing in the HSV color space is used as an example for explanation. It should be understood that the coloring methods described in this application can also be applied to other second color spaces.

[0063] In some embodiments, the solid color coloring method includes:

[0064] When the lines of the target image or the target area are rendered black on the template, the following formula is used: Obtain the value of the luminance channel;

[0065] When the lines of the target image or the target area are rendered white on the template, the following formula is used: and Obtain the values ​​for the saturation channel and the brightness channel;

[0066] in, This refers to the value of the luminance channel obtained using the described solid color coloring method. The value of the luminance channel of the template in HSV color space, I gray This refers to the lines in the target image or the grayscale values ​​of the target region. This refers to the saturation channel value obtained using the described solid color coloring method. This refers to the value of the saturation channel of the template in the HSV color space.

[0067] Specifically, the following embodiments use the example of coloring lines in a target image for illustration. (Reference) Figure 3 and Figure 4 , Figure 3 The top left image is the bright red template; the bottom left image is the input line drawing (partial); the right image is the processing result: after the input line is black, it turns white, and then the gray lines gradually transition from light red to the bright red template color. Figure 4 The top left image shows the orange template; the bottom left image shows the input line drawing (partial); the right image shows the processing result: after coloring the black parts of the input lines, they appear black; then the gray lines appear light orange, and finally gradually transition to the orange template color. This method is suitable when the template and the line drawing are blended, and the line drawing's color appears black or white on the template. Using this method, a natural color transition effect can be achieved.

[0068] If the lines need to appear black on the template, we only need to perform linear calculations in the V space, as shown in the following formula: This method achieves the effect by changing the brightness of the lines in the template. The closer the gray value of a line is to 1, the closer its brightness is to the template; conversely, when the brightness is close to 0, it appears black. As the line drawing value changes from 0 to 1, assuming the template background is pure red, the effect will transition from black to light red and then to pure red.

[0069] If the lines need to appear white on the template, both the S and V spaces need to be processed simultaneously.

[0070]

[0071] In the V-space, the processing function means that the closer the line drawing is to a value of 1, the closer the saturation is to the template; the closer the line is to a value of 0, the closer the saturation is to 255, thus making the line appear white in the template; as the value ranges from 0 to 1, assuming the template is pure red, the final image will transition from white to light red and then to pure red. Finally, the HSV space... By merging the three channels, the final solid color coloring result is as follows:

[0072]

[0073] In some embodiments, the texture extraction-based coloring method includes:

[0074] Using the formula: and Obtain the values ​​of the saturation channel and the brightness channel.

[0075] in, The value of the luminance channel obtained using the texture extraction-based coloring method. This refers to the saturation channel value obtained using the texture extraction-based coloring method described above. The value of the saturation channel of the template in HSV color space, I gray The grayscale value refers to the lines in the target image or the grayscale value of the target region.

[0076] Specifically, refer to Figure 5 , Figure 5 The top left corner shows a dark red template, and the bottom left corner shows a partial view of the input line drawing. The image on the right shows the processing result: after coloring the black parts of the input line, it becomes dark red, then the gray lines become light red, and finally gradually transition to white. This method is suitable for assigning any color or texture to lines. The specific formula is as follows:

[0077]

[0078] in First, keep the hue of the H space unchanged; in order to give the lines a stencil texture, it means... The closer the brightness is to 0, the higher the saturation. To ensure the final brightness is adjusted according to... The grayscale value varies. In the S-space processing, the saturation of lines closer to 1 is close to 0, and when the saturation is 0, it is pure white. At this point, V-space processing is needed to ensure the line's brightness reaches its maximum of 255 when the value is close to 1. Finally, the HSV space is processed... By merging the three channels, the final solid color coloring result is as follows:

[0079]

[0080] In some embodiments, the template adaptive coloring method includes:

[0081] Using the formula: and Obtain the values ​​of the saturation channel, hue channel, and brightness channel.

[0082] Among them, 0 <c<1, This refers to the value of the tone channel obtained using the template adaptive coloring method. This refers to the value of the luminance channel obtained using the template adaptive coloring method. This refers to the saturation channel value obtained using the template adaptive coloring method. This refers to the value of the hue channel of the template in the HSV color space. This refers to the saturation channel value of the template in the HSV color space. The value of the luminance channel of the template in HSV color space, I gray The grayscale value refers to the lines in the target image or the grayscale value of the target region.

[0083] Specifically, refer to Figure 6 , Figure 6 The top left corner shows the template image, divided into two colors: red and yellow. The bottom left corner shows the input line drawing (partial view). The right image shows the processing result: in the upper half, where the lines are black, the line color is redder than red; in the lower half, where the lines are black, the line color is yellower than yellow. This method works when the template and line drawing are blended, with the line color adaptively darkening according to the template color. If the template contains red, blue, and green blocks, the user will get a deeper red in the red areas, a bluer in the blue areas, and a greener in the green areas. The specific formula is as follows:

[0084]

[0085]

[0086]

[0087] In the S-space line drawing, when the value approaches 1, the saturation of the template remains unchanged; conversely, it maximizes saturation. Correspondingly, when the line value is close to 0, c is set to 0.5, and the brightness is halved; while when the line value is close to 1, the brightness in the resulting image remains unchanged. This processing method achieves adaptive coloring during the blending of lines and the template. Finally, the HSV space... By merging the three channels, the final solid color coloring result is as follows:

[0088]

[0089] In some embodiments, after performing the texture extraction-based colorization process and obtaining the template, the target image, and / or the target region converted back to the second color space, the method further includes:

[0090] Coloring is performed using a blending coloring method, which includes:

[0091] Using the formula: Obtain the coloring result.

[0092] in, Refers to the coloring result in the RGB space, I color The term "target image" refers to the target image or target region in the RGB space after being colored using the texture extraction-based coloring method, and "T" refers to the module in the RGB space.

[0093] Specifically, refer to Figure 7 and Figure 8 , Figure 7 The top left image is a watercolor texture template; the middle left image is a dark red template; the bottom left image is a partial input line drawing; the right image shows the processing result: the black parts of the line drawing are processed into dark red, the gray lines are processed into light red, and finally transition to the watercolor paper texture template. Figure 8 The colors of bright red, brown, skin tone, and lip color are obtained through texture extraction; the colors of lines on the face, such as the nose and chin, are obtained through template adaptive coloring, resulting in a skin tone that is darker than the cheeks. This method aims to overcome the shortcomings of both "solid coloring" and "texture extraction." With solid coloring, lines can only appear black or white on the template; with texture extraction, lines only have texture, while the rest remains white. The blending coloring method allows for various textures and colors for foreground lines when blending with the template, and also allows for the replacement of various template types as background. The colored line drawing obtained through texture extraction is denoted as... The final result of the line template fusion can be obtained through the following calculation formula:

[0094]

[0095] The four methods mentioned above are not only applicable to lines, but also to the masks extracted in "region extraction". The same principle applies to coloring. In the above embodiments, the target image corresponding to the line drawing is used as an example for illustration. These are all parameters related to the target image corresponding to the line drawing. It should be understood that the parameters corresponding to the target area can be used to replace them. Both are consistent in usage because each layer in the mask is also a grayscale value between 0 and 1. The only difference is that the mask is regional. Finally, users can choose from the four coloring methods mentioned above according to their specific needs. In the end, the processed HSV space result needs to be converted back to RGB space.

[0096] In step 104, the template, the target image, and / or the target region in the second color space after coloring are merged to obtain the coloring result.

[0097] In some embodiments, the method for fusing the template, the target image, and / or the target region in the second color space after coloring includes: fusing the target regions after regional coloring to obtain a fusion result, and fusing the fusion result with the lines of the target image and the template after coloring to obtain the coloring result.

[0098] refer to Figure 2 Specifically, if the photo uploaded by the user is a facial photo, then the corresponding facial analysis is obtained through the "region extraction" step. Therefore, this fusion operation is to combine... The masks, after being colored at each layer, are merged to obtain the overall colored area of ​​the face. Finally, this result is blended with the lines to obtain the facial features, thus obtaining the final colored face portrait.

[0099] By employing the methods described above and performing different linear processing calculations in the HSV space, users can freely change line textures or colors, blend them with templates in various texture forms, and achieve a more natural blend while preserving more line information. This satisfies both the visual display needs of online platforms and users' application requirements for line drawings on various personalized products.

[0100] Figure 9 A module block diagram of an automatic coloring device for artistic line drawings provided in this application embodiment. The device includes:

[0101] A memory 201; and a processor 202 connected to the memory 201, the processor 202 being configured to: acquire a line drawing generated from a user-input photo or the line drawing directly input by the user; preprocess the line drawing to adapt it to the mask position of a template; convert the preprocessed line drawing into a grayscale image and perform normalization processing to obtain a target image; based on the user's instruction, determine whether to perform regional coloring; if so, generate a corresponding parsing mask based on the photo; extract the area to be colored based on the parsing mask and perform normalization processing to obtain the target area; if not, skip this step; acquire the image converted to a first color space. A template is used to color the template, the target image, and / or the target area based on the depth of the template's background color and the characteristics of the line drawing. The colored template, target image, and / or target area in the first color space are then converted back to a second color space. The first color space includes LAB, HSV, or HSI space, and the second color space includes RGB space. The coloring methods include solid color coloring, texture extraction coloring, and / or template adaptive coloring. The colored template, target image, and / or target area in the second color space are then blended to obtain the final coloring result.

[0102] In some embodiments, the processor 202 is further configured such that the solid color coloring method includes:

[0103] When the lines of the target image or the target area are rendered black on the template, the following formula is used: Obtain the value of the luminance channel;

[0104] When the lines of the target image or the target area are rendered white on the template, the following formula is used: and Obtain the values ​​for the saturation channel and the brightness channel;

[0105] in, This refers to the value of the luminance channel obtained using the described solid color coloring method. The value of the luminance channel of the template in HSV color space, I gray This refers to the lines in the target image or the grayscale values ​​of the target region. This refers to the saturation channel value obtained using the described solid color coloring method. This refers to the value of the saturation channel of the template in the HSV color space.

[0106] In some embodiments, the processor 202 is further configured such that the texture extraction-based coloring method includes:

[0107] Using the formula: and Obtain the values ​​of the saturation channel and the brightness channel.

[0108] in, The value of the luminance channel obtained using the texture extraction-based coloring method. This refers to the saturation channel value obtained using the texture extraction-based coloring method described above. The value of the saturation channel of the template in HSV color space, I gray The grayscale value refers to the lines in the target image or the grayscale value of the target region.

[0109] In some embodiments, the processor 202 is further configured such that the template adaptive coloring method includes:

[0110] Using the formula: and Obtain the values ​​of the saturation channel, hue channel, and brightness channel.

[0111] Among them, 0 <c<1, This refers to the value of the tone channel obtained using the template adaptive coloring method. This refers to the value of the luminance channel obtained using the template adaptive coloring method. This refers to the saturation channel value obtained using the template adaptive coloring method. This refers to the value of the hue channel of the template in the HSV color space. This refers to the saturation channel value of the template in the HSV color space. The value of the luminance channel of the template in HSV color space, I gray The grayscale value refers to the lines in the target image or the grayscale value of the target region.

[0112] In some embodiments, the processor 202 is further configured to: after performing the texture extraction colorization process and obtaining the template, the target image, and / or the target region converted back to the second color space, further include:

[0113] Coloring is performed using a blending coloring method, which includes:

[0114] Using the formula: Obtain the coloring result.

[0115] in, Refers to the coloring result in the RGB space, I color The term "target image" refers to the target image or target region in the RGB space after being colored using the texture extraction-based coloring method, and "T" refers to the module in the RGB space.

[0116] In some embodiments, the processor 202 is further configured such that the preprocessing method includes:

[0117] Trim the excess white borders around the line drawing. Scale the line drawing according to its aspect ratio after trimming, so that it fits the mask position of the template.

[0118] In some embodiments, the processor 202 is further configured to: generate the corresponding parsing mask based on the photograph, including:

[0119] The photo is cropped, and the cropped photo is input into the corresponding image segmentation network to obtain the multi-channel parsing mask.

[0120] In some embodiments, the processor 202 is further configured to: fuse the template, the target image, and / or the target region of the colored second color space, including:

[0121] The target regions after being colored in different areas are merged to obtain a fusion result. The fusion result is then merged with the lines of the colored target image and the template to obtain the coloring result.

[0122] For specific implementation methods, please refer to the aforementioned method embodiments, which will not be repeated here.

[0123] This application may be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this application.

[0124] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0125] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0126] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0127] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0128] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0129] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0131] Note that, unless otherwise explicitly stated, all features disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by alternative features for achieving the same, equivalent, or similar purpose. Therefore, unless explicitly stated otherwise, each disclosed feature is merely one example of a set of equivalent or similar features. Where used, "further," "preferably," "even further," and "more preferably" are simple starting points for describing another embodiment based on the foregoing embodiments, the combination of which with the foregoing embodiments constitutes the complete configuration of another embodiment. Any combination of several "further," "preferably," "even further," or "more preferably" settings following the same embodiment constitutes yet another embodiment.

[0132] Although this application has been described in detail above with general descriptions and specific embodiments, some modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of this application fall within the scope of protection claimed in this application.

Claims

1. An automatic coloring method for artistic line drawings, characterized in that, Includes the following steps: Obtain a line drawing generated based on a photo input by the user or a line drawing directly input by the user; preprocess the line drawing to adapt it to the mask position of the template; convert the preprocessed line drawing into a grayscale image and perform normalization processing to obtain the target image. Based on the user's instructions, it is determined whether to perform regional coloring. If so, a corresponding parsing mask is generated based on the photo, and the region to be colored is extracted based on the parsing mask and normalized to obtain the target region. A template converted to HSV color space is obtained. Based on the depth of the template's background color and the characteristics of the line drawing, a coloring method is selected to color the template, the target image, and / or the target area. The colorized template, target image, and / or target area in HSV color space are then converted back to RGB color space. The coloring methods include solid coloring, texture extraction coloring, and template adaptive coloring. After performing texture extraction coloring and obtaining the template, target image, and / or target area converted back to the second color space, a blending coloring method is also included. This blending coloring method utilizes the formula: The coloring result is obtained, among which, The coloring result in RGB space. The target image or target region in the RGB space after being colored using the texture extraction-based coloring method; The template in the RGB space; The colored template, the target image, and / or the target area are merged to obtain a colored result, including merging the colored target areas to obtain a fusion result, and merging the fusion result with the lines of the colored target image and the template to obtain the colored result; The solid color coloring method includes: When the lines of the target image or the target area are rendered black on the template, the following formula is used: Obtain the value of the luminance channel; When the lines of the target image or the target area are rendered white on the template, the following formula is used: and Obtain the values ​​for the saturation channel and the brightness channel; in, This refers to the value of the luminance channel obtained using the described solid color coloring method. This refers to the value of the luminance channel of the template in the HSV color space. This refers to the lines in the target image or the grayscale values ​​of the target region. This refers to the saturation channel value obtained using the described solid color coloring method. The value of the saturation channel of the template in HSV color space; The texture extraction-based coloring method includes: Using the formula: and Obtain the values ​​of the saturation channel and the brightness channel. in, , , The value of the luminance channel obtained using the texture extraction-based coloring method. This refers to the saturation channel value obtained using the texture extraction-based coloring method described above. This refers to the saturation channel value of the template in the HSV color space. The grayscale value refers to the lines in the target image or the grayscale value of the target region. The template adaptive coloring method includes: Using the formula: , and Obtain the values ​​of the saturation channel, hue channel, and brightness channel. Among them, 0 <c<1, This refers to the value of the tone channel obtained using the template adaptive coloring method. This refers to the value of the luminance channel obtained using the template adaptive coloring method. This refers to the saturation channel value obtained using the template adaptive coloring method. This refers to the value of the hue channel of the template in the HSV color space. This refers to the saturation channel value of the template in the HSV color space. This refers to the value of the luminance channel of the template in the HSV color space. The grayscale value refers to the lines in the target image or the grayscale value of the target region.

2. The automatic coloring method for artistic line drawings according to claim 1, characterized in that, The preprocessing method includes: Trim the excess white borders around the line drawing, and scale the line drawing according to its aspect ratio to fit the mask position of the template.

3. The automatic coloring method for artistic line drawings according to claim 1, characterized in that, The method for generating the corresponding parsing mask based on the photo includes: The photo is cropped, and the cropped photo is input into the corresponding image segmentation network to obtain the multi-channel parsing mask.

4. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a machine, it implements the steps of the method as described in any one of claims 1 to 3.

Images