Oil painting style image generation method and device, electronic equipment and storage medium

Abstract

The application provides an oil painting style image generation method and device, electronic equipment and storage medium, and relates to the technical field of image processing. The method comprises the following steps: according to a to-be-processed image, a preset multi-layer importance threshold is used to process a multi-layer blank canvas image, and an importance map of the to-be-processed image is obtained; according to a color parameter of the to-be-processed image, the to-be-processed image is color remapped, and a color-remapped to-be-processed image is obtained; according to a plurality of brushes in a preset brush set and importance values of each pixel point, brush rendering parameters of a plurality of drop points in the color-remapped to-be-processed image are determined; and the brush rendering parameters of the plurality of drop points are used to render the plurality of drop points in the color-remapped to-be-processed image, and an oil painting style image corresponding to the to-be-processed image is obtained. The application can improve the fine effect of the generated oil painting style image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an oil painting style image generation method and device, electronic equipment and storage medium. BACKGROUND

[0002] Due to the threshold of painting technology for ordinary people, people need to convert their own images into oil painting images with the help of computer technology.

[0003] The existing oil painting style image generation method usually adopts the way of drawing stroke by stroke to simulate the habit of the painter to generate oil painting images, but the existing step-by-step drawing method does not distinguish the importance of different drop points in the original image, so that the effect of the oil painting image is very rough, or the drop point selection method is too complex, so that the speed of generating oil painting images is very slow. SUMMARY

[0004] The present application aims to overcome the deficiencies in the prior art, and provides an oil painting style image generation method, device, electronic equipment and storage medium, so as to realize efficient image style conversion and generate detailed oil painting images.

[0005] To achieve the above purpose, the technical scheme adopted by the embodiments of the present application is as follows:

[0006] In a first aspect, the embodiments of the present application provide an oil painting style image generation method, which comprises:

[0007] According to the to-be-processed image, a plurality of blank canvas images are processed by using a plurality of preset importance thresholds to obtain an importance map of the to-be-processed image, wherein the importance value of each pixel point in the importance map is used to indicate the importance of each pixel point in the to-be-processed image.

[0008] According to the color parameters of the to-be-processed image, the to-be-processed image is color remapped to obtain a color-remapped to-be-processed image.

[0009] According to the plurality of brushes in the preset brush set and the importance values of the pixel points, the brush rendering parameters of a plurality of drop points in the color-remapped to-be-processed image are determined.

[0010] The brush rendering parameters of the plurality of drop points are used to render the plurality of drop points in the color-remapped to-be-processed image to obtain an oil painting style image corresponding to the to-be-processed image.

[0011] Optionally, according to the to-be-processed image, a plurality of blank canvas images are processed by using a plurality of preset importance thresholds to obtain an importance map of the to-be-processed image, comprising:

[0012] According to the multi-layer importance threshold, a first target region in the to-be-processed image is drawn into the multi-layer blank canvas image respectively, to obtain a multi-layer region drawing image;

[0013] According to the mask of the second target region in the to-be-processed image, the mask of the third target region in the to-be-processed image and the multi-layer region drawing image, an importance map of the to-be-processed image is obtained, wherein the first target region, the second target region and the third target region are three associated regions of the same type of object in the to-be-processed image.

[0014] Optionally, the obtaining of the importance map of the to-be-processed image according to the mask of the second target region in the to-be-processed image, the mask of the third target region in the to-be-processed image and the multi-layer region drawing image comprises:

[0015] The mask of the second target region, the mask of the third target region and the multi-layer region drawing image are multiplied by corresponding importance thresholds respectively;

[0016] According to the maximum pixel value of each pixel point after multiplication, the importance map of the to-be-processed image is generated.

[0017] Optionally, the color remapping of the to-be-processed image according to the color parameter of the to-be-processed image comprises:

[0018] According to a first color of the to-be-processed image in a first color space, a preset color space conversion relationship is used to obtain a first color parameter of the first color in a second color space;

[0019] According to a hue component and a brightness component in the first color parameter and a preset random number, a preset three-dimensional color lookup table is used to determine a second color;

[0020] According to the second color, the color space conversion relationship is used to obtain a second color parameter of the second color in the second color space;

[0021] The saturation component in the second color parameter is replaced by the saturation component in the first color parameter to obtain a third color parameter;

[0022] The third color parameter is converted into a third color in the first color space by using the color space conversion relationship;

[0023] The first color and the third color are mixed on the to-be-processed image to obtain the color-remapped to-be-processed image.

[0024] Optionally, the method further comprises:

[0025] According to the importance value of each pixel point and the pixel point threshold corresponding to each brush, a brush rendering parameter of each brush is determined.

[0026] The preset pattern corresponding to each brush is determined as the brush pattern of the at least one drop point corresponding to each brush.

[0027] According to the color of each pixel point in the color remapped image, a drop center color of each drop point is determined.

[0028] According to the gradient of each pixel point in the image in two mutually perpendicular directions, a brush rotation angle of each drop point is calculated, and the brush rendering parameter of each drop point comprises the brush pattern, the drop center color and the brush rotation angle of each drop point.

[0029] Optionally, before the step of calculating the brush rotation angle of each drop point according to the gradient of each pixel point in the image in two mutually perpendicular directions, the method further comprises:

[0030] An edge image of the image is obtained.

[0031] After rotating the blank image with a plurality of white points having random positions, the blank image is weighted and mixed with the edge image to obtain a white point edge image.

[0032] The distance between each pixel point in the white point edge image and the nearest preset position point is calculated to obtain a distance image, wherein the value of each pixel point in the distance image is the distance between the corresponding pixel point in the white point edge image and the nearest preset position point, and the preset position point is an edge point or a white point.

[0033] The first initial gradient of each pixel point in the distance image in the two mutually perpendicular directions and the first initial gradient amplitude between the two mutually perpendicular directions are calculated.

[0034] The second initial gradient of each pixel point in the corresponding gray image of the image in the two mutually perpendicular directions and the second initial gradient amplitude between the two mutually perpendicular directions are calculated.

[0035] The first initial gradient amplitude and the second initial gradient amplitude are weighted and mixed to obtain a target gradient amplitude.

[0036] calculate an angle difference between the pixel point and the plurality of regional pixel points according to an initial direction gradient of the plurality of regional pixel points in a preset region range centered on the pixel point in one direction and an initial gradient of the pixel point;

[0037] calculate a gradient amplitude difference between the pixel point and the plurality of regional pixel points according to a target gradient amplitude of the plurality of regional pixel points and a target gradient amplitude of the pixel point;

[0038] respectively calculate gradients of the pixel point in the two mutually perpendicular directions according to the angle difference and the gradient amplitude difference in the corresponding direction.

[0039] Optionally, after the brush rendering parameters of the plurality of pen drop points are used to perform pointillism rendering on the plurality of pen drop points in the color remapped to-be-processed image, to obtain the oil painting style image corresponding to the to-be-processed image, the method further comprises:

[0040] respectively process the to-be-processed image according to preset face brightness enhancement parameters and preset portrait brightness enhancement parameters, to generate a face edge feature importance map and a portrait edge feature importance map;

[0041] generate an edge importance image according to the face region mask, the portrait region mask, the face edge feature importance map and the portrait edge feature importance map corresponding to the to-be-processed image;

[0042] adjust a saturation component and a brightness component of the to-be-processed image, to obtain an adjusted to-be-processed image;

[0043] mix the edge importance image, the adjusted to-be-processed image and the oil painting style image, to obtain a target oil painting style image.

[0044] In a second aspect, the embodiments of the present application further provide an oil painting style image generation device, the device comprising:

[0045] an importance map determination module configured to process a plurality of blank canvas images according to a to-be-processed image by using preset multi-layer importance thresholds, to obtain an importance map of the to-be-processed image, wherein an importance value of each pixel point in the importance map is used to indicate an importance degree of each pixel point in the to-be-processed image;

[0046] a color mapping module configured to perform color remapping on the to-be-processed image according to a color parameter of the to-be-processed image, to obtain a color remapped to-be-processed image;

[0047] The rendering parameter determination module is configured to determine brush rendering parameters of the plurality of landing points in the color-remapped to-be-processed image according to a plurality of brushes in a preset brush set and the importance values of the pixel points.

[0048] The rendering module is configured to render the plurality of landing points in the color-remapped to-be-processed image by using the brush rendering parameters of the plurality of landing points, to obtain an oil painting style image corresponding to the to-be-processed image.

[0049] Optionally, the importance map determination module comprises:

[0050] The target region drawing unit is configured to draw a first target region in the to-be-processed image into the plurality of blank canvas images respectively according to the plurality of importance thresholds, to obtain a plurality of region-drawn images.

[0051] The importance map determination unit is configured to obtain an importance map of the to-be-processed image according to a mask of a second target region in the to-be-processed image, a mask of a third target region in the to-be-processed image, and the plurality of region-drawn images, wherein the first target region, the second target region, and the third target region are three associated regions of a same type of object in the to-be-processed image.

[0052] Optionally, the importance map determination unit comprises:

[0053] The calculation subunit is configured to multiply the mask of the second target region, the mask of the third target region, and the plurality of region-drawn images respectively by corresponding importance thresholds.

[0054] The importance map determination subunit is configured to generate the importance map of the to-be-processed image according to maximum pixel values of each pixel point after multiplication.

[0055] Optionally, the color mapping module comprises:

[0056] The first color parameter calculation unit is configured to obtain first color parameters of a second color space of a first color of the to-be-processed image in a first color space by using a preset color space conversion relationship according to the first color.

[0057] The second color calculation unit is configured to determine the second color by using a preset three-dimensional color lookup table according to a hue component, a brightness component, and a preset random number in the first color parameters.

[0058] The second color parameter calculation unit is configured to obtain second color parameters of the second color in the second color space by using the color space conversion relationship according to the second color.

[0059] a third color parameter calculation unit, configured to replace a saturation component in the second color parameter with a saturation component in the first color parameter to obtain a third color parameter;

[0060] a third color calculation unit, configured to convert the third color parameter into a third color in the first color space by using the color space conversion relationship;

[0061] a color mixing unit, configured to mix the first color and the third color on the image to be processed to obtain the image after color remapping.

[0062] Optionally, the rendering parameter determination module comprises:

[0063] a pen drop point determination unit, configured to determine, from the image after color remapping, a pen drop set corresponding to each brush according to a pixel point threshold corresponding to each brush and the importance value of each pixel point, the pen drop set comprising at least one pen drop point corresponding to each brush;

[0064] a brush pattern determination unit, configured to determine a preset pattern corresponding to each brush as a brush pattern of the pen drop point corresponding to each brush;

[0065] a color confirmation unit, configured to determine a pen drop center color of each pen drop point according to the color of each pixel point in the image after color remapping;

[0066] an angle confirmation unit, configured to calculate a brush rotation angle of each pen drop point according to the gradient of each pixel point in the image in two mutually perpendicular directions, the brush rendering parameter of each pen drop point comprising the brush pattern, the pen drop center color and the brush rotation angle of each pen drop point.

[0067] Optionally, before the angle confirmation unit, the device further comprises:

[0068] an edge image acquisition unit, configured to acquire an edge image of the image to be processed;

[0069] a white point mixing unit, configured to perform weighted mixing on the rotated blank image with a plurality of white points having random positions and the edge image to obtain a white point edge image;

[0070] a distance calculation unit, configured to calculate the distance between each pixel point in the white point edge image and the nearest preset position point to obtain a distance image, the value of each pixel point in the distance image being the distance between the corresponding pixel point in the white point edge image and the nearest preset position point, the preset position point being an edge point or a white point;

[0071] a first initial gradient calculation unit, configured to calculate a first initial gradient of each pixel in the distance image in two mutually perpendicular directions and a first initial gradient amplitude between the two mutually perpendicular directions;

[0072] a second initial gradient calculation unit, configured to calculate a second initial gradient of each pixel in a gray-scale image corresponding to the to-be-processed image in the two mutually perpendicular directions and a second initial gradient amplitude between the two mutually perpendicular directions;

[0073] an amplitude calculation unit, configured to perform weighted mixing on the first initial gradient amplitude and the second initial gradient amplitude to obtain a target gradient amplitude;

[0074] an angle difference calculation unit, configured to calculate an angle difference between the pixel and a plurality of regional pixels in a preset region range centered on the pixel in one direction according to initial direction gradients of the plurality of regional pixels and an initial gradient of the pixel;

[0075] an amplitude difference calculation unit, configured to calculate a gradient amplitude difference between the pixel and the plurality of regional pixels according to target gradient amplitudes of the plurality of regional pixels and a target gradient amplitude of the pixel;

[0076] a gradient calculation unit, configured to calculate gradients of the pixel in the two mutually perpendicular directions respectively according to the angle difference between the two mutually perpendicular directions and the gradient amplitude difference in the corresponding direction.

[0077] Optionally, after the rendering module, the apparatus further includes:

[0078] an edge brightness enhancement module, configured to respectively process the to-be-processed image according to preset face brightness enhancement parameters and preset portrait brightness enhancement parameters to generate a face edge feature importance map and a portrait edge feature importance map;

[0079] an edge importance map generation module, configured to generate an edge importance image according to a face region mask, a portrait region mask, the face edge feature importance map and the portrait edge feature importance map corresponding to the to-be-processed image;

[0080] a parameter adjustment module, configured to adjust a saturation component and a brightness component of the to-be-processed image to obtain an adjusted to-be-processed image;

[0081] an image mixing module, configured to mix the edge importance image, the adjusted to-be-processed image and the oil painting style image to obtain a target oil painting style image.

[0082] In a third aspect, the embodiments of the present application further provide an electronic device, comprising a processor, a storage medium and a bus, the storage medium stores program instructions executable by the processor, when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to perform the steps of the oil painting style image generation method according to any one of the above embodiments.

[0083] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, the storage medium stores a computer program, and the computer program performs the steps of the oil painting style image generation method according to any one of the above embodiments when executed by a processor.

[0084] The beneficial effects of the present application are as follows:

[0085] The present application provides an oil painting style image generation method, device, electronic device and storage medium. According to a to-be-processed image, a preset multi-layer importance threshold is used to process a multi-layer blank canvas image to obtain an importance map of the to-be-processed image. The importance value of each pixel point in the importance map is used to indicate the importance of each pixel point in the to-be-processed image. According to the color parameter of the to-be-processed image, the to-be-processed image is color remapped to obtain a color-remapped to-be-processed image. According to the plurality of brushes in the preset brush set and the importance value of each pixel point, the brush rendering parameters of a plurality of pen drop points in the color-remapped to-be-processed image are determined. The brush rendering parameters of the plurality of pen drop points are used to render the plurality of pen drop points in the color-remapped to-be-processed image to obtain an oil painting style image corresponding to the to-be-processed image. The present application processes the multi-layer blank canvas image by using the multi-layer importance threshold, so that the importance map of the to-be-processed image obtained has rich levels. The brush rendering parameters of the pen drop points determined according to the importance map are used to render the oil painting style image, so that the details of the oil painting style image have a more layered effect, the fine effect of the generated oil painting style image is improved, and the oil painting style image is efficiently generated. BRIEF DESCRIPTION OF DRAWINGS

[0086] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0087] Figure 1 A flowchart of an oil painting style image generation method provided by an embodiment of the present application is shown in the figure.

[0088] Figure 2 A flowchart of another oil painting style image generation method provided by an embodiment of the present application is shown in the figure.

[0089] Figure 3 A flowchart of still another oil painting style image generation method provided by an embodiment of the present application is shown in FIG. 6;

[0090] Figure 4 A flowchart of still another oil painting style image generation method provided by an embodiment of the present application is shown in FIG. 6;

[0091] Figure 5 A flowchart of still another oil painting style image generation method provided by an embodiment of the present application is shown in FIG. 6;

[0092] Figure 6 A flowchart of still another oil painting style image generation method provided by an embodiment of the present application is shown in FIG. 6;

[0093] Figure 7 A flowchart of still another oil painting style image generation method provided by an embodiment of the present application is shown in FIG. 6;

[0094] Figure 8 A flowchart of still another oil painting style image generation method provided by an embodiment of the present application is shown in FIG. 6; DETAILED DESCRIPTION

[0095] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application.

[0096] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0097] In the description of the present application, it should be noted that if the terms "upper", "lower", etc. indicate the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the product of the present application is usually placed, only for the convenience of describing the present application and simplifying the description, and it is not intended to indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0098] In addition, the terms "first", "second", and the like in the description and in the claims of the present application and the above-described drawings are used to distinguish similar objects, and are not necessarily used to describe a particular sequential or chronological order. It should be understood that the data thus used can be interchanged under appropriate circumstances so that the embodiments of the application described herein can be implemented in other than the order illustrated or described herein. Furthermore, the terms "comprise" and "have", as well as any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a list of steps or units as non-limiting to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to such processes, methods, products, or apparatuses.

[0099] It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0100] The oil painting style image generation method provided by the embodiments of the present application is applied to image processing software installed in an electronic device. After a to-be-processed image is imported into the image processing software or the to-be-processed image is obtained by directly shooting in the image processing software, the oil painting style image generation method is executed to convert the to-be-processed image into an oil painting style image. In the conversion process, the to-be-processed image can be directly converted in style, and the to-be-processed image is replaced by the oil painting style image after conversion, and only the oil painting style image is saved in the electronic device. Alternatively, the to-be-processed image can be copied, the copied to-be-processed image is converted in style, and the converted to-be-processed image and the oil painting style image are both saved in the electronic device.

[0101] Please refer to Figure 1 A flowchart of an oil painting style image generation method provided by the embodiments of the present application is shown in FIG. 1. Figure 1 The method comprises the following steps.

[0102] S10: According to the to-be-processed image, a preset multi-layer importance threshold is used to process a multi-layer blank canvas image to obtain an importance map of the to-be-processed image.

[0103] In this embodiment, since the details contained in local images of different regions in the image to be processed may vary, multiple layers of blank canvas images are pre-set to reflect the level of detail in different local images. These multiple canvas images have different importance thresholds and brush radii. The brush radius of the bottom blank image is larger than that of the top blank image, and the brush radii of the multiple blank canvas images decrease arithmetically. The smaller the local image, the fewer details it contains, and the lower its importance. The size of the local image is compared with the importance threshold of each layer. If the size of the local image is smaller than the importance threshold of the corresponding layer, the local image is drawn on the blank image of the corresponding layer using the corresponding brush radius. The pixel values ​​of the drawn multiple blank canvas images are calculated to determine the importance map. The importance value of each pixel in the importance map indicates the importance of each pixel in the image to be processed. The higher the importance, the more image details the pixel contains; conversely, the lower the importance, the fewer image details the pixel contains.

[0104] In one alternative embodiment, before performing style conversion on the image to be processed, the image to be processed may be scaled to improve the execution efficiency of the oil painting style image generation method.

[0105] In this embodiment, the image to be processed I in The width is denoted as W. in The height is denoted as H. in Let the processing width be denoted as W, the processing height as H, and the maximum side length as E. limit Let the width be denoted as W and the height as H. For example, it could be:

[0106]

[0107]

[0108] If the processing width W and processing height H are related to the image to be processed I in Width W in and height H in If they are the same, then the size of the image to be processed remains unchanged. If the image to be processed I... in Width W in The image I to be processed is larger than the processing width W. in Height H in If the image height is greater than the processing height H, then the image to be processed will be scaled down to the processing height and width. For example, area-based interpolation can be used for scaling. If the above conditions are not met, bicubic interpolation can be used to scale the image to the processing height and width.

[0109] S20: performing color remapping on the to-be-processed image according to the color parameter of the to-be-processed image, to obtain a color-remapped to-be-processed image.

[0110] In this embodiment, since the oil painting has experienced time precipitation, the overall color of the oil painting is dark. In order to make the color of the generated oil painting style image closer to the oil painting, a predefined corresponding relationship between the image color and the oil painting color can be used to determine the color parameter of the corresponding oil painting color according to the color parameter of each pixel point in the to-be-processed image, and the to-be-processed image is color-remapped according to the color parameter of the oil painting color, to obtain a color-remapped to-be-processed image. The color of the color-remapped to-be-processed image is more rich.

[0111] S30: determining the brush rendering parameters of the multiple drop points in the color-remapped to-be-processed image according to the multiple brushes in the preset brush set and the importance values of the pixel points.

[0112] In this embodiment, the preset brush set includes multiple brushes, and the multiple brushes have different brush parameters. The brush parameters can at least include a brush pattern, a brush radius, and a brush threshold. The importance value of each pixel point can be determined according to the brush threshold and the importance value of each pixel point, to determine whether the importance value of each pixel point is less than the brush threshold of the multiple first brushes. If so, it is determined that the brush can drop on the pixel point, that is, it is determined that the pixel point is a drop point of the brush, to obtain a drop point set of each brush. The brush rendering parameters of the multiple drop points can at least include the brush pattern and the brush radius of the corresponding brush.

[0113] S40: rendering the multiple drop points in the color-remapped to-be-processed image by using the brush rendering parameters of the multiple drop points, to obtain an oil painting style image corresponding to the to-be-processed image.

[0114] In this embodiment, the brush rendering parameters of the multiple drop points are saved in a brush rendering queue Q render , and the brush rendering parameters of the multiple drop points in the brush rendering queue Q render are rendered on the to-be-processed image in sequence by using a preset graphics processing framework, to obtain an oil painting style image. For example, the preset graphics processing framework can be OpenGL (Open Graphics Library), Metal, or Vulkan. The style of the oil painting style image can be determined by the brush parameters, and the styles of the oil painting style images obtained by different brush parameters are different.

[0115] The oil painting style image generation method provided in the embodiments of the present application comprises the following steps: processing a plurality of blank canvas images according to a preset plurality of importance thresholds, to obtain an importance map of the to-be-processed image, wherein the importance value of each pixel point in the importance map is used to indicate the importance of each pixel point in the to-be-processed image; performing color remapping on the to-be-processed image according to the color parameters of the to-be-processed image, to obtain a color-remapped to-be-processed image; determining the brush rendering parameters of a plurality of drop points in the color-remapped to-be-processed image according to a plurality of brushes in a preset brush set and the importance values of the pixel points; and performing rendering on the plurality of drop points in the color-remapped to-be-processed image according to the brush rendering parameters of the plurality of drop points, to obtain an oil painting style image corresponding to the to-be-processed image. The plurality of blank canvas images are processed according to the plurality of importance thresholds, so that the importance map of the to-be-processed image obtained has rich levels, and the brush rendering parameters of the drop points determined according to the importance map have levels, so that the details of the oil painting style image obtained by rendering the brush rendering parameters have levels, the fine effect of the generated oil painting style image is improved, and the oil painting style image is generated efficiently.

[0116] On the basis of the above-mentioned embodiments, another oil painting style image generation method is further provided in the embodiments of the present application. Please refer to Figure 2 The flowchart of another oil painting style image generation method provided in the embodiments of the present application is shown in Figure 2 The above-mentioned S10 comprises:

[0117] S11: drawing a first target region in the to-be-processed image into a plurality of blank canvas images according to a plurality of importance thresholds, to obtain a plurality of region drawing images.

[0118] In the embodiment, the first target region is a key region in the to-be-processed image defined in advance in the process of generating the oil painting style image, and the first target region is the region with the highest importance in the to-be-processed image. The importance of the regions other than the first target region can be completely the same. The first target region comprises a plurality of target objects, and the importance of the plurality of target objects can be different. The size of the plurality of target objects and the plurality of importance thresholds are compared. If the size of a target object is smaller than the importance threshold of the corresponding layer, the image of the target object is drawn on the blank canvas image of the corresponding layer by using the corresponding brush radius. According to the size of the first target region, the blank canvas image corresponding to the first target region is determined, and the mask of the first target region is also drawn on the blank canvas image of the corresponding layer, to obtain the plurality of region drawing images.

[0119] For example, the plurality of blank canvas images comprise I1, I2, …, In. L-1 ,I L, all pixel values of all pixel points of the multi-layer blank canvas image are 0, if the to-be-processed image includes at least one portrait, the first target region can be at least one portrait region, a pre-trained deep learning method is used to perform deep recognition on the to-be-processed image to identify and output face points corresponding to each portrait in the to-be-processed image and the face mask The face points and the face mask are merged and stored in the face set , wherein N is used to represent the number of portraits in the to-be-processed image.

[0120] For each face in the face set S faces , the five features of each face can be a plurality of sub-regions corresponding to the first target region, the size of each feature of each face is calculated according to the face points of each face, and each feature corresponding to the blank canvas image is determined according to the size of each feature and the multi-layer importance threshold, and each feature is drawn on the blank canvas image of the corresponding layer according to the brush radius of the blank canvas image of the corresponding layer; the face frame size of each face is calculated according to the face points of each face, and the face mask of each face is drawn on the blank canvas image of the corresponding layer according to the corresponding brush radius, to obtain a multi-layer region drawing image I1, I2, …, I L-1 , I L . In an example, when generating an oil painting style image, it is found that the left eye, the left eyeball, the right eye, the right eyeball and the lips are more important than the eyebrows, the nose, the ears and other features, therefore, the calculation of the feature size can only calculate the size of the left eye, the right eye, the left eyeball and the right eyeball.

[0121] S12: obtaining an importance map of the to-be-processed image according to the mask of the second target region in the to-be-processed image, the mask of the third target region in the to-be-processed image and the multi-layer region drawing image.

[0122] In this embodiment, the first target region, the second target region and the third target region are three associated regions of the same type of object in the to-be-processed image, and the first target region and the third target region can be sub-regions of the second target region. A pre-trained mask deep learning method is used to recognize the to-be-processed image to identify and output the mask of the second target region and the mask of the third target region in the to-be-processed image, and the importance value of each pixel point is determined according to the pixel value of each pixel point in the mask of the second target region in the to-be-processed image, the mask of the third target region in the to-be-processed image and the multi-layer region drawing image, to obtain an importance map of the to-be-processed image.

[0123] In an alternative embodiment, according to the mask of the second target region in the image to be processed, the mask of the third target region in the image to be processed, and the multi-layer region drawing image, an importance map of the image to be processed is drawn, comprising:

[0124] The mask of the second target region, the mask of the third target region, and the multi-layer region drawing image are multiplied by the corresponding importance threshold value respectively; and according to the maximum pixel value of each pixel point after multiplication, an importance map of the image to be processed is generated.

[0125] In this embodiment, the mask of the second target region is multiplied by the importance threshold value of the second target region, the mask of the third target region is multiplied by the importance threshold value of the third target region, and the multi-layer region drawing image is multiplied by the multi-layer importance threshold value respectively, to obtain a plurality of pixel images, and the maximum pixel value of each pixel point in the plurality of pixel images is taken to generate an importance map of the image to be processed.

[0126] For example, it is assumed that the mask of the second target region is a portrait region mask M body , the importance threshold value of the portrait region mask M body is T body , the mask of the third target region is a skin region mask M skin , the importance threshold value of the skin region mask M skin is T skin , and the multi-layer region drawing image I1, I2, …, I L-1 , I L corresponds to the multi-layer importance threshold value T1, T2, …, T L-1 , T L respectively, then the formula for calculating the importance map can be:

[0127] I importance = max(M body ·T body , M skin ·T skin , I1·T1, I2·T2, …, I L-1 ·T L-1 , I L ·T L )

[0128] The oil painting style image generation method provided in the embodiments of the present application draws the first target region in the to-be-processed image into a plurality of blank canvas images according to a plurality of importance thresholds, to obtain a plurality of region drawing images, and obtains an importance map of the to-be-processed image according to a mask of a second target region in the to-be-processed image, a mask of a third target region in the to-be-processed image, and the plurality of region drawing images. The embodiments of the present application draw the first target region into blank canvas images of different layers, and generate the importance map according to the mask of the second target region, the mask of the third target region, and the plurality of region drawing images, so that the importance map can represent different details of different regions of the to-be-processed image, so as to make the details of the generated oil painting style image more hierarchical, and improve the effect of the generated oil painting style image.

[0129] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide another oil painting style image generation method. Please refer to Figure 3 The flowchart of another oil painting style image generation method provided in the embodiments of the present application is shown in Figure 3 S20, as shown in the above-mentioned S20, includes:

[0130] S21: obtaining first color parameters of the first color in the second color space according to the first color of the to-be-processed image in the first color space and using a preset color space conversion relationship.

[0131] In the embodiments, the first color space is an RGB (Red-Green-Blue) space, and the second color space is an HSV (Hue-Saturation-Value) space. The first color of the to-be-processed image in the first color space is converted into the first color parameters in the second color space according to the preset color space conversion relationship between the first color space and the second color space.

[0132] For example, the color space conversion relationship can be:

[0133]

[0134]

[0135] V = I max

[0136] wherein, Δ = I max -I min , I max = min(I i ' n [R], I i ' n [G], I i ' n [B]) , Imin =max(I i ' n [R],I i ' n [G],I i ' n [B]), I i ' n It is the image I to be processed in The result of normalizing the pixel values. The image to be processed, converted to the second color space, is I. hsv =combine(H,S,V).

[0137] S22: Based on the hue component, brightness component and preset random quantity in the first color parameter, a preset three-dimensional color lookup table is used to determine the second color.

[0138] In this embodiment, the preset three-dimensional color lookup table is composed of multiple hue lookup tables H, and each hue H lookup table is composed of multiple brightness lookup tables V. Each brightness V lookup table contains multiple preset colors, and each preset color has a unique weight. The multiple hues H include: red (H=0°), orange (H=30°), yellow (H=60°), green (H=120°), cyan (H=180°), blue (H=240°), purple (H=270°), and magenta (H=300°).

[0139] Based on the hue component H in the first color parameter, determine the two hues closest to this hue component, which are H0 and H1 respectively. lower and H upper Their corresponding indices are respectively and Calculate the hue component H to the two nearest hues H lower and H upper Using the distance as a preset probability, a random index corresponding to a hue is selected as the index of the hue component H in the hue dimension. For example, the formula for selecting the hue dimension index can be: That is, when the random number generated between 0 and 1 is less than the preset probability, the color H is used. lower index Index as the hue component H H When a random number generated between 0 and 1 is greater than or equal to a preset probability, the color H is selected. upper index Index as the hue component H H .

[0140] The index of the luminance component V in the luminance dimension is calculated in the same way as the index of the hue component H. For example, the selection formula of the index of the luminance dimension can be: That is, when the random number generated between 0 and 1 is less than the preset probability, the index of the luminance V lower is taken as the index Index of the hue component V. V When the random number generated between 0 and 1 is greater than or equal to the preset probability, the index of the luminance V upper is taken as the index Index of the luminance component V. V The preset random number is a number randomly generated between 0 and 1 as the index Index K of the weight dimension K.

[0141] The three-dimensional color lookup table is sampled to determine the corresponding hue component H according to the index Index H of the hue component H, determine the corresponding luminance component H according to the index Index H of the luminance component V under the hue component H, and determine the corresponding color C as the second color according to the index Index K of the weight dimension K under the luminance component H, and copy the second color to the color image I map corresponding to the pixel point.

[0142] S23: According to the second color, the color space conversion relationship is used to obtain the second color parameter of the second color in the second color space.

[0143] In this embodiment, after determining the corresponding second color of the first color in the three-dimensional color lookup table, the second color is converted into the second color parameter in the second color space again by using the color space conversion relationship, and the specific conversion manner is not described here.

[0144] S24: Replace the saturation component in the second color parameter with the saturation component in the first color parameter to obtain the third color parameter.

[0145] In this embodiment, since the second color parameter in the second color space changes after the first color is converted into the second color, in order to ensure that the saturation of each pixel point will not be reduced, the saturation component S in the second color parameter is replaced with the saturation component S in the first color parameter to obtain the third color parameter.

[0146] S25: Convert the third color parameter into the third color in the first color space by using the color space conversion relationship.

[0147] In the embodiment, the third color parameter is inversely converted by using the color space conversion relationship to obtain a third color of the third color parameter in the first color space, and a color image I' having the third color is obtained. map .

[0148] S26: mixing the first color and the third color on the to-be-processed image to obtain the color remapped to-be-processed image.

[0149] In the embodiment, the to-be-processed image I having the first color is mixed with the color image I' having the third color to obtain the color remapped to-be-processed image. in and the color image I' having the third color. map The colors of the corresponding pixel points are randomly mixed to obtain the color remapped to-be-processed image.

[0150] The oil painting style image generation method provided in the embodiment of the application comprises the following steps: obtaining a first color parameter of a first color of a to-be-processed image in a second color space by using a preset color space conversion relationship according to the first color of the to-be-processed image in a first color space; determining a second color by using a preset three-dimensional color lookup table according to a hue component and a brightness component in the first color parameter and a preset random number; obtaining a second color parameter of the second color in the second color space by using the color space conversion relationship according to the second color; replacing a saturation component in the second color parameter with a saturation component in the first color parameter to obtain a third color parameter; converting the third color parameter into a third color in the first color space by using the color space conversion relationship; and mixing the first color and the third color on the to-be-processed image to obtain a color remapped to-be-processed image. The color of the to-be-processed image is adjusted in three dimensions of hue, brightness and saturation, so that the color of the adjusted to-be-processed image is closer to the color of an oil painting, and the effect of the obtained oil painting style image is better.

[0151] On the basis of the above-mentioned embodiment, the embodiment of the application further provides another oil painting style image generation method. Please refer to Figure 4 The flowchart of another oil painting style image generation method provided in the embodiment of the application is shown in Figure 4 The above-mentioned S30 comprises:

[0152] S31: determining a pen drop set corresponding to each brush from each pixel point in the color remapped to-be-processed image according to a pixel point threshold value corresponding to each brush and an importance value of each pixel point, the pen drop set comprising at least one pen drop point corresponding to each brush.

[0153] In the embodiment, the brush threshold value can be a pixel point threshold value, each brush has a corresponding pixel point threshold value, each brush is sequentially taken out from the preset brush set, each pixel point is traversed using a preset step, if the importance value of the pixel point is less than the pixel point threshold value of the brush, the pixel point can be used as the landing point of the brush. After the preset traversal step is used to traverse the color remapped to-be-processed image, the preset traversal step is divided by 2, and the traversal is performed again until the preset traversal step is equal to 1, to obtain a landing set composed of at least one landing point corresponding to each brush. For example, the calculation formula of the traversal step can be:

[0154]

[0155] For example, the embodiment provides a method for determining a landing set, which can be:

[0156] A pigment stack thickness map I is created in advance stack Each pixel point of the pigment stack thickness map I stack is assigned a value of 0, and the size of the pigment stack thickness map I stack is the same as that of the color remapped to-be-processed image. It is assumed that a plurality of brushes in the preset brush set include B body , B skin , B1, B2, …, B L , B body is a brush for a portrait area, B skin is a brush for a skin area, B1, B2, …, B L are brushes for different detail layers, and the radii of the brushes B1, B2, …, B L are in an arithmetic progression, and the pixel point threshold values of the brushes are sequentially increased, that is, a brush with a small brush radius can draw a pixel point with a small importance value, but a brush with a large brush radius cannot draw a pixel point with a large importance value.

[0157] The plurality of brushes B body , B skin , B1, B2, …, B L are added to the drawing queue, one brush is sequentially taken out from the drawing queue, a preset traversal step Step=4 is set, and the preset traversal step is used to traverse the pigment stack thickness map I stackIf the pixel value of the pixel point is less than 1 and the importance value of the pixel point in the image to be processed is less than the pixel point threshold of the brush, the coordinates of the pixel point are added to the candidate set S of the brush. The candidate set S includes a plurality of candidate drop points. Each brush has a preset pigment stacking thickness, which is generally between 0 and 1. It is determined whether the pigment stacking thickness of the position of the candidate drop point is less than 1. When it is less than 1, the candidate drop point is added to the drop set of the brush. After each drop point is determined, the pigment stacking thickness of the position of the drop point is increased according to the pigment stacking thickness of the brush. When the pigment stacking thickness of the position of the candidate drop point is greater than 1, it indicates that the position cannot be dropped, and the candidate drop point is discarded.

[0158] Further, the brush pattern of the first brush corresponding to the first drop point is rotated by θ°, and after the long side is enlarged, the horizontal offset Δ stack and the vertical offset Δ X of the top left corner of the brush pattern and the pigment stacking thickness map I Y are determined. According to the pixel value of the pixel point at the horizontal offset Δ X and the vertical offset Δ Y of the pixel point, the thickness of the pixel point at the horizontal offset Δ X and the vertical offset Δ Y is updated.

[0159] For example, the update formula of the thickness can be:

[0160] I stack (i+Δ X ,j+Δ Y )=I stack (i+Δ X ,j+Δ Y )+B(i,j)

[0161] Where B(i,j) is the thickness of the brush corresponding to the pixel point (i,j).

[0162] S32: Determine the preset pattern corresponding to each brush as the brush pattern of the drop point corresponding to each brush.

[0163] In this embodiment, the brush parameters of each brush include a preset pattern, which is a brush pattern corresponding to different oil painting styles. For the drop points in the drop set of each brush, the preset image of the brush is used as the brush pattern of the drop point.

[0164] S33: Determine the drop center color of each drop point according to the color of each pixel point in the color remapped image to be processed.

[0165] In this embodiment, according to the position of each first pen drop point in the pigment stack thickness map I stack , the color of the pixel point at the corresponding position in the color remapped to-be-processed image is determined as the pen drop center color C center of the pen drop point.

[0166] S34: The brush rotation angle of each pen drop point is calculated according to the gradient of each pixel point in the to-be-processed image in two mutually perpendicular directions. The brush rendering parameters of each pen drop point include the brush pattern, the pen drop center color and the brush rotation angle of each pen drop point.

[0167] In this embodiment, the first gradient of each pixel point in two mutually perpendicular directions includes a first horizontal gradient G X in the horizontal direction and a first vertical gradient G Y in the vertical direction. The horizontal direction and the vertical direction are mutually perpendicular. The brush rotation angle θ is calculated according to the first horizontal gradient G X and the first vertical gradient G Y .

[0168] For example, the calculation formula of the brush rotation angle θ can be:

[0169]

[0170] The oil painting style image generation method provided by the embodiments of the present application determines the pen drop set corresponding to each brush from the pixels in the color remapped to-be-processed image according to the threshold of the pixel corresponding to each brush and the importance value of each pixel, the pen drop set includes at least one pen drop point corresponding to each brush, determines the preset pattern corresponding to each brush as the brush pattern of the pen drop point corresponding to each brush, determines the pen drop center color of each pen drop point according to the color of each pixel in the color remapped to-be-processed image, and calculates the brush rotation angle of each pen drop point according to the gradient of each pixel in the to-be-processed image in two mutually perpendicular directions. The embodiments of the present application can obtain accurate brush rendering parameters, so that the effect of the oil painting image rendered according to the brush rendering parameters is closer to the color remapped to-be-processed image, and the effect of the oil painting style is better.

[0171] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide another oil painting style image generation method. Please refer to Figure 5 , the flowchart of another oil painting style image generation method provided by the embodiments of the present application is shown in Figure 5 , before the above-mentioned S34, the method further includes:

[0172] S351: An edge image of the to-be-processed image is obtained.

[0173] In this embodiment, a preset edge extraction method is used to extract edges from the image to be processed, thereby obtaining an edge image corresponding to the image to be processed. The edge image includes the edge lines of each object in the image to be processed. For example, the edge extraction method can be the ED (Edge Drawing) method.

[0174] In an optional implementation, if the image to be processed includes a human figure, then after determining the edge image of the image to be processed, the face set S obtained above is used as a basis for further processing. faces For each face in the image, the outlines of the left eye, right eye, left eyeball, right eyeball, nose, and lips are drawn onto the edge image to enhance the edges of the facial features.

[0175] In one alternative implementation, before edge extraction is performed on the image to be processed, the width and height of the image to be processed are scaled, and the width W of the image used for brush direction processing is calculated. flow and height H flow Furthermore, after scaling the image to be processed, the processing width W and processing height H are obtained. Based on the processing width W and processing height H, the width W of the image used for brush direction processing is calculated. flow and height H flow Among them, the scaling parameter P scale The value can be any positive integer. For example, the width W... flow and height H flow The calculation formula can be:

[0176]

[0177]

[0178] If the scaling parameter P scale If the parameter value is not 1, the image needs to be reduced in size. flow ·P scale If ≠ W, then denote Pad. R For W flow ·P scale -W, if H flow ·P scale If ≠ W, then denote Pad. B For H flow ·P scale -W, if Pad R or Pad B If the value is not 0, then the image to be processed, I... in right-side extended Pad R Pixels, bottom extended Pad B Each pixel is used in the extended region to process the image I.in , and the edge-padded image I tmp is obtained. in , and the edge-padded image I tmp is obtained. tmp , the width and height of the edge-padded image I flow are reduced to W flow and H edge respectively by using the pixel area method, and the reduced image is stored in the image I scale . If the parameter value of the scaling parameter P in is 1, the to-be-processed image I edge is directly copied to the image I

[0179] Further, the image I edge is processed by using an anisotropic diffusion filter algorithm to enhance the directionality of the image.

[0180] Further, a super pixel segmentation algorithm is implemented by using a simple linear iterative clustering (SLIC) to process the image I edge , so as to reduce the color of the image and enhance the boundary of the image edge.

[0181] Further, the image I edge is processed by using a small-radius median filter to reduce the sawtooth of the super pixel segmentation edge and remove relatively small blocks.

[0182] S352: After rotating the blank image with a plurality of white points with random positions, the white point edge image is obtained by weighted mixing of the blank image and the edge image.

[0183] In this embodiment, a plurality of white points are randomly drawn on the blank image I dots , wherein the pixel value of the white point is 255, and the pixel value of other pixel points on the blank image I dots except the white point is 0. The position of the randomly generated white point on the blank image is as follows:

[0184]

[0185]

[0186] I dots (i,j) = 255 0≤i<W rotated ,0≤j<H rotated

[0187] wherein, is a distance parameter of two adjacent points in the same row in the horizontal direction, is a distance parameter of two adjacent points in the same column in the vertical direction, shake is a jitter distance proportion parameter, rotated and H rotated are the size reserved after the image with a width of W flow and a height of H flow is rotated by P θ °. For example, the calculation formula of W rotated and H rotated may be as follows:

[0188]

[0189]

[0190] After the blank image with multiple white points is rotated by P θ °, the blank image is mixed with the edge image by weighting to obtain a white point edge image.

[0191] In an optional embodiment, before the blank image with white points and the edge image are mixed by weighting, the upper, lower, left and right edges of the edge image are expanded, and the expanded area is filled with 0. The expansion size of the upper edge is denoted as Pad T , the expansion size of the lower edge is denoted as Pad B , the expansion size of the left edge is denoted as Pad L , and the expansion size of the right edge is denoted as Pad R . For example, the calculation formula of the expansion size may be as follows:

[0192]

[0193]

[0194]

[0195]

[0196] In an optional embodiment, if the image to be processed includes a portrait, the image corresponding to the portrait region mask is used as a weight to mix the blank image with multiple white points and the edge image to obtain a white point edge image.

[0197] Specifically, if the image to be processed includes a portrait, the upper, lower, left and right edges of the portrait region mask are respectively expanded by Pad T , Pad B , Pad L and Pad R, the extended area is filled with 0, and an image I corresponding to the portrait region mask is obtained body For example, a formula for mixing the blank image and the edge image with multiple white points using the image corresponding to the portrait region mask as a weight can be:

[0198]

[0199] S353: Calculate the distance between each pixel point in the white point edge image and the nearest preset position point to obtain a distance image; the value of each pixel point in the distance image is the distance between the corresponding pixel point in the white point edge image and the nearest preset position point; the preset position point is an edge point or a white point.

[0200] In this embodiment, a preset distance calculation method is used to calculate the distance between each pixel point in the white point edge image and the nearest edge point or white point to obtain a distance image I corresponding to the white point edge image e dge For example, the distance calculation method can be a JFA (Jump Flooding Algorithm) algorithm.

[0201] S354: Calculate the first initial gradient of each pixel point in the distance image in two mutually perpendicular directions and the first initial gradient amplitude between the two mutually perpendicular directions.

[0202] In this embodiment, the first initial gradient of each pixel point in the two mutually perpendicular directions includes a first initial horizontal gradient G X and a first initial vertical gradient G Y The first initial horizontal gradient G X is used to represent the change in the horizontal direction from the pixel point (i-1, j) to the pixel point (i, j), and the first initial vertical gradient G Y is used to represent the change in the vertical direction from the pixel point (i, j-1) to the pixel point (i, j). When a region in the distance image is relatively smooth, the gray scale normalized value changes less, and the corresponding gradient value is also smaller. The first initial gradient amplitude G may be

[0203] For example, the Sobel operator can be used to calculate the initial horizontal gradient G X and the initial vertical gradient G Y , and the calculation formula can be:

[0204]

[0205]

[0206] ​S355: Calculate the second initial gradient of each pixel in the grayscale image corresponding to the image to be processed in two mutually perpendicular directions and the magnitude of the second initial gradient between the two mutually perpendicular directions.

[0207] For example, after converting and normalizing the image to be processed, image I is obtained. i ' n Calculate image I i ' n The second initial horizontal gradient, the second initial vertical gradient, and the second initial gradient magnitude The scheme is the same as the scheme for calculating the first initial horizontal gradient, the first initial vertical gradient, and the magnitude of the first initial gradient, and will not be repeated here.

[0208] S356: Weight the first initial gradient magnitude and the second initial gradient magnitude to obtain the target gradient magnitude.

[0209] In this embodiment, by adjusting the first initial gradient magnitude... Second initial gradient magnitude Weighted mixing is performed to obtain the target gradient magnitude.

[0210] In one optional embodiment, if the image to be processed includes a human figure, a hair region mask M in the image to be processed is obtained according to a preset mask deep learning method. hair Extend the Pad along the top, bottom, left, and right edges of the mask for the hair area. T Pad B Pad L and Pad R The extended region is filled with 0s to obtain the image I corresponding to the hair region mask. hair For example, the image corresponding to the hair region mask is used as the weight for the first initial gradient magnitude. Second initial gradient magnitude The formula for weighted mixing can be:

[0211]

[0212] S357: Calculate the angle difference between the pixel and the multiple region pixels based on the initial directional gradient of the pixel and the initial gradient of the pixel within a preset area centered on the pixel in one direction.

[0213] In this embodiment, taking the horizontal direction as an example, the process of calculating the angle difference is explained. Centered on pixel (i,j), within a radius R in the horizontal direction (i+R,j), the angle difference θ between the initial horizontal gradient of pixel (i,j) and the initial horizontal gradients of the multiple regions of pixels (i+R,j) is calculated. The same method is used in the vertical direction, with multiple regions of pixels (i,j+R). The calculation method is not elaborated here.

[0214] S358: Calculate the gradient magnitude difference between a pixel and the pixels in multiple regions based on the target gradient magnitude of pixels in multiple regions and the target gradient magnitude of a pixel.

[0215] In this embodiment, the gradient magnitude difference Δ between the initial gradient magnitude of pixel (i,j) and the initial gradient magnitudes of multiple region pixels (i+R,j) is calculated. Mag This yields multiple gradient magnitude differences Δ corresponding to pixel points (i+R,j) in multiple regions. Mag .

[0216] S359: Calculate the gradient of the pixel in the two mutually perpendicular directions based on the angle difference between the two mutually perpendicular directions and the gradient magnitude difference in the corresponding directions.

[0217] In this embodiment, let the horizontal weighted gradient G' X and vertical weighted gradient G' Y To initialize to zero, first, in the horizontal direction, calculate the horizontal weighted gradient G'. X and vertical weighted gradient G' Y Specifically: when the angle difference θ is less than or equal to ±90°, multiple gradient magnitude differences Δ are used. Mag As weights, the results of multiplying by the initial horizontal gradients of multiple region pixels (i+R,j) and the horizontal weighted gradient G' are respectively... X The results are accumulated to obtain a horizontally weighted blending result, which is then multiplied by the initial vertical gradients of multiple region pixels (i+R,j) and combined with the vertically weighted gradient G'. Y The results are accumulated to obtain a vertically weighted mixture; when the angle difference θ is greater than ±90°, the horizontally weighted gradient G' is used. X Subtract the difference in magnitude of multiple gradients Δ Mag As weights, the results of multiplying by the initial horizontal gradients of multiple region pixels (i+R,j) respectively yield the horizontally weighted mixing result, with the vertical weighted gradient G' as the weight. Y Subtract the difference in magnitude of multiple gradients Δ Mag As weights, the initial vertical gradients of multiple region pixels (i+R,j) are multiplied to obtain the vertically weighted mixing result. The horizontally weighted mixing result and the vertically weighted gradient are then normalized to obtain the first horizontal gradient G. Xand the first vertical gradient G Y .

[0218] For example, for the horizontal weighted gradient G X and the vertical weighted gradient G Y , the calculation formula can be:

[0219]

[0220]

[0221] Similarly, on the basis of the first horizontal gradient G X and the first vertical gradient G Y , the horizontal weighted gradient G X and the vertical weighted gradient G Y are further calculated in the vertical direction in the same way, which will not be repeated here. For example, for the horizontal weighted gradient G X and the vertical weighted gradient G Y , the calculation formula can be:

[0222]

[0223]

[0224] Further, the above S357-S359 can be repeated K times to increase the difference of each pixel point in the horizontal direction and the vertical direction to obtain the first horizontal gradient and the first vertical gradient.

[0225] The oil painting style image generation method provided in the embodiments of the present application comprises the following steps: obtaining an edge image of the image to be processed; performing weighted mixing on a blank image with a plurality of white points with random positions after rotation and the edge image to obtain a white point edge image; calculating distances between each pixel point in the white point edge image and the nearest preset position point to obtain a distance image; the value of each pixel point in the distance image is the distance between the corresponding pixel point in the white point edge image and the nearest preset position point; the preset position point is an edge point or a white point; calculating a first initial gradient of each pixel point in the distance image in the two mutually perpendicular directions and a first initial gradient amplitude between the two mutually perpendicular directions; calculating a second initial gradient of each pixel point in a corresponding gray image of the image to be processed in the two mutually perpendicular directions and a second initial gradient amplitude between the two mutually perpendicular directions; performing weighted mixing on the first initial gradient amplitude and the second initial gradient amplitude to obtain a target gradient amplitude; calculating an angle difference between the pixel point and a plurality of regional pixel points in a preset region range centered on the pixel point in one direction according to the initial direction gradient of the plurality of regional pixel points and the initial gradient of the pixel point; calculating a gradient amplitude difference between the pixel point and the plurality of regional pixel points according to the target gradient amplitude of the plurality of regional pixel points and the target gradient amplitude of the pixel point; and calculating the gradient of the pixel point in the two mutually perpendicular directions according to the angle difference between the two mutually perpendicular directions and the gradient amplitude difference in the corresponding direction. The brush direction calculated according to the gradient presents circular rotation at the position of the random white point, and the oil painting style image is generated.

[0226] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide another oil painting style image generation method. Please refer to Figure 6 The flowchart of the another oil painting style image generation method provided in the embodiments of the present application is shown in Figure 6 The method further comprises the following steps:

[0227] S51: generating a face edge feature importance map and a portrait edge feature importance map by respectively processing the image to be processed according to preset face brightness enhancement parameters and preset portrait brightness enhancement parameters.

[0228] In the embodiments, in order to further enhance the face details of the oil painting style image, the second gradient of each pixel point in the two mutually perpendicular directions in the image to be processed is calculated according to the preset portrait brightness enhancement parameters, and the second gradient comprises a second horizontal gradient and a second vertical gradient.

[0229] In an optional implementation, in order to better increase the portrait details of the oil painting style image, the second gradient of each pixel point in the two mutually perpendicular directions in the image to be processed is calculated according to the preset portrait brightness enhancement parameters, and the second gradient comprises a second horizontal gradient and a second vertical gradient. infiltering to obtain a filtered image I median , the filtered image I median is calculated respectively. For example, the filtering mode can be median filtering, and the calculation mode can be:

[0230]

[0231]

[0232] wherein P brightness is a face brightness enhancement parameter, and a second gradient amplitude I is calculated according to the second horizontal gradient and the second vertical gradient

[0233] A color mapping function L edge is calculated for enhancing details edge , and a face edge map after enhancement is determined according to the color mapping function L magnitude and the second gradient amplitude I

[0234] For example, the color mapping function L edge may be calculated according to the following calculation formula:

[0235]

[0236]

[0237] wherein P shadows is a shadow parameter, P medians is a mid-tone parameter, P highlights is a highlight parameter, P minimum is a minimum value of a color scale, and P maximum is a maximum value of a color scale.

[0238] After the second gradient amplitude I magnitude is converted into a gray-scale image, the color mapping function L edge is adopted, and color inversion is performed to obtain a face edge map after enhancement For example, the calculation formula can be:

[0239]

[0240] An image edge map is generated by adopting a portrait brightness enhancement parameter by using the same method of calculating the face edge map

[0241] After the face edge map and the image edge map are obtained, a target region of each face is drawn to the image I faces according to face points of each face in a face set S maskThe target region can be, for example, an eye region and a lip region. The image I mask is subjected to appropriate erosion and Gaussian blur, and the face edge image and the portrait edge image are obtained by using the image I mask is mixed to obtain a face edge feature importance map and a portrait edge feature importance map For example, the calculation formula can be, for example:

[0242]

[0243]

[0244] S52: An edge importance image is generated according to the face region mask, the portrait region mask, the face edge feature importance map and the portrait edge feature importance map corresponding to the image to be processed.

[0245] In this embodiment, the face region mask M face and the face edge feature importance map are mixed to obtain a face edge feature importance map body and the portrait edge feature importance map are mixed to obtain an edge importance image I edge . For example, the calculation formula can be:

[0246]

[0247]

[0248]

[0249] S53: The saturation component and the brightness component of the image to be processed are adjusted to obtain an adjusted image to be processed.

[0250] In this embodiment, the image I color is subjected to three times of erosion, and the image I color is converted from the RGB color space to the HSV color space, the saturation component S is enhanced, the brightness component V is weakened, and then the image I edge is converted from the HSV color space to the RGB color space to obtain a color-adjusted image to be processed.

[0251] S54: The edge importance image, the adjusted image to be processed and the oil painting style image are mixed to obtain a target oil painting style image.

[0252] In this embodiment, the edge importance image I edge and the color-adjusted image to be processed I edgeMixing to oil painting style image I out The target oil painting style image is obtained. For example, the calculation formula can be:

[0253]

[0254] The oil painting style image generation method provided in the embodiments of the present application processes the to-be-processed image according to the preset face brightness enhancement parameter and the preset portrait brightness enhancement parameter, generates a face edge feature importance map and a portrait edge feature importance map, generates an edge importance image according to the face region mask, the portrait region mask, the face edge feature importance map and the portrait edge feature importance map corresponding to the to-be-processed image, adjusts the saturation component and the brightness component of the to-be-processed image, obtains an adjusted to-be-processed image, mixes the edge importance image, the adjusted to-be-processed image and the oil painting style image, and obtains a target oil painting style image. The embodiments of the present application further enhance the face edge and the portrait edge, enhance the edge detail effect of the oil painting image, and make the detail effect of the target oil painting image better.

[0255] In an optional embodiment, the method further includes: superimposing a random texture material on the oil painting image, adjusting the color of each pixel point on the oil painting image by using a preset two-dimensional color lookup table, and obtaining an oil painting style image with adjusted color.

[0256] In the embodiment, a random texture material image I canvas is loaded from a preset file, and then is mixed into the oil painting style image by superimposition. For example, the superimposition formula of the brush texture image or the random texture material image on the oil painting style image can be:

[0257]

[0258] Wherein, A is the oil painting style image, B is the random texture material image, and C is the oil painting style image after superimposition.

[0259] A two-dimensional color lookup table material image I lut is loaded from a preset file, and then the three-dimensional color C in of each pixel point on the oil painting image is adjusted by using the two-dimensional color lookup table material image I lut to find the output color C out . The color of the oil painting image is adjusted by using the output color C out , and an oil painting image with adjusted color is obtained. The color of the image with adjusted color is more inclined to the color of the oil painting.

[0260] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide an oil painting style image generation device. Please refer to Figure 7A structure schematic diagram of an oil painting style image generation device provided by an embodiment of the present application is shown in FIG. 1. The device includes: Figure 7

[0261] An importance map determination module 10 is configured to process the multi-layer blank canvas image according to the preset multi-layer importance threshold based on the to-be-processed image, to obtain an importance map of the to-be-processed image, wherein the importance value of each pixel point in the importance map is used to indicate the importance of each pixel point in the to-be-processed image.

[0262] A color mapping module 20 is configured to perform color remapping on the to-be-processed image according to the color parameter of the to-be-processed image, to obtain the to-be-processed image after color remapping.

[0263] A rendering parameter determination module 30 is configured to determine the brush rendering parameters of the multiple drop points in the to-be-processed image after color remapping according to the multiple brushes in the preset brush set and the importance value of each pixel point.

[0264] A rendering module 40 is configured to perform rendering on the multiple drop points in the to-be-processed image after color remapping by using the brush rendering parameters of the multiple drop points, to obtain the oil painting style image corresponding to the to-be-processed image.

[0265] Optionally, the importance map determination module 10 includes:

[0266] A target region drawing unit is configured to draw the first target region in the to-be-processed image into the multi-layer blank canvas image according to the multi-layer importance threshold, to obtain the multi-layer region drawing image.

[0267] An importance map determination unit is configured to obtain the importance map of the to-be-processed image according to the mask of the second target region in the to-be-processed image, the mask of the third target region in the to-be-processed image, and the multi-layer region drawing image, wherein the first target region, the second target region, and the third target region are three associated regions of the same type of object in the to-be-processed image.

[0268] Optionally, the importance map determination unit includes:

[0269] A calculation subunit is configured to multiply the mask of the second target region, the mask of the third target region, and the multi-layer region drawing image by the corresponding importance threshold.

[0270] An importance map determination subunit is configured to generate the importance map of the to-be-processed image according to the maximum pixel value of each pixel point after multiplication.

[0271] Optionally, the color mapping module 20 includes:

[0272] ​The first color parameter calculation unit is configured to obtain a first color parameter of a first color of the image to be processed in a second color space according to a preset color space conversion relationship based on the first color of the image to be processed in a first color space;

[0273] The second color calculation unit is configured to determine a second color according to a hue component and a brightness component in the first color parameter, a preset random number, and a preset three-dimensional color lookup table;

[0274] The second color parameter calculation unit is configured to obtain a second color parameter of the second color in the second color space according to the second color and the color space conversion relationship;

[0275] The third color parameter calculation unit is configured to replace a saturation component in the second color parameter with a saturation component in the first color parameter to obtain a third color parameter;

[0276] The third color calculation unit is configured to convert the third color parameter into a third color in the first color space according to the color space conversion relationship;

[0277] The color mixing unit is configured to mix the first color and the third color on the image to be processed to obtain a color remapped image to be processed.

[0278] Optionally, the rendering parameter determination module 30 comprises:

[0279] The pen drop point determination unit is configured to determine a pen drop set corresponding to each brush from each pixel point in the color remapped image to be processed according to a pixel point threshold value corresponding to each brush and an importance value of each pixel point, the pen drop set comprising at least one pen drop point corresponding to each brush;

[0280] The brush pattern determination unit is configured to determine a preset pattern corresponding to each brush as a brush pattern of the pen drop point corresponding to each brush;

[0281] The color confirmation unit is configured to determine a pen drop center color of each pen drop point according to a color of each pixel point in the color remapped image to be processed;

[0282] The angle confirmation unit is configured to calculate a brush rotation angle of each pen drop point according to gradients of each pixel point in the image to be processed in two mutually perpendicular directions, the brush rendering parameter of each pen drop point comprising: the brush pattern, the pen drop center color, and the brush rotation angle of each pen drop point.

[0283] Optionally, before the angle confirmation unit, the device further comprises:

[0284] The edge image acquisition unit is configured to acquire an edge image of the image to be processed;

[0285] The white point mixing unit is configured to perform weighted mixing of the blank image with the plurality of white points having random positions after rotation and the edge image to obtain a white point edge image.

[0286] The distance calculation unit is configured to calculate distances between each pixel point in the white point edge image and a nearest preset position point to obtain a distance image, wherein a value of each pixel point in the distance image is a distance between the corresponding pixel point in the white point edge image and the nearest preset position point, and the preset position point is an edge point or a white point.

[0287] The first initial gradient calculation unit is configured to calculate a first initial gradient of each pixel point in the distance image in two mutually perpendicular directions and a first initial gradient amplitude between the two mutually perpendicular directions.

[0288] The second initial gradient calculation unit is configured to calculate a second initial gradient of each pixel point in a corresponding gray-scale image of the to-be-processed image in two mutually perpendicular directions and a second initial gradient amplitude between the two mutually perpendicular directions.

[0289] The amplitude calculation unit is configured to perform weighted mixing of the first initial gradient amplitude and the second initial gradient amplitude to obtain a target gradient amplitude.

[0290] The angle difference calculation unit is configured to calculate an angle difference between the pixel point and a plurality of region pixel points according to initial direction gradients of the plurality of region pixel points within a preset region range centered on the pixel point in one direction and an initial gradient of the pixel point.

[0291] The amplitude difference calculation unit is configured to calculate a gradient amplitude difference between the pixel point and the plurality of region pixel points according to target gradient amplitudes of the plurality of region pixel points and a target gradient amplitude of the pixel point.

[0292] The gradient calculation unit is configured to calculate gradients of the pixel point in two mutually perpendicular directions according to the angle difference between the two mutually perpendicular directions and the gradient amplitude difference in the corresponding direction.

[0293] Optionally, after the rendering module 40, the device further comprises:

[0294] The edge brightness enhancement module is configured to respectively process the to-be-processed image according to preset face brightness enhancement parameters and preset portrait brightness enhancement parameters to generate a face edge feature importance map and a portrait edge feature importance map.

[0295] The edge importance map generation module is configured to generate an edge importance image according to a face region mask, a portrait region mask, the face edge feature importance map, and the portrait edge feature importance map corresponding to the to-be-processed image.

[0296] The parameter adjustment module is configured to adjust a saturation component and a brightness component of the to-be-processed image to obtain an adjusted to-be-processed image.

[0297] The image mixing module is configured to mix the edge importance image, the adjusted to-be-processed image, and the oil painting style image to obtain a target oil painting style image.

[0298] The device is configured to execute the method provided in the foregoing embodiments, and has similar implementation principles and technical effects, which will not be described here again.

[0299] The above modules can be one or more integrated circuits configured to implement the above method, for example, one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of a processing element scheduling code, the processing element can be a general-purpose processor, for example, a central processing unit (CPU) or other processor capable of invoking program code. For another example, the modules can be integrated together to implement in the form of a system on a chip (SOC).

[0300] For another example, the modules can be integrated together to implement in the form of a system on a chip (SOC). Figure 8 For another example, the modules can be integrated together to implement in the form of a system on a chip (SOC).

[0301] Optionally, the present application further provides a computer readable storage medium, and the storage medium stores a computer program. The computer program is executed by a processor to perform the method embodiments described above.

[0302] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. The units as divided can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0303] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0304] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into a unit. The integrated unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0305] The integrated unit implemented in the form of software functional units can be stored in a computer readable storage medium. The software functional units stored in the storage medium include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute some steps of the method described in the various embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, and various program code storage media.

[0306] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An oil painting style image generation method characterized by comprising: The method comprises: According to the image to be processed, a preset multi-layer importance threshold is used to process a multi-layer blank canvas image to obtain an importance map of the image to be processed, wherein the importance value of each pixel point in the importance map is used to indicate the importance of each pixel point in the image to be processed; According to the color parameters of the image to be processed, the image to be processed is color remapped to obtain a color-remapped image to be processed; According to a plurality of brushes in a preset brush set and the importance values of the pixel points, brush rendering parameters of a plurality of drop points in the color-remapped image to be processed are determined; The plurality of drop points in the color-remapped image to be processed are rendered by using the brush rendering parameters of the plurality of drop points to obtain an oil painting style image corresponding to the image to be processed. According to the image to be processed, a preset multi-layer importance threshold is used to process a multi-layer blank canvas image to obtain an importance map of the image to be processed, comprising: According to the multi-layer importance threshold, a first target region in the image to be processed is drawn into the multi-layer blank canvas image to obtain a multi-layer region drawing image; According to the mask of a second target region in the image to be processed, the mask of a third target region in the image to be processed and the multi-layer region drawing image, corresponding importance thresholds are multiplied respectively; According to the maximum pixel value of each pixel point after multiplication, the importance map of the image to be processed is generated, wherein the first target region, the second target region and the third target region are three associated regions of the same type of object in the image to be processed.

2. The method of claim 1, wherein, According to the color parameters of the image to be processed, the image to be processed is color remapped to obtain a color-remapped image to be processed, comprising: According to a first color of the image to be processed in a first color space, a preset color space conversion relationship is used to obtain a first color parameter of the first color in a second color space; According to a hue component, a brightness component in the first color parameter and a preset random number, a preset three-dimensional color lookup table is used to determine a second color; According to the second color, the color space conversion relationship is used to obtain a second color parameter of the second color in the second color space; The saturation component in the second color parameter is replaced by the saturation component in the first color parameter to obtain a third color parameter; The third color parameter is converted into a third color in the first color space by using the color space conversion relationship; The first color and the third color are mixed on the image to be processed to obtain the color-remapped image to be processed.

3. The method of claim 1, wherein, According to a plurality of brushes in a preset brush set and the importance values of the pixel points, brush rendering parameters of a plurality of drop points in the color-remapped image to be processed are determined, comprising: According to the pixel point threshold corresponding to each brush and the importance value of each pixel point, a landing set corresponding to each brush is determined from each pixel point in the color remapped to-be-processed image, and the landing set includes at least one landing point corresponding to each brush; A preset pattern corresponding to each brush is determined as a brush pattern of the landing point corresponding to each brush; According to the color of each pixel point in the color remapped to-be-processed image, a landing center color of each landing point is determined; According to the gradient of each pixel point in two mutually perpendicular directions in the to-be-processed image, a brush rotation angle of each landing point is calculated, and the brush rendering parameter of each landing point includes the brush pattern, the landing center color and the brush rotation angle of each landing point.

4. The method of claim 3, wherein, Before the gradient of each pixel point in two mutually perpendicular directions in the to-be-processed image is calculated to obtain the brush rotation angle of each landing point, the method further includes: An edge image of the to-be-processed image is obtained; After the blank image with a plurality of white points with random positions is rotated, the white point edge image is obtained by weighted mixing of the blank image and the edge image; The distance between each pixel point in the white point edge image and the nearest preset position point is calculated to obtain a distance image, and the preset position point is an edge point or a white point; A first initial gradient of each pixel point in the two mutually perpendicular directions in the distance image and a first initial gradient amplitude between the two mutually perpendicular directions are calculated; A second initial gradient of each pixel point in the two mutually perpendicular directions in a gray image corresponding to the to-be-processed image and a second initial gradient amplitude between the two mutually perpendicular directions are calculated; The first initial gradient amplitude and the second initial gradient amplitude are weighted mixed to obtain a target gradient amplitude; According to the initial direction gradient of a plurality of region pixel points in a preset region range centered on the pixel point in one direction and the initial gradient of the pixel point, an angle difference between the pixel point and the plurality of region pixel points is calculated; According to the target gradient amplitude of the plurality of region pixel points and the target gradient amplitude of the pixel point, a gradient amplitude difference between the pixel point and the plurality of region pixel points is calculated; According to the angle difference between the two mutually perpendicular directions and the gradient amplitude difference in the corresponding direction, the gradient of the pixel point in the two mutually perpendicular directions is calculated.

5. The method of claim 1, wherein, After the pointillism rendering of the plurality of landing points in the color remapped to-be-processed image is performed by using the brush rendering parameters of the plurality of landing points to obtain the oil painting style image corresponding to the to-be-processed image, the method further includes: According to the preset face brightness enhancement parameter and the preset portrait brightness enhancement parameter, the to-be-processed image is processed to generate a face edge feature importance image and a portrait edge feature importance image; According to the face region mask, the portrait region mask, the face edge feature importance image and the portrait edge feature importance image corresponding to the to-be-processed image, an edge importance image is generated. Adjust a saturation component and a brightness component of the to-be-processed image to obtain an adjusted to-be-processed image; Blend the edge importance image, the adjusted to-be-processed image and the oil painting style image to obtain a target oil painting style image.

6. An oil painting style image generation apparatus characterized by comprising: The device comprises: An importance map determination module configured to process a plurality of blank canvas images according to a to-be-processed image and a plurality of preset importance thresholds to obtain an importance map of the to-be-processed image, wherein an importance value of each pixel point in the importance map is used to indicate an importance degree of each pixel point in the to-be-processed image; A color mapping module configured to perform color remapping on the to-be-processed image according to a color parameter of the to-be-processed image to obtain a color-remapped to-be-processed image; A rendering parameter determination module configured to determine brush rendering parameters of a plurality of landing points in the color-remapped to-be-processed image according to a plurality of brushes in a preset brush set and the importance values of the pixel points; A rendering module configured to perform rendering on the plurality of landing points in the color-remapped to-be-processed image by using the brush rendering parameters of the plurality of landing points to obtain an oil painting style image corresponding to the to-be-processed image. The importance map determination module comprises: A target region drawing unit configured to draw a first target region in the to-be-processed image into the plurality of blank canvas images respectively according to the plurality of importance thresholds to obtain a plurality of region-drawn images; An importance map determination unit configured to multiply a mask of a second target region in the to-be-processed image, a mask of a third target region in the to-be-processed image and the plurality of region-drawn images respectively according to corresponding importance thresholds, and generate an importance map of the to-be-processed image according to maximum pixel values of each pixel point after the multiplication, wherein the first target region, the second target region and the third target region are three associated regions of a same type of object in the to-be-processed image.

7. An electronic device, comprising: A processor, a storage medium and a bus, the storage medium stores program instructions executable by the processor, when the electronic device is running, the processor and the storage medium communicate through the bus, the processor executes the program instructions to execute the steps of the oil painting style image generation method in any one of claims 1 to 5. The storage medium stores a computer program, when the computer program is run by the processor, the steps of the oil painting style image generation method in any one of claims 1 to 5 are executed.

8. A computer-readable storage medium, characterized in that, ​

Images