Image style transfer method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image style transfer method, device, electronic device and storage medium, and relates to the field of image processing technology, wherein the method includes: obtaining an original image and initial drawing parameters; using multiple pixel points in the original image as multiple sampling points, and obtaining the colors corresponding to the multiple sampling points; obtaining multiple mapping points corresponding to the multiple sampling points on a virtual drawing canvas; generating a virtual drawing brush according to the initial drawing parameters and the colors corresponding to the multiple sampling points; based on the multiple mapping points, using the virtual drawing brush to draw on the virtual drawing canvas to generate a target image after style transfer. The method uses a multi-layer simulation of a real user painting an oil painting, and performs multi-layer drawing on a blank virtual drawing canvas according to the original image to obtain a target image after style transfer, which can obtain a close-to-real oil painting effect, improve the degree of restoration of the original image, and improve the image style transfer effect.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image style transfer method, device, electronic device and storage medium. Background Art

[0002] At present, with the help of image processing software, users can create oil paintings without any art foundation, which are widely used in games, animations, advertisements, film and television special effects and other fields. In related technologies, a style transfer algorithm is used to transfer the style of input images to obtain images with oil painting style. Specifically, a picture A with a target style can be pre-selected. For any input picture B, the style of picture A can be transferred to picture B through the style transfer algorithm to obtain an output picture B', so that the output picture B' has the style of picture A while maintaining the content of input picture B.

[0003] However, in this method, the user needs to pre-select a picture A with the target style, which is inconvenient. In addition, the style transfer algorithm cannot guarantee that the output picture B' maintains the content of the input picture B and the style of the input picture A at the same time, thus failing to guarantee the oil painting effect of the output picture. Summary of the invention

[0004] The present disclosure provides an image style transfer method, device, electronic device and storage medium, so as to at least solve the problem that a style transfer algorithm is used in the related art to transfer the style of an input image to obtain an output image with an oil painting style, which is inconvenient to operate and cannot guarantee the oil painting effect of the output image.

[0005] The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image style transfer method, comprising:

[0007] Get the original image and initial drawing parameters;

[0008] Using a plurality of pixel points in the original image as a plurality of sampling points, and obtaining colors respectively corresponding to the plurality of sampling points;

[0009] Acquire a plurality of mapping points respectively corresponding to the plurality of sampling points on a virtual drawing canvas;

[0010] Generate a virtual drawing brush according to the initial drawing parameters and the colors corresponding to the plurality of sampling points;

[0011] Based on the multiple mapping points, the virtual drawing brush is used to draw on the virtual drawing canvas to generate a target image after style transfer, wherein the drawing process includes multiple drawing stages, in each drawing stage, the corresponding virtual drawing brush is used to draw on the virtual drawing canvas, and in each drawing stage, the drawing is based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method.

[0012] As a first possible scenario of the embodiment of the present disclosure, after generating a virtual drawing brush according to the initial drawing parameters and the colors respectively corresponding to the plurality of sampling points, the method further includes:

[0013] Acquire a plurality of adjacent pixel points of the plurality of sampling points in the original image;

[0014] Generating a drawing path corresponding to the sampling point according to the plurality of adjacent pixel points;

[0015] The parameters of the virtual drawing brush are adjusted according to the drawing path.

[0016] As a second possible situation of the embodiment of the present disclosure, the generating a drawing path corresponding to the sampling point according to the plurality of adjacent pixel points includes:

[0017] Respectively obtaining pixel differences between the sampling point and a plurality of adjacent pixel points of the sampling point;

[0018] Selecting an adjacent pixel point with the largest pixel difference from the plurality of adjacent pixel points as a target pixel point;

[0019] The drawing path is generated according to the sampling point and the target pixel point.

[0020] As a third possible situation of the embodiment of the present disclosure, the initial drawing parameter includes a mask identifier, and the method further includes:

[0021] Detecting the original image to obtain a target object and a target object type in the original image;

[0022] Determine a mask area of ​​the target object according to the type of the target object;

[0023] Setting the mask area in the virtual drawing canvas;

[0024] Drawing is performed in the mask area and in a non-mask area other than the mask area in the virtual drawing canvas using a drawing method that matches the mask identifier.

[0025] As a fourth possible situation of the embodiment of the present disclosure, the drawing method matching the mask identifier is adopted to draw in the mask area and the non-mask area other than the mask area in the virtual drawing canvas, including:

[0026] When the mask identifier is a first identifier, extracting a mask image corresponding to the mask area from the original image; wherein the first identifier is used to indicate not to draw in the mask area and to draw in the non-mask area;

[0027] The mask image is added to the mask area in the target image to generate a processed target image.

[0028] As a fifth possible situation of the embodiment of the present disclosure, the drawing method matching the mask identifier is adopted to draw in the mask area and the non-mask area other than the mask area in the virtual drawing canvas, including:

[0029] When the mask identifier is a second identifier, extracting a non-mask image corresponding to the non-mask area from the original image; wherein the second identifier is used to indicate to draw in the mask area and not to draw in the non-mask area;

[0030] The non-mask image is added to the non-mask area in the target image to generate a processed target image.

[0031] As a sixth possible situation of the embodiment of the present disclosure, the drawing method matching the mask identifier is adopted to draw in the mask area and the non-mask area other than the mask area in the virtual drawing canvas, including:

[0032] When the mask identifier is the third identifier, based on the multiple mapping points, the virtual drawing brush is used to draw in the mask area and the non-mask area to generate the target image; wherein the third identifier is used to indicate drawing in the mask area and drawing in the non-mask area, and the virtual drawing brush parameters in the mask area are different from the virtual drawing brush parameters in the non-mask area.

[0033] As a seventh possible situation of the embodiment of the present disclosure, the method further includes:

[0034] According to the drawing operation in the virtual drawing canvas, drawing is synchronously performed in the virtual texture canvas to generate a texture image; wherein the drawing process includes a plurality of drawing stages, in each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on an image obtained by drawing in a previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method;

[0035] The target image and the texture image are merged to generate a processed target image.

[0036] As an eighth possible situation of the embodiment of the present disclosure, synchronously drawing in a virtual texture canvas to generate a texture picture according to the drawing operation in the virtual drawing canvas includes:

[0037] Acquire a plurality of position points respectively corresponding to the plurality of mapping points in the virtual texture canvas;

[0038] In each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush based on the multiple position points;

[0039] At the plurality of position points, generating textures corresponding to the position points according to a height model corresponding to the virtual drawing brush;

[0040] A texture image corresponding to the drawing stage is generated according to the textures of the multiple position points.

[0041] As a ninth possible situation of the embodiment of the present disclosure, the method further includes:

[0042] In each rendering stage, a corresponding texture image is generated according to the rendering operation, and the target image and the texture image are fused in each rendering stage.

[0043] The image style transfer method of the disclosed embodiment uses a plurality of pixel points in the acquired original image as a plurality of sampling points, and obtains the colors corresponding to the plurality of sampling points, then obtains a plurality of mapping points corresponding to the plurality of sampling points on a virtual drawing canvas, generates a virtual drawing brush based on the acquired initial drawing parameters and the colors corresponding to the plurality of sampling points, and uses the virtual drawing brush to draw on the virtual drawing canvas based on the plurality of mapping points to generate a target image after style transfer, wherein the drawing process includes a plurality of drawing stages, in each drawing stage, drawing is performed on the virtual drawing canvas using the corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change mode. Thus, the user only needs to input an original image to obtain the corresponding target image after style transfer, without selecting an image with the target style, which can simplify the user operation, and through the multi-layer simulation of a real user painting oil painting, on a blank virtual drawing canvas, multi-layer drawing is performed based on the original image to obtain the target image after style transfer, which can obtain a close-to-real oil painting effect, improve the degree of restoration of the original image, and improve the image style transfer effect.

[0044] According to a second aspect of an embodiment of the present disclosure, there is provided an image style transfer apparatus, comprising:

[0045] A first acquisition module is configured to acquire an original image and initial drawing parameters;

[0046] A processing module is configured to use a plurality of pixel points in the original image as a plurality of sampling points, and obtain colors respectively corresponding to the plurality of sampling points;

[0047] A second acquisition module is configured to acquire a plurality of mapping points respectively corresponding to the plurality of sampling points on the virtual drawing canvas;

[0048] A first generating module is configured to generate a virtual drawing brush according to the initial drawing parameters and the colors respectively corresponding to the plurality of sampling points;

[0049] The first drawing module is configured to execute drawing on the virtual drawing canvas using the virtual drawing brush based on the multiple mapping points to generate a target image after style transfer, wherein the drawing process includes multiple drawing stages, in each drawing stage, drawing is performed on the virtual drawing canvas using the corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method.

[0050] As a first possible situation of the embodiment of the present disclosure, the device further includes:

[0051] A third acquisition module is configured to execute acquisition of a plurality of adjacent pixel points of the plurality of sampling points in the original image;

[0052] A second generating module is configured to generate a drawing path corresponding to the sampling point according to the plurality of adjacent pixel points;

[0053] The adjustment module is configured to adjust the parameters of the virtual drawing brush according to the drawing path.

[0054] As a second possible situation of the embodiment of the present disclosure, the second generating module is specifically configured to execute:

[0055] Respectively obtaining pixel differences between the sampling point and a plurality of adjacent pixel points of the sampling point;

[0056] Selecting an adjacent pixel point with the largest pixel difference from the plurality of adjacent pixel points as a target pixel point;

[0057] The drawing path is generated according to the sampling point and the target pixel point.

[0058] As a third possible situation of the embodiment of the present disclosure, the initial drawing parameter includes a mask identifier, and the device further includes:

[0059] A detection module is configured to detect the original image to obtain a target object and a target object type in the original image;

[0060] A determination module, configured to determine a mask area of ​​the target object according to the type of the target object;

[0061] A setting module, configured to execute setting of the mask area in the virtual drawing canvas;

[0062] The second drawing module is configured to execute drawing in a drawing mode matching the mask identifier, and draw in the mask area and a non-mask area of ​​the virtual drawing canvas except the mask area.

[0063] As a fourth possible situation of the embodiment of the present disclosure, the second drawing module is specifically configured to execute:

[0064] When the mask identifier is a first identifier, extracting a mask image corresponding to the mask area from the original image; wherein the first identifier is used to indicate not to draw in the mask area and to draw in the non-mask area;

[0065] The mask image is added to the mask area in the target image to generate a processed target image.

[0066] As a fifth possible situation of the embodiment of the present disclosure, the second drawing module is specifically configured to execute:

[0067] When the mask identifier is a second identifier, extracting a non-mask image corresponding to the non-mask area from the original image; wherein the second identifier is used to indicate to draw in the mask area and not to draw in the non-mask area;

[0068] The non-mask image is added to the non-mask area in the target image to generate a processed target image.

[0069] As a sixth possible situation of the embodiment of the present disclosure, the second drawing module is specifically configured to execute:

[0070] When the mask identifier is the third identifier, based on the multiple mapping points, the virtual drawing brush is used to draw in the mask area and the non-mask area to generate the target image; wherein the third identifier is used to indicate drawing in the mask area and drawing in the non-mask area, and the virtual drawing brush parameters in the mask area are different from the virtual drawing brush parameters in the non-mask area.

[0071] As a seventh possible situation of the embodiment of the present disclosure, the device further includes:

[0072] A third drawing module is configured to execute drawing operations in the virtual drawing canvas and synchronously draw in the virtual texture canvas to generate a texture image; wherein the drawing process includes a plurality of drawing stages, in each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on an image obtained by drawing in a previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method;

[0073] The first fusion module is configured to perform fusion of the target image and the texture image to generate a processed target image.

[0074] As an eighth possible situation of the embodiment of the present disclosure, the third drawing module is specifically configured to execute:

[0075] Acquire a plurality of position points respectively corresponding to the plurality of mapping points in the virtual texture canvas;

[0076] In each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush based on the multiple position points;

[0077] At the plurality of position points, generating textures corresponding to the position points according to a height model corresponding to the virtual drawing brush;

[0078] A texture image corresponding to the drawing stage is generated according to the textures of the multiple position points.

[0079] As a ninth possible situation of the embodiment of the present disclosure, the device further includes:

[0080] The second fusion module is configured to generate a corresponding texture image according to the drawing operation at each drawing stage, and fuse the target image with the texture image at each drawing stage.

[0081] The image style transfer device of the disclosed embodiment uses a plurality of pixel points in the acquired original image as a plurality of sampling points, and obtains the colors corresponding to the plurality of sampling points, then obtains a plurality of mapping points corresponding to the plurality of sampling points on a virtual drawing canvas, generates a virtual drawing brush based on the acquired initial drawing parameters and the colors corresponding to the plurality of sampling points, and uses the virtual drawing brush to draw on the virtual drawing canvas based on the plurality of mapping points to generate a target image after style transfer, wherein the drawing process includes a plurality of drawing stages, in each drawing stage, drawing is performed on the virtual drawing canvas with a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change mode. Thus, the user only needs to input an original image to obtain the corresponding target image after style transfer, without selecting an image with a target style, which can simplify user operations, and by simulating a real user painting oil painting in multiple layers, on a blank virtual drawing canvas, multiple layers of drawing are performed according to the original image to obtain the target image after style transfer, which can obtain a close-to-real oil painting effect, improve the degree of restoration of the original image, and improve the image style transfer effect.

[0082] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image style transfer method proposed in the embodiment of the first aspect of the present disclosure.

[0083] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the image style transfer method proposed in the embodiment of the first aspect of the present disclosure.

[0084] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided. When an instruction processor in the computer program product is executed, the image style transfer method proposed in the embodiment of the first aspect of the present disclosure is implemented.

[0085] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0087] Figure 1 is a flowchart of an image style transfer method according to an exemplary embodiment 1;

[0088] Figure 2a is a schematic diagram of an original picture according to an exemplary embodiment;

[0089] Figure 2b is a schematic diagram of a picture obtained by drawing at each drawing stage according to an exemplary embodiment;

[0090] Figure 3 is a flowchart of an image style transfer method according to a second exemplary embodiment;

[0091] Figure 4 is a schematic diagram of a drawing parameter adjustment tool interface according to an exemplary embodiment;

[0092] Figure 5 is a flowchart of an image style transfer method shown in accordance with a third exemplary embodiment;

[0093] Figure 6 is a flowchart of an image style transfer method shown in a fourth exemplary embodiment;

[0094] Figure 7 is a schematic diagram of a texture image obtained by drawing at each drawing stage according to an exemplary embodiment;

[0095] Figure 8 is a schematic diagram of a target image according to an exemplary embodiment;

[0096] Fig. 9 is a flowchart of an image style transfer method according to a fifth exemplary embodiment;

[0097] Fig.10 is a block diagram of an image style transfer apparatus according to a sixth exemplary embodiment;

[0098] Fig.11 The invention is a block diagram of an electronic device for image style transfer according to an exemplary embodiment. DETAILED DESCRIPTION

[0099] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0100] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0101] Figure 1 is a flowchart of an image style transfer method shown according to an exemplary embodiment.

[0102] The embodiment of the present disclosure takes the image style transfer method configured in an image style transfer device as an example. The image style transfer device can be applied to any electronic device so that the electronic device can perform the image style transfer function.

[0103] Among them, the electronic device can be any device with computing capabilities, such as a personal computer (PC), a mobile terminal, a server, etc. The mobile terminal can be, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a car-mounted device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0104] like Figure 1 As shown, the image style transfer method may include the following steps:

[0105] In step S101, the original image and initial drawing parameters are obtained.

[0106] In the embodiments of the present disclosure, the original image may be an image stored locally in the electronic device, such as an image pre-downloaded or taken by a user, or a new image obtained by the user by processing an image based on an image processing application, or the original image may be an image currently collected by the electronic device, etc. The present disclosure does not impose any restrictions on this.

[0107] In the disclosed embodiment, the initial drawing parameters may be selected in advance by the designer according to preferences and / or design requirements, and the initial drawing parameters may include drawing parameters such as the brush radius of the virtual drawing brush, the minimum length of the virtual drawing brush, the maximum length of the virtual drawing brush, the sampling interval of the virtual drawing brush, and the brush type of the virtual drawing brush.

[0108] In the embodiments of the present disclosure, a camera for collecting pictures may be provided on the electronic device, wherein the number of cameras provided is not limited, for example, it may be one, or it may be multiple. The form in which the camera is provided on the electronic device is not limited, for example, it may be a camera built into the electronic device, or it may be a camera external to the electronic device, for another example, it may be a front camera or a rear camera. The camera may be any type of camera, for example, the camera may be a color camera, a black and white camera, a depth camera, a telephoto camera, a wide-angle camera, etc., which are not limited here.

[0109] When the electronic device is provided with multiple cameras, the multiple cameras may be cameras of the same type or cameras of different types, and the present disclosure does not limit this. For example, they may all be color cameras or all be black and white cameras, or one of the cameras may be a telephoto camera and the other cameras may be wide-angle cameras, and so on.

[0110] In the disclosed embodiment, the image style transfer device can detect user operations, and in response to the user operations, obtain the original image, and the image style transfer device can also obtain initial drawing parameters. For example, a drawing parameter adjustment tool can be provided, and the designer can open the drawing parameter adjustment tool and select or set the corresponding initial drawing parameters on the tool interface. Accordingly, the image style transfer device can obtain the initial drawing parameters that the designer has preset from the configuration information or background data.

[0111] In step S102, a plurality of pixel points in the original image are used as a plurality of sampling points, and the colors respectively corresponding to the plurality of sampling points are obtained.

[0112] In the disclosed embodiment, multiple sampling points can be determined in the original image according to a preset sampling interval, that is, the number of pixels between adjacent sampling points is the same. For example, if the marked sampling interval is T pixels, the number of pixels between adjacent sampling points is T.

[0113] In the disclosed embodiment, the pixel points in the original image can be determined at preset sampling intervals along the horizontal and vertical axes in the pixel coordinate system as corresponding sampling points. After all sampling points are determined, the color corresponding to each sampling point in the original image can be obtained, or after each sampling point is determined, the color corresponding to the sampling point in the original image can be obtained. The pixel coordinate system is a common pixel coordinate system for computer vision, the origin of the pixel coordinate system can be the upper left corner of the original image, the X axis is horizontal to the right, the Y axis is horizontal downward, and the unit is pixel.

[0114] In step S103, a plurality of mapping points respectively corresponding to the plurality of sampling points are obtained on the virtual drawing canvas.

[0115] In the embodiments of the present disclosure, the size of the virtual drawing canvas may be the same or consistent with the size of the original image, or the size of the virtual drawing canvas may be different from the size of the original image. For example, the size of the virtual drawing canvas may be a set multiple of the size of the original image, wherein the set multiple may be an integer or a floating point number, and the present disclosure does not impose any limitation on this.

[0116] In the embodiment of the present disclosure, after a plurality of sampling points in the original image are determined, for each sampling point, a mapping point corresponding to the sampling point in the virtual drawing canvas may be determined.

[0117] As an example, when the size of the virtual drawing canvas is the same or consistent with the size of the original picture, for each sampling point, the corresponding mapping point in the virtual drawing canvas can be determined according to the position of the sampling point in the original picture. Among them, the position of the mapping point in the virtual drawing canvas is the same as the position of the corresponding sampling point in the original picture. Thus, a virtual drawing canvas of the same size as the original picture can be obtained, so that a target picture of the same size as the original picture can be obtained later, so that the generated target picture can meet the actual drawing needs of the user. In addition, the size of the virtual drawing canvas is consistent with the size of the original picture, which can also facilitate calculation, thereby improving the drawing efficiency of the target picture.

[0118] As another example, when the size of the virtual drawing canvas is different from the size of the original image, for example, when the size of the virtual drawing canvas is a set multiple of the size of the original image, for each sampling point, the corresponding mapping point in the virtual drawing canvas can also be determined according to the position of the sampling point in the original image. For example, if the length and width of the virtual drawing canvas are both set multiples of the length and width of the original image, the position of the mapping point in the virtual drawing canvas can be a set multiple of the position of the corresponding sampling point in the original image. In this way, a virtual drawing canvas of a different size from the original image can be obtained, so that a target image of a different size from the original image can be obtained later, thereby improving the applicability of the method.

[0119] It should be noted that, in order to avoid reducing the effect of image style transfer, the color of the virtual drawing canvas should not be too bright. For example, the color of the virtual drawing canvas can be simple colors such as black and white.

[0120] In step S104, a virtual drawing brush is generated according to the initial drawing parameters and the colors corresponding to the plurality of sampling points.

[0121] In the disclosed embodiment, a virtual drawing brush may be generated according to the initial drawing parameters and the colors corresponding to the multiple sampling points. For example, a virtual drawing brush may be generated according to the brush radius and the drawing brush type of the drawing brush and the colors corresponding to the multiple sampling points.

[0122] In step S105, based on multiple mapping points, a virtual drawing brush is used to draw on the virtual drawing canvas to generate a target image after style transfer, wherein the drawing process includes multiple drawing stages, in each drawing stage, a corresponding virtual drawing brush is used to draw on the virtual drawing canvas, and in each drawing stage, drawing is performed based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method.

[0123] In the embodiment of the present disclosure, the drawing process includes multiple drawing stages. In each drawing stage, a virtual drawing brush corresponding to the drawing stage is used to draw at multiple mapping points on a virtual drawing canvas, and in each drawing stage, the drawing stage is drawn based on the picture obtained by drawing in the previous drawing stage. The number of drawing stages is preset. For example, the number of drawing stages can be preset by a built-in program of the electronic device, or the number of drawing stages can also be set by the designer according to actual design requirements. For example, the number of drawing stages can be included in the initial drawing parameters, and the present disclosure does not limit this. Optionally, the number of marking drawing stages is N, where N is a positive integer greater than 1. For example, the number of drawing stages can be 4, 5, 6, and so on.

[0124] In the embodiment of the present disclosure, the virtual drawing brush of each drawing stage changes according to a preset change mode, wherein the preset change mode corresponding to each drawing stage can be set according to actual design requirements, and the preset change mode corresponding to each drawing stage can be the same or different, and the present disclosure does not limit this. For example, the preset change mode can be to gradually reduce the brush radius of the virtual drawing brush, gradually increase the brush radius of the virtual drawing brush, the virtual drawing brush shapes corresponding to the foreground area and the background area are different, the virtual drawing brush radius is gradually reduced within the preset range of the central pixel point, and the virtual drawing brush radius is gradually increased outside the preset range, the virtual drawing brush radius is reduced in the foreground area, and the virtual drawing brush radius is kept unchanged in the background area, etc., and the present disclosure does not limit this.

[0125] It should be understood that the color corresponding to each sampling point may be different. Therefore, when a virtual drawing brush is generated and used to draw at each mapping point, the drawing color corresponding to each mapping point of the virtual drawing brush may be different. In the embodiment of the present disclosure, in each drawing stage, when a virtual drawing brush is used to draw at multiple mapping points on a virtual drawing canvas, the drawing color of the virtual drawing brush is the same as the color of the sampling point corresponding to the mapping point.

[0126] In a possible implementation of the embodiment of the present disclosure, in order to make the target image have similar content to the original image, in each drawing stage, when using the virtual drawing brush to draw at each mapping point on the virtual drawing canvas, the initial starting point can be determined. For example, a mapping point can be randomly selected from multiple mapping points as the initial starting point, or the mapping point corresponding to the upper left corner of the virtual drawing canvas can be used as the initial starting point, or other mapping points can be selected as the initial starting point, and the present disclosure does not limit this. Then, the sampling point corresponding to the initial starting point can be determined, and the color gradient direction near the sampling point can be determined. According to the color gradient direction, the next sampling point to be drawn is determined, so that the virtual drawing brush can draw at the mapping point corresponding to the next sampling point to be drawn. Afterwards, when drawing at each mapping point, the sampling point corresponding to each mapping point can continue to be determined, and the color gradient direction near each sampling point can be determined. According to the color gradient direction, the next sampling point to be drawn is determined, so that the virtual drawing brush can draw at the mapping point corresponding to the next sampling point to be drawn.

[0127] That is to say, in the present disclosure, when drawing at each mapping point, the next sampling point to be drawn can be determined according to the sampling point corresponding to the mapping point and the color gradient direction near the sampling point in the original image, so that the virtual drawing brush can be used to draw at the mapping point corresponding to the next sampling point to be drawn, thereby drawing according to the color gradient direction in the original image, so that the target image obtained by drawing has similar content to the original image. Moreover, when the virtual drawing brush draws at each mapping point, the brush center of the virtual drawing brush is the corresponding mapping point, and the virtual drawing brush can cover multiple pixel points, thereby, a complete image can be drawn.

[0128] It is understandable that, in each drawing stage, the larger the brush radius of the virtual drawing brush is, the larger the area covered by the virtual drawing brush is, the more pixels with the same or similar colors are, the lower the degree of restoration of the original image is, and the more abstract the target image is. Therefore, as a possible implementation method of the embodiment of the present disclosure, the preset change method can be to gradually reduce or sequentially reduce the brush radius of the virtual drawing brush.

[0129] As an example, when the preset change method is to gradually reduce the brush radius of the virtual drawing brush, for the virtual drawing brush in each drawing stage, the initial value of the brush radius corresponding to the virtual drawing brush in the drawing stage can be determined, so that when the virtual drawing brush is controlled to draw based on the picture drawn in the previous drawing stage, the brush radius of the virtual drawing brush can be controlled to gradually decrease from the corresponding initial value according to the preset change method to obtain the picture drawn in the drawing stage.

[0130] The initial value of the brush radius of the virtual drawing brush in each drawing stage may be a fixed value, or the initial value of the brush radius of the virtual drawing brush in each drawing stage may also be determined according to the brush radius of the virtual drawing brush in the previous drawing stage. For example, the initial value of the brush radius of the virtual drawing brush in each drawing stage may be determined according to the brush radius of the virtual drawing brush in the previous drawing stage and a preset brush radius reduction value, or according to the brush radius of the virtual drawing brush in the previous drawing stage and a preset brush radius increase value, and the present disclosure does not impose any limitation on this.

[0131] For example, when the initial value of the brush radius of the virtual drawing brush in each drawing stage is determined according to the brush radius of the virtual drawing brush in the previous drawing stage, in order to improve the drawing effect of the oil painting, the initial value of the brush radius corresponding to the drawing brush of the virtual drawing brush in the second drawing stage can be smaller than the brush radius corresponding to the virtual drawing brush in the first drawing stage, the initial value of the brush radius corresponding to the virtual drawing brush in the third drawing stage can be smaller than the initial value of the brush radius corresponding to the virtual drawing brush in the second drawing stage, and so on. In this way, the degree of restoration of the original image can be improved and the generation effect of the target image can be improved.

[0132] For example, after drawing the picture obtained in the first drawing stage, the brush radius of the virtual drawing brush can be reduced, and the virtual drawing brush with the reduced brush radius can be controlled to draw on the picture obtained in the first drawing stage according to a preset change method to obtain the picture obtained in the second drawing stage, and then the brush radius of the virtual drawing brush can be reduced again, and the virtual drawing brush with the reduced brush radius can be controlled to draw on the picture obtained in the second drawing stage according to a preset change method to obtain the picture obtained in the third drawing stage, and so on, the picture obtained in the Nth drawing stage can be obtained, and the picture obtained in the Nth drawing stage can be used as the target picture after style transfer. Therefore, by simulating the real user painting oil paintings in multiple layers, the target picture can be obtained by performing multiple layers of painting according to the original picture on a blank virtual drawing canvas, and a close-to-real oil painting effect can be obtained.

[0133] That is to say, in the present disclosure, for the 2nd, 3rd, ..., Nth drawing stages, when using a virtual drawing brush to draw on the image drawn in the previous drawing stage, the brush radius of the virtual drawing brush in each drawing stage can be dynamically adjusted according to a preset change method to meet actual drawing needs.

[0134] It should be noted that the preset change modes corresponding to the virtual drawing brushes in each drawing stage may be the same or different, and the present disclosure does not impose any limitation on this. For ease of explanation, the above example only takes the preset change modes corresponding to the virtual drawing brushes in each drawing stage as the same.

[0135] As another example, when the preset change mode is to gradually reduce the brush radius of the virtual drawing brush, when the virtual drawing brush of each drawing stage draws on the picture drawn in the previous drawing stage, it can also draw with a fixed brush radius. Specifically, after drawing the picture drawn in the first drawing stage, the brush radius of the virtual drawing brush can be reduced, and the virtual drawing brush with the reduced brush radius (i.e., the virtual drawing brush drawn in the second drawing stage) is used to draw on the picture drawn in the first drawing stage to obtain the picture drawn in the second drawing stage, and then the brush radius of the virtual drawing brush is reduced again, and the virtual drawing brush with the reduced brush radius (i.e., the virtual drawing brush in the third drawing stage) is used to draw on the picture drawn in the second drawing stage to obtain the picture drawn in the third drawing stage, and so on, the picture drawn in the Nth drawing stage can be obtained, and the picture drawn in the Nth drawing stage is used as the final target picture. Thus, by simulating the real user painting oil paintings in multiple layers, on a blank virtual drawing canvas, multiple layers of drawing are performed according to the original picture to obtain the target picture, and a close-to-real oil painting effect can be obtained.

[0136] The brush radius that is reduced each time may be a fixed value, or, in order to improve the applicability of the method, the brush radius that is reduced each time may also be a non-fixed value. For example, the brush radius of the virtual drawing brush in the i+1th drawing stage may be a preset multiple of the brush radius of the virtual drawing brush in the i-th drawing stage, wherein the preset multiple is a floating point number less than 1, and the present disclosure does not impose any restrictions on this.

[0137] It should be noted that the above is only an example of gradually reducing the brush radius of the virtual drawing brush in a preset change manner. In actual application, the preset change manner can be set according to actual design requirements, and the present disclosure does not limit this.

[0138] For example, when the preset change method is to gradually increase the brush radius of the virtual drawing brush, the initial value of the corresponding brush radius can be determined for the virtual drawing brush in the second drawing stage, so that when the virtual drawing brush in the second drawing stage is controlled to draw on the picture drawn in the first drawing stage, the brush radius of the virtual drawing brush in the second drawing stage can be controlled to gradually increase from the corresponding initial value according to the preset change method to obtain the picture drawn in the second drawing stage. Similarly, for the virtual drawing brush of the third drawing stage, the initial value of the corresponding brush radius can be determined. When the virtual drawing brush of the third drawing stage is controlled to draw on the picture drawn in the second drawing stage, the brush radius of the virtual drawing brush of the third drawing stage can be controlled to gradually increase from the corresponding initial value according to a preset change method to obtain the picture drawn in the third drawing stage. Similarly, for the virtual drawing brush of the Nth drawing stage, the initial value of the corresponding brush radius can be determined. When the virtual drawing brush of the Nth drawing stage is controlled to draw on the picture drawn in the N-1th drawing stage, the brush radius of the virtual drawing brush of the Nth drawing stage can be controlled to gradually increase from the corresponding initial value according to a preset change method to obtain the picture drawn in the Nth drawing stage.

[0139] As an example, the original image can be Figure 2a As shown, taking the number of drawing stages N as 5 and the color of the virtual drawing canvas as black as an example, the pictures are drawn in sequence in each drawing stage, that is, the pictures drawn in the first drawing stage, the pictures drawn in the second drawing stage, the pictures drawn in the third drawing stage, the pictures drawn in the fourth drawing stage, and the pictures drawn in the fifth drawing stage can be as shown in FIG. Figure 2b As shown, the picture drawn in the fifth drawing stage can be used as the final target picture after style transfer.

[0140] The image style transfer method of the disclosed embodiment uses a plurality of pixel points in the acquired original image as a plurality of sampling points, and obtains the colors corresponding to the plurality of sampling points, then obtains a plurality of mapping points corresponding to the plurality of sampling points on a virtual drawing canvas, generates a virtual drawing brush based on the acquired initial drawing parameters and the colors corresponding to the plurality of sampling points, and uses the virtual drawing brush to draw on the virtual drawing canvas based on the plurality of mapping points to generate a target image after style transfer, wherein the drawing process includes a plurality of drawing stages, in each drawing stage, drawing is performed on the virtual drawing canvas using the corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change mode. Thus, the user only needs to input an original image to obtain the corresponding target image after style transfer, without selecting an image with the target style, which can simplify the user operation, and through the multi-layer simulation of a real user painting oil painting, on a blank virtual drawing canvas, multi-layer drawing is performed based on the original image to obtain the target image after style transfer, which can obtain a close-to-real oil painting effect, improve the degree of restoration of the original image, and improve the image style transfer effect.

[0141] It should be noted that the drawing process of the virtual drawing brush is similar to the classic graphics rendering algorithm in computer graphics, so the rendering pipeline based on the graphics processing unit (GPU) shader can be reused to perform parallel calculations on each pixel of the output result. In other words, the color of each pixel of the output result depends only on the input image and the generated virtual drawing brush, and has nothing to do with other pixels of the output result. Therefore, as a possible implementation method of the embodiment of the present disclosure, in each drawing stage, when the virtual drawing brush is used to draw at each mapping point on the virtual drawing canvas, multiple mapping points can be drawn in parallel in the GPU to improve the drawing efficiency of the target image.

[0142] In a possible implementation of the embodiment of the present disclosure, in order to further enhance the oil painting effect of the target image, the parameters of the virtual painting brush may be adjusted according to the painting path. The above process is described in detail below in conjunction with the second embodiment.

[0143] Figure 3 is a flowchart of an image style transfer method shown according to a second exemplary embodiment.

[0144] like Figure 3 As shown, in Figure 1 Based on the illustrated embodiment, after step S104, the image style transfer method may further include the following steps:

[0145] In step S201, a plurality of adjacent pixel points of a plurality of sampling points in an original image are obtained.

[0146] In the embodiment of the present disclosure, for each sampling point in the original image, a plurality of adjacent pixel points adjacent to the sampling point may be determined.

[0147] In step S202, a drawing path corresponding to a sampling point is generated according to a plurality of adjacent pixel points.

[0148] In the disclosed embodiment, the drawing path may be the direction of lines in the original image, such as the contours of the facial features or face area, the direction of hair, etc. The drawing path corresponding to each sampling point may be determined according to the color gradient direction in the original image.

[0149] In step S203, the parameters of the virtual drawing brush are adjusted according to the drawing path.

[0150] In the disclosed embodiment, after determining the drawing path corresponding to each sampling point, the parameters of the virtual drawing brush can be adjusted according to the drawing path. For example, the shape, brush radius and other parameters of the virtual drawing brush can be adjusted according to the drawing path corresponding to each sampling point.

[0151] For example, the initial image displayed by the virtual drawing brush on the virtual drawing canvas can be determined based on the brush radius and brush type of the virtual drawing brush. When the drawing path is different, the initial image displayed by the virtual drawing brush on the virtual drawing canvas will change. At this time, the shape and brush radius of the virtual drawing brush are changed.

[0152] As an example, see Figure 4 , there are 17 types of virtual drawing brushes in area 41. Assuming that the brush type of the virtual drawing brush in the initial drawing parameters selected by the designer is the first one from the left in area 41, after the brush type is selected, the graphic that the virtual drawing brush spreads along the drawing path on the virtual drawing canvas can be determined. For example, when the drawing path is a straight line, the graphic that the virtual drawing brush spreads along the drawing path on the drawing canvas can be an elliptical virtual drawing brush, and when the drawing path is a curve (such as the direction of a hair), the graphic that the virtual drawing brush spreads along the drawing path on the drawing canvas can be a virtual drawing brush with a certain width and arcs at both ends.

[0153] If the designer does not select a brush type, Figure 4Among the 17 brush types shown, a brush type is randomly selected as the brush type of the virtual drawing brush. It should be noted that the above examples are only exemplary and cannot be regarded as limitations of the present disclosure. In actual use, the brush type is not limited to Figure 4 The 17 types shown in the figure can be modified and other optional brush types or shapes can be added according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0154] In a possible implementation of the embodiment of the present disclosure, for each sampling point, the pixel difference between the sampling point and multiple adjacent pixel points can be determined, and the adjacent pixel point with the largest pixel difference is selected from the multiple adjacent pixel points as the target pixel point, so that the drawing path corresponding to the sampling point can be generated according to the sampling point and the target pixel point. Thus, the drawing path corresponding to the sampling point is determined according to the pixel point with the largest pixel difference, which can ensure the accuracy of the drawing path calculation result, so that the parameters of the virtual drawing brush are adjusted according to the drawing path, and the virtual drawing brush with adjusted parameters is used to draw on the virtual drawing canvas, which can ensure that the drawn target image has similar content to the original image, thereby improving the oil painting effect of the target image.

[0155] In a possible implementation of the disclosed embodiment, in order to improve the applicability of the method and make the drawing effect of the target image more realistic, a mask area can be set in the virtual drawing canvas, and whether to draw the mask area is determined according to the designer's design requirements. If not, the mask image of the corresponding area in the original image is retained to highlight the picture details of the mask image and improve the drawing effect of the target image. The above process is described in detail below in conjunction with Example 3.

[0156] Figure 5 is a flowchart of an image style transfer method shown according to a third exemplary embodiment.

[0157] like Figure 5 As shown, the initial drawing parameters may also include a mask identifier. Figures 1 to 3 Based on the illustrated embodiment, the image style transfer method may further include the following steps:

[0158] In step S301, the original image is detected to obtain a target object and a target object type in the original image.

[0159] In the embodiment of the present disclosure, the target object may be the main object in the original image, and the main object may be in the foreground area or in the background area, and the present disclosure does not impose any limitation on this.

[0160] In the disclosed embodiment, the target object in the original image can be identified based on the target detection algorithm, and then the original image can be detected based on the object recognition model to determine the type of the target object in the original image. A large number of sample images with labeled object types can be obtained in advance to train the initial object recognition model, so that the trained object recognition model can learn the corresponding relationship between the image and the object type.

[0161] In step S302, a mask area of ​​the target object is determined according to the type of the target object.

[0162] In the embodiment of the present disclosure, when the target object type is different, the mask area of ​​the target object can be different. For example, in order to make the generated target image more vivid, when the target object type is a person, the mask area can be the eye area. Or, when the target object is a landscape, the mask area can be the background area of ​​the original image, and so on.

[0163] In the embodiments of the present disclosure, the correspondence between each target object type and the mask area can be preset. Therefore, in the present disclosure, when determining the target object type in the original image, the above correspondence can be queried to determine the corresponding mask area, which is simple to operate and easy to implement.

[0164] In step S303, a mask area is set in the virtual drawing canvas.

[0165] In the embodiment of the present disclosure, after the mask area is determined, the mask area can be set in the virtual drawing canvas. Specifically, the position and boundary information of the mask area in the original image can be determined, and the position and boundary of the mask area in the virtual drawing canvas can be determined based on the position and boundary information of the mask area in the original image, so that the mask area can be set in the virtual drawing canvas based on the position and boundary of the mask area in the virtual drawing canvas.

[0166] In step S304, drawing is performed in the mask area and in the non-mask area other than the mask area in the virtual drawing canvas using a drawing method that matches the mask identifier.

[0167] In the embodiment of the present disclosure, when the mask identifiers are different, the drawing methods in the mask area and the non-mask area other than the mask area in the virtual drawing canvas are different.

[0168] In a possible implementation of an embodiment of the present disclosure, when the mask identifier is a first identifier, a mask image corresponding to the mask area can be extracted from the original image, and the mask image is added to the mask area in the target image to generate a processed target image; wherein the first identifier is used to indicate that no drawing is performed in the mask area, and that drawing is performed in the non-mask area.

[0169] In the disclosed embodiment, when the mask identifier is the first identifier, drawing is performed only in the non-mask area, and no drawing is performed in the mask area. Specifically, the mask image corresponding to the mask area can be extracted from the original picture. For example, when the mask area is the eye area, the mask image corresponding to the eye can be identified and extracted from the original picture. Alternatively, when the mask area corresponds to the background area, the foreground area and the background area of ​​the original picture can be identified, and after the background area is identified, the mask image corresponding to the background area can be extracted from the original picture. After the mask image is extracted, the mask image can be added to the mask area in the target picture to generate a processed target picture. In this way, the picture details of the mask area in the target picture can be highlighted, and the drawing effect of the target picture can be further improved.

[0170] For example, when the target object type is a person, the mask area is the eye area, so when drawing, it is not necessary to draw in the eye area to preserve the picture details of the eye. It should be noted that the above examples are only exemplary and cannot be regarded as limitations of the present disclosure. In actual use, when the target object type is a person, the mask area can also be other facial features areas, such as the mouth area, or can also be the entire face area, etc., and the embodiments of the present disclosure do not limit this.

[0171] In another possible implementation of the embodiment of the present disclosure, when the mask identifier is the second identifier, a non-mask image corresponding to the non-mask area is extracted from the original image, and the non-mask image is added to the non-mask area in the target image to generate a processed target image; wherein the second identifier is used to indicate that drawing is to be performed in the mask area and that no drawing is to be performed in the non-mask area.

[0172] In the disclosed embodiment, when the mask identifier is the second identifier, drawing is performed only in the mask area, and no drawing is performed in the non-mask area. Specifically, a non-mask image corresponding to the non-mask area can be extracted from the original image. For example, when the mask area is the eye area, the non-mask image other than the eye can be identified and extracted from the original image. Alternatively, when the mask area corresponds to the background area, the foreground area and the background area can be identified for the original image, and after the background area is identified, the non-mask image (i.e., the image corresponding to the foreground area) other than the background area can be extracted from the original image. After the non-mask image is extracted, the non-mask image can be added to the non-mask area in the target image to generate a processed target image. Thus, the picture details of the non-mask area in the target image can be highlighted, further improving the drawing effect of the target image.

[0173] In another possible implementation of the embodiment of the present disclosure, when the mask identifier is a third identifier, a virtual drawing brush can be used to draw in the mask area and the non-mask area based on multiple mapping points to generate a target image; wherein the third identifier is used to indicate drawing in the mask area and drawing in the non-mask area, and the virtual drawing brush parameters in the mask area are different from the virtual drawing brush parameters in the non-mask area.

[0174] In the disclosed embodiment, when the mask identifier is the third identifier, drawing is performed in the mask area and the non-mask area at the same time. Furthermore, in order to highlight the picture details in the mask area, the virtual drawing brush parameters in the mask area are different from the virtual drawing brush parameters in the non-mask area. For example, the brush radius of the virtual drawing brush in the mask area can be smaller than the brush radius of the virtual drawing brush in the non-mask area. Specifically, when drawing, based on multiple mapping points, a virtual drawing brush can be used to draw in the mask area and the non-mask area. For example, a virtual drawing brush can be used to draw in the non-mask area first, and then the brush radius of the virtual drawing brush can be reduced, and the virtual drawing brush with the reduced brush radius can be used to draw in the mask area to generate a target image.

[0175] As an example, the mask identifier may be Figure 4 The face mask shown can take values ​​of 0, 1, and 2, where the face mask is set to 0, indicating that no mask is used when drawing the oil painting, and the oil painting is drawn on the entire image; the face mask is set to 1, indicating that the oil painting is drawn outside the mask area, and the corresponding mask image in the original image is retained within the mask area; the face mask is set to 2, indicating that the oil painting is drawn within the mask area, and the non-mask image in the original image except the mask image is retained outside the mask area.

[0176] It should be noted that the above disclosure only uses the first identifier as 1, the second identifier as 2, and the third identifier as 0 as an example. In actual application, the corresponding values ​​of each identifier can be set according to design requirements and preferences. The disclosure does not impose any restrictions on this. For example, the first identifier can also be character A, the second identifier can be character B, and the third identifier can be character C.

[0177] Figure 4The texture in is the texture of the virtual drawing brush, which is designed by the designer in advance according to preferences and requirements as a reusable virtual drawing brush shape map. The sampling interval indicates that the pixels on the original image are sampled according to the sampling interval to determine the corresponding sampling points, so that when the virtual drawing brush draws on the virtual drawing canvas, it draws with the virtual drawing brush every sampling interval to ensure that the virtual brush covers the entire virtual drawing canvas as much as possible, and the uncovered area will show the background color of the virtual drawing canvas, and there will be no oil painting effect, thereby ensuring that the output target image has content similar to the original image.

[0178] It should be noted that the foreground area and the background area in the original image can be determined according to the following method:

[0179] In the first way, the electronic device may also include a camera capable of collecting depth information, such as a time of flight (TOF) camera, a structured light camera, an RGB-D (Depth) camera, and other depth cameras. When collecting the original picture, the depth camera can be controlled to synchronously collect the depth map, wherein the depth map is used to indicate the depth information corresponding to each pixel in the original picture, so that the foreground area and background area in the original picture can be determined according to the depth map. Generally speaking, when the depth information indicates that the photographed object is close to the plane where the camera is located and the depth value is small, it can be determined that the object is the foreground, otherwise, the object is the background. Then, the foreground area and background area of ​​the original picture can be determined based on the depth information of each object in the depth map.

[0180] In the second method, the electronic device may include a main camera and a secondary camera, and the original image is captured by one of the main camera and the secondary camera, and the test image is captured by the other of the main camera and the secondary camera, so that the depth information of the original image can be determined based on the test image and the original image, and the foreground area and the background area can be determined based on the depth information of the original image.

[0181] Specifically, due to the certain distance between the main camera and the secondary camera, the two cameras have parallax, and the images taken by different cameras should be different. According to the principle of triangulation, the depth information of the same object in the original image and the test image can be calculated, that is, the distance of the object from the plane where the main camera and the secondary camera are located. After calculating the depth information of the original image, it is possible to determine whether the object is the foreground or the background based on the depth information of each object in the original image, and then the foreground area and background area in the original image can be obtained.

[0182] Of course, the foreground area and the background area in the original image may also be identified according to other algorithms, and the present disclosure does not limit this.

[0183] In a possible implementation of the embodiment of the present disclosure, in order to better generate the effect of thickness and layering of oil painting brushstrokes, in the present disclosure, when drawing the target image in each drawing stage, it is also possible to synchronously draw in the virtual texture canvas according to the drawing operation in the virtual drawing canvas to generate a texture image, so that the target image and the texture image can be merged to generate a processed target image. The above process is described in detail below in conjunction with Example 4.

[0184] Figure 6 is a flowchart of an image style transfer method according to a fourth exemplary embodiment.

[0185] like Figure 6 As shown, in Figures 1 to 5 Based on the illustrated embodiment, the image style transfer method may further include the following steps:

[0186] In step S401, according to the drawing operation in the virtual drawing canvas, drawing is synchronously performed in the virtual texture canvas to generate a texture image; wherein the drawing process includes multiple drawing stages, in each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method.

[0187] In the disclosed embodiment, the size of the virtual texture canvas may be consistent with the size of the virtual drawing canvas to facilitate the subsequent image fusion process.

[0188] In the disclosed embodiment, the drawing process of the texture image also includes multiple drawing stages, and the number of drawing stages is the same as the number of drawing stages corresponding to the target image, which is N. Drawing can be performed synchronously in the virtual texture canvas according to the drawing operation in the virtual drawing canvas to generate a texture image.

[0189] For example, when drawing with an initial starting point in a virtual drawing canvas, the next drawing sampling point is determined based on the sampling points corresponding to each mapping point and the color gradient direction in the original image, so that the virtual drawing brush can be used to draw at the mapping point corresponding to the next drawing sampling point. Therefore, in the present disclosure, the initial position point of the initial starting point in the virtual texture canvas can be determined, and multiple position points corresponding to each mapping point in the virtual texture canvas can be determined. Therefore, when drawing with the initial position point in the virtual texture canvas, the next position point to be drawn can also be determined based on the mapping point corresponding to the next drawing sampling point, and the virtual drawing brush can be used to draw at the next position point to be drawn.

[0190] As an example, taking N as 5, the texture image drawn in the first drawing stage, the texture image drawn in the second drawing stage, the texture image drawn in the third drawing stage, the texture image drawn in the fourth drawing stage, and the texture image drawn in the fifth drawing stage can be as follows: Figure 7 As shown. Among them, Figure 7 Take the virtual texture canvas as an example with black color. The initial virtual texture canvas has no protrusions on the surface. Each time a virtual drawing brush is drawn, the thickness is accumulated in the coverage area of ​​the virtual drawing brush according to the brush shape, and the result is as follows: Figure 7 The texture maps corresponding to each drawing stage are shown. Figure 7 In the , the brighter the area, the more raised the virtual texture canvas is.

[0191] In step S402, the target image and the texture image are fused to generate a processed target image.

[0192] In the embodiment of the present disclosure, after the texture image is drawn, the target image and the texture image can be merged to generate a processed target image. Figure 7 The texture image drawn in the fifth drawing stage is the same as Figure 2b The pictures drawn in the fifth drawing stage are fused to generate Figure 8 Target image shown.

[0193] Specifically, the color of the target image can be perturbed according to the texture image, and then the result of the color perturbation can be superimposed on the target image to replace the color of the target image to obtain the fused target image. Among them, the color perturbation algorithm simulates the three-dimensional shadow of a real oil painting, that is, according to the local convexity of the virtual texture painting represented by the texture image, the shadow of the canvas under a selected lighting direction is calculated. For example, if the convexity corresponding to the pixels around pixel A is higher, the pixel A will be blocked by the surrounding pixels from the incident light and cast a shadow, so the color value of the pixel A will become darker.

[0194] The image style transfer method of the disclosed embodiment generates a processed target image by fusing the target image and the texture image, so that the fused target image can have a good sense of thickness and layering of oil painting brushstrokes, further improving the oil painting effect of the target image.

[0195] In a possible implementation of the disclosed embodiment, when a virtual drawing brush draws on a virtual texture canvas, the texture of each drawing point can be generated according to the height model corresponding to the virtual drawing brush, thereby obtaining a texture image according to the texture of each drawing point. The above process is described in detail below in conjunction with Example 5.

[0196] Fig. 9 is a flowchart of an image style transfer method shown according to a fifth exemplary embodiment.

[0197] like Fig. 9 As shown, in Figure 6 Based on the illustrated embodiment, step S401 may specifically include the following steps:

[0198] In step S501, a plurality of position points respectively corresponding to a plurality of mapping points are obtained in the virtual texture canvas.

[0199] In the disclosed embodiment, for each mapping point, the position point corresponding to the mapping point in the virtual texture canvas can be determined according to the position of the mapping point on the virtual drawing canvas, wherein the position of the mapping point in the virtual drawing canvas is the same as the position of the corresponding position point in the virtual texture canvas.

[0200] In step S502 , in each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush based on a plurality of position points.

[0201] In the disclosed embodiment, at each drawing stage, drawing can be performed on the virtual texture canvas with a corresponding virtual drawing brush based on multiple position points. For example, at each drawing stage, drawing can be performed on the virtual drawing canvas with an initial starting point, and the next drawing sampling point can be determined according to the sampling point corresponding to each mapping point and the color gradient direction in the original image, so that the virtual drawing brush can be used to draw at the mapping point corresponding to the next drawing sampling point.

[0202] Similarly, in each drawing stage, the initial position point of the initial starting point in the virtual texture canvas can be determined, and multiple position points corresponding to each mapping point in the virtual texture canvas can be determined. Therefore, when drawing in the virtual texture canvas with the initial position point, the next position point to be drawn can also be determined according to the mapping point corresponding to the next drawing sampling point, and the virtual drawing brush can be used to draw at the next position point to be drawn.

[0203] In step S503, at a plurality of position points, textures of corresponding position points are generated according to a height model corresponding to the virtual drawing brush.

[0204] In the disclosed embodiment, the height model corresponding to the virtual drawing brush is the height distribution corresponding to the virtual drawing brush. When drawing at multiple positions, the texture of the corresponding position can be generated according to the height model corresponding to the virtual drawing brush, wherein the texture of each position is another dimension in the virtual drawing canvas in addition to the RGB color value, that is, the convexity or thickness of the surface of the virtual drawing canvas. Intuitively, the more paint is applied by the virtual drawing brush, the thicker the virtual drawing canvas is.

[0205] In step S504, a texture image corresponding to the drawing stage is generated according to the textures of the plurality of position points.

[0206] In the disclosed embodiment, a texture image of a corresponding drawing stage may be generated according to the textures of a plurality of position points.

[0207] It should be understood that the virtual drawing brush has a certain brush radius, and it does not cover only one pixel, but covers multiple pixels. Therefore, on the virtual texture canvas, not only multiple position points have textures, but also the surrounding pixels of the multiple position points have textures. When the sampling interval between multiple sampling points is small, the virtual drawing brush can cover the entire virtual texture canvas and the virtual drawing canvas, thereby drawing a complete target image and texture image.

[0208] The image style transfer method of the disclosed embodiment obtains multiple position points corresponding to multiple mapping points in the virtual texture canvas, and in each drawing stage, draws on the virtual texture canvas with a corresponding virtual drawing brush based on the multiple position points; wherein, in each drawing stage, at multiple position points, the texture of the corresponding position points is generated according to the height model corresponding to the virtual drawing brush, and the texture image of the corresponding drawing stage is generated according to the texture of the multiple position points. Therefore, by generating the texture of the corresponding position points according to the height model corresponding to the virtual drawing brush, the reliability of the texture image generation results of each drawing stage can be improved, thereby improving the oil painting effect of the subsequent target image.

[0209] In a possible implementation of the embodiment of the present disclosure, a corresponding texture image can be generated according to the drawing operation at each drawing stage, and the target image and the texture image can be fused at each drawing stage. Specifically, for each drawing stage, the color of the target image can be disturbed according to the texture image of the drawing stage, and then the result of the color disturbance can be superimposed on the target image to replace the color of the target image to obtain a fused target image. Thus, in each drawing stage, the target image obtained by drawing is processed so that the fused target image has a better sense of thickness and layering of the oil painting brushstrokes, thereby improving the oil painting effect of the target image at each drawing stage.

[0210] As an example, the user can input an original image, and the designer can pre-process it by Figure 4 The tool interface shown has selected the corresponding initial drawing parameters. The image style transfer device can generate multiple layers of virtual drawing brushes along the local direction of the original image according to the original image and the initial drawing parameters, and use the generated virtual drawing brushes to draw layer by layer starting from a blank virtual drawing canvas (black by default), and superimpose the drawing results, thereby simulating the real process of hand-painted oil paintings to obtain a target image with an oil painting style.

[0211] The color of the virtual drawing brush corresponding to each layer or each drawing stage is the color of the original image at the center of the virtual drawing brush, and the shape of each virtual drawing brush can be determined by the thickness of the virtual drawing brush (brush radius), the drawing path and the selected brush type. The virtual drawing brush corresponding to each layer or each drawing stage is randomly sampled and drawn on the entire drawing canvas at a certain sampling interval to ensure that the virtual drawing brush covers the entire virtual drawing canvas as much as possible, thereby ensuring that the output image has content similar to the original image.

[0212] If you want to achieve a three-dimensional shadow effect, you can draw the target image in the virtual drawing canvas and draw on the virtual texture canvas at the same time to generate a texture map, which is used to indicate the height texture or convexity of the target image. In addition, in order to highlight the details of the eye area of ​​the portrait, a mask algorithm based on facial key points can be introduced to perform special operations on the drawing of the eye area.

[0213] Among them, the configurable global parameters of oil painting stylization may include: the actual width of the virtual drawing canvas and the virtual texture canvas, the density of the virtual drawing brush (the larger the canvas, the more virtual drawing brushes, and the richer the details of the drawing result), whether to turn on the shadow effect, the shadow granularity, etc.

[0214] The virtual painting brush parameters of each layer or each painting stage that can be configured for oil painting stylization may include: brush radius, minimum length of the virtual painting brush, maximum length of the virtual painting brush (wherein the maximum length should be greater than or equal to the minimum length), sampling interval of the virtual painting brush, and method of using face mask (e.g. Figure 4 0 means drawing the entire image without using a mask; 1 means drawing only outside the mask area; 2 means drawing only within the mask area), brush type number, etc.

[0215] Therefore, designers can adjust various parameters in advance, and intuitively and interactively adjust and control the style of the output image so that the style of the output image can meet the actual design needs. In addition, various parameters can be set according to actual needs to improve the applicability of this method and meet different design needs. In addition, different oil painting texture details can be obtained by selecting different brush types. Through efficient parallel computing with GPU shader, the target image can be quickly drawn.

[0216] In order to implement the above embodiment, the present disclosure proposes an image style transfer device.

[0217] Fig.10 is a block diagram of an image style transfer apparatus according to a sixth exemplary embodiment.

[0218] Reference Fig.10 The image style transfer device 100 may include: a first acquisition module 101, a processing module 102, a second acquisition module 103, a first generation module 104, and a first drawing module 105.

[0219] The first acquisition module 101 is configured to acquire the original image and initial drawing parameters.

[0220] The processing module 102 is configured to use a plurality of pixels in the original image as a plurality of sampling points, and obtain colors corresponding to the plurality of sampling points.

[0221] The second acquisition module 103 is configured to acquire a plurality of mapping points respectively corresponding to the plurality of sampling points on the virtual drawing canvas.

[0222] The first generating module 104 is configured to generate a virtual drawing brush according to the initial drawing parameters and the colors corresponding to the plurality of sampling points.

[0223] The first drawing module 105 is configured to execute drawing on a virtual drawing canvas based on multiple mapping points using a virtual drawing brush to generate a target image after style transfer, wherein the drawing process includes multiple drawing stages, in each drawing stage, drawing is performed on the virtual drawing canvas using a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on an image obtained by drawing in a previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method.

[0224] In a possible implementation form of the embodiment of the present disclosure, the image style transfer apparatus 100 may further include:

[0225] The third acquisition module is configured to acquire multiple adjacent pixel points of multiple sampling points in the original image.

[0226] The second generating module is configured to generate a drawing path corresponding to a sampling point according to a plurality of adjacent pixel points.

[0227] The adjustment module is configured to adjust the parameters of the virtual drawing brush according to the drawing path.

[0228] In a possible implementation form of the embodiment of the present disclosure, the second generation module is specifically configured to execute: respectively obtaining the pixel difference between the sampling point and multiple adjacent pixel points of the sampling point; selecting the adjacent pixel point with the largest pixel difference from the multiple adjacent pixel points as the target pixel point; and generating a drawing path according to the sampling point and the target pixel point.

[0229] In a possible implementation form of the embodiment of the present disclosure, the initial drawing parameters include a mask identifier, and the image style transfer apparatus 100 may further include:

[0230] The detection module is configured to perform detection on the original image to obtain the target object and the target object type in the original image.

[0231] The determination module is configured to determine a mask area of ​​the target object according to the type of the target object.

[0232] The setting module is configured to execute setting of a mask area in the virtual drawing canvas.

[0233] The second drawing module is configured to execute drawing in a drawing mode matching the mask identifier, and draw in the mask area and a non-mask area other than the mask area in the virtual drawing canvas.

[0234] In a possible implementation form of the embodiment of the present disclosure, the second drawing module is specifically configured to execute: when the mask identifier is the first identifier, extracting a mask image corresponding to the mask area from the original image; wherein the first identifier is used to indicate not to draw in the mask area and to draw in the non-mask area; adding the mask image to the mask area in the target image to generate a processed target image.

[0235] In a possible implementation form of an embodiment of the present disclosure, the second drawing module is specifically configured to execute: when the mask identifier is the second identifier, extracting a non-mask image corresponding to the non-mask area from the original image; wherein the second identifier is used to indicate drawing in the mask area and not drawing in the non-mask area; adding the non-mask image to the non-mask area in the target image to generate a processed target image.

[0236] In a possible implementation form of the embodiment of the present disclosure, the second drawing module is specifically configured to execute: when the mask identifier is the third identifier, based on multiple mapping points, drawing in the mask area and the non-mask area using a virtual drawing brush to generate a target image; wherein the third identifier is used to indicate drawing in the mask area and drawing in the non-mask area, and the virtual drawing brush parameters in the mask area are different from the virtual drawing brush parameters in the non-mask area.

[0237] In a possible implementation form of the embodiment of the present disclosure, the image style transfer apparatus 100 may further include:

[0238] The third drawing module is configured to execute drawing operations in the virtual drawing canvas and synchronously draw in the virtual texture canvas to generate a texture image; wherein the drawing process includes multiple drawing stages, in each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method.

[0239] The first fusion module is configured to perform fusion of the target image and the texture image to generate a processed target image.

[0240] In a possible implementation form of the embodiment of the present disclosure, the third drawing module is specifically configured to execute: obtaining multiple position points corresponding to multiple mapping points in the virtual texture canvas; in each drawing stage, drawing on the virtual texture canvas with a corresponding virtual drawing brush based on the multiple position points; generating textures of the corresponding position points at the multiple position points according to the height model corresponding to the virtual drawing brush; generating a texture image of the corresponding drawing stage according to the textures of the multiple position points.

[0241] In a possible implementation form of the embodiment of the present disclosure, the image style transfer apparatus 100 may further include:

[0242] The second fusion module is configured to generate a corresponding texture image according to the drawing operation at each drawing stage, and fuse the target image and the texture image at each drawing stage.

[0243] It should be noted that, regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0244] The image style transfer device of the disclosed embodiment uses a plurality of pixel points in the acquired original image as a plurality of sampling points, and obtains the colors corresponding to the plurality of sampling points, then obtains a plurality of mapping points corresponding to the plurality of sampling points on a virtual drawing canvas, generates a virtual drawing brush based on the acquired initial drawing parameters and the colors corresponding to the plurality of sampling points, and uses the virtual drawing brush to draw on the virtual drawing canvas based on the plurality of mapping points to generate a target image after style transfer, wherein the drawing process includes a plurality of drawing stages, in each drawing stage, drawing is performed on the virtual drawing canvas with a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change mode. Thus, the user only needs to input an original image to obtain the corresponding target image after style transfer, without selecting an image with a target style, which can simplify user operations, and by simulating a real user painting oil painting in multiple layers, on a blank virtual drawing canvas, multiple layers of drawing are performed according to the original image to obtain the target image after style transfer, which can obtain a close-to-real oil painting effect, improve the degree of restoration of the original image, and improve the image style transfer effect.

[0245] In order to implement the above embodiment, the embodiment of the present disclosure also proposes an electronic device.

[0246] Among them, electronic equipment includes:

[0247] processor;

[0248] a memory for storing processor-executable instructions;

[0249] The processor is configured to execute instructions to implement the image style transfer method as described above.

[0250] As an example, Fig.11 is a block diagram of an electronic device 200 for image style transfer according to an exemplary embodiment. Fig.11As shown, the electronic device 200 may further include:

[0251] The memory 210 and the processor 220, a bus 230 connecting different components (including the memory 210 and the processor 220), the memory 210 stores a computer program, and when the processor 220 executes the program, the image style transfer method described in the embodiment of the present disclosure is implemented.

[0252] Bus 230 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0253] The electronic device 200 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 200, including volatile and non-volatile media, removable and non-removable media.

[0254] The memory 210 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 240 and / or cache memory 250. The electronic device 200 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 260 may be used to read and write non-removable, non-volatile magnetic media ( Fig.11 not shown, usually called a "hard drive"). Although Fig.11 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 230 via one or more data medium interfaces. The memory 210 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0255] A program / utility 280 having a set (at least one) of program modules 270 may be stored, for example, in the memory 210, such program modules 270 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 270 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0256] The electronic device 200 may also communicate with one or more external devices 290 (e.g., keyboard, pointing device, display 291, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 200, and / or communicate with any device that enables the electronic device 200 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface 292. Furthermore, the electronic device 200 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) via a network adapter 293. As shown, the network adapter 293 communicates with other modules of the electronic device 200 via a bus 230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0257] The processor 220 executes various functional applications and data processing by running the programs stored in the memory 210 .

[0258] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the image style transfer method of the embodiment of the present disclosure, and will not be repeated here.

[0259] In order to implement the above embodiment, the embodiment of the present disclosure also proposes a storage medium.

[0260] When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image style transfer method as described above.

[0261] In order to implement the above embodiments, the present disclosure also provides a computer program product. When an instruction processor in the computer program product executes, the electronic device can perform the image style transfer method as described above.

[0262] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0263] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for image style transfer, characterized in that: include: Get the original image and initial drawing parameters; Using a plurality of pixel points in the original image as a plurality of sampling points, and obtaining colors respectively corresponding to the plurality of sampling points; Acquire a plurality of mapping points respectively corresponding to the plurality of sampling points on a virtual drawing canvas; Generate a virtual drawing brush according to the initial drawing parameters and the colors corresponding to the plurality of sampling points; Based on the multiple mapping points, the virtual drawing brush is used to draw on the virtual drawing canvas to generate a target image after style transfer, wherein the drawing process includes multiple drawing stages, in each drawing stage, the corresponding virtual drawing brush is used to draw on the virtual drawing canvas, and in each drawing stage, the drawing is based on the image obtained by drawing in the previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method; Wherein, the method further comprises: According to the drawing operation in the virtual drawing canvas, drawing is synchronously performed in the virtual texture canvas to generate a texture image; wherein the drawing process includes a plurality of drawing stages, in each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on an image obtained by drawing in a previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method; The number of drawing stages corresponding to the drawing process of the texture image is the same as the number of drawing stages corresponding to the drawing process of the target image; The target image and the texture image are merged to generate a processed target image.

2. The image style transfer method according to claim 1, characterized in that: After generating a virtual drawing brush according to the initial drawing parameters and the colors respectively corresponding to the plurality of sampling points, the method further includes: Acquire a plurality of adjacent pixel points of the plurality of sampling points in the original image; Generating a drawing path corresponding to the sampling point according to the plurality of adjacent pixel points; The parameters of the virtual drawing brush are adjusted according to the drawing path.

3. The image style transfer method according to claim 2, characterized in that: The step of generating a drawing path corresponding to the sampling point according to the plurality of adjacent pixel points includes: Respectively obtaining pixel differences between the sampling point and a plurality of adjacent pixel points of the sampling point; Selecting an adjacent pixel point with the largest pixel difference from the plurality of adjacent pixel points as a target pixel point; The drawing path is generated according to the sampling point and the target pixel point.

4. The image style transfer method according to claim 1, characterized in that: The initial drawing parameters include a mask identifier, and the method further includes: Detecting the original image to obtain a target object and a target object type in the original image; Determine a mask area of ​​the target object according to the type of the target object; Setting the mask area in the virtual drawing canvas; Drawing is performed in the mask area and in a non-mask area other than the mask area in the virtual drawing canvas using a drawing method that matches the mask identifier.

5. The image style transfer method according to claim 4, characterized in that: The drawing method matching the mask identifier is used to draw in the mask area and a non-mask area other than the mask area in the virtual drawing canvas, including: When the mask identifier is a first identifier, extracting a mask image corresponding to the mask area from the original image; wherein the first identifier is used to indicate not to draw in the mask area and to draw in the non-mask area; The mask image is added to the mask area in the target image to generate a processed target image.

6. The image style transfer method according to claim 4, characterized in that: The drawing method matching the mask identifier is used to draw in the mask area and a non-mask area other than the mask area in the virtual drawing canvas, including: When the mask identifier is a second identifier, extracting a non-mask image corresponding to the non-mask area from the original image; wherein the second identifier is used to indicate to draw in the mask area and not to draw in the non-mask area; The non-mask image is added to the non-mask area in the target image to generate a processed target image.

7. The image style transfer method according to claim 4, characterized in that: The drawing method matching the mask identifier is used to draw in the mask area and a non-mask area other than the mask area in the virtual drawing canvas, including: When the mask identifier is the third identifier, based on the multiple mapping points, the virtual drawing brush is used to draw in the mask area and the non-mask area to generate the target image; wherein the third identifier is used to indicate drawing in the mask area and drawing in the non-mask area, and the virtual drawing brush parameters in the mask area are different from the virtual drawing brush parameters in the non-mask area.

8. The image style transfer method according to claim 1, characterized in that: The step of synchronously drawing in a virtual texture canvas to generate a texture image according to the drawing operation in the virtual drawing canvas includes: Acquire a plurality of position points respectively corresponding to the plurality of mapping points in the virtual texture canvas; In each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush based on the multiple position points; At the plurality of position points, generating textures corresponding to the position points according to a height model corresponding to the virtual drawing brush; A texture image corresponding to the drawing stage is generated according to the textures of the multiple position points.

9. The image style transfer method according to any one of claims 1 to 6, characterized in that: The method further comprises: In each rendering stage, a corresponding texture image is generated according to the rendering operation, and the target image and the texture image are fused in each rendering stage.

10. An image style transfer device, characterized in that: include: A first acquisition module is configured to acquire an original image and initial drawing parameters; A processing module is configured to use a plurality of pixels in the original image as a plurality of sampling points, and obtain colors respectively corresponding to the plurality of sampling points; A second acquisition module is configured to acquire a plurality of mapping points respectively corresponding to the plurality of sampling points on the virtual drawing canvas; A first generating module is configured to generate a virtual drawing brush according to the initial drawing parameters and the colors respectively corresponding to the plurality of sampling points; A first drawing module is configured to execute drawing on the virtual drawing canvas using the virtual drawing brush based on the multiple mapping points to generate a target image after style transfer, wherein the drawing process includes multiple drawing stages, in each drawing stage, drawing is performed on the virtual drawing canvas using a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on an image obtained by drawing in a previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method; Wherein, the device further comprises: A third drawing module is configured to execute drawing operations in the virtual drawing canvas and synchronously draw in the virtual texture canvas to generate a texture image; wherein the drawing process includes a plurality of drawing stages, in each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush, and in each drawing stage, drawing is performed based on an image obtained by drawing in a previous drawing stage, and the virtual drawing brush in each drawing stage changes according to a preset change method; The number of drawing stages corresponding to the drawing process of the texture image is the same as the number of drawing stages corresponding to the drawing process of the target image; The first fusion module is configured to perform fusion of the target image and the texture image to generate a processed target image.

11. The image style transfer device according to claim 10, characterized in that: The device also includes: A third acquisition module is configured to execute acquisition of a plurality of adjacent pixel points of the plurality of sampling points in the original image; A second generating module is configured to generate a drawing path corresponding to the sampling point according to the plurality of adjacent pixel points; The adjustment module is configured to adjust the parameters of the virtual drawing brush according to the drawing path.

12. The image style transfer device according to claim 11, characterized in that: The second generating module is specifically configured to execute: Respectively obtaining pixel differences between the sampling point and a plurality of adjacent pixel points of the sampling point; Selecting an adjacent pixel point with the largest pixel difference from the plurality of adjacent pixel points as a target pixel point; The drawing path is generated according to the sampling point and the target pixel point.

13. The image style transfer device according to claim 10, characterized in that: The initial drawing parameters include a mask identifier, and the device further includes: A detection module is configured to detect the original image to obtain a target object and a target object type in the original image; A determination module, configured to determine a mask area of ​​the target object according to the type of the target object; A setting module, configured to execute setting of the mask area in the virtual drawing canvas; The second drawing module is configured to execute drawing in a drawing mode matching the mask identifier, and draw in the mask area and a non-mask area of ​​the virtual drawing canvas except the mask area.

14. The image style transfer device according to claim 13, characterized in that: The second drawing module is specifically configured to execute: When the mask identifier is a first identifier, extracting a mask image corresponding to the mask area from the original image; wherein the first identifier is used to indicate not to draw in the mask area and to draw in the non-mask area; The mask image is added to the mask area in the target image to generate a processed target image.

15. The image style transfer device according to claim 13, characterized in that: The second drawing module is specifically configured to execute: When the mask identifier is a second identifier, extracting a non-mask image corresponding to the non-mask area from the original image; wherein the second identifier is used to indicate to draw in the mask area and not to draw in the non-mask area; The non-mask image is added to the non-mask area in the target image to generate a processed target image.

16. The image style transfer device according to claim 13, characterized in that: The second drawing module is specifically configured to execute: When the mask identifier is the third identifier, based on the multiple mapping points, the virtual drawing brush is used to draw in the mask area and the non-mask area to generate the target image; wherein the third identifier is used to indicate drawing in the mask area and drawing in the non-mask area, and the virtual drawing brush parameters in the mask area are different from the virtual drawing brush parameters in the non-mask area.

17. The image style transfer device according to claim 10, characterized in that: The third drawing module is specifically configured to execute: Acquire a plurality of position points respectively corresponding to the plurality of mapping points in the virtual texture canvas; In each drawing stage, drawing is performed on the virtual texture canvas with a corresponding virtual drawing brush based on the multiple position points; At the plurality of position points, generating textures corresponding to the position points according to a height model corresponding to the virtual drawing brush; A texture image corresponding to the drawing stage is generated according to the textures of the multiple position points.

18. The image style transfer device according to any one of claims 10 to 16, characterized in that: The device also includes: The second fusion module is configured to generate a corresponding texture image according to the drawing operation at each drawing stage, and fuse the target image with the texture image at each drawing stage.

19. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image style transfer method as described in any one of claims 1 to 9.

20. A storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the image style transfer method as described in any one of claims 1 to 9.

Images