Image processing method, image generation method, device, equipment, and medium

Abstract

An image processing method, an image generating method, an apparatus, a device and a medium are provided, the image processing method including the steps of obtaining an original image, determining a cartoon image corresponding to the original image, obtaining line information and color block information of the cartoon image, and determining a papercut image corresponding to the original image according to the line information and color block information of the cartoon image. The embodiments of the present disclosure ensure high-quality display effect of the papercut image, enrich the image processing function supported by the electronic device, and thereby enrich the conversion type of the style image supported by the electronic device.

Description

[Technical field]

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, an image generating method, an apparatus, a device, and a medium. [Background technology]

[0002] Currently, as the functionality of video interaction applications becomes richer, image style transfer has become an interesting new way to play. Image style transfer refers to transferring the style of one or more images to generate a style image that meets the needs of the user.

[0003] However, the types of style transfer supported by traditional video interaction applications are still limited and cannot meet the needs of users to generate personalized style images. Summary of the Invention [Problem to be solved by the invention]

[0004] SUMMARY OF THE DISCLOSURE The embodiments of the present disclosure provide an image processing method, an image generating method, an apparatus, a device, and a medium to completely solve or at least partially solve the above technical problems. [Means for solving the problem]

[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising: obtaining an original image; determining a cartoon image corresponding to the original image; obtaining line information and color block information of the cartoon image; determining a cutout image corresponding to the original image based on line information and color block information of the cartoon image.

[0006] In a second aspect, embodiments of the present disclosure provide a method of generating an image, the method comprising: obtaining an image to be processed; and obtaining a paper-cut style image corresponding to the target image by utilizing a paper-cut style image generation model; The papercut-style image generation model is obtained by training based on an original image and a papercut image corresponding to the original image, and the papercut image is obtained based on any of the image processing methods provided by the embodiments of the present disclosure.

[0007] In a third aspect, an embodiment of the present disclosure provides an image processing apparatus, the apparatus comprising: an original image acquisition module used for acquiring an original image; a cartoon image determination module used to determine a cartoon image corresponding to the original image; a line and color block information acquisition module used for acquiring line information and color block information of the cartoon image; a cutout image generation module used for determining a cutout image corresponding to the original image based on line information and color block information of the cartoon image.

[0008] In a fourth aspect, embodiments of the present disclosure provide an image generating device, the device comprising: a processing target image acquisition module used to acquire a processing target image; a paper-cut style image generation module adapted to obtain a paper-cut style image corresponding to the target image by utilizing a paper-cut style image generation model; The papercut-style image generation model is obtained by training based on an original image and a papercut image corresponding to the original image, and the papercut image is obtained based on any of the above image processing methods provided by the embodiments of the present disclosure.

[0009] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device including a memory and at least one processor, wherein the memory stores a computer program, and when the computer program is executed by the at least one processor, the electronic device realizes any of the above image processing methods or image generation methods provided by the embodiments of the present disclosure.

[0010] In a sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored therein, the computer program being executed by a computing device, causing the computing device to realize any of the image processing or image generation methods provided by the embodiments of the present disclosure.

[0011] In a seventh aspect, an embodiment of the present disclosure provides a computer program product including a computer program, the computer program being stored in a readable storage medium, at least one processor of an electronic device being capable of reading the computer program from the readable storage medium, and when the at least one processor executes the computer program, the electronic device realizing any of the above image processing methods or image generation methods provided by the embodiment of the present disclosure.

[0012] In an eighth aspect, an embodiment of the present disclosure further provides a computer program, the computer program being stored in a readable storage medium, at least one processor of an electronic device being capable of reading the computer program from the readable storage medium, and when the at least one processor executes the computer program, the electronic device realizing any of the above image processing methods or image generation methods provided by the embodiments of the present disclosure. Effect of the Invention

[0013] Compared with the prior art, the technical solutions provided by the embodiments of the present disclosure have at least the following advantages: In the embodiments of the present disclosure, line information and color block information are extracted based on the cartoon image of the original image, the effect of extracting the line information and color block information of the original image is improved, and the line information and color block information obtained by extraction based on the cartoon image are used to generate the required cut-out image, thereby providing a new cut-out image generation method, ensuring the high-quality display effect of the cut-out image, enriching the image processing functions supported by the electronic device, thereby enriching the conversion types of style images supported by the electronic device, and solving the problem that the conversion types of style images supported by the previous solutions are limited and cannot meet the style image generation needs of users. [Brief description of the drawings]

[0014] The drawings herein are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the principles of the present disclosure.

[0015] In order to more clearly describe the embodiments of the present disclosure or the technical solutions in the prior art, the drawings that need to be used in the description of the embodiments or the prior art are briefly described below. Of course, those skilled in the art can obtain other drawings based on these drawings without creative efforts. [Figure 1] 1 is a flowchart of one image processing method provided by an embodiment of the present disclosure. [Diagram 2] FIG. 13 is a schematic diagram of an extraction result of line information and color block information provided by an embodiment of the present disclosure. [Diagram 3] 1 is a flowchart of another image processing method provided by an embodiment of the present disclosure. [Figure 4] FIG. 11 is a schematic diagram illustrating a stitched cutout image obtained based on the extraction results of line information and color block information of multiple information extraction regions provided by an embodiment of the present disclosure; [Diagram 5]FIG. 1 is a schematic diagram of obtaining a cutout image by preset processing provided by an embodiment of the present disclosure. [Figure 6] 1 is a flowchart of an image generation method provided by an embodiment of the present disclosure. [Figure 7] FIG. 2 is a schematic diagram of one paper-cutting style template image provided by an embodiment of the present disclosure. [Figure 8] 13 is a schematic diagram of another paper-cutting style template image provided by an embodiment of the present disclosure; FIG. [Figure 9] 1A-1C are comparative schematic diagrams of paper-cutting style images provided by embodiments of the present disclosure; [Figure 10] FIG. 2 is a structural schematic diagram of an image processing device provided according to an embodiment of the present disclosure; [Figure 11] FIG. 2 is a structural schematic diagram of an image generating device provided according to an embodiment of the present disclosure. [Figure 12] FIG. 2 is a structural schematic diagram of an electronic device provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] In order to make the above objectives, features and advantages of the present disclosure more clearly understood, the solutions of the present disclosure are further described below. Note that the embodiments and features in the embodiments of the present disclosure can be combined with each other unless they are inconsistent.

[0017] In the following description, many details are described in order to fully understand the present disclosure, but the present disclosure can be implemented in other forms different from those described in this specification. Of course, the embodiments in this specification are only a part of the embodiments of the present disclosure, and are not all of the embodiments.

[0018] FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, which can be applied to the scene of how to generate a cutout image, and the method can be executed by a cutout image processing device, which includes software and / or orIt can be implemented using hardware and can be integrated into any electronic device with computing capabilities, such as various user terminals or servers.

[0019] In the embodiment of the present disclosure, the papercut image is a style image with papercut effect, which is formed by lines and color blocks of preset colors. For any original image, the image processing method provided by the embodiment of the present disclosure can be used to obtain a papercut image corresponding to the original image, which enriches the needs of image stylization and improves the user experience. In addition, the papercut image obtained by adopting the method according to the embodiment of the present disclosure not only maintains a high similarity to the original image, but also has smooth lines, clean color blocks, and no noise. The preset color of the papercut image can be determined according to the specific needs of papercut image generation, for example, a color related to a specific holiday, such as red, and the embodiment of the present disclosure is not particularly limited thereto.

[0020] As shown in FIG. 1, the image processing method provided by the embodiment of the present disclosure may include the following steps.

[0021] In S101, the original image is acquired.

[0022] The original image may be any type of image, such as a face image, an animal image, a landscape image, etc. Exemplarily, the electronic device may obtain the original image by calling a photographing device to take a photograph, by a user's image upload operation, or by collecting data using the Internet.

[0023] At S102, a cartoon image corresponding to the original image is determined.

[0024] A cartoon image is an image in a cartoon style, for example, an image in a cartoon style of a specific region. A cartoon image corresponding to an original image has a high similarity to the original image. For example, a cartoon style image generation model obtained by training in advance can be used to obtain a cartoon image corresponding to the original image. The cartoon style image generation model has a cartoon image generation function, and can be obtained by training based on a generative adversarial network model, for example, but is naturally not limited thereto.

[0025] In S103, line information and color block information of the cartoon image are obtained.

[0026] After obtaining the cartoon image, any available algorithm with specific frequency information extraction function can be used to extract line information and color block information on the cartoon image, for example, any available band-pass filtering algorithm such as the xdog algorithm can be used.

[0027] Compared with the original image in which the pixel color distribution is disordered and the extracted line information and color block information are disordered, the cartoon image has a single pixel color and uniform color distribution, the extracted line information is smooth, and the color block information is relatively neat. Thus, the embodiment of the present disclosure extracts line information and color block information based on the cartoon image, thereby ensuring a high-quality display effect of the papercut image.

[0028] Optionally, the step of obtaining line information and color block information of the cartoon image includes the steps of:

[0029] One or more information extraction regions on the cartoon image are identified, and the specific number of information extraction regions may be preset or may be determined by identifying different color regions on the cartoon image, and the identification of the color regions may be achieved using any available conventional image processing algorithm (e.g., a target detection and identification algorithm, etc.), and the embodiments of the present disclosure are not particularly limited thereto.

[0030] According to the extraction parameters corresponding to each information extraction region, the line information and color block information in each information extraction region are obtained, and the line information and color block information of the cartoon image are obtained.

[0031] There are differences in the colors of the pixels in different information extraction regions, that is, different information extraction regions can correspond to image feature information of different frequencies. Therefore, by setting extraction parameters corresponding to each information extraction region, it is possible to narrow down and extract the line information and color block information within each information extraction region, thereby ensuring the extraction results of the line information and color block information and contributing to further ensuring the high-quality display effect of the paper cut-out image.

[0032] The extraction parameters are used to determine image feature information of a specific frequency to be extracted within each information extraction region, and the specific content of the extraction parameters can be set according to the extraction algorithm, and the embodiments of the present disclosure do not particularly limit it.

[0033] Exemplarily, in the embodiment of the present disclosure, the original image is an original image including a target object, the cartoon image is a cartoon image of the target object, the cartoon image is a cartoon image corresponding to the original image, the target object can include, for example, a target person, and accordingly, the information extraction region can be the target part region (face region, hair region, body region, etc.) of the target person and other regions (image background region, etc.) other than the target part region on the cartoon image, and the body region is the remaining region other than the face region and hair region, etc. in the target person image of the cartoon image. At this time, the step of acquiring line information and color block information in each information extraction region based on the extraction parameters corresponding to each information extraction region may be, for example, a step of acquiring line information and color block information of the face region based on the extraction parameters corresponding to the face region, a step of acquiring line information and color block information of the hair region based on the extraction parameters corresponding to the hair region, or a step of acquiring line information and color block information of the body region based on the extraction parameters corresponding to the body region, etc.

[0034] Of course, the target object may also be a target animal, specifically similar to a target person, and therefore will not be described again here.

[0035] 2 is a schematic diagram of the line information and color block information extraction result provided by the embodiment of the present disclosure, specifically, the embodiment of the present disclosure is illustrated by taking an original image including a target person as an example, but this should not be understood as specifically limiting the embodiment of the present disclosure. As shown in FIG. 2, when the line information (e.g., the line information of the person's hair region) and the color block information (e.g., the color block information of the person's clothing region, etc.) are directly extracted based on the original image, the line information of the hair region has many jagged edges, the color block information of the clothing region has many noises, etc., the lines and color blocks obtained by extraction are messy, and the display effect of the final cutout image is also poor. However, when the original image is converted into a cartoon image and the line information and color block information are extracted based on the cartoon image, the lines obtained by extraction are smooth, the color blocks are clean, and the display effect of the final cutout image is good.

[0036] In S104, a cut-out image corresponding to the original image is determined based on the line information and color block information of the cartoon image.

[0037] For example, the cutout image may be determined based on a preset image color (i.e., a preset cutout color) and a preset image resolution, specifically, based on the needs of generating the cutout image, and image rendering is performed based on the extracted line information and color block information to obtain a cutout image corresponding to the original image.

[0038] The papercut images and corresponding original images obtained by adopting the image processing method provided by the embodiments of the present disclosure can be used as training samples to train a papercut-style image generation model, which has the function of generating papercut-style images and can be realized based on any available neural network, such as Conditional Generative Adversarial Nets (CGAN) and pixel2pixel network.

[0039] In the embodiment of the present disclosure, line information and color block information are extracted based on the cartoon image of the original image, the extraction effect of the line information and color block information of the original image is improved, and the line information and color block information obtained by extraction based on the cartoon image are used to generate the required cut-out image, thereby providing a new cut-out image generation method, ensuring a high-quality display effect of the cut-out image, enriching the image processing functions supported by the electronic device, thereby enriching the conversion types of style images supported by the electronic device, and solving the problem that the conversion types of style images supported by the traditional solutions are limited, the stylization effect is single, and the diversification needs of users' style image generation cannot be met.

[0040] 3 is a flowchart of another image processing method provided by an embodiment of the present disclosure, which can be optimized and expanded based on the above embodiment and used in combination with each of the above optional embodiments. As shown in FIG. 3, the image processing method provided by the embodiment of the present disclosure can include the following steps:

[0041] In S201, the original image is obtained.

[0042] In S202, a cartoon image corresponding to the original image is determined.

[0043] In S203, one or more information extraction regions on the cartoon image are identified.

[0044] The specific number of information extraction regions may be preset or may be determined by identifying different color regions on the cartoon image, and the identification of color regions may be achieved using any available conventional image processing algorithm (e.g., an image segmentation algorithm, etc.), and the embodiments of the present disclosure are not particularly limited thereto.

[0045] In S204, line information and color block information within each information extraction region are obtained based on the extraction parameters corresponding to each information extraction region.

[0046] The extraction parameters are used to determine image feature information of a specific frequency to be extracted within each information extraction region, and the specific content of the extraction parameters can be set according to the extraction algorithm, and the embodiments of the present disclosure do not particularly limit it.

[0047] There are differences in the colors of the pixels in different information extraction regions, that is, different information extraction regions can correspond to image feature information of different frequencies. Therefore, by setting extraction parameters corresponding to each information extraction region, it is possible to narrow down and extract the line information and color block information within each information extraction region, thereby ensuring the extraction results of the line information and color block information, and further ensuring the high-quality display effect of the paper cut-out image.

[0048] Optionally, the step of obtaining line information and color block information in each information extraction region according to extraction parameters corresponding to each information extraction region includes: According to the extraction parameters corresponding to each information extraction region, respectively obtaining candidate line information and candidate color block information corresponding to each extraction parameter in the cartoon image; and determining line information and color block information within each information extraction region from candidate line information and candidate color block information of the cartoon image corresponding to the extraction parameters based on the position of each information extraction region in the cartoon image.

[0049] In an optional embodiment, after determining one or more information extraction regions on the cartoon image, an extraction operation of line information and color block information is performed once or multiple times on the entire cartoon image using extraction parameters corresponding to each information extraction region, and the extraction results of each time are set as candidate line information and candidate color block information of the cartoon image, and then, based on the position of each information extraction region in the cartoon image, line information and color block information belonging to each information extraction region can be determined from the candidate line information and candidate color block information of the cartoon image corresponding to each extraction parameter. For example, an extraction operation of line information and color block information is performed on the entire cartoon image using extraction parameters corresponding to a face region, and the extraction results are set as candidate line information and candidate color block information of the cartoon image (which includes line information and color block information of the face region and also includes line information and color block information of other regions other than the face region on the cartoon image), and then, based on the position of the face region in the entire cartoon image, line information of the face region is determined from the candidate line information and candidate color block information of the cartoon image. And Ka Determine the laravel block information.

[0050] Fig. 4 is a schematic diagram of obtaining a spliced ​​cutout image based on the extraction results of line information and color block information of multiple information extraction regions provided by the embodiment of the present disclosure, specifically, an original image including a target person and a cartoon image of the target person are taken as examples to exemplify the embodiment of the present disclosure, but this should not be understood as specifically limiting the embodiment of the present disclosure. As shown in Fig. 4, the information extraction region on the cartoon image can include a face region (including facial features), a hair region, and a target body region (which can include a person's neck part and upper body clothing region), and corresponding extraction parameters are set for each information extraction region, for example, taking the xdog algorithm as an example, the face region corresponds to one set of xdog parameters, the hair region corresponds to another set of xdog parameters, and the target body region corresponds to another set of xdog parameters. For example, when using extraction parameters corresponding to a face region to extract line information and color block information from a cartoon image to obtain one candidate line information and candidate color block information for the cartoon image, refer to the schematic diagram of the information extraction result for the face region shown in Figure 4; when using extraction parameters corresponding to a hair region to extract line information and color block information from a cartoon image to obtain one candidate line information and candidate color block information for the cartoon image, refer to the schematic diagram of the information extraction result for the hair region shown in Figure 4; and when using extraction parameters corresponding to a target body region to extract line information and color block information from a cartoon image to obtain one candidate line information and candidate color block information for the cartoon image, refer to the schematic diagram of the information extraction result for the target body region shown in Figure 4.

[0051] In addition, the extraction result corresponding to the background region obtained by extracting line information and color block information from the cartoon image using the extraction parameters corresponding to any one of the other information extraction regions for the background region on the cartoon image is the line information and color block information in the background region. As shown in FIG. 4, after extracting line information and color block information from the cartoon image using the extraction parameters corresponding to the face region, the extraction result corresponding to the background region from among the candidate line information and candidate color block information of the cartoon image can be the line information and color block information in the background region, and at this time, it may be considered that the background region and the face region adopt the same extraction parameters. In addition, the extraction parameters corresponding to the background region may be set individually, and the embodiment of the present disclosure is not particularly limited thereto, and can be set according to the display needs of the image background.

[0052] Optionally, the step of obtaining line information and color block information in each information extraction region according to extraction parameters corresponding to each information extraction region includes: performing region segmentation on the cartoon image based on the position of each information extraction region in the cartoon image to obtain region sub-images corresponding to each information extraction region; respectively obtaining line information and color block information of the regional sub-images corresponding to each information extraction region according to extraction parameters corresponding to each information extraction region.

[0053] In an optional embodiment, an image segmentation algorithm can be used to detect each information extraction region on a cartoon image, and then region segmentation can be performed on the cartoon image based on the region area of ​​each information extraction region to obtain region sub-images corresponding to each information extraction region; for example, after detecting a face region, a hair region, and a target body region on a cartoon image containing a target person, the cartoon image can be segmented based on these regions to obtain multiple region sub-images; and, based on extraction parameters corresponding to each information extraction region, line information and color block information can be extracted individually for each corresponding region sub-image.

[0054] In S205, the line information and color block information in each information extraction region are spliced ​​together based on the position of each information extraction region in the cartoon image to obtain line information and color block information of the cartoon image.

[0055] 4, after obtaining the line information and color block information in each information extraction region, the line information and color block information in each information extraction region are spliced ​​according to the position of each information extraction region in the cartoon image, thereby obtaining the line information and color block information of the cartoon image. For example, the line information and color block information in each information extraction region are spliced ​​according to the positional relationship of the face region, hair region, body region, etc. in the cartoon image, thereby obtaining the line information and color block information of the cartoon image including the target person.

[0056] In S206, a cut-out image corresponding to the original image is determined based on the line information and color block information of the cartoon image.

[0057] In the embodiment of the present disclosure, the line information and color block information on the cartoon image are extracted based on different information extraction regions to improve the information extraction effect, but because the extraction parameters of different information extraction regions are different, after the extraction results of each information extraction region are spliced, the overall style is not integrated as a result of the splicing, and there are still many jagged edges and noises on the lines, which are factors that affect the display effect of the cutout image, so the embodiment of the present disclosure can further optimize the generation process of the cutout image. That is, optionally, the step of determining a cutout image corresponding to the original image based on the line information and color block information of the cartoon image may include the steps of obtaining a candidate image based on the line information and color block information of the cartoon image, and performing a preset process on the candidate image and obtaining a cutout image corresponding to the original image based on the result of the preset process. The preset process includes at least one of removing the jagged edges of the line information and removing the noise of the line information. The removal of the jagged edges and noise can be realized by using any available algorithm with the function of removing jagged edges and noise in the prior art, and the embodiment of the present disclosure is not particularly limited thereto.

[0058] In an optional embodiment, the step of performing a preset process on the candidate image and obtaining a cutout image corresponding to the original image based on a result of the preset process includes the steps of:

[0059] The candidate image is converted into a vectorized image, specifically, the candidate image (a type of bitmap) can be converted into a vectorized image by using any available conventional vectorization technology, and after vectorization, any enlargement process can be performed on the image to avoid image distortion, and the vectorization process can reduce the jaggedness and noise of the candidate image, thereby realizing optimization of the candidate image.

[0060] A rendering is performed on the vectorized image based on a preset image color and a preset image resolution to obtain a cutout image corresponding to the original image. Optionally, the preset image color includes a color associated with preset holiday information, such as red.

[0061] FIG. 5 is a schematic diagram of obtaining a cutout image by a preset processing provided by an embodiment of the present disclosure, which is used to illustratively explain an embodiment of the present disclosure. As shown in FIG. 5, the cutout image obtained by the above-mentioned preset processing has little jaggedness and noise, and displays a good image effect.

[0062] In the embodiment of the present disclosure, based on each information extraction area on the cartoon image corresponding to the original image, line information and color block information are narrowed down and extracted, improving the extraction effect of the line information and color block information of the original image, and the required cut-out image is generated by using the line information and color block information extracted based on the cartoon image. By providing a new cut-out image generation method, it is possible to ensure a high-quality display effect of the cut-out image and enrich the image processing functions supported by the electronic device, thereby enriching the conversion types of style images supported by the electronic device, and solving the problem that the conversion types of style images supported by the traditional solutions are limited and cannot meet the users' style image generation needs.

[0063] 6 is a flowchart of an image generating method provided by an embodiment of the present disclosure, which can be applied to the scene of how to generate a paper-cut style image. The image generating method can be performed by an image generating device, which can be realized using software and / or hardware, and can be integrated into any electronic device with computing capabilities, such as a mobile smart terminal, a tablet computer, a notebook computer, etc.

[0064] The image generation method provided by the embodiment of the present disclosure is realized by pre-training a papercut style image generation model, and the papercut style image generation model is obtained by training based on an original image and a papercut image corresponding to the original image, and the papercut image is obtained based on any available image processing method provided by the embodiment of the present disclosure. That is, any image processing method provided by the embodiment of the present disclosure can be used to obtain a papercut image corresponding to the original image, and the original image and the corresponding papercut image are paired training data used to train and obtain the papercut style image generation model. Compared with the training data obtained by directly drawing the papercut image, the training data obtained by using the above image processing method of the present disclosure is more efficient and less costly. Exemplarily, the papercut style image generation model can be realized based on any available neural network, such as a conditional generative adversarial network (CGAN) or a pixel2pixel network. For a specific description of the image processing method, the contents of the above embodiment can be referred to.

[0065] In addition, in the embodiments of the present disclosure, the image generation method may be realized in the form of an independent application program or an applet integrated into a public platform, or may be realized as a functional module integrated into an application program or applet with image editing function, and the application program or applet may include, but is not limited to, a video interaction application program or a video interaction applet.

[0066] As shown in FIG. 6, the image generating method provided by the embodiment of the present disclosure may include the following steps.

[0067] In S301, the image to be processed is acquired.

[0068] The processing target image may be any type of image, such as a face image, an animal image, a landscape image, etc. Exemplarily, the electronic device may obtain the processing target image by calling a photographing device to take a photograph, or may obtain the processing target image by a user's image upload operation.

[0069] In S302, a paper-cut style image corresponding to the processing target image is obtained using a paper-cut style image generation model.

[0070] The papercut-style image generation model is obtained by training based on an original image and a papercut image corresponding to the original image, and the papercut image is obtained based on any available image processing method provided by the embodiments of the present disclosure.

[0071] Optionally, the image generating method provided by the embodiment of the present disclosure further includes, before the step of obtaining the processing target image, Responding to a user's request to generate a paper-cut style image, the method includes the step of displaying an original template image; The image generation method provided by the embodiment of the present disclosure further includes, after the step of obtaining a paper-cut style image corresponding to the processing target image by using a paper-cut style image generation model, The method includes displaying the paper-cut style image overlaid on a paper-cut style template image corresponding to the original template image to generate a target image.

[0072] The original template image is a preset background image and may include any type of preset image, and the embodiment of the present disclosure is not particularly limited thereto. Exemplarily, the user may trigger the paper-cut style image generation request by touching a prop icon corresponding to the template image displayed on the screen of the electronic device, or the user may trigger the paper-cut style image generation request by directly touching the shooting button displayed on the screen of the electronic device, or the user may trigger the paper-cut style image generation request using means such as gesture control or voice control, and the electronic device may respond to the user's paper-cut style image generation request and randomly display the original template image, or may display the original template image according to a preset rule, and the preset rule may include, for example, a rule for displaying according to the popularity of the original template image (or the number of times used by a large number of users, specifically, the server stats in the background), or a rule for displaying according to the default order of the original template image, and the embodiment of the present disclosure is not particularly limited thereto, and of course, after responding to the user's paper-cut style image generation request, the electronic device may also select an operation according to the user's original template image and display the original template image selected by the user.

[0073] The paper-cutting style template image is a template image obtained by stylizing an original template image with a paper-cutting style. The paper-cutting style template image corresponding to each original template image may be generated in advance, and during target image generation, the paper-cutting style template image corresponding to the currently used original template image can be called up according to the correspondence between the original template image and the paper-cutting style template image.

[0074] Illustratively, the original template image provided by an embodiment of the present disclosure may include a template image associated with preset holiday information, and the color of the paper-cut style image and the paper-cut style template image may include a color associated with the preset holiday information, such as red.

[0075] FIG. 7 is a schematic diagram of one paper-cut style template image provided by an embodiment of the present disclosure. As shown in FIG. 7, the paper-cut style template image may include elements related to the Chinese zodiac, such as holiday-related elements such as lanterns, where the corresponding original template image includes lantern elements before paper-cut stylization, and the paper-cut style template image includes lantern elements after paper-cut stylization. FIG. 8 is a schematic diagram of another paper-cut style template image provided by an embodiment of the present disclosure. As shown in FIG. 8, the paper-cut style template image may include elements related to a specific plant. It should be understood that FIG. 7 and FIG. 8 are used to illustratively illustrate the paper-cut style template images according to the embodiment of the present disclosure, but they should not be understood as specifically limiting the embodiment of the present disclosure.

[0076] Displaying the paper-cut style template image together with the generated paper-cut style image contributes to improving the display effect of the style image. The display effect of the target image can be seen in Fig. 9, which specifically shows the display effect of the target image in the case where the paper-cut style template image and the processing target image shown in Fig. 8 are face images.

[0077] In an optional embodiment, the step of displaying the paper-cutting style image overlaid on a paper-cutting style template image corresponding to the template image to generate the target image comprises: performing region segmentation on the paper-cut style image based on a position of the target object in the paper-cut style image to obtain a region sub-image corresponding to the target object; and overlaying the regional sub-image corresponding to the target object on a paper-cut style template image corresponding to the original template image to generate a target image.

[0078] The target object, also called the shooting target, is associated with the type of the target image. For example, if the target object is a target person, the target image and the corresponding paper-cut style image are images including the target person, and the area sub-image corresponding to the target object on the paper-cut style image is an image in which only the target person is displayed. By displaying the area sub-image corresponding to the target object on the paper-cut style template image corresponding to the original template image in a superimposed manner, the influence of the background area on the paper-cut style image on the overall display effect of the target image can be avoided, that is, by using the background on the paper-cut style template image as the background of the target image, the uniformity of the display style of the entire image can be ensured, and the image display effect is high.

[0079] In an alternative embodiment, the image generation method provided by the embodiments of the present disclosure includes , cut After the step of obtaining a paper-cut style image corresponding to the processing target image by using the paper-cut style image generation model, The method includes performing feature enhancement and / or deblurring processing on the papercut-style image.

[0080] Considering that the size of the input image of the papercut style image generation model is usually a preset size, if the user's processing target image does not meet the preset size, processing operations such as zooming are usually required until the requirement for the size of the input image is met, and then input into the model, and output a style image through model processing; finally, the style image output from the model is zoomed again to the same size as the processing target image, and the papercut style image is obtained and displayed to the user; in this process, due to the zoom operation, it is easy to lead to a blurred display effect of the papercut style image, so in order to improve the display effect of the papercut style image, feature enhancement processing and / or blur removal processing can be performed on the papercut style image.

[0081] In Example 1, the step of performing feature enhancement processing on the paper-cut style image includes: performing a blurring process on the paper-cut style image to obtain a blurred style image, the blurring process can be realized using any available blurring algorithm, and the embodiment of the present disclosure is not particularly limited thereto; determining a residual image corresponding to the papercut style image based on the papercut style image and the blur style image; performing a feature enhancement process on the residual image, and performing an overlay process on the feature enhanced residual image and the blurred style image to obtain a feature enhanced papercut style image.

[0082] For example, a Gaussian blur algorithm can be used to perform blurring on the papercut style image A to obtain a blurred style image B, where the blurred information is low-frequency information; and a residual image C is determined based on the papercut style image A and the blurred style image B, where the residual image C can be expressed as C=AB (representing subtraction of image features at the same image position), i.e., the residual image can be used to represent high-frequency information on the papercut style image; and finally, a feature enhancement process is performed on the residual image, i.e., feature enhancement is performed on the high-frequency information, for example, by multiplying the residual image C by a preset parameter t, and performing an overlap process on the residual image subjected to the feature enhancement process and the blurred style image to obtain a feature enhanced papercut style image D, which can be expressed as D=B+t·C, where the larger the value of the preset parameter t, the clearer the obtained feature enhanced papercut style image D.

[0083] In Example 2, the steps of performing a deblurring process on a paper-cut style image include: The method includes a step of performing high-frequency information extraction on the paper-cut style image to obtain a paper-cut style image that has been subjected to blur removal processing. Since the low-frequency information on the paper-cut style image is low-frequency information, high-frequency information extraction is performed on the paper-cut style image, for example, by using the xdog algorithm to perform high-frequency information extraction, a blur removal effect can be achieved.

[0084] 9 is a comparative schematic diagram of a papercut style image provided by an embodiment of the present disclosure, which is used to exemplarily describe the papercut style image before and after the feature enhancement and / or blur removal processing according to the embodiment of the present disclosure, but should not be understood as specifically limiting the embodiment of the present disclosure. As shown in FIG. 9, the papercut style image after the feature enhancement and / or blur removal processing has a clearer display effect.

[0085] In the embodiment of the present disclosure, a paper-cut style image generation model is trained based on an original image and a paper-cut image corresponding to the original image, and a paper-cut style image corresponding to the image to be processed is obtained by using the paper-cut style image generation model, thereby solving the problem that the conversion types of style images supported by conventional solutions are limited and cannot meet the style image generation needs of users, enriching the style image conversion types supported by electronic devices, realizing the highly efficient generation of paper-cut style images, and improving the fun of style image generation.

[0086] FIG. 10 is a structural schematic diagram of an image processing device 400 provided by an embodiment of the present disclosure, which can be implemented using software and / or hardware, and can be integrated into any electronic device with computing capabilities, such as various user terminals or servers.

[0087] As shown in FIG. 10 , the image processing device 400 provided by an embodiment of the present disclosure may include an original image acquisition module 401, a cartoon image determination module 402, a line and color block information acquisition module 403, and a cut-out image generation module 404. The original image acquisition module 401 is used to acquire an original image; The cartoon image determination module 402 is used to determine a cartoon image corresponding to the original image; The line and color block information acquisition module 403 is used to acquire line information and color block information of the cartoon image; The cutout image generation module 404 is used to determine a cutout image corresponding to the original image based on the line information and color block information of the cartoon image.

[0088] Optionally, the line and color block information acquisition module 403 includes: an information extraction region identification unit, used for identifying one or more information extraction regions on the cartoon image; a line and color block information extraction unit, which is used to obtain line information and color block information in each information extraction region according to extraction parameters corresponding to each information extraction region; and an information splicing unit, which is used to perform splicing on the line information and color block information in each information extraction region based on the position of each information extraction region in the cartoon image, to obtain the line information and color block information of the cartoon image.

[0089] Optionally, the line and color block information extraction unit comprises: A first information acquisition sub-unit is used for respectively acquiring candidate line information and candidate color block information corresponding to each extraction parameter in the cartoon image according to the extraction parameters corresponding to each information extraction region; a first information determination subunit used to respectively determine line information and color block information in each information extraction region from candidate line information and candidate color block information of the cartoon image corresponding to the extraction parameters based on the position of each information extraction region in the cartoon image.

[0090] Optionally, the line and color block information extraction unit comprises: a region segmentation subunit, which is used to perform region segmentation on the cartoon image according to the position of each information extraction region in the cartoon image, and obtain a region sub-image corresponding to each information extraction region; and a second information acquisition sub-unit, which is used for respectively acquiring line information and color block information of the regional sub-image corresponding to each information extraction region according to the extraction parameters corresponding to each information extraction region.

[0091] Optionally, the cutout image generation module 404 includes: a candidate image determining unit, which is used to obtain a candidate image according to the line information and the color block information of the cartoon image; The present invention includes a preset processing unit used for performing preset processing on a candidate image and obtaining a cut-out image corresponding to the original image based on a result of the preset processing, the preset processing including at least one of removing jagged edges in line information and removing noise in line information.

[0092] Optionally, preset processing Reason Knit is an image conversion subunit used to convert the candidate image into a vectorized image; and an image rendering subunit that is used to perform rendering on the vectorized image based on a preset image color and a preset image resolution to obtain a cutout image corresponding to the original image.

[0093] The image processing device provided by the embodiment of the present disclosure can execute any image processing method provided by the embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method. For contents not described in detail in the embodiment of the device of the present disclosure, reference can be made to the description of any embodiment of the method of the present disclosure.

[0094] FIG. 11 is a structural schematic diagram of an image generating device provided by an embodiment of the present disclosure, which can be implemented using software and / or hardware and can be integrated into any electronic device with computing capabilities, such as a mobile smart terminal, a tablet computer, a notebook computer, etc.

[0095] As shown in FIG. 11 , an image generating device 500 provided by an embodiment of the present disclosure may include a processing target image acquiring module 501 and a paper-cut style image generating module 502 . The processing target image acquisition module 501 is used to acquire a processing target image. the paper-cut style image generation module 502 is used to obtain a paper-cut style image corresponding to the processing target image by utilizing a paper-cut style image generation model; The papercut-style image generation model is obtained by training based on an original image and a papercut image corresponding to the original image, where the papercut image is obtained based on any of the image processing methods provided by the embodiments of the present disclosure.

[0096] Optionally, the image generating device 500 provided by the embodiment of the present disclosure further comprises: a template image display module used for displaying an original template image in response to a user's request for generating a paper-cut style image; and a target image generation module adapted to overlay the paper-cut style image on a paper-cut style template image corresponding to the template image to generate a target image.

[0097] Optionally, the target image generation module further comprises: a segmentation unit, which is used to perform segmentation on the paper-cut style image based on a position of a target object in the paper-cut style image to obtain a region sub-image corresponding to the target object; an overlay display unit used to overlay the regional sub-image corresponding to the target object onto a paper-cut style template image corresponding to the original template image to generate a target image.

[0098] Optionally, the image generating device 500 provided by the embodiment of the present disclosure further comprises: It may also include a styled image processing module that is used to perform feature enhancement and / or deblurring processing on the papercut styled images.

[0099] Optionally, the style image processing module further comprises: a blurring processing unit used for performing a blurring process on the paper-cut style image to obtain a blurred style image; a residual image determining unit, which is used for determining a residual image corresponding to the papercut style image according to the papercut style image and the blur style image; and an image overlay processing unit, which is used to perform feature expansion processing on the residual image and to perform overlay processing on the feature expanded residual image and the blurred style image to obtain a feature enhanced papercut style image.

[0100] Optionally, the style image processing module further comprises: The apparatus includes a high frequency information extraction unit that is used to perform high frequency information extraction on the paper cut-out style image to obtain a deblurred paper cut-out style image.

[0101] The image generating device provided by the embodiment of the present disclosure can execute any image generating method provided by the embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method. For contents not described in detail in the embodiment of the device of the present disclosure, reference can be made to the description of any embodiment of the method of the present disclosure.

[0102] FIG. 12 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure, and is used to exemplarily describe an electronic device that realizes an image processing method or an image generating method provided by an embodiment of the present disclosure. The electronic device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs, desktop computers, smart home devices, wearable electronic devices, and servers. The electronic device shown in FIG. 12 is merely an example and should not impose any limitations on the functions and scope of use of the embodiment of the present disclosure.

[0103] As shown in FIG. 12, the electronic device 600 includes one or more processors 601 and a memory 602 .

[0104] The processor 601 may be a central processing unit (CPU) or other type of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 600 to perform desired functions.

[0105] The memory 602 may include one or more computer program products, which may include various types of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or high-speed cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. The computer readable storage medium may store one or more computer program instructions, and the processor 601 may execute the program instructions, thereby realizing the image processing method or image generation method provided by the embodiments of the present disclosure, and may also realize other functions as desired. The computer readable storage medium may further include various contents, such as an input signal, a signal component, a noise component, etc.

[0106] In one aspect, an image processing method provided by an embodiment of the present disclosure may include the steps of obtaining an original image, determining a cartoon image corresponding to the original image, obtaining line information and color block information of the cartoon image, and determining a papercut image corresponding to the original image based on the line information and color block information.

[0107] In another aspect, an image generating method provided by an embodiment of the present disclosure may include obtaining a target image and obtaining a paper-cut style image corresponding to the target image using a paper-cut style image generation model, the paper-cut style image generation model being trained based on an original image and a paper-cut image corresponding to the original image, and the paper-cut image being obtained based on any image processing method provided by an embodiment of the present disclosure.

[0108] The electronic device 600 can also perform other optional embodiments provided by method embodiments of the present disclosure.

[0109] In one example, the electronic device 600 may further include input devices 603 and output devices 604, and these components may be connected together by a bus system and / or other type of connection mechanism (not shown).

[0110] The input device 603 may further include a keyboard, a mouse, and the like.

[0111] The output device 604 can output various information including the determined distance information, direction information, etc. The output device 604 can include, for example, a display, a speaker, a printer, a communication network, and a remote output device connected thereto.

[0112] Of course, for the sake of brevity, Fig. 12 shows only some of the components of the electronic device 600 that are relevant to the present disclosure, and omits components such as buses and input / output ports, and the electronic device 600 may include any other suitable components depending on the specific application.

[0113] In addition to the above methods and apparatus, embodiments of the present disclosure further provide a computer program product including a computer program or computer program instructions, which, when executed by a computing device, cause the computing device to implement any of the image processing or image generation methods provided by embodiments of the present disclosure.

[0114] In a computer program product, program code for carrying out operations of embodiments of the present disclosure may be written in one or a combination of programming languages, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user electronic device, as a separate software package, partially on the user electronic device and partially on the remote electronic device, or entirely on the remote electronic device.

[0115] Additionally, embodiments of the present disclosure further provide a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a computing device, cause the computing device to implement any of the image processing or image generation methods provided by embodiments of the present disclosure.

[0116] In one aspect, an image processing method provided by an embodiment of the present disclosure may include the steps of obtaining an original image, determining a cartoon image corresponding to the original image, obtaining line information and color block information of the cartoon image, and determining a papercut image corresponding to the original image based on the line information and color block information.

[0117] In another aspect, an image generation method provided by an embodiment of the present disclosure may include the steps of obtaining a target image and obtaining a paper-cut style image corresponding to the target image using a paper-cut style image generation model, the paper-cut style image generation model being obtained by training based on an original image and a paper-cut image corresponding to the original image, and the paper-cut image being obtained based on any image processing method provided by an embodiment of the present disclosure.

[0118] It should be understood that the computer program instructions, when executed by a computing device, can cause the computing device to implement other alternative embodiments provided by the method embodiments of the present disclosure.

[0119] The computer readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic memory component, or any suitable combination of the above.

[0120] It should be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another and do not necessarily require or imply any actual relationship or order between those entities or operations. Also, the terms "comprise", "comprises", or any other variation thereof are intended to include a non-exclusive inclusion, whereby a process, method, article, or device that includes a set of elements includes not only those elements, but also other elements not expressly listed or that are inherent to such process, method, article, or device. Unless more limited, an element that is qualified by the phrase "comprises..." does not exclude the presence of further identical elements within the process, method, article, or device in which the element is included.

[0121] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or practice the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to these embodiments herein, but is accorded the widest scope consistent with the principles and special features disclosed herein.

[0122] This application was filed with the China Patent Office on February 9, 2021 and has application number 2021 1 No. 0180562.9, entitled “Image Processing Method, Image Generating Method, Apparatus, Device and Medium,” the entire contents of which are incorporated herein by reference.

Claims

1. 1. An image processing method, comprising: obtaining an original image; determining a cartoon image corresponding to the original image; obtaining line information and color block information of the cartoon image; determining a cut-out image corresponding to the original image based on line information and color block information of the cartoon image; determining a cutout image corresponding to the original image based on line information and color block information of the cartoon image; obtaining a candidate image based on line information and color block information of the cartoon image; performing a preset process on the candidate image and obtaining a cutout image corresponding to the original image based on a result of the preset process, the preset process including at least one of removing jagged edges of line information and removing noise of line information; The step of performing a preset process on the candidate image and obtaining a cutout image corresponding to the original image based on a result of the preset process includes: converting the candidate image into a vectorized image; and performing a rendering on the vectorized image based on a preset image color and a preset image resolution to obtain a cutout image corresponding to the original image.

13. An image processing method comprising:

2. The step of obtaining line information and color block information of the cartoon image includes: identifying one or more information extraction regions on the cartoon image; obtaining line information and color block information within each information extraction region according to extraction parameters corresponding to each information extraction region; 2. The method of claim 1, further comprising: performing splicing on line information and color block information in each information extraction region based on a position of each information extraction region in the cartoon image to obtain line information and color block information of the cartoon image.

3. The step of obtaining line information and color block information in each information extraction region according to extraction parameters corresponding to each information extraction region includes: respectively obtaining candidate line information and candidate color block information corresponding to each extraction parameter in the cartoon image according to extraction parameters corresponding to each information extraction region; and determining line information and color block information in each of the information extraction regions from candidate line information and candidate color block information of the cartoon image corresponding to the extraction parameters based on a position of each of the information extraction regions in the cartoon image, respectively.

4. The step of obtaining line information and color block information in each information extraction region according to extraction parameters corresponding to each information extraction region includes: performing region segmentation on the cartoon image based on the position of each of the information extraction regions in the cartoon image to obtain region sub-images corresponding to each of the information extraction regions; 3. The method according to claim 2, further comprising: respectively obtaining line information and color block information of a region sub-image corresponding to each of the information extraction regions according to extraction parameters corresponding to each of the information extraction regions.

5. An image processing device, an original image acquisition module used for acquiring an original image; a cartoon image determination module used to determine a cartoon image corresponding to the original image; a line and color block information acquisition module used for acquiring line information and color block information of the cartoon image; a cutout image generation module used for determining a cutout image corresponding to the original image based on line information and color block information of the cartoon image; The cutout image generation module includes: a candidate image determining unit, which is used to obtain a candidate image according to the line information and the color block information of the cartoon image; a preset processing unit used for performing a preset processing on the candidate image and obtaining a cutout image corresponding to the original image based on a result of the preset processing, the preset processing including at least one of removing jaggedness of line information and removing noise of line information; The preset processing unit includes: an image conversion subunit used to convert the candidate image into a vectorized image; an image rendering subunit, which is used to perform rendering on the vectorized image based on a preset image color and a preset image resolution to obtain a cutout image corresponding to the original image; 13. An image processing device comprising:

6. The line and color block information acquisition module includes: an information extraction region identification unit, used for identifying one or more information extraction regions on the cartoon image; a line and color block information extraction unit, which is used to obtain line information and color block information in each information extraction region according to extraction parameters corresponding to each information extraction region; 6. The image processing device according to claim 5, further comprising: an information splicing unit, which is used to perform splicing on the line information and color block information in each information extraction region based on the position of each information extraction region in the cartoon image, to obtain the line information and color block information of the cartoon image.

7. The line and color block information extraction unit includes: A first information acquisition sub-unit is used for respectively acquiring candidate line information and candidate color block information corresponding to each extraction parameter in the cartoon image according to the extraction parameters corresponding to each information extraction region; 7. The image processing device according to claim 6, further comprising: a first information determining subunit used for respectively determining line information and color block information in each information extraction region from candidate line information and candidate color block information of the cartoon image corresponding to the extraction parameters based on the position of each information extraction region in the cartoon image.

8. The line and color block information extraction unit includes: a region segmentation subunit, which is used to perform region segmentation on the cartoon image according to the position of each information extraction region in the cartoon image, and obtain a region sub-image corresponding to each information extraction region; 7. The image processing device according to claim 6, further comprising: a second information acquisition subunit, which is used for respectively acquiring line information and color block information of the regional sub-image corresponding to each information extraction region according to extraction parameters corresponding to each information extraction region.

9. A computer-readable storage medium having a computer program stored therein, the computer program causing the computing device to realize the image processing method according to any one of claims 1 to 4 when executed by the computing device.

10. A computer program causing a computer device to execute the image processing method according to any one of claims 1 to 4.

Images