Image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an original text image and a prompt text for the original text image, and enabling the text in the original text image to be presented in a first font; determining a plurality of control conditions of the original text image, and processing the original text image according to the plurality of control conditions to obtain a plurality of condition control images; processing the original text image according to the prompt text and the plurality of condition control images to obtain a target text image; wherein the text in the target text image is presented in a second font, and the first font is different from the second font. In this way, the fonts of the text in the original text image are adjusted through the multiple control conditions, the adjusted fonts are more creative and diversified, the expected effect of a user can be achieved, and the time of the user is saved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] Currently, for text images, the text therein is usually presented in a specific font, but users have a need to adjust the font of the text in the text image. To this end, users usually use image editing tools to adjust the font of the text in the text image, which wastes time and the effect of the adjustment does not achieve the expected effect. Summary of the invention

[0003] In order to solve the technical problem that users usually use image editing tools to adjust the font of text in text images, which wastes time and the adjustment effect does not achieve the expected effect, the embodiment of the present application provides an image processing method, device, electronic device and storage medium. The specific technical solution is as follows:

[0004] In a first aspect of an embodiment of the present application, an image processing method is first provided, the method comprising:

[0005] Acquire an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font;

[0006] Determine a plurality of control conditions of the original text image, and process the original text image according to the plurality of control conditions to obtain a plurality of condition-controlled images;

[0007] Processing the original text image according to the prompt text and the plurality of condition control images to obtain a target text image;

[0008] The text in the target text image is presented in a second font, and the first font is different from the second font.

[0009] In an optional implementation, the original text image is processed according to the plurality of control conditions to obtain a plurality of condition-controlled images, including:

[0010] For any of the control conditions, the original text image is processed using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition.

[0011] In an optional implementation, when the control condition is graffiti, the processing of the original text image using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition includes:

[0012] Performing edge detection on the original text image using a first edge detection algorithm to obtain a first edge image;

[0013] A convolution operation is performed on the first edge image using a Gaussian kernel to obtain a convolution image, and a threshold segmentation is performed on the convolution image to obtain a graffiti image.

[0014] In an optional implementation, performing threshold segmentation on the convolution image to obtain a graffiti image includes:

[0015] For the pixel value of any pixel in the convolution image, the following processing is performed to obtain a graffiti image:

[0016] When the pixel value is greater than a preset pixel value threshold, setting the pixel value to a preset first pixel value;

[0017] When the pixel value is less than or equal to the preset pixel value threshold, the pixel value is set to a preset second pixel value.

[0018] In an optional implementation, when the control condition is a line, the processing of the original text image using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition includes:

[0019] Performing edge detection on the original text image using a second edge detection algorithm to obtain a second edge image;

[0020] In the case where the control condition is depth, the processing of the original text image using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition includes:

[0021] A depth detection algorithm is used to perform depth detection on the original text image to obtain a depth image.

[0022] In an optional implementation, the processing of the original text image according to the prompt text and the plurality of conditional control images to obtain a target text image includes:

[0023] For any of the conditional control images, determining a conditional control network corresponding to the conditional control image;

[0024] Inputting the conditional control image into the conditional control network to obtain conditional control features;

[0025] The prompt text, the original text image and a plurality of the conditional control features are input into an image generation network to obtain a target text image.

[0026] In an optional embodiment, the conditional control network includes a first zero convolution layer, a second zero convolution layer, an encoding network, and a first intermediate block;

[0027] The step of inputting the conditional control image into the conditional control network to obtain conditional control features includes:

[0028] Using the first zero convolution layer to perform convolution processing on the conditional control image to obtain a first convolution result;

[0029] Inputting the first convolution result into the encoding network to obtain a conditionally controlled encoding feature;

[0030] Inputting the conditional control coding feature into the first intermediate block to obtain a first refined feature;

[0031] The first extracted feature is convolved using the second zero convolution layer to obtain a second convolution result, and the second convolution result is determined as a conditional control feature.

[0032] In an optional implementation, the coding network includes M conditional control coding blocks, where M is a positive integer;

[0033] The step of inputting the first convolution result into the encoding network to obtain a conditionally controlled encoding feature includes:

[0034] Iterate and perform the following operations until the Mth conditional encoding feature is obtained and then stop the following operations:

[0035] Inputting the i-th conditional feature into the i-th conditional control coding block to obtain the i-th conditional coding feature;

[0036] Among them, the i-th conditional coding feature is the i+1-th conditional feature, the 1st conditional feature is the first convolution result, the M-th conditional coding feature is the conditional control coding feature, and i is 1, 2, 3,..., M respectively.

[0037] In an optional embodiment, the image generation network includes a first encoder, a first decoder, a second encoding network, a second decoding network, and a second intermediate block;

[0038] The step of inputting the prompt text, the original text image, and the plurality of conditional control features into an image generation network to obtain a target text image comprises:

[0039] Inputting the original text image into the first encoder to obtain a first encoding feature;

[0040] Inputting the prompt text and the first coding feature into the first coding network to obtain a second coding feature;

[0041] Determine a first weight corresponding to each of the plurality of conditional control features, and perform weighted summation on the plurality of conditional control features according to the first weight to obtain a first weighted feature;

[0042] Inputting the prompt text, the second encoding feature and the first weighted feature into the second intermediate block to obtain a second refined feature;

[0043] Inputting the prompt text and the second refined features into the second decoding network to obtain decoding features;

[0044] The decoded features are input into the first decoder to obtain a target text image.

[0045] In an optional implementation, the second coding network includes M second coding blocks, where M is a positive integer;

[0046] The step of inputting the prompt text and the first coding feature into the first coding network to obtain a second coding feature includes:

[0047] Iterate and perform the following operations until the Mth intermediate encoding feature is obtained and then stop the following operations:

[0048] Inputting the prompt text and the i-th first feature into the i-th second coding block to obtain the i-th intermediate coding feature;

[0049] Among them, the i-th intermediate coding feature is the i+1-th first feature, the 1st first feature is the first coding feature, the M-th intermediate coding feature is the second coding feature, and i is 1, 2, 3, ..., M respectively.

[0050] In an optional implementation, the second decoding network includes M second decoding blocks, where M is a positive integer;

[0051] The step of inputting the prompt text and the second refined features into the second decoding network to obtain the decoding features includes:

[0052] Iterate the following steps until the first intermediate decoding feature is obtained and stop the following operations:

[0053] Inputting the prompt text and the j-th second feature into the j-th second decoding block to obtain the j-th intermediate decoding feature;

[0054] Among them, the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M,...,3,2,1 respectively.

[0055] In an optional embodiment, each conditional control network further includes M third zero convolutional layers, where M is a positive integer;

[0056] The step of inputting the prompt text and the j-th second feature into the j-th second decoding block to obtain the j-th intermediate decoding feature includes:

[0057] For any conditional control network, determining a j-th conditional control coding block of a coding network in the conditional control network;

[0058] The j-th conditional control coding block of the coding network in the conditional control network is at the same level as the j-th second decoding block in the second decoding network;

[0059] Using the jth third zero convolution layer, convolution processing is performed on the jth conditional coding feature output by the jth conditional control coding block of the coding network in the conditional control network to obtain a jth second convolution result;

[0060] Determine a second weight corresponding to each of a plurality of j-th second convolution results, and perform weighted summation on the plurality of j-th second convolution results according to the second weight to obtain a j-th second weighted feature;

[0061] The prompt text, the jth second feature, and the jth second weighted feature are input into the jth second decoding block to obtain the jth intermediate decoding feature.

[0062] In an optional implementation, the step of inputting the prompt text, the jth second feature, and the jth second weighted feature into the jth second decoding block to obtain the jth intermediate decoding feature includes:

[0063] In the first iteration, the Mth second feature and the Mth second weighted feature are input into the Mth second decoding block to obtain the Mth intermediate decoding feature;

[0064] In a non-first iteration process, the prompt text, the j-th second feature, and the j-th second weighted feature are input into the j-th second decoding block to obtain the j-th intermediate decoding feature;

[0065] Among them, the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M-1,...,3,2,1 respectively.

[0066] In a second aspect of the embodiments of the present application, an image processing device is further provided, the device comprising:

[0067] An image and text acquisition module, used to acquire an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font;

[0068] A condition determination module, used for determining a plurality of control conditions of the original text image;

[0069] A first image processing module, used for processing the original text image according to the plurality of control conditions to obtain a plurality of condition-controlled images;

[0070] A second image processing module, configured to process the original text image according to the prompt text and the plurality of conditional control images to obtain a target text image;

[0071] The text in the target text image is presented in a second font, and the first font is different from the second font.

[0072] In an optional implementation, the first image processing module specifically includes:

[0073] The original text image processing submodule is used to process the original text image according to any of the control conditions using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition.

[0074] In an optional implementation, when the control condition is graffiti, the original text image processing submodule specifically includes:

[0075] An edge detection unit, configured to perform edge detection on the original text image using a first edge detection algorithm to obtain a first edge image;

[0076] An image convolution unit, configured to perform a convolution operation on the first edge image using a Gaussian kernel to obtain a convolution image;

[0077] The image segmentation unit is used to perform threshold segmentation on the convolution image to obtain a graffiti image.

[0078] In an optional implementation, the image segmentation unit is specifically used to:

[0079] For the pixel value of any pixel in the convolution image, the following processing is performed to obtain a graffiti image:

[0080] When the pixel value is greater than a preset pixel value threshold, setting the pixel value to a preset first pixel value;

[0081] When the pixel value is less than or equal to the preset pixel value threshold, the pixel value is set to a preset second pixel value.

[0082] In an optional implementation, when the control condition is a line, the original text image processing submodule is specifically used for:

[0083] Performing edge detection on the original text image using a second edge detection algorithm to obtain a second edge image;

[0084] When the control condition is depth, the original text image processing submodule is specifically used for:

[0085] A depth detection algorithm is used to perform depth detection on the original text image to obtain a depth image.

[0086] In an optional implementation, the second image processing module specifically includes:

[0087] A network determination submodule, used for determining, for any of the conditional control images, a conditional control network corresponding to the conditional control image;

[0088] An image processing submodule, used for inputting the condition control image into the condition control network to obtain condition control features;

[0089] The image generation submodule is used to input the prompt text, the original text image and a plurality of the conditional control features into an image generation network to obtain a target text image.

[0090] In an optional embodiment, the conditional control network includes a first zero convolution layer, a second zero convolution layer, an encoding network, and a first intermediate block;

[0091] The image processing submodule specifically includes:

[0092] A first convolution unit, configured to perform convolution processing on the conditional control image using the first zero convolution layer to obtain a first convolution result;

[0093] A feature generation unit, used for inputting the first convolution result into the encoding network to obtain a conditionally controlled encoding feature;

[0094] A feature extraction unit, used for inputting the conditional control coding feature into the first intermediate block to obtain a first extracted feature;

[0095] The second convolution unit is used to perform convolution processing on the first refined feature by using the second zero convolution layer to obtain a second convolution result, and determine the second convolution result as a conditional control feature.

[0096] In an optional implementation, the encoding network includes M conditional control encoding blocks, where M is a positive integer; and the feature generation unit is specifically used for:

[0097] Iterate and perform the following operations until the Mth conditional encoding feature is obtained and then stop the following operations:

[0098] Inputting the i-th conditional feature into the i-th conditional control coding block to obtain the i-th conditional coding feature;

[0099] Among them, the i-th conditional coding feature is the i+1-th conditional feature, the 1st conditional feature is the first convolution result, the M-th conditional coding feature is the conditional control coding feature, and i is 1, 2, 3,..., M respectively.

[0100] In an optional embodiment, the image generation network includes a first encoder, a first decoder, a second encoding network, a second decoding network, and a second intermediate block;

[0101] The image generation submodule specifically includes:

[0102] A first coding feature generating unit, used for inputting the original text image into the first encoder to obtain a first coding feature;

[0103] A second coding feature generating unit, used for inputting the prompt text and the first coding feature into the first coding network to obtain a second coding feature;

[0104] A feature weighting unit, used to determine a first weight corresponding to each of the plurality of conditional control features, and perform weighted summation on the plurality of conditional control features according to the first weight to obtain a first weighted feature;

[0105] A second refined feature generating unit, used for inputting the prompt text, the second encoding feature and the first weighted feature into the second intermediate block to obtain a second refined feature;

[0106] A decoding feature generating unit, used for inputting the prompt text and the second refined feature into the second decoding network to obtain a decoding feature;

[0107] The image generating unit is used to input the decoding feature into the first decoder to obtain a target text image.

[0108] In an optional implementation, the second coding network includes M second coding blocks, where M is a positive integer; and the second coding feature generating unit is specifically configured to:

[0109] Iterate and perform the following operations until the Mth intermediate encoding feature is obtained and then stop the following operations:

[0110] Inputting the prompt text and the i-th first feature into the i-th second coding block to obtain the i-th intermediate coding feature;

[0111] Among them, the i-th intermediate coding feature is the i+1-th first feature, the 1st first feature is the first coding feature, the M-th intermediate coding feature is the second coding feature, and i is 1, 2, 3, ..., M respectively.

[0112] In an optional implementation, the second decoding network includes M second decoding blocks, where M is a positive integer; and the decoding feature generation unit specifically includes:

[0113] The iterator unit is used to iteratively perform the following steps until the first intermediate decoding feature is obtained and then stop the following operations:

[0114] an intermediate decoding feature generating subunit, configured to input the prompt text and the j-th second feature into the j-th second decoding block to obtain the j-th intermediate decoding feature;

[0115] Among them, the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M,...,3,2,1 respectively.

[0116] In an optional implementation, each conditional control network further includes M third zero convolutional layers, where M is a positive integer; and the intermediate decoding feature generation subunit is specifically used for:

[0117] For any conditional control network, determining a j-th conditional control coding block of a coding network in the conditional control network;

[0118] The j-th conditional control coding block of the coding network in the conditional control network is at the same level as the j-th second decoding block in the second decoding network;

[0119] Using the jth third zero convolution layer, convolution processing is performed on the jth conditional coding feature output by the jth conditional control coding block of the coding network in the conditional control network to obtain a jth second convolution result;

[0120] Determine a second weight corresponding to each of a plurality of j-th second convolution results, and perform weighted summation on the plurality of j-th second convolution results according to the second weight to obtain a j-th second weighted feature;

[0121] The prompt text, the jth second feature, and the jth second weighted feature are input into the jth second decoding block to obtain the jth intermediate decoding feature.

[0122] In an optional implementation, the intermediate decoding feature generation subunit is specifically used for:

[0123] In the first iteration, the Mth second feature and the Mth second weighted feature are input into the Mth second decoding block to obtain the Mth intermediate decoding feature;

[0124] In a non-first iteration process, the prompt text, the j-th second feature, and the j-th second weighted feature are input into the j-th second decoding block to obtain the j-th intermediate decoding feature;

[0125] Among them, the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M-1,...,3,2,1 respectively.

[0126] In a third aspect of the embodiments of the present application, there is further provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0127] Memory, used to store computer programs;

[0128] The processor is used to implement any image processing method described in the first aspect when executing the program stored in the memory.

[0129] In a fourth aspect of the embodiments of the present application, a storage medium is further provided, wherein instructions are stored in the storage medium, and when the storage medium is run on a computer, the computer executes the image processing method described in any one of the first aspects above.

[0130] In a fifth aspect of the embodiments of the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned image processing methods.

[0131] The technical solution provided by the embodiment of the present application obtains an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font, determines multiple control conditions of the original text image, processes the original text image according to the multiple control conditions, respectively, obtains multiple condition control images, processes the original text image according to the prompt text and the multiple condition control images, and obtains a target text image, wherein the text in the target text image is presented in a second font, and the first font is different from the second font.

[0132] By acquiring multiple control conditions of the original text image, the original text image is processed respectively according to the multiple control conditions to obtain multiple condition control images, and the original text image is processed according to the prompt text for the original text image and the multiple condition control images to obtain a target text image to complete the adjustment of the text font in the original text image. In this way, the font of the text in the original text image is adjusted through multiple control conditions, and the adjusted font is more creative and diverse, which can achieve the user's expected effect and save the user's time. BRIEF DESCRIPTION OF THE DRAWINGS

[0133] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0134] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0135] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0136] Figure 1 A schematic diagram of an implementation flow of an image processing method shown in an embodiment of the present application;

[0137] Figure 2 A schematic diagram of an original text image shown in an embodiment of the present application;

[0138] Figure 3 A schematic diagram of a target text image shown in an embodiment of the present application;

[0139] Figure 4 It is a schematic diagram of an implementation flow of another image processing method shown in an embodiment of the present application;

[0140] Figure 5 A schematic diagram of a graffiti image shown in an embodiment of the present application;

[0141] Figure 6 is a schematic diagram of a second edge image shown in an embodiment of the present application;

[0142] Figure 7 A schematic diagram of a depth image shown in an embodiment of the present application;

[0143] Figure 8 It is a schematic diagram of an implementation flow of another image processing method shown in an embodiment of the present application;

[0144] Fig. 9 A schematic diagram of the structure of a conditional control network shown in an embodiment of the present application;

[0145] Fig.10 It is a structural schematic diagram of another conditional control network shown in an embodiment of the present application;

[0146] Fig.11 A schematic diagram of the structure of an image generation network and a condition control network shown in an embodiment of the present application;

[0147] Fig.12 It is a structural schematic diagram of another image generation network and condition control network shown in the embodiment of the present application;

[0148] Fig.13 A schematic diagram of the structure of an image processing device shown in an embodiment of the present application;

[0149] Fig.14 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0150] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0151] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0152] For text images, the text therein is usually presented in a specific font. If you want to adjust the font of the text, you can provide hint text (the hint text is used to describe the text style) and control conditions (such as lines), and use the control algorithm corresponding to the control condition (such as edge detection algorithm) to process the text image to obtain a conditional control image (such as an edge image). Combine the conditional control image and the hint text to adjust the font of the text in the text image.

[0153] Based on the above invention concept, Figure 1 FIG. 1 is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0154] S101, obtaining an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font.

[0155] In an embodiment of the present application, an original text image may be obtained, wherein the text in the original text image is presented in a first font. For example, an original text image is obtained, wherein the text "apple" exists in the original text image, and the text "apple" in the original text image is presented in a first font, such as Figure 2 shown.

[0156] In addition to obtaining the above-mentioned original text image, it is also necessary to obtain a prompt text for the original text image. The prompt text is generally used to describe the text style and is used to adjust the style of the text in the original text image. For example, the prompt text can be "fruit", then the style of the text in the original text image will be adjusted to a style related to "fruit".

[0157] It should be noted that the original text image is usually a text image with white characters on a black background, such as Figure 2 As shown, the embodiments of the present application are not limited to this.

[0158] S102, determining a plurality of control conditions of the original text image, and processing the original text image according to the plurality of control conditions to obtain a plurality of condition-controlled images.

[0159] In an embodiment of the present application, for an original text image, multiple control conditions of the original text image are determined. For multiple control conditions, the adjustment effect of the first font of the text in the original text image can be affected, which means that the above-mentioned prompt text and multiple control conditions can jointly affect the adjustment effect of the first font of the text in the original text image.

[0160] To this end, the original text image is processed according to multiple control conditions to obtain multiple conditional control images, which means that for any control condition, the original text image is processed according to the control condition to obtain the corresponding conditional control image, thus obtaining multiple conditional control images.

[0161] It should be noted that the control conditions may be, for example, graffiti, lines, depth, etc. For this purpose, the original text image is processed separately according to multiple control conditions to obtain multiple conditional control images, which may be graffiti images, edge images, depth images, etc. The embodiments of the present application are not limited to this.

[0162] S103, processing the original text image according to the prompt text and the multiple condition control images to obtain a target text image, wherein the text in the target text image is presented in a second font, and the first font is different from the second font.

[0163] In an embodiment of the present application, after the above-mentioned processing, multiple conditional control images and prompt texts for the original text image can be obtained, thereby the original text image can be processed according to the prompt texts and the multiple conditional control images to obtain a target text image, wherein the text in the target text image is presented in a second font, and the first font is different from the second font.

[0164] It should be noted that, for the original text image and the target text image, the text contents of the two are the same, for example, both are "apple", and the only difference is the font of the text, that is, the text in the original text image is presented in a first font, and the text in the target text image is presented in a second font, such as Figure 3 As shown, the embodiments of the present application are not limited to this.

[0165] Through the above description of the technical solution provided in the embodiment of the present application, an original text image and a prompt text for the original text image are obtained, wherein the text in the original text image is presented in a first font, a plurality of control conditions of the original text image are determined, and the original text image is processed according to the plurality of control conditions respectively to obtain a plurality of conditional control images, and the original text image is processed according to the prompt text and the plurality of conditional control images to obtain a target text image, wherein the text in the target text image is presented in a second font, and the first font is different from the second font.

[0166] Through multiple control conditions of the original text image, the original text image is processed according to the multiple control conditions to obtain multiple condition control images. According to the prompt text for the original text image and the multiple condition control images, the original text image is processed to obtain the target text image to complete the adjustment of the text font in the original text image. In this way, the font of the text in the original text image is adjusted through multiple control conditions. The adjusted font is more creative and diverse, which can achieve the user's expected effect and save the user's time.

[0167] like Figure 4 FIG. 1 is a schematic diagram of an implementation flow of another image processing method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0168] S401, obtaining an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font.

[0169] In the embodiment of the present application, this step is similar to the above-mentioned step S101, and the embodiment of the present application will not be described one by one here.

[0170] S402, determining multiple control conditions of the original text image, and for any control condition, processing the original text image using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition.

[0171] In an embodiment of the present application, for an original text image, multiple control conditions of the original text image are determined. For multiple control conditions, the adjustment effect of the first font of the text in the original text image can be affected, which means that the above-mentioned prompt text and multiple control conditions can jointly affect the adjustment effect of the first font of the text in the original text image.

[0172] For this purpose, for any control condition, the original text image is processed using the conditional control algorithm corresponding to the control condition to obtain the conditional control image corresponding to the control condition, thereby obtaining multiple conditional control images, wherein the conditional control algorithms corresponding to each control condition are different.

[0173] When the control condition is graffiti, the conditional control algorithm corresponding to the control condition is used to process the original text image to obtain a conditional control image corresponding to the control condition. Specifically, the first edge detection algorithm may be used to perform edge detection on the original text image to obtain a first edge image, a Gaussian kernel may be used to perform a convolution operation on the first edge image to obtain a convolution image, and the convolution image may be threshold segmented to obtain a graffiti image.

[0174] For example, the HED edge detection algorithm is used to perform edge detection on the original text image to obtain the first edge image A, and a 3x3 Gaussian kernel is used to perform a convolution operation on the first edge image A to obtain a convolution image B. The convolution image B is threshold segmented to obtain a graffiti image, such as Figure 5 shown.

[0175] Among them, threshold segmentation of the convolution image is essentially to perform the following threshold segmentation processing on the pixel value of any pixel in the convolution image to obtain a graffiti image: when the pixel value is greater than a preset pixel value threshold (for example, 4), the pixel value is set to a preset first pixel value (for example, 255); when the pixel value is less than or equal to the preset pixel value threshold, the pixel value is set to a preset second pixel value (for example, 0).

[0176] In addition, when the control condition is a line, the conditional control algorithm corresponding to the control condition is used to process the original text image to obtain the conditional control image corresponding to the control condition. Specifically, the second edge detection algorithm can be used to perform edge detection on the original text image to obtain the second edge image. For example, the Canny edge detection algorithm is used to perform edge detection on the original text image to obtain the second edge image, such as Figure 6 As shown, the second edge image is a binary image.

[0177] In addition, when the control condition is depth, the conditional control algorithm corresponding to the control condition is used to process the original text image to obtain the conditional control image corresponding to the control condition. Specifically, the depth detection algorithm can be used to perform depth detection on the original text image to obtain a depth image. For example, the MiDaS depth detection algorithm is used to perform depth detection on the original text image to obtain a depth image, such as Figure 7 shown.

[0178] It should be noted that, for the above-mentioned first edge detection algorithm, in addition to the above-mentioned HED edge detection algorithm, it can also be other edge detection algorithms. Similarly, for the above-mentioned second edge detection algorithm, in addition to the above-mentioned Canny edge detection algorithm, it can also be other edge detection algorithms. Similarly, for the above-mentioned depth detection algorithm, in addition to the above-mentioned MiDaS depth detection algorithm, it can also be other depth detection algorithms. The embodiments of the present application do not limit this.

[0179] S403, processing the original text image according to the prompt text and the multiple condition control images to obtain a target text image, wherein the text in the target text image is presented in a second font, and the first font is different from the second font.

[0180] In the embodiment of the present application, this step is similar to the above-mentioned step S103, and the embodiment of the present application will not be described one by one here.

[0181] Through multiple control conditions of the original text image, the original text image is processed according to the multiple control conditions to obtain multiple condition control images. According to the prompt text for the original text image and the multiple condition control images, the original text image is processed to obtain a target text image to complete the adjustment of the text font in the original text image. In this way, the font of the text in the original text image is adjusted through multiple control conditions, and the adjusted font is more creative and diverse.

[0182] like Figure 8 FIG. 1 is a schematic diagram of an implementation flow of another image processing method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0183] S801, obtaining an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font.

[0184] In the embodiment of the present application, this step is similar to the above-mentioned step S101, and the embodiment of the present application will not be described one by one here.

[0185] S802, determining multiple control conditions of the original text image, and for any control condition, processing the original text image using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition.

[0186] In the embodiment of the present application, this step is similar to the above-mentioned step S402, and the embodiment of the present application will not be described one by one here.

[0187] S803: For any conditional control image, determine a conditional control network corresponding to the conditional control image.

[0188] S804, inputting the conditional control image into the conditional control network to obtain conditional control features.

[0189] In an embodiment of the present application, multiple conditional control images, such as graffiti images, edge images, depth images, etc., can be obtained through the above steps. For any conditional control image, the conditional control network corresponding to the conditional control image is determined, and the conditional control image is input into the conditional control network to obtain the conditional control features.

[0190] It should be noted that the network structure and parameters of the conditional control network corresponding to each conditional control image are the same, and the only difference is that different conditional control images are processed. For example, for a graffiti image, there is a corresponding graffiti network, for an edge image, there is a corresponding line network, and for a depth image, there is a corresponding depth network. The network structures and parameters of the graffiti network, the line network, and the depth network are the same, and the embodiments of the present application do not limit this.

[0191] Among them, for the conditional control network, it mainly includes the first zero convolution layer, the second zero convolution layer, the encoding network, and the first intermediate block (used to further refine the features output by the encoding network), such as Fig. 9 As shown. Thus, the conditional control image is convolved using the first zero convolution layer to obtain a first convolution result, the first convolution result is input into the encoding network to obtain a conditional control encoding feature, the conditional control encoding feature is input into the first intermediate block to obtain a first refined feature (the size of the first refined feature is, for example, 8x8), the first refined feature is convolved using the second zero convolution layer to obtain a second convolution result, and the second convolution result is determined as the conditional control feature.

[0192] For the encoding network, it usually includes M conditional control encoder blocks (Encoder Block), where M is a positive integer, and is formed by stacking M conditional control encoder blocks from top to bottom. The conditional control encoder block can be considered as a stack of multiple encoders. For example, the encoding network consists of 4 conditional control encoder blocks, namely conditional control encoder block 1, conditional control encoder block 2, conditional control encoder block 3, and conditional control encoder block 4. The 4 conditional control encoder blocks are stacked from top to bottom, such as Fig.10 As shown in the figure, the size of the conditional coding features generated by each conditional control coding block is different. For example, the size of the conditional coding features generated by conditional control coding block 1 is 64x64, the size of the conditional coding features generated by conditional control coding block 2 is 32x32, the size of the conditional coding features generated by conditional control coding block 3 is 16x16, and the size of the conditional coding features generated by conditional control coding block 4 is 8x8, and the sizes gradually decrease.

[0193] Thus, the first convolution result is input into the coding network to obtain the conditional control coding feature, which can be specifically performed iteratively until the Mth conditional coding feature is obtained and then the following operation is stopped: the i-th conditional feature is input into the i-th conditional control coding block to obtain the i-th conditional coding feature; wherein the i-th conditional coding feature is the i+1th conditional feature, the 1st conditional feature is the first convolution result, the Mth conditional coding feature is the conditional control coding feature, and i is 1, 2, 3, ..., M. The sizes of the above-mentioned conditional coding features are different and gradually decrease.

[0194] S805, inputting the prompt text, the original text image and a plurality of conditional control features into an image generation network to obtain a target text image, wherein the text in the target text image is presented in a second font, and the first font is different from the second font.

[0195] In an embodiment of the present application, after the above processing, multiple conditional control features can be obtained, thereby the prompt text for the original text image, the original text image and the multiple conditional control features can be input into the image generation network to obtain the target text image, wherein the text in the target text image is presented in a second font, and the first font of the text in the original text image is different from the second font.

[0196] For the image generation network, for example, it can be Stable Diffusion, which can specifically include a first encoder, a first decoder, a second encoding network, a second decoding network, and a second intermediate block (for further refining the features output by the second encoding network), such as Fig.11 As shown. The first encoder and the first decoder are, for example, VAE Encoder (encoder of variational differential autoencoder) and VAE Decoder (decoder of variational differential autoencoder), respectively, which are not limited in the embodiments of the present application.

[0197] Thus, the original text image is input into the first encoder to obtain the first encoding feature; the prompt text and the first encoding feature are input into the first encoding network to obtain the second encoding feature; the first weights corresponding to the multiple conditional control features are determined, and the multiple conditional control features are weighted and summed according to the first weights to obtain the first weighted feature; the prompt text, the second encoding feature and the first weighted feature are input into the second intermediate block to obtain the second refined feature (the size of the second refined feature is, for example, 8x8); the prompt text and the second refined feature are input into the second decoding network to obtain the decoded feature; the decoded feature is input into the first decoder to obtain the target text image.

[0198] It should be noted that the value range of the first weight corresponding to each of the multiple conditional control features is randomly generated, for example, between 0.4 and 1.0. In this way, the first weight corresponding to each of the multiple conditional control features is randomly generated and limited to between 0.4 and 1.0. This can achieve the effect of creatively generating the target text image, making the adjustment of the font of the text in the original text image more creative.

[0199] Among them, for the second encoding network, it usually includes M second encoding blocks (Encoder Block), M is a positive integer, and is formed by stacking M second encoding blocks from top to bottom. The second encoding block can be considered as a stack of multiple encoders, and the sizes of the intermediate encoding features output by each second encoding block are different. For example, for the second encoding network, it includes 4 second encoding blocks, namely second encoding block 1, second encoding block 2, second encoding block 3, and second encoding block 4. Fig.12As shown, the size of the intermediate coding features output by the second coding block 1 is 64x64, the size of the intermediate coding features output by the second coding block 2 is 32x32, the size of the intermediate coding features output by the second coding block 3 is 16x16, and the size of the intermediate coding features output by the second coding block 4 is 8x8, and the size decreases gradually.

[0200] Thus, the prompt text and the first coding feature are input into the first coding network to obtain the second coding feature. Specifically, the following operations may be performed iteratively until the Mth intermediate coding feature is obtained and the following operations are stopped: the prompt text and the i-th first feature are input into the i-th second coding block to obtain the i-th intermediate coding feature; wherein the i-th intermediate coding feature is the i+1th first feature, the 1st first feature is the first coding feature, the M-th intermediate coding feature is the second coding feature, and i is 1, 2, 3, ..., M. The sizes of the intermediate coding features are different and gradually decrease.

[0201] In addition, the second decoding network also includes M second decoding blocks (Decoder Block), where M is a positive integer, and is formed by stacking M second decoding blocks from bottom to top. The second decoding block can be understood as being formed by stacking multiple decoders. For example, the second decoding network includes 4 second decoding blocks, namely, second decoding block 1, second decoding block 2, second decoding block 3, and second decoding block 4. Fig.12 As shown, the size of the intermediate decoding feature output by the second decoding block 1 is 8x8, the size of the intermediate decoding feature output by the second decoding block 2 is 16x16, the size of the intermediate decoding feature output by the second decoding block 3 is 32x32, and the size of the intermediate decoding feature output by the second decoding block 4 is 64x64, and the sizes gradually increase.

[0202] Thus, the prompt text and the second refined features are input into the second decoding network to obtain the decoding features. Specifically, the following steps may be iteratively performed until the first intermediate decoding feature is obtained and the following operation is stopped: the prompt text and the jth second feature are input into the jth second decoding block to obtain the jth intermediate decoding feature; wherein the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M, ..., 3, 2, 1. The sizes of the intermediate decoding features are different and gradually increase.

[0203] In addition, in the embodiment of the present application, for the encoding network in the conditional control network, there is a short connection between it and the second decoding network, that is, there is a short connection between the conditional control encoding block of the encoding network in the conditional control network at the same level and the second decoding block of the second decoding network in the image generation network, such as Fig.12 shown.

[0204] Based on this, the above-mentioned prompt text and the jth second feature are input into the jth second decoding block to obtain the jth intermediate decoding feature. Specifically, for any conditional control network, the jth conditional control coding block of the encoding network in the conditional control network is determined; the jth conditional control coding block of the encoding network in the conditional control network is at the same level as the jth second decoding block in the second decoding network; using the jth third zero convolution layer, the jth conditional coding feature output by the jth conditional control coding block of the encoding network in the conditional control network is convolved to obtain the jth second convolution result; the second weights corresponding to each of the multiple jth second convolution results are determined, and according to the second weights, the multiple jth second convolution results are weighted and summed to obtain the jth second weighted feature; the prompt text, the jth second feature, and the jth second weighted feature are input into the jth second decoding block to obtain the jth intermediate decoding feature.

[0205] For example, taking the fourth second decoding block as an example, for any conditional control network, the fourth conditional control coding block of the coding network in the conditional control network is determined, and the fourth conditional control coding block of the coding network in the conditional control network is at the same level as the fourth second decoding block in the second decoding network, such as Fig.12 As shown, the 4th conditional coding feature output by the 4th conditional control coding block of the coding network in the conditional control network is convolved using the 4th third zero convolution layer to obtain the 4th second convolution result; the second weights corresponding to the multiple 4th second convolution results are determined, and the multiple 4th second convolution results are weighted and summed according to the second weights to obtain the 4th second weighted feature; the prompt text, the 4th second feature, and the 4th second weighted feature are input into the 4th second decoding block to obtain the 4th intermediate decoding feature.

[0206] It should be noted that the second weight corresponding to each of the multiple j-th second convolution results has a value range that is randomly generated, for example, between 0.4 and 1.0. In this way, the second weight corresponding to each of the multiple j-th second convolution results is randomly generated and limited to between 0.4 and 1.0, which can achieve the effect of creatively generating the target text image, making the adjustment of the font of the text in the original text image more creative.

[0207] In addition, for the second decoding network, no prompt text is required during the first iteration, that is, the input of the first iteration does not require prompt text, such as Fig.12As shown. To this end, in the first iteration, the Mth second feature and the Mth second weighted feature are input into the Mth second decoding block to obtain the Mth intermediate decoding feature; in the non-first iteration, the prompt text, the jth second feature, and the jth second weighted feature are input into the jth second decoding block to obtain the jth intermediate decoding feature; wherein the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the 1st intermediate decoding feature is the decoding feature, and j is M-1, ..., 3, 2, 1 in sequence.

[0208] Through multiple control conditions of the original text image, the original text image is processed according to the multiple control conditions to obtain multiple condition control images. According to the prompt text for the original text image and the multiple condition control images, the original text image is processed to obtain a target text image to complete the adjustment of the text font in the original text image. In this way, the font of the text in the original text image is adjusted through multiple control conditions, and the adjusted font is more creative and diverse.

[0209] Corresponding to the above method embodiment, the present application embodiment also provides an image processing device, such as Fig.13 As shown, the device may include: an image and text acquisition module 1310 , a condition determination module 1320 , a first image processing module 1330 , and a second image processing module 1340 .

[0210] An image and text acquisition module 1310 is used to acquire an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font;

[0211] A condition determination module 1320, configured to determine a plurality of control conditions of the original text image;

[0212] A first image processing module 1330 is used to process the original text image according to the multiple control conditions to obtain multiple condition control images;

[0213] A second image processing module 1340 is used to process the original text image according to the prompt text and the plurality of conditional control images to obtain a target text image;

[0214] The text in the target text image is presented in a second font, and the first font is different from the second font.

[0215] In an optional implementation, the first image processing module specifically includes:

[0216] The original text image processing submodule is used to process the original text image according to any of the control conditions using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition.

[0217] In an optional implementation, when the control condition is graffiti, the original text image processing submodule specifically includes:

[0218] An edge detection unit, configured to perform edge detection on the original text image using a first edge detection algorithm to obtain a first edge image;

[0219] An image convolution unit, configured to perform a convolution operation on the first edge image using a Gaussian kernel to obtain a convolution image;

[0220] The image segmentation unit is used to perform threshold segmentation on the convolution image to obtain a graffiti image.

[0221] In an optional implementation, the image segmentation unit is specifically used to:

[0222] For the pixel value of any pixel in the convolution image, the following processing is performed to obtain a graffiti image:

[0223] When the pixel value is greater than a preset pixel value threshold, setting the pixel value to a preset first pixel value;

[0224] When the pixel value is less than or equal to the preset pixel value threshold, the pixel value is set to a preset second pixel value.

[0225] In an optional implementation, when the control condition is a line, the original text image processing submodule is specifically used for:

[0226] Performing edge detection on the original text image using a second edge detection algorithm to obtain a second edge image;

[0227] When the control condition is depth, the original text image processing submodule is specifically used for:

[0228] A depth detection algorithm is used to perform depth detection on the original text image to obtain a depth image.

[0229] In an optional implementation, the second image processing module specifically includes:

[0230] A network determination submodule, used for determining, for any of the conditional control images, a conditional control network corresponding to the conditional control image;

[0231] An image processing submodule, used for inputting the condition control image into the condition control network to obtain condition control features;

[0232] The image generation submodule is used to input the prompt text, the original text image and a plurality of the conditional control features into an image generation network to obtain a target text image.

[0233] In an optional embodiment, the conditional control network includes a first zero convolution layer, a second zero convolution layer, an encoding network, and a first intermediate block;

[0234] The image processing submodule specifically includes:

[0235] A first convolution unit, configured to perform convolution processing on the conditional control image using the first zero convolution layer to obtain a first convolution result;

[0236] A feature generation unit, used for inputting the first convolution result into the encoding network to obtain a conditionally controlled encoding feature;

[0237] A feature extraction unit, used for inputting the conditional control coding feature into the first intermediate block to obtain a first extracted feature;

[0238] The second convolution unit is used to perform convolution processing on the first refined feature by using the second zero convolution layer to obtain a second convolution result, and determine the second convolution result as a conditional control feature.

[0239] In an optional implementation, the encoding network includes M conditional control encoding blocks, where M is a positive integer; and the feature generation unit is specifically used for:

[0240] Iterate and perform the following operations until the Mth conditional encoding feature is obtained and then stop the following operations:

[0241] Inputting the i-th conditional feature into the i-th conditional control coding block to obtain the i-th conditional coding feature;

[0242] Among them, the i-th conditional coding feature is the i+1-th conditional feature, the 1st conditional feature is the first convolution result, the M-th conditional coding feature is the conditional control coding feature, and i is 1, 2, 3,..., M respectively.

[0243] In an optional embodiment, the image generation network includes a first encoder, a first decoder, a second encoding network, a second decoding network, and a second intermediate block;

[0244] The image generation submodule specifically includes:

[0245] A first coding feature generating unit, used for inputting the original text image into the first encoder to obtain a first coding feature;

[0246] A second coding feature generating unit, used for inputting the prompt text and the first coding feature into the first coding network to obtain a second coding feature;

[0247] A feature weighting unit, used to determine a first weight corresponding to each of the plurality of conditional control features, and perform weighted summation on the plurality of conditional control features according to the first weight to obtain a first weighted feature;

[0248] A second refined feature generating unit, used for inputting the prompt text, the second encoding feature and the first weighted feature into the second intermediate block to obtain a second refined feature;

[0249] A decoding feature generating unit, used for inputting the prompt text and the second refined feature into the second decoding network to obtain a decoding feature;

[0250] The image generating unit is used to input the decoding feature into the first decoder to obtain a target text image.

[0251] In an optional implementation, the second coding network includes M second coding blocks, where M is a positive integer; and the second coding feature generating unit is specifically configured to:

[0252] Iterate and perform the following operations until the Mth intermediate encoding feature is obtained and then stop the following operations:

[0253] Inputting the prompt text and the i-th first feature into the i-th second coding block to obtain the i-th intermediate coding feature;

[0254] Among them, the i-th intermediate coding feature is the i+1-th first feature, the 1st first feature is the first coding feature, the M-th intermediate coding feature is the second coding feature, and i is 1, 2, 3, ..., M respectively.

[0255] In an optional implementation, the second decoding network includes M second decoding blocks, where M is a positive integer; and the decoding feature generation unit specifically includes:

[0256] The iterator unit is used to iteratively perform the following steps until the first intermediate decoding feature is obtained and then stop the following operations:

[0257] an intermediate decoding feature generating subunit, configured to input the prompt text and the j-th second feature into the j-th second decoding block to obtain the j-th intermediate decoding feature;

[0258] Among them, the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M,...,3,2,1 respectively.

[0259] In an optional implementation, each conditional control network further includes M third zero convolutional layers, where M is a positive integer; and the intermediate decoding feature generation subunit is specifically used for:

[0260] For any conditional control network, determining a j-th conditional control coding block of a coding network in the conditional control network;

[0261] The j-th conditional control coding block of the coding network in the conditional control network is at the same level as the j-th second decoding block in the second decoding network;

[0262] Using the jth third zero convolution layer, convolution processing is performed on the jth conditional coding feature output by the jth conditional control coding block of the coding network in the conditional control network to obtain a jth second convolution result;

[0263] Determine a second weight corresponding to each of a plurality of j-th second convolution results, and perform weighted summation on the plurality of j-th second convolution results according to the second weight to obtain a j-th second weighted feature;

[0264] The prompt text, the jth second feature, and the jth second weighted feature are input into the jth second decoding block to obtain the jth intermediate decoding feature.

[0265] In an optional implementation, the intermediate decoding feature generation subunit is specifically used for:

[0266] In the first iteration, the Mth second feature and the Mth second weighted feature are input into the Mth second decoding block to obtain the Mth intermediate decoding feature;

[0267] In a non-first iteration process, the prompt text, the j-th second feature, and the j-th second weighted feature are input into the j-th second decoding block to obtain the j-th intermediate decoding feature;

[0268] Among them, the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M-1,...,3,2,1 respectively.

[0269] The present application also provides an electronic device, such as Fig.14As shown, it includes a processor 141, a communication interface 142, a memory 143 and a communication bus 144, wherein the processor 141, the communication interface 142, and the memory 143 communicate with each other through the communication bus 144.

[0270] Memory 143, used for storing computer programs;

[0271] The processor 141 is used to execute the program stored in the memory 143 to implement the following steps:

[0272] Acquire an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font; determine multiple control conditions of the original text image, and process the original text image according to the multiple control conditions to obtain multiple conditional control images; process the original text image according to the prompt text and the multiple conditional control images to obtain a target text image; wherein the text in the target text image is presented in a second font, and the first font is different from the second font.

[0273] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0274] The communication interface is used for communication between the above electronic device and other devices.

[0275] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0276] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0277] In another embodiment provided in the present application, a storage medium is provided. The storage medium stores instructions. When the instructions are executed on a computer, the computer executes any one of the image processing methods in the above embodiments.

[0278] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the image processing methods in the above embodiments.

[0279] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a storage medium, or transmitted from one storage medium to another storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0280] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0281] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0282] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font; Determine a plurality of control conditions of the original text image, and process the original text image according to the plurality of control conditions to obtain a plurality of condition-controlled images; According to the prompt text and the plurality of condition control images, the original text image is processed to obtain a target text image; The text in the target text image is presented in a second font, and the first font is different from the second font.

2. The method according to claim 1, characterized in that The processing of the original text image according to the plurality of control conditions to obtain a plurality of condition-controlled images includes: For any of the control conditions, the original text image is processed using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition.

3. The method according to claim 2, characterized in that In the case where the control condition is graffiti, the processing of the original text image using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition includes: Performing edge detection on the original text image using a first edge detection algorithm to obtain a first edge image; A convolution operation is performed on the first edge image using a Gaussian kernel to obtain a convolution image, and a threshold segmentation is performed on the convolution image to obtain a graffiti image.

4. The method according to claim 3, characterized in that The step of performing threshold segmentation on the convolution image to obtain a graffiti image includes: For the pixel value of any pixel in the convolution image, the following processing is performed to obtain a graffiti image: When the pixel value is greater than a preset pixel value threshold, setting the pixel value to a preset first pixel value; When the pixel value is less than or equal to the preset pixel value threshold, the pixel value is set to a preset second pixel value.

5. The method according to claim 2, characterized in that: In the case where the control condition is a line, the original text image is processed using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition, including: Performing edge detection on the original text image using a second edge detection algorithm to obtain a second edge image; In the case where the control condition is depth, the processing of the original text image using a conditional control algorithm corresponding to the control condition to obtain a conditional control image corresponding to the control condition includes: A depth detection algorithm is used to perform depth detection on the original text image to obtain a depth image.

6. The method according to claim 1, characterized in that The processing of the original text image according to the prompt text and the plurality of condition control images to obtain a target text image includes: For any of the conditional control images, determining a conditional control network corresponding to the conditional control image; Inputting the conditional control image into the conditional control network to obtain conditional control features; The prompt text, the original text image and a plurality of the conditional control features are input into an image generation network to obtain a target text image.

7. The method according to claim 6, characterized in that The conditional control network includes a first zero convolution layer, a second zero convolution layer, an encoding network, and a first intermediate block; The step of inputting the conditional control image into the conditional control network to obtain conditional control features includes: Using the first zero convolution layer to perform convolution processing on the conditional control image to obtain a first convolution result; Inputting the first convolution result into the encoding network to obtain a conditionally controlled encoding feature; Inputting the conditional control coding feature into the first intermediate block to obtain a first refined feature; The first extracted feature is convolved using the second zero convolution layer to obtain a second convolution result, and the second convolution result is determined as a conditional control feature.

8. The method according to claim 7, characterized in that The coding network includes M conditional control coding blocks, where M is a positive integer; The step of inputting the first convolution result into the encoding network to obtain a conditionally controlled encoding feature includes: Iterate and perform the following operations until the Mth conditional encoding feature is obtained and then stop the following operations: Inputting the i-th conditional feature into the i-th conditional control coding block to obtain the i-th conditional coding feature; Among them, the i-th conditional coding feature is the i+1-th conditional feature, the 1st conditional feature is the first convolution result, the M-th conditional coding feature is the conditional control coding feature, and i is 1, 2, 3,..., M respectively.

9. The method according to claim 6, characterized in that The image generation network includes a first encoder, a first decoder, a second encoding network, a second decoding network, and a second intermediate block; The step of inputting the prompt text, the original text image, and the plurality of conditional control features into an image generation network to obtain a target text image comprises: Inputting the original text image into the first encoder to obtain a first encoding feature; Inputting the prompt text and the first coding feature into the first coding network to obtain a second coding feature; Determine a first weight corresponding to each of the plurality of conditional control features, and perform weighted summation on the plurality of conditional control features according to the first weight to obtain a first weighted feature; Inputting the prompt text, the second encoding feature and the first weighted feature into the second intermediate block to obtain a second refined feature; Inputting the prompt text and the second refined features into the second decoding network to obtain decoding features; The decoded features are input into the first decoder to obtain a target text image.

10. The method according to claim 9, characterized in that The second coding network includes M second coding blocks, where M is a positive integer; The step of inputting the prompt text and the first coding feature into the first coding network to obtain a second coding feature includes: Iterate and perform the following operations until the Mth intermediate encoding feature is obtained and then stop the following operations: Inputting the prompt text and the i-th first feature into the i-th second coding block to obtain the i-th intermediate coding feature; Among them, the i-th intermediate coding feature is the i+1-th first feature, the 1st first feature is the first coding feature, the M-th intermediate coding feature is the second coding feature, and i is 1, 2, 3, ..., M respectively.

11. The method according to claim 9, characterized in that The second decoding network includes M second decoding blocks, where M is a positive integer; The step of inputting the prompt text and the second refined features into the second decoding network to obtain the decoding features includes: Iterate the following steps until the first intermediate decoding feature is obtained and stop the following operations: Inputting the prompt text and the j-th second feature into the j-th second decoding block to obtain the j-th intermediate decoding feature; Among them, the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M,...,3,2,1 respectively.

12. The method according to claim 11, characterized in that Each conditional control network further includes M third zero convolutional layers, where M is a positive integer; The step of inputting the prompt text and the j-th second feature into the j-th second decoding block to obtain the j-th intermediate decoding feature includes: For any conditional control network, determining a j-th conditional control coding block of a coding network in the conditional control network; The j-th conditional control coding block of the coding network in the conditional control network is at the same level as the j-th second decoding block in the second decoding network; Using the jth third zero convolution layer, convolution processing is performed on the jth conditional coding feature output by the jth conditional control coding block of the coding network in the conditional control network to obtain a jth second convolution result; Determine a second weight corresponding to each of a plurality of j-th second convolution results, and perform weighted summation on the plurality of j-th second convolution results according to the second weight to obtain a j-th second weighted feature; The prompt text, the jth second feature, and the jth second weighted feature are input into the jth second decoding block to obtain the jth intermediate decoding feature.

13. The method according to claim 12, characterized in that The step of inputting the prompt text, the j-th second feature, and the j-th second weighted feature into the j-th second decoding block to obtain the j-th intermediate decoding feature comprises: In the first iteration, the Mth second feature and the Mth second weighted feature are input into the Mth second decoding block to obtain the Mth intermediate decoding feature; In a non-first iteration process, the prompt text, the j-th second feature, and the j-th second weighted feature are input into the j-th second decoding block to obtain the j-th intermediate decoding feature; Among them, the jth intermediate decoding feature is the j-1th second feature, the Mth second feature is the second refined feature, the first intermediate decoding feature is the decoding feature, and j is M-1,...,3,2,1 respectively.

14. An image processing device, characterized in that: The device comprises: An image and text acquisition module, used to acquire an original text image and a prompt text for the original text image, wherein the text in the original text image is presented in a first font; A condition determination module, used for determining a plurality of control conditions of the original text image; A first image processing module, used for processing the original text image according to the plurality of control conditions to obtain a plurality of condition-controlled images; A second image processing module, used for processing the original text image according to the prompt text and the plurality of conditional control images to obtain a target text image; The text in the target text image is presented in a second font, and the first font is different from the second font.

15. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-13 when executing a program stored in a memory.

16. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images