A character generation method and device, electronic equipment and storage medium

Abstract

The application discloses a character generation method and device, electronic equipment and storage medium, and the method comprises the steps of obtaining a to-be-processed image corresponding to to-be-processed characters and reference characters respectively; inputting the to-be-processed image into a target font style fusion model to obtain target characters of the to-be-processed characters under a target font style; wherein the target font style is determined by fusing a reference font style of the reference characters and a to-be-processed font style of the to-be-processed characters based on the font style fusion model. The problem that the existing font style cannot meet the user's demand is solved, and the effect of generating characters between two font styles is realized by fusing the font styles of any two characters.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to a character generation method and device, electronic equipment and storage medium. BACKGROUND

[0002] Existing style transfer or picture translation technology is better at modifying the texture of a picture and is not good at modifying the structural information of the picture. The interval structure is exactly an important distinguishing point between various fonts, which leads to the fact that there are many bad cases (such as stroke breakage, edge unsmoothness, stroke loss or redundancy, etc.) in the generated font in the existing technology performing the style transfer or picture translation task on font data, which makes a great gap between the result obtained by AI font fusion and the actual landing requirement. SUMMARY

[0003] The present application provides a character generation method and device, electronic equipment and storage medium to realize the effect of generating a character with a font style between two font styles.

[0004] In a first aspect, an embodiment of the present application provides a character generation method, which comprises:

[0005] obtaining a to-be-processed image corresponding to a to-be-processed character and a reference image corresponding to a reference character respectively;

[0006] inputting the to-be-processed image into a target font style fusion model to obtain a target character of the to-be-processed character under a target font style.

[0007] The target font style is determined based on the font style fusion model fusing a reference font style of the reference character and a to-be-processed font style of the to-be-processed character.

[0008] In a second aspect, an embodiment of the present application further provides a character generation device, which comprises:

[0009] a to-be-processed image acquisition module configured to acquire a to-be-processed image corresponding to a to-be-processed character and a reference image corresponding to a reference character respectively;

[0010] a target character determination module configured to input the to-be-processed image into a target font style fusion model to obtain a target character of the to-be-processed character under a target font style.

[0011] The target font style is determined based on the font style fusion model fusing a reference font style of the reference character and a to-be-processed font style of the to-be-processed character.

[0012] In a third aspect, an embodiment of the present application further provides an electronic equipment, which comprises:

[0013] one or more processors;

[0014] a memory device for storing one or more programs,

[0015] when the one or more programs are executed by the one or more processors, the one or more processors implement the text generation method according to any of the embodiments of the present disclosure.

[0016] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium containing computer executable instructions for executing the text generation method according to any of the embodiments of the present disclosure when executed by a computer processor.

[0017] The technical solution of the embodiments of the present disclosure obtains a to-be-processed image corresponding to the to-be-processed text and a reference text respectively, fuses the to-be-processed text font style and the reference font style based on a target font style fusion model to obtain any font style between the to-be-processed text and the reference text, and repeatedly fuses the font style according to the user's demand until the font style of the text consistent with the user's demand is obtained. The to-be-processed image is input into the target font style fusion model to obtain a target text of the to-be-processed text in the target font style, which meets the user's demand of converting the font style of the to-be-processed text into the target font style. The problem that the text with a font style between two font styles cannot be generated is solved, and the effect of generating the text corresponding to any font style between two font styles is achieved by fusing the two font styles into any target font style between the two font styles and generating the text consistent with the target font style. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, same or similar reference numerals can represent same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0019] Figure 1 A flowchart of a text generation method provided by the first embodiment of the present disclosure;

[0020] Figure 2 A schematic diagram of a target font style fusion model provided by the first embodiment of the present disclosure;

[0021] Figure 3 A schematic diagram of a target text style provided by the first embodiment of the present disclosure;

[0022] Figure 4A flowchart of a text generation method provided for Embodiment Two of the present disclosure is shown in FIG. 1.

[0023] Figure 5 A structural diagram of a text generation apparatus provided for Embodiment Three of the present disclosure is shown in FIG. 2.

[0024] Figure 6 A structural diagram of an electronic device provided for Embodiment Four of the present disclosure is shown in FIG. 3. DETAILED DESCRIPTION

[0025] Embodiments of the present disclosure will be described in more detail by referring to the drawings. Although certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, but rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of the present disclosure.

[0026] It should be understood that the various steps in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this respect.

[0027] The term "comprising" and variations thereof as used in the present disclosure are open-ended, that is, "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions are given throughout the description.

[0028] It should be noted that the terms "first", "second", and the like in the present disclosure are merely used to distinguish different devices, modules or units, and do not imply the order or interdependence of the functions performed by these devices, modules or units.

[0029] It should be noted that the terms "one", "multiple" in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that "one or more" should be understood unless otherwise explicitly stated in the context.

[0030] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the scope of the messages or information.

[0031] Embodiment One

[0032] Figure 1A flowchart of a character generation method provided by Embodiment One of the present disclosure is shown. Embodiment One can be applied to the case of fusing the font styles of two fonts to obtain a character with a font style between the font styles of the two fonts. The method can be executed by a character generation device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC terminal, or a server, etc.

[0033] Before introducing the technical solution, an exemplary application scenario is described. The technical solution can be applied to a scenario of generating a font style between two font styles based on the obtained font styles. The obtained font styles can be a font style with copyright, such as the Songti style or the Kaishu style in a drop-down menu of font style selection, or a font style of user handwriting, which is not limited herein. That is, the user wants to convert the font style of the character to a font style between any two font styles in the drop-down menu of font style selection, that is, the user expects to generate a character with a font style that contains both the A font style and the B font style, but is not completely consistent with the A font style or the B font style. Based on the solution of the present embodiment, a character with a font style between any two font styles can be generated, and the font style of the generated character is based on the font style between the two font styles input by the user.

[0034] As shown in FIG. 1, Figure 1 The method of the present embodiment includes the following steps.

[0035] In S110, an image to be processed corresponding to the character to be processed and a reference character is obtained.

[0036] The character to be processed can be understood as a character that the user expects to convert the font style. The character to be processed can be a character selected by the user from a font library, or a character written by the user. For example, after the user writes a character, the written character is recognized by image recognition, and the recognized character is taken as the character to be processed. The reference character can be understood as a character whose font style needs to be fused with the font style of the character to be processed. For example, the font style of the reference character can include a font style with copyright, such as the Kaishu style, the Lishu style, the Xingshu style, the Caoshu style, the Songti style, or a user's handwriting font style, etc. The image to be processed can be understood as an image corresponding to the character to be processed or an image corresponding to the reference character.

[0037] Specifically, an image corresponding to the to-be-processed text or an image corresponding to the reference text can be obtained from a text database, and the obtained image is taken as the to-be-processed image; or the user can write the text by himself / herself, and then take the written text as an image, and the image corresponding to the text written by the user is taken as the to-be-processed image. Further, after obtaining the to-be-processed image, the font style and font features of the to-be-processed text and the reference text can be obtained by recognizing the text in the to-be-processed image. It should be noted that the font style of the to-be-processed text and the reference text can be the same or different.

[0038] Optionally, the obtaining of the to-be-processed image corresponding to the to-be-processed text and the reference text respectively includes: generating the to-be-processed image corresponding to the to-be-processed text and the reference text based on the to-be-processed text and the reference text edited in the editing control.

[0039] The editing control can be understood as a control for inputting the to-be-processed text or the reference text. For example, the editing control can be arranged in the interface of the font selection system, so as to facilitate the user to input the to-be-processed text or the reference text. After the to-be-processed text or the reference text is input in the editing control, the image processing module in the font selection system can process the to-be-processed text or the reference text to obtain the to-be-processed image corresponding to the to-be-processed text and the reference text.

[0040] Specifically, the editing control is arranged in the text selection system, and the user can edit the to-be-processed text and the reference text in the editing control and click to confirm the to-be-processed text and the reference text. Then, the to-be-processed text and the reference text are sent to the image processing module in the text selection system, and the image processing module converts the to-be-processed text or the reference text into an image to obtain the to-be-processed image corresponding to the to-be-processed text and the reference text. Optionally, the to-be-processed text and the reference text can also be the handwritten text of the user, and after the handwriting is completed, the handwritten text of the user is taken as an image as the to-be-processed image.

[0041] S120, inputting the to-be-processed image into a target font style fusion model to obtain a target text of the to-be-processed text in a target font style.

[0042] Among them, the target font style fusion model can be understood as a model that fuses different font styles. The target font style fusion model can be a pre-trained neural network model, such as a convolutional neural network model. The format of the input data of this model is in image format, and correspondingly, the format of the output data is also in image format. The target font style can be understood as being fused according to the text styles of the text to be processed and the reference text, obtaining any font style between the two font styles. It should be noted that there can be multiple fused font styles, and any one of them can be used as the target font style. The target text can be understood as the text with the target font style.

[0043] Specifically, input the image to be processed corresponding to the text to be processed and the image to be processed corresponding to the reference text into the target font style fusion model. Refer to Figure 2 , input the image to be processed corresponding to the text "仓" as the text to be processed and the image to be processed corresponding to the reference text "颉" into the target font style fusion model. Among them, the font styles of the texts in the two images are different. After processing the two images to be processed based on the target font style fusion model, an image of the character "仓" in the font style of the character "颉" can be obtained. For example, an image of the character "仓" with the same font style as the character "颉" can be obtained, or an image of the character "仓" between the font style of the text to be processed and the font style of the reference text can be obtained. Take any one of the font styles as the target font style and obtain the target text corresponding to the target font style.

[0044] It should be noted that if the obtained target font style does not match the font style required by the user, the user can use the text with the target font style as the text to be processed and continue to fuse the font style until a font style satisfactory to the user is obtained.

[0045] Exemplarily, taking the font style processing of "济" as an example, refer to Figure 3The font style corresponding to the "Ji" character in the figure is a font style with copyright, and is only used as an example to illustrate the font style copyright, and is not limited to the font style copyright. Inputting the image to be processed corresponding to number 1 and the image to be processed corresponding to number 10 into the target font style fusion model can obtain any font style between the two font styles, for example, any font style between number 2 and number 9 can be obtained, and any font style can be used as a target font style. For example, if the target font style obtained is the font style of number 5, and the font style actually required by the user is the font style of number 8, that is, the target font style obtained is different from the font style expected by the user, the font style can be further fused based on the font style fusion model. Optionally, number 5 and number 10 are input as images to be processed into the target font style fusion model for processing until a target font style consistent with the font style expected by the user is obtained.

[0046] Optionally, based on the target font style fusion model, the to-be-used characters of each character in the target font style are generated, and the character package is generated based on the to-be-used characters.

[0047] Optionally, based on the target font style fusion model, the to-be-used characters of each character in the target font style are generated, and the character package is generated based on the to-be-used characters.

[0048] Optionally, when it is detected that the font style selected from the font style list is the target font style and it is detected that the to-be-processed character is edited, the target character corresponding to the to-be-processed character is obtained from the character package.

[0049] Optionally, based on the target font style fusion model, the to-be-used characters of each character in the target font style are generated, and the character package is generated based on the to-be-used characters.

[0050] Specifically, the existing font style is included in the font style list, and the font style generated based on the target font style fusion model is also included in the font style list. The font style selected by the user in the font style list is taken as the target font style. Then, when the user-edited to-be-processed text is detected, the same text as the to-be-processed text is obtained from the text package, so that the font style of the to-be-processed text is adapted to the font style selected by the user.

[0051] For example, the font style selected by the user in the font style list is the fused font style A. When the input to-be-processed text "ke" is received, the "ke" character can be determined from the text package corresponding to the target font style A and displayed as the target text. It should be noted that the technical solution can be applied to an existing office software, integrated into the office software, integrated into the text package, or integrated into an application software.

[0052] The technical solution of the embodiment of the present disclosure obtains a to-be-processed image corresponding to the to-be-processed text and a reference image corresponding to the reference text, respectively, to fuse the font styles of the to-be-processed text and the reference text based on the target font style fusion model, obtain any font style between the font styles of the to-be-processed text and the reference text, and repeatedly fuse the font styles according to the user's demand until the font style of the text consistent with the user's demand is obtained. The to-be-processed image is input into the target font style fusion model to obtain the target text of the to-be-processed text in the target font style, which meets the user's demand of converting the font style of the to-be-processed text into the target font style. The problem of being unable to generate text with a font style between two font styles is solved. By fusing the two font styles into any target font style between the two font styles and generating text consistent with the target font style, the effect of generating text corresponding to any font style between the two font styles is achieved.

[0053] Embodiment Two

[0054] Figure 4 The flowchart of the text generation method provided by the second embodiment of the present disclosure is based on the foregoing embodiment. The target font style fusion model includes a font style extraction sub-model, a stroke feature extraction sub-model, an image feature extraction sub-model, and a coding sub-model. Before the font styles of the two fonts are fused based on the target font style fusion model, the stroke feature extraction sub-model can be pre-trained to construct a to-be-trained font style fusion model based on the stroke feature extraction sub-model, and then the font style fusion model is trained. The same or corresponding technical terms as in the above embodiment are not repeated here.

[0055] AsFigure 4 As shown, the method specifically includes:

[0056] S210. Train the stroke feature extraction sub-model in the target font style fusion model.

[0057] In this embodiment, the training of the stroke feature extraction sub-model in the target font style fusion model includes: obtaining a first training sample set; where the first training sample set includes multiple first training samples, and each first training sample includes a first image corresponding to a first training character and a first stroke vector; for each first training sample, use the first image of the current first training sample as the input parameter of the to-be-trained stroke feature extraction sub-model, and use the corresponding first stroke vector as the output parameter of the to-be-trained stroke feature extraction sub-model, and train the to-be-trained stroke feature extraction sub-model to obtain the stroke feature extraction sub-model.

[0058] Among them, the stroke feature extraction sub-model can be used to extract the stroke features of characters. In the actual application process, in order to improve the accuracy of the model, as many training samples as possible can be obtained to adjust the model parameters by training the model on a large number of training samples. The first sample set includes first images and first stroke vectors corresponding to multiple first training characters. The first training characters can be understood as the characters trained based on the stroke feature extraction sub-model. Since the model mostly processes images, before inputting the first training characters into the model for training, the first training characters can be converted into corresponding images, that is, the first images. Before determining the first stroke vector, a reference stroke vector can be constructed based on the character with the largest number of strokes. For example, the character with the most strokes usually has 29 strokes, and correspondingly, a 1×29-order vector can be constructed. When constructing the stroke vector of each first training character, it can be determined whether the corresponding stroke exists in the 1×29-order vector. If it exists, mark its position as 1; if it does not exist, mark it as 0.

[0059] Exemplarily, taking the determination of the first stroke vector of the character "仓" as an example, first construct a 1×29-order vector according to the character with the most stroke features in the text, and this vector contains all stroke features. The stroke features in the character "仓" include "撇", "捺", "横折钩", and "竖弯钩". Then, determine the first stroke vector corresponding to the character "仓" according to whether the corresponding stroke features exist in the pre-constructed first stroke vector. For example, the first stroke vector corresponding to the character "仓" can be obtained as {101001010……}, and this vector is 1×29-order. Among them, 1 in the vector indicates that there is a stroke feature corresponding to the character "仓" in the pre-constructed first stroke vector; 0 indicates that there is no stroke feature corresponding to the character "仓" in the pre-constructed first stroke vector.

[0060] Specifically, a plurality of to-be-trained characters are obtained as first training samples, each to-be-trained character is converted into a corresponding first image, and a vector corresponding to each character is constructed as a first stroke vector. In actual application, when the stroke feature extraction sub-model extracts stroke features of each first training sample, the first image corresponding to the first training character can be used as the input parameter, and the first stroke vector corresponding to the first training character can be used as the output parameter.

[0061] It should be noted that before using the stroke feature extraction sub-model, the model needs to be trained first. By training a large number of first training sample sets, a stroke feature extraction sub-model is obtained to accurately extract stroke features of each first training character based on the stroke feature extraction model.

[0062] S220, training the target font style fusion model.

[0063] On the basis of the above, after the stroke feature extraction sub-model is trained, the to-be-trained font style fusion model can be constructed based on the stroke feature extraction sub-model, and after the construction is completed, the to-be-trained font style fusion model is trained.

[0064] The to-be-trained font style fusion model constructed includes a font style extraction sub-model, a stroke feature extraction sub-model, an image feature extraction sub-model, and an encoding sub-model. Referring to Figure 2, the image feature extraction sub-model in block 1 is used to extract image features corresponding to the to-be-processed text. The stroke feature extraction sub-model in block 2 is used to extract stroke features of the to-be-processed text. The font style extraction sub-model (i.e., the font style extractor) can input the reference text “Jie” and the font style label corresponding to the “Jie” text, so as to extract the reference font style of the reference text. The encoding sub-model can be used to encode the extracted font style of the reference text. Then, the encoding result of the font style of the reference text and the stroke feature extraction result of the to-be-processed text are jointly input into the decoder, so as to obtain a text with a font style between the font styles of the to-be-processed text and the reference text through the decoder. In addition, the stroke order prediction sub-model is connected after the encoding sub-model, which is used to predict the stroke order of the input text. For example, any text can be input into the target font style fusion model. Taking the input text “Cang” as an example, the stroke order features corresponding to the “Cang” are “Poi”, “Na”, “Hengzhedogou”, and “Shuangan’gou”, respectively. After the “Cang” is input into the model, the stroke order features corresponding to the “Cang” can be stored in the ht vector, and the vector ht = {h1, h2, h3, and h4} can be obtained according to the stroke order. Then, the obtained stroke order vector is input into the stroke order prediction model, and the stroke order features are analyzed based on the neural network (such as a convolutional neural network). After the training of the to-be-trained style fusion model is completed, the stroke order features of each text can be predicted, so as to avoid the situation that the stroke order is missing or incorrect in the output text result.

[0065] The training of the target font style fusion model comprises: obtaining a second training sample set; wherein the second training sample set comprises a plurality of second training samples, each second training sample comprising a second training image of a second training character, a third training image of a third training character, and a font style label of the third training character; the font style of the second training character and the third training character is the same or different; for each second training sample, the current second training sample is input into the to-be-trained font style fusion model, the first training character and the third training image are processed based on the to-be-trained font style extraction sub-model to obtain a to-be-fused font style, the second training image is processed based on the to-be-trained image feature extraction sub-model to obtain a to-be-fused content feature, the second training character in the second training image is processed based on the stroke feature extraction sub-model to obtain a stroke feature, and the to-be-fused font style, the to-be-fused content feature, and the stroke feature are processed based on the to-be-trained encoding sub-model to obtain an actual output image; wherein the to-be-trained font style fusion model comprises a to-be-trained font style extraction sub-model, a to-be-trained image feature extraction sub-model, and a to-be-trained encoding sub-model; the actual output image and a corresponding theoretical output image are processed based on at least one loss function to determine a loss value, and each model parameter in the to-be-trained font style fusion model is corrected based on the loss value; convergence of the at least one loss function is taken as a training target to obtain the target font style fusion model.

[0066] It should be noted that one loss function used in the technical solution comprises a reconstruction loss function, a stroke loss function, an adversarial loss function, a style encoding loss function, and a font distinguishability function.

[0067] Next, the specific role of each loss function in the model will be introduced in detail.

[0068] The first loss function is a reconstruction loss function (Rec Loss), which is used to intuitively constrain whether the network output meets the expectation. When training based on images to be processed corresponding to characters in two different font styles, a font style between the two font styles can be obtained, and if the obtained font style does not meet the user's demand, the model parameters can be adjusted through the reconstruction loss function to make the output result of the model more consistent with the user's demand.

[0069] The second loss function is a stroke order loss function, which can be used to pre-train a self-designed recurrent neural network (RNN) that can predict stroke order information. The number of nodes in the RNN is the maximum number of strokes of Chinese characters. The features predicted by each node are combined through a connection function to form a stroke order feature matrix. Before the training of the target font style fusion model is completed, the output of the model may have incorrect or missing stroke orders. In this case, the model can be continuously adjusted based on the stroke order loss function to obtain the stroke order corresponding to each input character. By training and adjusting the model, the prediction of the stroke order of the input character can be realized to improve the accuracy of the stroke order prediction of the model.

[0070] The third loss function is an adversarial loss function (Adv Loss). The discriminator structure with an auxiliary classifier (ACGAN) can be used. The discriminator not only judges the authenticity of the generated font, but also classifies the generated font style. When the reference character is input into the font style extraction sub-model, the font style label corresponding to the reference character is also input. According to the adversarial loss function, it can be determined whether the generated font matches the input font style label. Then, according to the matching result and the adversarial loss function, the model parameters of the training font fusion model are trained to make the model output a font style that matches the font style label.

[0071] The fourth loss function is a style encoding loss function (Triplet loss), which can be used to constrain the two-norm of the font style encoding generated by different fonts to be as close to 0 as possible. That is, the style encoding loss function can obtain the two-norm between two different font styles. According to the value of the two-norm, it can be determined which font style the obtained font style is more biased towards. In order to make the fusion of different font styles have continuity, the value of the two-norm is kept as close to 0 as possible, so that the fused font style is between the two font styles and does not bias towards any of the two font styles.

[0072] The fifth loss function is a font distinguishability function (Style Regularization (SR) loss), which can be used to constrain the font style encoding generated by different fonts to have sufficient distinguishability. Based on the fourth loss function, the font distinguishability function can be used to distinguish the obtained font style encoding.

[0073] It should be noted that the above five loss functions can be used in combination or individually, and the model parameters of the font style fusion model to be processed are modified based on at least one loss function. Among them, through the mutual constraint between SR loss and Triplet loss, the style encoding distribution of different fonts is finally different but as continuous as possible. Therefore, the method can continuously control the style of the font while generating the font.

[0074] The advantage of such arrangement is that at least one loss function can better constrain the training of the font style fusion model to be trained to obtain the best target font style fusion model, and the font style conversion of the text contained in the actual output image obtained when different fonts are fused based on the target font style fusion model is more natural.

[0075] After determining each loss function, the model can be trained based on the loss function, and at this time, the second training sample set can be obtained to train the target font style fusion model based on the second training sample set.

[0076] Among them, the second training sample set includes two groups of training data. Respectively, the second training text and the second training image, and the third training image and the font style label corresponding to the third training text.

[0077] The current second training sample can be understood as a training sample that is about to be input into the font style fusion model to be trained for fusion. The actual output image can be understood as an image fused by the font style based on the trained font style fusion model to be trained. For example, the second training sample set input includes "Cang" of Kai font style and "Jie" of Song font style. Based on the second sample set, the font style fusion model to be trained can be fused to obtain the actual output image corresponding to the "Cang" character, and the font style of the output "Cang" character is between Kai font style and Song font style. Among them, Kai font style and Song font style used here are existing font styles with copyright, only for illustrative purposes, not limited to font styles with copyright. The loss function can be understood as a function for evaluating the difference between the predicted value and the true value of the model, so as to guide the next step of training to the correct direction. The better the loss function, the better the performance of the model. The theoretical output image can be understood as the text image corresponding to the first training text in a specific font based on the target font style fusion model. The loss value can be understood as the deviation value of the actual image and the theoretical image based on the loss function. The training target can be understood as the loss value based on each loss function as a condition for detecting whether the loss function converges.

[0078] Specifically, the second training image containing the second training character and the third training image containing the third training character and the font style label of the third training character in the second training sample set, the character style of the second training character and the third training character can be the same or different. First, input the second training character in the to-be-trained font style fusion model, which can contain the font features of the character, such as stroke features, and then input the third training character and the font style label of the third training character. Based on the target font style fusion model, the second training sample set is trained, the font features of the second training character are fused with the font style of the third training character, and the image corresponding to the fused character is taken as the actual output image.

[0079] Exemplarily, based on the to-be-trained font style fusion model, the "cang" of A font style and the "jue" of B font style are fused, and the "cang" of C font style is generated as the actual output image after fusion, and the "cang" with B font style is taken as the theoretical output image. Wherein, the C font style is based on the font style between A font style and B font style. Considering that the actual output image and the theoretical output image have differences before the to-be-trained font style fusion model is well trained, for example, there can be stroke missing or character output error in the actual output image, and the obtained actual output image is not ideal, therefore, the actual output image and the theoretical output image can be loss processed based on at least one loss function to determine the loss value of the actual output image.

[0080] It can be understood that when determining the loss value, it is necessary to judge whether the training error of the loss function is less than the preset error or whether the error change trend is stable, or whether the current iteration number is equal to the preset number. If the convergence condition is detected, such as the training error of the loss function is less than the preset error, or the error change trend is stable, it indicates that the to-be-trained font style fusion model is trained, at this time the iteration training can be stopped. If it is detected that the current convergence condition is not reached, the actual output image and the corresponding theoretical output image can be further obtained to continue training the model until the training error of the loss function is within the preset range. When the training error of the loss function converges, the to-be-trained font style fusion model trained can be taken as the target font style fusion model.

[0081] Specifically, loss processing is performed on the actual output image and the corresponding theoretical output image according to at least one loss function to determine the loss value corresponding to each loss function, and the loss values are summed to obtain the final loss value. The deviation between the actual output image and the corresponding theoretical output image can be determined through the obtained loss value, and then the model parameters in the to-be-trained font style fusion model are corrected based on the loss value. When at least one loss function reaches the convergence condition of the loss function, the to-be-trained font style fusion model is trained, and the target font style fusion model is obtained.

[0082] S230. Obtain the to-be-processed images corresponding to the to-be-processed text and the reference text respectively.

[0083] S240. Input the to-be-processed image into the target font style fusion model to obtain the target text of the to-be-processed text in the target font style.

[0084] In practical applications, optionally, the target font style fusion model further includes a stroke feature extraction sub-model. The step of inputting the to-be-processed image into the target font style fusion model to obtain the target text of the to-be-processed text in the target font style includes: extracting the stroke features of the to-be-processed text based on the stroke feature extraction sub-model; correspondingly, the step of processing the reference font style and the image features based on the encoding sub-model to obtain the target text of the to-be-processed text in the target font style includes: processing the reference font style, stroke features and image features based on the encoding sub-model to obtain the target text of the to-be-processed text in the target font style.

[0085] Specifically, input the to-be-processed image corresponding to the to-be-processed text "Cang" into the target font style fusion model. Based on the pre-trained stroke feature extraction sub-model, the stroke features of the to-be-processed text can be extracted. At the same time, input the to-be-processed image corresponding to the reference text "Jie" into the target font style fusion model, and extract the font style features of the reference text "Jie" based on the font style extraction sub-model in the target font style fusion model. Input the extracted font style features into the encoding sub-model to perform encoding processing on the font style, and then input the obtained result into the compiler. In the compiler, further processing is performed on the font style and the above splicing result to obtain the target text with the target font style.

[0086] S250. Extract the stroke features of the to-be-processed text based on the stroke feature extraction sub-model.

[0087] Among them, the stroke feature extraction sub-model can be understood as a model for extracting the stroke features of text, which can be a Convolutional Neural Network (CNN) or a stroke feature extractor. It is set in the target font style fusion model and is used to extract the stroke features of the text to be processed after the user inputs the text to be processed. The stroke features of the text can include the stroke content features of the text. For example, the stroke features of the character "warehouse" can include "left-falling stroke", "right-falling stroke", "horizontal fold hook", and "vertical hook".

[0088] Specifically, similar to the font style extraction sub-model, before using the stroke feature extraction sub-model, it is necessary to train the model first and adjust the model parameters to improve the accuracy of the model in extracting the stroke features of the text in the image. After determining the optimal model parameters of the model, based on the model, the stroke features of the text in the input image to be processed are extracted. Through the stroke feature extraction sub-model, the stroke features of the text to be processed can be determined, including the specific stroke features of the text to be processed.

[0089] S260. Extract the image features corresponding to the text to be processed based on the image feature extraction sub-model; among them, the image features include content features and the font style features to be processed.

[0090] Among them, the content features can be understood as the stroke features, stroke order features, and structural features of the text, etc.

[0091] Specifically, the image feature extraction sub-model is a pre-trained model with fixed model parameters. By inputting the image containing the text to be processed into this model, the stroke features, stroke order features, structural features, and font style features of the text to be processed can be determined through the image feature extraction sub-model. So as to fuse the image features of the text to be processed with the font styles of other fonts.

[0092] S270. Process the reference font style, stroke features, and image features based on the encoding sub-model to obtain the target text of the text to be processed in the target font style.

[0093] Among them, the encoding sub-model can be understood as a model for encoding the image features of text. The image features of text can be input into the encoding sub-model in the format of a sequence. Based on the encoding sub-model, the sequences are spliced to fuse the image features.

[0094] Specifically, the font style features of the reference characters and the stroke features of the to-be-processed characters are spliced in the image feature input encoding sub-model, the font style of the reference characters and the font style of the to-be-processed characters are fused together, the to-be-processed characters with the target font style are obtained, and the processed to-be-processed characters are taken as the target characters.

[0095] The technical scheme of the embodiment of the present disclosure extracts the reference font style of the reference characters based on the font style extraction sub-model, determines the features of the reference font style, and fuses the font style of the to-be-processed characters based on the reference font style to obtain the font style between the font style of the to-be-processed characters and the font style of the reference characters. The stroke features of the to-be-processed characters are extracted based on the stroke feature extraction sub-model, and the stroke features, stroke order features, and image features of the to-be-processed characters are obtained. The image features corresponding to the to-be-processed characters are extracted based on the image feature extraction sub-model, and the image features corresponding to the to-be-processed characters are fused with the font style of the reference characters. The reference font style, the stroke features, and the image features are processed based on the encoding sub-model to obtain the target characters of the to-be-processed characters in the target font style, so as to provide the characters expected by the user. The obtained target characters have the stroke features and the image features of the to-be-processed characters, and the font style features are between the font style of the to-be-processed characters and the font style of the reference characters. The problem that the font style of the target characters does not match the font style expected by the user is solved, and the effect of generating characters with the target font style is achieved.

[0096] Embodiment three

[0097] Figure 5 A structure schematic diagram of a character generation apparatus provided in the third embodiment of the present disclosure is provided, and the apparatus specifically includes: a to-be-processed image acquisition module 310 and a target character determination module 320.

[0098] The to-be-processed image acquisition module 310 is configured to acquire to-be-processed images corresponding to to-be-processed characters and reference characters respectively.

[0099] The target character determination module 320 is configured to input the to-be-processed images into a target font style fusion model to obtain target characters of the to-be-processed characters in a target font style.

[0100] The target font style is determined based on the font style fusion model fusing the reference font style of the reference characters and the to-be-processed font style of the to-be-processed characters.

[0101] The technical scheme of the embodiment of the present disclosure is that a to-be-processed image corresponding to to-be-processed text and reference text is obtained, to-be-processed text font style and reference font style are fused based on a target font style fusion model, and any font style between the to-be-processed text and the reference text is obtained. The font style can be repeatedly fused according to the user's demand until the text with the font style consistent with the user's demand is obtained. The to-be-processed image is input into the target font style fusion model, and target text of the to-be-processed text under the target font style is obtained, which meets the user's demand of converting the font style of the to-be-processed text into the target font style. The problem that the text with the font style between two font styles cannot be generated is solved. By fusing the two font styles into any target font style between the two font styles, and generating the text consistent with the target font style, the effect of generating the text corresponding to any font style between the two font styles is realized.

[0102] On the basis of the above technical scheme, optionally, the to-be-processed image acquisition module is configured to:

[0103] Based on the to-be-processed text and the reference text edited in the editing control, a to-be-processed image corresponding to the to-be-processed text and the reference text is generated.

[0104] On the basis of the above technical scheme, optionally, the target text determination module comprises:

[0105] The reference font style determination submodule is configured to extract a reference font style of the reference text based on a font style extraction submodule in the target font style fusion model;

[0106] The image feature extraction submodule is configured to extract image features corresponding to the to-be-processed text based on the image feature extraction submodule; wherein the image features comprise content features and to-be-processed font style features;

[0107] The target text determination submodule is configured to process the reference font style and the image features based on the encoding submodule, and obtain target text of the to-be-processed text under the target font style.

[0108] On the basis of the above technical scheme, optionally, the target text determination module comprises:

[0109] The stroke feature extraction submodule is configured to extract stroke features of the to-be-processed text based on the stroke feature extraction submodule;

[0110] Correspondingly, the target text determination submodule comprises:

[0111] a target character determination sub-module, configured to process the reference font style, the stroke feature, and the image feature based on the encoding sub-model to obtain a target character of the to-be-processed character under a target font style.

[0112] On the basis of the technical solution described above, optionally, the character generation apparatus further includes:

[0113] a character package generation module, configured to generate to-be-used characters of each character under the target font style based on the target font style fusion model, and generate a character package based on the to-be-used characters.

[0114] On the basis of the technical solution described above, optionally, the character package generation module is further configured to:

[0115] when it is detected that the font style selected from the font style list is the target font style and it is detected that the to-be-processed character is edited, obtaining the target character corresponding to the to-be-processed character from the character package.

[0116] On the basis of the technical solution described above, optionally, the stroke feature extraction sub-module further includes:

[0117] a stroke feature extraction sub-model determination unit, configured to train a stroke feature extraction sub-model in the target font style fusion model;

[0118] the stroke feature extraction sub-model determination unit includes:

[0119] a first training sample set acquisition sub-unit, configured to acquire a first training sample set; wherein the first training sample set includes a plurality of first training samples, and each first training sample includes a first image corresponding to a first training character and a first stroke vector;

[0120] a stroke feature extraction sub-model determination sub-unit, configured to, for each first training sample, take the first image of the current first training sample as an input parameter of a to-be-trained stroke feature extraction sub-model, take the corresponding first stroke vector as an output parameter of the to-be-trained stroke feature extraction sub-model, and train the to-be-trained stroke feature extraction sub-model to obtain the stroke feature extraction sub-model.

[0121] On the basis of the technical solution described above, optionally, the reference font style determination sub-module includes:

[0122] a target font style fusion model determination unit, configured to train the target font style fusion model;

[0123] the target font style fusion model determination unit includes:

[0124] The second training sample set acquisition subunit is configured to acquire a second training sample set. The second training sample set includes a plurality of second training samples. Each second training sample includes a second training image of a second training character, a third training image of a third training character, and a font style label of the third training character. The second training character and the third training character have the same font style or different font styles.

[0125] The actual output image determination subunit is configured to input each second training sample into a to-be-trained font style fusion model, to process the font style label of the first training character and the third training image based on a to-be-trained font style extraction submodel, to obtain a to-be-fused font style, to perform content feature extraction on the second training image based on a to-be-trained image feature extraction submodel, to obtain to-be-fused content features, to perform stroke feature extraction on the second training character in the second training image based on a stroke feature extraction submodel, to obtain stroke features, and to process the to-be-fused font style, the to-be-fused content features, and the stroke features based on a to-be-trained encoding submodel, to obtain an actual output image. The to-be-trained font style fusion model includes the to-be-trained font style extraction submodel, the to-be-trained image feature extraction submodel, and the to-be-trained encoding submodel.

[0126] The model parameter correction subunit is configured to perform loss processing on the actual output image and a corresponding theoretical output image based on at least one loss function, to determine a loss value, and to correct each model parameter in the to-be-trained font style fusion model based on the loss value.

[0127] The target font style fusion model determination subunit is configured to take convergence of the at least one loss function as a training target, to obtain the target font style fusion model.

[0128] Optionally, the at least one loss function includes a reconstruction loss function, a stroke order loss function, an adversarial loss function, a style encoding loss function, and a font distinguishability function.

[0129] The character generation apparatus provided in the embodiments of the present disclosure can perform the character generation method provided in any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of performing the method.

[0130] It should be noted that each unit and module included in the above apparatus is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be implemented; in addition, the specific name of each functional unit is only for convenient mutual distinction, and does not serve to limit the protection scope of the embodiments of the present disclosure.

[0131] Embodiment Four

[0132] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this disclosure. Refer to the following... Figure 6 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 6 The diagram below shows the structure of the terminal device or server 400. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0133] like Figure 6 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 406 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. Processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0134] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 406 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0135] In particular, according to embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 406, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the text generation method of the embodiments of the present disclosure are executed.

[0136] Embodiment Five

[0137] Embodiment Five of the present disclosure provides a computer storage medium having stored thereon a computer program, which, when executed by a processor, implements the text generation method provided by the above-mentioned embodiments.

[0138] It should be noted that the computer readable medium of the present disclosure described above can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0139] In some embodiments, the client, server, or both can communicate using any known or future developed network protocols such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., communication networks). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current or future developed networks.

[0140] The computer readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device and be not assembled into the electronic device.

[0141] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to: obtain a to-be-processed image corresponding to to-be-processed text and reference text respectively; input the to-be-processed image into a target font style fusion model to obtain target text of the to-be-processed text under a target font style; wherein the target font style is determined based on the font style fusion model fusing a reference font style of the reference text and a to-be-processed font style of the to-be-processed text.

[0142] Alternatively, the computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to: obtain a to-be-processed image corresponding to to-be-processed text and reference text respectively; input the to-be-processed image into a target font style fusion model to obtain target text of the to-be-processed text under a target font style; wherein the target font style is determined based on the font style fusion model fusing a reference font style of the reference text and a to-be-processed font style of the to-be-processed text.

[0143] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0144] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0145] The units described in the embodiments of the present disclosure can be implemented by hardware, software, or a combination thereof. In some cases, the names of the units do not constitute a limitation on the units themselves. For example, the first obtaining unit can also be described as a unit that obtains at least two Internet protocol addresses.

[0146] The functions described in this specification can be performed at least in part by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0147] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] According to one or more embodiments of the present disclosure, Example One provides a text generation method, which comprises:

[0149] obtaining a to-be-processed image corresponding to a to-be-processed text and a reference text respectively;

[0150] inputting the to-be-processed image into a target font style fusion model to obtain a target text of the to-be-processed text under a target font style;

[0151] wherein the target font style is determined based on a reference font style of the reference text and a to-be-processed font style of the to-be-processed text by the font style fusion model.

[0152] According to one or more embodiments of the present disclosure, Example Two provides an image processing method, which further comprises:

[0153] Optionally, the obtaining a to-be-processed image corresponding to a to-be-processed text and a reference text respectively comprises:

[0154] generating a to-be-processed image corresponding to the to-be-processed text and the reference text based on the to-be-processed text and the reference text edited in the editing control.

[0155] According to one or more embodiments of the present disclosure, Example Three provides a text generation method, which further comprises:

[0156] Optionally, the target font style fusion model comprises a font style extraction sub-model, an image feature extraction sub-model, and an encoding sub-model, and the inputting the to-be-processed image into the target font style fusion model to obtain the target text of the to-be-processed text under the target font style further comprises:

[0157] extract a reference font style of the reference text based on the font style extraction sub-model;

[0158] extract an image feature corresponding to the text to be processed based on the image feature extraction sub-model; wherein the image feature comprises a content feature and a text to be processed font style feature;

[0159] process the reference font style and the image feature based on the encoding sub-model to obtain a target text of the text to be processed under a target font style.

[0160] According to one or more embodiments of the present disclosure,

Example Four

[0161] Optionally, the target font style fusion model further comprises a stroke feature extraction sub-model, and the inputting the image to be processed into the target font style fusion model to obtain the target text of the text to be processed under the target font style comprises:

[0162] extract a stroke feature of the text to be processed based on the stroke feature extraction sub-model;

[0163] Correspondingly, the processing the reference font style and the image feature based on the encoding sub-model to obtain the target text of the text to be processed under the target font style comprises:

[0164] processing the reference font style, the stroke feature and the image feature based on the encoding sub-model to obtain the target text of the text to be processed under the target font style.

[0165] According to one or more embodiments of the present disclosure,

Example Five

[0166] Optionally, based on the target font style fusion model, generate a text to be used of each text under the target font style, and generate a text package based on the text to be used.

[0167] According to one or more embodiments of the present disclosure,

Example Six

[0168] Optionally, when it is detected that the font style selected from the font style list is the target font style and it is detected that the text to be processed is edited, obtain a target text corresponding to the text to be processed from the text package.

[0169] According to one or more embodiments of the present disclosure,

Example Seven

[0170] Optionally, the stroke feature extraction sub-model in the target font style fusion model is trained.

[0171] The stroke feature extraction sub-model in the target font style fusion model is trained.

[0172] A first training sample set is obtained, wherein the first training sample set includes a plurality of first training samples, and each first training sample includes a first image corresponding to a first training character and a first stroke vector.

[0173] For each first training sample, the first image of the current first training sample is used as an input parameter of a to-be-trained stroke feature extraction sub-model, and the corresponding first stroke vector is used as an output parameter of the to-be-trained stroke feature extraction sub-model, and the to-be-trained stroke feature extraction sub-model is trained to obtain the stroke feature extraction sub-model.

[0174] According to one or more embodiments of the present disclosure, Example Eight provides a character generation method, the method further includes:

[0175] Optionally, the target font style fusion model is trained.

[0176] The target font style fusion model is trained.

[0177] A second training sample set is obtained, wherein the second training sample set includes a plurality of second training samples, and each second training sample includes a second training image of a second training character, a third training image of a third training character, and a font style label of the third training character; the font styles of the second training character and the third training character are the same or different.

[0178] For each second training sample, the current second training sample is input into a to-be-trained font style fusion model, so as to process the font style label of the first training character and the third training image based on a to-be-trained font style extraction sub-model to obtain a to-be-fused font style, extract content features of the second training image based on a to-be-trained image feature extraction sub-model to obtain to-be-fused content features, extract stroke features of the second training character in the second training image based on the stroke feature extraction sub-model to obtain stroke features, and process the to-be-fused font style, the to-be-fused content features, and the stroke features based on a to-be-trained encoding sub-model to obtain an actual output image; wherein the to-be-trained font style fusion model includes a to-be-trained font style extraction sub-model, a to-be-trained image feature extraction sub-model, and a to-be-trained encoding sub-model.

[0179] determine a loss value based on the loss value, and correct each model parameter in the to-be-trained font style fusion model based on the loss value;

[0180] Converge the at least one loss function as a training target to obtain the target font style fusion model.

[0181] According to one or more embodiments of the present disclosure, Example Nine provides a character generation method, the method further comprising:

[0182] Optionally, the at least one loss function comprises a reconstruction loss function, a stroke order loss function, an adversarial loss function, a style encoding loss function, and a font distinctiveness function.

[0183] According to one or more embodiments of the present disclosure, Example Ten provides a character generation apparatus, the apparatus comprising:

[0184] A to-be-processed image acquisition module configured to acquire a to-be-processed image corresponding to a to-be-processed character and a reference character, respectively;

[0185] A target character determination module configured to input the to-be-processed image into a target font style fusion model to obtain a target character of the to-be-processed character under a target font style.

[0186] The target font style is determined based on a reference font style of the reference character and a to-be-processed font style of the to-be-processed character by the font style fusion model.

[0187] The above description is merely preferred embodiments of the present disclosure and a description of the principles of the technology applied. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

[0188] Moreover, while operations are depicted in a particular order, this should not be understood as requiring such an order nor infringing on the scope of the disclosure. Certain of the operations described in the discussion are combinable into a single operation, and certain operations can be separated into several operations. In some embodiments, the operations described in the discussion can be performed in an order different than presented in the discussion. In some embodiments, the operations described in the discussion can be performed concurrently. Also, while several specific implementation details are discussed in the discussion, these should not be interpreted as limiting the scope of the disclosure. Rather, certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0189] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A text generation method characterized by, The method comprises: obtaining a to-be-processed image corresponding to to-be-processed text and reference text respectively, wherein the to-be-processed text and the reference text have different font styles; inputting the to-be-processed image into a target font style fusion model to obtain target text of the to-be-processed text in a target font style, wherein the target font style fusion model is a model for fusing different font styles, and the to-be-processed image includes an image corresponding to the to-be-processed text and an image corresponding to the reference text; wherein the target font style is determined based on the reference font style of the reference text and the to-be-processed font style of the to-be-processed text by the font style fusion model; the target font style fusion model includes a font style extraction sub-model, an image feature extraction sub-model, and an encoding sub-model, and the inputting the to-be-processed image into the target font style fusion model to obtain the target text of the to-be-processed text in the target font style comprises: extracting the reference font style of the reference text based on the font style extraction sub-model; extracting an image feature corresponding to the to-be-processed text based on the image feature extraction sub-model, wherein the image feature includes a content feature and a to-be-processed font style feature, and the content feature includes stroke features, stroke order features, and interval structure features of the text; processing the reference font style and the image feature based on the encoding sub-model to obtain the target text of the to-be-processed text in the target font style.

2. The method of claim 1, wherein, The obtaining the to-be-processed image corresponding to the to-be-processed text and the reference text respectively comprises: generating the to-be-processed image corresponding to the to-be-processed text and the reference text based on the to-be-processed text and the reference text edited in an editing control.

3. The method of claim 1, wherein, The target font style fusion model further includes a stroke feature extraction sub-model, and the inputting the to-be-processed image into the target font style fusion model to obtain the target text of the to-be-processed text in the target font style comprises: extracting stroke features of the to-be-processed text based on the stroke feature extraction sub-model; Correspondingly, the processing the reference font style and the image feature based on the encoding sub-model to obtain the target text of the to-be-processed text in the target font style comprises: processing the reference font style, the stroke features, and the image feature based on the encoding sub-model to obtain the target text of the to-be-processed text in the target font style.

4. The method of claim 1, wherein, Further comprising: generating to-be-used text of each text in the target font style based on the target font style fusion model, and generating a text package based on the to-be-used text.

5. The method of claim 4, wherein, Further comprising: when it is detected that a font style selected from a font style list is the target font style and it is detected that the to-be-processed text is edited, obtaining target text corresponding to the to-be-processed text from the text package.

6. The method of claim 3, wherein, Further comprising: training the stroke feature extraction sub-model in the target font style fusion model; the training the stroke feature extraction sub-model in the target font style fusion model comprises: obtaining a first training sample set; wherein the first training sample set comprises a plurality of first training samples, and each first training sample comprises a first image corresponding to a first training character and a first stroke vector; For each first training sample, taking the first image of the current first training sample as an input parameter of a to-be-trained stroke feature extraction sub-model, and taking the corresponding first stroke vector as an output parameter of the to-be-trained stroke feature extraction sub-model, training the to-be-trained stroke feature extraction sub-model to obtain the stroke feature extraction sub-model.

7. The method of claim 6, wherein, Further comprising: training the target font style fusion model; training the target font style fusion model comprises: obtaining a second training sample set; wherein the second training sample set comprises a plurality of second training samples, and each second training sample comprises a second training image of a second training character, a third training image of a third training character, and a font style label of the third training character; the font styles of the second training character and the third training character are the same or different; For each second training sample, input the current second training sample into a to-be-trained font style fusion model, process the font style label of the first training character and the third training image based on a to-be-trained font style extraction sub-model to obtain a to-be-fused font style, extract content features of the second training image based on a to-be-trained image feature extraction sub-model to obtain to-be-fused content features, extract stroke features of the second training character in the second training image based on the stroke feature extraction sub-model to obtain stroke features, and process the to-be-fused font style, the to-be-fused content features, and the stroke features based on a to-be-trained encoding sub-model to obtain an actual output image; wherein the to-be-trained font style fusion model comprises a to-be-trained font style extraction sub-model, a to-be-trained image feature extraction sub-model, and a to-be-trained encoding sub-model; based on at least one loss function, the actual output image and the corresponding theoretical output image loss processing, determine the loss value, based on the loss value correct each model parameter in the to-be-trained font style fusion model; converge the at least one loss function as a training target to obtain the target font style fusion model.

8. The method of claim 7, wherein, The at least one loss function comprises a reconstruction loss function, a stroke order loss function, an adversarial loss function, a style encoding loss function, and a font distinctiveness function.

9. A character generating apparatus characterized by comprising: comprising: a to-be-processed image acquisition module configured to acquire a to-be-processed image corresponding to a to-be-processed character and a reference character, wherein the to-be-processed character and the reference character have different font styles; a target character determination module configured to input the to-be-processed image into a target font style fusion model to obtain a target character of the to-be-processed character in a target font style, wherein the target font style fusion model is a model that fuses different font styles, and the to-be-processed image comprises an image corresponding to the to-be-processed character and an image corresponding to the reference character; The target font style is determined based on a font style fusion model, a reference font style of the reference text, and a to-be-processed font style of the to-be-processed text. The target font style fusion model includes a font style extraction sub-model, a stroke feature extraction sub-model, an image feature extraction sub-model, and a coding sub-model. A reference font style determination sub-module is configured to extract a reference font style of the reference text based on the font style extraction sub-model. An image feature extraction sub-module is configured to extract image features corresponding to the to-be-processed text based on the image feature extraction sub-model. The image features include content features and to-be-processed font style features, and the content features include stroke features, stroke order features, and interval structure features.

10. An electronic device, comprising: A target text determination sub-module is configured to process the reference font style and the image features based on the coding sub-model to obtain target text of the to-be-processed text under a target font style. The electronic device includes: One or more processors; A storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the text generation method of any one of claims 1-8.

11. A storage medium containing computer-executable instructions for performing the text generation method of any one of claims 1-8 when executed by a computer processor.

Images