Training method and device of font generation model, equipment and storage medium

Abstract

The application discloses a font generation model training method and device, equipment and a storage medium, and belongs to the field of machine learning. The method comprises the following steps: obtaining font style images of n sample fonts, font content images of a first character, and sample character images of the first character under the n sample fonts; calling a font generation model to learn style features in the font style images of the n sample fonts, and learn content features in the font content images of the first character, so as to predict predicted character images of the first character under the n sample fonts; and training the font generation model according to n groups of errors between the predicted character images under the n sample fonts and the sample character images under the n sample fonts. The application can make the font generation model adapt to multiple fonts at the same time, strengthen the feature extraction capability of the font generation model on the font detail differences of different fonts, improve the quality of font generation, and improve the stability.

Description

Technical Field

[0001] This application relates to the field of machine learning, and in particular to a training method, apparatus, device, and storage medium for a font generation model. Background Technology

[0002] Personalized fonts have been applied in various scenarios. For example, in instant messaging applications, users can choose to use a personalized font and communicate with other users using the selected personalized font.

[0003] Machine learning models can generate a large number of other text images with the same font style but different characters from a small number of text images of a custom font, thus forming a font set for the custom font. In related technologies, this can be achieved using an Encoder-Mixer-Decoder (EMD). The EMD includes a font style encoder, a font content encoder, a mixer, and a decoder. The font style encoder extracts style features from a first text image, and the font content encoder extracts content features from a second text image. The style features reflect the font style of the characters in the first text image, and the content features reflect the content of the characters in the second text image. The EMD then mixes the style and content features using a mixer, and the decoder processes the mixed features to predict the generated text image. The font of the characters in the generated text image is determined by the first text image, and the content of the characters is determined by the second text image. According to the EMD, a large number of text images with the same font but different characters can be generated using a small number of first text images of the same custom font and a large number of second text images with different characters.

[0004] When using EMD to generate font images of different fonts, the font content encoder of EMD has a weak ability to process new font styles. As a result, the font content in the font image generated for a new font style cannot adapt to the new font style, leading to low and unstable font generation quality. Summary of the Invention

[0005] This application provides a training method, apparatus, device, and storage medium for a font generation model, which can improve the quality and stability of font generation. The technical solution is as follows:

[0006] According to one aspect of this application, a method for training a font generation model is provided, the method comprising:

[0007] Obtain font style images of n sample fonts, font content images of the first character, and sample text images of the first character under the n sample fonts, where n is greater than or equal to 2;

[0008] The font generation model is invoked to learn the style features in the font style images of the n sample fonts and the content features in the font content image of the first character, thereby predicting the predicted text image of the first character under the n sample fonts.

[0009] The font generation model is trained based on n sets of errors between the predicted text images under the n sample fonts and the sample text images under the n sample fonts.

[0010] Each of the n sets of errors is the error between the predicted text image and the sample text image of the first text under the same sample font.

[0011] According to another aspect of this application, a training apparatus for a font generation model is provided, the apparatus comprising:

[0012] The acquisition module is used to acquire font style images of n sample fonts, font content images of the first text, and sample text images of the first text under the n sample fonts, wherein n is greater than or equal to 2;

[0013] The prediction module is used to call the font generation model to learn the style features in the font style images of the n sample fonts and to learn the content features in the font content image of the first character, so as to predict the predicted text image of the first character under the n sample fonts.

[0014] The training module is used to train the font generation model based on n sets of errors between the predicted text images under the n sample fonts and the sample text images under the n sample fonts.

[0015] Each of the n sets of errors is the error between the predicted text image and the sample text image of the first text under the same sample font.

[0016] In an optional design, the prediction module is used for:

[0017] The font generation model is invoked to extract features from the font style images of the n sample fonts, resulting in n style feature vectors corresponding to the n sample fonts. The style feature vectors are used to reflect the font style of the sample fonts. The font generation model is also invoked to extract features from the font content image of the first character, resulting in a content feature vector. The content feature vectors are used to reflect the content of the text in the font content image of the first character.

[0018] The font generation model is invoked to predict the predicted text image of the first character under the n sample fonts, based on the n style feature vectors and the content feature vector of the first character.

[0019] In an optional design, the font generation model includes a multi-style encoder; the prediction module is used for:

[0020] The multi-style encoder is invoked to extract features from the font style images of the n sample fonts, thereby obtaining the n style feature vectors;

[0021] The multi-style encoder is invoked to extract features from the font content image of the first text, thereby obtaining the content feature vector.

[0022] In an optional design, the multi-style encoder includes n style encoders; the prediction module is used for:

[0023] The n style encoders are invoked to extract features from one of the font style images of the n sample fonts, respectively, to obtain the n style feature vectors.

[0024] In an optional design, the multi-style encoder includes a content encoder; the prediction module is used for:

[0025] The content encoder is invoked to extract features from the font content image of the first text, thereby obtaining the content feature vector.

[0026] In an optional design, the font generation model includes a mixer; the prediction module is used for:

[0027] The mixer is invoked to fuse the n style feature vectors with the content feature vectors respectively, resulting in n fused feature vectors;

[0028] The font generation model is invoked to predict the predicted text images for the n sample fonts based on the n fused feature vectors.

[0029] In an optional design, the font generation model further includes a decoder; the prediction module is used for:

[0030] The decoder is invoked to decode the n fused feature vectors respectively, so as to obtain the predicted text images under the n sample fonts.

[0031] In an optional design, the training module is used for:

[0032] Based on the predicted text images under the n sample fonts and the images of the sample text images under the n sample fonts, the images of the same sample font dimension, and the minimum absolute value deviation function, n sets of correction losses are determined;

[0033] Based on the product of the correction loss, the first parameter, and the second parameter in each group, determine n groups of target correction losses;

[0034] The font generation model is trained based on the n sets of target correction losses;

[0035] The first parameter is negatively correlated with the number of black pixels in the sample text image, and the second parameter is determined based on the normalized exponential function and the average pixel value of the black pixels in the sample text image, wherein the black pixels are pixels with pixel values ​​below a threshold.

[0036] In an optional design, the font generation model further includes a discriminator; the training module is used for;

[0037] The predicted text images under the n sample fonts, the images of the font style images of the n sample fonts at the same sample font dimension, and the font content image of the first text are respectively input into the discriminator to obtain n first prediction labels. The first prediction labels are used to reflect whether the predicted text image is a real image. The sample text images under the n sample fonts, the images of the font style images of the n sample fonts at the same sample font dimension, and the font content image of the first text are respectively input into the discriminator to obtain n second prediction labels. The second prediction labels are used to reflect whether the sample text image is a real image.

[0038] Based on the n first labels and the n second labels, the labels corresponding to the same sample font dimension, and the loss function of the generative adversarial network, n sets of adversarial losses are determined;

[0039] The font generation model is trained based on the n sets of target correction losses and the n sets of adversarial losses corresponding to the same sample font dimension.

[0040] In an optional design, the training module is used for:

[0041] Based on the sum of the losses corresponding to the same sample font dimension in the n sets of weighted target correction losses and n sets of weighted adversarial losses, n sets of target losses are determined. The n sets of weighted target correction losses are determined by the product of each set of target correction losses and the first weight. The n sets of weighted adversarial losses are determined by the product of each set of adversarial losses and the second weight.

[0042] The font generation model is trained based on the n sets of target losses.

[0043] In an optional design, the acquisition module is used to acquire the font style image of the target font and the font content image of the second text;

[0044] The prediction module is used to input n identical font style images of the target font and the font content image of the second character into the font generation model to obtain the generated text image of the second character under the target font predicted by the font generation model.

[0045] In an optional design, the prediction module is used for:

[0046] Input n identical font style images of the target font into the font generation model to obtain n target style feature vectors, which are used to reflect the font style of the target font; input the font content image of the second text into the font generation model to obtain target content feature vectors, which are used to reflect the content of the text in the font content image of the second text;

[0047] The font generation model is invoked to determine the generated text image based on the n target style feature vectors and the target content feature vectors.

[0048] In an optional design, the prediction module is used for:

[0049] Determine the average value of the n target style feature vectors;

[0050] The font generation model is invoked, and the generated text image is determined based on the average value of the n target style feature vectors and the target content feature vector.

[0051] According to another aspect of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the training method of the font generation model as described above.

[0052] According to another aspect of this application, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the training method of the font generation model as described above.

[0053] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a training method for a font generation model provided in various alternative implementations of the above aspects.

[0054] The beneficial effects of the technical solution provided in this application include at least the following:

[0055] By learning the stylistic features of various sample fonts and the content features of the text, the model can predict text images. This enables the font generation model to learn the stylistic features of different fonts, effectively identify and analyze the differences between them, and adapt the font content in the generated text images to different font styles. This application enhances the font generation model's ability to extract features related to the subtle differences between different fonts, allowing it to process text images of various fonts, improve the quality and stability of font generation, and exhibit high robustness when generating text images of novel fonts. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of the training process of a font generation model provided in an exemplary embodiment of this application;

[0058] Figure 2 This is a flowchart illustrating a training method for a font generation model provided in an exemplary embodiment of this application;

[0059] Figure 3 This is a flowchart illustrating a training method for a font generation model provided in an exemplary embodiment of this application;

[0060] Figure 4 This is a schematic diagram illustrating the generation of text images provided in an exemplary embodiment of this application;

[0061] Figure 5 This is a schematic diagram of the structure of a training device for a font generation model provided in an exemplary embodiment of this application;

[0062] Figure 6 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application.

[0063] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0065] Figure 1 This is a schematic diagram illustrating the training process of a font generation model provided in an exemplary embodiment of this application. For example... Figure 1 As shown, the font generation model includes a multi-style encoder 101, a mixer 102, a decoder 103, and a discriminator 104. The multi-style encoder 101, mixer 102, decoder 103, and discriminator 104 constitute a Generative Adversarial Network (GAN). Specifically, the multi-style encoder 101, mixer 102, and decoder 103 form the generator in the GAN. The multi-style encoder 101 includes at least two style encoders and a content encoder 1014, for example, it includes three style encoders: a first style encoder 1011, a second style encoder 1012, and a third style encoder 1013. The dashed line connecting the content encoder 1014 and the decoder 103 represents a skip connection.

[0066] During the training phase, the computer device acquires font style images 105 of n sample fonts, font content images 106 of the first character, and sample text images 107 of the first character under the n sample fonts. Here, n is greater than or equal to 2. For example, if n is 3, then font style image 105 includes the first font style image 1051 of the first sample font, the second font style image 1052 of the second sample font, and the third font style image 1053 of the third sample font, collectively referred to as font style image 105. Sample text images 107 include images of the first character under the first sample font, the second sample font, and the third sample font, respectively.

[0067] The computer device extracts features from the font style images corresponding to one of the n sample fonts using various style encoders, obtaining n style feature vectors corresponding to the n sample fonts. For example, the style feature vector of the first font style image 1051 is extracted by the first style encoder 1011, the style feature vector of the second font style image 1052 is extracted by the second style encoder 1012, and the style feature vector of the third font style image 1053 is extracted by the third style encoder 1013. The computer device then extracts features from the font content image 106 using the content encoder 1014, obtaining a content feature vector. The style feature vector reflects the font style of the sample font, and the content feature vector reflects the content of the text in the font content image 106 of the first character. The computer device then fuses each style feature vector with the content feature vector using a mixer 102, obtaining n fused feature vectors 108. The decoder 103 decodes each of the n fused feature vectors 108 to obtain the predicted text image 109 of the first character under the n sample fonts.

[0068] The computer device determines the correction loss based on the images of the same sample font dimension in the predicted text image 109 and sample text images 107 under n sample fonts, and the Least Absolute Deviations (LAD, also known as L1) function. It determines the adversarial loss based on the first predicted label of the predicted text image 109 and the second predicted label of the sample text image 107, corresponding to the same sample font dimension, and the loss function of the generative adversarial network. The font generation model is then trained based on the sum of the correction loss and the adversarial loss. The first predicted label reflects whether the predicted text image 109 is a real image, and the second predicted label reflects whether the sample text image 107 is a real image.

[0069] In the application phase, the computer device acquires the font style image of the target font and the font content image of the second character. Then, n identical font style images of the target font are input into each style encoder to obtain n target style feature vectors. The font content image of the second character is input into the content encoder 1014 to obtain the target content feature vector. The computer device then determines the average value of each target style feature vector and fuses the average value and the target content feature vector through a mixer 102. The fused result is input into the decoder 103 to obtain the generated text image, which is the text image of the second character predicted by the font generation model under the target font.

[0070] During the training of the font generation model, multiple style encoders are used to extract style features from text images of various fonts. These different style encoders learn the style features of different fonts, enabling effective identification and analysis of the differences between them. This allows the content encoder to adapt to multiple fonts simultaneously, and content encoding can then process text images of different fonts and better extract their content features. This improves the quality and stability of font generation, demonstrating high robustness when generating text images of novel fonts.

[0071] Figure 2 This is a flowchart illustrating a training method for a font generation model provided in an exemplary embodiment of this application. This method can be used with a computer device. Figure 2 As shown, the method includes:

[0072] Step 202: Obtain the font style images of n sample fonts, the font content image of the first character, and the sample text image of the first character under the n sample fonts.

[0073] Where n is an integer greater than or equal to 2, and n is determined by the computer device, for example, the computer device sets n based on the structure of the font generation model. The sample font is a manually set font.

[0074] Optionally, the font style images include one or more text images for each of the n sample fonts. The number of text images for each sample font may be the same or different, the text in each text image may be the same or different, and the text in each font style image may be the same or different.

[0075] The first character can be any character, and the font content image is an image containing the first character. The font content image may consist of one or more character images. The font of the characters in the font content image may or may not belong to any of the n sample fonts. The number of font content images may be the same as or different from the number of font style images for each of the n sample fonts.

[0076] The sample text image includes one or more images of each of the n sample fonts, and the text in the sample text image is the first text.

[0077] For example, the font style images include r images corresponding to each of the n sample fonts, where r is a positive integer. For instance, n is 3 and r is 10. The sample fonts include a first sample font, a second sample font, and a third sample font. The font style images include 10 images of the first sample font, 10 images of the second sample font, and 10 images of the third sample font. The font content images of the first text also include r images, for example, 10 images. The sample text images include n images, for example, images of the first text under the first sample font, the first text under the second sample font, and the first text under the third sample font. Furthermore, the text in the font style images is different from the text in the font content images; the text in the font style images is completely different for each character. The font of the text in the font content images is not any of the n sample fonts.

[0078] Step 204: Call the font generation model to learn the style features in the font style images of n sample fonts, and learn the content features in the font content image of the first character, so as to predict the predicted text image of the first character under n sample fonts.

[0079] This font generation model predicts the text image of a target character in a target font, which is determined by a computer device. Optionally, the font generation model is constructed using a Convolutional Neural Network (CNN). Style features reflect the font style of the sample font, while content features reflect the content of the text in the font content image of the first character. For example, style features include the texture, skeleton, and thickness of the text. Content features include the strokes and structure of the text; content features are features that can indicate which specific character the text in the font content image belongs to.

[0080] Optionally, the computer device invokes a font generation model to extract features from the font style images of n sample fonts, obtaining a style feature vector corresponding to each of the n sample fonts, thus obtaining n style feature vectors. The computer device also extracts features from the font content image of the first character, thus obtaining a content feature vector corresponding to the first character. Then, based on the n style feature vectors and the content feature vector of the first character, the font generation model can predict the predicted text image of the first character under the n sample fonts.

[0081] Step 206: Train the font generation model based on the n sets of errors between the predicted text images under n sample fonts and the sample text images under n sample fonts.

[0082] In this model, each of the n error groups represents the error between the predicted text image and the sample text image for the first character under the same sample font. The computer determines the n correction losses based on the images of the predicted and sample text images under the n sample fonts, along with the minimum absolute value deviation function, for the same sample font dimension. The font generation model is then trained using these correction losses.

[0083] Optionally, the aforementioned computer equipment is a server, which may be a single server, a server cluster consisting of several servers, or a virtual server in a cloud computing service center, etc.

[0084] In summary, the method provided in this embodiment predicts text images by learning the style features of multiple sample fonts and the content features of the text. This enables the font generation model to learn the style features of different fonts, effectively identify and analyze the differences between different fonts, and adapt the font content in the text images generated by the font generation model to different font styles. This application enhances the font generation model's ability to extract features related to the subtle differences between different fonts, enabling the font generation model to process text images of different fonts, improve the quality and stability of font generation, and exhibit high robustness when generating text images of novel fonts.

[0085] Figure 3 This is a flowchart illustrating a training method for a font generation model provided in an exemplary embodiment of this application. This method can be used with a computer device. Figure 3 As shown, the method includes:

[0086] Step 302: Obtain the font style images of n sample fonts, the font content image of the first character, and the sample text image of the first character under the n sample fonts.

[0087] Where n is greater than or equal to 2. The sample fonts are manually set fonts. Optionally, the font style image includes one or more text images for each of the n sample fonts. The first character is any character, and the font content image is an image containing the first character. The sample text image includes one or more images for each of the n sample fonts, where the text in the sample text image is the first character.

[0088] For example, when acquiring the above images, the computer device first acquires a sample set, which includes a large number of text images with different fonts and characters.

[0089] The computer device will select k images. (k=1, 2, ..., k) are used as sample text images, wherein c represents text content, that is, the first character. represents k different fonts, k=n, represents n sample fonts. For example, the sample text images include text images (3 images) of the character "Yong" under a first sample font, a second sample font and a third sample font respectively.

[0090] The font content images are representing r text images with content being the first character but different fonts, (i=1, 2, ..., r) represents r different fonts, and these fonts are randomly selected from a sample set after removing the n sample fonts in , that is for example, the font content images include text images (10 images) of the character "Yong" under 10 different fonts respectively, and the 10 different fonts do not include the first sample font, the second sample font and the third sample font.

[0091] In font style images, each sample font corresponds to one image set. Moreover, all fonts of the font style images in the image set corresponding to a certain sample font are the sample font. Therefore, the font style images include k image sets. The font style images in the k-th image set are representing r images with the same font (all are ) but different text contents. Wherein, (i=1, 2, ..., r) represents the r-th font style image in the image set, represents the character in the r-th font style image. represents that the characters in the font style images do not include the first character. For example, the font style images include text images of 10 different characters under the first sample font, text images of 10 different characters under the second sample font, and text images of 10 different characters under the first sample font (3 image sets, 10 images in each image set). And the 10 different characters do not include the character "Yong".

[0092] Optionally, in consideration of the efficiency of font generation model training and the learning effect, the above k can be set to 3, and the above r can be set to 10. The resolution of all images is 80×80. Then at this time, the sample text images include 3 text images of the first character under 3 sample fonts. The font content images include 10 text images of the first character. The font style images include one image set corresponding to each of the 3 sample fonts, and each image set includes 10 text images all in the corresponding sample font.

[0093] Step 304: invoking a font generation model, performing feature extraction on font style images of n sample fonts to obtain n style feature vectors corresponding to the n sample fonts, and performing feature extraction on the font content image of the first character to obtain a content feature vector.

[0094] The font generation model is used to predict the text image of a target text in a target font, which is determined by a computer device. There is a one-to-one correspondence between n style feature vectors and n sample fonts; that is, each style feature vector has a unique corresponding sample font.

[0095] The computer device invokes a font generation model to extract features from the font style image of each sample font, thereby obtaining a style feature vector corresponding to each sample font, resulting in n style feature vectors. This style feature vector reflects the font style of the sample font, while the content feature vector reflects the content of the text in the font content image of the first character.

[0096] Optionally, the font generation model includes a multi-style encoder. The computer device invokes the multi-style encoder to extract features from the font style images of n sample fonts, obtaining n style feature vectors. The multi-style encoder is then invoked to extract features from the font content image of the first character, obtaining a content feature vector.

[0097] The aforementioned multi-style encoder can include n style encoders, for example, the style encoder is... (k = 1, 2, ..., k). A computer device calls n style encoders to extract features from one of the font style images of n sample fonts, resulting in n style feature vectors, for example... in, Optionally, all n style encoders have the same structure.

[0098] For example, Table 1 shows the specific structure of the style encoder.

[0099] Table 1

[0100]

[0101] As shown in Table 1, each of the n style encoders consists of 8 CNN layers. The kernels and stride of each CNN layer are shown in Table 1. For example, the 5×5×64 kernel in the first layer represents 64 kernels of size 5×5. The stride represents the step size the kernel moves during each convolution. Each CNN layer includes an additional layer: a batch normalization (BN) layer, with the activation function being the Leaky Rectified Linear Unit (Leaky ReLU) function, where 0.2 represents the coefficient of the Leaky ReLU function. This additional layer structure accelerates the convergence speed of the font generation model during training.

[0102] For example, the font style image includes an image set corresponding to each of the three sample fonts, and each image set includes 10 text images of the corresponding sample font. When the computer device inputs the font style image into the font generation model, it inputs the image sets corresponding to the three sample fonts into different style encoders. That is, for each style encoder, the input is 80×80×10 (80×80 represents the resolution of the font style image, and 10 represents the number of input font style images). By extracting features from the input font style image using the style encoders shown in Table 1, a 1×1×512-dimensional style feature vector can be obtained.

[0103] The aforementioned multi-style encoder can also include a content encoder, for example, a content encoder as shown above. A computer device invokes the content encoder to extract features from the font content image of the first text, obtaining a content feature vector, for example, as shown above. Wherein, ...

[0104] For example, Table 2 shows the specific structure of the content encoder.

[0105] Table 2

[0106]

[0107] As shown in Table 2, the content encoder consists of 8 CNN layers. The kernel and stride of each CNN layer are shown in Table 2. Each CNN layer includes a batch normalization (BN) layer, and the activation function is the Leaky ReLU function, where 0.2 represents the coefficient of the Leaky ReLU function. The structure of this content encoder is the same as that of the style encoder.

[0108] For example, the font content image includes a set of images corresponding to the first character, which consists of 10 text images of the first character. The computer device input to the content encoder is 80×80×10 (80×80 represents the resolution of the font content image, the same as the font style image, and 10 represents the number of input font content images). By performing feature extraction on the input font content image using the content encoder shown in Table 2, a 1×1×512 dimensional content feature vector can be obtained.

[0109] Step 306: Call the font generation model to predict the predicted text image of the first character in the n sample fonts based on the n style feature vectors and the content feature vector of the first character.

[0110] The predicted text image of the first character in n sample fonts includes the text image of the first character in each of the n sample fonts. For example, the predicted text image of the first character in the first sample font, the predicted text image of the first character in the second sample font, and the predicted text image of the first character in the third sample font.

[0111] Optionally, the font generation model also includes a mixer network. The computer device invokes the mixer to fuse the n style feature vectors with the content feature vectors, resulting in n fused feature vectors. Since the style feature vectors are uncorrelated, the computer device processes these style feature vectors in parallel, fusing each style feature vector with the content feature vector separately. The computer device then invokes the font generation model, and based on these n fused feature vectors, can predict the predicted text image of the first character in the n sample fonts.

[0112] For example, computer devices will The features are fused separately to obtain the fused feature vector. The process of fusion using a mixer can be represented as follows: * indicates cross product, and W is a tensor. The dimension of the sum is 1*F. It is a 1*M dimensional feature vector. W is an F*M*F dimensional tensor. For example, when the style encoder has the structure in Table 1 and the content encoder has the structure in Table 2, M = F = 512.

[0113] Optionally, the font generation model also includes a decoder network. The computer device calls the decoder to decode the n fused feature vectors respectively, thus obtaining predicted text images for n sample fonts. For example, the predicted text image is... k = 1, 2, ..., k.

[0114] For example, Table 3 shows the specific structure of the decoder.

[0115] Table 3

[0116]

[0117] As shown in Table 3, the decoder consists of 8 CNN layers. The convolutional kernels and strides of each CNN layer are listed in Table 3. Each CNN layer includes a batch normalization (BN) layer, and the activation function is the Leaky ReLU function, where 0.2 represents the coefficients of the Leaky ReLU function. The decoder structure corresponds to the decoding structure of the style encoder and content encoder.

[0118] For example, after extracting n 1×1×512-dimensional style feature vectors and 1×1×512-dimensional content feature vectors, the computer device fuses each style feature vector and content feature vector using a mixer to obtain n 1×1×512-dimensional fused vectors. These n 1×1×512-dimensional fused vectors are then input into the decoder shown in Table 3, resulting in n predicted text images (one image output at a time). The resolution of each predicted text image is the same as that of the font style image and the font content image, for example, 80×80.

[0119] Step 308: Train the font generation model based on the n sets of errors between the predicted text images under n sample fonts and the sample text images under n sample fonts.

[0120] In this model, each of the n error sets represents the error between the predicted text image and the sample text image for the first character under the same sample font. Optionally, the computer device determines the n sets of correction losses based on the images of the predicted text images and the sample text images under the n sample fonts, along with the images along the same sample font dimension, and the Least Absolute Deviations (LAD) function, also known as the L1 function. The font generation model can then be trained using these n sets of correction losses.

[0121] Using the L1 loss function to determine the correction loss for training a font generation model will cause the model to focus more on the thicker, darker parts of the font in the image, while ignoring the thinner, brighter parts. To reduce the impact of this problem on the model's generation results, the computer can also determine n sets of target correction losses based on the product of each set of correction losses, the first parameter, and the second parameter. The font generation model is then trained based on these n sets of target correction losses. The first parameter is negatively correlated with the number of black pixels in the sample text image, and the second parameter is determined based on the normalization exponent (softmax) function and the average pixel value of the black pixels in the sample text image. These black pixels are those with pixel values ​​below a threshold set by the computer, for example, 127. When determining the target correction loss, the first and second parameters used for the correction loss are influenced by the sample text image corresponding to the same sample font as the correction loss. The sample font corresponding to the correction loss is the sample font in both the sample text image used to determine the correction loss and the sample text image used for prediction.

[0122] For example, the formula for calculating the target correction loss is as follows:

[0123]

[0124] in, As the first parameter, Represents sample text images The number of black pixels in the middle. For the second parameter, Sample text image The average pixel value of the black pixels in the image. Indicates based on sample text images and sample text images Predicted text images in the same sample font dimension Between these, the corrected loss is determined based on the L1 loss function.

[0125] The formula for calculating the softmax function is as follows:

[0126]

[0127] Here, t represents the variable of the function.

[0128] To further improve the accuracy of the font generation model, computer devices can also train the font generation model using a generative adversarial network (GAN) structure. Optionally, the font generation model also includes a discriminator. This discriminator, along with the aforementioned multi-style encoder, mixer, and decoder, forms the GAN. The multi-style encoder, mixer, and decoder together constitute the generator of the GAN.

[0129] The computer device inputs the predicted text images of n sample fonts, the font style images of n sample fonts (images along the same font dimension), and the font content image of the first character into a discriminator to obtain n first predicted labels. The computer device then inputs the sample text images of n sample fonts, the font style images of n sample fonts (images along the same font dimension), and the font content image of the first character into the discriminator to obtain n second predicted labels.

[0130] The first prediction label reflects whether the predicted text image is a real image, and the second prediction label reflects whether the sample text image is a real image. If it is a real image, it means that the image being judged is predicted to be an unprocessed image; if it is not a real image, it means that the image being judged is predicted to be an image generated by the generator.

[0131] For example, Table 4 shows the specific structure of the discriminator.

[0132] Table 4

[0133]

[0134] As shown in Table 4, the discriminator consists of four CNN layers. The convolutional kernels and strides of each CNN layer are shown in Table 4. The additional layers of the first, second, and third CNN layers include Batch Normalization (BN) layers, and the activation function is the Leaky ReLU function, where 0.2 represents the coefficient of the Leaky ReLU function.

[0135] For example, after determining the predicted text images, the computer device inputs each predicted text image, a font style image of the same sample font corresponding to the predicted text image, and a font content image into a discriminator. The discriminator then determines whether the predicted text images match the font style images and the font content images. For example, it can determine a 10×10 dimensional feature vector. The authenticity of the predicted text image can be determined based on this feature vector. The calculation process is as follows:

[0136]

[0137] Where Dis represents the discriminator.

[0138] The computer device also inputs each sample text image, a font style image of the same sample font as the sample text image, and a font content image into the discriminator. The discriminator will determine whether the sample text image matches the font style image and the font content image, for example, it can determine a 10×10 dimensional feature vector. The authenticity of a sample text image can be determined based on this feature vector. The calculation process is as follows:

[0139]

[0140] Computer equipment according to The mean of the values ​​can be used to determine whether the predicted text image is real or fake. Computer equipment uses... The mean of the values ​​can be used to determine the authenticity of sample text images.

[0141] The computer device determines n sets of adversarial losses based on the labels corresponding to the same sample font dimension from n first labels and n second labels, as well as the loss function of the generative adversarial network. Then, the font generation model is trained using the n sets of target correction losses and the n sets of adversarial losses corresponding to the same sample font dimension.

[0142] For example, the formula for calculating adversarial loss is as follows:

[0143] L adv =E x，y [logDis(x, y)] + E x，y [lo g (1-Dis(x, G(x)))];

[0144] Where y represents the sample text image, x is the input to the font generation model, specifically including the font style image and font content image of the same sample font as the sample text image. G represents the generator, and G(x) is the predicted text image, x is the input to the font generation model, specifically including the font style image and font content image of the same sample font as the predicted text image. Dis represents the discriminator. E represents the expectation.

[0145] When training the font generation model using target correction loss and adversarial loss, the computer device can also determine n sets of target losses based on the sum of the losses corresponding to the same sample font dimension from the n sets of weighted target correction losses and n sets of weighted adversarial losses, and train the font generation model based on the n sets of target losses. Specifically, the n sets of weighted target correction losses are determined by multiplying each target correction loss by a first weight, and the n sets of weighted adversarial losses are determined by multiplying each adversarial loss by a second weight.

[0146] For example, the training objective of the font generation model is as follows:

[0147]

[0148] in, Represents a set of target losses. As the first weight, w adv The second weight is used to adjust the target correction loss and adversarial loss, while the first and second weights are used to adjust these. The training objective of the generative adversarial network (GAN) is to maximize the target loss of the discriminator and minimize the target loss of the generator. During training, the computer alternately trains the generator and the discriminator. Introducing the discriminator allows the generator to generate text images that the discriminator cannot distinguish as real images, thereby improving the accuracy of the generator's output. Optionally, during training, the second weight is initially set to 1, and then the first weight is adjusted to achieve the optimal effect. For example, the first weight can ultimately be set to 420.

[0149] Step 310: Obtain the font style image of the target font and the font content image of the second text.

[0150] The target font is a manually set font. The target font belongs to one of the n sample fonts, or it is a font other than one of the n sample fonts. The font used for the text in the font style image is the target font.

[0151] The font content image of the second character is a text image containing the second character, which may be the same as or different from the first character. The font of the text in the font content image of the second character may be the same as or different from the target font, and the text in the font style image of the second character and the target font may be the same as or different.

[0152] For example, the target font's style image includes r text images, and the second text's content image includes r text images, where r is 10. When validating the font generation model's output, the target font belongs to n sample fonts, and the second text can be the first text or any other text. When using the font generation model to generate a new font, if the target font does not belong to n sample fonts, or if the target font belongs to n sample fonts (in the case where the font generation model is trained using a small number of new font text images), the second text is different from the first text.

[0153] Step 312: Input n identical target font style images and second text font content images into the font generation model to obtain the generated text image of the second text predicted by the font generation model under the target font.

[0154] The generated text image is a text image where the text predicted by the font generation model is the second text and the font is the target font. Optionally, the computer device copies the font style image of the target font into n copies and inputs these n identical font style images into the trained font generation model to obtain n target style feature vectors. The computer device also inputs the font content image of the second text into the trained font generation model to obtain target content feature vectors. The target style feature vectors reflect the font style of the target font, and the target content feature vectors reflect the content of the text in the font content image of the second text. Then, the computer device calls the trained font generation model and, based on the n target style feature vectors and the target content feature vectors, determines the text image to be generated.

[0155] Optionally, the computer device inputs n identical target font style images into different style encoders in the font generation model to obtain n target style feature vectors. The font content image of the second character is then input into the content encoder of the font generation model to obtain target content feature vectors. The computer device then determines the average of the n target style feature vectors and calls the font generation model to determine the generated text image based on the average of the n target style feature vectors and the target content feature vectors. For example, the average of the n target style feature vectors and the target content feature vectors can be mixed using the font generation model's mixer, and then the font generation model's decoder can predict the mixed result to obtain the generated text image.

[0156] For example, Figure 4 This is a schematic diagram illustrating the generation of text images according to an exemplary embodiment of this application. For example... Figure 4 As shown, the computer device acquires 10 font style images 401 of the target font and 10 font content images 402 of the second character. Then, the font style images 401 are copied n times and input together with the font content images 402 into the font generation model 403, thereby obtaining the generated text image of the second character under the target font predicted by the font generation model 403.

[0157] In practical applications, evaluation metrics can be used to analyze the accuracy of the font generation model's output. Evaluation metrics include L1 (LAD), Structural Similarity (SSIM), and Frechet Inception Distance (FID).

[0158] Where L1 represents the sum of the absolute distances between each pixel in the real text image (sample text image) and the predicted text image (predicted text image), used to evaluate the accuracy of the output. The calculation formula is as follows:

[0159] L1 = |f(x) - Y|;

[0160] Where f(x) represents the predicted text image and Y represents the actual text image.

[0161] SSIM evaluates the accuracy of its output by calculating the similarity between real and predicted text images based on brightness, contrast, and structure. The calculation formula is as follows:

[0162] SSIM(I t I g )=[1(I t I g )] α [c(I t I g )] β [s(I t I g )] γ ;

[0163] Among them, I t Representing real text images, I g This represents the predicted text image. 1 represents the calculation for brightness, c represents the calculation for contrast, and s represents the calculation for structure. α, β, and γ are constants representing the importance of brightness, contrast, and structure, respectively.

[0164] The formula for calculating FID is as follows:

[0165] FID = ||μ r -μ g || 2 +T r (∑ r +∑ g -2(∑ r ∑ g ) 1 / 2 );

[0166] Where, μ r μ represents the mean pixel value of a real text image. g ∑ represents the mean of the pixel values ​​in the predicted text image. r The covariance of the actual text image, ∑ g This represents the covariance of the predicted text image.

[0167] A higher SSIM indicates higher accuracy in the generated result, while lower L1 and FID values ​​also indicate higher accuracy. By combining these evaluation metrics with sample text images and predicted text images, it can be determined that the text generation model provided in this application embodiment has highly accurate generation results.

[0168] It should be noted that steps 302 to 308 above can be implemented independently, serving as the training method for the font generation model on the model training side. Steps 310 and 312 above can be implemented independently, serving as the feature extraction method on the model application side.

[0169] In summary, the method provided in this embodiment predicts text images by learning the style features of multiple sample fonts and the content features of the text. This enables the font generation model to learn the style features of different fonts, effectively identify and analyze the differences between different fonts, and adapt the font content in the text images generated by the font generation model to different font styles. This application enhances the font generation model's ability to extract features related to the subtle differences between different fonts, enabling the font generation model to process text images of different fonts, improve the quality and stability of font generation, and exhibit high robustness when generating text images of novel fonts.

[0170] The method provided in this embodiment also extracts style feature vectors and content feature vectors separately, and determines the predicted text image based on the style feature vectors and content feature vectors. This enables the simultaneous and accurate learning of style features and content features, thereby improving the accuracy of the model.

[0171] The method provided in this embodiment also uses a multi-style encoder to extract style feature vectors and content feature vectors of multiple sample fonts, providing a way to extract style feature vectors and content feature vectors of multiple sample fonts.

[0172] The method provided in this embodiment also uses multiple style encoders to extract style feature vectors of different sample fonts, enabling the font generation model to learn the style features of different font styles, thereby effectively identifying and analyzing the differences between different fonts and improving the accuracy of the model.

[0173] The method provided in this embodiment also uses a content encoder to extract content feature vectors. Combined with a style encoder, the content encoder can adapt to multiple fonts at the same time. The content encoding can then process text images of different fonts and better extract the content features of different fonts, thereby improving the accuracy of the model.

[0174] The method provided in this embodiment also uses a mixer to fuse different style feature vectors and content feature vectors respectively, thereby achieving the prediction of the output result based on the fused feature vector.

[0175] The method provided in this embodiment also decodes the fused feature vector output by the mixer through a decoder, thereby predicting the output result.

[0176] The method provided in this embodiment also trains the font generation model by using a target correction loss jointly determined by the correction loss, the first parameter, and the second parameter. This avoids the problem that the model focuses more on the thicker and darker parts of the font in the image and ignores the thinner and brighter parts of the font in the image during the training process.

[0177] The method provided in this embodiment also trains the font generation model together with target correction loss and adversarial loss, and trains the machine learning model by means of generative adversarial network, which can effectively improve the accuracy of the machine learning model.

[0178] The method provided in this embodiment also adjusts the target correction loss and adversarial loss through the first weight and the second weight, which enables the trained model to output more accurate results.

[0179] The method provided in this embodiment also obtains the font style image of the target font and the font content image of the second character, and uses a font generation model to predict the generated text image of the second character under the target font, thereby achieving the prediction of the text image of the character under the new font.

[0180] The method provided in this embodiment also extracts the target style feature vector and the target content feature vector respectively, and determines the generated text image based on the target style feature vector and the target content feature vector. This enables the simultaneous and accurate prediction of style features and content features, thereby improving the accuracy of the predicted generated text image.

[0181] The method provided in this embodiment also determines the generated text image by determining the average value of the target style feature vector and the target content feature vector, providing a way to determine the generated text image based on multiple target style feature vectors and content feature vectors.

[0182] It should be noted that the order of the method steps provided in the embodiments of this application can be appropriately adjusted, and the steps can also be added or removed as appropriate. Any method variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0183] In a specific example, the structure of the font generation model is as follows: Figure 1As shown, during the training phase of the font generation model, the computer device acquires font style images, font content images, and sample text images for three existing fonts. For each existing font, there are 10 font style images, 10 font content images, and 3 sample text images. The text in the font content images differs from that in the font style images but is identical to that in the sample text images. The computer device then uses the font generation model, the font style images, and the font content images to determine three predicted text images for the three existing fonts. Based on the predicted text images and sample text images corresponding to the same existing font, three sets of target losses are determined to train the font generation model.

[0184] In the font generation model application stage, the computer acquires a manually created font style image of the target font and a font content image of the text to be converted. The trained font generation model can predict the generated text image of the text to be converted under the target font based on the font style image and the font content image of the text to be converted. By controlling the font content image of the text to be converted, the font generation model can generate different text images under the target font, thus obtaining a complete set of text images for the target font. Since the output image of the font generation model is a bitmap, it needs to be processed to obtain a vector graphic. Then, the vector graphic is encapsulated to obtain the complete set of target fonts.

[0185] Figure 5 This is a schematic diagram of the structure of a training apparatus for a font generation model provided in an exemplary embodiment of this application. This apparatus can be used in a computer device. Figure 5 As shown, the device includes:

[0186] The acquisition module 501 is used to acquire font style images of n sample fonts, font content images of the first character, and sample text images of the first character under n sample fonts, where n is greater than or equal to 2.

[0187] The prediction module 502 is used to call the font generation model to learn the style features in the font style images of n sample fonts and the content features in the font content image of the first character, so as to predict the predicted text image of the first character under n sample fonts.

[0188] Training module 503 is used to train the font generation model based on n sets of errors between the predicted text images of n sample fonts and the sample text images of n sample fonts. Each of the n sets of errors represents the error between the predicted text image and the sample text image of the first character under the same sample font.

[0189] In an optional design, prediction module 502 is used for:

[0190] The font generation model is invoked to extract features from the font style images of n sample fonts, resulting in n style feature vectors corresponding to the n sample fonts. These style feature vectors reflect the font style of the sample fonts. The font generation model is then invoked to extract features from the font content image of the first character, resulting in a content feature vector. This content feature vector reflects the content of the text in the font content image of the first character. Finally, the font generation model is invoked to predict the predicted text image of the first character under the n sample fonts, based on both the n style feature vectors and the content feature vector of the first character.

[0191] In an optional design, the font generation model includes a multi-style encoder. Prediction module 502 is used for:

[0192] The multi-style encoder is invoked to extract features from the font style images of n sample fonts, resulting in n style feature vectors. The multi-style encoder is then invoked to extract features from the font content image of the first character, resulting in a content feature vector.

[0193] In an optional design, the multi-style encoder includes n style encoders. Prediction module 502 is used for:

[0194] Call n style encoders to extract features from one of the font style images of n sample fonts, and obtain n style feature vectors.

[0195] In an optional design, the multi-style encoder includes a content encoder. Prediction module 502 is used for:

[0196] The content encoder is invoked to extract features from the font content image of the first text, resulting in a content feature vector.

[0197] In an optional design, the font generation model includes a mixer. Prediction module 502 is used for:

[0198] The mixer is invoked to fuse the n style feature vectors with the content feature vectors respectively, resulting in n fused feature vectors. The font generation model is then invoked to predict the text images for the n sample fonts based on the n fused feature vectors.

[0199] In an optional design, the font generation model also includes a decoder. Prediction module 502 is used for:

[0200] The decoder is called to decode the n fused feature vectors respectively, and the predicted text images under n sample fonts are obtained.

[0201] In an optional design, training module 503 is used for:

[0202] Based on the predicted text images for n sample fonts and the images of the same sample font dimension from the sample text images for n sample fonts, and the minimum absolute value deviation function, n sets of correction losses are determined. Based on the product of each set of correction losses, the first parameter, and the second parameter, n sets of target correction losses are determined. The font generation model is then trained using these n sets of target correction losses.

[0203] The first parameter is negatively correlated with the number of black pixels in the sample text image, and the second parameter is determined based on the normalized exponential function and the average pixel value of the black pixels in the sample text image. Black pixels are pixels with pixel values ​​below a threshold.

[0204] In an optional design, the font generation model also includes a discriminator. Training module 503 is used for...

[0205] The discriminator inputs the predicted text images for n sample fonts, the images of the same sample font dimension from the font style images of the n sample fonts, and the font content image of the first character into the discriminator to obtain n first prediction labels. The first prediction labels are used to reflect whether the predicted text image is a real image. The discriminator inputs the sample text images for n sample fonts, the images of the same sample font dimension from the font style images of the n sample fonts, and the font content image of the first character into the discriminator to obtain n second prediction labels. The second prediction labels are used to reflect whether the sample text image is a real image.

[0206] Based on the n first labels and n second labels corresponding to the same sample font dimension, and the loss function of the generative adversarial network, n sets of adversarial losses are determined. The font generation model is then trained based on the n sets of target correction losses and the n sets of adversarial losses corresponding to the same sample font dimension.

[0207] In an optional design, training module 503 is used for:

[0208] The n-group target loss is determined by summing the losses corresponding to the same sample font dimension from the n-group weighted target correction loss and the n-group weighted adversarial loss. The n-group weighted target correction loss is determined by multiplying each target correction loss by the first weight, and the n-group weighted adversarial loss is determined by multiplying each adversarial loss by the second weight. The font generation model is then trained using the n-group target losses.

[0209] In an optional design, module 501 is used to acquire the font style image of the target font and the font content image of the second character. Prediction module 502 is used to input n identical font style images of the target font and font content images of the second character into the font generation model to obtain the generated text image of the second character under the target font predicted by the font generation model.

[0210] In an optional design, prediction module 502 is used for:

[0211] Input n identical target font style images into the font generation model to obtain n target style feature vectors, which reflect the font style of the target font. Input the font content image of the second text into the font generation model to obtain target content feature vectors, which reflect the content of the text in the font content image of the second text. Call the font generation model to determine the generated text image based on the n target style feature vectors and the target content feature vectors.

[0212] In an optional design, prediction module 502 is used for:

[0213] Determine the average value of n target style feature vectors. Invoke the font generation model and, based on the average value of the n target style feature vectors and the target content feature vector, determine the text image to be generated.

[0214] It should be noted that the font generation model training device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the font generation model training device and the font generation model training method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0215] Embodiments of this application also provide a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the training method of the font generation model provided in the above-described method embodiments.

[0216] Alternatively, the computer device is a server. For example, Figure 6 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application.

[0217] The computer device 600 includes a central processing unit (CPU) 601, a system memory 604 including random access memory (RAM) 602 and read-only memory (ROM) 603, and a system bus 605 connecting the system memory 604 and the CPU 601. The computer device 600 also includes a basic input / output system (I / O system) 606 to facilitate information transfer between various components within the computer device, and a mass storage device 607 for storing the operating system 613, application programs 614, and other program modules 615.

[0218] The basic input / output system 606 includes a display 608 for displaying information and an input device 609 for user input, such as a mouse or keyboard. Both the display 608 and the input device 609 are connected to the central processing unit 601 via an input / output controller 610 connected to the system bus 605. The basic input / output system 606 may also include the input / output controller 610 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 610 also provides output to a display screen, printer, or other types of output devices.

[0219] The mass storage device 607 is connected to the central processing unit 601 via a mass storage controller (not shown) connected to the system bus 605. The mass storage device 607 and its associated computer-readable storage media provide non-volatile storage for the computer device 600. That is, the mass storage device 607 may include computer-readable storage media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0220] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage devices, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 604 and mass storage device 607 described above can be collectively referred to as memory.

[0221] The memory stores one or more programs, which are configured to be executed by one or more central processing units 601. The one or more programs contain instructions for implementing the above method embodiments, and the central processing unit 601 executes the one or more programs to implement the methods provided by the various method embodiments described above.

[0222] According to various embodiments of this application, the computer device 600 can also be connected to a remote computer device on a network, such as the Internet. That is, the computer device 600 can be connected to a network 612 via a network interface unit 611 connected to the system bus 605, or the network interface unit 611 can be used to connect to other types of networks or remote computer device systems (not shown).

[0223] The memory further includes one or more programs stored in the memory, and the one or more programs include steps performed by a computer device in the methods provided in the embodiments of this application.

[0224] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set. When the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor of a computer device, the training method for the font generation model provided in the above-described method embodiments is implemented.

[0225] This application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the font generation model training method provided in the above-described method embodiments.

[0226] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0227] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent switching, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A training method for a font generation model, characterized in that, The font generation model includes: a multi-style encoder, a mixer, and a decoder. The multi-style encoder includes n style encoders and a content encoder, where n is greater than or equal to 2. The content encoder and the decoder are connected using a skip-pass connection. The method includes: Obtain font style images of n sample fonts, font content images of the first text, and sample text images of the first text under the n sample fonts; The n style encoders are invoked to extract features from the font style image corresponding to one of the n sample fonts, respectively, to obtain n style feature vectors. One of the style feature vectors is used to reflect the font style of the corresponding sample font. The content encoder is invoked to extract features from the font content image of the first character to obtain a content feature vector, which is used to reflect the content of the text in the font content image of the first character. The mixer is invoked to fuse the n style feature vectors with the content feature vectors respectively, resulting in n fused feature vectors; The decoder is invoked to decode the n fused feature vectors respectively, thereby obtaining the predicted text images under the n sample fonts; The font generation model is trained based on n sets of errors between the predicted text images under the n sample fonts and the sample text images under the n sample fonts. Each of the n sets of errors is the error between the predicted text image and the sample text image of the first text under the same sample font.

2. The method according to claim 1, characterized in that, The step of training the font generation model based on n sets of errors between the predicted text images under the n sample fonts and the sample text images under the n sample fonts includes: Based on the predicted text images under the n sample fonts and the images of the sample text images under the n sample fonts, the images of the same sample font dimension, and the minimum absolute value deviation function, n sets of correction losses are determined; Based on the product of the correction loss, the first parameter, and the second parameter in each group, determine n groups of target correction losses; The font generation model is trained based on the n sets of target correction losses; The first parameter is negatively correlated with the number of black pixels in the sample text image, and the second parameter is determined based on the normalized exponential function and the average pixel value of the black pixels in the sample text image, wherein the black pixels are pixels with pixel values ​​below a threshold.

3. The method according to claim 2, characterized in that, The font generation model further includes a discriminator; training the font generation model based on the n sets of target correction losses includes: The predicted text images under the n sample fonts, the images of the font style images of the n sample fonts at the same sample font dimension, and the font content image of the first text are respectively input into the discriminator to obtain n first prediction labels. The first prediction labels are used to reflect whether the predicted text image is a real image. The sample text images under the n sample fonts, the images of the font style images of the n sample fonts at the same sample font dimension, and the font content image of the first text are respectively input into the discriminator to obtain n second prediction labels. The second prediction labels are used to reflect whether the sample text image is a real image. Based on the n first predicted labels and the n second predicted labels, the labels corresponding to the same sample font dimension, and the loss function of the generative adversarial network, n sets of adversarial losses are determined; The font generation model is trained based on the n sets of target correction losses and the n sets of adversarial losses corresponding to the same sample font dimension.

4. The method according to claim 3, characterized in that, The step of training the font generation model based on the loss corresponding to the same sample font dimension from the n sets of target correction losses and the n sets of adversarial losses includes: Based on the sum of the losses corresponding to the same sample font dimension in the n sets of weighted target correction losses and n sets of weighted adversarial losses, n sets of target losses are determined. The n sets of weighted target correction losses are determined by the product of each set of target correction losses and the first weight. The n sets of weighted adversarial losses are determined by the product of each set of adversarial losses and the second weight. The font generation model is trained based on the n sets of target losses.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Obtain the font style image of the target font and the font content image of the second character; Input n identical font style images of the target font and the font content image of the second character into the font generation model to obtain the generated text image of the second character under the target font predicted by the font generation model.

6. The method according to claim 5, characterized in that, The step of inputting n identical font style images of the target font and the font content image of the second character into the font generation model to obtain the generated text image of the second character under the target font predicted by the font generation model includes: Input n identical font style images of the target font into the font generation model to obtain n target style feature vectors, which are used to reflect the font style of the target font; input the font content image of the second text into the font generation model to obtain target content feature vectors, which are used to reflect the content of the text in the font content image of the second text; The font generation model is invoked to determine the generated text image based on the n target style feature vectors and the target content feature vectors.

7. The method according to claim 6, characterized in that, The step of calling the font generation model and determining the generated text image based on the n target style feature vectors and the target content feature vector includes: Determine the average value of the n target style feature vectors; The font generation model is invoked, and the generated text image is determined based on the average value of the n target style feature vectors and the target content feature vector.

8. A training device for a font generation model, characterized in that, The font generation model includes: a multi-style encoder, a mixer, and a decoder. The multi-style encoder includes n style encoders and a content encoder, where n is greater than or equal to 2. The content encoder and the decoder are connected via a skip-pass connection. The device includes: The acquisition module is used to acquire font style images of n sample fonts, font content images of the first text, and sample text images of the first text under the n sample fonts; The prediction module is used to invoke the n style encoders to extract features from the font style image corresponding to one of the n sample fonts, obtaining n style feature vectors, each style feature vector reflecting the font style of the corresponding sample font; invoke the content encoder to extract features from the font content image of the first character, obtaining a content feature vector, the content feature vector reflecting the content of the text in the font content image of the first character; invoke the mixer to fuse the n style feature vectors with the content feature vector, obtaining n fused feature vectors; and invoke the decoder to decode the n fused feature vectors to obtain the predicted text image under the n sample fonts. The training module is used to train the font generation model based on n sets of errors between the predicted text images under the n sample fonts and the sample text images under the n sample fonts. Each of the n sets of errors is the error between the predicted text image and the sample text image of the first text under the same sample font.

9. The apparatus according to claim 8, characterized in that, The training module is used for: Based on the predicted text images under the n sample fonts and the images of the sample text images under the n sample fonts, the images of the same sample font dimension, and the minimum absolute value deviation function, n sets of correction losses are determined; Based on the product of the correction loss, the first parameter, and the second parameter in each group, determine n groups of target correction losses; The font generation model is trained based on the n sets of target correction losses; The first parameter is negatively correlated with the number of black pixels in the sample text image, and the second parameter is determined based on the normalized exponential function and the average pixel value of the black pixels in the sample text image, wherein the black pixels are pixels with pixel values ​​below a threshold.

10. The apparatus according to claim 9, characterized in that, The font generation model further includes a discriminator; the training module is used for: The predicted text images under the n sample fonts, the images of the font style images of the n sample fonts, the images of the same sample font dimension, and the font content image of the first text are respectively input into the discriminator to obtain n first prediction labels. The first prediction labels are used to reflect whether the predicted text image is a real image. The sample text images under the n sample fonts and the font style images of the n sample fonts, the images on the same sample font dimension, and the font content image of the first text are respectively input into the discriminator to obtain n second prediction labels. The second prediction labels are used to reflect whether the sample text image is a real image. Based on the n first predicted labels and the n second predicted labels, the labels corresponding to the same sample font dimension, and the loss function of the generative adversarial network, n sets of adversarial losses are determined; The font generation model is trained based on the n sets of target correction losses and the n sets of adversarial losses corresponding to the same sample font dimension.

11. The apparatus according to claim 10, characterized in that, The training module is used for: Based on the sum of the losses corresponding to the same sample font dimension in the n sets of weighted target correction losses and n sets of weighted adversarial losses, n sets of target losses are determined. The n sets of weighted target correction losses are determined by the product of each set of target correction losses and the first weight. The n sets of weighted adversarial losses are determined by the product of each set of adversarial losses and the second weight. The font generation model is trained based on the n sets of target losses.

12. The apparatus according to any one of claims 8 to 11, characterized in that, The acquisition module is also used for: Obtain the font style image of the target font and the font content image of the second character; The prediction module is also used for: Input n identical font style images of the target font and the font content image of the second character into the font generation model to obtain the generated text image of the second character under the target font predicted by the font generation model.

13. The apparatus according to claim 12, characterized in that, The prediction module is also used for: Input n identical font style images of the target font into the font generation model to obtain n target style feature vectors, which are used to reflect the font style of the target font; input the font content image of the second text into the font generation model to obtain target content feature vectors, which are used to reflect the content of the text in the font content image of the second text; The font generation model is invoked to determine the generated text image based on the n target style feature vectors and the target content feature vectors.

14. The apparatus according to claim 13, characterized in that, The prediction module is used for: Determine the average value of the n target style feature vectors; The font generation model is invoked, and the generated text image is determined based on the average value of the n target style feature vectors and the target content feature vector.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the training method for the font generation model as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the training method for the font generation model as described in any one of claims 1 to 7.

17. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the training method for the font generation model as described in any one of claims 1 to 7.

Images