A method and system for generating Chinese characters in calligraphic style

Abstract

The present application discloses a method and system for generating Chinese characters in the style of a calligrapher, belonging to the technical field of text image processing. The optimization method comprises: collecting initial Chinese characters in calligraphy and preprocessing them, generating Chinese characters in calligraphy, obtaining corresponding standard Chinese characters images, and forming multiple sets of paired data sets; constructing a Chinese characters in calligraphy style generation model based on a generative adversarial network, wherein the generator comprises a global generation network, an encoder, a decoder, and a feature extraction network, and introduces a conditional attention mechanism; designing a multi-scale discriminator to distinguish between real Chinese characters in calligraphy style and pseudo Chinese characters in calligraphy style generated by the generator; constructing an overall loss function, combining adversarial loss, multi-scale loss, category loss, and feature loss, and iteratively optimizing the model; inputting a specific standard Chinese character image, and generating a Chinese characters in calligraphy style with a specific calligrapher. The method can improve the generation quality of Chinese characters in calligraphy style.

Description

Technical Field

[0001] The present invention belongs to the technical field of text image processing, and in particular relates to a method and system for generating Chinese characters in the style of a calligrapher. Background Art

[0002] Calligraphy is an art form unique to Chinese characters with a long history. However, some calligraphy characters have been damaged to varying degrees due to their age. In order to pass on the calligraphy that carries the cultural traditions of the nation and the country, and to allow learners to improve their calligraphy skills by copying outstanding works from past dynasties, a large number of researchers have tried to generate calligraphy characters with the style of calligraphers.

[0003] Calligraphy Chinese character generation is also a kind of image generation, so we can focus on the relevant methods in the field of image generation. At present, the methods of image generation can be divided into traditional methods and methods based on deep learning. Traditional methods mainly include methods based on image restoration fusion and methods based on manually designed features. Since deep learning technology can capture and express complex data more effectively, image generation models based on deep learning are gradually emerging. The methods based on deep learning mainly include autoregressive model (AR), variational autoencoder model (VAE), and generative model based on generative adversarial network (GAN).

[0004] Compared with traditional methods, the use of deep learning-based methods to generate calligraphy Chinese characters in the style of calligraphers can generate more complex and varied calligraphy Chinese character images. However, there are still problems such as low quality of generated images, blurred details, loss of fine stroke features or trace features, and it is difficult to learn the corresponding calligrapher style with a small sample data set, resulting in poor generation quality of calligraphy Chinese character images in the style of calligraphers. Summary of the invention

[0005] Based on this, the present application proposes a method and system for generating Chinese characters in calligraphy style in a calligrapher's style, aiming to solve the problem of poor generation quality of Chinese characters in calligraphy style due to low quality of generated images, blurred details, loss of fine stroke features or trace connection features, and difficulty in learning the corresponding calligrapher style from a small sample data set.

[0006] The first aspect of the present application provides a method for generating Chinese characters in a calligrapher's style, the method comprising:

[0007] Collecting initial calligraphy Chinese character images, preprocessing the initial calligraphy Chinese character images to generate calligraphy Chinese character images, acquiring standard Chinese character images corresponding to the calligraphy Chinese character images, forming a plurality of groups of paired data from the calligraphy Chinese character images and the standard Chinese character images, and forming a paired data set based on the plurality of groups of paired data;

[0008] A calligrapher-style calligraphy Chinese character generation model based on a generative adversarial network is constructed, which includes: setting a global generation network G1 in a generator, the global generation network G1 including an encoder and a decoder, adding a feature extraction network before the encoder, introducing a conditional attention mechanism in the decoder, and generating a pseudo-calligrapher-style calligraphy Chinese character image from an input standard Chinese character image through the generator; designing a multi-scale discriminator for receiving the standard Chinese character image, the calligrapher-style calligraphy Chinese character image and the pseudo-calligrapher-style calligraphy Chinese character image to distinguish the calligrapher-style calligraphy Chinese character image from the pseudo-calligrapher-style calligraphy Chinese character image;

[0009] Constructing an overall loss function for training the calligrapher-style calligraphy Chinese character generation model, wherein the overall loss function includes an adversarial loss function, a multi-scale loss function, a category loss function, and a feature loss function;

[0010] According to the overall loss function, using the paired data set, iteratively optimize the calligrapher style calligraphy Chinese character generation model to obtain an optimized calligrapher style calligraphy Chinese character generation model;

[0011] Determine the calligraphy Chinese characters that need to be generated and the specific calligraphy style, obtain the specific standard Chinese character images corresponding to the calligraphy Chinese characters that need to be generated, input the specific standard Chinese character images into the optimized calligraphy Chinese character generation model of the calligraphy style, and obtain the calligraphy Chinese characters of the specific calligraphy style.

[0012] Compared with the prior art, the present application provides a method for generating Chinese characters in a calligraphic style. The method for generating Chinese characters in a calligraphic style forms a paired data set including calligraphy Chinese character images and standard Chinese character images by collecting initial calligraphy Chinese character images, preprocessing and forming multiple groups of paired data. This provides rich sample data for subsequent model training, which is helpful for the model to learn the characteristics and style of calligraphy Chinese characters. A calligraphic style calligraphy Chinese character generation model based on a generative adversarial network is constructed, including a global generation network G1, a multi-scale discriminator, etc. The feature extraction network and conditional attention mechanism in the global generation network G1 are helpful to generate pseudo-calligrapher style calligraphy Chinese character images with more calligraphic style. The multi-scale discriminator can better judge and distinguish between real calligrapher style calligraphy Chinese character images and pseudo-calligrapher style calligraphy Chinese character images generated by the generator at different scales, thereby improving the discrimination ability of the model. A calligrapher style calligraphy Chinese character generation model is constructed. The overall loss function of training includes adversarial loss function, multi-scale loss function, category loss function and feature loss function. The comprehensive application of these loss functions prompts the model to generate calligraphy Chinese characters not only close to the calligrapher's style in appearance, but also consistent with standard Chinese characters in semantics, while ensuring that the generated images have multi-scale features and diversity; the calligrapher-style calligraphy Chinese character generation model is iteratively optimized using paired data sets to obtain an optimized model, and the parameters of the model are continuously adjusted to make it closer to the calligrapher's style when generating calligraphy Chinese characters, thereby improving the accuracy and quality of the generation; according to the calligraphy Chinese characters to be generated and the specific calligrapher's style, specific standard Chinese character images are obtained and input into the optimized model to obtain calligraphy Chinese characters with a specific calligraphy style. This process realizes the function of generating calligraphy Chinese characters with a specific calligrapher's style according to user needs, thereby meeting the user's needs for personalized calligraphy works.

[0013] Mainly, the calligrapher-style calligraphy Chinese character generation method based on generative adversarial network is a method of generating calligraphy Chinese characters by improving the generative adversarial network model in the field of image generation. This method solves the problem that small sample data sets are difficult to learn by adding a pre-trained feature extraction network before the encoder in the generator; solves the problems of low quality of generated images and loss of details by using a multi-scale architecture of the generator and the discriminator; enhances the stability of training by proposing a multi-scale discriminator loss function; and introduces a conditional attention module to enable the model to better capture and replicate the detailed features of the font and enhance the generalization ability of the model. This method performs well in the generation of calligrapher-style calligraphy Chinese characters.

[0014] As an optional implementation of the first aspect, the steps of collecting the initial calligraphy Chinese character image, preprocessing the initial calligraphy Chinese character image, and generating the calligraphy Chinese character image include:

[0015] Image processing technology is used to crop and segment the collected initial calligraphy Chinese character images to extract single-character images;

[0016] Performing background removal processing on the single word image to retain the handwriting, and performing binarization processing on the single word image after the background removal processing to convert it into a black and white single word image;

[0017] The black and white single word image is converted into a corresponding TTF font format using font making software to generate a calligraphy Chinese character image.

[0018] As an optional implementation of the first aspect, the calligraphy Chinese character images and the standard Chinese character images are grouped into multiple sets of paired data, and the step of forming a paired data set according to the multiple sets of paired data includes:

[0019] use Represents the standard Chinese character image, using Represents calligraphy Chinese character images, using Represents each set of paired data, where , ,and and Represented by the same Chinese character.

[0020] As an optional implementation of the first aspect, the step of generating a pseudo-calligrapher style calligraphy Chinese character image from an input standard Chinese character image by the generator includes:

[0021] The generator includes a global generation network G1 and a local enhancement network G2, the font framework of the standard Chinese character image is generated by the global generation network G1, and the font features of the standard Chinese character image are extracted by the local enhancement network G2;

[0022] The font framework and the font features are feature fused to generate a font fusion, and the font fusion is input into the local enhancement network G2 to generate a pseudo-calligrapher style calligraphy Chinese character image.

[0023] As an optional implementation of the first aspect, the global generation network G1 includes a feature extraction network VGG-19, an encoder, a decoder introducing a conditional attention mechanism, and a discriminator.

[0024] The features of the input standard Chinese character image are extracted by the feature extraction network VGG-19, the extracted features of the input standard Chinese character image are converted into a feature map by the encoder, the detailed features of the image are extracted from the feature map by the decoder introducing the conditional attention mechanism, and the font frame corresponding to the detailed features is discriminated by the discriminator.

[0025] As an optional implementation manner of the first aspect, the step of designing a multi-scale discriminator includes:

[0026] Multi-scale discriminators D1, D2 and D3 are designed, and D1, D2 and D3 are set to have the same structure, and D1, D2 and D3 are set to process large, medium and small sized images respectively. Discriminator D1 is used to focus on the detailed features of the image, and discriminator D3 is used to focus on the overall features of the image.

[0027] As an optional implementation manner of the first aspect, the overall loss function is:

[0028] ,

[0029] Where L represents the overall loss function, represents a generator, represents the discriminator, represents the adversarial loss function, represents the multi-scale loss function, represents the category loss function, represents the feature loss function, λ1, λ2 and λ3 are the relative weights of the multi-scale loss function, category loss function and feature loss function respectively.

[0030] A second aspect of the embodiment of the present application provides a calligraphy Chinese character generation system in the style of a calligrapher, the calligraphy Chinese character generation system comprising:

[0031] A data acquisition module is used to collect initial calligraphy Chinese character images, pre-process the initial calligraphy Chinese character images, generate calligraphy Chinese character images, acquire standard Chinese character images corresponding to the calligraphy Chinese character images, form multiple sets of paired data with the calligraphy Chinese character images and the standard Chinese character images, and form a paired data set based on the multiple sets of paired data;

[0032] A model module is constructed for constructing a calligrapher-style calligraphy Chinese character generation model based on a generative adversarial network, which includes: setting a global generation network G1 in a generator, the global generation network G1 including an encoder and a decoder, adding a feature extraction network before the encoder, introducing a conditional attention mechanism in the decoder, and generating a pseudo-calligrapher-style calligraphy Chinese character image from an input standard Chinese character image through the generator; designing a multi-scale discriminator for receiving the standard Chinese character image, the calligrapher-style calligraphy Chinese character image and the pseudo-calligrapher-style calligraphy Chinese character image to distinguish the calligrapher-style calligraphy Chinese character image from the pseudo-calligrapher-style calligraphy Chinese character image;

[0033] Constructing a loss function module, which is used to construct an overall loss function for training the calligrapher style calligraphy Chinese character generation model, wherein the overall loss function includes an adversarial loss function, a multi-scale loss function, a category loss function, and a feature loss function;

[0034] A model training module, used for iteratively optimizing the calligrapher style calligraphy Chinese character generation model using the paired data set according to the overall loss function to obtain an optimized calligrapher style calligraphy Chinese character generation model;

[0035] The data generation module is used to determine the calligraphy Chinese characters that need to be generated and the specific calligraphy style, obtain the specific standard Chinese character images corresponding to the calligraphy Chinese characters that need to be generated, input the specific standard Chinese character images into the optimized calligraphy Chinese character generation model of the calligraphy style, and obtain the calligraphy Chinese characters of the specific calligraphy style.

[0036] A third aspect of an embodiment of the present application provides a computer device, which includes a memory, a processor, and a processing program stored in the memory and executable on the processor, wherein the processing program, when executed by the processor, implements the above-mentioned method for generating calligraphy Chinese characters in a calligrapher's style.

[0037] A fourth aspect of an embodiment of the present application provides a storage medium on which a processing program is stored. When the processing program is executed by a processor, the method for generating calligraphy Chinese characters in the style of a calligrapher is executed.

[0038] Additional aspects and advantages of the present application will be given in part in the following description, and in part will become apparent from the following description, or will be understood through the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a method for generating Chinese characters in a calligraphic style proposed in the first embodiment of the present application;

[0040] Figure 2 This is a structural diagram of a calligrapher-style calligraphy Chinese character generation model based on a generative adversarial network in the first implementation of this application;

[0041] Figure 3 A structural diagram of the global generation network G1 in the first implementation of the present application;

[0042] Figure 4 A schematic diagram of the structure of a calligrapher-style calligraphy Chinese character generation system proposed in the second embodiment of the present application.

[0043] The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0046] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0047] See also Figure 1 , is a flow chart of a method for generating Chinese characters in a calligraphic style proposed in the first embodiment of the present application, the proposed method comprising:

[0048] S01: Collect initial calligraphy Chinese character images, pre-process the initial calligraphy Chinese character images to generate calligraphy Chinese character images, obtain standard Chinese character images corresponding to the calligraphy Chinese character images, group the calligraphy Chinese character images and the standard Chinese character images into multiple groups of paired data, and form a paired data set based on the multiple groups of paired data.

[0049] It should be noted that deep learning cannot be separated from data sets, but there is a lack of public data sets in the field of calligraphy Chinese character generation. Therefore, this method needs to construct a data set suitable for calligraphy Chinese character generation for model training.

[0050] Specifically, the step S01 includes S011 to S014, wherein:

[0051] S011: Use image processing technology to perform image cropping and single-word segmentation on the collected initial calligraphy Chinese character images to extract single-word images.

[0052] It should be noted that calligraphy images are collected from different calligraphy works, including but not limited to calligraphy inscriptions by historical masters, collections of works by modern calligraphers, and high-definition calligraphy images on the Internet. To ensure data diversity, works of different calligraphy styles should be collected as much as possible. The purpose of this step is to extract individual Chinese characters from complex calligraphy images as the basic unit for subsequent processing.

[0053] S012: performing background removal processing on the single word image to retain the handwriting, and performing binarization processing on the single word image after the background removal processing to convert it into a black and white single word image.

[0054] It should be noted that this step can further simplify the complexity of the image.

[0055] S013: Convert the black and white single word image into a corresponding TTF font format using font making software to generate a calligraphy Chinese character image.

[0056] S014: Merge the standard Chinese character image and the calligraphy Chinese character image corresponding to the calligraphy Chinese character image into paired data to perform calligraphy style conversion training.

[0057] Optionally, use Represents the standard Chinese character image, using Represents calligraphy Chinese character images, using Represents each set of paired data, where , ,and and Represented by the same Chinese character.

[0058] S02: Construct a calligrapher-style calligraphy Chinese character generation model based on a generative adversarial network, which includes setting a global generation network G1 in the generator, the global generation network G1 includes an encoder and a decoder, adding a feature extraction network in front of the encoder, introducing a conditional attention mechanism in the decoder, and generating a pseudo-calligrapher-style calligraphy Chinese character image from the input standard Chinese character image through the generator; designing a multi-scale discriminator for receiving the standard Chinese character image, the calligrapher-style calligraphy Chinese character image and the pseudo-calligrapher-style calligraphy Chinese character image to distinguish the calligrapher-style calligraphy Chinese character image from the pseudo-calligrapher-style calligraphy Chinese character image.

[0059] It should be noted that the calligrapher-style calligraphy Chinese character generation model based on the generative adversarial network is based on the encoder-decoder architecture of the generative adversarial network as the backbone network. First, in order to solve the problem that small sample data sets are difficult to learn and have slow training speed, this model adds a pre-trained feature extraction network VGG-19 in front of the encoder in the generator. The encoder part is responsible for converting the input image into a feature map, while the feature extraction network is responsible for further feature extraction. Secondly, in order to address the problems of low quality of generated images and loss of details, this model uses a multi-scale architecture, which includes multiple generators and discriminators working at different scales. This design allows the model to capture and generate image details more carefully. At the same time, in order to enable the model to better capture and replicate the detailed features of the font, such as stroke thickness, tilt angle, etc., and enhance the generalization ability of the model, a conditional attention module is introduced in the decoder of the model.

[0060] Specifically, Figure 2 As shown in the figure, it is a structural diagram of the calligrapher style calligraphy Chinese character generation model based on the generative adversarial network, showing the multi-scale architecture of the generator and the discriminator.

[0061] The core parts of the generators of different scales are the global generation network G1 and the local enhancement network G2. The data processing process of the generator specifically includes:

[0062] The font framework of the standard Chinese character image is generated by the global generation network G1, and the font features of the standard Chinese character image are extracted by the local enhancement network G2; the font framework and the font features are feature fused to generate a font fusion, and the font fusion is input into the local enhancement network G2 to generate a pseudo-calligrapher style calligraphy Chinese character image.

[0063] As you can understand, G1 is responsible for generating the basic framework of the font, and G2 is mainly composed of convolution blocks, one part of which is responsible for extracting features and combining these features with the features before the final output layer of G1 to achieve deep fusion of information. The fused information is then fed into another part of G2 to generate high-resolution image results.

[0064] The discriminator is a characteristic of the generative adversarial network. It is responsible for determining whether the data obtained is a real sample or a pseudo sample generated by the generator. The data processing process of the discriminator specifically includes:

[0065] Multi-scale discriminators D1, D2 and D3 are designed, and D1, D2 and D3 are set to have the same structure, and D1, D2 and D3 are set to process large, medium and small sized images respectively. Discriminator D1 is used to focus on the detailed features of the image, and discriminator D3 is used to focus on the overall features of the image.

[0066] Furthermore, if Figure 3 As shown in the figure, it is a structural diagram of the global generative network G1, which includes a pre-trained VGG-19 network (responsible for feature extraction), an encoder (the input image is converted into a specific vector, and then the merged vector of the category embedding and the character embedding is added), a decoder (responsible for generation, introducing a conditional attention mechanism to allow the model to better capture the detailed features of the font), and a discriminator. The data processing process of the global generative network G1 includes:

[0067] The features of the input standard Chinese character image are extracted by the feature extraction network VGG-19, the extracted features of the input standard Chinese character image are converted into a feature map by the encoder, the detailed features of the image are extracted from the feature map by the decoder introducing the conditional attention mechanism, and the font frame corresponding to the detailed features is discriminated by the discriminator.

[0068] S03: Constructing an overall loss function for training the calligrapher style calligraphy Chinese character generation model, wherein the overall loss function includes an adversarial loss function, a multi-scale loss function, a category loss function and a feature loss function.

[0069] It should be noted that the model proposed by this method needs to construct a loss function during training. The loss function includes adversarial loss function, multi-scale loss function, category loss function and feature loss function. Finally, the above four loss functions are combined to obtain the final overall loss function. Represents the standard Chinese character image, represents the calligraphy Chinese character image of the target calligrapher style, then the training process of the model can be expressed as: given paired data and In this case, the proposed model and loss function are used to learn the transformation from standard Chinese characters to target calligraphic Chinese characters.

[0070] Specifically, step S03 includes S031 to S035, wherein:

[0071] S031: Construct adversarial loss function;

[0072] It can be understood that the adversarial loss function is a conditional adversarial loss function. The adversarial loss function is:

[0073] ,

[0074] in, represents a generator, represents the discriminator, represents the expected value of the distribution function, represents random noise.

[0075] S032: Construct multi-scale loss function;

[0076] It is understandable that in order to enhance the stability of training, the multi-scale loss not only adds the losses of the discriminators at each scale of the true and false Chinese character images, but also adds the multi-scale loss of the feature map output by the discriminator. The multi-scale loss function is:

[0077] ,

[0078] in, is the number of layers, is the number of elements in each layer.

[0079] S033: Construct category loss function;

[0080] Understandably, in order to prevent the model from confusing and mixing various calligraphy styles together, a category loss function is used to penalize the generator. The category loss function is:

[0081] ,

[0082] in, is the number of categories, is the one-hot encoding of the true category label, is the class distribution predicted by the model that generated the images.

[0083] S034: Construct feature loss function;

[0084] It is understandable that the standard Chinese characters and the target calligraphy Chinese characters are very similar in structure, so they have similar positions in the embedding space. For this reason, the feature loss function is used to force the decoder to retain the structural features of the generated Chinese characters, narrow the searchable range, and speed up the convergence. The feature loss function is:

[0085] .

[0086] S035: Comprehensively obtain the final overall loss function of the model.

[0087] The overall loss function of the model can be expressed as:

[0088] ,

[0089] Where L represents the overall loss function, represents the adversarial loss function, represents the multi-scale loss function, represents the category loss function, represents the feature loss function, λ1, λ2 and λ3 are the relative weights of the multi-scale loss function, category loss function and feature loss function respectively.

[0090] S04: According to the overall loss function, the calligrapher style calligraphy Chinese character generation model is iteratively optimized using the paired data set to obtain an optimized calligrapher style calligraphy Chinese character generation model.

[0091] It should be noted that after the data set, model and loss function are built, the model training begins. In multiple rounds of training, the model gradually learns the conversion from standard Chinese characters to target calligraphy Chinese characters. After the training, the model's generation ability is tested, that is, inputting a standard Chinese character image, it can output a calligraphy Chinese character image in the style of a specified calligrapher, achieving excellent results.

[0092] S05: Determine the calligraphy Chinese characters to be generated and the specific calligraphy style, obtain the specific standard Chinese character images corresponding to the calligraphy Chinese characters to be generated, input the specific standard Chinese character images into the optimized calligraphy Chinese character generation model of the calligraphy style, and obtain the calligraphy Chinese characters of the specific calligraphy style.

[0093] It should be noted that after the model training and verification are completed, the model's generation ability can be used to complete the task of generating Chinese characters in the style of a calligrapher. First, determine the Chinese characters and the specific calligrapher style you want to generate, then generate pictures that can be used as model input from standard Chinese characters, call the model according to the selected calligrapher style, and finally generate Chinese characters in the specific calligrapher style.

[0094] In summary, this method is dedicated to exploring the problem of calligraphy Chinese character generation in the style of calligraphers, and proposes a new model based on generative adversarial networks to generate calligraphy Chinese characters in the style of calligraphers. The model constructed in this method solves the problem of small sample data sets being difficult to learn and slow to train by adding a pre-trained feature extraction network before the encoding stage of the generator. In addition, a multi-scale architecture and a novel multi-scale discriminator loss function are adopted to improve the quality of calligraphy Chinese character images generated by the model and the stability of training. The model also introduces a conditional attention mechanism in the decoding stage to more accurately describe and reproduce the details of the font. This method effectively solves the problems of fuzzy structure and unclear stroke details when generating calligraphy Chinese characters, and performs well in the task of generating calligraphy Chinese characters in the style of calligraphers.

[0095] See also Figure 4 , which is a schematic diagram of the structure of a calligraphy Chinese character generation system in the style of a calligrapher proposed in the second embodiment of the present application, and the calligraphy Chinese character generation system includes:

[0096] The data acquisition module 10 is used to collect initial calligraphy Chinese character images, pre-process the initial calligraphy Chinese character images, generate calligraphy Chinese character images, obtain standard Chinese character images corresponding to the calligraphy Chinese character images, form multiple sets of paired data with the calligraphy Chinese character images and the standard Chinese character images, and form paired data sets according to the multiple sets of paired data;

[0097] The model building module 20 is used to build a calligrapher-style calligraphy Chinese character generation model based on a generative adversarial network, which includes: setting a global generation network G1 in the generator, the global generation network G1 includes an encoder and a decoder, adding a feature extraction network before the encoder, introducing a conditional attention mechanism in the decoder, and generating a pseudo-calligrapher-style calligraphy Chinese character image from an input standard Chinese character image through the generator; designing a multi-scale discriminator for receiving the standard Chinese character image, the calligrapher-style calligraphy Chinese character image and the pseudo-calligrapher-style calligraphy Chinese character image to distinguish the calligrapher-style calligraphy Chinese character image from the pseudo-calligrapher-style calligraphy Chinese character image;

[0098] A loss function construction module 30 is used to construct an overall loss function for training the calligrapher style calligraphy Chinese character generation model, wherein the overall loss function includes an adversarial loss function, a multi-scale loss function, a category loss function, and a feature loss function;

[0099] A model training module 40 is used to iteratively optimize the calligrapher style calligraphy Chinese character generation model using the paired data set according to the overall loss function to obtain an optimized calligrapher style calligraphy Chinese character generation model;

[0100] The data generation module 50 is used to determine the calligraphy Chinese characters that need to be generated and the specific calligraphy style, obtain the specific standard Chinese character images corresponding to the calligraphy Chinese characters that need to be generated, input the specific standard Chinese character images into the optimized calligraphy Chinese character generation model of the calligraphy style, and obtain the calligraphy Chinese characters of the specific calligraphy style.

[0101] On the other hand, the present application also proposes a computer device, which includes a memory, a processor, and a processing program stored in the memory and executable on the processor, and the processing program, when executed by the processor, implements the above-mentioned method for generating calligraphy Chinese characters in the style of a calligrapher.

[0102] On the other hand, the present application further proposes a storage medium on which a processing program is stored. When the processing program is executed by a processor, the method for generating Chinese characters in calligraphy in the style of a calligrapher is executed.

[0103] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0105] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for generating Chinese characters in a calligrapher's style, characterized in that: The calligraphy Chinese character generation method comprises: Collecting initial calligraphy Chinese character images, preprocessing the initial calligraphy Chinese character images to generate calligraphy Chinese character images, acquiring standard Chinese character images corresponding to the calligraphy Chinese character images, forming a plurality of groups of paired data from the calligraphy Chinese character images and the standard Chinese character images, and forming a paired data set based on the plurality of groups of paired data; A calligrapher-style calligraphy Chinese character generation model based on a generative adversarial network is constructed, which includes: setting a global generation network G1 in a generator, the global generation network G1 including an encoder and a decoder, adding a feature extraction network before the encoder, introducing a conditional attention mechanism in the decoder, and generating a pseudo-calligrapher-style calligraphy Chinese character image from an input standard Chinese character image through the generator; designing a multi-scale discriminator for receiving the standard Chinese character image, the calligrapher-style calligraphy Chinese character image and the pseudo-calligrapher-style calligraphy Chinese character image to distinguish the calligrapher-style calligraphy Chinese character image from the pseudo-calligrapher-style calligraphy Chinese character image; Constructing an overall loss function for training the calligrapher-style calligraphy Chinese character generation model, wherein the overall loss function includes an adversarial loss function, a multi-scale loss function, a category loss function, and a feature loss function; According to the overall loss function, using the paired data set to iteratively optimize the calligrapher style calligraphy Chinese character generation model to obtain an optimized calligrapher style calligraphy Chinese character generation model; Determine the calligraphy Chinese characters that need to be generated and the specific calligraphy style, obtain the specific standard Chinese character images corresponding to the calligraphy Chinese characters that need to be generated, input the specific standard Chinese character images into the optimized calligraphy Chinese character generation model of the calligraphy style, and obtain the calligraphy Chinese characters of the specific calligraphy style.

2. The method for generating Chinese characters in a calligrapher's style according to claim 1, characterized in that: The steps of collecting the initial calligraphy Chinese character image, preprocessing the initial calligraphy Chinese character image, and generating the calligraphy Chinese character image include: Image processing technology is used to crop and segment the collected initial calligraphy Chinese character images to extract single-character images; Performing background removal processing on the single word image to retain the handwriting, and performing binarization processing on the single word image after the background removal processing to convert it into a black and white single word image; The black and white single word image is converted into a corresponding TTF font format using font making software to generate a calligraphy Chinese character image.

3. The method for generating Chinese characters in a calligrapher's style according to claim 1, characterized in that: The calligraphy Chinese character images and the standard Chinese character images are combined into a plurality of paired data sets, and the steps of forming a paired data set according to the plurality of paired data sets include: use Represents the standard Chinese character image, using Represents calligraphy Chinese character images, using Represents each set of paired data, where , ,and and Represented by the same Chinese character.

4. The method for generating Chinese characters in a calligrapher's style according to claim 1, characterized in that: The step of generating a pseudo-calligrapher style calligraphy Chinese character image from an input standard Chinese character image by the generator comprises: The generator includes a global generation network G1 and a local enhancement network G2, the font framework of the standard Chinese character image is generated by the global generation network G1, and the font features of the standard Chinese character image are extracted by the local enhancement network G2; The font framework and the font features are feature fused to generate a font fusion, and the font fusion is input into the local enhancement network G2 to generate a pseudo-calligrapher style calligraphy Chinese character image.

5. The method for generating Chinese characters in a calligrapher's style according to claim 4, characterized in that: The global generation network G1 includes a feature extraction network VGG-19, an encoder, a decoder and a discriminator that introduces a conditional attention mechanism. The features of the input standard Chinese character image are extracted by the feature extraction network VGG-19, the extracted features of the input standard Chinese character image are converted into a feature map by the encoder, the detailed features of the image are extracted from the feature map by the decoder introducing the conditional attention mechanism, and the font frame corresponding to the detailed features is discriminated by the discriminator.

6. The method for generating Chinese characters in a calligrapher's style according to claim 1, characterized in that: The steps of designing a multi-scale discriminator include: Multi-scale discriminators D1, D2 and D3 are designed, and D1, D2 and D3 are set to have the same structure, and D1, D2 and D3 are set to process large, medium and small sized images respectively. Discriminator D1 is used to focus on the detailed features of the image, and discriminator D3 is used to focus on the overall features of the image.

7. The method for generating Chinese characters in a calligrapher's style according to claim 6, characterized in that: The overall loss function is: ， Where L represents the overall loss function, represents a generator, represents the discriminator, represents the adversarial loss function, represents the multi-scale loss function, represents the category loss function, represents the feature loss function, λ1, λ2 and λ3 are the relative weights of the multi-scale loss function, category loss function and feature loss function respectively.

8. A calligraphic Chinese character generation system in the style of a calligrapher, characterized in that: The calligraphy Chinese character generation system comprises: A data acquisition module is used to collect initial calligraphy Chinese character images, pre-process the initial calligraphy Chinese character images, generate calligraphy Chinese character images, acquire standard Chinese character images corresponding to the calligraphy Chinese character images, form multiple sets of paired data with the calligraphy Chinese character images and the standard Chinese character images, and form a paired data set based on the multiple sets of paired data; A model module is constructed for constructing a calligrapher-style calligraphy Chinese character generation model based on a generative adversarial network, which includes: setting a global generation network G1 in a generator, the global generation network G1 including an encoder and a decoder, adding a feature extraction network before the encoder, introducing a conditional attention mechanism in the decoder, and generating a pseudo-calligrapher-style calligraphy Chinese character image from an input standard Chinese character image through the generator; designing a multi-scale discriminator for receiving the standard Chinese character image, the calligrapher-style calligraphy Chinese character image and the pseudo-calligrapher-style calligraphy Chinese character image to distinguish the calligrapher-style calligraphy Chinese character image from the pseudo-calligrapher-style calligraphy Chinese character image; Constructing a loss function module, which is used to construct an overall loss function for training the calligrapher style calligraphy Chinese character generation model, wherein the overall loss function includes an adversarial loss function, a multi-scale loss function, a category loss function, and a feature loss function; A model training module, used for iteratively optimizing the calligrapher style calligraphy Chinese character generation model using the paired data set according to the overall loss function to obtain an optimized calligrapher style calligraphy Chinese character generation model; The data generation module is used to determine the calligraphy Chinese characters that need to be generated and the specific calligraphy style, obtain the specific standard Chinese character images corresponding to the calligraphy Chinese characters that need to be generated, input the specific standard Chinese character images into the optimized calligraphy Chinese character generation model of the calligraphy style, and obtain the calligraphy Chinese characters of the specific calligraphy style.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a processing program stored in the memory and executable on the processor. When the processing program is executed by the processor, a method for generating Chinese characters in calligraphy in the style of a calligrapher as described in any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores a processing program, and when the processing program is executed by the processor, the method for generating Chinese characters in the style of a calligrapher as described in any one of claims 1 to 7 is implemented.

Images