Font generation method based on interactive fusion of Chinese character description sequence and style structure

Abstract

The invention discloses a font generation method based on interactive fusion of a Chinese character description sequence and a style structure, and the method comprises the following steps: 1, constructing a Chinese character style conversion data set and a Chinese character font decomposition table, and dividing the data set into a training set and a test set; 2, constructing a Chinese character style conversion model, wherein the model comprises a Chinese character sequence decomposition module, a style encoder, a structure style interaction fusion module, a decoder and a style discriminator; step 3, training the Chinese character style conversion model, pairing fonts in a training set according to characters, training the model by using paired font images in the training set, and carrying out loop training until a set upper limit of the number of iterations is reached or the loss is less than or equal to a preset threshold value; and 4, inputting the test set into the trained model to obtain a target Chinese character font image of a corresponding style. The problem that in the prior art, the quality of generated fonts is not high is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a font generation method based on interactive fusion of Chinese character description sequence and style structure. BACKGROUND

[0002] In today's globalization, the diversification and internationalization of font styles further promote cross-cultural communication and deepen people's understanding and respect for different cultures. Different fonts express different meanings, convey different visual effects and cultural characteristics, and are indispensable important elements in design. Not only the information conveyed should be considered, but also the beautiful form should be presented to convey a pleasing visual effect. Each style can make the viewer have different visual tendencies, which can further bring people the corresponding psychological connection, so that the viewer can feel the ups and downs of emotions through the font. For example, serif fonts make the text look more elegant and formal, and are suitable for business documents, literature or formal occasions; the variety body is rounded at the stroke transition, so that the font has a visual feeling of being soft in the middle of being hard, is stable, powerful and soft, has artistic, professional and modern characteristics, and can express an artistic fashion; the thin body with its simple style caters to modern aesthetics, and the characteristics are delicate, clean and clear strokes, and the intuitive horizontal and vertical structure and clear layout make reading more relaxed. This font style is suitable for texts that need detailed expression, and is also very suitable for the main text of high-visual publications such as fashion magazines.

[0003] At present, people's desire for font styles with unique personality is growing, which has promoted the development of diversified font generation technology to become a hot research field. However, the large number, different shapes and complex structure of Chinese characters make it extremely challenging to design personalized fonts.

[0004] At present, the technology of Chinese font generation mainly includes two kinds: one is the traditional method relying on computer graphics, which extracts strokes and recombines isolated strokes to create fonts, but the complexity and high accuracy requirements limit the effect. The other is the modern method based on deep learning, which is considered a powerful tool to solve the problem of personalized font design due to its excellent performance in many fields. Although deep learning can be used to automatically generate and style convert fonts at present, the existing method focuses on decoupling the content features and style features of the character shape during conversion, and generates a new character shape by combining the content features of the source character shape with the style features of the reference character shape. However, due to the complexity of the character structure, the feature decoupling process faces significant challenges, often resulting in generated character shapes significantly influenced by the source character shape style and prone to artifacts, causing low quality of the converted font. SUMMARY

[0005] The application aims to provide a font generation method based on interactive fusion of Chinese character description sequence and style structure, and solve the problem of low font quality in the prior art.

[0006] The technical scheme adopted by the application is a font generation method based on interactive fusion of Chinese character description sequence and style structure, comprising the following steps: Step 1, constructing a Chinese character style conversion dataset and a Chinese character form decomposition table, and dividing the dataset into a training set and a test set; Step 2, constructing a Chinese character style conversion model, which comprises a Chinese character sequence decomposition module, a style encoder, a structure style interactive fusion module, a decoder, and a style discriminator; Step 3, training the Chinese character style conversion model, pairing the fonts in the training set by characters, training the model using the paired font images in the training set, and cyclically training until reaching a set upper limit of iteration times or a loss less than or equal to a preset threshold; Step 4, inputting the test set into the trained model to obtain a target Chinese character font image of a corresponding style.

[0007] The application is further characterized in that In step 1, Chinese characters are collected, and each Chinese character is rendered into an image of 128*128 resolution under different fonts, and a bicubic interpolation method is used to keep the proportions consistent.

[0008] The Chinese character form decomposition table is based on the Unicode public Chinese character decomposition table and is simplified and standardized.

[0009] Step 3 specifically pairs the fonts in the training set by characters into reference forms and target forms, inputs the reference forms into the Chinese character sequence decomposition module to decompose them into Chinese character form sequences, and encodes the Chinese character form sequences into semantic features; then extracts the style features of the reference forms through the encoder, aggregates the global and local style features using the structure style interactive fusion module, inputs the style features output by the structure style interactive fusion module together with the semantic features into the decoder, generates a target font mark sequence through self-recurrence, and then decodes the target font mark sequence through VQGAN to obtain a target font image; The generated target font image and the target form image in the training set are input into the contrast loss module of the style discriminator, the model parameters are updated through calculation of the loss function in the reverse direction, the above process is cyclically trained until reaching a set upper limit of iteration times or a loss less than or equal to a preset threshold, and the training of the model is completed.

[0010] The extraction of the style features of the reference forms through the encoder comprises: A convolutional neural network structure is adopted to extract multi-layer features of the image through downsampling operation, and convert the input image into a low-dimensional feature map; After the input image is encoded to extract high-level semantic features, the channel number is adjusted to be consistent with the embedding dimension of the vector quantizer through a 1*1 convolution layer, the processed feature vector is input into the vector quantizer, and the discrete code word in the codebook is matched through the nearest neighbor search to realize feature discretization.

[0011] The structure style interactive fusion module includes potential feature extraction of reference glyphs, similarity calculation of target glyphs and reference glyphs, global style feature fusion, local style feature fusion, and style feature fusion and output; The potential feature extraction of the reference glyphs obtains an intermediate feature set through the following formula: ; In the formula, is the number of reference glyphs, is the spatial dimension of the feature map, and c is the feature dimension; The similarity calculation of the target glyphs and the reference glyphs specifically calculates the structural similarity of the target glyphs and each reference glyph using a similarity module Sim to generate a similarity weight sequence, which is expressed as shown in the following formula: ; The global style feature fusion specifically normalizes the similarity weights through a softmax function, and obtains the global feature by weighted fusion of the potential features of the reference glyphs based on the normalized weights The local style feature fusion specifically reorganizes the potential features of the reference glyphs into a unified representation space to generate the query vector of the target glyphs, the key vector and the value vector of the reference features, calculates the attention score matrix through the attention mechanism and normalizes it, and obtains the local feature by aggregating the reference value vectors based on the normalized weights. Finally, the global feature and the local feature are spliced through the channel dimension, and the output is obtained after layer normalization.

[0012] The decoder adopts a multi-layer Transformer block architecture, each layer of which integrates a self-attention layer, a cross-attention layer and a multi-layer perceptron, and the specific processing process includes: The semantic feature is taken as the initial token , cross-attention fusion is performed, and the style feature is fused into each Transformer through the cross-attention mechanism, wherein the generated token is taken as the query; as the key and the value, which is expressed as shown in the following formula: V; Then, starting from the initial token , the next token is predicted each time, which is added to the sequence until the complete token sequence ; the predicted token sequence indices corresponding to the VQGAN codebook, each index pointing to a discrete code word convert the token sequence into continuous features by codebook lookup quantize the features reconstructed into the target font image by the decoder of the VQGAN.

[0013] update the model parameters by backpropagation of the autoregressive loss and the contrastive loss, specifically after obtaining the output target font image, and derived from the adversarial loss, as shown in the following expression: ; + ; wherein, is a multi-style discriminator that distinguishes both the style and the content category of the generated sample, is a set of generated samples, is a set of real samples; s represents the style feature; c represents the content feature; the expression of the contrastive loss function is as follows: ; update the model parameters for optimization training based on the weighted combination of the autoregressive loss, the contrastive loss and the adversarial loss, and the weighted expression is as follows: ; wherein, is the autoregressive loss function.

[0014] The beneficial effects of the present application are: The font generation method based on the interactive fusion of Chinese character description sequence and style structure of the present application transmits content information by using Chinese character description sequence, no longer uses source glyphs, that is, does not need to perform feature decoupling, and therefore there is no risk of incomplete decoupling; the font generation is reconstructed as a sequence prediction task, and the target is to generate the token of the target glyph based on the given Chinese character description sequence and the reference glyph. By utilizing the prior knowledge of the embedded quantization codebook, the influence of the source glyph on the output is reduced, and the artifact phenomenon is also reduced.

[0015] The structure-aware gating unit provided by the application realizes structure-sensitive dynamic fusion by obtaining information of the Chinese character sequence decomposition module, automatically adjusts global-local weights according to the number of components and nesting levels, and avoids oversmoothing of simple characters and structural confusion of complex characters; secondly, the style adaptation guided by semantics, combining component semantics (such as shape and sound) to optimize feature aggregation, improves the semantic and style consistency of generated characters; and the structure features are directly obtained from the Chinese character content serialization module, without manual annotation, and are suitable for large-scale character sets. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a flowchart of the font generation method based on the interactive fusion of Chinese character description sequence and style structure of the application; Figure 2 is a Chinese character font decomposition table constructed by the application; Figure 3 is a decomposition diagram of the Chinese character sequence decomposition module of the application; Figure 4 is a target Chinese character image generated after conversion in embodiment 6 of the application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0018] Embodiment 1 The font generation method based on the interactive fusion of Chinese character description sequence and style structure of the application, as shown in Figure 1 includes the following steps: Step 1, constructing a Chinese character style conversion dataset and a Chinese character font decomposition table, dividing the dataset into a training set and a test set; Specifically, a plurality of Chinese character fonts are collected, and each Chinese character is rendered into an image with a resolution of 128*128 in different fonts, and a bicubic interpolation method is used to keep the proportions consistent. If necessary, black edges are filled to unify the size.

[0019] The basis of the Chinese character font decomposition table is the Unicode public Chinese character decomposition table, but there are a large number of repeated structure descriptions in the public decomposition table, such as Figure 2As shown, for example, multiple equivalent decompositions of the same Chinese character (such as "tree" which is decomposed into both "wood pair" and "wood and again") are repeatedly recorded, resulting in semantic analysis confusion during model training and increasing computational complexity. Therefore, the Chinese character decomposition table is disclosed here to be simplified and standardized, redundant entries, circular references, and partially missing characters in the Chinese character decomposition table are deleted, and the structure of Chinese characters is reduced to 12, which is sufficient to handle most commonly used Chinese characters. Finally, the sequence of basic components and some single character Chinese characters is set to itself, and the Chinese character font decomposition table required by the present method is obtained. For example Figure 3 As shown, 12 basic structure descriptions are extracted from the original complex system to cover more than 90% of the structure types of commonly used Chinese characters.

[0020] Step 2, build a Chinese character style conversion model, which includes a Chinese character sequence decomposition module, a style encoder, a structure style interactive fusion module, a decoder, and a style discriminator; Step 3, train the Chinese character style conversion model, pair the fonts in the training set by the characters, use the paired font images in the training set to train the model, and loop until the set iteration limit is reached or the loss is less than or equal to the preset threshold; Step 4, input the test set into the trained model to obtain the target Chinese character font image of the corresponding style.

[0021] Embodiment 2 Based on the above embodiment 1, the training of the Chinese character style conversion model in step 3 of the present application is as follows: the fonts in the training set are paired by the characters into reference characters and target characters, the reference characters are input into the Chinese character sequence decomposition module to be decomposed into Chinese character sequence, and the Chinese character sequence is encoded into semantic features; then the style features of the reference characters are extracted through the encoder, and the global and local style features are aggregated through the structure style interactive fusion module; the style features output by the structure style interactive fusion module are input into the decoder together with the semantic features, and the target font mark sequence is generated through autoregression, and then the target font image is obtained through VQGAN decoding; The generated target font image and the target character image in the training set are input into the contrast loss module of the style discriminator, the model parameters are updated through the calculation of the loss function, and the above process is looped until the set iteration limit is reached or the loss is less than or equal to the pre-set threshold, that is, the training of the model is completed.

[0022] Further, the Chinese character sequence decomposition module, when determining the decomposition sequence of the Chinese character, does not rely on the pre-defined decomposition table, but analyzes the structure of the Chinese character. For Chinese characters with left-middle-right or top-middle-bottom structure, multiple equivalent decomposition sequences are constructed by random selection, such as Figure 3The shown. As "tree", not only can be decomposed into common form, but also can have other equivalent representation. Then, the input Chinese character and reference Chinese character are decomposed into and respectively, and is filled to the maximum sequence length in the Chinese character sequence decomposition module, and is encoded into semantic features , so as to effectively extract semantic information from Chinese character structure while maintaining computing efficiency.

[0023] In order to realize efficient feature quantization and generation, the quality and efficiency of font generation are improved. The vector quantization technology of VQ-GAN can discretize the continuous image feature space, map the high-dimensional image features to a low-dimensional discrete space, and generate a compact codebook. In font generation, the features of the reference glyph can be quantized into discrete tokens, reducing the data dimension and computational complexity, making the subsequent processing more efficient. At the same time, this kind of discrete representation can better capture and express the style features of the font, providing a simple and effective way for style coding, avoiding the complexity and ambiguity caused by continuous feature processing in traditional methods.

[0024] Specifically, the style features of the reference glyph are extracted by the encoder, including: Structural design, using convolutional neural network structure, extracting multi-layer features of the image through downsampling operation, and converting the input image into low-dimensional feature map; Feature processing flow, after the input image is extracted by the encoder to obtain high-level semantic features, the channel number is adjusted to be consistent with the embedding dimension of the vector quantizer through a 1x1 convolution layer, and the processed feature vector is input into the vector quantizer to match the discrete code in the codebook through nearest neighbor search, realizing feature discretization.

[0025] Embodiment 3 Based on the above-mentioned embodiment 2, the structure style interaction fusion module of the present application includes latent feature extraction of the reference glyph, similarity calculation between the target glyph and the reference glyph, global style feature fusion, local style feature fusion, and style feature fusion and output; Latent feature extraction of the reference glyph, through the feature extraction of the input glyph by the style encoder in the last step, the VQGAN maps the reference glyph to the latent feature space through the pre-trained VQGAN encoder, and the intermediate feature set is obtained through the following formula (1): (1); In the formula, is the number of reference glyphs, is the spatial dimension of the feature map, and c is the feature dimension; The similarity calculation of the target glyph and the reference glyph is to evaluate the structural matching of the target glyph and the reference glyph, to calculate the similarity index through a structure perception gating unit (SAGU), to realize the structure-sensitive dynamic fusion, and to automatically adjust the global-local weight according to the component quantity and the nesting level, so as to avoid the over-smoothing of simple characters and the structural confusion of complex characters; the specific process is as follows: The structural similarity of the target glyph and each reference glyph is calculated by using the similarity module Sim, and a similarity weight sequence is generated, as shown in the following formula (2): (2); The weight calculated by the formula reflects the similarity degree of the target and the reference character in structure or component and the complexity of the glyph, and when the glyph is more complex, the model will pay more attention to the global features, and when the glyph is relatively simple, the model will pay more attention to the local features.

[0026] The global style feature fusion is specifically to normalize the similarity weight through a softmax function, to obtain the global feature by weighting and fusing the latent features of the reference glyphs based on the normalized weight.

[0027] The local style feature fusion is specifically to reorganize the latent features of the reference glyphs into a unified representation space, to generate the query vector of the target glyph, the key vector and the value vector of the reference features, to calculate the attention score matrix through an attention mechanism and normalize it, and to obtain the local feature by aggregating the reference value vectors based on the normalized weight. Finally, the global feature and the local feature are spliced through the channel dimension, and then output after layer normalization.

[0028] Embodiment 4 Based on Embodiment 3, the global style feature fusion process of the application is the core of realizing accurate extraction of style features, and the detailed process is as follows. First, the similarity weight calculation and normalization are performed, and the input is the structural complexity and the similarity of the target glyph and the reference glyph set , wherein , the structural matching degree and the structural complexity (such as the number of strokes, shared components, etc.) of the target character and the first reference character are measured. Next, the similarity weight is normalized through a softmax function to meet the probability distribution characteristics, and the expression of the softmax function is shown in the following formula (3): (3); ​​The operation can make the model increase the weight of global features when facing higher complexity of glyphs, increase the weight of local features when facing simpler glyphs, and at the same time, can make the reference glyphs more similar to the target glyph obtain higher weight. Next, the aggregation of coarse-grained features is carried out, and the input is the latent feature of the reference glyph after VQGAN encoding wherein is the spatial dimension of the feature map, c is the feature dimension, and the weighted fusion formula is shown in the following formula (4): (4); The global style of the reference glyph is aggregated by weighted average, for example, the weights of the overall features and the local features are dynamically adjusted according to the complexity of the strokes, the overall layout of the font, the distribution of stroke thickness, the inclination angle and other coarse-grained features. If the reference glyph is similar to the target glyph (such as left-right structure or top-bottom structure), the features will be preferentially fused.

[0029] The local feature fusion, that is, the fine-grained feature fusion, is specifically the following four steps: First, the feature dimension needs to be reorganized from multiple references to a unified representation space, and the latent feature of the reference glyph contains k reference samples, and the feature map size of each sample is , and the channel number is c. In order to enable the target glyph to interact with all reference features, the batch dimension and the spatial dimension are combined by the flatten2 operation: wherein, the spatial dimension of the k reference features is expanded to one dimension to form a sequence with a length of , which is convenient for subsequent attention mechanism processing. At the same time, the operation unifies the multiple reference features into a single linear sequence, so that each semantic position of the target glyph can interact with the local area of all reference features, solving the feature alignment problem between different reference samples. Next, linear projection is performed to generate the core vector of the attention mechanism. First, the generation of the query vector (Query) of the target glyph is as follows: the semantic feature of the target glyph wherein is the glyph sequence length (such as 35), and c is the feature dimension. After projection by the linear layer , the distribution is stabilized by layer normalization (LayerNorm), and the formula is shown in the following formula (5): (5); always represents the semantic demand of the target character, which is used to query the corresponding local style in the reference feature. Then the key vector (Key) and the value vector (Value) of the reference feature are generated. We reorganize the reference feature After input, the key vector K is generated, and the specific process is as follows: the linear layer After projection, normalization is performed, and the formula is shown in equation (6): (6); The value vector V is directly projected through the linear layer The formula is shown in equation (7): (7); K is used to query the matching degree of the reference feature and the target query, and V contains the local style details (such as stroke edges and corner shapes) to be extracted. After obtaining K and V, cross-attention calculation is performed. First, the attention score matrix is constructed, and the similarity matrix of the target query and the reference key K is calculated. The scale factor is used to stabilize the gradient, and the specific formula is shown in equation (8): (8); In the formula, the matrix element represents the attention degree of the i-th position of the target character to the j-th local area of the reference feature, and dropout is used to prevent overfitting.

[0030] For example, if the target character is "tree" (decomposed as "tree: ⿰wood ⿰again inch"), the position corresponding to the "wood" side in the query vector Q will give a high attention score to the feature area of "wood" in the reference character, achieving precise matching of semantic components and style features. Next, the attention matrix A is normalized by row to obtain the probability distribution, as shown in equation (9): (9); Based on the normalized weight, the reference value vector V is aggregated to generate local features, and the formula is shown in equation (10): (10); Each corresponds to the local feature style of the i-th position of the target character , which integrates all the details related to the semantic position in the reference character. The captured information includes stroke-level details such as the starting arc of horizontal strokes and the ending thickness of strokes; component relationships such as the spacing of three points next to "water" and the angle of the horizontal hook next to "speech"; style variations such as the tendency of connected writing in handwriting and the sharpness of corners in printing.

[0031] The local feature fusion realizes part-level migration through a "semantic query-reference matching" mechanism, such as ensuring that the style beside "wood" is only extracted from the "wood" part of the reference character, avoiding cross-part interference; improving the adaptability of few-shot, in the 1-shot (i.e. only one reference character) scenario, making up for the lack of reference through attention focusing; and improving the anti-artifact capability, filtering irrelevant features through dynamic weights, and reducing the common stroke ambiguity in traditional methods.

[0032] Finally, the global feature is concatenated with the local style feature , where is the spatial dimension of the reference feature, is the sequence length of the target character, and c is the number of feature channels. Next, the feature concatenation operation is performed to combine the two into a fusion feature, as shown in equation (11): (11); This operation makes the fusion feature contain both coarse-grained layout information and fine-grained stroke details . Then, layer normalization is performed on the concatenated feature to eliminate the scale difference between different dimensions, as shown in equation (12): (12); The normalization function is to ensure the stability of the feature distribution and avoid optimization difficulties caused by the numerical range difference between global and local features, while enhancing the model's adaptability to different style reference samples. As a style feature input into the Transformer decoder, it guides the autoregressive generation together with the character semantic feature . The decoder takes as the key and value of cross-attention, so that each generated token can dynamically align with the reference style.

[0033] Embodiment 5 Based on the above embodiment 4, the decoder uses a multi-layer Transformer block architecture, containing 10 Transformer layers. Each layer integrates a self-attention layer, a cross-attention layer, and a multi-layer perceptron. The self-attention layer handles the internal dependencies of the generated token sequence, the cross-attention layer is used to align the semantic feature and the style feature, and the multi-layer perceptron is used to enhance the feature expression capability.

[0034] The specific processing process includes: First, initialize the input, and concatenate the semantic feature​ As initial mark ,Right now = This operation converts the character structure semantics (such as "⿰木尌") into the starting point of sequence generation, and It contains the hierarchical structure information of Chinese character components (such as left-right structure, top-bottom structure), forcing the generated sequence to conform to the character semantic rules and avoid structural errors.

[0035] Next, cross attention fusion is performed, style features It is integrated into each Transformer through the cross-attention mechanism, where the generated token is used as the query; As keys and values, the expressions are as shown in Equation (13): V (13); This mechanism enables each generation step to dynamically align stylistic details (e.g., stroke thickness, corner curvature) of the reference glyph.

[0036] Then from the initial mark Start by predicting the next token each time , adding it to the sequence until the complete labeled sequence ; The probability modeling for prediction is as follows, conditional probability Through the softmax output of Transformer, the formula for maximizing sequence likelihood is shown in the following formula (14): (14); Finally, decode from token to glyph and convert the predicted token sequence into Corresponding to the index of the VQGAN codebook, each index points to a discrete codeword , convert the tag sequence into continuous features through codebook search ,Right now Quantitative features The decoder of VQGAN is upsampled and reconstructed into the target font image.

[0037] Furthermore, after the target font image is output, the model parameters are updated by calculating the autoregressive loss and the contrast loss. and Derived from the adversarial loss, the expressions are shown in the following equations (15) and (16): (15); + (16); Where, It is a multi-style discriminator that distinguishes the style and content category of the generated samples. To generate the sample set, For the real sample set; s represents the style feature; c represents the content feature.

[0038] The total loss function also includes the following parts, first is the autoregressive loss Optimizing the likelihood of the mark sequence ensures semantic and stylistic consistency. Then is the contrastive loss , which pulls similar style features closer and pushes different style features farther apart. The expression of the contrastive loss function is shown in equation (17) as follows: (17); Based on the weighted combination of autoregressive loss, contrastive loss and adversarial loss, the model parameters are updated and optimized. The weighted expression is shown in equation (18) as follows: (18); In the formula, is the autoregressive loss function.

[0039] Embodiment 6 This embodiment is based on the font generation method of Chinese character description sequence and style structure interactive fusion, which includes the following steps: Step 1: Collect 464 fonts from the Internet, covering various styles such as printed, handwritten and artistic fonts, to ensure the diversity of the data set. Select 3500 common Chinese characters as the basic character set to ensure coverage of high-frequency usage scenarios. Next, render each Chinese character in different fonts into 128*128 resolution images.

[0040] The data set is divided into training set and test set, wherein the training set randomly selects 3300 Chinese characters and 424 fonts, which are used for model parameter optimization and cover mainstream structure and style. The test set is divided into UFSC (unseen font-visible character), which uses the same 3300 Chinese characters, but replaces 40 fonts that are not involved in training, to test the generalization ability of the model to new styles. UFUC (unseen font-unseen character): contains the remaining 200 untrained Chinese characters and 40 new fonts, to verify the generation ability of the model to unfamiliar characters.

[0041] Meanwhile, a Chinese character font decomposition table is constructed.

[0042] Step 2, construct a Chinese character style conversion model, which includes a Chinese character sequence decomposition module, a style encoder, a structure and style interactive fusion module, a decoder, and a style discriminator; Step 3, train the Chinese character style conversion model, pair the fonts in the training set by character, use the paired font images in the training set to train the model, and cycle the training until the set iteration limit is reached or the loss is less than or equal to the preset threshold; Step 4, input the test set into the trained model to obtain the target Chinese character font image corresponding to the style. The converted target Chinese character font image is shown in Figure 4 The left character is the reference character shape, and the right character is the target Chinese character content generated according to the style of the reference character shape. It can be seen that, since the Chinese character description sequence is used instead of the Chinese character image to extract the content features, the Chinese character image generated by the model not only learns the style features of the reference character shape, but also retains the structure information of the Chinese character very completely, without artifacts, blurring, and stroke loss, etc.

[0043] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0044] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A font generation method based on the interactive fusion of Chinese character description sequence and style structure, characterized in that: The following steps are involved: Step 1: Construct a Chinese character style conversion dataset and a Chinese character glyph decomposition table, and divide the dataset into a training set and a test set; Step 2: Build a Chinese character style transfer model, which includes a Chinese character sequence decomposition module, a style encoder, a structure-style interaction fusion module, a decoder, and a style discriminator. Step 3: training the Chinese character style transfer model by pairing the fonts in the training set by character, using the paired font images in the training set to train the model, and looping the training until the set upper limit of the number of iterations is reached or the loss is less than or equal to a preset threshold; Step 4: Input the test set into the trained model to obtain the target Chinese font image of the corresponding style.

2. The font generation method based on the interactive fusion of Chinese character description sequence and style structure according to claim 1 is characterized in that: In step 1, Chinese characters are collected and each Chinese character is rendered as a 128*128 resolution image in different fonts, and a bicubic interpolation method is used to maintain a consistent proportion.

3. The font generation method based on the interactive fusion of Chinese character description sequence and style structure according to claim 1 is characterized in that: The Chinese character glyph decomposition table is simplified and standardized based on the Unicode public Chinese character decomposition table.

4. The font generation method based on the interactive fusion of Chinese character description sequence and style structure according to claim 1 is characterized in that: The step 3 specifically comprises dividing the fonts in the training set into reference glyphs and target glyphs according to character pairing, inputting the reference glyphs into a Chinese character sequence decomposition module to decompose them into a Chinese character glyph sequence, and encoding the Chinese character glyph sequence into semantic features; then extracting the style features of the reference glyphs through an encoder, and aggregating the global and local style features using a structural style interaction fusion module; inputting the style features and semantic features output by the structural style interaction fusion module into a decoder, generating a target font tag sequence through autoregression, and then decoding the target font image through a VQGAN. The generated target font image and the target glyph image in the training set are input into the contrast loss module of the style discriminator. The model parameters are updated by calculating the loss function and transferring it backwards. The above process is trained repeatedly until the set upper limit of iterations is reached or the loss is less than or equal to the pre-set threshold, thus completing the training of the model.

5. The font generation method based on the interactive fusion of Chinese character description sequence and style structure according to claim 4 is characterized in that: The extracting of the style features of the reference glyph by the encoder includes: Using a convolutional neural network structure, the multi-layer features of the image are extracted through downsampling operations, and the input image is converted into a low-dimensional feature map; After the encoder extracts high-level semantic features from the input image, the number of channels is adjusted to be consistent with the embedding dimension of the vector quantizer through a 1×1 convolutional layer. The processed feature vector is input into the vector quantizer, and the discrete codewords in the codebook are matched through nearest neighbor search to achieve feature discretization.

6. The font generation method based on the interactive fusion of Chinese character description sequence and style structure according to claim 4 is characterized in that: The structural style interactive fusion module includes potential feature extraction of reference glyphs, similarity calculation between target glyphs and reference glyphs, global style feature fusion, local style feature fusion, and style feature fusion and output; The potential features of the reference glyph are extracted to obtain an intermediate feature set by the following formula: ； Where, is the reference glyph number, is the spatial dimension of the feature map, and c is the feature dimension; The similarity calculation between the target glyph and the reference glyph is specifically calculated using the similarity module Sim. The structural similarity of , generates a similarity weight sequence, the expression is shown as follows: ； The global style feature fusion is specifically to normalize the similarity weights through the softmax function, and to obtain the global features by weighted fusion of the potential features of the reference glyphs based on the normalized weights. The local style feature fusion specifically reorganizes the latent features of the reference glyphs into a unified representation space, generates the query vector of the target glyph, the key vector and value vector of the reference features, calculates the attention score matrix through the attention mechanism and normalizes it, and aggregates the reference value vector based on the normalized weights to obtain the local features; Finally, the global features and local features are concatenated through the channel dimension and output after layer normalization.

7. The font generation method based on the interactive fusion of Chinese character description sequence and style structure according to claim 4 is characterized in that: The decoder adopts a multi-layer Transformer block architecture, each layer integrating a self-attention layer, a cross-attention layer, and a multi-layer perceptron. The specific processing process includes: Semantic features As initial mark , cross attention fusion, style features It is integrated into each Transformer through the cross-attention mechanism, where the generated token is used as the query; As keys and values, the expressions are as follows: V； Then from the initial mark Start by predicting the next token each time , adding it to the sequence until the complete labeled sequence ; The predicted tag sequence Corresponding to the index of the VQGAN codebook, each index points to a discrete codeword , convert the tag sequence into continuous features through codebook search , quantitative features The decoder of VQGAN is upsampled and reconstructed into the target font image.

8. The font generation method based on the interactive fusion of Chinese character description sequence and style structure according to claim 4 is characterized in that: The method of updating the model parameters by calculating the autoregressive loss and the contrast loss and backpropagating is specifically to obtain the output target font image. and Derived from the adversarial loss, the expression is as follows: ； + ； Where, It is a multi-style discriminator that distinguishes the style and content category of the generated samples. To generate a sample set, is a real sample set; s represents style features; c represents content features; The expression of the contrast loss function is as follows: ； Based on the weighted combination of autoregressive loss, contrast loss and adversarial loss, the model parameters are updated for optimization training. The weighted expression is shown in the following formula: ； Where, is the autoregressive loss function.

Images